What I really meant was stronger than that.

I respect people who do the most impactful work they can — whether they work 70 hours a week because they can, 30 hours so they can be home with their kid, or 15 hours because of illness or burnout.

I admire those who go above and beyond. But I don’t expect that of everyone. Working long hours isn’t required to earn my respect, nor do I think it should be the standard that we hold as a community. I want it to be okay to say "that doesn't work for me".

It feels like donations: I admire people who give away 50%, but I don’t expect it. I still deeply respect someone who gives 10% to the most impactful charities.

As a productivity coach I see a lot of ambitious EAs struggling to live up to their own (often unrealistic) expectations. The sense of what they “should” do is sky high, and often unconnected with any reasonable path from where they are now. The unreasonableness is especially stark with people learning to live with a new chronic health condition, as they constantly compare their present capacity with what they could do before.

Personally, I average less than 40 hours due to chronic health issues. Accepting that has been hard. There’s never a clear line between pushing myself and resting enough — but I’m happier and more productive when I balance work with “happiness time,” without beating myself up for needing more rest than others. 

I want others to have that same freedom. If setting your own sustainable pace were seen as morally okay, maybe I would see fewer people constantly pushing themselves into unhealthy, unhappy situations. (Of course, accommodations depend on the job and are often a privilege — but that’s a reason it’s hard, not a reason not to try.) I want to support a community where there’s affordances for respecting your sustainable limits.  

So, in case you needed someone to give you permission -- it’s okay to set a sustainable, happy pace that works for you even if someone else can do more.

Other things – like family, friends, gardening, hobbies that make you happy – are worth valuing, and it’s okay (even good) that your happiness and identity are diversified beyond just your work.

I think I need those things that I care about just for me, not because someone else will metaphorically pat me on the head for doing a good job. I think the world is a better place when I take the time to tend my plants, cuddle my dog, and create beautiful things and spaces. 

Additionally, if the world changes and there's no longer work for me and most other humans, or if I change and can no longer do the work I find meaningful, I still want something important that can thrive in that world. And that means having something I value and love that isn't about my ability to contribute.

For me, that comes partly from accepting my limits as gracefully as I can, and partly from filling the space around them with the most meaningful bits of self and creativity and connection I can find. So I'll put in my hours, then rest into something good. And I'm going to treat it like I do donating 10% -- it's admirable to do more, but this is enough. I'm still a good person, and I think you are too.

If this resonated with you, I highly recommend Julia Wise’s excellent pieces Cheerfully and You have more than one goal, and that's fine. And I’m going to mollify my anxious instinct to puts caveats everywhere with one link to the Rule of Equal and Opposite Advice.

Well, yes, as a matter of fact. You probably can’t eliminate all of your uncertainties. But that isn't the point. You’re not trying to achieve certainty. You’re trying to reduce your uncertainty so that you can make better decisions. 

I took a class with an unusually eccentric professor during my undergrad. This professor claimed that every possible outcome is equally probable, and hence we can’t make inferences about what is more likely to happen based on what happened in the past. Now, I don’t know if this women literally believed that the sun was equally likely to rise in the east or west tomorrow, but that is the literal interpretation of her stated beliefs. 

This stance probably seems ludicrous to you. It certainly did to me. But I think a weaker version of this belief is actually fairly common. The weaker belief is that we can’t be certain, so it isn’t worth trying to get more information. 

Not at all! The picture isn’t black and white, but there are still darker and lighter shades of gray. We have a lot of data that we can use to narrow our guesses about what the outcome of different decision would be - in other words, the probabilities of those outcomes. Even given our uncertainty, it can matter a lot to go from 50% certainty to 80% certainty. So don’t toss up your hands yet. 

It might help to think in probabilities. For example, is it 50% or 90% certain that you’ll get that promotion? Or what is the range of options that you’re 90% confident contains the real answer? You can say something is “medium” vs “very” likely to happen, but it’s easier to get confused. How often do “medium” likely things happen? How often do “very” likely things happen? What is the difference, and is it consistent? By the time you get to the end of that train of thought, you’ve already put rough numbers to the words, and those numbers may be easier to check and track. 

Relatedly, spending a bit of time improving the accuracy and calibration of your guesses is low hanging fruit for making better decisions. The book How to Measure Anything has a great chapter on calibration training, and the Open Philanthropy project commissioned a calibration training tool you can try out here. 

Uncertainty reduction is always proportional to how much you already know. If you know very little, then it is very easy to reduce your uncertainty at least a little bit. For those of you familiar with Bayesian reasoning, reducing your uncertainty is equivalent to updating your prior probabilities. 

Similarly, it’s extremely common to find uncertainty aversive. Not knowing exactly what to do may be the single most common cause of procrastination. When you have a big or important decision in front of you, it can feel overwhelming even choosing what exactly you need to decide on. 

So start by decomposing your decision. What do you already understand? What smaller steps do you know how to do? Who can you ask about the parts you’re confused by? You already have a lot of information - start there! And it’s not cheating to start with small steps like asking for help if you’re really confused. 

Finally, you’ll often have to accept that you’re never going to be entirely sure. Once you’ve removed what uncertainty is worth removing, you need to make a decision despite the remaining uncertainty. You can always gather more or better information. Ultimately decision-making is based on judgment calls. Let’s just make them well-informed judgement calls.

1. I think it’s a good case study for iterated depth decision-making in general and reevaluating your career in particular, and 

2. I want to let you know about my exciting plans! I’m doing the Tarbell Fellowship for early-career journalists for the next nine months. I’m excited to dive in and see if AI journalism is a good path for me long-term. I’ll still be doing coaching, but my availability will be more limited. 

Background

I love being a productivity coach. It’s awesome watching my clients grow and accomplish their goals. 

But the inherent lack of scalability in 1:1 work frustrated me. There was a nagging voice in the back of my head that kept asking “Is this really the most important thing I can be doing?” This voice grew more pressing as it became increasingly clear artificial intelligence was going to make a big impact on the world, for good or bad. 

I tried out a string of couple-month projects. While good, none of them grew into something bigger. I had some ideas but they weren’t things that would easily grow without deliberate effort. (Needing the space to explore these ideas prompted me to try CBT for perfectionism.) 

I always had this vague impostery feeling around my ideas, like they would just come crashing down at some point if I continued. I wasn’t confident in my decision-making process, so I wasn’t confident in the plans it generated. 

So at the beginning of last year, I set out to do a systematic career review. I would sit down, carefully consider my options, seek feedback, and find one I was confident in. 

This is the process I used, including the specific tools I used to tackle each of my sticking points. 

Deciding Which Problem to Work On

I’m a big proponent of theories of change, and think that the cause I pick to work on heavily influences how much impact I can make. I also need to match my personal fit to specific career opportunities, so it’s harder than “just” determining what the most important cause is and choosing a job working on that.

My solution was to write about what I thought were the most pressing problems in the world. I kept that info in the back of my mind later when brainstorming options and considering theories of change for specific options. 

Example

My short answer from Jan 2023:

AI seems like the clear #1 cause area, while most of the other main EA cause areas seem like good second options. The outputs from ChatGPT, Stable Diffusion, Midjourney, etc. make me feel like we’re on the cusp of world transforming AI. AI might not replace humans en masse for years, but they’re already sparking backlash from threatened artists. My guess is that AI seems like one of the top areas to focus on even if I set aside AI x-risk, because AI has so much potential to improve the world if it’s used well (which doesn’t seem likely without a lot of people working together to make it go well). However, alignment still seems like the most important area, because we need alignment both for x-risk and for maximizing the potential for AI. 

I don’t have an especially detailed view of what the bottlenecks are in AI safety. (Based on poor past performance trying out coding, I’m not a good candidate for direct technical work.) Generically, [improving the productivity of people working on AI safety] and [helping promising candidates skill up to start working in the field] both seem promising. 

Writing for larger audiences on productivity, epistemic/rationality, and EA also seems promising (for AI and other issues), but I feel like my theory of change via outreach is underdeveloped. My likely avenues of impact are increasing productivity for people working in impactful roles, instilling better reasoning skills in readers, and/or spreading EA ideas. 

Tips for you to apply

Tool: Write a couple paragraphs about which causes you think are most important and why. 

Tip: Fortunately, there are some good resources to help locate promising cause areas, like 80,000 Hours’ Problem Profiles, so you can focus most of your energy on finding the best roles for you within one of them.

Don’t get bogged down here. There's no single 'best' cause area for everyone. Your goal is to identify promising causes where you can find suitable roles. If it helps, I expect you will have more impact in a great role in a good cause area than you would in a poor role in a great cause area. 

I call this the “all else not equal” clause. You can’t just make the theoretically best decision as if all else is equal between your options - practical considerations in the real world usually dominate the decision. For example, (for most people) finding out which jobs you can get is more important than theoretically deciding which of your top options would be most impactful if you could get it.

Big Brainstorm of Ideas 

I’ve been feeling uncertain partly because it felt like I hadn’t considered all of the options systematically. So I wanted to balance being systematic and practical, since I couldn’t realistically explore a bunch of ideas in depth. 

Over the course of a few weeks, I brainstormed around 30 roles and independent project ideas I could explore. (I’ve seen other people make lists with 10-100 options.) I wrote down options I’d already considered and brainstormed more by looking through the 80,000 Hours job board, talking to some close friends, and thinking about potential bottlenecks in the causes I’d listed in my theory of change brainstorm. 

Example

Big brainstorm of work ideas, very roughly grouped by category

• More measured/outcome-based coaching, working toward specific goals 

• Do another impact evaluation 

• CBT provider course 

• Roleplay training sessions with other peers 

• Record calls and analyze them for ways to improve 

• Blogging about psych/productivity/ea

• Do a deep dive into expertise 

• AI Impacts style research 

• Deliberate practice writing skills (e.g. Gladwell exercises, reasoning transparency, rationalist discourse norms) 

• Trying to get a job writing for a news source 

• Engage a lot with short posts on fb, write a handful of longer posts based on what feels most important/gets traction after doing so 

• Write fiction 

• Spinning up skilling up courses

• Work with BlueDot

• Help run ARENA

• Intro to bio course 

• Research exploration/training course/cohort 

• Map the EA landscape talent gaps

• Operations training workshops 

• Charity Entrepreneurship

• Take over the MHN

• Do operations for some EA org 

• Project manager, e.g. for some AI safety org

• Grant making, e.g. Longview or Open Phil or Founders Pledge 

• Do lots of broader social organizing for the London community 

• Do retreats/weekends away for community building 

• Host workshops or teach at workshops, like ESPER and Spark

• Do productivity workshops 

• Host retreats for coaches 

• Host retreats for people working on mental health 

• Host retreats for people working on rationality 

Tips for you to apply

Tool: Make a list of 10-100 possible career options.

Tip: Don’t evaluate your ideas while brainstorming. You will get stalled and think of fewer ideas if you’re thinking about whether they are good or bad as you go. Instead just try to think of as many ideas as possible. Many of these ideas will be bad – that’s fine. You’re just trying to find numerous and novel ideas, so that the top 3-10 are probabilistically more likely to include some really good ideas. 

Here are some ways to broaden your options:

• Write down any opportunity you already know about which might be very good (e.g. using your capacity for earning to give as a benchmark to try to beat).

• 80,000 Hours job board

• You can filter jobs by problem area, role type and location.

• Also use the job board to discover promising orgs even if they are not currently listing a job you can do right away. See the listed jobs and the ‘Organizations we recommend’ section.

• Ask people who share or understand your values what the most valuable opportunities they are aware of are.

• List big problems you have insight into along with possible solutions. 

• Look through 80,000 Hours’ list of promising career paths. 

• Look through Open Phil's grantees, which can be filtered by focus area.

• Don’t forget career capital. You want to be doing high-impact work within the next few years. To do that you might focus on building skills or other resources in the near term. Everything here applies to this as much as for jumping into a directly valuable role.

For those interested, you can read Kit Harris’s list of 50 ideas from a similar brainstorm he did years ago. They span operations, generalist research, technical and strategic AI work, grantmaking, community building, earning to give and cause prioritization research.

Quick Ranking

In the past, I’ve found a long list of career ideas overwhelming. This time (based on Kit’s suggestion), I roughly sorted the ideas by how promising they seemed. “Promising” roughly meant [expected impact x how excited I felt about the idea]. Then I selected my top ideas to explore further. This made exploration much more approachable. 

I did a quick sanity check by asking my partner if he would have ranked any of the excluded ideas above the ones I prioritized for deeper dives. 

Tips for you to apply

Tool: Roughly rank your possible career options by how promising they seem. 

If you have a default option that you can definitely pursue (such as continuing in a job you already have), that’s a good threshold. You can immediately rule out any options that seem less promising than your default. 

Note on using spreadsheets 

I wanted to balance prioritizing quickly and considering ideas I might not have thought about before. After all, I just did a big brainstorm to get new ideas. One way to do this is to put all of the ideas in a spreadsheet and spend a few minutes roughly ranking each one, then see how the averages compare to each other. I quickly did this with a 1-5 scale for personal fit, impact, and skill building potential. 

I didn’t personally find this exercise that useful, but I’m still optimistic that spreadsheets can sometimes be helpful for decision making. 

Project Briefs 

For each of my top options, I spent thirty min to two hours writing a project brief. (Ones for outside roles were shorter, while independent projects were longer.) This consolidated my understanding of the role/project and helped me identify key uncertainties to test. 

The questions from my project briefs were drawn from this post. Specifically I answered:

• One-paragraph description of the role/plan 

• What is the expected impact? 

• How much do I expect to enjoy this? How motivated do I feel?  

• What hypotheses am I trying to test? What are possible one-month tests of my hypotheses? 

• What metrics could I track to address the most likely failure modes with this plan? 

The project briefs were mostly to identify my key uncertainties. Fleshing out the above questions helped me identify the cruxes that would make me choose or abandon each option, so I could start investigating. 

Example

Scale up ARENA with BlueDot 

Outline

Work with BlueDot to set up a refined course on their platform and set up training for facilitators to scale the course for potential ML research engineers (using my ideas for a full-time course with dedicated technical facilitators). Ideally, I could then hand the course over to them to run going forward. 

Benefit

This course could be used for an accelerated study program where people learn to do ML engineering well enough to get jobs at AI safety orgs. By targeting people who specifically want to work on AI safety, this would help with the bottleneck to AI safety research. (Rohin and other people think this is a bottleneck to safety research.) 

Motivation and enjoyment 

6/10, need some force to pay attention to all of the details. (I’m excited about the program, but not super excited about running it myself.) 

What hypotheses am I trying to test? 

That ARENA will accelerate the skilling up process to work at AI safety orgs.

That BlueDot will be able to effectively implement and run this program. 

Whether these kinds of programs in general are worth creating and running. 

Whether BlueDot in particular is good to work with and whether they are doing high impact work. 

What are possible one-month tests of my hypotheses? 

Discuss whether Matt and Callum and Rohin think ARENA is promising for skilling up research engineers. 

Discuss with Callum and BlueDot whether we should scale up the course. 

Try implementing the course on BlueDot’s platform, and run a trial program with their facilitators. 

How long will this project take? 

Probably 3-10 hours spread out over a couple weeks to figure out if we should try implementing. 

Highly uncertain about implementation, maybe range of 20-200 hours? So a couple weeks to 5 months. Worst case scenario is even longer, because it’s all the hours of work plus waiting on other people a bunch. 

If we move forward with this project, we definitely need intermediate milestones and reevaluation points. Maybe the first point at whether we attempt it at all. Second one month into implementing – how close are we to opening this to a trial group? Second three months in – is this promising enough that it’s worth going further, if it requires more time than this to implement? Monthly reevaluations after that until running with trial cohort. Third 4 weeks into first trial cohort – is this going well, should we majorly shift course? At end of first trial cohort, how did this go? Is it worth running regularly? 

Counterfactual impact 

Probably the online course will quietly die unless I push it forward, since Matt isn’t continuing. 

Metrics to measure

We don’t know whether the course helps people skill up. 

Success rate of participants at getting jobs, how many apply for ea jobs, how many get jobs, how many need to do more study before they’re ready for jobs, how many do something else, we could measure lead of how many want ea jobs and lag of how many get them

How many graduate, how highly they rate the course, also good metrics. 

How much would an EA funder be willing to pay me to do this project? 

Matt’s program was potentially costing millions and people seemed willing (admittedly that was FTX). Open Phil funded the stipends after FTX, so they thought at least that much met their bar. 

Personal blogging  

Outline

I would continue blogging about psych/productivity/ea/rationality in an attempt to sharpen my own thinking and spread better ideas through the community. I could do some deliberate practice on writing skills (e.g. Gladwell exercises, reasoning transparency, rationalist discourse norms), plus try out my old idea of doing lots of short posts on FB or twitter to refine my understanding of which ideas are most promising for longer posts. Hopefully, those two things would make my writing more useful and compelling. 

Benefit

There’s probably something useful in spreading good ideas, and written form is easily spread and scaled. I could reach many times more people than I can in 1:1 coaching, probably including high-impact people. 

Motivation and enjoyment 

8/10, need some force to follow through on all the exercises and finish posts, but I’m mostly excited about the writing process. 

What hypotheses am I trying to test? 

Can I write well enough/fast enough that I'm regularly putting out actionable, engaging posts that thousands of people will read?

That my writing will reach a large and/or high impact audience, probably measured by email subscriptions, engagement, and feedback. 

That my writing will help the people is reaches, probably measured via self-reports of impact and via engagement. 

Whether my FB idea to test engagement works. 

Whether I should pursue writing as my main impactful work, including getting a job somewhere with mentorship. 

That I can practice and improve the speed/quality of my writing. 

What are possible one-month tests of my hypotheses? 

Try the FB idea with daily posts for a month, plus at least one blog post. 

Read Holden’s post and other materials about the value of journalism (is there an ea forum post about vox FP? 80k post on journalism?) Use these to think about what kinds of messages and audience I want to target. 

Ask around for some advice on growing my audience, and implement to see if I can reach a bigger audience (e.g. reposting to hacker news). Measure a bunch of things and see what seems to resonate. 

Ask Nuño or someone like Rohin/Kit for estimates of value from posts? 

Try out ChatGPT to aid in writing, see if this speeds me up. 

How long will this project take? 

As much or as little time as I want to give to it. Probably 20+ hours a month if I want to write and read a bunch and publish at least one high quality post a month. 

I should have reevaluation points and metrics if I pursue this. Maybe ask for a re-evaluation of donor funding worth after 3 months or 1 year, based on what I produced in that time and how much my audience has grown? Maybe have some metric for how much I should expect my audience to grow? Monthly reevaluations for what I should be doing differently in writing and what to focus on. 

Counterfactual impact 

There are lots of other writers. None of them are doing exactly what I would be, but maybe I’m not adding that much novel and new? 

Metrics to measure

I can imagine I get to the end of a year of blogging, without clearly understanding if this worked. Having specific targets in mind (and ways to measure those targets) for audience growth, engagement, posts published, and donor-evaluated value (or audience? Patron?) could help here. 

12 posts published, stretch 24

Audience doubled, to 1000 subscribers (can I measure rss feeds?)

Engagement – I get at least at least one comment on the majority of my posts? Maybe I want substantial comments or people telling me the post was useful for the majority of posts? 

Donor-evaluated value – what value would be worth trying this experiment? $200 x number of hours to create a post is a good threshold. Maybe anything that’s higher than the arena course donor value? What value would be worth continuing at the end? 

How much would an EA funder be willing to pay me to do this project? 

Logan got 80,000 for high variance work on naturalism, with Habyrka expressing expectation that it likely wouldn’t work – this seems relevant but different from what I’m thinking about. Personal blogging is probably below this bar. 

Guestimate model 

I also put together a guestimate model to see if any option was clearly higher impact. While the exercise was helped clarify some uncertainties, it didn’t cause me to update much. Mostly I learned that my confidence intervals were wide and overlapping, so no option was clearly best. 

Tips for you to apply

Tool: Write a short project brief for each of your top options to identify your key uncertainties. 

This post has many good questions you can consider. I recommend using a reasoning transparency-inspired style if you’re struggling, laying out your reasons and the evidence for those conclusions.

Planning Tests 

In my case, writing the project briefs turned up numerous questions I could investigate. For example, I identified two key uncertainties that would easily change my plans. 

1. Can I write well enough/fast enough for professional journalism or blogging? 

2. Do I and the other stakeholders want to scale up the ARENA pilot? 

Additionally, I think there’s often value from just doing as close to the role I want as possible. Do I like it? Am I good at it? Does anything surprise me? 

With these in mind, I developed quick tests meant to give me more information about my top options. 

My brief explorations included:

• Talking to people doing the job

• Reading about the career path (e.g. journalism) 

• Talking with potential collaborators 

• Applying to a program

• Getting feedback on my plans from people who know me

• Talking with people in the field about how promising a project was 

My deeper explorations included:

• Spending a day at an org I was interested in 

• Co-running a ten-week trial program of an AI skilling-up course 

• Spending three months blogging regularly 

• Taking a relevant course 

• Pitching posts to publications 

• Applying to the Tarbell Fellowship for journalism

Example 

A couple ideas got dropped because other people took up the mantle. A couple more were directions I could take coaching in, so I had fewer uncertainties. I could learn more from focusing my tests on options where I had less preexisting information.  

In the end, I ended up doing the most tests with journalism, which you can read about in this excerpt from my Making Hard Decisions post:

I knew basically nothing about what journalism actually involved day-to-day and I had only a vague theory of change. So my key uncertainties were: What even is journalism? Would journalism be high impact/was there a good theory of change? Would I be a good fit for journalism? 

Cheapest experiments: 

So I started by doing the quickest, cheapest test I could possibly do: I read 80,000 Hours’ profile on journalism and a few other blog posts about journalism jobs. This was enough to convince me that journalism had a reasonable chance of being impactful.

Meanwhile, EA Global rolled around and I did the second cheapest quick test I could do: I talked to people. I looked up everyone on Swapcard (the profile app EAG uses) who worked in journalism or writing jobs and asked to chat. Here my key uncertainties were: What was the day-to-day life of a journalist like? Would I enjoy it? 

I quickly learned about day-to-day life. For example, the differences between staff and freelance journalism jobs, or how writing is only one part of journalism – the ability to interview people and get stories is also important. I also received advice to test personal fit by sending out freelance pitches.

Deeper experiment 1: 

On the personal fit side, one key skill the 80,000 Hours’ profile emphasized was the ability to write quickly. So a new, narrowed key uncertainty was: Can I write fast enough to be a journalist?

So I tried a one-week sprint to draft a blog post each day (I couldn’t), and then a few rounds of deliberate practice exercises to improve my writing speed. I learned a bunch about scoping writing projects. (Such as: apparently, I draft short posts faster than I do six-thousand-word research posts. Shocking, I know.) 

It was, however, an inconclusive test for journalism fit. I think the differences between blogging and journalism meant I didn’t learn much about personal fit for journalism. In hindsight, if I was optimizing for “going where the data is richest”, I would have planned a test more directly relevant to journalism. For example, picking the headline of a shorter Vox article, trying to draft a post on that topic in a day, and then comparing with the original article. 

Deeper experiment 2: 

At this point, I had a better picture of what journalism looked like. My questions had sharpened from “What even is this job?” to “Will I enjoy writing pitches? Will I get positive feedback? Will raising awareness of AI risks still seem impactful after I learn more?” 

So I proceeded with a more expensive test: I read up on how to submit freelance pitches and sent some out. In other words, I just tried doing journalism directly. The people I’d spoken with had suggested some resources on submitting pitches, so I read those, brainstormed topics, and drafted up a few pitches. One incredibly kind journalist gave me feedback on them, and I sent the pitches off to the black void of news outlets. Unsurprisingly, I heard nothing back afterwards. Since the response rate for established freelance writers is only around 20%, dead silence wasn’t much feedback. 

Instead, I learned that I enjoyed the process and got some good feedback. I also learned that all of my pitch ideas had been written before. Someone, somewhere had a take on my idea already published. The abundance of AI writing undermined my “just raise awareness” theory of change. 

Deeper experiment 3: 

Since I was now optimistic I would enjoy some jobs in journalism, my new key uncertainties were: Could I come up with a better, more nuanced theory of change? Could I get pieces published or get a job in journalism?  

I applied to the Tarbell Fellowship. This included work tests (i.e. extra personal fit tests), an external evaluation, and a good talk about theories of change, which left me with a few promising routes to impact. (Yes, applying to roles is scary and time consuming! It’s also often a very efficient way to test whether a career path is promising.) 

Future tests: 

Now my key uncertainties are about how I’ll do on the job: Will I find it stressful? Will I be able to write and publish pieces I’m excited about? Will I still have a plausible theory of change after deepening my models of AI journalism? 

It still feels like I’m plunging into things I’m not fully prepared for. I could spend years practicing writing and avoiding doing anything so scary as scaling down coaching to work at a journalism org – at the cost of dramatically slowing down the rate at which I learn. 

Tips for you to apply

Tool: Do tests to more deeply investigate your top options, especially your key uncertainties. 

Start with the cheapest, easiest tests, and work up to deeper tests. I also recommend doing cheap tests of a few things before going deeper with one, especially if you’re early in your career. This post has advice and tons of case studies on making career decisions. 

You want to address the key uncertainties you identified. Key uncertainties are the questions that make you change your plans. You’re not trying to know everything - you’re trying to make better decisions. This is very important. There are a thousand and one things you could learn, and most of them are irrelevant. You need to focus your attention on the questions that might change your decision. 

Spread out asking for advice, roughly from easy to hard to access. So, if you can casually chat with your housemate about a decision, start there! Later when you need feedback on more developed ideas, reach out to the harder to access people, e.g. experts on the topic or more senior people who you don’t want to bother with lots of questions. Some possible questions: Does this seem reasonable to you given what you know? What is most likely to fail? What would you do differently? Here is additional good advice on asking feedback from Michelle Hutchinson. 

Learning from Tests 

While planning my tests, I realized I needed a system of tracking them. I’d miss important lessons if I didn’t have a reliable method for checking the result afterwards, updating on the new information, and planning the next set of experiments. 

So I started using what I called hypothesis-driven loops. 

Each week during my weekly planning, I would also plan what data I was collecting to help with career exploration. I wrote down my questions, my plan for collecting data, and what I currently predicted I would find. 

All of this made it easier to notice when I was surprised. I was collecting data from recent experiences when it was fresh in my mind. Because I was writing in advance what I guessed I’d find, it was easy to notice when I actually found something quite different. 

It allowed me to plan what I wanted to learn from my goals each week, have that in the back of my mind, and reliably circle around the next week to write down any info. This left a written trail documenting the evolution of my plans.  

Example

Do a 1-week experiment where I draft a new post every day. Hypothesis: ADHD meds will make it easier for me to write quickly and reliably. Drafting a post a day will still be difficult but might be achievable. (80% I'll be able to work on a new draft each day. 70% I'll finish at least 2 drafts. 30% I'll finish all 5.) 

•  I was able to do 4/5 drafts, but got a headache the last day. I finished two of them basically, and have 2 more in progress. Results consistent with ADHD meds making it a lot easier for me to write, though I want to evaluate the posts, test revisions, and check whether this pace is sustainable next week. 

Talk with Matt and Callum about BlueDot. Hypothesis: It will pass this first test for scalability (65%). 

•  It passed the scalability test, but Callum is going to try exploring himself. I think he's a better fit than I am, so happy to let him try it out. I'll reevaluate later if he doesn't continue or I get new information. 

Reach out to people/schedule interviews/have interviews/draft post in 1 week: ambitious goal, but seems good to try for my goal of writing faster. 30% I can have a draft by the end of the week. 70% I can do at least 2 interviews. 

•  No draft, but I did six interviews.

Tips for you to apply

Set it up somewhere you’ll reliably come back to. The entire point is coming back to update from what you learned, so this exercise is basically useless if you don’t close the loop. I have a todo that I write down the experiments in while doing weekly planning on Saturday, and then I set the due date to be the following Saturday so I’m reminded each week. 

State what your plan is for getting information. I tie this into my goals for the week. Usually it looks something like glancing over my goals for the week and seeing if any of my goals are at least partially there to learn from. Then I make hypotheses based on those. This ties together the learning goal with the plan for getting the information. 

Tracking Probabilities 

While I was doing tests, I tracked the probability I assigned to different options, as well as noting what information caused big swings in my probabilities. This let me notice and track changes to my expectations around which option was best. 

I updated my spreadsheet weekly for the first couple months while I was exploring and learning about options (roughly corresponding to the period when I was brainstorming, writing project briefs, and doing tests taking <1 week). Once I started longer tests, I switched to monthly, until I committed to Tarbell. 

Example

Tips for you to apply

Here’s an excerpt from my post on How Much Career Exploration Is Enough?

Track when you stop changing your mind about which option is best. Even with an important decision, you only want to spend more time exploring as long as that exploration is changing your mind about which job is most valuable. 

To measure how much your mind is changing, you can set up a spreadsheet with probabilities for how likely it is you’ll do each of the options you’re considering. 

Try to write down numbers that feel reasonable and add up to 100%. (To avoid having to manually make the numbers equal 100, you can add numbers on a 1-10 scale for how likely each option feels, and then divide each number by the total.)

Template sheet with example 

Each week while you’re exploring, put in your new numbers. As you get more information and your decision is more solid, the numbers will slowly stop changing (and one option will probably be increasingly closer to 100%).

When your probabilities are changing wildly, you’re still gaining more information. Once they taper off, you’ve neared the end of productive returns to your current exploration. 

Reevaluation Points

Both tests for productivity blogging and AI journalism were both promising. I would be happy continuing productivity blogging, if AI journalism turns out to be a poor fit. 

In the end though, AI just seems too important to ignore. So I committed to the Tarbell Fellowship. I spent the spring doing an AI journalism spinning up program, and this month I’m starting the nine-month main portion of the fellowship. 

After those nine months, I’ll reevaluate. I’ve got a set of key questions I’m testing, including about theory of change and personal fit. I also expect I’ll learn tons of things that I don’t know enough now to ask about. 

I already learned so much about journalism – including that several of my assumptions during my early tests were wrong. For example, I focused so much on writing speed, whereas now I would have focused on reporting. 

Despite all the flaws in my understanding at the time, I still think the tests were a reasonable approach to making better informed decisions. The iterative information gathering allowed me to make measured investments until I was ready to make a big commitment.  

Now I’ll dive into journalism. At the end of the fellowship, I’ll decide what comes next. Right now, I hope to continue. I’m confident this is worth trying, even though I’m not confident what the end result will be. But I’m doing my best to set myself up for success along the way, including telling all of you that I’m doing this (if you have stories that should be told, please let me know!) You might already have noticed the changes to my website. 

In the meantime, I’ll be writing less on this blog. Come check out my substack for AI writing instead! 

Tips

After you commit to one career path, reevaluate periodically to check your work is a good fit and as impactful as you hoped. Your work may have natural evaluation points (such as a nine-month fellowship). Otherwise, about once a year often works well. At each reevaluation point, ask yourself if you learned anything that might make you want to change plans? Are there new options that might be much better than what you’re currently doing? If so, do another deep dive. If not, keep doing the top option. 

Credits

Thanks to everyone who gave me advice and feedback on this long journey. Special thanks to Kit Harris, Anna Gordon, Garrison Lovely, Miranda Dixon-Luinenburg, Joel Becker, Cillian Crosson, Shakeel Hashim, and Rohin Shah.

Want to understand why particular approaches fail?

Read on! 

Some career decisions are easy. There’s a clear best option. 

Many are not easy. Where are you a good fit? What would you be best at? What work is most important? 

These questions can feel baffling. How do you even start getting the information to make these decisions well? 

Do you wish you could just look at how successful people made their decisions? What made them confident in their plans?

Well, here you can. Here are 10+ real stories of career decisions. Each story is a real account from a real person. You can see what works -- and what doesn’t. 

Even better, these stories illustrate three big tools for making important, complex decisions. You’ll learn high-level strategies for efficiently gathering vital information, so that you’ll be able to confidently make decisions.  

In case you’re wondering how I came up with these, I brainstormed 25+ examples, clustered them, got feedback from a few people, and then selected the best examples for the most important clusters.

So, why should you read 6k words of stories? Why don’t I just give you the tools in a bullet list? 

Because abstract concepts alone aren’t enough. There are already lots of resources for generic advice. But hard career decisions are nuanced and complex. 

When you only have the abstract advice, it can be hard to apply to all of the situations where you need it. Personal fit tests for someone exploring bio careers are going to look very different than my tests for journalism. It’s also easy to think you understand the advice, when actually you’re missing important details. 

For example, 80,000 Hours has some great advice on investigating uncertainties. However, beyond talking to people and reading their posts, their advice for testing is “Look for ways to test your uncertainties.” They give one example of what this might look like, but one example isn’t enough. That sentence deserves its own entire blog post! 

This is that blog post. The stories teach you the details to flexibly apply these tools. They’re meant to convey the tools in depth so you can apply them yourself in a variety of situations. Because, of course, career decisions don’t stop once you have a job. Doing research, starting a charity, or planning your team’s next project all benefit from the same tools.

And, hey, if you’re not sold on reading the case studies, you can always jump to the conclusion to read the super high level summary of the whole process. 

But I hope you’ll read the whole thing. I think most of the value comes from seeing the strategies in action. 

Data rich experiments 

Identify your key uncertainties, then find the richest real-world data possible to help you reduce them. 

For data rich experiments, you want to get as close to the real-world setting as possible. If you’re testing job fit, you want to try doing work that’s as close as possible to the real job. 

You’ll need to start by identifying your key uncertainties, then find ways to quickly get data to reduce those uncertainties. 

Data rich experiments usually structure effort in the order that decreases your uncertainty most quickly (e.g. start with most likely to fail and work towards least likely to fail). You want to tackle your biggest uncertainties quickly with tests that yield a lot of info.

Rich data is usually external. There’s a detailed, nuanced world out there, and you need to have lots of data points from bumping into it. Get your hands dirty! Go talk to people, try things, learn how the world works. 

For my career exploration, “finding the richest data” meant looking for experiments where I expected to be surprised. I couldn’t easily predict what the outcome would be. For example, I didn’t know what I would discover by submitting pitches. I expected more information on whether I liked the activity, but it was novel enough that I expected to learn surprising things. Just trying something new is often a rich source of data since my models were poor before. (If I’m repeating things I’ve done many times, I don’t expect to learn much.)

I did some tests as “sanity checks” to confirm I didn’t have gaping misunderstandings. But in general, if I expect a certain outcome, I don’t learn much by testing it. So I look for those tests where I can’t confidently predict the outcome. 

Better yet, I look for tests that will give me detailed, nuanced feedback. If I’m just testing whether I can draft a post in a day, the answer is yes or no. A richer data set is investigating what factors predict how fast I draft a post. An even richer data set (for career exploration) would have been testing how quickly I can draft a journalism-style piece and comparing it against a published piece. Here the data is getting richer the more detail I’m extracting and the closer the test is to the real-world scenario I’m testing. 

Question to ask yourself: What are my key uncertainties right now? What are the most detailed, closest-to-real scenario tests I could run to reduce those uncertainties? 

Extra:

1. This method clicked for me when I sought out tests where the result was high variance - they could go either way, and I couldn't predict in advance what I would learn. I suspect that’s often a good litmus test for telling whether a test might yield rich data. 

Asking other people for advice or reading works about the topic are a good starting point for finding data rich tests. My career tests were based on advice from people in the field. My user testing interviews were based on the process in the book Sprint. 

2. Do what’s uncomfortable. 

By definition, testing things means you’re not sure what the outcome will be. That’s often uncomfortable. For example, you can spend hours debating the relative merits of two jobs, but if your biggest uncertainty is whether you can get job offers, you should probably just apply and then compare the job offers. You’ll save yourself a lot of time that wasn’t ultimately moving you forward. 

In my therapy platform project, I was so tempted to just start designing evaluations. I could have worked quite happily to create a project I loved – without realizing it had serious flaws that would later make it fail. Instead, I got feedback early, far before I felt ready to do so. Because I got that feedback early, I could pivot without wasting too much effort on the discarded idea. 

Don’t wait until you feel totally ready to share your idea. Test your assumptions, get feedback, and crash your idea against reality early and often. 

Question to ask yourself: Is there a test that would give me valuable information that I’m avoiding because it feels ughy or scary or uncomfortable? 

Ben Kuhn’s hiring process

Ben Kuhn uses a careful approach to planning behavioral interviews to extract enough data that he can make informed hiring decisions. 

“One of the worst mistakes you can make in a behavioral interview is to wing it: to ask whatever follow-up questions pop into your head, and then at the end try to answer the question, “did I like this person?” If you do that, you’re much more likely to be a “weak yes” or “weak no” on every candidate, and to miss asking the follow-up questions that could have given you stronger signal.”

He plans detailed questions in order to get rich data that is close to how they would likely perform if he hired them. For example, he emphasizes digging into the details. “Almost everyone will answer the initial behavioral interview prompt with something that sounds vaguely like it makes sense, even if they don’t actually usually behave in the ways you’re looking for. To figure out whether they’re real or BSing you, the best way is to get them to tell you a lot of details about the situation—the more you get them to tell you, the harder it will be to BS all the details.” 

Giving What We Can’s failed forum 

Michelle: When we were building a forum for Giving What We Can, we were trying to build a forum that was exactly of the kind that we wanted, and it ended up really blowing out as a project. 

I hadn't managed a tech project before. We were still mostly volunteers, and it ended up taking many months and just totally wasn't worth it. It didn't end up being used very much at all. 

I think that's a case where I hadn't properly zoomed out and been like, "Okay, how important actually is this, and at what point should we pivot away from working on this, even if we put quite a bit of time into it?" 

Lynette: Sounds good. Knowing what you do now about zooming out, what would you have done differently early on? Concretely, what would that have looked like?

Michelle: I think it probably would have looked like doing more of a minimum viable product. I think we considered this at the time and our worry was that a forum only works if you get enough people on it. If you do something that's fine but not great, then you get a few people, and you just don't get enough. It's bound to fail. 

I think, for example, the Giving What We Can Community Facebook group, which I think is what we ended up going with, has actually done pretty well and got fairly good engagement on it. I think I might have just ended up sticking with that. 

I might have tried something like a Google Group with some different threads and then sent that around and been like, "Do people want something like this?" I guess something that was quick to build and could immediately see whether people were using it. 

Testing key uncertainties in my career exploration

Around the beginning of 2023, I started a career review to decide if I wanted to do something besides coaching. 

I liked writing and thought journalism seemed potentially impactful. However, I knew basically nothing about what journalism actually involved day-to-day and I had only a vague theory of change. So my key uncertainties were: What even is journalism? Would journalism be high impact/was there a good theory of change? Would I be a good fit for journalism? 

Cheapest experiments: 

So I started by doing the quickest, cheapest test I could possibly do: I read 80,000 Hours’ profile on journalism and a few other blog posts about journalism jobs. This was enough to convince me that journalism had a reasonable chance of being impactful.

Meanwhile, EA Global rolled around and I did the second cheapest quick test I could do: I talked to people. I looked up everyone on Swapcard (the profile app EAG uses) who worked in journalism or writing jobs and asked to chat. Here my key uncertainties were: What was the day-to-day life of a journalist like? Would I enjoy it? 

I quickly learned about day-to-day life. For example, the differences between staff and freelance journalism jobs, or how writing is only one part of journalism – the ability to interview people and get stories is also important. I also received advice to test personal fit by sending out freelance pitches.

Deeper experiment 1: 

On the personal fit side, one key skill the 80,000 Hours’ profile emphasized was the ability to write quickly. So a new, narrowed key uncertainty was: Can I write fast enough to be a journalist?

So I tried a one-week sprint to draft a blog post each day (I couldn’t), and then a few rounds of deliberate practice exercises to improve my writing speed. I learned a bunch about scoping writing projects. (Such as: apparently, I draft short posts faster than I do six-thousand-word research posts. Shocking, I know.) 

It was, however, an inconclusive test for journalism fit. I think the differences between blogging and journalism meant I didn’t learn much about personal fit for journalism. In hindsight, if I was optimizing for “going where the data is richest”, I would have planned a test more directly relevant to journalism. For example, picking the headline of a shorter Vox article, trying to draft a post on that topic in a day, and then comparing with the original article. 

Deeper experiment 2: 

At this point, I had a better picture of what journalism looked like. My questions had sharpened from “What even is this job?” to “Will I enjoy writing pitches? Will I get positive feedback? Will raising awareness of AI risks still seem impactful after I learn more?” 

So I proceeded with a more expensive test: I read up on how to submit freelance pitches and sent some out. In other words, I just tried doing journalism directly. The people I’d spoken with had suggested some resources on submitting pitches, so I read those, brainstormed topics, and drafted up a few pitches. One incredibly kind journalist gave me feedback on them, and I sent the pitches off to the black void of news outlets. Unsurprisingly, I heard nothing back afterwards. Since the response rate for established freelance writers is only around 20%, dead silence wasn’t much feedback. 

Instead, I learned that I enjoyed the process and got some good feedback. I also learned that all of my pitch ideas had been written before. Someone, somewhere had a take on my idea already published. The abundance of AI writing undermined my “just raise awareness” theory of change. 

Deeper experiment 3: 

Since I was now optimistic I would enjoy some jobs in journalism, my new key uncertainties were: Could I come up with a better, more nuanced theory of change? Could I get pieces published or get a job in journalism?  

I applied to the Tarbell Fellowship. This included work tests (i.e. extra personal fit tests), an external evaluation, and a good talk about theories of change, which left me with a few promising routes to impact. (Yes, applying to roles is scary and time consuming! It’s also often a very efficient way to test whether a career path is promising.) 

Future tests: 

Now my key uncertainties are about how I’ll do on the job: Will I find it stressful? Will I be able to write and publish pieces I’m excited about? Will I still have a plausible theory of change after deepening my models of AI journalism? 

It still feels like I’m plunging into things I’m not fully prepared for. I could spend years practicing writing and avoiding doing anything so scary as scaling down coaching to work at a journalism org – at the cost of dramatically slowing down the rate at which I learn. 

Learning my idea sucked 

At EAG 2022, I stumbled onto some conversations about the need for vetted EA therapists and coaches. Could we thoroughly evaluate a few providers? We could provide detailed insight into their methods and results, so that EAs could find providers with solid evidence of impact. Would it be possible to identify coaches who get 10x the results for their clients? 

I was really excited about this idea. 

As I’m also a fan of lean methods, I designed some user testing interviews for my idea. If I was to build something like a mini-GiveWell-style evaluator for therapists and coaches, it would probably take months or years of effort. Before I invested that, I wanted to refine my idea. The goal wasn’t to build the vetting process more quickly at first, the goal was to figure out what vetting process to build. 

So I followed the process outlined in Sprint. I built mock prototypes with fake data showing the kinds of evaluation info I hoped to include, plus Calendly links and availability data. I scheduled user testing interviews and watched how they interacted with the prototypes. I also hopped on calls with a few providers to discuss the possibility of including them in the vetting process. 

Basically, I jumped into the data as richly as possible. And it told me that my precious, exciting idea...sucked.

First, the providers had limited space for new clients. One had a ten-month waiting list to work with her. Even if each provider could take on 20 or 30 clients, they already had full schedules and would only gradually have space for new clients. My plan for “a few, highly vetted providers” would only benefit a small number of clients.  

Second, the potential users didn’t place much weight on the evaluations. They cared more about whether a provider was covered by their insurance or whether they specialized in a particular issue. 

Both of these indicated that a larger database of providers would be better, even if that meant forgoing the deep vetting process I’d wanted.  

So I let go of my original plan. That sucked a bit. I’d been excited about it. 

One piece of advice I got to help this kind of thing was “It’s much easier to go towards rich data if you’re curious or open. If you’re finding internal resistance, then try to ramp up curiosity.” Here I felt curious about what would help EAs most, so I got excited about the new idea pretty quickly. 

I worked with the EA Mental Health Navigator to revamp and expand their provider database. This was a much smaller and quicker project that, I think, turned out strictly better than my original idea. 

Jacob Steinhardt’s approach to failing fast in research 

For those of you more mathematically minded, Jacob Steinhardt has a beautiful post about how to reduce uncertainty most efficiently. Specifically, he recommends that you structure your research in the order that decreases your uncertainty most quickly. E.g. start with most likely to fail and work towards least likely to fail. (Additionally, multiple clients have told me it was helpful for speeding up their research as well.)

He claims this method quadrupled his research output. 

I’ve included a few excerpts from it here: 

Suppose you are embarking on a project with several parts, all of which must succeed for the project to succeed. For instance, a proof strategy might rely on proving several intermediate results, or an applied project might require achieving high enough speed and accuracy on several components. What is a good strategy for approaching such a project? For me, the most intuitively appealing strategy is something like the following:

(Naive Strategy)
Complete the components in increasing order of difficulty, from easiest to hardest.

This is psychologically tempting: you do what you know how to do first, which can provide a good warm-up to the harder parts of the project. This used to be my default strategy, but often the following happened: I would do all the easy parts, then get to the hard part and encounter a fundamental obstacle that required scrapping the entire plan and coming up with a new one. For instance, I might spend a while wrestling with a certain algorithm to make sure it had the statistical consistency properties I wanted, but then realize that the algorithm was not flexible enough to handle realistic use cases.

The work on the easy parts was mostly wasted--it wasn't that I could replace the hard part with a different hard part; rather, I needed to re-think the entire structure, which included throwing away the "progress" from solving the easy parts.

Rather, we should prioritize tasks that are more likely to fail (so that we remove the risk of them failing) but also tasks that take less time (so that we've wasted less time if one of the tasks does fail, and also so that we get information about tasks more quickly).

A Better Strategy: Sorting by Information Rate

We can incorporate both of the above desiderata by sorting the tasks based on which are most informative per unit time.

(Better Strategy)
Do the components in order from most informative per unit time to least informative per unit time.

Example 1: All of the steps of a project have roughly equal chance of success (80%, say) but take varying amounts of time to complete.

In this example we would want to do the quickest task first and slowest last, since the later a task occurs, the more likely we will get to skip doing it. Sorting "easiest to hardest" is therefore correct here, but it is rare that all steps have equal success probability.

Example 2: An easy task has a 90% success probability and takes 30 minutes, and a hard task has a 40% success probability and takes 4 hours.

Here we should do the easy task first: if it fails we save 240 minutes, so 0.1 * 240 = 24 minutes in expectation; conversely if the hard task is done first and fails, we save 30 minutes, for 0.6 * 30 = 18 minutes in expectation. But if the hard task takes 2 hours or the easy task has a 95% chance of success, we should do the hard task first.

Thus, in this method we formalized "most informative per unit time" by looking at how much time we save (in expectation) by not having to do the tasks that occur after the first failure. 

Iterated depth

The idea of iterated depth is that you start with a wide range of options, shallowly explore them, move the most promising onto the next round of exploration, and repeat with increasingly in-depth explorations. 

1. Start with small tests and build up. 

Start with the cheapest, easiest tests that are teaching you new information. For Kit, this meant asking a friend about their job, reading a blog post about what it’s like to work in that industry, or looking at available job postings. For Allan or CE, it means doing a 30-minute write up before committing to spending hours or months on a project. 

Go deeper with more extensive and expensive tests as you need to but ONLY as you need to. Don’t do a month-long project before you’ve spent a weekend trying it out. Don’t do a weekend-long project until you’ve spent an hour reading about the job. 

Question to answer: What are the smallest, cheapest tests that would have a decent chance of changing my mind about which options are the best fit for me? 

2. Start with a big top of the funnel and gradually narrow. 

Don’t make decisions based on only one data point. If the first option you tried goes okay, still try more options. You need the ability to compare multiple options to see when something is going unusually well vs just okay vs unusually horribly. 

Serendipity and chance play a big role in career exploration, but you want at least a few data points so that you’re optimizing for the best out of five or ten options, rather than the best out of one or two. 

Start with more career paths or specific jobs or project ideas than you can follow, and do a tiny bit of exploration in several. Then go deeper on a select few, narrowing the number you’re trying with each step. By the time you’re accepting a job or committing to a project, you want to have explored several other paths. 

Question to answer: Am I considering at least three options (ideally ten plus options) for early stages of the funnel? 

Extra:

You likely won’t feel convinced you know enough to cut off all the other paths. You’re trying to gain more robust clarity – where it’s harder to get new information that would change your choice. (See How much career exploration is enough?) 

You might end up going back up a step if a deep dive doesn’t pan out, say if an internship doesn’t go as well as you would like. That’s a good time to reconsider other paths that seemed promising. 

Kit’s career exploration

When Kit Harris wanted to reassess his career, he started from scratch. (You can read his longer account of the process here.)

He identified 50 potentially high-impact roles spanning operations, generalist research, technical and strategic AI work, grantmaking, community building, earning to give, and cause prioritization research.

Then he roughly ranked them by promisingness and explored the top 10 ideas more deeply. “At first, the idea of choosing 1 next step from 50 ideas was quite overwhelming. Explicitly arranging the ideas made exploration much more approachable.”

He started with brief explorations, such as: 

• Talking to someone working in a similar role

• Talking briefly with potential collaborators

• Beginning but not finishing an application, learning from the process what might make me a good fit

Then he advanced to deeper investigations of the most promising ideas, such as: 

• Selecting small projects which seemed representative of the work and trying them

• Applying for a position which had work tests in the application process

• Contracting for a relevant organization

• Interviewing relevant people and presenting an organization with a project plan

At the end of the process, he spent time contracting for Effective Giving UK (now Longview Philanthropy), and ultimately accepted a full-time job there. Kit wrote that he felt “quite confident” in his decision. 

Founding Charities 

To choose which charities to found, Charity Entrepreneurship uses an iterative depth approach. 

“With our 2020 charity research, that meant doing a quick 30-minute prioritization of hundreds of ideas, then a longer two-hour prioritization of dozens of ideas, and, finally, an 80-hour prioritization of the top five to ten. Each level of depth examines fewer ideas than the previous round, but invests considerably more time into each one.” From How to launch a high-impact nonprofit

GiveWell and Open Philanthropy use a similar process for selecting priority charities. 

You can read a longer description here of Charity Entrepreneurship’s process from an early round.

Allan Dafoe’s shallow research tests

Jade Leung’s account of how her former professor, Allan Dafoe, selected research projects: 

“He always has a running list of research ideas, way more than he could ever get done. He explores more ideas than he would actually be able to follow through with.

By this process of light experimentation, he delves into the idea and tries to understand it a little bit better. Maybe writing up a couple of pages on it and getting into the mode of actually investigating it. 

I think that gave him a bunch more data about whether this was a project that actually had the legs that he thought it could have, whether it felt pleasant and fun to work on, whether it felt like it was exploring a bunch of other ideas, or whether it felt a bit flat.”

An easy way to try this is to keep your attention open for promising ideas and have a place you can jot these down with minimal friction.

Closing the loop

Closing the loop is about having systems to constantly experiment, learn from the tests, and plan new experiments based on your updated hypothesis. This process of constantly iterating enables the iterated depth and data rich methods above. 

There’s a leap of faith to committing to a career path. You have to make commitments before you have deep models that can really inform you, so you’re always making the choice to start something based on less information than you will have later. 

But you want to reevaluate that path once you get more information. You want to check whether you’re still on the right path as your model becomes more granular. Related: Theory of Change as a Hypothesis

To do this, you want to be able to run longer experiments, and know you’re going to circle around and update your hypothesis based on the data you gathered. 

For me, reevaluation points are a good way to balance bigger experiments with frequent check-ins. Quarterly reevaluation points especially help when I’m feeling down or pessimistic about my work. When this happens, my instinct is to immediately reevaluate my entire career path. Having a record of the reasons behind my plans is immensely helpful when I feel like my blog draft is terrible and should never see the light of day and maybe I should immediately quit blogging forever. 

Extra: 

Keep your feedback loops small. 

When running experiments, you don’t want the end of a year “exploring” research to be the first time you analyze how it’s going. Even for longer experiments, you want smaller loops of collecting more granular data. 

Build up to big conclusions with regular bits of data. Keep close to the data, so that you’re constantly making little bits of contact with it. 

Failing to learn from exploration 

I spent the year after college doing psych research as a test for whether I wanted to do grad school. At the end, disillusioned with psych research, I knew the answer was no. 

I just had no idea what I did want to do. 

I spent a year “testing” whether I wanted to do research, but at no point was I actually collecting data or even thinking about what I wanted to learn from that test. 

I wasn’t repeatedly circling around to plan what data to collect, collecting it, updating beliefs, and then planning new tests. In other words, I wasn’t closing the loop. 

I could have broken my key uncertainties (e.g. will I be a good fit for a PhD?) into small components and looked for data (or ways to get data) on them. I could have kept a log of which elements of my job I enjoyed and which I slogged through. I could have noted where I got praised and where I received silence or negative feedback. I could have done some tests of other options in my free time. I could have asked other people about their labs, to check whether my experience was typical for the field.

Ideally, I would have had a system where I thought weekly about what I learned and kept a log of updates. I could have planned questions in advance to pay attention to during the week. 

Instead, I just did my work and tried to reflect at the end. Then, I could generate career ideas but I didn’t have good data points from which to evaluate my ideas. I didn’t have a good sense of my strengths or weaknesses. I had to work to even figure out what I liked. 

I think this was really a wasted opportunity. 

Developing a weekly system 

I wanted a good way to regularly circle back to close the loop on little experiments. So I came up with this process of hypothesis driven loops. 

The basic idea was: while planning my week, I would also plan what data I was collecting to help with career exploration. I wrote down my questions, how I was collecting data, and what I currently predicted I would find (plus how confident I was). 

All of this made it easier to notice when I was surprised. I was collecting data from recent experiences when it was fresh in my mind. Because I was writing in advance what I guessed I’d find, it was easy to notice when I actually found something quite different. 

It allowed me to plan what I wanted to learn from my goals each week, have that in the back of my mind, and reliably circle around the next week to write down any info. This left a written trail documenting the  evolution of my plans.  

Some examples: 

When I was trying to assess whether I should continue working on the ML engineer accelerator program ARENA after the first iteration. One of my key uncertainties was whether the other leaders running ARENA thought it was worth continuing. 

Hypothesis: When I talk with Matt and Callum about ARENA, it will pass this first test for scalability (65%). 

Result: It passed the scalability test, but Callum is going to try exploring himself. I think he's a much better fit than I am, so I’m happy to let him take it on. I'll reevaluate later if he doesn't continue or I get new information. 

When I was drafting the ADHD post. I wanted to test how long it took me to conduct interviews, since I hadn’t done that for a post previously. 

Hypothesis: Reach out to people/schedule interviews/have interviews/draft post in 1 week: ambitious goal, but seems good to try for my goal of writing faster. 30% I can have a draft by the end of the week. 70% I can do at least 2 interviews. 

Result: I didn’t get around to drafting, but I did six interviews. 

Cedric Chin’s hypothesis system 

You Aren't Learning If You Don't Close the Loops by Cedric Chin, author of CommonCog, emphasizes the importance of studying the results of experiments to actually learn from your trials. His system is a good example of systems for exploring bigger key uncertainties over time. Excerpt: 

“Here at Commoncog we spent four months working on our Burnout Guide, summarizing the bulk of burnout research in one easy-to-read piece. The original intention for this guide was to see if we could use a set of commonly known SEO techniques to grow the site’s rankings. But the guide went viral on launch. I was so distracted by the distribution, the positive feedback and the attention that the guide was getting that I nearly forgot about the original hypotheses that we had. It was only when I consulted the original 6-pager I wrote at the outset that I realised we needed to test certain things; the virality was nice but not the main purpose of this particular execution loop.

(This is, by the way, an explicit recommendation to write out your hypotheses before you execute. It doesn’t matter if you jot it down in a 6-pager format or a Google Doc or whatever; the point is that you’re likely to forget your original goals by the end of a loop if you don’t put things down on a page.)”

Conclusion

The next time you need to make a complex, important decision where it’s worth it to put in 100 hours, you should upfront plan a process that looks something like this. 

1. Start with brainstorming a broad top of the funnel.

2. Identify your key uncertainties. 

3. Get data points as close to the real-world setting as possible so you can compare.

4. Iteratively narrow and deepen the best ones. 

5. Structure your effort in the order that decreases your uncertainty most quickly. E.g. start with most likely to fail and work towards least likely to fail.  

6. Have systems that let you regularly circle back to check what you’re learning and update your hypotheses.

Hopefully the case studies helped you think about how to do that. If you want more help, please reach out! I’d love to help design great career tests. 

Related resources: 

My blog post Theory Of Change As A Hypothesis: Choosing A High-Impact Path When You’re Uncertain was an earlier attempt to apply the importance of regular iteration for reducing uncertainty in career choice.  

Logan Strohl’s naturalism posts are one way to think more about getting in close contact with the data. My guess is that you’ll know pretty quickly whether you click with his style, so feel free to check it out and move on if this one isn’t for you. 

Reading about LEAN methods, particularly The Lean Startup and similar takes on lean as a “method for iterating quickly to reduce uncertainty” is another helpful take.

Ideally, you want to trade off the chance new information changes your mind against the effort it takes to get new information. The longer you spend exploring, the less time you have to actually get stuff done. (More or less -- it’s not a strict trade off since some exploration is also directly useful.)

Except, it’s really hard to do that calculation robustly.

One alternative is to explore, and track when you stop changing your mind about which option is best. Even with an important decision, you only want to spend more time exploring as long as that exploration is changing your mind about which job is most valuable.

To measure how much your mind is changing, you can set up a spreadsheet with probabilities for how likely it is you’ll do each of the options you’re considering.

Try to write down numbers that feel reasonable and add up to 100%. (To avoid having to manually make the numbers equal 100, you can adding numbers on a 1-10 scale for how likely each options feels, and then divide each number by the total.)

Template sheet with example

Each week while you’re exploring, put in your new numbers. As you get more information and your decision is more solid, the numbers will slowly stop changing (and one option will probably be increasingly closer to 100%).

When your probabilities are changing wildly, you’re still gaining more information:

Once they taper off, you’ve neared the end of productive returns to your current exploration:

Ideally, they taper off when you’re confident in one option, as in the above graph. Sometimes, however, they taper off to where you have stable preferences, even if you’re not confident they are optimal, like in the below graph. In this case, you still probably want to commit, at least for a set period of time.

When you have several weeks without new information that changes your probabilities (and you don’t have information you’re expecting to learn soon), you’ve hit diminishing returns on new information. It’s probably time to commit.

After that point, you can reevaluate periodically (I’d recommend about once a year). At each reevaluation point, ask yourself if you learned anything that might make you want to change plans? Are there new options that might be much better than what you’re currently doing? If so, do another deep dive. If not, keep doing the top option.

Related posts:

80,000 Hours’ How much should you research your career?

Greg Lewis’s Terminate deliberation based on resilience, not certainty

2023 has been a big year for me. I got married and brought home an adorable puppy named Rosie.

Perhaps appropriately, my first post this year was Rewriting My Mindset: My Experience With CBT For Perfectionism, in which I shared the particular exercises I found helpful for relaxing some of my more counterproductive mindsets. With the benefit of hindsight, I can say that I felt more comfortable and less anxious publishing posts and trying things this year.

My first theme this year was stimulants. My interest stemmed from finding stimulants helpful for the brain fog I’d always attributed to my chronic health issues. I started with Inside The Minds Of ADHD, where I explored what ADHD looked like from the candid stories of people diagnosed with ADHD. Then How to Get Diagnosed with ADHD where I looked at the practical side of the medical process, and finally with What is ADHD? I pushed back against the framing of “Do you have the medical condition ADHD?” instead of the more (in my opinion) useful framing of “Will stimulants help you?”. I’m planning one more post, about how my thoughts have changed over a year of using stimulants.  

In Do Deadlines Make Us Less Creative?, I explored the psychology of creativity and the tradeoffs we make for productivity.

Detouring to a fun, lighthearted post – Why do Tattoos last? explores why tattoos don’t slough off with dead skin cells. (Hint, the cells holding the ink do die, but the body has a way around throwing out the ink with the dead cells.)

My second major theme this year was studying learning, particularly deliberate practice. I started exploring some of the theoretical claims made by deliberate practice supporters in Leveling Up Or Leveling Off?, then did a deep dive into applying deliberate practice* to unstructured contexts in DIY Deliberate Practice. (*Technically, more accurately I was applying “purposeful practice”, but I’m going to ignore the pedantry to use the more commonly understood phrase.) I finished up with Of Practice And Paintings on the benefits of practicing within the scope of your normal work, rather than purely in isolated exercises.

More broadly, I spent a good amount of time this year studying learning. How quickly and reliably we can learn new skills, bodies of knowledge, or from our own experiments seems incredibly important for what we can accomplish. So I’m studying a variety of approaches to learning, including career exploration, lean methods, naturalism, and hypothesis driven experiments.

 What’s coming next year?

I expect you’ll see more posts about optimizing learning next year! In particular, I’m hoping to refine a few tools I use so you can try them for yourself.

Additionally, I’m trying out letting people commission posts. My goal for this blog is to produce valuable, actionable insights you can apply yourself, by sharing my journey, concrete examples, and underlying theories. One way to test value is seeing what people will pay for.

To that end, if there’s a particular topic you wanted to hear more from me about, this is your chance! Ideas for posts you could commission include:

• Guides for particular productivity tools or strategies.

• Reviews of different strategies for addressing a particular problem.

• Exploring how robust a particular productivity tool is to scientific literature and practical application.

• Trying to quantify what effect you might experience from trying a specific tool or strategy.

• An experiment that I run and write up the results.

While I’m figuring out exactly how this will work, I’m offering a $50/hour rate (this will go up later to closer to what I value an hour at). I’d start with an ~5-hour project to test we’re a good fit, which can continue if we’re both excited about where the project is going. 

If you’d like to discuss a potential post, schedule a call with me here. 

Thanks for coming along on this journey. I hope you found these posts as interesting and useful to read as I did to research and write them.

Cheers to 2024,

Lynette

“What on earth is a penguin doing in the bathtub?!?”

___________________________________________________

This is how I wish we reacted when our plans failed.

I set a deadline for myself, and completely missed it? Astonishing! Something surprising happened here, and I want to figure out why!

Instead, it often feels like goals and deadlines are treated like vague aspirations. You simply fail to start writing. The Friday deadline slips past without a whimper. You shrug and move on, because you didn’t really expect it to happen.

Now imagine you treated it like a penguin in the bathtub instead?

You would never just shrug and go about your day if you found a penguin in your bathtub. You would be asking questions and trying to get to the bottom of the mystery.

Similarly, you shouldn’t shrug that your best plans went awry. Maybe it’s okay if that happens every now and then, but if you expect your plans to succeed 80% of the time, then you should notice if they only succeeded 20% of the time! This should be surprising. It’s not inevitable - you can learn how to be accurately calibrated about your plans, though there may be many steps on the path there.

Try to figure out the mystery. Treat yourself as a system and actually try to optimize it. Can you trace back the steps that led to this point? How do you fix it?

If it's Saturday and you didn't finish the paper sections you’d planned to draft on Friday, ask yourself “How did I get here?” --> Huh, so on Friday I felt tired and unmotivated. I felt tired because the three nights before I ended up going to bed late, while still getting up early to push through my work week. By Friday I sort of ran out of gas, and the thought of tackling the paper seems overwhelming. Also, I forgot to schedule coworking sessions.

...thus, maybe, if I want to increase the likelihood of my writing happening, I need to make sure I get an early night on Thursday, and set a reminder to schedule coworking sessions in advance, and break down the paper into smaller segments that feel less daunting…

___________________________________________________

Notably absent when finding a penguin in your bathtub is any sense of shame. You wouldn’t beat yourself up – you didn’t do anything wrong!

Similarly, failing to meet your goals isn’t a moral failing. It means you haven’t figured out how to make good enough plans yet, and you need to slap on your detective hat to figure out how you can do better. How did we get here? What needs to happen next so this problem doesn’t happen again? If it does happen again, you know you haven’t solved the mystery. So, you try again. Because you want to solve the mystery.

The usual response I get here is that feeling bad makes sense because you’re responsible. You’re not responsible for the penguin being in the bathtub. One person said “I would feel differently if I’d carefully made a nest in the bathtub the night before and gently tucked a penguin egg inside.”

I hear you. I get that you feel responsible. But it’s not useful to sit there feeling bad about it. It’s useful to figure out how to do differently next time. Sometimes feeling a bit bad can help motivate you to figure out how to do better, but it seems more common that feeling bad just makes you not want to think about it. Totally counterproductive.

But also, “I’m responsible and should feel bad” assumes you just ... could have done it. Like that was an option you had at your finger tips. That’s not usually the case. The post Not yet gods describes my response so well that I’ll just direct you there. “Acting as you wish doesn't happen for free, it only happens after tweaking the environment and training your brain.”

In the meantime, don’t beat yourself up. Don’t shrug. Just figure out why there is a forking penguin in your bathtub!  

All of my other paintings were conceived with a final piece in mind. A sunrise over the ocean, a towering tree, the delicate petals of a sunflower... I find myself learning as I go along, but the knowledge is a byproduct of painting something beautiful.

I think this reveals something interesting about what practice looks like.

Most of the reward in painting – at least for me -- is in creating beauty. Encouraging a picture to take shape on my canvas is fun. Painting rocks over and over? Not so much.

But does only painting entire pieces make me the best painter I could be?

Perhaps.

Despite my love of deliberate practice, the ideal of practicing established exercises under the guidance of an expert teacher is impractical for most skills. Without clear direction on which skills to practice, I’m not sure we should aim for mastering one piece of a skill through repetitive.

It might work better to practice while producing output you're genuinely interested in. The key, however, is to still practice, not just “do the task”!

Practicing, not just doing

I learned to paint clouds and waves by following YouTube tutorials to paint a few seascapes, slowly but surely absorbing the techniques needed to portray the capricious nature of water and sky. Both the paintings below were made with a reference photo but no tutorial. You can judge for yourself how the left painting (before the tutorials) compares with the right painting (after a few tutorials). (Both were painted from just a reference photo.)

Following the tutorials was still very much practicing, not doing something I already knew how to do. The paintings took longer, I needed to learn new techniques, I used the comparison of my painting and the instructor’s as rapid feedback, and I was entirely focused on making each stroke look more like hers. For comparison, once I’d learned the techniques, I could make a painting in half the time while feeling more relaxed.

Is one method better?

It’s certainly easier to spend more time on the bottleneck skill if you don’t also have to paint the rest of the picture for each practice attempt.

However, I’m not sure that practicing in context is less effective, controlling for the actual amount of time spent practicing. I tried comparing my progress painting rocks vs clouds. (While I dedicated a few hours to practicing rocks, clouds were only given the spotlight as part of a complete painting.)

While I saw more noticeable improvement in painting rocks in one practice session, I’m estimating that my overall time painting rocks and clouds along with a tutorial are similar or a couple hours more on clouds. However, I subjectively feel that my improvement in clouds is more than on rocks, so this example doesn’t indicate much difference in the two types of practice.

However, it’s easier to identify your bottlenecks when practicing a skill in the context of a real task. It prevents you from inadvertently learning the wrong skills. Since you can see immediately if the practice is helping, you’re less likely to take away the wrong lessons from a toy environment. The immediate feedback from your canvas allows you to correct your course. If you’re not sure what success looks like, then practicing in the real environment is crucial.

For example, when I practiced outlining posts to limit post length and speed up writing posts, I then drafted posts from the outlines. I wouldn’t have realized how poorly I stuck to outlines if I hadn't put them to use in drafting my posts.

I think I was underestimating the benefit of seeing whether your practice improved your work. However, there’s probably an even more important consideration.

Empirically, I practice more in context. At least for me, practicing a particular technique demands more willpower and energy. Yet, it was relatively easy to practice in the middle of a painting if part of the painting looks bad. I want a beautiful painting – it annoys me if it’s not turning out the way I envisioned!

I don’t know if I’d learn faster practicing in isolation, but I’m do know I’m more likely to practice at all in context. In the end, that’s probably enough of a reason to focus on practicing while producing output.

When 'Practice' Morphs into 'Doing'

However, I’m worried that people will think they’re practicing in context when, in fact, they're merely performing the task as they've always done it or they’re not receiving feedback on how they’re doing. This process really only works if you’re focusing on trying a new method and getting feedback about how it’s going. 

Honestly, unless you deliberately pay attention to learning better techniques, you might never even realize there are better ways of doing the task. When I started blogging a couple years ago, I assumed that posting once a month was a good rate. I once wrote and posted a short post in the same day, but I thought it was a lucky fluke. Afterall, most of my blog posts took weeks of research and rewriting. It didn't even cross my mind that I could dissect the process and learn how to regularly churn out daily posts.

That blindness to opportunities to improve is my primary concern with practicing in context. It’s easy to think of the task as “the set of steps you already know and have done before to complete this task”. That’s fine sometimes. Sometimes you just want to get the job done.

But it’s not going to make you better, at least not quickly. You need new techniques and you need good feedback loops for that.

If you want to set those up, I’m happy to help! You can schedule a free 30-minute call to talk about it here.

What exactly is deliberate practice?

Ericsson’s recipe for practice starts with what he calls “purposeful practice”:

• Purposeful practice takes place outside of your comfort zone, pushing what you can already do. You should be trying new techniques, not just repeating what you’ve done before. Think “try differently”, not “try harder”!

• Purposeful practice demands you actively think about what you’re doing -- you shouldn't be able to daydream about dinner while doing it!

• Purposeful practice involves well-defined, specific goals broken down for step-by-step improvement (NOT vaguely “trying to improve”). You don't want to "practice the piano piece" you want to "practice the tricky section with the left hand until you can play it three times through at the correct speed without mistakes."

• Purposeful practice involves quick feedback and changing what you’re doing in response. Ideally, immediate feedback so that you can improve your approach mid practice session.

Ericsson adds one more criteria to graduate from “purposeful practice” to “deliberate practice”: well-developed knowledge of what and how to practice. Deliberate practice is when you’re purposefully practicing optimized strategies for improving the skill. Ideally, you want a highly developed field where experts have identified the most effective techniques and the best training strategies to develop those skills, plus a teacher who can lead you through the process.

Lacking that, do your best to find proven techniques and hope for the best. I ask more experienced people how they developed their skills or what they recommend I practice, and use that as a starting point. (Tips for informational interviews to learn how more experienced people developed skills.)

My initial goal

My goal was to write faster. I didn’t have an instructor, but I did have a benchmark: several journalists and bloggers had shared that they could write a post each day. One blogger who I respect advised me to try publishing a post each day for a month. So I set the more modest goal of writing one post each day for a week.

My first…and second…and third attempts

Day 1: I began by enthusiastically plunking out a short post around a great career planning tip I’d recently learned. I got the full thing drafted, but it seemed a bit forlorn. Surely it would be better if I went back and wrote a longer post that also included the other career planning tips I found most useful?

Day 2: Sticking to my intention to draft a new post each day, I set aside my career tools idea. Instead, I started drafting what became my CBT post. I’d stitched together most of the main post by time evening rolled around, but I wanted to go through the resources I’d been compiling to make a nice resource list.

Day 3: I whipped together a little post on an intuition I had about AI. However, when I spoke with my partner in the evening (who works in the field), he agreed that a solid example would improve the post. It too went on the stack of posts awaiting revising.

Day 4: I tried putting together a short post on ADHD…and only got as far as an outline. The more I tried to nail down what I wanted to say, the more I realized there was to cover. In the end, I set it aside to await a round of interviews. (It eventually grew into nine thousand words across three posts.) 

Day 5: A migraine killed all attempts at writing.

Apart from the migraine, I ended the week feeling good. I hadn’t quite met my goal, but I had four exciting posts in the works. So I decided to repeat the challenge…right after I spent two months revising the posts I’d drafted the first time around.

My second attempt followed a similar path: I drafted a couple shorter posts, then got lost on the gargantuan mess of trying to untangle the science behind skill plateaus. Ditto my third attempt.

Applying deliberate practice

As a way to get myself diving into new posts, the week-long drafting challenges were great.

However, as a way to learn how to write a post in a day, they were terrible! I didn’t have the skill-building-blocks yet to be able to draft a good post each day. I should have been practicing smaller sub skills, like outlining posts or studying how to write narratives.

The deliberate practice principles would have served me better. I was pushing myself and diligently paying attention. However, “write a post each day” is too vague and high level; it lacks both specific steps to practice and quick feedback loops.

I also spent most of the time on writing posts (in the way I already knew how to do), rather than on a specific subskill that I needed to practice. So effectively I was only spending a tiny percentage of that time outside my comfort zone.

The fact that I set essentially the same goal three times should have clued me in that it wasn’t working. Nothing in my approach changed between attempts - I was attempting to “try harder” rather than “try differently”! A good deliberate practice goal should be composed of small enough incremental steps that I make visible progress each time or change something significant between attempts.

My revised plan

If I actually wanted to learn how to draft a post in a day, I needed a new approach. Specifically, I needed to:

1.      Break the bottleneck down into specific steps that practice important subskills.

2.      Plan a quick feedback loop so that I can adjust my plan in real time based on how it’s going.

Lacking a writing teacher, I used the previous challenges as data to identify bottlenecks that make my writing process slower. What had gone wrong before?

Well, the biggest problem was that my posts inevitably grew into behemoths as I added more and more information. I don’t expect myself to write six thousand words each day. So I needed a better way to plan short posts. 

Second, I often wanted to do more research, write examples illustrating the point, or workshop the post with other people. I could probably learn to write examples more quickly. Research is probably just slow – I should kick those posts until I have more time.

Third, editing my drafts usually takes me at least as long as the initial draft. Heck, completely restructuring the post is common. No way I’m going to be writing finished posts in a day until I change that.

I brainstormed a long list of steps that might help me improve on the above (see Appendix A for my scribblings).

Ultimately, I decided to practice outlining and planning posts before I drafted them. This seemed like a good exercise to help me write more clearly from the beginning (and hence needing less revising) and know in advance how long a post would be.

I planned to outline a post each day and then try drafting it according to the outline. Because I’m creating the outline and then immediately drafting it, I could get feedback about whether the outline was working that day, then update my approach the next day if needed.  

How did it go?

Day 1: I outlined a post on a new concept, but felt really tired when trying to draft it. So I had ChatGPT produce a crude draft that I could revise later. Like 25% successful, but the concept was too vague in my mind. There was too much heavy lifting clarifying something new for this to be a good short post. I should look for topics that feel easy and I can envision the whole outline easily.

Day 2: I whipped out the Tattoos post -- short, contained, I just followed the outline without going down the potential rabbit holes of discussing “noticing confusion” or going really deep into the science. Success!

Day 3: I drafted the first version of this deliberate practice post. I rewrote the deliberate practice principles section to be more concise, but mostly ended the day with a post matching my outline (including the unfinished “How did it go?” section awaiting the rest of the data). 75% successful. 

Day 4: I knocked out a draft of a deliberate practice-tangential post, but wasn’t totally satisfied with the structure. I think I got the main points in the outline, but the structure and examples could be clearer. I should workshop the examples in my outlines more. Still managed to keep it short!

Day 5: Migraine, again. Ugh. I guess things came full circle…

Betrayals of my addled brain aside, this was a great success. While I didn’t cut down on editing (all three other than the Tattoo post required revision), I did make progress on keeping my drafts short. None grew into a behemoth, and it only took two weeks to revise them (instead of two months). I think I’ll be better equipped to plan what size of post I’m tackling going forward.

More importantly, the deliberate practice principles feel like an easy framework to apply to future growth goals. Breaking down my bottlenecks to a tiny goal to practice for a short period, and then moving on to the next one, feels way more promising than repeatedly slamming myself against the whole problem. As Ericsson would say, "Try differently, not harder!"

Appendix A

• Drafting shorter posts – takes me about 4 hours to write a short post, which requires that I’ve narrowed down a small idea I can cover in <3 pages including examples, and that I already know mostly what I’m going to say. If I need to do research, that takes longer (though is sometimes more valuable). Kelsey’s posts are mostly short.

  • Predict how long a post will be before I draft it.

  • Create an outline with the main points I want to hit, try to actually draft that (my posts usually end up wandering away from my outlines). 

• Whittling down my ideas and discarding large chunks that can’t fit within a post.

  • Anything that feels like a separate thread, put in an optional section to be picked up later. (haha)

• Revise quickly

  • When I realize that I’ve lots the thread in a post, I’ll start rewriting a paragraph and go from there (taken from Duncan)

  • Revision pyramid – big content/structural changes, then flow of ideas and paragraphs covering the right things, then copy editing pass (Newport)

• Writing more clearly from the start – I less certain about which steps would fix this 

  • Summarizing a passage you admire and then reproducing it as closely as possible to the original, observing the differences you make after you are done. (Taken from Ben Franklin’s autobiography.)

  • Duncan recommended practicing expressing an idea in one sentence, one paragraph, and one page/blog post – to clarify the idea in your head and practice expressing it with different levels of details. 

  • Review my copy editor’s comments for common errors I make that need to be corrected later.

  • Record myself writing and review the footage for clues.

• Get better at writing examples quickly

  • Practice/study elements of story telling

  • Write some fiction

  • Summarize and replicate other people’s examples that I thought were well done, so that I deeply understand how they did them.

  • Have a database of examples to pull from – in obsidian.

• Hiring a copy editor

• Use ChatGPT more

So, how does tattoo ink remain in the skin, practically unchanged, for decades? If the cells containing ink are dividing and dying, shouldn't the ink become diluted across cells and eventually cleared away with dead cells?

I noticed I was confused here. Noticing confusion is great for building better, more accurate models of the world, so I went down a rabbit hole to figure it out. (This concludes the tenuous link to productivity – the rest of this post is pure nerd sniping!)

Was my model of how cells divide accurate?

We know the answer thanks, at least in part, to bombs. Apparently Cold War nuclear tests temporarily altered the ratio of carbon isotopes being fixed into new cells. In a rare case of beneficial serendipity, the radiation gave researchers the chance to measure when specific cells were added to the body. They could even identify the rate at which slow-changing cells, like skeletal and heart cells, are replaced.

According to this fascinating bio-numbers website, it appears that most cells in the body are replaced in under a year. Fat, heart, nervous system, skeleton, eye lens, and (in women) egg cells are the only ones that replace themselves less frequently than a year (or not at all, as far as we can currently measure).

Therefore, skin cells should divide and dead cells be cleared out in under a year, not the decades that tattoos would suggest. It doesn't seem that the ink getting into deep tissue skin cells would change that, since nearly all the cells in the body are regularly replaced.

Still confused, I googled how tattoos work. Ignoring the citation-less pages insisting ink lasts because it gets trapped in a deep skin layer, basically all of the other results cite a 2018 study where scientists tattooed mice tails.

Studying the green tattoos, the scientists identified one type of cell that “eats” the ink: dermal macrophages. Macrophages are a type of white blood cell that engulfs and digests foreign particles as an immune response, but they apparently can’t break the ink down. Thus, each macrophage releases the ink when it dies, only for another macrophage to recapture the ink. Successive generations of these immune cells just hold onto the tattoo ink for years.

Even newer studies suggest that other cells also contain tattoo ink. The main second type are fibroblasts, which can become phagocytic when inflammation occurs – in other words, they are probably following the same process as the macrophages. I’m still confused by traces of ink in skin cells. They seem to be a minority (though I didn’t find good numbers on this), so perhaps the answer is simply that a few stray traces of ink get passed down through successive skin cells despite the majority being eliminated with dead cells. 

Anyway, the answer to my confusion is that tattoos don’t rely on skin cells to survive. Rather, your immune system is just throwing the ink in a perpetually revolving prison.

This is a slight exaggeration – you don’t want a doctor in their first year out of medical school.[1] After that, it’s less clear. One review found thirty-two studies indicating that the older a doctor was, the worse their medical outcomes; that review only found one study indicating that all outcomes got better with increasing age.[2] Other analyses suggest that middle-aged doctors might do better than younger doctors (though the effect is not statistically significant)[3], but older doctors are still clearly worse than middle-aged doctors.[4]

It’s not like doctors become terrible with more experience, but they are measurably worse. In one study, an extra twenty years of experience translated to about one additional elderly patient dying out of every hundred treated.[5] Why would twenty extra years of practice make a doctor worse at helping people?

Similarly, some research on famous painters, writers[6], and composers[7] found that it’s most common for these artists to produce their best work before the age of 45. A famous composer is more likely to have written their best piece of music in their 20s than their 40s, and they’re almost twice as likely to have written their best piece in their 30s instead of their 50s.[8]

Again, creatives are doing worse after a couple of extra decades of experience. This phenomenon is called skill plateaus. The idea is that performance stops improving after a relatively short time. After that, putting in thousands more hours on the job won’t reliably make you much better.

What’s going on here?

Deliberate practice                      

Anders Ericsson thought he had the answer. Ericsson was a psychologist famous for his research on expertise and human performance. He claims that most people practice a new skill until they can perform it adequately for their purposes.

For example, someone might practice tennis until they’re good enough to play with their friends. At that point, they stop practicing to just enjoy the casual games. Many people assume that they’ll continue to slowly get better just by playing games. However, that doesn’t seem to be the case. If they just play games using the skills they already know, they stagnate or even get worse over time.[9]

Ericsson sought to study what separated the truly outstanding top performers from those who were merely good. He concluded that the key factor was a particular approach to improvement, which he called “deliberate practice”.

Deliberate practice is not just any focused practice.

Deliberate practice is breaking your work down into individual steps, finding the optimal technique for each step, and practicing each technique until you execute it automatically. It requires clear, rapid feedback so that you know what you’re doing right and what you need to work on (this feedback usually comes from a coach).

Say our amateur tennis player wanted to continue improving. They might get a tennis lesson. The coach would identify what is holding the player back -- perhaps they hold the racket poorly. Here the coach breaks tennis down into individual subskills and identifies which are making their performance worse.

So, the coach demonstrates a better way to grip the racket and marks their racket at the correct place to grip. Here the coach gives the player a better technique and a training method to practice it.

The player practices gripping the racket correctly, takes one swing, and then stops to check their grip. Here the player repeatedly practices the skill so that they learn it in their muscle memory. They get feedback by checking whether their hands align with the coach’s marks on the racket after each swing. They might get more feedback if the coach corrects their grip as they practice.

After repeating the drill a few dozen times, they have mastered the better grip and are ready to move on to the next improvement.

These repeated small loops are good for improving your skill effectively, but don’t try them during a game! During the practice loops, you practice one tiny skill over and over until it’s automatic. Then during a real game, you put all of the tiny skill improvements together to play better. Experts have often practiced hundreds or thousands of these individual subskills to build up their expertise.

In Ericsson’s mind, ceasing to deliberately practice causes the performance plateau. People stop learning better techniques. Their performance slowly worsens over time as their autopilot skills atrophy.

Does his theory hold up?

Does deliberate practice explain which people are successful?

There’s a good amount of evidence that deliberate practice can work well – and a lot of caveats.

There are plenty of studies supporting deliberate practice. However, I focused on the studies trying to disprove deliberate practice to see what the counter arguments were.

The result was kind of funny. The authors of one such paper wrote a New York Times piece called “Sorry, Strivers: Talent Matters” attempting to debunk deliberate practice by highlighting their finding that working memory determined 7% of success in their study. Meanwhile, their study also found that deliberate practice determined 45% of success.[10]

Another study found that deliberate practice “only” determined one third of success[11], and a meta-analysis found that deliberate practice only predicted 11%.[12] Even the naysayers agree that deliberate practice is important; they just don’t believe that it entirely explains success.

The skeptics could have made a stronger argument by pointing at the limited domains in which deliberate practice is strongest.

In the above studies, the ones finding that deliberate practice accounts for 45% and one third of variance in performance were only looking at highly developed fields like music and chess. Highly developed fields have objective methods of evaluating performance (such as music competitions or chess ratings) and effective training methods, usually developed over decades or centuries. Meanwhile, the meta-analysis (which found that deliberate practice explained 11% of the variance in performance) looked at multiple fields.

In highly developed fields like music and chess, this meta-analysis also found that deliberate practice was important, predicting more than 20% of success. Yet, it found that deliberate practice explained only 1% of the variance for professional performances such as computer coding or selling insurance.

This feels a bit like the growth mindset debate. “Researcher becomes famous by discovering the secret of learning. Critics later claim that the effect is bogus. Fierce back and forth ensues.” Likely the result will be similar here: “Yes, the effect is probably real, just smaller than everyone initially assumed when reading glowing news articles.”

But why this discrepancy?

Why would deliberate practice work so well when learning chess, and so poorly in professional work?

Remember that deliberate practice is about finding better techniques.

If you have to invent these good techniques from scratch, this is really hard. If other people have put thousands of hours into identifying the best techniques for doing a task, and the best training methods for those techniques, then it is easy.

In one of the early deliberate practice studies, the study subject started out able to remember a string of seven numbers. Once he developed a system for grouping numbers, he could remember up to 18. But then he struggled to continue learning.

He had to develop a meta-system for remembering groups of numbers before he could break that plateau. It took him more than 200 hours of practice to develop the techniques to be able to remember nearly 80 digits.[13] That’s a lot of time to develop techniques from scratch for a relatively simple task.

Ericsson claims that running, swimming, piano, and countless other fields have far higher standards today than they did a hundred years ago – because we’ve learned better techniques and training methods.[14] It took Newton to invent calculus, but since then we’ve refined the field and discovered better teaching methods. Now it's a skill that can be learned by the average high school student.

Meanwhile, the meta-analysis lumped computer programming, military aircraft piloting, soccer refereeing, and insurance selling under “professional performances.” None of these fields have refined their optimal techniques or training methods as much as chess or piano has.

It seems possible that doctors and creatives stopped practicing because they couldn’t find new techniques to practice. Maybe we occasionally get a good, easy-to-use technique that lots of doctors adopt, like using checklists that reduce patient mortality by 23%.

But maybe most of the time, doctors aren’t sure what sub-skills would reduce mortality or improve their bedside manner. Furthermore, taking several hours each week to practice -- instead of producing immediate output -- is hard. So maybe the doctors don’t practice anything if they’re not certain what would make them better.

It’s not clear which elements of deliberate practice are essential for skill improvement. Doctors could find some bottlenecks, look up techniques, and practice it themselves. How would the results compare to deliberate practice? I’m not sure yet.

However, even in the contexts like professional sports and music -- where people are highly incentivized to deliberately practice and coached on well-refined techniques – deliberate practice explains less than half of the variance in success. That should be our upper bound on how useful deliberate practice is.

What else explains why doctors don’t get better?

I think the people who say  “Talent matters!” are pointing at a big chunk of the answer.[15] Fortunately, we can build a more detailed model of what it means to say someone is “talented”.

Psychiatrist Scott Alexander looked at med school data and found that medical students’ scores plateaued after about their third year. Students got steadily better in their first year. In year two, they learned a bit less but were still improving. But after year three, they barely increased their test scores. “Compare fourth-year and fifth-year surgeons, and it’s pretty close to 50-50 which of them will know more surgery.”

Scott says students in his program were still spending just as much time attempting to learn in their fourth and fifth years. They were far below the maximum score, and some students were doing better than other students. So why would their learning level off?

Because human minds can’t remember the entirety of medical knowledge.

A cognitive constraint is a limitation to how much we can learn that is imposed by the architecture and processing capabilities of the human brain.

One deliberate practice study found that working memory explained 7% of success at sight reading music. Working memory capacity is one cognitive constraint at the point of learning. Other constraints impact the ability to remember information over time or process new information efficiently.

These cognitive constraints differ from person to person, and can explain what we mean when we talk about “talent”. If someone is less limited by the constraints that bottleneck other people, they improve more quickly and seem naturally talented.

For example, people forget things over time unless they review the information at some gradually expanding intervals. (This is the widely accepted idea behind spaced repetition learning techniques.) Scott suggests that in order to remember a fact, people need to review that fact within different time intervals: some might need to review it after a week, others after a few months. This means that some people build up larger memory trees than others.[16] 

Scott writes: “Suppose that you forget any fact you haven’t reviewed in X amount of time (X might be shorter or longer depending on your intelligence/memory/talent). And suppose that an average doctor sees 5 diseases ~weekly, another 5 diseases ~monthly, and another 5 diseases ~yearly. A bad doctor might forget anything she sees less than once a week, a mediocre doctor might forget anything she sees less than once a month, and a great doctor might forget anything she sees less than once a year. So, the bad doctor will end up knowing about 5 diseases, the mediocre doctor 10, and the great doctor 15.”

I’m guessing these constraints explain a lot of the rest of skill plateaus. Individuals have a certain range of ability to improve. For example, doctors have some limit to how many medical facts they can memorize. If medical students are already hitting their limit after three years, no wonder doctors don’t keep improving for decades.

Forget the old myth that humans only use 10% of their brains. If doctors are constantly operating at their max cognitive capacity, they’re already at 100%. If this is the case, we should expect any small cognitive decline as they age to translate directly into decreased performance. If the doctor was operating at 100% of their ability and then that ability goes down by 1%, we should expect them to only operate at 99% of their former capacity.

Can we tease apart whether cognitive constraints or lack of deliberate practice contributes more to skill plateaus?

To attempt to disentangle which factors are contributing to skill plateaus, we can check if a skill known for deliberate practice follows the same skill plateau pattern as doctors and creatives. Doctors and creatives possibly get better until their thirties or early forties, then they get worse.[17]

If deliberate practice can solve skill plateaus, we should expect to see chess players continue to improve their skills later in life. They have more skills to keep practicing, so they can continue to improve over time.

However, if cognitive constraints are the main factor, we would expect to see chess players following the same pattern of decline as doctors and creatives.

Do chess players also get steadily worse after their mid 30s?

Mostly yes, but there’s a twist.

Studies on chess players found that their performance declined after the age of 35 or 40.[18] Younger players got sharply better, then their performance tapered off as they got older. Additionally, if you look at the top chess players, almost all are in their twenties and thirties.[19] So, chess players fit a similar pattern of decline to doctors and creatives.

However, chess players also demonstrated a cohort shift.

Over a hundred years, players got better. Subsequent generations made a higher percentage of optimal moves than the previous generation. This pattern has persisted across four generations.

Additionally, younger players experienced faster improvement earlier in life compared to previous generations.

My guess is that there are two things going on here.

First, across a field, practitioners on average decline after approximately the age of 40. Doctors, creatives, and chess players all seem to get worse at their craft after about that age.

This could be consistent with a story where “just doing the job” actually improves performance for the first few decades; and with a different story where they do deliberate practice but run out of obvious techniques after 20 years.

However, I’m guessing this pattern is better explained by cognitive decline. We found similar results in each field regardless of whether the field tends to do deliberate practice, which makes me feel that this pattern is less likely explainable by deliberate practice.

Similarly, if raw talent just improves skills for a while, that doesn’t explain why that pattern would reverse and people start getting worse. The theory of deliberate practice tried to explain this by saying that skills decay without deliberate practice, but the “just doing the job” case doesn’t have that excuse.

If people are already hitting their cognitive constraints given the current techniques, then performance will go down with even slight cognitive decline. If that decline is fairly stable across age, then we’d expect to see a pattern like this. (Note: all of these examples indicated fairly gradual performance declines, so we probably don’t need to panic until at least our sixties.)

Second, within a field, practice probably explains part of success, though it might depend a lot on the field. Deliberate practice probably doesn’t explain why people start getting worse in their 40s, but it might explain which already-talented people become the best.

The original deliberate practice research focused on what separated the very best from the merely decent within a highly specialized field. If you compare two highly skilled practitioners within a highly developed field, it seems plausible that deliberate practice determines up to half of which is better.

The finding that the learning curve for chess was shifting earlier and earlier introduces another possibility. Perhaps regardless of where the max performance tops out, deliberate practice speeds up the learning curve. A chess player or doctor will learn better skills more quickly, so they have more time to use their skills.

What do we actually do with this?

I set out to investigate whether deliberate practice or cognitive constraints could better explain skill plateaus. Now, I’m guessing the answer is mostly cognitive constraints, at least for the original definition of skill plateaus. 

However, if I ask “which is more important to focus on?”, then deliberate practice matters more. We can’t yet prevent cognitive decline, but we can do something about deliberate practice.

When done well, deliberate practice – i.e. better techniques and teaching methods – makes a big difference in continued improvement. For the medical students, some practice did matter. Students’ scores definitely went up for the first few years, and I doubt Frank Abagnale[20] learned as much pretending to be a doctor as med students do in school.

Furthermore, cognitive constraints and deliberate practice aren’t totally independent.

My guess is that cognitive limitations definitely matter, but they are more like a multiplier for the techniques you’re using. Without deliberate practice, everyone just does whatever random technique they stumbled upon when they first learned to do the thing. If the common techniques are cognitively demanding, then cognitive constraints will stop most people from becoming really good at the skill.

However, if you give them less cognitively demanding frameworks or tools, they will keep getting better. So we should be able to mitigate cognitive constraints with better techniques or better training methods.

So, as long as the natural ability to memorize symptoms determines doctor quality, then doctors will be limited by their cognitive constraints to a certain range of skill. If we start having doctors do spaced repetition or give them an AI tool that reminds the doctor of unusual options, that limit changes.

This would imply that we can raise the bar for entire fields, but we need specific skills to practice.

Maybe individuals should put more effort into deliberate practice. More importantly, societies need people researching and developing better techniques and training methods so that individuals can do deliberate practice.[21]

Footnotes:

[1] Association of Hospitalist Years of Experience With Mortality in the Hospitalized Medicare Population

Observed hospital mortality slightly improved after the first year (3.33% for patients cared for by first-year hospitalists vs 2.96% for second-year hospitalists), but didn’t change between the second year and subsequent years of experience.

[2] Systematic Review: The Relationship between Clinical Experience and Quality of Health Care

A systematic review of 62 studies found: “32 of the 62 (52%) evaluations reported decreasing performance with increasing years in practice for all outcomes assessed; 13 (21%) reported decreasing performance with increasing experience for some outcomes but no association for others; 2 (3%) reported that performance initially increased with increasing experience, peaked, and then decreased (concave relationship); 13 (21%) reported no association; 1 (2%) reported increasing performance with increasing years in practice for some outcomes but no association for others; and 1 (2%) reported increasing performance with increasing years in practice for all outcomes.”

[3] Association between surgeon age and postoperative complications/mortality: a systematic review and meta-analysis of cohort studies

In this meta-analysis of ten studies, older surgeons (above 50 or 60, depending on the study) had worse mortality outcomes than middle-aged (some subset of 40-60). It looks like young surgeons had slightly worse mortality than middle-aged (but not statistically significant).

The mortality in patients undergoing surgery by young surgeons was 1.02 compared to those by middle-aged surgeons. The mortality in patients undergoing surgery by old-aged surgeons was 1.14 compared to those by middle-aged surgeons.

[4] Physician age and outcomes in elderly patients in hospital in the US: observational study

This one is a single study rather than a meta-analysis or systematic review but has a more easily understandable finding: patient mortality gets slightly but steadily worse for each decade of physicians’ age. Patients’ adjusted 30-day mortality rates were 10.8% for physicians aged <40, 11.1% for physicians aged 40-49, 11.3% for physicians aged 50-59, and 12.1% for physicians aged ≥60.

[5] Physician age and outcomes in elderly patients in hospital in the US: observational study

This one is a single study rather than a meta-analysis or systematic review but has a more easily understandable finding: patient mortality gets slightly but steadily worse for each decade of physicians’ age. Patients’ adjusted 30-day mortality rates were 10.8% for physicians aged <40, 11.1% for physicians aged 40-49, 11.3% for physicians aged 50-59, and 12.1% for physicians aged ≥60.

[6] When Did Nobel Prize Laureates in Literature Make Their Best Work?

An analysis of 189 modern art painters’ highest-priced works found that the painters were on average aged 41.9 when they created their best works. Meanwhile, 89 Nobel Prize laureates in literature wrote their most important work at the average age of 44.7. The distribution of ages when they completed their best piece of literature looks roughly normal.

[7] When Did Classic Composers Make Their Best Work?

An analysis of the most popular works by the 100 most popular classic composers found that the average age of peak creativity was around 39. The distribution of ages when they completed their best piece looks roughly normal.

[8] When Did Classic Composers Make Their Best Work?

Link to table with histogram.

[9] Ericsson, Anders; Pool, Robert. Peak : Secrets from the New Science of Expertise (p. 10-13).

[10] Deliberate Practice Is Necessary but Not Sufficient to Explain Individual Differences in Piano Sight-Reading Skill: The Role of Working Memory Capacity

“It has even been suggested that deliberate practice is sufficient to account for expert performance. Less clear is whether basic abilities, such as working memory capacity (WMC), add to the prediction of expert performance, above and beyond deliberate practice.”

“For piano players, deliberate practice accounted for nearly half the variance (45.1%) in sight-reading performance. However, WMC accounted for a significant proportion of the variance (7.4%), above and beyond deliberate practice.”

[11] Deliberate practice: Is that all it takes to become an expert?

Deliberate practice accounted for about one-third of the reliable variance in performance in music and chess, leaving most of the variance explainable by other factors.

[12] Deliberate Practice and Performance in Music, Games, Sports, Education, and Professions: A Meta-Analysis

“More than 20 years ago, researchers proposed that individual differences in performance in such domains as music, sports, and games largely reflect individual differences in amount of deliberate practice, which was defined as engagement in structured activities created specifically to improve performance in a domain. This view is a frequent topic of popular science writing—but is it supported by empirical evidence? To answer this question, we conducted a meta-analysis covering all major domains in which deliberate practice has been investigated. We found that deliberate practice explained 26% of the variance in performance for games, 21% for music, 18% for sports, 4% for education, and less than 1% for professions. We conclude that deliberate practice is important, but not as important as has been argued.”

[13] Acquisition of a Memory Skill

“After more than 230 hours of practice in the laboratory, a subject was able to increase his memory span from 7 to 79 digits.”

[14] Ericsson, Anders; Pool, Robert. Peak (p. 6-8). HarperCollins. Kindle Edition.

[15] Another theory I saw was that people aren’t actually getting worse as they age, they just get busier and less motivated.

This could explain why creatives are more likely to produce their great works earlier. But doctors? Maybe I’m naive, but I don’t think doctors would get apathetic and start letting patients die if it was purely a matter of motivation. But if their cognitive capacity declines a bit? Sure, that makes sense.

[16] Skills Plateau Because Of Decay And Interference

I was originally inspired to look into skill plateaus because I wanted to understand how Scott’s memory constraints explanation and the deliberate practice explanation popularized in Cal Newport’s book “So Good They Can’t Ignore You” fit together. I ended up wanting to focus on the bigger picture of cognitive constraints and decline, but here’s the summary of Scott’s posts that I made along the way. It’s a good example of two cognitive constraints.

Scott Alexander writes a popular blog on medicine, neuroscience, rationality, and [insert topic of your choice]. He also daylights as a psychiatrist.

Scott proposes two cognitive constraints that together (he thinks) answer the dilemma of skill plateaus.

To understand these, it will be helpful to think of knowledge as trees of complex facts/concepts branching off from personal experience and things you learned. If I understand Scott’s idea correctly, the number of relevant branches you can remember determines how good you are at something. A doctor who can look at a list of ten symptoms and tell you only the most common possible diagnoses is a worse doctor than the one who can look at the list, tell you the most common diagnoses, and then also tell you all the uncommon ones if it turns out that the common ones are wrong.

Scott first proposes the Decay Hypothesis, which says that people forget things over time unless the idea is reviewed at some gradually expanding intervals. This is the widely accepted idea behind spaced repetition learning techniques . He suggests that people have different intervals over which they can review the fact, such that some people build up larger memory trees than others.

Secondly, Scott proposes the Interference Hypothesis, which says that memories that are too similar to each other are easier to “collapse” – e.g., it starts getting hard for our hypothetical doctor to recall which diseases this particular set of symptoms suggests, because a bunch of diseases have very similar sets of symptoms. Things that stand out in memory more vividly are easier to remember because they don’t get smushed into all the other similar memories.

Scott speculates that these constraints may explain overall skill plateaus, but he acknowledges that his evidence is mostly for more narrow learning, such as a constraint on how much you can learn per day (e.g. twenty words of Spanish vocabulary a day).

My guess is that Scott is pointing at two important ideas, but I’m not convinced they explain all or even a majority of the skill plateau that is due to cognitive constraints. I speculate that they are two among a large number of cognitive constraints.

For example, working memory seems like a cognitive constraint at the point of learning. That’s completely different from the interference or decay hypotheses, which impact the ability to remember information over time.

[17] The studies mentioned in the introduction indicate that the creatives probably got better for a bit before they got worse - e.g. composers were more likely to produce their most popular piece in their 30s rather than their 20s, then it sharply declines. The studies on doctors were ambiguous as to whether doctors in their 40s were better or worse than younger doctors, but clear that middle-aged doctors were better than older doctors.

[18] Life cycle patterns of cognitive performance over the long run

“This study presents evidence for the dynamics of life cycle patterns of cognitive performance over the past 125 y based on an analysis of data from professional chess tournaments. Individual move-by-move performance in more than 24,000 games is evaluated relative to an objective benchmark that is based on the respective optimal move suggested by a chess engine. This provides a precise and comparable measurement of individual performance for the same individual at different ages over long periods of time, exploiting the advantage of a strictly comparable task and a comparison with an identical performance benchmark. Repeated observations for the same individuals allow disentangling age patterns from idiosyncratic variation and analyzing how age patterns change over time and across birth cohorts. The findings document a hump-shaped performance profile over the life cycle and a long-run shift in the profile toward younger ages that is associated with cohort effects rather than period effects. This shift can be rationalized by greater experience, which is potentially a consequence of changes in education and training facilities related to digitization.”

[19] Ages of top 10 on the FIDE list in classical chess on May 25, 2023: 19, 27, 28, 29, 30, 30, 32, 32, 35, 53. Only two are not in their twenties or thirties: only one was above 35.

[20] An infamous conman who claims to have illegally practiced medicine as an untrained teenager.

[21] If you've made it through all the footnotes, I feel like you deserve the caveats that I'm told make rationalist writing clunky and terrible. This post is my current best guess for how to interpret a bunch of data points. As with all grand theories explaining complex concepts, there's a decent chance I'll change my mind with new information.

• Skills peak and start declining somewhere between 30 and 50. – 75% confidence. This seems to be true for doctors and creatives. Furthermore, the pattern held when I checked if chess players also peaked around those ages, leading me to think the trend is likely to generalize.
• Deliberate practice explains between 5% and 50% of the variance in success between people in well-developed fields. – 50% confidence. This range is my best guess from the deliberate practice studies, but I wouldn’t be surprised to find new evidence that shifts this range.
• Additional high-quality research would make me conclude that Ericsson’s specific deliberate practice process is much better than other forms of practice. – 30% confidence. It’s not currently clear to me how necessary Ericsson’s exact process is to effective practice. It’s possible that other practice strategies, like learning by addressing bottlenecks as they arise in everyday work, might also lead to significant improvement. My current guess is that Ericsson’s process is good when possible, but that you can change many elements and still get meaningful skill improvement. Further experimentation is needed here.
• Cognitive constraints determine a big chunk of the variance in success between people – 90% confidence. Though I don’t have good estimates on how much, since ‘cognitive constraints’ is such a wide category. I would be surprised if it was less than half.
• Deliberate practice shifts the learning curve earlier, making people become better faster. – 50% confidence. This seems intuitively likely, but I only have one study supporting the idea.
• Better techniques improve fields – 90% confidence. This seems intuitively and almost tautologically true. Consistent with this idea, the fields of sports, music, and chess all demonstrated improvements over decades. Things like The Toyota Way in car manufacturing seem to indicate that fields like business can have similar improvements via better techniques, so I expect this to apply to all fields, at least in theory.
• Deliberate practice improves fields above and beyond better techniques – 60% confidence. It seems true that deliberate practice would help people learn about and master the better techniques, but I’m not sure if it’s always necessary. I want to do more research on this.

Thanks to Amber Ace for editing. 

Honestly, it’s complicated.

ADHD isn’t a disease, like a coronavirus or malaria. We don’t know of a clear underlying medical cause for the symptoms of ADHD. No doctor can look at a brain scan or blood test and say “Yes, you have ADHD!”

We know that it’s partly inheritable and that dopamine and norepinephrine deficits seem to be involved. ADHD stimulants increase these chemicals, which impact executive function, motivation, and reward perception – basically everything you think of as “self-control.” (See an easy introduction video to dopamine here.)

Instead, ADHD is the label given to a cluster of symptoms around difficulty managing attention, impulsivity, and hyperactivity. A scale like the Adult ASRS Scale is often used to measure these symptoms.

I find this frustrating. “When you have a task that requires a lot of thought, how often do you avoid or delay getting started?” is so unhelpfully vague that I want to scream.

Should I say “often” if I answer emails for an hour before getting started on work once a week? What if I procrastinate on starting work for a couple of hours on the majority of days? Or should it only be if I procrastinate for days, and get nothing done in the meantime? These three scenarios have wildly different consequences for my ability to function, but the questions don’t give you any baseline to compare yourself against![1]

One person told me that they’d looked at an ADHD quiz several years before getting diagnosed officially:

“I did the 6-question assessor and was like ‘trouble paying attention? Nah, not more than anyone else!’ I typical-mind-fallacied my way right out of a diagnosis.”[2]

This wasn’t an isolated experience; over half of the people I interviewed expressed uncertainty about their diagnosis. Did they “really” have ADHD? Which symptoms were due to ADHD, and which to just being unmotivated or their personality?

I would love a scale that could tell me that “50% of people procrastinate getting started for X-minutes a day on average, whereas the best 10% only procrastinate for Y-minutes and the worst 10% procrastinate for Z-minutes.” You could check your day against the scale and have a good idea where you fall.

That scale doesn’t exist.

As a result, ADHD diagnoses are arbitrary.

Everyone experiences some degree of ADHD symptoms. If your experiences pass some arbitrary threshold of severity according to the particular doctor you’re talking to, then you get diagnosed with ADHD.

According to psychiatrist Scott Alexander:

“Ability to concentrate” is a normally distributed trait, like IQ. We draw a line at some point on the far left of the bell curve and tell the people on the far side that they’ve “got” “the disease” of “ADHD”.

The prevalence of diagnosed ADHD is about 10%, so doctors are essentially cordoning off the worst 10% of the population on this bell curve.[3] 

So to all the people commenting on my ADHD post that “Everyone experiences those symptoms. It doesn’t mean I have ADHD!” …You’re sort of right?

We should expect most people to occasionally experience symptoms of ADHD. If we could magically line up everyone based on how severely ADHD symptoms impacted their life, we’d see the burden from symptoms gradually increasing as we went down the line.

At one end of the line, we’d see people who were barely impacted by ADHD symptoms at all. At some point, we’d start seeing people who are doing fine but could accomplish more if they had stimulants. Further down the line, we’d find people who are struggling every day but can still hold down a job. Eventually we’d reach people who’ve lost multiple jobs or relationships.

Where on this line do we start giving people stimulants? 

If different brains make different amounts of dopamine (or whatever the magic chemical is), then it’s not surprising that lots of people who make enough to function normally still sometimes benefit from a bit extra. I don’t know if 20% or 50% or 70% of people would benefit, but I would strongly guess that it’s more than 10%.

Unfortunately, the medical establishment treats stimulants as a medicine to treat a specific disease – a medicine which they strictly gatekeep. The medical establishment says that finding ADHD medication helpful doesn’t confirm that you have ADHD (and thus should be allowed stimulants), since most people benefit from ADHD medication whether or not they meet the criteria for diagnosis.[4]

Instead, doctors want to track the presence of symptoms from childhood to the present day, and check that you say “often” or “very often” sufficiently frequently to questions like “How often do you have difficulty keeping your attention when you are doing boring or repetitive work?”[5]

But now we’re back to arbitrary, hackable questions gatekeeping a drug that’s helpful to a potentially large fraction of the population. People who are genuinely trying to figure out if they “really” have ADHD aren’t able to do so from these questions.

People who’ve given up on that can just say “often” or “very often” to all the questions, and get meds.

Seriously.

Psychiatrist Scott Alexander wrote:

“If by some chance a psychiatrist doesn’t give a patient Adderall, that patient practically always goes to another psychiatrist, and that next psychiatrist does. Trust me, no matter how unsuitable a candidate you are, no matter how bad a liar you are, somewhere there is a psychiatrist who will give you Adderall. And by “somewhere”, I mean it will take you three tries, tops.”

Makes one feel rather cynical about the world, doesn’t it?

Personally, I care if people are accessing medication that helps them function better. I wish we didn't have any of the gatekeeping or labels or screening tests except insofar as we used it to predict who would benefit from meds.

My ideal solution would be for everyone to try stimulants, see if they notice a difference, and then take them if it makes their life better (and for all of this to be completely legal).

Of course, we’d inform people about the possible side effects like irritability, insomnia, fast heart rate, and high blood pressure – then we’d let them decide for themselves if the benefit was worth the side effects.

I’m also in a weird social bubble of unusually high conscientiousness and openness-to-experience[6] people where almost everyone will read at least Scott’s post about the risks of Adderall before trying drugs, and almost no one would even consider taking 25 pills at once. This post is probably redundant for a quarter of that social bubble who have already investigated stimulants, but probably another quarter should check whether stimulants help them.

I know my vision is too optimistic (probably because of that weird social bubble), and we need some gatekeeping to avoid abuse. And obviously, doctors and society don’t quite agree. They have extra hoops you’ll need to jump through.

But if you’re a sensible person just trying to optimize your productivity, and you haven’t tried stimulants, maybe check you aren’t typical-mind-fallacying yourself out of a helpful tool?

Footnotes:

[1] Tip, the first answer probably doesn’t indicate ADHD. The second one is ambiguous but might, and the last answer sure sounds like ADHD.

[2] The typical mind fallacy is the mistake of assuming other people's experiences are similar to your own.

[3] This analysis and this analysis of US federal data found that 11% and 9.8% respectively of kids had been diagnosed with ADHD by a doctor. UpToDate, a well-trusted database of medical reviews for doctors, estimates that between 9 and 15 percent of school-aged children have ADHD. Meanwhile, this meta-analysis found that 6.7% of adults have ADHD, and UpToDate estimates 4.4%. 

[4] It seems to be a talking point on the internet that “ADHD meds don’t help if you don’t have ADHD,” but the limited studies we have on stimulant’s’ effects on non-ADHD populations don’t support this claim. This review focuses on the risks of misuse, but if you read the section “Effects of stimulants on cognition in individuals without ADHD,” all of the studies show neutral or positive effects, including on memorization tasks, working memory, cognitive control, and even creativity, though often with a smaller effect than for more impaired people.

[5] These doctors say it’s important that you had symptoms as a child, since current diagnostic criteria mostly say that you can’t develop ADHD as an adult. I’m confused by this, since there seems to be growing research that many people have ADHD as adults who didn’t have symptoms as kids. It’s possible that this will change in the future, similarly to how diagnostic criteria used to say you could not have both ADHD and autism, then changed when new research showed this was incorrect.

[6] Conscientiousness and openness to experience are two of the Big Five personality traits, and I expect being high on both to correlate with being cautious but willing to try potentially helpful medication.

Is this a reasonable worry?

To find out, let’s look at the psychology literature on pressure and creativity.

There’s a classic psychology experiment called the “candle problem”. Participants are shown matches, a box of thumbtacks, and a candle as in the picture below. The experimenter then instructs the participants to mount the candle on the wall using the available materials. "The problem is considered solved when the candle can be firmly affixed to the wall, burn properly, and does not drip wax on the table or floor."

If you’ve heard of this problem before, you probably know the answer. However, if this is new to you, take thirty seconds to try solving it before reading on.

.

.

.

.

.

Done?

If it’s still difficult, just imagine the thumbtacks on the table next to the box as in the picture below. Once you do that, suddenly it’s easy to guess that you should tack the empty box to the wall and put the candle inside it.

When the box is filled with tacks, our brain writes it off as a “tack holder”, instead of seeing it as a possible “candle holder.” This is called “functional fixedness.” It takes creativity to see an object being used in one way, and then break that default association to work out what other uses it could be put to.

This experiment is designed to study creativity, particularly the ability to find unusual or “out of the box” solutions to a problem. The experimenter can easily randomize whether the participants see the box full of tacks or empty (i.e. hard or easy creative thinking), plus add whatever other interactions they want to test.

Which brings us to the point of this whole experiment—how long does it take the participants to find the solution under different conditions?

In one study, the experimenters used a simple 2x2 design: participants were randomized so that half saw a picture of tacks in a box (which requires more creativity), and the rest saw tacks on the table and an empty box (which makes the puzzle easier). Half were told they’d receive a $20 bonus if they were the fastest in the group ($5 if they were in the top 25%); the other half heard no mention of a bonus.

Now, adjusted for inflation since 1962, $20 is almost $175 dollars. So, participants had a strong motive to complete the task faster when offered that bonus.

Did they?

Only when they saw the empty box. When the picture showed an empty box, participants solved the problem about a minute faster if they were offered a reward (taking on average 3.67 minutes, compared to 4.99 minutes for the non-rewarded group). If instead the participant saw the box full of tacks and was offered a reward, they took over three minutes longer than those who saw a full box but never heard about a reward (11.08 vs 7.41 minutes respectively).

What’s happening here?

The basic theory is that when you add pressure, people get better at tasks they already know how to do, but worse at doing novel tasks. This finding has been repeated in studies that use financial rewards, performance evaluation, and even self-evaluation as the source of pressure.

Think of it as tunnel vision. When you are particularly focused on one problem or motivated to get it done quickly, you get better at doing what you already know exactly how to do. But you get worse at looking around for novel solutions, because you get stuck thinking about the problem in one narrow way.

Are these findings sound?

I didn’t find any red flags: googling one of the papers plus the keywords “myth” or “replication” didn’t turn up anything damning, and the Wikipedia page on functional fixedness didn’t highlight reasons to doubt the theory. It’s a neatly-packaged theory but it hasn’t been headline news—so I’m not applying the additional skepticism I add for surprisingly interesting findings.

That said, there are some yellow flags: it’s mostly an older body of literature, which might mean worse methods. For example, the study I described included only male psychology undergrads. However, there is a 2009 study that came to the same conclusion.

Possibly we should be skeptical of all psychology findings, given that some attempts to replicate studies can reproduce less than half of the original findings. Even when studies replicate, the effect size often changes wildly. One study found that effect sizes decreased by 50% on average when replicated.

I don’t think we need to toss out these results completely, though. I wouldn’t be surprised if the effect size decreased, but I’d rate this finding as slightly more likely to replicate than the average psych study. 

Are these findings meaningful?

Even psych studies that replicate don’t always matter - the real world impact is so tiny that it’s not worth implementing. In this case, I revisited these studies because a few clients were worried that setting deadlines would make them less creative. Is this true?

My short answer? “Probably yes, but it’s still often worth setting the deadlines.”

If we can take the candle problem study at face value (which is a big assumption), then adding pressure made participants 50% slower at finding the answer. That’s pretty important, if it generalizes.

However, there are two big caveats that make me inclined to recommend deadlines anyway.

First, participants found the solution faster under pressure when it was easier to figure out what to do. Do you have a decent idea of what you’re supposed to do, but are struggling to focus on it and get it done quickly? Then adding pressure should speed you up, according to this study.

Second, while for the harder task participants in the high-pressure condition did take longer than those in the low-pressure condition, the participants in both conditions were under enough pressure to be actively working the whole time. In contrast, what if you’re a PhD student struggling to sit down to write your thesis? Or a knowledge worker struggling to make time for important-but-not-urgent work instead of answering more emails?

In that case, maybe adding pressure makes you go slower than if you were doing the task without pressure, but probably not slower than you go if you’re not spending time on the task at all.

For most tasks that you’re struggling to put enough time towards, I expect the benefits of deadlines to outweigh even working at half speed occasionally.

There’s a tradeoff between carving out time to slowly explore and setting up incentives to quickly get things done.

If you’re already working long hours, feel free to take a long walk to puzzle over a problem without pressure. Famous mathematician Richard Hamming set aside every Friday afternoon after lunch to think "great thoughts." This was his time to ask, “How will computers change science?”, “How can I change that path?”, and other big questions. Reserving that time probably helped him create the mental space needed to be unusually creative.

So if you’re constantly trying to cram eighteen tasks into fifteen minutes and wondering why you aren’t having any new ideas, then feel free to take some pressure off. Make time to be bored.

However, if you’re struggling to put enough time towards your tasks, don’t let concerns about your creativity become an excuse to avoid any pressure to increase what you accomplish.

Getting an ADHD diagnosis is sufficiently complicated that there’s a joke, "You know which people don't have ADHD, because no one with ADHD could make it through the process to get diagnosed." 

Hopefully this quick overview of my diagnosis process can make finding a doctor and getting diagnosed more approachable. 

Note, my diagnosis process was in the UK. The process is similar in the US, but I understand that system less well.  

How to find a psychiatrist 

Step one is finding a doctor to diagnose you. 

If you need to go through a provider covered by your insurance or (if you’re in the UK) the NHS, you probably want to start by asking your primary care physician for a referral or contacting your insurance provider for a list of covered psychiatrists. It will probably be a huge hassle. I asked my GP for a referral – only to be informed that there is a four-year waitlist to get an appointment. 

In the US, you can also try asking your primary care provider if they can diagnose you and prescribe meds. I think it’s common for them to say no, but it might be the simplest option if they do agree. 

If you can afford to drop one to two thousand quid, you can go the private route and get diagnosed in a few weeks to a few months. 

UK

I used The ADHD Centre, an online private clinic. The process took three weeks from scheduling to an initial diagnosis call, then about a month to get medical tests and a follow up appointment to prescribe medication. So altogether about two months, and cost about £1,500 (£795 for the initial appointment and £225 for a few monthly follow-up appointments until I was stable on meds that worked for me.) As a second data point, a friend also had a good experience with them. 

Another friend had a good experience with Psychiatry-UK. They signed up for an account, looked over a list of doctors with availability in the next few days, and scheduled an appointment in a couple days for £360 per session. You might be able to have the NHS cover this or other ADHD diagnosis services through Right to Choose. 

Canada

One person recommended the online service Frida. 

USA

I’ve tried to list some US options, but they’re not great. Sorry! The services available vary state-to-state in the US, so you’re going to have to check if any of these are available where you are. In addition, telehealth availability will probably change in the next few weeks as the extended telehealth permissions from Covid are rolled back. If you send me updated information or additional services, I’ll try to update this list. 

I asked a few people for their recommendations, and got:

• Bay Area Adult ADHD worked pretty well for one person. I think they’re an assessment clinic that can assess you and recommend a treatment plan to your doctor, but not able to directly prescribe medication. 

• One person worked with Boston Neurobehavioral Associates exclusively through Telehealth. 

• Online services like Done are relatively accessible, but are under scrutiny as the telehealth rules change. 

• One recommendation on the EA Mental Health Navigator was for Talkiatry, an online “group psychiatry practice that operates in multiple states, accepts most major insurances, and offers primarily medication management.”  

The EA Mental Health Navigator provider database has more recommendations for providers who can prescribe medication (you can filter by specialty for ADHD). 

If none of these work for you, you can try asking for recommendations from friends and family members who have received treatment for ADHD. Based on the examples I found so far, I’m guessing that you probably want to look for online practices that advertise ADHD diagnoses, so you’re less likely to run into a doctor who will refuse to prescribe medication. 

Do your own research 

This step is optional, but I’d recommend you at least read Scott’s post on ADHD medication (linked below), since the doctor may or may not give you enough information to make sensible choices for yourself. (My doctor basically told me “Here are four medications. Two are short acting, two are long acting. Which do you want?”) 

Psychiatrist Scott Alexander has an excellent overview here with the medication he recommends. He also covers most of the other questions I hear -- possible side effects, tolerance, addiction, etc. 

If you want more sources, UpToDate.com (a medical overview website for doctors) covers the treatment of ADHD in adults, including citing relevant studies. For a different approach, StuffThatWorks aggregates thousands of patient reviews to see which treatments work best. 

tl;dr: The other two sources mostly seem to support Scott’s claim that Adderall is slightly preferable to Ritalin, and that both usually work well for people diagnosed with ADHD. 

You can also check your own symptoms against the questionnaire the doctor will use. My doctor used the Adult ASRS Scale, which I think it’s the most common one. Scoring “often” or “very often” on 6 or more of the questions is suggestive of ADHD. (Specifically, I think my doctor was checking that I scored “often” or “very often” on 6 or more in childhood and 5 or more in adulthood.) The doctor will look at more factors, especially whether the symptoms are negatively impacting your life, but this gives you a good baseline. 

The diagnosis process with the psychiatrist 

Before the diagnostic appointment, the doctor sent me three questionnaires. I filled in one, and the others were filled in by who knows me now and someone who knew me as a kid. I think these were all versions of the Adult ASRS Scale. These checked whether I’d had symptoms spanning from childhood into adulthood that impacted my life now. 

During the diagnostic appointment, my doctor used the following criteria based on the DSM-V: 

1) Having 6 or more than 6/9 symptoms of attention deficit and/or hyperactivity/impulsivity in childhood and 5 or 5/9 in adulthood. 

The doctor administered the Adult ASRS Scale separately for me as an adult and me in childhood. The scale has 18 questions addressing different aspects of inattentive and hyperactive presentations. Essentially, scoring “often” or “very often” for more than 6 of these is the threshold for an ADHD diagnosis, though that’s not sufficient. It’s possible other doctors would more heavily weigh the six questions in part A. 

2) Having a lifelong pattern of symptoms. 

The doctor checked that I’d had symptoms starting in childhood and continuing to adulthood, even if they were more minor and/or never diagnosed as a child. 

3) Symptoms / impairment are expressed in at least two domains of functioning in childhood (before the age of 12)

Basically, did ADHD symptoms negatively impact at least two life areas such as schoolwork, social relationships, etc.? Examples from childhood included things like making spelling mistakes, getting distracted from homework, failing to finish tasks, or talking at inappropriate times.  

4) Symptoms / impairment are expressed in at least two domains of functioning in adulthood

ADHD symptoms need to negatively impact at least two “domains of functioning” (like work, romantic relationships, friendships, money management, or well-being) to be diagnosed. Examples from adulthood include things like spending money impulsively, having difficulties in your relationships because you forget things, getting fired or losing a job because of poor performance. 

5) The symptoms cannot be better explained by the presence of any other psychiatric disorder.

The doctor screened for autism and asked some questions about mental health to see if e.g. depression better explains the symptoms. 

6) Clear interference / reduction in level of functioning and well-being in more than one domain

Is the negative impact from ADHD symptoms in at least one area severe enough that it’s making life worse? I think this last area is pretty important. If ADHD symptoms aren’t negatively impacting you, the doctor understandably is less likely to give you medication. 

Examples might be if: you’re anxious about work because you can’t reliably meet deadlines, your work performance is worse than it should be, you’ve gotten bad feedback about being late, you’ve lost a job, you have a hard time managing money because you spend impulsively, you have problems in a romantic relationship because you zone out or forget things. 

Medical screening 

The doctor also asked some questions about my health (smoking, drinking, drugs, heart problems, family medical history, meds I take) that screen for any potential risks for stimulant medication. I’ve heard elsewhere that taking illegal drugs can be a dealbreaker. 

Medical checkup

My doctor had me take a few medical tests before starting medication (ECG, a few blood tests, a doctor listening to my heart with a stethoscope). I think this is standard if you have a heart condition or other medical issue that might interact with stimulants. 

It took me a few weeks of trying to get my GP to schedule an appointment before I gave up and just scheduled one myself at a private clinic. (I used SameDayDoctor in London, which was extremely convenient.) 

Titration 

Finally, the doctor prescribed medication. 

I had to meet with the doctor monthly until we figured out which medication and dose was best for me. I tracked a bunch of metrics over the titration period (especially the first month) to measure how the meds worked. You can see those experiments here. 

After that, you just get the medication regularly refilled by your doctor. 

In the UK, after you’re stable on a medication, you can have the ADHD doctor hand over prescribing responsibility to your GP by sending them a shared care agreement. My GP refused (which I’m told is unusual), so I switched to Babylon GP at Hand, an online GP that accepts shared care agreements in London.

I’m a productivity coach who frequently works with people who have ADHD. I’d studied diagnostic questionnaires and read about the official symptoms of ADHD.  

I’ve also been struggling with those symptoms for decades. I was unable to consistently focus on demand, felt constantly tired, and often needed to force myself to get started on my projects. Yet I never seriously considered that I might have ADHD. 

Why not? 

Because my experiences didn’t match the picture of ADHD in my head. I thought I understood the symptoms of ADHD, but I didn’t really. (How much trouble focusing counts as “difficulty focusing” anyway?) 

I missed out on the benefits of stimulant medication for well over a decade because I didn’t understand what ADHD actually looks like. I’m guessing there’s a good number of adults (maybe including you!) who are also missing out on that benefit because of similar misconceptions. 

So I collected stories from six people who were diagnosed with ADHD as adults (including myself). These people are intelligent, high-performing individuals who nevertheless struggled for years with undiagnosed ADHD. 

I’ve structured the first half of this post around my interviewees’ responses to a typical ADHD questionnaire, so you can see what kinds of experiences you might expect if you have ADHD. The second half covers their stories prediagnosis and with medication. If these responses feel familiar to your experience, I encourage you to consider whether you might benefit from ADHD medication. 

So, what does adult ADHD actually look like? 

Hint: Adult ADHD is dominated by what I’m calling the Terrible Trifecta: trouble getting started, keeping focused, and finishing up projects. Every one of my interviewees struggled with these three areas, while most said that the other symptoms had less of a detrimental impact on them. 

Note, I mainly focus on the ‘inattentive’ presentation of ADHD, since it’s easiest to miss. If you’d like to learn more about the other types of ADHD, I recommend the youtube channel How to ADHD. 

When you have a task that requires a lot of thought, how often do you avoid or delay getting started?

Every person I spoke with thought this was a major problem. They described procrastinating, trying to get started but being unable to focus, or spirals of feeling motivated and wanting to make progress but then…just not doing the task. 

This isn’t just for tasks that people dislike. People would bounce off even tasks that they enjoyed once they got into them.

For me, this is especially likely when a project is distant in my mind. Resuming drafting a post after a weekend feels aversive. I can’t remember exactly what I was planning or why I wanted to write the post. If I don’t sit down and mentally “boot up” the project, it’s tempting to start something easier instead, which means that I don’t tackle the planned task until hours later. 

For some respondents, procrastination was particularly likely when they were trying to meet really high standards. If they didn’t feel that they were able to do a great job right then, they would instead go do something else until a deadline forced their hand. (I definitely resonated with this!) 

It felt like I would have such a high standard for each sentence that it felt like any word choice was totally wrong. I would try and make myself start, and then it wouldn't feel on fire enough to actually start doing it. Then I very easily get sucked into some other internet distraction. This would keep going until it was like midnight or 1 AM and then it would be like, “Okay, If I'm going to get any sleep tonight, I need to start this.” ~Richard 

Other people describe making a plan, but then just not feeling able to start doing it. It might take multiple attempts to do something very basic. 

I'm trying to read one page of a report. First of all, it's incredibly difficult to just get started on this task. Instead of doing the next step, even though I know what I want to do, I will read news. I’ll continue to follow the links…and links and links and links…and not do the thing I really wanted to do. 

It both feels like just following my curiosity and learning new things (that's exciting!), and it also feels like a part of me is slowly dying because I'm like, “Oh no, this is not what I'm supposed to do, but I can't stop”. I feel very frustrated with myself.  

Sometimes procrastination would drag on indefinitely, especially for tasks that didn’t have an external deadline. One interviewee described trying – and failing -- to work on a project for nine months. Her manager (understandably) assumed that she didn't think of the project as something she was working on, because she had, in fact, not worked on it…at all…ever.

How often do you have difficulty keeping your attention when you are doing boring or repetitive work?

Again, every single person described difficulty staying focused while working. 

Difficulty getting started would blend right into trouble staying focused while working. In the stories, I found it difficult to separate “getting started” and “staying focused,” because they generally overlapped. People would try to get started, make a plan or do a little bit of work, get distracted or want to do something else, try again… You can see the pattern here. 

I started today with one task. I know that basically I just need to open my document and read the notebook. But I just don't look at my notebook. Instead, I go on the internet…and then the Pomodoro ends. 

The next Pom, I open the notebook, but then I need to close all the windows that were open from the previous day. They are all very tempting and I sort of need them later on, so it's super difficult. Then I finally look at the notebook. I realize I can just copy down what’s in the notebook…and it’s the end of the Pom and time to take a break. 

So, I take a break, and then the whole cycle starts again. I could just write down literally what's in the notebook and put it in the document, but instead I check slack because something could have happened.  

Difficulty staying focused didn’t just apply to boring or repetitive work either. Work that required a lot of thought and concentration seemed to be the most common challenge, even if people enjoyed the task once they were able to focus on it. 

For me, difficulty focusing feels like my brain is mush. I’m trying to force my mushy brain to do something, but my working memory is shot and every minute is effortful. 

Other people describe similar experiences of feeling unable to force their attention, no matter how hard they try. 

“When it comes to, like, reading something I find boring, it's like my mind just won't Stay. On. The. Page.” 

Several people were easily distracted by things around them, such as noises or people. I find it easiest to work alone so I’m not distracted. Another interviewee only realized they were extremely distractible when they suddenly became more productive after they happened to move to a new desk with a privacy barrier. In contrast, other people with ADHD have told me that e.g. working in a café or library with other people around makes it easier for them to focus. I suspect that these people feel more accountable and stimulated with background noise or other people around. 

Most people also found it super tempting to just start doing something easier. 

I would just reflexively reach for my phone or go to some other tab. ~Richard 

I’ve seen arguments that ADHD is really just people reacting badly to addictive phones or superstimuli on the internet. I don’t entirely buy this: why would ADHD meds make our phones no longer superstimuli? However, there is something here. Some people with ADHD say it helps to be in an environment where there is no internet or other distractions. As I understand it, ADHD brains have low dopamine, so people with ADHD are unusually tempted by things that quickly release dopamine (like videogames or interesting videos). Being distractible AND having so many things that will quickly give you dopamine makes focusing harder. (Though removing distractions doesn’t always solve the problem – several of my interviewees mentioned that daydreaming or mental restlessness was their worst distraction.) 

[My experience of ADHD] would be starting the day ready, feeling pretty good about what I'm going to do, having a plan, and then being interrupted and distracted. Feeling more and more despair accumulate over time. And then finally by the end of the day, focusing for a little bit. After like 75% of the day being a bit crushed, finally at the end of the day, being able to really work for 25% and having a somewhat decent outcome.  But I feel shame that I've worked so inefficiently. 

Similar to starting work, my respondents said that open-ended, deadline-less tasks without accountability were the biggest challenge. 

• Compared to school, my work had fewer deadlines. This was a lot harder because I never got to the point of like, “Oh no, my big essay is due and I've got to write the whole essay tonight.” Instead I just kept being like, “Okay, I guess I'll make more progress next week.” ~Richard 

Many people with ADHD have found workarounds by setting up deadlines and accountability, such as telling their supervisor that they’ll get a particular doc to them by a certain date or coworking with people who will hold them accountable. 

I can’t overexaggerate how important deadlines and accountability were to people with ADHD. Almost all of my respondents told me (unprompted) some version of “I can’t do open-ended tasks until the deadline is close or there’s accountability.” 

How often do you have trouble wrapping up the final details of a project, once the challenging or interesting parts have been done?

All my interviewees said they struggled to finish the projects they started – if there wasn’t a clear deadline. Most of the people I spoke with found things like planning an event easy, because there was a clear deadline that other people counted on. They had to do the work. 

Some projects are easier for me. Some projects are just like, “Oh, this needs to get done” Like a hiring round or any event organizing. But then afterwards, there's some things you can do that just make it a bit nicer, like having an evaluation. Those tend to not happen. 

Any task without a deadline – for example, writing a post or an independent research project – might never get done. Everyone had stories of half-finished projects left on the shelf. 

I feel like I just can't [finish projects]. So I'm delegating. Even if I don't have a super high budget, I just need to hire people to do the citations and the refining, or I need a very strong deadline. Otherwise it doesn't get done. I have an EA forum post that I'm almost done with. I just need to ask one person to add to it and…it's been two months. 

And more stories…

I do the hardest parts, like writing several murder mysteries with plots and character profiles, but haven’t finished any to where someone could play it. It just doesn’t feel enjoyable to do – the part that felt enjoyable was figuring out how the characters and plots interact with each other. Once that’s been decided, actually having it be printable for people to play it doesn’t feel exciting. 

One contributing factor is that people with ADHD seem to often start new projects impulsively. They get excited about something new and start working on it, only to find that they’re much less excited about completing the project later. 

I'll get halfway through writing something. And then I'll just be like, “I don't know where to take this,” or another idea will inspire me. ~Anisha 

A couple of people commented that “finishing the final details” of a project is rarely a small task. I’ve heard before that having a full draft of something written is more like 50% done than 80%, because “just polishing it up” is actually so much work. 

I like to say that I've never finished a project but it's not quite true. But it’s kind of true for every project that didn't need to be finished.

For example, I have a post. It's not the first draft. It's not the second draft. It's like the third or some later draft, but I thought it wasn't ready to go out. I started writing it in 2020. I haven’t touched it in a long time. I should probably just post it at this point, right? 

But then I just think that rewriting a draft to a final polished post, especially when you still want to make structural changes, is just very challenging. Like, I'm not sure I want to call this finishing up the final details. 

I loved how Anisha put this struggle for the final push that you never quite manage. 

It feels like chasing butterflies sometimes. You don't end up catching them, but it feels like you get really close.

How often do you make careless mistakes when you have to work on a boring or difficult project?

The results were split fifty-fifty here. Half of the people said they thought this was a problem for them, half said it wasn’t. 

The people who said they struggle with “careless” mistakes mostly mentioned homework. I’ve also heard people say that they frequently need to resend emails because they had typos. You might struggle with this if you have gotten feedback that you’re making typos too often and need to work on it, but find it difficult to improve when you try. 

According to one interviewee: 

It's not like sometimes I will make careless mistakes. I just assume I will have made careless mistakes. It's just a thing. When I do math, I just assume that something is wrong, because something is usually wrong. 

My most embarrassing story was from some application thing. I meant to send out a mail merge to the people that we wanted to interview before we admitted them to the next stage. But instead, I sent them all an email saying they were accepted to the program. It was like only eight people or something, but I was so embarrassed. 

How often do you have difficulty concentrating on what people say to you, even when they are speaking to you directly?

All but one of the people I spoke with said they sometimes struggled here. However, for most of them, it was only with people they felt fairly comfortable with or things like lectures. 

I have enough attention control to not [space out] with people that I don’t feel comfortable doing that with. But I think it's more like I'm putting effort in. 

How often do you have difficulty getting things in order when you have to do a task that requires organization?

I think this question is poorly defined. Everyone said they struggled here, but they all gave different examples of what they struggled with! These included:

• Physical space organization (like “where did I put those notes?”)

• Being messy

• Regular planning (le.g. making a weekly plan)

• Realistic planning (these people can easily make plans, but the plans aren’t realistic)

• Breaking a task down (e.g. renewing a driver’s license)

• Scoping tasks (like being efficient with research and not disappearing into rabbit holes) 

A recent thing was that my old ID card expired nine months ago. In order to get a new ID, I need to make an appointment at the DMV and gather a couple of documents to prove where I live. This was a task that I started a couple times and then I stopped part way through, because I couldn’t figure out where to find some document. If I had really focused, I could have figured out like, “Oh, you can find this tax form and make it work.” Or the DMV is right by my parents’ house, and I’d want to go when I could also see them.  

It feels like everything had to be perfect, so I would wait another few weeks to make the appointment. It didn't have a deadline beyond the one that had already passed, so there was no urgent moment to do it. Just that little trivial inconvenience of gathering the right papers meant that when I started it, I didn't complete it. And then I would have to go back to square one when I restarted. ~Richard 

The most common challenge was being consistently too optimistic with deadlines.

Not plans like “I will meet you at this place at 4:00 p.m.” type of plans but like “I will hand you this report by Friday” - that sort of plans. The part of my brain that is making these promises is not actually doing the hard work of imagining buckling down and doing the work that it takes to do that. Part of my brain is taking the easy way out. So I always would schedule a fairly optimistic time frame. ~Richard 

People described making plans and fully intending to follow them, but also not being surprised that they didn’t follow the plan or that they didn’t meet the deadline they set. 

When I plan my own time, I say “I'm going to do this then” but I don't believe it in my mind. I'm already imagining I'm not going to do it. 

It’s possible that over-optimistic planning happens because people with ADHD have poor time perception.

For purposes of self-diagnosis, I think the point that I remember jumping out at me most was the extreme reliance on "the last minute", and how remember resonating I would be totally unable to actually start doing the important task until it really did feel dire. If you want the one thing that I most with past Richard, it's that. ~Richard 

Some research shows that people with ADHD are worse at estimating the passage of time. A couple of people said it feels like there is “now” and “not-now,” and planning regarding anything that’s not-now doesn’t happen. 

You either do [the thing] now, or it doesn’t happen. ~Anisha 

How often do you have problems remembering appointments or obligations?

Most of my interviewees said they didn’t struggle with this – at least, not anymore. People said that they used to struggle with forgetting things, or that they struggle if they forget to write stuff down, or that they are often a few minutes late to most things. 

If I forgot to write down a homework assignment, I might forget to do it entirely. ~Anisha 

However, by the time they’re adults, they’ve mostly developed coping strategies. One person described telling herself she needs to leave by an earlier time than she actually does, so that, when she inevitably leaves later than planned, she’ll still arrive mostly on time. 

Almost everyone mentioned using calendars and/or todo lists scrupulously. “Out of sight, out of mind” fit me to a tee. If it’s on my todo list in TickTick or if I have a physical reminder present, I’m almost guaranteed to remember it. If it’s out of sight, I won’t. I’m really, really good at writing down todos immediately so I don’t forget them. 

How often do you experience mood swings or difficulty dealing with stress? 

Mood swings aren’t one of the standard diagnostic criteria for ADHD, but they came up a lot when I was reading about it. Five out of six people said they often had mood swings. 

For me, this means going from excited to pessimistic about a project based on my mood, even if the project hasn’t changed. Another person said they had a “mood of the week.” 

From the inside, it doesn’t feel like I’ve had these mood swings forever and that’s just how I work. From the inside, it feels like, “Oh, I had this big thing that made me sad, but now I’m on the up. Oh, I have a new big thing that was exceptional, but now I’m on the up…” 

It took a while until I looked back at it to go, “Wait, that doesn't seem to be what's going on.” I think part of the story is that I immediately forget how I felt. 

At some point I had a really bad weekend, like I had a Saturday that was possibly one of the mentally worst days of my life. On Tuesday, somebody asked me how my weekend was. I was feeling good that day, so I said that I had a great weekend. Then I wanted to tell them more about what I did, so I thought about what actually happened last weekend. “Wait, I was miserable last weekend!” 

In other cases, the symptoms of ADHD frequently put people in situations where they are understandably anxious, depressed, or frustrated with themselves. If ADHDers continually feel like they’re not living up to their ideals, they might also qualify as depressed. (Both “feeling bad about yourself” and “trouble concentrating” are diagnostic questions for depression.) Or if they are continually stressed about meeting (or failing to meet) deadlines, they might get anxious about whether they’ll be able to do it this time. 

Half of the people I spoke with had experienced depression or anxiety that was probably caused by ADHD. This meta-analysis found that about 50% of people with ADHD also have anxiety, and 18% to 53% of people with ADHD have depression (and 9% to 16% of people with depression have ADHD!) 

I haven’t seen good advice on how to tell if you’re more likely anxious/depressed, ADHD, or anxious/depressed because you have ADHD. A therapist told me that you should treat both possibilities at once: “If someone is experiencing anxiety or depression (manifesting as sadness, guilt, irritability, low self-confidence, and helplessness) due to ADHD-related challenges, such as difficulty focusing, forgetfulness, disorganization, and restlessness, it's important to consider ADHD. However, it's also possible to have ADHD and depression unrelated to ADHD. Treatment plans should typically address all conditions simultaneously, rather than separately. If someone suffers from ADHD and an anxiety disorder, the treatment plan should be designed to target both issues at once.” 

My impression is that when doctors and therapists are presented with one diagnosis, they don’t always consider whether there’s a comorbid condition that better explains the situation. I would guess that if most of your symptoms are around poor concentration and feeling tired (rather than low mood), your difficulties concentrating continue even when your mood is better, or you feel bad about yourself or anxious primarily because of your work, you might want to ask directly about the possibility of ADHD.

Before you know you have ADHD

The above descriptions have one thing in common: they are from people who already have an ADHD diagnosis. Knowing you have ADHD is a lens that subtly changes how you perceive your own experiences. Before getting diagnosed with ADHD, the people I spoke with still had the same symptoms. But they often chalked them up to “just not trying hard enough” or some health issue. Sometimes, they didn’t even realize they were struggling more than other people. 

“Having ADHD without knowing you have it” is really different from “having ADHD and knowing you have it.” So, the story that is the most like me “having ADHD and not knowing,” is probably losing my job and being surprised at my incompetence in the office when I was assigned stuff I didn't find interesting.

Whereas, having ADHD and knowing that you have it, I guess just feels a lot more like, “Oh, I understand the types of things that I'm better at. I understand that if I want to do some of the things I'm not as good at naturally, I'm going to need some systems in place. I'll probably want to prepare those before I get started.” 

My respondents only realized they might have ADHD when someone else suggested it or they heard about a friend getting diagnosed and realized they’d had the same symptoms the friend was describing. Before they were diagnosed, they interpreted their experiences very differently. 

One person thought they had chronic fatigue, and that’s why they felt unmotivated and could sleep for fourteen hours a day. None of the things meant to help chronic fatigue (like iron and exercise) worked for them, but it’s common for treatment to fail to improve chronic fatigue symptoms. 

Others attributed their experiences to personality traits or personal failings. One person said they used to think they were just a night owl, which is why they wouldn’t get anything done until late at night. Another said she was always being told to focus more, and just thought she wasn’t trying hard enough.

Several people told me that before they were diagnosed, they felt like they were trying their hardest but just not getting quite as much done as other people. They usually assumed that others were just working harder. 

The hallmark experiences of undiagnosed ADHD seem to be saying “I just need to try harder” over and over for years, or kicking yourself for intending to start work and then not getting much done, or wondering “How does that person get so much done?” It feels like you should be able to do more, but you just can’t. Some people blame themselves. Others assume that something must be wrong with them, but they think it’s anxiety, depression, fatigue, or some other health problem. 

I previously attributed my troubles focusing to brain fog from POTS and headaches/migraines, and possibly depression. I also previously blamed personal failings but had mostly worked through that in therapy before I ever considered ADHD.

These were reasonable guesses. Brain fog is a symptom of POTS. Feeling unmotivated and tired are symptoms of depression. Almost everyone describes some difficulty focusing on hard work like writing. I knew I was worse than normal at focusing, but it’s hard to pin down how much worse when there are such huge variations between people already. And all of my experiences fit neatly with the theory that my health or emotional issues were just making it harder to focus than ideal. If I said I wanted to do something but couldn’t get myself to do it, maybe I didn’t care about it as much as I thought?  

The hard thing is that all of this is probably partially true. If I’m tired or don’t really care about some task, I’m going to have more difficulty focusing, ADHD or no ADHD. Working through emotional issues (e.g. perfectionism) did help me get more done.

But there’s another layer on top of all of those. My brain is deficient in the chemicals that make tasks rewarding. It’s impossible for me to feel as motivated as I “should” be relative to how much I care about the task. 

It’s not surprising that I didn’t notice I was less motivated than I should be. There’s not a good motivation meter we can check the readings on. 

And ADHD is a measure of degree; there’s not a clear line between having it and not having. Everyone struggles with focus sometimes. Everyone sometimes doesn’t want to get started on a hard task. So if you want to determine whether you have ADHD, you need to ask yourself not “Do I sometimes struggle to focus?” but “Are these symptoms negatively impacting my life?”  

Medication 

Missing that I have ADHD meant that I spent over a decade struggling when there is a simple solution. 70-90% of people with ADHD find stimulant medication highly effective.(1) 

When I actually get things done, I just feel like my heart is blossoming and it's like “Finally, I can be the person I want to be.” 

Of the people I spoke with, all six said stimulants helped them focus. Four said meds worked quite well for them. 

Of the other two, one person tried one medication, which helped her focus but had bad side effects (losing an unhealthy amount of weight and emotional blunting). She hasn’t tried another kind yet. This meta-analysis found that 10 percent of adults with ADHD discontinued stimulant medications due to side effects. It’s fairly common to need to try a couple medications before you find the one that works best for you, but it only takes few weeks to test each one.

The sixth person said that meds helped her focus, but that she still needed “to fight the rabbit hole” while on them. I think this experience isn’t unusual. ADHD stimulant medications help people focus, but they work best alongside learning prioritization and productivity strategies. 

Of the people who said that meds worked well, some take them every work day, while others find that they develop some tolerance if they take them every day. In those cases, it’s more common to take ADHD meds three or four days a week (often cycled with caffeine on the other days). I’ve heard that other people with milder symptoms only take meds when they need to focus for a big task, like writing an article. 

The research on stimulant medication for ADHD finds a large effect size (Cohen’s D of 0.73). Put simply, this means that people with ADHD who take medication report fewer, less intense symptoms on average, than those who don’t. It’s not easy to directly quantify reduction in ADHD symptoms, but for comparison, an intervention to increase IQ with a Cohen’s D of 0.73 would increase IQ by 11 points. 

Here are the stories of how meds helped the people I spoke with: 

Medications proved to me that I need them. Before I tried anything, well, I was still a little skeptical, like maybe I really am just not trying hard enough. But after trying medications and having them work, I know what it looks like to experience a day where I have normal amounts of energy and focus. I’m able to consistently jump from one task to the next, without always needing a break in between. 

I’m not getting sidetracked by things. Like, when I have meds, if another task comes up, it's just like, “Okay, write it down, go back to what I'm doing.” Because there's a sort of feeling of, I don't want to be pulled away from what I'm doing. ~Anisha 

It puts me in a mode where I feel more “let’s go!” It energizes me. The idea of checking things off my to-do list feels as addictive and “pulling me towards it” as random internet distractions usually feel. 

With a lot of tasks, if I can just get myself to the point of starting the task, I go, “Oh, actually, this is interesting.” But just doing that first five minutes would be a huge lift on a lot of days, and when I have taken Adderall, then it just would be a lot easier to force myself to get through the first five minutes and pick up some momentum.  ~Richard 

I can just feel energized to do something as soon as it's needed. I mean, obviously I have to decide to take them, but then I have energy for the next few hours. 

For me, medication has led to more than a 25% improvement in my general quality of life. I feel like I must be an outlier because the effect was so huge in so many areas of life. For one example, writing feels so much easier – the little voice that’s obsessing over each word is gone. I can just plan a post and write it out. Previously I had to be in the right frame of mind, or I would try writing and feel like I was pushing at mush in my brain. I felt scattered by possibilities at every choice - how to structure the post, what to cover, how to phrase each sentence. It was hard to narrow my focus to any one thing or feel satisfied when I did. Now I’m reliably able to imagine a structure and quickly hammer it out, without feeling like I’m forcing myself. 

This How to ADHD video beautifully captures my internal experience of taking meds vs not. 

If you’re worried about side effects or addiction or something else, I highly recommend psychiatrist Scott Alexander’s page on Adderall. I trust Scott’s analysis and feel pretty comfortable accepting risks he compared to “the risks of eating one extra strip of bacon per day.” I also checked the UpToDate page for risks of stimulants; their findings broadly seem to agree with Scott’s analysis. 

When I wondered aloud why we treat these stimulants so differently to caffeine, I was informed that just because I’m a prude who never considered taking 25 times the prescribed dose, doesn’t mean that there aren’t people that crazy. If you take 25 times the prescribed dose, ADHD meds are more dangerous than caffeine. Don’t do that, please. 

Conclusion

Medication shouldn’t stand alone. If you have ADHD, you’ll want to learn coping strategies and productivity habits, like coworking and setting external deadlines, to use alongside the meds. 

But those strategies aren’t a replacement for meds! Without stimulants, I will always be working harder to do the same task as someone with more dopamine. 

If you’re saying, “Yes, but how do I know if ADHD meds will help me?,” you could take an ADHD questionnaire, like this one. Or you could look back at the Terrible Trifecta:

1. Do you struggle to get started working, frequently procrastinating? 

2. When you’re trying to work, do you find yourself getting distracted or having a hard time producing output? 

3. Once you’ve made progress on a project without a deadline, do you have a hard time finishing it up? 

Have any of these symptoms made you lose a job or underperform in your job? Do you feel like you have to constantly work as hard as you can to do a “normal” amount of work? Do you get feedback that you don’t accomplish enough or don’t accomplish it fast enough? Do you feel unable to force yourself to do work, even if you think it’s important? 

None of the above are conclusive evidence you have ADHD. These experiences might also be explained by anxiety, depression, chronic health problems, poor prioritization, being inefficient, or having a bad manager. However, they are still good indicators of whether you might benefit from meds.  

Honestly though, there’s probably a simpler way to test for yourself. 

One therapist told me that they sometimes recommend a client take stimulants to see how they respond, such as trying a friend’s pill (note: this might be technically illegal, depending on where you live). I suspect that caffeine might also help with ADHD symptoms, as long as you don’t regularly consume it (e.g., no more than once a week). (Before my diagnosis, I had a super strong response to taking caffeine once a week.) 

If you have a strong response to either, there’s a good chance that ADHD stimulants will be helpful and it’s worth going through the process of getting a diagnosis. 

A weak response doesn’t rule out stimulants being helpful (you often need to try a few kinds or doses to find what works best for you), but it probably makes it less likely that you’ll have a huge positive response to ADHD meds. 

One of my respondents said: 

The first evening that I properly considered that I might have ADHD, I just started crying. Because I felt so relieved and I had just realized that I felt extremely guilty before. I think the thing that triggered this was me thinking that “Actually I'm trying really hard. It's not, in fact, the case that I'm just lazy and not putting any effort in.” 

In the end, it’s that simple. Are you struggling, and can medication help? If it can, I want you to have that help. 

A huge thank you to Richard, Anisha, and my other interviewees for sharing their stories. Thanks to Amber Ace for editing. 

Footnote:

(1) Papers on ADHD cite that stimulants work for 70% to 80% or 90%, often without bothering to give a citation. Ones that offer a citation link to studies like this one that found Methylphenidate reduced ADHD symptoms in 76% of patients, compared to 19% of the placebo group. Meanwhile, StuffThatWorks aggregates reviews from people with ADHD of which medication they’ve tried and which has been helpful, kind of like Yelp reviews for medical treatments. Based on more than 10,000 reviews each, the website finds that 91% and 86% of users found Adderall and Methylphenidate (Ritalin) helpful, respectively. This is consistent with psychiatrist Scott Alexander’s claim that most people slightly prefer Adderall.

Luisa Rodriguez’s Imposter Syndrome post hit me hard when I read it last year. 

I’d inadvertently adopted a mindset that you had to be one of the best people in the world to have an impact. Anything less would meet with failure and harsh judgment. 

Unsurprisingly, this was bad for both my mental health and my productivity. I’d go through periods of feeling quite pessimistic that my work was valuable or that I could do really challenging, high-impact work. Often I’d feel that my coaching was mediocre because I wasn’t clearly transforming all my clients’ lives, despite generally good reviews and a full client load. 

From the inside, perfectionism didn’t feel like striving for perfection or something silly like that.(1) Perfectionism felt like struggling (and failing) to meet perfectly reasonable standards that other people were somehow meeting. Then, of course, I felt bad about the failures and resolved to try harder….you can probably see the pattern here. It never felt obvious from the inside that my standards were unrealistic -- maybe my standards were a little bit high, but not unreasonable.(2)

For example, I sometimes found that I spent too long on projects because I was afraid of seeming incompetent. I’d hoped to get one of my side projects (the interview series) out the door in 2-3 calendar months, and it ended up taking nearly six. 

That extra time cost me. I could have created an entire second project. Instead, I spent a lot of time polishing some supplemental materials that almost no one read. 

I wanted to do a deeper career exploration last fall, including exploring some bigger ideas that I didn’t know if I could succeed at. I felt scared just thinking about the ideas, and terrified at the thought of telling others I was attempting them. What if I tried and failed? Or just decided it wasn’t worth it after all? Would they write me off as a failure or someone not worth working with in the future? 

Those fears were my wake up call. I couldn’t really explore career options if just thinking about them felt scary. So I scheduled some sessions with a cognitive behavioral therapy (CBT) therapist and started Overcoming Perfectionism by Roz Shafran (a CBT workbook). I found both of these helpful for relaxing some of my implicit beliefs that were sabotaging my work. 

I’m sharing some of my exercises from the process below, to hopefully make this more accessible for other EAs who might find a similar process helpful. 

All the usual caveats: I’m not a mental health professional. This was my experience, yours may vary. 

Perfectionism diagram 

In the first or second session, the therapist walked me through a perfectionism model on the call. We laid out a bunch of the implicit thoughts I was experiencing.

I got the impression my therapist wouldn’t have said I meet the criteria for clinical perfectionism (e.g. I already could offer counters to many of the implicit thoughts), but the framework was still helpful for connecting some of my implicit assumptions (“I’m being harshly judged with little room for forgiveness”) with the actions I’d identified as counterproductive (“I have to keep working until this unimportant post is flawless”). 

In particular, this exploration helped identify which “beliefs” felt true, even if I didn’t necessarily endorse them. 

CBT Log – exploring the issue 

My homework for several weeks was to fill in a CBT log each time I noticed myself feeling anxious, avoiding a task, or noticing relevant thoughts. This was super helpful for getting a more detailed picture of the situations and thoughts that triggered perfectionist behavior. It also helped me separate thoughts and feelings more reliably. (Perfectionist behavior for me includes anxiety, procrastination, and poor prioritization). 

I set a reminder in my daily todo list to fill in the CBT log, which helped me remember to regularly go back to it. 

A few example entries: 

Identifying specific beliefs/patterns from the log

After I’d kept the original log for a few weeks, I looked over it and identified themes that came up repeatedly in my thoughts. For me, common themes included: my peers were harshly evaluating me and my work, feeling pessimistic about my work meant it was bad/wasn’t going well, trying something and then stopping was failing, not currently doing the most important thing was failing, and not gritting though lots of effortful work meant I would fail. 

We discussed these themes in therapy sessions, plus I did some exercises on them (see behavioral experiments and pros/cons sections). 

I already rationally knew that some of them were silly, but I needed this exercise to remind me over and over until I started remembering on my own. (E.g. trying an experiment and deciding it wasn’t worth continuing was a learning success, not a failure. I just had to remind myself that this was an experiment with the goal of learning, and it’s still a success if I learn it’s not worth doing more of.) 

Some of the other worried I don’t think were completely unfounded, just exaggerated. For these, it helped to carefully evaluate the evidence for how severe and likely the risk actually was. (E.g. my peers do hold high standards for writing, but they’re probably not going to write me off after one bad post and a lot of writing success is hits-based. It’s fine if not every post is great.) 

I incorporated the reframes into my CBT log, and practiced evaluating each situation through those perspectives. I added notes on the reframes I was practicing to the “behavior” column to remind myself to check those. 

A few examples from this period in the log: 

Pros and cons of perfectionism

One exercise the therapist suggested early on was to create a “pros and cons of perfectionism” list. This felt a lot like a short internal double crux exercise, where I listed the best cases for and against the standard next to each other, and I could consider evidence for both perspectives. 

A few examples: 

Behavioral experiments

I did behavioral experiments based on the template outlined in Overcoming Perfectionism. These were basically me just doing the thing that felt anxious or prompted perfectionistic tendencies, and evaluating the evidence I got. 

I’m noticing that I’m feeling nervous about sharing the following example: “Ah, what if a future employer reads this, and thinks I’m not up for doing whatever job they otherwise thought I was a good fit for?” I want to add lots of caveats about, “But I can do more now with the ADHD medication!” Honestly though, I think it’s good to share this vulnerable experience of working through my feelings about something I wasn’t sure if I could do or not, in order to get to a place where I could accept that uncertainty and put myself out there. 

Example: Applying to things

1. Belief:

If I apply to things, I’ll be rejected, judged harshly, and waste other peoples’ time, while also feeling incredibly anxious, wasting my time, and risking suffering depressive episodes afterwards. If I am accepted, it will be because I somehow falsely convinced them I was more competent than I am, and I’ll ultimately fail, be judged even more harshly, feel worse about myself, and be in a more precarious work situation going forward.

2. Identify the prediction in general:

If I apply to X job, 99% chance I’ll be rejected. So, I am probably wasting a few hours of my time and 15 minutes of theirs by applying. (Though I do get a small amount of information from whether I get to the first round or not, plus a very small probability of being surprised by their evaluation if I make it through multiple rounds. So not completely wasted time.)

I did feel incredibly anxious applying (tears were involved), and days later still feel mildly anxious thinking about someone reading my application. (100%)

Maybe 10% chance it will make the people evaluating my application think less of me than their baseline evaluation from knowing my name in the community? No, I think that’s wrong. Like, what I ended up writing shouldn’t make people downgrade their evaluation, unless their evaluation was incredibly uncalibrated and somewhat optimistic. Eh, we’ll go with 5% that I’ll be judged, but no way to evaluate this directly in my case (most likely) and it’s a black box of uncertainty. A survey on how often people thought less of someone after seeing what they’ve accomplished might help. (Ask Rohin)

I ballpark a 70% chance I could support the level of work required to do a normal 9-5 job. There might be ways to work around this in the role, and I’m upfront with my medical issues going in. But I think there’s a legitimate chance I’d be stressed and unable to keep up with demands. This prediction has high uncertainty and would need to be tested by doing a regular job – but I’ll note that doing the three-day 80k work trial exhausted me (it’s possible but not definite that I could keep that pace up). 

I would need to stop doing the coaching, so I would be slightly worse off if I took the job and then needed to switch back. This is an overcomeable risk.

3. Specify the prediction precisely (specify behaviors and rate intensity of beliefs and emotions):

90% I don’t make it to the first round (of interviews/work trials).

99% I don’t get the job.

80% I feel “slightly” anxious about the application a week after submitting it. (I won’t feel super bad or have it actively bothering me, but it will feel aversive when I think of it.)

20% Rohin will say seeing work/resume makes him downgrade his opinion of people often, 40% he’ll think that others often do. (Insufficiently specified)

4. Experiment:

Apply.

Survey Rohin.

5. Results:

Rohin says he has a low prior on candidates so doesn’t often negatively update (which I’d anticipated), but also he doesn’t positively update much because he’s mostly doing a very basic filtering criteria on resumes and relies more on interviews. He doesn’t know how others approach applications.

I made it to the first round of work tests.

One week later, I still feel mildly aversive about the job. I don't feel anxious (which I had predicted), but I feel like I don't want the job and feel mildly aversive about doing the tasks.

I didn't make it to the second round of work tests.

6. Reflection:

After filling in this, it feels good that I applied to something that felt out of my comfort zone, and I don't feel anxious that I'm being judged for applying. 

7. Revised belief:

If I apply to something, I'll likely be rejected, feel somewhat anxious, and ultimately not have anything to show for the time. But I probably won't be judged harshly or suffer depressive episodes afterwards, and I have a chance at getting good opportunities. 

Wrapping up therapy doc  

After a few months working together, I felt more in control of my perfectionism behavior. So we decided to pause therapy. In our last session, we filled in a “wrapping up therapy” template that included prompts for my key unhelpful thoughts and their reframes. 

My most helpful takeaway from this process was the reminder that my beliefs about reality and my emotions are only moderately correlated. By going through these steps and discussing them with my therapist, I identified several beliefs that felt true even if I didn’t rationally endorse them. The log, behavioral exercises, and discussion helped me test what I reflectively endorse, so that I can come back later to the below list as a reminder when I’m next feeling pessimistic. 

Conclusion

CBT didn’t magically make all my insecurities go away. I’m still (reasonably) unsure if my more ambitious goals will succeed in having the outcomes I want. I still get discouraged and pessimistic sometimes. 

However, I am better at being more realistic about my pessimism. I don’t think you need to be the best in the world to have an impact; we need way too many people working on the world’s problems to be able to leave it all to the top .001% to solve. 

Nor do I need to already be accomplishing everything I want to. I’m focusing on building towards big goals, by setting points when I'll check in on the big picture and otherwise focusing on the intermediate goal in front of me. It’s enough if I’m making incremental progress toward my goals. 

I’m less worried about being silently and harshly judged all the time, which makes me more relaxed about asking for and receiving feedback. I also feel less pressure to make something “perfect”, and more relaxed about iteration and making something “good enough” for my purposes. 

I started this process because I felt terrified of telling people I was trying something, in case it ultimately didn’t work out. So I’ll finish today by telling you: I’m testing making writing my main work focus. I still feel nervous sharing this -- what if I’m not a good enough writer or I decide something else is more impactful? It feels like sharing my dream publicly raises the stakes. 

But sharing doesn’t terrify me, and that feels like the best evidence of the progress I’ve made. 

Resources 

If this post resonated with you, I’ve put together some resources below for you to check out. 

If you just want one resource, I’d read this post by Luisa Rodriguez. Excellent first-person example of imposter syndrome/perfectionism in person, which helped me understand the experience more clearly. Solid tips for actions you can take if it’s impacting your life. 

The CBT Log template I used. 

Overcoming Perfectionism by Roz Shafran is a CBT perfectionism workbook that both Luisa and Howie recommended (Howie’s podcast linked below). I worked through most of it, and my recommendation would be to only read it if you’re actually ready to go through the exercises as you read. The book is dry and I found just listening to it much less compelling than reading Luisa’s account or most of the ones below. HOWEVER, I expect this resource will be more effective at helping you make longer term changes if you actually do the exercises. (Bonus, it’s been tested in multiple studies that found it often helped reduce perfectionism and related symptoms, although less than also seeing a therapist.) One good point that the book clarified for me: Perfectionism is the negative effects of trying to be perfect. It's NOT “having high standards.”

The EA Mental Health Navigator provider database includes ~80 therapists recommended by community members. You can filter “Specialties” for “Perfectionism” to find providers who work with this topic. 

This 80,000 Hour interview with Tim LeBon (the therapist I worked with) breaks down perfectionism especially well from a clinician’s perspective.

The Replacing Guilt series on Minding Our Way seems to resonate with a bunch of EAs. I found it to be a good balance between the usual therapist “take care of yourself” line and the hardnose EA “we might be the only ones who can literally save the world” line. Personally, I got something like “stop running yourself ragged trying to satisfy the vague feeling you’re not doing enough and figure out how to actually make effective progress toward the big goal. If effective progress means you should take a nap right now, take a frickin nap right now.” I expect people similar to me to come away from reading this series motivated to (or maybe given permission to?) take care of themselves along the way to saving the world. (However, it does not have exercises for actually doing so, and I expect Overcoming Perfectionism will be more effective for actually dealing with perfectionism.) 

Firsthand accounts from EAs:

• https://80000hours.org/podcast/episodes/depression-anxiety-imposter-syndrome/ "Having a successful career with depression, anxiety and imposter syndrome" from the 80,000 Hour Podcast is another good personal take. 

• Among effective altruists, there’s a particular pattern of mental health problems related to feeling guilty about not doing enough to help the world: feeling guilty setting personal boundaries, or worrying that you’re not smart enough to make a difference, or thinking that what you’re doing is good but just not “good enough” to matter. Desperation Hamster Wheels is a great description of one EA’s experience with this.

• Helen Toner’s sustainable motivation talk covers some stories of her experience with pushing herself too much and what she did instead (it’s not just “work fewer hours”). 

• Tessa hits some good points on slack and moving targets in “Aiming for the minimum of self-care is dangerous,” along with her personal story of hitting a wall. 

• Jesse Galef’s mathematical model of his perfectionism. 

• “You get to go home” is another good personal anecdote from Michelle Hutchinson.

• For a very different approach, Kat Woods wrote a post claiming that doing self-directed loving kindness meditation for an hour a day for one week eliminated her imposter syndrome and work anxiety.

The Gifts of Imperfections by Brene Brown was recommended to me by a therapist once, and is a great read for someone looking to spend an extended time meditating on how you’re actually a worthy human being despite being very imperfect. To my knowledge, there are no studies showing this works for any purpose. That said, I remember liking the book as a lens for spotting certain ways/reasons I was hard on myself and being able to address those. 

The CBT Workbook for Perfectionism by Sharon Martin. I haven’t looked at this one, but it’s from Ewelina Tur’s list and is highly rated on Amazon (4.7 out of 600+ ratings). 

Perfectly Hidden Depression: How to Break Free from the Perfectionism That Masks Your Depression by Margaret Rutherford is another Ewelina recommendation.

Footnotes:

1) Perfectionism and imposter syndrome technically have different clinical criteria, but in practice I can’t tell them apart when reading personal accounts. Both usually dismiss successes for some reason and focus on perceived failures, with negative consequences emotionally and performance-wise.

2)  I also started ADHD meds shortly after finishing the CBT sessions, which made me realize why it always seemed like some people could just easily meet certain standards that were unrealistic for me without meds. So there’s a somewhat confusing jumble of experiences around “Is this standard reasonable, but just not one that I can meet?” and “Is this an unreasonable standard generally?”

Thanks to Maja Cernja, Ewelina Tur, Daniel Kestenholz, and Damon Sasi for their feedback on this post. 

I respect how carefully the effective altruists around me try to use reason and logic to make good decisions. However, I sometimes see these people spending way too much effort to make careful decisions in situations where the outcome doesn’t matter.

You might not literally be destroying your life, but you’re probably flushing a good chunk of time down the toilet if you’re spending hours googling which lamp to buy.

You might spend time on a decision because you’re afraid of making a mistake or regretting your decision. Or you might not have considered the cost of your time – it can feel intuitive to keep researching until you’ve found the best decision, without noticing that the return on your time is miniscule.

Alternatively, you could just make the simpler decision and have all of that time back. Your time is probably incredibly valuable. Learning to respect that is a key productivity breakthrough to accomplishing more.

Caveats:

1. This advice is mainly aimed at people who have the privilege to value their time more than getting the best deal.

2. It’s even better advice for people who can absorb a mistake or two (like forgetting to mail back an Amazon return or giving away a new-but-ill-fitting shirt).

3. This advice is NOT aimed at people making important and hard decisions, like “what career should I pick?” These big questions can have huge differences in how impactful one decision vs the other is, so you often want to spend a good chunk of time making a careful decision.

Because some decisions matter and others don’t, I have a couple questions that help decide how important a decision is to get right:

1. Is the decision easily reversed or undone?

One heuristic I use: Can you replace it for less than thirty dollars in less than thirty minutes? (Or more precisely, can you replace/undo it for less than ½ as much as you value an hour, in less than thirty minutes?)

Examples: Amazon returns, buying a cheap replacement for a forgotten item while traveling, tickets with a flexible change policy.

If so, you shouldn’t spend more than half an hour total researching it. There’s a good chance you should spend much less time on it.

2. Are the options about equally good?

Is it difficult to decide because both options have pros and cons that about equally cancel each other out?

Then all the books in the world probably won’t substantially improve your decision – the options that are too similar to each other to matter. Just flip a coin and be done with it.

Caveat: can you get information that will tell you one is in fact better for substantially less than the difference in value between the options?

1. Spending hours researching your career decision = expensive time cost but big payoff for getting the better option -> totally worth it.

2. Spending ten minutes researching throw blankets = small time cost, small payoff -> probably fine.

3. Spending hours researching which throw blanket to buy = high cost, small payoff -> probably a waste of your valuable, limited time.

When faced with a situation where it genuinely doesn’t matter much which option you choose, consider flipping a coin or going with a fast recommendation (e.g. picking the top Wirecutter recommendation or first 4+ star Amazon review).

If you’re going back and forth on which takeout to get, just flip a forking coin and be done with it.

The following are real stories of people that I have permission to share under fake names.

Ariel

Ariel started to work with me near the end of her internship because she couldn’t get herself to work on the final project. She loved the work, but she’d procrastinated on starting the project for weeks. She described that the work was interesting, and she really enjoyed it when she did it earlier in the internship. However, nothing she tried could force herself to start work. And she didn’t know why. Why did she procrastinate so much on work she liked?

Since Ariel felt confused about what the issues even were, we tried a bunch of mini experiments. Would breaking the task down help? What about starting with a one-minute task so it didn’t feel so overwhelming? Maybe recording herself explaining a section, and transcribing that as a rough draft? 

Many of these experiments didn’t work. For example, financial penalties were a disaster. She still failed to do the task and felt worse about losing the money.

Which makes sense, as it turned out. As we discussed her goals and motivations, we found that Ariel really cared about this project. She wanted to do well, and she felt like to do well, she had to be able to sit down and write a good draft as the starting point. 

Except, she never felt ready to sit down to write a good draft. So she never started.

As she avoided the task, it loomed worse in her mind, until she couldn’t even think about it without feeling guilty. The pressure she was putting on herself paralyzed her. She felt like a failure because she couldn’t do this task as well as she had convinced herself was necessary.

So, one of the first layers we had to work with was perfectionism. Here, she had to work on internalizing the idea that she could iterate on her work. She could write a crappy first draft and come back several times to edit it until it was good. In fact, starting with a bad draft resulted in a better output than trying to start with a good draft.

We continued working together as she started a full-time job following the internship. As we continued peeling back layers, Ariel learned to sit with and manage anxiety. She worked on better productivity routines (such as setting lots of little deadlines instead of a few big ones) and getting rid of distractions (even putting her phone in a timed lock box so it didn’t distract her). She also got more comfortable asking for help and standing up for herself.

I saw her confidence blossom, but this process took a while -- and that’s normal. 

It can take a while to see progress when working through layers of maladaptive mindsets, inaccurate beliefs, and bad habits. Making progress is all the more difficult because often clients hold themselves up against unrealistic standards, so they feel bad about themselves even when they are improving. In these cases, it really helps to focus on progress over time.   

Pat

Now, many of the cases I work with are much simpler.

Pat worked in tech, but he wanted to transition to working on AI safety. In order to do that, he needed to self-study machine learning alongside his regular job. He was having a hard time staying motivated to study after work.

Pat came to me to increase his motivation, but talking with him, it became apparent that he didn’t have a clear plan. He didn’t know exactly what he needed to study, and that sapped his motivation before he could even try studying.

So, we started by having him put together a plan for what he thought he needed to learn. Then, he sanity checked that plan by talking to a couple people who worked at the orgs he was interested in.

That turned out to be immensely valuable. First, they told him that the list basically looked good, but about half of the things weren’t that important. So, he reduced what he needed to study by 50%, basically meaning he was learning the important content twice as fast.

Second, he started volunteering at one of the orgs. He ended up deciding to keep his regular job and continue volunteering on the side.

In Pat’s case, his desire to spend more time studying was a proxy for what he really cared about – making progress toward doing work he found meaningful. Setting clear goals, prioritizing only the most important tasks, and getting feedback early all helped him make much faster progress to that goal than he would have if he had just powered through more hours studying.

Phil

Phil started coaching as a senior studying computer science. He was preparing to start his master’s program. Except he hadn’t done that well in undergrad – he procrastinated chronically, started his assignments the night before they were due, and generally didn’t think he was prepared to get the most out of his master’s program. It was a make-or-break point. He was paying for this program, but he had to step up if he was going to make that investment worthwhile.

We started working on three main goals: daily planning, starting projects in advance, and implementing more effective learning strategies. However, over the first couple meetings, he only completed about half of the plans we made.

After setting the same goals a couple times, we discussed whether this was really the right approach. He really thought these were the goals he cared about - despite failing several times - so we agreed for him to try out financial commitments. If he didn’t complete the action item, he would pay a penalty. This seemed to help him get over the initial hump of trying out interventions.

During this process, we ran a bunch of experiments. With the goal of focusing better, some of his experiments included different work locations, organizing coworking groups, studying with his girlfriend, and penalties for individual tasks.

We had twelve calls over the span of four months reinforcing those key habits. He felt pretty happy with that progress, so we stopped coaching.

He returned a year and a half later. He’d done well in the master’s program and was working at a startup that he liked. However, he wasn’t getting as much done as he wanted. We set out to increase his productivity.

However, as we discussed why he wasn’t motivated, we found signs that maybe the job wasn’t a great fit for him. He liked the work but didn’t think the job was meaningful. He started exploring potential jobs and found some that felt very exciting.

However, in the process of choosing a specific goal to work towards, he noticed that it felt scary to think about changing. He liked his job and felt like he was learning a lot. Stepping into the unknown felt scary. Because of that fear, it had been easier not to really consider leaving in the first place. Instead, he could feel like he was making progress toward being more productive -- without having to face that decision directly.

This emotional hurdle was what he really needed to work past before he could make progress in his career. Last I heard from Phil, he had just started a new job he’s excited about and thinks could be really impactful.

Originally, Phil just needed to experiment. He needed to try a lot of things, with some accountability to help him follow through on them, to find the habits that would help him accomplish his goals. The second time, however, what originally seemed like a problem with not getting enough done, turned out to be an emotional hurdle that he needed to work around.

Conclusion

As you can see from these stories, productivity coaching covers a lot of ground. People frequently come to me with some version of “I want to get more done”, but the following process often includes experimenting, self-discovery, and zooming out to find the best path to accomplish their goals. It isn’t enough to just mindlessly apply a toolbox of productivity advice (however good the advice) – there isn’t a prescribed solution that will solve any particular problem. I know tips that often help, but a lot of the process is figuring out which strategies work for this particular person and situation.

I think this is also why it can be helpful to have a coach who works specifically with effective altruists. It gives me a shared mindset with my clients, so it’s easier for me to try to understand their perspective.

This ninth and final post covers general advice from my guests.

You can view bios of my guests and other posts in the series here.

__________________________

Backchain

We want to think, figure out some things to do, and then, if we do those things, the world will be better. An important part of that, obviously, is making sure that the things you think about, matter for the outcomes you want to cause to happen. 

In practice, it seems to me that what happens is people get into an area, look around, look at what other people are doing. They spend a few minutes, possibly hours thinking about, “Okay, why would they be doing this?” This seems great as a way to get started in a field. It's what I did. 

But then they just continue and stay on this path, basically, for years as far as I can tell, and they don't really update their models of "Okay, and this is how the work that I'm doing actually leads to the outcome." They don't try to look for flaws in that argument or see whether they're missing something else. 

Most of the time when I look at what a person is doing, I don't really see that. I just expect this is going to make a lot of their work orders of magnitude less useful than it could be.

Rohin Shah

Fail often

One thing on my to-do list is to make sure I fail every month at something because then that means that you have been reaching your max. When I was applying for fellowships, scholarships, internships in policy and whatever, if I had looked at every single rejection as a failure rather than just expected, then I wouldn't be here. 

Abi Olvera

Guilt is a symptom, not a motivator 

I think guilt is a feeling I have reasonably often, not too intensely, and not all the time. It doesn't generally cause me to be motivated. It's more usually a symptom of being in an unmotivated place. Or being in a place where there's not so many hours I can pour into my thing and I'm wandering in the desert. I'll often feel, less frequently now but still sometimes, I'll feel guilty and frustrated that I haven't moved on something yet, things haven't moved forward, I haven't put in a lot of hours.

Guilt can get you to do some things, like call your parents or maybe even run an extra lap or something. The bottleneck for finding traction in research isn't something that I find I've been able to usefully address by feeling guilty about it. It's more like an effect of being in a place where I'm for whatever reason not that productive.

Ajeya Cotra

It’s okay to skill up before doing impactful work 

I made a mistake of feeling I needed to be working on some really impactful thing right away. I think that worked out okay, in my case. I got somewhat lucky in that regard, but I should have been totally happy to take at least a year or two and just try to find the place where I could skill up as well as possible.

Daniel Ziegler 

You don’t need to be impactful immediately in your career 

Maybe don't stress out about the fact that you're not impactful now. 

Abi Olvera

Competing with the very best takes a lot of time and effort 

It's really hard to be the top percent, to compete with people who are the 20 who get the national scholarship. I feel like if I would have eased up, it wouldn't have happened and then I would have had outcomes that are more typical of people from my socioeconomic background. I would say I actually wasn't unhappy but I had reached, probably, the max of what I could do, at least in terms of time. 

Abi Olvera

Calibrate recommendations 

Maybe one thing is trying to get a sense of either calibration or confidence of recommendations. I think my impression is that EA land has gone wrong in the past along the lines of miscommunications like, "Oh, org x says y." While y is meant as a tentative suggestion, we take that as a cast-iron recommendation and then make lots of decisions. Which then, because it was tentative, actually turned out to be mistaken, and we're a bit screwed over by that. One way of avoiding this, is trying to interrogate—“Have you examples of this? Are there counter-examples? Can you give a credence on this?” 

Greg Lewis 

You don’t have to be a superstar to do good 

[EA] is a group project, and you can only decide for yourself as an individual what your career's going to be, but EA's made up of a bunch of different people with different careers and different projects and different skills. Yours may or may not be this great success story, but we don't all have to be. 

Back in Earning to Give, I was just like, "I just can't see my way to it. I don't know what I'm good at that would earn a bunch of money, there's nothing.” If we'd stayed in that world where we're really funding constrained, I just wouldn't have been able to contribute that much. I still would have tried, but there just would have been a pretty different niche. I think trying to make peace sometimes with, "Okay, I'm going to have this niche, and it's not going to be cool or famous or something, but, that's just how it is." 

I think, if we had stayed in the days where I was only saving people's lives by donating to GiveWell charities, that's still really good. Michelle Hutchison has a good post about this, that it's just like having some absolute impact. Even if other people are having more impact than you, it doesn't reduce what you are able to accomplish. In the beginning, we were all just like, "What? I can save a life for $2,000 or whatever?" And that's still there. The numbers changed around, but you can still do good stuff without being a superstar, and no one can take that away from you.

Julia Wise

You might not need a lot of education to do the thing 

I assumed that a lot of the world had more accumulated knowledge than it really did. When we were initially setting up CEA, it really felt like, “Well, I assume that there's a lot to note before you can set up a company. That it’s this complicated business that a bunch of people have studied hard to do, and that's the only way you can set up a company.” Whereas it turned out, in fact, you can just read Companies House website and then just go and do it.

I think part of this is coming from people who are interested in academic things and do quite a bit of study. That gives you a false perspective on the world because you do a master's in philosophy and then end up doing a PhD and it's all aiming towards academia. The idea is that you need to have done 10 years of education in order to do this thing.

The idea that there could be something like setting up a company that actually you can just do by reading the website and then filling in the form feels really surprising.

I see this quite a lot in the people I talk to as well, particularly the ones who have done PhDs. Someone will have done a PhD in chemistry or something and be interested in going into policy and feel like, "Oh, presumably, therefore, I need to do a bunch of training in policy or do some policy degrees or something" when actually there are a bunch of fellowships directly designed to get people who have PhDs in science directly into Congress or something, but that feels very counterintuitive when you are in this environment where you need to have done tons of education in order to do a job like a postdoc.

Michelle Hutchinson 

Sometimes it’s good to finish things even if you’re not sure they’re important 

I had decided that actually, this approach I was thinking of was not likely to work, so probably it wasn't that important. I ended up finishing it anyway, out of a principle of: it's good to finish projects because, as you go through them, you often think that they will be bad or not that useful, but sometimes they'll be impactful in a way that surprises you. At the very least, you want to tell everyone else, "This is what I did, here's why I think it's not that good, and you shouldn't be thinking about it."

Rohin Shah

This eighth post covers thoughts on mentors, colleagues, and feedback.

You can view bios of my guests and other posts in the series here.

How did you develop your most meaningful mentors?

Admit your vulnerability and ask 

I’ve developed great mentorships by asking questions, and sometimes coming clean about my total ignorance about a topic, tactfully. Without seeming incompetent, you can definitely talk about your vulnerability and knowledge gaps. For example, after some big meeting, I can tell that there's some background information that I'm missing, maybe an ulterior motive someone seems to exhibit. To know the whole picture, I'll privately ask a trusted higher-up for the things between the lines. People are often glad to teach others the ropes.

Abi Olvera

Having role models can open doors 

Setting up a charity feels like this big intimidating thing that someone coming straight out of university would have no idea how to do.

Charity Entrepreneurship is saying, "Hey, you too can do this. Here are some things that you need, you need to be a smart person and have a good idea and stuff like that. Hey, we're going to vet people and figure out if you seem like actually a good fit for this. Also, we're going to describe some projects that you could do that are worth trying. Also, we're going to model it. Look, here are a couple of charities that have gone really well. You too can do this."

I think that kind of thing is also pretty useful because as an undergrad, I just hadn't really come across anyone who was setting up companies or charities or anything. I had no idea that you would be able to do this without too much work. The kind of thing that made it feel way more viable was talking to some people who had done it and were just like, "Yes, no, this isn't that hard. Here are the kinds of things you do. Also, here are some other people you can ask for help from," that kind of thing.

I think putting more information out there, I guess including the kind of thing that you're doing with your interview series, can help people realize that these are the kinds of things that they could have a go at as well.

Michelle Hutchinson

Learn by working with senior people

It's important to hear what more senior people think, who’ve sort of gone through this process. Especially because senior people tend to be more plugged in and have a lot of accumulated knowledge. So trying to pick their brains on things tends to be more valuable. You can also learn a lot through collaborating with people. I'm working with this guy right now, Stefano DellaVigna, who's fantastic. And it's been a great learning experience for me to see how he works on grant applications and how he phrases things. It's been really wonderful working with him. So some of that is also just learning by working with other people.

Eva Vivalt

Work with people who can mentor you

I think because the field that I'm in is very relationship-based, relationships have been pretty important. Two really obvious examples have been the two main bosses I've worked with, Holden Karnofsky and Jason Matheny. Just trying to have good relationships with them, trying to do good work so that they trust me in my work. I think they both have supported me a lot. That's been very, very important. 

Helen Toner

Encouragement is sometimes necessary 

I tended not to find it easy to come up with ideas and just decide to execute on them. It meant that it was really useful for me to be surrounded by people who were ambitious and had ideas.

In particular, I'm very motivated by helping the people around me. As soon as there was someone doing some ambitious project and they were like, "We want to do this thing. Do you think you can do this part of it or something?" that immediately feels way more appealing to me than coming up with that idea and deciding to do it. Things like running Giving What We Can, I just definitely don't think I would have, on my own steam, decided that I was ready for. Whereas having someone be like, "Nope, I think you are ready for it, and you're going to do it, go get them" made all the difference to me.

I think some of that can come somewhat quickly. I think part of what advising does often is, look at someone's CV, talk to them for half an hour and be like, "Yeah no, I just think you can do this, maybe won't go well, but I'm pretty sure you can do it. You certainly can try. No one will think you're ridiculous for trying," that kind of thing.

I think it doesn't require many months of getting to know someone super well, you can just be like, "You're the kind of profile for whom this is reasonable." 

Michelle Hutchinson 

How do you elicit useful feedback? How important is being surrounded with or working with other great people? 

Use objections you’ve already heard to elicit better feedback

Trying to anticipate objections and listing objections that you've already heard and asking people to say if they agree with them or if that sparks any thoughts, is pretty helpful and will get you a lot more responses.

Ajeya Cotra

Being surrounded by great people seems important for research

I think I've definitely benefited from having people to follow or at least people to try to join in their projects. I've benefited from that a lot. As far as research ideas goes, this definitely feels true to me. It does seem really important to have people that bounce ideas off.

Some people also seem to just go off on their own, do a bunch of really good stuff. I don't know how that would work.

Daniel Ziegler

Get feedback opportunistically or go where it’s easy

The main way I get feedback is actually by talking with particular individuals. Like there’s some person who is in my field, and I reach out to them directly and try to chat with them. It can also be a little bit more opportunistic—so it's maybe somebody who's not quite in my field, but you know, maybe they're in my department or I run into them in the hallway or see them at a conference or something like that. So your environment also matters. 

Eva Vivalt

Ask observant, kind people for feedback 

I think if you can identify people who are both observant and reasonably kind, and try to seek out feedback. One of my coworkers in that first social work job just gave me some kind feedback about like, "Here's some things you're doing that are annoying people." She did it in this gentle way and I'm just like, "Oh my gosh, I wish you had told me that six months ago.” That was very helpful.

I think if I had said, "Hey, you've been in this field a lot longer than I have. If you had three tips to give me, what would they be?" then I wouldn't have been so embarrassed when I realized, "Oh my gosh, I've just been screwing up these things," and not knowing it because my manager was too busy until like three months in to be like, "Here are all the things that annoy your coworkers."

Julia Wise 

Pick people’s brains about areas they know well 

People can really point you to the things that you should be paying attention to, which is really useful. It's not only the case that you can come to them with specific questions that are useful for your work, but also that they can notice other things that they know about that are useful for your work until you can really find the most useful things. I find the EA community great for this just very disproportionately if you write to a person like, "Hey, 'I have some questions about your specific job,'' they're happy to chat. 

Then I prepare beforehand to make sure that I have specific questions. For example, during advising sessions, I'll try to write down afterward whatever came up that I didn't know the answer to and that way, when I'm talking to someone if I notice like, "Hey, I feel like I have a few finance questions," I can reach out to someone in finance and be like, "Hey, do you want to chat?" and then go through my questions.

Michelle Hutchinson

Try copying beliefs to build models

I think an example of this is when I was trying to build my models of how AI will work. 

One of the major things I did was “Can I take some person who's already working in AI safety and has written a decent amount about their views? Can I inhabit that person's perspective and explain why they have these views?” Frequently, I couldn't, but some of the times I could. I think even when I couldn't, the act of trying gave me a lot of information. 

Imagine that is like giant haystack of possible things you could think about AI safety, and then there's this one needle that's Eliezer Yudkowsky. Then you're like, "Okay, well, I don't really know exactly where Eliezer's needle is in the entire haystack, but I know it's in this tiny little portion over here based on things that he's written." 

A lot of the things, that already narrows it down a lot, and then I can think about, "Okay, well, it can't be this particular view because that makes no sense. Maybe it's this other thing." It narrows down your search space a lot. Rather than sifting through the whole haystack, you're like, here are these 10 different spots where it could be out of millions of possible spots, and search only those 10.

Rohin Shah