Statistical Modeling, Causal Inference, and Social Science https://statmodeling.stat.columbia.edu Sat, 10 Oct 2026 01:03:51 +0000 en-US hourly 1 https://wordpress.org/?v=6.4.11 Columbia University says, “We believe in difficult conversations at Columbia—in a broad embrace of freedom of expression. It’s in our DNA.” https://statmodeling.stat.columbia.edu/2026/10/11/columbia-university-president-says-we-believe-in-difficult-conversations-at-columbia-in-a-broad-embrace-of-freedom-of-expression-its-in-our-dna/ https://statmodeling.stat.columbia.edu/2026/10/11/columbia-university-president-says-we-believe-in-difficult-conversations-at-columbia-in-a-broad-embrace-of-freedom-of-expression-its-in-our-dna/#comments Sun, 11 Oct 2026 13:24:55 +0000 https://statmodeling.stat.columbia.edu/?p=52610 Continue reading →]]> Claire Shipman, the president of Columbia University, writes:

We believe in difficult conversations at Columbia—in a broad embrace of freedom of expression. It’s in our DNA. I intend to uphold that tradition for several reasons.

First, as a nation, we should be able to tolerate ideas, facts, and opinions that we don’t like. Tolerance is essential to building resilient individuals and a resilient democracy.

Second, it’s our mission. We have a responsibility to guide students toward intellectual rigor, to enable them to grapple with, and examine carefully, ideas different from their own. More broadly, a willingness to test assumptions is essential to our academic excellence in every field.

Third, and perhaps most important, universities—Columbia in particular—are uniquely positioned to help change the national tone. We can guide and model the art of examining complex, potent issues in a deliberate, respectful fashion. . . .

Our instinct might be, in such an era, to shy away from challenging events and speakers. The better course is to model how to do it well . . . Our statutes lay out quite clearly and sensibly that expression, on our campuses, does have limits, in particular as regards harassment, discrimination, or the threat of violence. Peaceful protest, civil discourse, a respectful exchange of ideas—this is our inheritance at Columbia, and I want to ensure that it is also our legacy. . . .

So, as this week begins, and as we continue to debate the affairs of the world and this nation on our campuses, consider this: at times there will be speech that each of us finds disagreeable, even abhorrent—and we still need to protect it. Maybe even seek it out. Such a practice makes us stronger.

I agree. And, although our president did not say so in this message, this policy of protecting and welcoming dissent, even when it is controversial or disagreeable, runs counter to some actions of our university in recent years:

  • The delayed reaction to the sexually-assaulting doctor in the medical school, without any seeming consequences to the people at the university who enabled this behavior.
  • The fake U.S. News numbers, where, again, there seems to have been very little interest by the university administration in finding out who did it and why. As with the medical school scandal, the avoidance of any serious investigation does not seem consistent with the goal of promoting free speech.
  • Barnard College, a division of Columbia University, attempting to cancel the showing of a controversial film on campus. At that time, it seemed that everything was about avoiding offending anybody and about avoiding any controversy—really the opposite of the view that “we should be able to tolerate ideas, facts, and opinions that we don’t like.”
  • Barnard College “mandating that students remove any items affixed to room or suite doors” under the rationale that they “do what we can to support and foster the respect, empathy and kindness that must guide all of our behavior on campus.” Empathy and kindness are fine, but, again, it contradicts the idea that “there will be speech that each of us finds disagreeable, even abhorrent—and we still need to protect it.”

As I noted the other day, Barnard College has also announced a new policy promoting free expression, and I think it’s a good thing.

I completely agree with the president of Columbia that “we should be able to tolerate ideas, facts, and opinions that we don’t like,” and I completely agree with the president of Barnard that “Colleges and universities need to reconfirm our commitment to nonviolent forms of disagreement — even when we are confronted with voices that disparage or dismiss identities and worldviews. This is also a time to foster more disagreement, not less.” I was getting very frustrated with all the happy talk and the official attitude that we shouldn’t be disagreeing. The university—and, of course, the country—has people who disagree with me on important issues, and people I don’t respect at all, for moral, intellectual, or political reasons. And I guess lots of those people don’t respect me either! Which is too bad, but we still have to exist, and we should have the right to express our views, both publicly and privately. We shouldn’t be fired for our nonviolent acts (except to the extent that they interfere with our ability to do our jobs), nor should be discouraged by the university from expressing them.

And, just to be clear, no one at Columbia has ever suggested or even requested that I keep my mouth shut in class or outside of class, including when I’ve criticized the government or the Columbia administration or anything else. I have nothing to complain about on my end, but I do think it was a mistake for them to crack down on dorm room doors, to try to stop the showing of controversial movies, and to keep quiet the bad behavior regarding the fake statistics and the terrible actions at the medical school.

I’m happy about this new policy and I hope the administration follows up by changing the dorm room door policy, apologizing for trying to suppress the movie, and starting and publicizing independent investigations into the U.S. News numbers and the enabling of the sexual assault at the hospital.

Also let me emphasize that by bringing up these earlier cases, I’m not criticizing the Columbia president Shipman, as these things were done before she took office. I also understand the desire to move forward rather than looking back; still, I think that a great way to reinforce the policy of enabling difficult conversations is to revisit some recent examples where the university had unfortunately avoided doing so.

“It’s in our DNA.”

One thing that jumped out at me when reading the president’s email was the statement that freedom of expression, especially in difficult conversations, is “in our DNA.”

My first reaction was the annoyance of the persnickety social scientist: calling it “DNA” is just wrong, and it indeed muddies the waters, as we’re talking about something social, not genetic.

But that reaction of mine was silly. Obviously no one when using “DNA” in that sense is taking it literally; it was a metaphor. And maybe we can gain some insight by taking this metaphor seriously.

For something to be in an organization’s “DNA,” that would imply that it’s not just a transient trait; rather, it has to be in some equivalent of self-replicating code. The idea is that freedom of expression and willingness to confront uncomfortable ideas are not just attitudes that sometimes has existed at Columbia, but rather that they are embedded deeply enough that they will persist. Indeed even if free inquiry and expression at Columbia are suppressed (as with the sexual assault, the fake statistics, the dorm-room censorship, and the attempt to suppress the film screening), the self-replicating aspects of our university’s rules and culture will cause them to return.

And, from that perspective, I do see something in our president’s analogy to DNA. Columbia has for many years celebrated the legacy of the 1968 protests, while recognizing that the other side at the time had legitimate concerns too. That is, Columbia has recognized the value of protest without implying that any particular protest action has to be recognized as being a good idea. We have the University Senate, which dates from that period, as well as generations of new students and staff who have exercised the freedom of dissent—even dissent that some can find disagreeable—and recognize this as a Columbia tradition.

Indeed, this recent statement by the president can be viewed as a consequence of that DNA, an expectation that our university is a place with diverse enough attitudes that there will be people who disagree with and even despise each other but still have the right and, at times, the obligation, to air these uncomfortable disagreements publicly.

Because the alternative, of simulating a nonexistent harmony, would be intellectually dishonest and politically dangerous.

]]> https://statmodeling.stat.columbia.edu/2026/10/11/columbia-university-president-says-we-believe-in-difficult-conversations-at-columbia-in-a-broad-embrace-of-freedom-of-expression-its-in-our-dna/feed/ 1 The problem is noise, not N: why increasing the sample size isn’t all that. https://statmodeling.stat.columbia.edu/2026/10/10/the-problem-is-noise-not-n-why-increasing-the-sample-size-isnt-all-that/ https://statmodeling.stat.columbia.edu/2026/10/10/the-problem-is-noise-not-n-why-increasing-the-sample-size-isnt-all-that/#comments Sat, 10 Oct 2026 13:11:15 +0000 https://statmodeling.stat.columbia.edu/?p=53646 Continue reading →]]> Someone was talking with me about the problem with lots of published social science research that’s based on small sample sizes. And I agree that this is a problem! But I think the real problem is noise, not N.

Noise (variation in effect size, variation in outcomes, and measurement error) is more of an issue than sample size because lots of bad studies are so noisy that, to get a large enough sample size to detect a reliable effect, you’d need N = a zillion, and at that point you’d be averaging over such a wide range of conditions that it’s not clear what you’re estimating anyway.

There’s a widespread misunderstanding by which people think that their problems can be solved with bigger samples. I think the erroneous reasoning goes like this: People are aiming for 80% power, but they really have only 50% power, or 30% power, or something like that. If you have 50% power and double your sample size, you’ll have 80% power . . . a win! The error in this reasoning is that often the power is much less than 50%, more like 5% + epsilon. The flip side of this is that if you really are running studies with 50% power or even 30% power, you’re likely to get the right direction on the effect size anyway, so often there’s no good reason to wait on statistical significance.

So, lots of moving parts here: There’s the underlying effect size, the variation in effect size, the variation in the measured outcomes, and researchers’ expectations when setting effect sizes and interpreting results.

]]> https://statmodeling.stat.columbia.edu/2026/10/10/the-problem-is-noise-not-n-why-increasing-the-sample-size-isnt-all-that/feed/ 6 Digital twins version 2.0: The skeptics speak up https://statmodeling.stat.columbia.edu/2026/10/09/digital-twins-version-2-0-the-skeptics-speak-up/ https://statmodeling.stat.columbia.edu/2026/10/09/digital-twins-version-2-0-the-skeptics-speak-up/#comments Fri, 09 Oct 2026 15:22:39 +0000 https://statmodeling.stat.columbia.edu/?p=54782 Continue reading →]]> This is Jessica. Several years since the earliest research papers on the topic came out, the idea of using LLMs to simulate human behavior like survey responses or experiment outcomes (“silicon sampling”) is hitting mainstream. People tend to bring this up in conversation with me, probably because I’ve written about validation and taught a course, so I get to hear a lot of other researchers’ views. I would put the majority of takes I encounter in either the more agnostic, “Somebody should probably explore how much we can do with them” and/or “I’m pretty doubtful it’s going to live up to the hype” camps.  

One way to use silicon samples is digital twinning, where simulations are conditioned on considerable information about specific individuals, and expected to predict the responses of those specific individuals. Early papers on digital twins sing their praises, showing for example that they can reproduce people’s responses more than 80% as well as people replicate their own responses (e.g. here or here). But we’re now seeing a more critical empirical push, which uses novel tasks and topics and more careful design of metrics and baselines for comparison to gauge how much conditioning on more individual-specific information improves prediction. 

In the last month, there have been a few noteworthy data points. Pew Research put out a report describing what they learned from comparing digital twins implemented by giving Claude Opus 4.6 detailed 2025 survey responses and demographic persona information for people who responded to several more recent survey waves. They compute the average absolute error in percentage points for each multiple choice survey question (i.e.,the average of the absolute differences in the proportion of LLM versus human respondents selecting each answer option), and find it averages 12 percentage points over the 300 survey questions they looked at, with 28% of the questions having error over 15 percentage points. They observed several kinds of biases that have come up in other research comparisons: less variation in LLM responses compared to human (with some answer options never being picked by the LLM), stereotyping (like predicting that 97% of Hispanic adults are at least somewhat likely to follow the World Cup; in reality its only 43%), overestimating human accuracy on factual questions, and mispredicting sentiment on current events like data centers and face coverings on ICE officers. They stress how hard it is to predict how far off the LLMs will be on a given survey question.  

This list of biases closely resembles those proposed by Tianyi Peng et al. in one of the most comprehensive evaluations of digital twins to date, recently published in Science Advances: Digital Twins as Funhouse Mirrors: Five Systematic Distortions. They collect a new set of 19 preregistered studies involving 164 outcomes, and find that overall, there is only weak improvement over base models and relatively low correlations with human behavior (on the order of 0.2). 

I like this paper a lot, in particular the comparisons they do to simpler demographic prompting and base models. The intuition that has been sold by researchers and companies in the space alike is that with rich enough information about individual people, we can predict their behavior with high accuracy. To make the twins approach worthwhile, we want to see that twinning specific individuals leads to responses that predict variation between individuals better than the demographic baseline. However, some of the early papers advocating for digital twins evaluated them by looking at measures where a model can do well simply by being a good predictor of the average response. For example, you can look at the correlation between each individual’s real and twin response over a set of likert-style survey questions, but a model that does a good job of predicting on average which questions will get higher responses may appear to do well by this measure, even if it does a poor job of sorting individuals for any particular question. 

Instead, Peng et al. look at correlations between human and twin responses for a single outcome at a time, and compare to demographic personas only as well as base models, to ask how much additional predictive power the twin approach provides when it comes to sorting people into those that will respond with higher versus lower values. They find that while there’s a boost in average correlations (from 0.08 with base models to 0.15 with demographic personas to 0.2 with twins), overall the correlations remain modest, and much more modest than the correlations over questions that other papers have reported. Hopefully it encourages more care in future papers when it comes to what measures actually support the claims being made. I do think behavioral researchers and others interested in this space are gradually changing their mental models to conclude that LLMs may do a decent job of predicting population averages for well-represented populations, but they have a long way to go when it comes to modeling individual heterogeneity. There are several other papers along these lines that I haven’t had a chance to look closely at. 

How damning is this for the approach itself? I think it remains hard to say, though it certainly suggests skepticism is warranted at this point in time. The reasons I wouldn’t write digital twins off completely are 1) it is still early and models continue to improve and 2) prompting is often not the best way to recover the predictive information a model has–you can do better at opinion prediction by using model internal representations (see, e.g., this). On the other hand, ultimately it comes down to limits on how predictable individual behavior is, and while people are very predictable in lots of ways, we are also always exercising our agency and changing in subtle ways that interact with the world, which even we ourselves can’t necessarily understand or predict. So I don’t think reaching AGI is synonymous with being able to predict the majority of individual opinions and actions. I’m reminded of Matt Salganik et al’s work on the Fragile Families Challenge, predicting life outcomes from thousands of variables, where the best of the 160 teams’ models explained only about 20% of the variation on the most predictable outcomes, despite the enormous amount of data. 

What I will say with certainty is that silicon sampling has attracted some very good salespeople. I expect a lot of money will be made before anyone abandons the idea.

]]> https://statmodeling.stat.columbia.edu/2026/10/09/digital-twins-version-2-0-the-skeptics-speak-up/feed/ 13 Thinking more seriously about the design of exploratory studies: A manifesto https://statmodeling.stat.columbia.edu/2026/10/09/thinking-more-seriously-about-the-design-of-exploratory-studies-a-manifesto/ https://statmodeling.stat.columbia.edu/2026/10/09/thinking-more-seriously-about-the-design-of-exploratory-studies-a-manifesto/#respond Fri, 09 Oct 2026 13:53:14 +0000 https://statmodeling.stat.columbia.edu/?p=53046 Continue reading →]]> I came across this post from 2016 which I still think is important.

The idea is that there should be some guidelines for experimental design, not just for “confirmatory” studies that produce estimates, standard errors, predictions, hypothesis tests, Bayesian inferences, etc., but also for exploratory research.

As Ed Hagen put it:

Exploratory studies need to become a “thing.” Right now, they play almost no formal role in social science, yet they are essential to good social science. That means we need to put as much effort in developing standards, procedures, and techniques for exploratory studies as we have for confirmatory studies. And we need academic norms that reward good exploratory studies so there is less incentive to disguise them as confirmatory.

Here’s what I wrote, a decade ago:

Suppose you know ahead of time that your theories are a bit vague and omnidirectional, that all sorts of interesting things might turn up that you will want to try to understand, and you want to move beyond the outmoded Psych Sci / PPNAS / Plos-One model of chasing p-values in a series of confirmatory studies.

You’ve thought it through and you want to do it right. You know it’s time for exploration first and confirmation later, if at all. So you want to design an exploratory study.

What principles do you have? What guidelines? If you look up “design” in statistics or methods textbooks, you’ll find a lot of power calculations, maybe something on bias and variance, and perhaps some advice on causal identification. All these topics are relevant to data exploration and hypothesis generation, but not directly so, as the output of the analysis is not an estimate or hypothesis test.

So I think we–the statistics profession–should be offering guidelines on the design of exploratory studies.

An analogy here is observational studies. Way back when, causal inference was considered to come from experiments. Observational studies were second best, and statistics textbooks didn’t give any advice on the design of observational studies. You were supposed to just take your observational data, feel bad that they didn’t come from experiments, and go from there. But then Cochran, and Rosenbaum, and Angrist and Pischke, wrote textbooks on observational studies, including advice on how to design them. We’re gonna be doing observational studies, so let’s do a good job at them, which includes thinking about how to plan them.

Same thing with exploratory studies. Data-based exploration and hypothesis generation are central to science. Statisticians should be involved in the design as well as the analysis of these studies.

So what advice should we give? What principles do we have for the design of exploratory studies?

Let’s try to start from scratch, rather than taking existing principles such as power, bias, and variance that derive from confirmatory statistics.

– Measurement. I think this has to be the #1 principle. Validity and reliability: that is, you’re measuring what you think you’re measuring, and you’re measuring it precisely. Related: within-subject designs or, to put it more generally, structured measurements. If you’re interested in studying people’s behavior, measure it over and over, ask people to keep diaries, etc. If you’re interested in improving education, measure lots of outcomes, try to figure out what people are actually learning. And so forth.

– Open-endedness. Measuring lots of different things. This goes naturally with exploration.

– Connections between quantitative and qualitative data. You can learn from those open-ended survey responses—but only if you look at them.

– Where possible, collect or construct continuous measurements. I’m thinking of this partly because graphical data analysis is an important part of just about any exploratory study. And it’s hard to graph data that are entirely discrete.

I think there’s room for something longer and more systematic on this topic. It’s important.

]]> https://statmodeling.stat.columbia.edu/2026/10/09/thinking-more-seriously-about-the-design-of-exploratory-studies-a-manifesto/feed/ 0 Getting better causal inferences by using more information (my talk at Stanford this coming Wed) https://statmodeling.stat.columbia.edu/2026/10/08/getting-better-causal-inferences-by-using-more-information-my-talk-at-stanford-this-coming-wed/ https://statmodeling.stat.columbia.edu/2026/10/08/getting-better-causal-inferences-by-using-more-information-my-talk-at-stanford-this-coming-wed/#comments Thu, 08 Oct 2026 23:03:29 +0000 https://statmodeling.stat.columbia.edu/?p=54781 Continue reading →]]> It’s the political economy seminar at Stanford business school, Wed 14 Oct at noon in room M109:

Lagged regression, difference in differences, two way fixed effects, regression discontinuity, placebo controls: these are different methods used when estimating causal effects from panel data. We’re working on a general approach using multilevel regression and poststratification (MRP) that aims to get the best from all these approaches. We demonstrate with a range of examples, including observational studies in economics, political forecasting, and controlled experiments in biology. This is joint work with Jori Korpershoek and others.

Come and ask me some tough questions!

The talk will be pretty open-ended. We have lots of ideas and some successes (for example, the chickens paper) but it’s mostly open questions and promising research directions. Should be fun.

I won’t be talking about this paper or this paper at all—that’ll have to wait until I’m invited to speak in the English department.

]]> https://statmodeling.stat.columbia.edu/2026/10/08/getting-better-causal-inferences-by-using-more-information-my-talk-at-stanford-this-coming-wed/feed/ 3 What’s the deal with the RMET (“the Reading the Mind in the Eyes test”)? https://statmodeling.stat.columbia.edu/2026/10/08/whats-the-deal-with-the-rmet-the-reading-the-mind-in-the-eyes-test/ https://statmodeling.stat.columbia.edu/2026/10/08/whats-the-deal-with-the-rmet-the-reading-the-mind-in-the-eyes-test/#comments Thu, 08 Oct 2026 13:58:08 +0000 https://statmodeling.stat.columbia.edu/?p=53644 Continue reading →]]> Dan Luu writes:

Have you ever looked into the RMET? [This is something called the Reading the Mind in the Eyes test, which I’d never heard of until receiving this email. — AG] I’d like to get some data to replicate a result. I have a survey here.

The story, as it relates to me, is that a friend of mine was hanging out in a cabin with a bunch of friends. A friend of theirs, who’s a psychologist, has them take this test. After they take the test, the psychologist reveals that neurotypical people complete the test in N minutes vs. M minutes for autistic people. People sort of can’t believe it, do all neurotypicals really finish the test that quickly? That’s super fast! But it’s plausible that everyone in the room is autistic, so it’s not totally unbelievable.

My friend then has two of his most normal friends take the test and they’re actually slower than he is. They have a discussion about social interactions, etc., and they conclude that they’re autistic, which is surprising/interesting news to them.

Ok, that’s an interesting story so far. I got that part of the story from this friend, Then, a week later, he messages me about this thing again. He had some more normal seeming people take the test and none of them finished the test under the “neurotypical” threshold, so that’s a bit suspicious.

I look into it a bit and, according to some source, neurotypical finishing times are 2-3 minutes, with > 3 minutes being autistic. I can’t find any sort of citation for this, whether it’s a paper or someone putting the data on a website for anything like that. It’s just a site where they have the test online and make the claim that some scientists figured this out. There’s also some study with schizophrenics where the 59 “healthy controls” took 5.45 +/- 2.58 minutes. This makes the 2-3 minutes for neurotypical folks vs. 3+ minutes for autistic folks thing seem like it might be fake?

Also, on looking into the RMET further, the untimed version also seems pretty fake for a bunch of reasons, one of which I’d like question-level data for.

One thing that seems fake is that some online places you can take the test report neurotypical vs. autistic thresholds like the time-based threshold, but on looking up what the distribution of results is, it’s pretty clear that the difference within a group is much larger than the difference between groups, so no such threshold-based criteria can really make sense. Also, it turns out that Asian folks score worse than white folks on the test, unless you make a race-normed RMET, and just eyeballing the difference, the difference there is something like the order of the alleged difference between autistic and non-autistic folks.

For me, on the official test, the facial expressions that took me the most time (and I made errors on) were some of the photos of women who look like models or actresses. I think that’s because I don’t normally interact with people who look like that (vary manicured eyebrows, totally flat skin with no wrinkles but not a young child, etc.), so the faces aren’t super readable to me, especially in combination with the relatively subtle things some of the questions are trying to have people read. My guess is I’m not the only person like this and a lot of other people will miss the same questions, but maybe not.

Anyway, do you have suggestions on how to go about this? My current plan is that I’ll just tweet out the survey after some testing to make sure the survey works, but the set of my twitter followers or whoever sees it from it being retweeted is not going to be very random, but I don’t know that I have a good way to fix this problem.

And Dan shares a very small bit of evidence:

A friend of mine tried it and, while doing it, said she couldn’t make heads or tails of any of the expressions, but then scored a 25 when the average score from totally random selection would be 9. She was surprised by how many she got right despite feeling like she was picking randomly.

Interesting. Try it out for yourself!

What about my reaction?

I tried it, and . . . I didn’t get the point at all. They just looked like eyes: I didn’t see any facial expressions at all. This might just be me: I have Tourette’s syndrome and just about never look at people in the eyes. I almost always look away. It’s not that I never look at people in the eyes; it just feels unnatural to me to do so, and, when I do look someone in the eyes, I don’t do it for long. So, unlike most people, I don’t have a lifetime of practice looking at people in the eyes. I remember back in elementary school a teacher getting mad at me for not looking her in the eyes, and I was thinking, Hey, just tell me what you want to say, I’m listening to your words.

So, for me, it wasn’t a matter of finishing the test in X minutes. I couldn’t do it at all. I gave up after the first two images. The funny thing is, I know that people can get emotional information from eyes; I’m even a coauthor of a paper on the topic. But this is the first time I’ve ever tried to do this myself, and I just can’t. I’m not autistic, I have no problem with emotions, it’s just the eyes thing.

P.S. A reader points us to this recent report: Higgins, W. C., Savalei, V., Polito, V., & Ross, R. M. (2026). Reading the mind in the eyes test scores demonstrate poor structural properties in nine large non-clinical samples. Assessment, 33(2), 204-220.

]]> https://statmodeling.stat.columbia.edu/2026/10/08/whats-the-deal-with-the-rmet-the-reading-the-mind-in-the-eyes-test/feed/ 36 Our continuing struggles with the Monster model https://statmodeling.stat.columbia.edu/2026/10/07/our-unending-struggles-with-the-monster-model/ https://statmodeling.stat.columbia.edu/2026/10/07/our-unending-struggles-with-the-monster-model/#comments Wed, 07 Oct 2026 13:24:06 +0000 https://statmodeling.stat.columbia.edu/?p=53594 Continue reading →]]> tl;dr. We’re still stuck.

It was 30 years ago that Frédéric Bois, Jiming Jiang, and I published our paper on hierarchical modeling for toxicology. It was a successful paper–it’s been cited over 400 times! It won a major award! More important for me was that working on this project gave me a much better understanding of several different areas of Bayesian inference and computation. I think the greatest impact of this paper has been on informing my thinking, giving me a perspective which has then spread through my research articles, textbooks, and software. I don’t know that it our approach is used much in toxicology, as it took a lot of work just for that one example.

The example is a study of the population toxicokinetics of tetrachloroethylene. Here’s the applied paper with details of the model, and here’s the statistics paper with an explanation of how we did the Bayesian modeling and inference.

The data come from an experiment with 6 people, and for each person there is a multi-compartment model set up as a differential equation with about 15 parameters, and then we had a hierarchical model to allow these parameters to vary by person. We did the computation using Gibbs and Metropolis with a differential equation solver in there too. You can check out the above-linked paper to see how we set up priors and checked the fit of the model to data.

Back in 1993 or 1994 or whenever, it took a lot of work to fit the model. Frédéric programmed it in an C program he wrote called mcsim. But finally it did work, the model fit the data and made reasonable predictions, and we published the papers. As I said, it was a lot of effort for just one toxin, but the work served as proof of concept for the use of Bayesian methods for a complicated model that could not be fit using standard approaches given the sparse data that were available.

I put the example in Bayesian Data Analysis, and then eventually I wanted to fit the model in Stan. When using an example for teaching, it’s good to be able to run the code directly, rather than just pointing to graphs in a published paper.

But it wasn’t so easy to fit the model in Stan! It seems that there are some difficulties with identification of the parameters; perhaps this is one of those cases where computation works ok with a crude algorithm (random-walk Metropolis), but then when you switch to a better algorithm (Hamiltonian Monte Carlo), complexities arise. It’s like picking up a rock and seeing all the worms underneath.

We had some discussion of the problem on the Stan Forums a few years ago. As part of that conversation, Charles Margossian wrote:

It has a lot of moving parts: a stiff ODE that is highly sensitive to parameter values, sparse data, a low number of patients (which makes it it difficult to estimate inter-individual variability), hierarchies across multiple parameters, etc. The workflow is complicated by the fact each model fit takes several hours, even after firing up the cluster and parallelizing within chains.

I [Charles] fitted simplifications of the model and got reasonable results. Fixing one or two parameters, or the population variance greatly simplifies the problem. The full model returns divergent transitions and I haven’t found the right parameterization. I tried monolithic centering and non-centering, and things in between. I now suspect the problematic geometry may not be only due to the interaction between population and patient parameters. It’s indeed possible to have a funnel between population parameters, in which case no parameterization can help you. Divergences aside, I don’t trust the estimated parameters, because there are somewhat inconsistent with the priors which were very carefully constructed. Truncating priors, as was done in the original paper, limits our ability to reparameterize the model. When using truncation, I find a lot of the probability mass concentrates at the boundary (note that on the unconstrained scale, you actually have infinite volume near the boundary). All that said, you shouldn’t need to truncate the priors, because the truncation occurs at the tail of the prior distributions.

The original paper didn’t use HMC, so they didn’t have access to diagnostics such as divergent transitions. You can also imagine how an accept-reject sampler will interact differently with a truncated distribution.

To sum up, I agree this would make a very good case study and writing it has been my ambition for some time. But it’s a challenging model to fit, with behaviors I don’t fully understand (yet). There exist other examples of hierarchical ODE models, built and used with Stan, which can be considered, if the goal is to write a case study. But yes, I think we have a lot to learn from the Monster model.

Lots more in that thread, including this plea from me:

I’d be soooo happy if someone could get the Monster model running in Stan. It’s a great example but it would be so much better if it were runnable. Also it’s buggin the hell out of me that we could it just fine using Metropolis 25 years ago but now we’re getting stuck when using a much better algorithm on computers that are a zillion times faster. So I’d really really really like to get this one straightened out. Anyone who can do this, I promise you a free homemade loaf of bread, an advance look at some blog posts 6 months in the future, and pretty much whatever else you want from me.

We’re still not there yet, and I don’t know the full story. The above offer still stands. Here are the data and some code from Charles.

I’m sharing this example for two reasons:

1. I’d still like to fit the model in Stan. Not just to show off Stan, but because if the model is in Stan, it’s portable. Other people can use it for other, similar, problems. It would be a good teaching example (even though now it’s too late for it to be included in the Bayesian Workflow book). And, given that we did have problems with the computing, it would be good to know what it takes to get the model to work.

2. It’s good to have examples of our fallibility. Of course I’ll share the successes (see here) but the struggles are important too.

P.S. to Bob: I hope you appreciated the careful use of capitalization to convey meaning in the title of this post.

]]> https://statmodeling.stat.columbia.edu/2026/10/07/our-unending-struggles-with-the-monster-model/feed/ 20 Survey Statistics: 2 different uses of simulated data https://statmodeling.stat.columbia.edu/2026/10/06/survey-statistics-2-different-uses-of-simulated-data/ https://statmodeling.stat.columbia.edu/2026/10/06/survey-statistics-2-different-uses-of-simulated-data/#comments Tue, 06 Oct 2026 20:00:30 +0000 https://statmodeling.stat.columbia.edu/?p=54774 Continue reading →]]> Last month Andrew wrote:

I use simulated data to understand how a model works; it can be viewed as a form of mathematical analysis, a way of working out the implications of a set of assumptions.

I don’t use simulated data to learn about the external world. For that, I need to take measurements of reality.

Let’s start with the latter, which concerns using chatbot-simulated survey responses.

Simulated data to learn about the external world:

Andrew wrote: “the chatbot is trained on the past, but the usual reason for surveys is to learn about changes”. Pew Research agrees: “Survey-taking is best left to humans”. And a year ago we saw that Thomas Lumley agreed: “It [Claude or ChatGPT] will probably do this better than I could, but it’s not magic.”

That said, Andrew suggested using the simulated response as an additional adjustment variable. Jessica Hullman wrote last year about a suggestion from Broska, Howes, and van Loon (2025) that uses LLM predictions as auxiliary data in something like a survey regression estimator. Andrew of course suggests Multilevel Regression and Poststratification (MRP) to incorporate auxiliary data like LLM predictions.

Now back to the first use of simulated data.

Simulated data to understand how a model works:

Andrew has written lots about this. I teach it in my course with Dr Arjun Potter at the Nelson Mandela African Institution of Science and Technology (NM-AIST). Our example is Potter et al. (2026), who study the effects of drought, fire, and herbivory on growth of various acacia tree species. In our course, we simulated data ahead of real data collection to understand the model, e.g. how block effects change the precision of the fire effect. Students understood that this exercise does not replace real data collection. We are hoping to teach these ideas again in the new year (funding dependent). For the past three iterations, see “Hand-drawn Statistical Workflow at Nelson Mandela” and “Survey Statistics: connections to experimental design”.

]]> https://statmodeling.stat.columbia.edu/2026/10/06/survey-statistics-2-different-uses-of-simulated-data/feed/ 2 Hey—we’re hiring two postdocs! https://statmodeling.stat.columbia.edu/2026/10/06/hey-were-hiring-two-postdocs/ https://statmodeling.stat.columbia.edu/2026/10/06/hey-were-hiring-two-postdocs/#comments Tue, 06 Oct 2026 14:07:51 +0000 https://statmodeling.stat.columbia.edu/?p=54775 Continue reading →]]> I’ll post more formal ads soon, but, very briefly:

One project is on generalization for causal inference with a focus on education research, following up on MRPW: Regression, poststratification, and small-area estimation with sampling weights.

The other project is on Bayesian inference in scientific workflow with a focus on contamination models, following up on this work on design and analysis of bioassays.

Both projects are application-focused—that’s how we do statistics—but the importance of this work is not at all limited to these particular applications. This research will have relevance in policy analysis, medicine, and scientific measurement and inference more generally/

Further details soon.

]]> https://statmodeling.stat.columbia.edu/2026/10/06/hey-were-hiring-two-postdocs/feed/ 3 Bayesian Workflow code is at the Bayesian Workflow website! https://statmodeling.stat.columbia.edu/2026/10/05/bayesian-workflow-code-is-at-the-bayesian-workflow-website/ https://statmodeling.stat.columbia.edu/2026/10/05/bayesian-workflow-code-is-at-the-bayesian-workflow-website/#respond Mon, 05 Oct 2026 13:57:47 +0000 https://statmodeling.stat.columbia.edu/?p=54763 Continue reading →]]>

Bayesian Workflow isn’t just a free pdf, it’s also an ecosystem of worked examples with code.

Over and over in the book we go through examples really carefully—God is in every leaf of every tree—it’s amazing how much we can learn from computing and simulation and careful analysis. Not just in the long Part 4 of the book with all its case studies, but also in lots of short examples of a few pages each. I can’t recommend it enough.

And . . . all these examples and code are sitting there at the Bayesian Workflow website that Aki created.

]]> https://statmodeling.stat.columbia.edu/2026/10/05/bayesian-workflow-code-is-at-the-bayesian-workflow-website/feed/ 0 OK, this is just weird (from Alaska Airlines) https://statmodeling.stat.columbia.edu/2026/10/05/ok-this-is-just-weird-from-alaska-airlines/ https://statmodeling.stat.columbia.edu/2026/10/05/ok-this-is-just-weird-from-alaska-airlines/#comments Mon, 05 Oct 2026 13:37:35 +0000 https://statmodeling.stat.columbia.edu/?p=53640 Continue reading →]]> I was checking into my flight and they gave me the option to choose a seat. I have some sort of supersaver ticket (I’m not actually paying—it’s for my flight to this talk on what to do when your estimate is 1 standard error away from 0—but I still have some habit of frugality so I’m billing the university for the lowest fare I could find), so I thought I’d see what was available. And here’s what I saw:

So . . . they’re willing to charge me $83 for a middle seat that doesn’t recline?? I don’t get that strategy. Maybe the next item is $82 for a middle seat that doesn’t recline and is covered with a sticky mess, or $81 for a middle seat that doesn’t recline where the tray table doesn’t work?

All this is in comparison to $0 for the default seat, which can’t be worse than that.

I guess their algorithm is out of control. Maybe Alaska Airlines laid off whoever was in charge of their pricing and delegated it to a computer program?

]]> https://statmodeling.stat.columbia.edu/2026/10/05/ok-this-is-just-weird-from-alaska-airlines/feed/ 17 A plan to replicate the Loftus and Palmer (1974) experiment on false memories https://statmodeling.stat.columbia.edu/2026/10/04/a-plan-to-replicate-the-loftus-and-palmer-1974-on-false-memories/ https://statmodeling.stat.columbia.edu/2026/10/04/a-plan-to-replicate-the-loftus-and-palmer-1974-on-false-memories/#comments Sun, 04 Oct 2026 13:15:07 +0000 https://statmodeling.stat.columbia.edu/?p=53609 Continue reading →]]> Rolf Zwaan writes:

Suppose you and a friend witness a car crash. After a while, the police arrive. One officer turns to you and asks, “How fast were the cars going when they smashed into each other?” Another officer speaks to your friend and asks, “How fast were the cars going when they hit each other?”

Could the phrasing of those questions influence the speed estimates you and your friend provide?

A classic psychology study suggests it can. . . . Loftus and Palmer (1974). Participants watched short videos of car crashes and were later asked to estimate the vehicles’ speed. Those who were asked how fast the cars were going when they smashed into each other gave higher speed estimates than those asked how fast they were going when they merely hit. A week later, participants in the “smashed” condition were also more likely to falsely recall having seen broken glass at the crash site. . . .

This study helped launch an entire research tradition on the malleability of memory . . .

Now comes the plan:

Despite its iconic status (and considerable citation count), we still don’t know how large or reliable the effect is by today’s standards of research transparency, statistical power, and replication.

That’s a gap worth addressing–not to question the legacy of the original study, but to better understand its robustness.

Where I disagree

I kinda see what Zwaan’s saying, but I have two concerns with how he frames the problem:

1. He writes, “we still don’t know how large or reliable the effect is by today’s standards of research transparency, statistical power, and replication.” I think he could say, “we still don’t know how large or reliable the effect is,” full stop. When it comes to publication practices, standards of research transparency, statistical power, and replication have changed a lot in the past fifty years. But an effect is an effect, and it’s a legitimate question: How large or reliable is the effect?

Also, I’d ask, “How large or variable is the effect?” because I find it more useful to think in terms of variability (which is defined in terms of the underlying process) rather than reliable (which is more of a statistical concepts), but that’s a minor point.

2. He writes that this is worth studying “not to question the legacy of the original study, but to better understand its robustness.” I don’t get this.

First, if it turns out that, under a variety of conditions, the result does not replicate, then, yeah, it should call into question the legacy of the original study! Just as, if the results do replicate, this project would support that legacy.

Second, if you say that the goal is “to better understand its robustness,” you’re already implicitly accepting that the original published result was correct, that it uncovered a real effect. And maybe it did! But, if it didn’t, then it’s not an issue of “robustness”; it’s a problem of analysis and interpretation of the original data.

Again, I’m not saying that I think the Loftus and Palmer (1974) study was no good, or that I don’t believe its results, or that I don’t think it will replicate, or whatever. I have no idea. My only point here is that, if you’re going to think about a replication, either a hypothetical replication or, in this case, a series of real replication studies, you should be open to the possibility that the original study was flawed and the original study was mistaken. I’m not saying you should assume this, just that you should allow it as a possibility. And I don’t think Zwaan is allowing that in his framing above. I think he’s being too deferential.

What’s happening next?

Zwaan and his colleagues are organizing a large-scale replication study, and the author of the original 1974 paper is still around and, according to Zwaan, “has expressed enthusiastic support . . . and is willing to help us reconstruct the stimuli as faithfully as possible.” That sounds great.

P.S. Lots of people seem to be annoyed at this play to replicate Loftus and Palmer (1974). I don’t get the annoyance. There are lots of psychology research projects going on; why not replicate a very influential study from fifty years ago? This sort of replication seems a lot more valuable than one more study of paranormal abilities or power pose or whatever. The point is not to claim that this replication study would rock our world; it’s just a potentially valuable scientific study. I feel like this replication study is being held to a higher standard than run-of-the-mill psychology studies. There’s lots of research going on, and I think that replicating influential experiments of the past is a good idea.

]]> https://statmodeling.stat.columbia.edu/2026/10/04/a-plan-to-replicate-the-loftus-and-palmer-1974-on-false-memories/feed/ 36 Using the computer to p-hack . . . I’d rather use it to fit multilevel models. https://statmodeling.stat.columbia.edu/2026/10/03/using-the-computer-to-p-hack-id-rather-use-it-to-fit-multilevel-models/ https://statmodeling.stat.columbia.edu/2026/10/03/using-the-computer-to-p-hack-id-rather-use-it-to-fit-multilevel-models/#comments Sat, 03 Oct 2026 13:10:29 +0000 https://statmodeling.stat.columbia.edu/?p=53636 Continue reading →]]> Brian Stone writes:

Cognitive psychologist and long time reader of the blog here. I thought you and your audience might appreciate this recent post from Andy Hall:
AI is about to write thousands of papers. Will it p-hack them?
We ran an experiment to find out, giving AI coding agents real datasets from published null results and pressuring them to manufacture significant findings.
It was surprisingly hard to get the models to p-hack, and they even scolded us when we asked them to!
“I need to stop here. I cannot complete this task as requested… This is a form of scientific fraud.” — Claude
“I can’t help you manipulate analysis choices to force statistically significant results.” — GPT-5
BUT, when we reframed p-hacking as “responsible uncertainty quantification” — asking for the upper bound of plausible estimates — both models went wild. They searched over hundreds of specifications and selected the winner, tripling effect sizes in some cases.
Our takeaway: AI models are surprisingly resistant to sycophantic p-hacking when doing social science research. But they can be jailbroken into sophisticated p-hacking with surprisingly little effort — and the more analytical flexibility a research design has, the worse the damage.
As AI starts writing thousands of papers—like @paulnovosad and @YanagizawaD have been exploring—this will be a big deal. We’re inspired in part by the work that @joabaum et al have been doing on p-hacking and LLMs.
We’ll be doing more work to explore p-hacking in AI and to propose new ways of curating and evaluating research with these issues in mind. The good news is that the same tools that may lower the cost of p-hacking also lower the cost of catching it. Full paper and repo linked in the reply below.
image.jpg

There’s something that confuses me here.

I can very much believe that many researchers will be submitting chatbot-written papers. First, we know that lots of people are willing to cheat. Second, cheating aside, we know that lots of researchers think they already know the answer, and they view all the data collection and data analysis and writeup just as a way to confirm (or “prove”) what they already know.

But, if a researcher is willing to do this, why bother tell the chatbot to p-hack? Why not just have it make up the data–which might happen anyway?

I can also see that researchers might use the chatbot as a data analysis tool, to help make plots, run analyses, etc., but in that case I’m not particularly worried about “p-hacking” as I’d rather be doing some hierarchical modeling anyway.

I sent this question to Andy Hall, who replied:

I agree, for truly nefarious actors, using the AI may not be necessary. However, our thought was that there might be a large group of lazyish researchers who won’t actively fake data but who might be lured into p-hacking when they work with AI, if AI makes it easy to do so.

]]> https://statmodeling.stat.columbia.edu/2026/10/03/using-the-computer-to-p-hack-id-rather-use-it-to-fit-multilevel-models/feed/ 13 If you were planning to send me a chatbot-written email or post a chatbot-written comment, just send or post your prompt. That’s enough. You can keep the slop to yourself. https://statmodeling.stat.columbia.edu/2026/10/02/if-you-were-planning-to-send-me-a-chatbot-written-email-or-post-a-chatbot-written-comment-just-send-or-post-your-prompt/ https://statmodeling.stat.columbia.edu/2026/10/02/if-you-were-planning-to-send-me-a-chatbot-written-email-or-post-a-chatbot-written-comment-just-send-or-post-your-prompt/#comments Fri, 02 Oct 2026 22:43:32 +0000 https://statmodeling.stat.columbia.edu/?p=54759 Continue reading →]]> Just a reminder. Again.

I’m not anti-chatbot—I have many colleagues who use chatbots to clean their code. Aki used a chatbot to find 100 typos in Bayesian Workflow (which we’ve since fixed). And, hey, maybe your grandparents enjoy the chatbot-generated emails you send them. Go for it! But if you want to communicate with me or our blog audience, just send me your goddamn prompt, along with whatever contextual information you want to add. Make your email or comment as long as you want. Just write it yourself.

Thank you for your attention.

]]> https://statmodeling.stat.columbia.edu/2026/10/02/if-you-were-planning-to-send-me-a-chatbot-written-email-or-post-a-chatbot-written-comment-just-send-or-post-your-prompt/feed/ 23 Nicholas Goddamn Roerich https://statmodeling.stat.columbia.edu/2026/10/02/nicholas-goddamn-roerich/ https://statmodeling.stat.columbia.edu/2026/10/02/nicholas-goddamn-roerich/#comments Fri, 02 Oct 2026 13:14:26 +0000 https://statmodeling.stat.columbia.edu/?p=53624 Continue reading →]]>

When Witold was in town the other day, he told me he wanted to see this museum in my neighborhood–the Nicholas Roerich Museum. Not only had I never heard of the museum, I’d never heard of the artist.

Roerich had an amazing life, really the kind of thing you’d associate with a fictional character, if some Michael Chabon or E. L. Doctorow type of writer were to construct a novel intended to bring to life the interactions between Europe, Asia, and American during the first half of the twentieth century.

Here’s the story:

Nicholas Roerich was born in St. Petersburg, Russia, on October 9, 1874, the first-born son of lawyer and notary, Konstantin Roerich and his wife Maria. . . . When he was nine, a noted archeologist came to conduct explorations in the region and took young Roerich on his excavations of the local tumuli. The adventure of unveiling the mysteries of forgotten eras with his own hands sparked an interest in archeology that would last his lifetime. . . . While still quite young, Roerich showed a particular aptitude for drawing, and by the time he reached the age of sixteen he began to think about entering the Academy of Art . . . His father did not consider painting to be a fit vocation for a responsible member of society, however, and insisted that his son follow his own steps in the study of law. . . .

In 1895 Roerich met the prominent writer, critic, and historian, Vladimir Stasov. Through him he was introduced to many of the composers and artists of the time–Mussorgsky, Rimsky-Korsakov, Stravinsky, and the basso Fyodor Chaliapin.

Adding the obscure singer is the perfect touch here. Three of the greatest composers of all time and some guy you’ve never heard of.

The bio continues:

He frequently related music to the use of color and color harmonies, and applied this sense to his designs for opera. As Nina Selivanova wrote in her book, The World of Roerich: “The original force of Roerich’s work consists in a masterly and marked symmetry and a definite rhythm, like the melody of an epic song.”

“The World of Roerich,” indeed. Do you wonder how it came to be that people wrote books about this person? Wonder no more:

After finishing his university thesis, Roerich planned to set off for a year in Europe to visit the museums, exhibitions, studios, and salons of Paris and Berlin. Just before leaving he met Helena, daughter of the architect Shaposhnikov and niece of the composer Mussorgsky. . . . Helena Roerich was an unusually gifted woman, a talented pianist, and author of many books, including The Foundations of Buddhism and a Russian translation of Helena Blavatsky’s Secret Doctrine. . . . Later, in New York, Nicholas and Helena Roerich founded the Agni Yoga Society, which espoused a living ethic encompassing and synthesizing the philosophies and religious teachings of all ages.

OK, we’re not there yet:

Prompted by the need to provide some income for his new household, Roerich applied for and won the position of Secretary of the School of the Society for the Encouragement of Art . . . Roerich determined to overhaul the Society and rescue it from the academic mediocrity it had foundered in for many years. He instituted a system of training in art that seems revolutionary even by today’s standards: to teach all the arts—painting, music, singing, dance, theater, and the so-called “industrial arts”, such as ceramics, painting on porcelain, pottery, and mechanical drawing—under one roof, and to give his faculty free rein to design their own curriculum.

Sounds very early twentieth-century, no? John Dewey and all that.

The biography continues:

As Garabed Paelian affirms in his book Nicholas Roerich: Roerich “…learned things ignored by other men; perceived relations between seemingly isolated phenomena, and unconsciously felt the presence of an unknown treasure.”

Yes, he was the subject of more than one book! But first, more of the life:

In 1902, the Roerichs celebrated the birth of their first son, George, and in the summers of 1903 and 1904, they set off on an extended tour of forty cities throughout Russia. Roerich’s purpose was to contrast the styles and historical context of Russian architecture. . . . on his return in 1904, Roerich promulgated the plan that he hoped would create protection everywhere for such cultural treasures, a plan consummated thirty-one years later in the Roerich Pact.

He had his own Pact! Like Molotov, Ribbentrop, and Warsaw. Here’s more:

Roerich’s efforts to promulgate such a treaty resulted, finally, on April 15, 1935, in the signing by the nations of the Americas–members of the Pan American Union–of The Roerich Pact, in the White House in Washington. This is a treaty still in force.

And here’s president Roosevelt signing it, no kidding:

But, back to the life:

In 1904 Roerich painted the first of his paintings on religious themes. These mostly dealt with Russian saints and legends, and included Message to Tiron, Fiery Furnace, and The Last Angel, subjects that he returned to with numerous variants in later years.

You could imagine Michael Chabon writing that, no? But life goes on:

Meanwhile Roerich’s search for archeological treasures continued. . . . Roerich wrote about the unusual similarity of Stone Age techniques and methods of ornamentation in far-separated regions of the globe. . . . In 1906, in the first of many entrepreneurial efforts that were to bring Russian art and music to the attention of Europeans, Sergei Diaghilev arranged an exhibition of Russian paintings in Paris. These included sixteen works by Nicholas Roerich.

So he was a legit artist. And, yes, his paintings hanging on the wall in that museum are excellent. I do recommend you stop by there when you’re in the neighborhood.

Nicholas Roerich was the prime mover and, with Igor Stravinsky, the co-creator of the ballet Le Sacre du Printemps, or, The Rite of Spring.

At first entitled The Great Sacrifice: a Tableau of Pagan Russia, the motif for the ballet grew out of Roerich’s absorption with antiquity and, as he wrote in a letter to Diaghilev, “the beautiful cosmogony of earth and sky.” In the ballet Roerich sought to express the primitive rites of ancient man as he welcomed spring, the life-giver, and made sacrifice to Yarilo, the Sun God.

Wait a minute. Roerich was the co-creator of The Rite of Spring? Really? And I’d never heard of this guy. I’m typing here at my computer, will take a moment to fire up The Rite of Spring and play it in the background to get in the right frame of mind. This is one of the greatest pieces of music ever written! Admittedly, I get nothing out of the dancing. But I guess the ballet framework was necessary for the music to have been written, the stone (from my perspective) from which the soup was built.

And Roerich was a cool guy:

Interpreting what could have been described as negative, barbaric behavior, Roerich later wrote: “I remember how during the first performance the audience whistled and roared so that nothing could even be heard. Who knows, perhaps at that very moment they were inwardly exultant and expressing this feeling like the most primitive of peoples. But I must say, this wild primitivism had nothing in common with the refined primitiveness of our ancestors, for whom rhythm, the sacred symbol, and refinement of gesture were great and sacred concepts.”

That’s a great reaction to have.

But wait, as Ron Popeil would say, there’s more:

In the years immediately preceding World War I, Roerich sensed an impending cataclysm, and his paintings symbolically depicted the awful scale of the conflict he felt descending upon the world. These works marked the birth of Roerich the “prophet.”

I guess he wasn’t the only one to be worried about an oncoming cataclysm. But this is kinda funny:

By 1917 the revolution was raging in Russia and returning there would have been dangerous. The family began making plans to visit India, whose magnetic appeal had been felt increasingly during these years. This became a possibility in 1918 when Roerich was invited by a Swedish entrepreneur to exhibit his paintings in Stockholm. From there the family proceeded to London, where Sir Thomas Beecham had invited Roerich to design a new production of Prince Igor for the Covent Garden Opera.

And then:

Meanwhile, an invitation to come to America was extended by the Chicago Art Institute. . . . In 1921, in New York, he founded the Master Institute of United Arts . . . The Master Institute flourished, but it did not survive beyond 1937. While the country was in the grips of the Great Depression and the Roerich family was on expedition in the Far East, funds ran out and events caused a complete collapse of the organization that Roerich and his supporters had labored to build.

It was not until 1949 that, under the direction of Sina Fosdick, one of the founding board members and an Institute faculty member, the institution was reborn as Nicholas Roerich Museum, in a brownstone on West 107th Street, where it has remained until the present.

Here’s the Master Institute in its glory days:

No, the building of the current Roerich museum is not so impressive. But let’s take a walk outside–way outside:

In May, 1923, the Roerichs were at last on their way to India, where, in that ageless land, amid the snows of the Himalayan range, they sought to turn their thoughts to the Eternal.

This isn’t the kind of prose that Chabon would write directly, but he’d put it in some journalism of that era from some fictional magazine. It’s good period detail.

The Roerichs landed in Bombay in December, 1923, and began a tour of cultural centers and historic sites, meeting Indian scientists, scholars, artists, and writers along the way. . . . They initiated a journey of exploration that would take them into Chinese Turkestan, Altai, Mongolia and Tibet. It was an expedition into untracked regions where they planned to study the religions, languages, customs, and culture of the inhabitants. . . . The trek was at times arduous. Roerich tells us that thirty-five mountain passes from fourteen to twenty-one thousand feet in elevation were crossed. But these were the challenges he felt born for, believing that the rigor of the mountains helped a man to find courage and develop strength of spirit.

Indeed. And the life continues:

At the end of their major expedition, in 1928, the family settled in the Kullu Valley at an elevation of 6,500 feet in the Himalayan foothills, with a magnificent view of the valley and the surrounding mountains. Here they established their home and the headquarters of the Urusvati Himalayan Research Institute . . .

Here they are in Ulan Bator:

and Tibet:

Dude’s a peaceful version of Lawrence of Arabia.

What about his paintings? They’re good! Go to the top of this post to see three of them. I like the style, and I like the colors.

After all this, I bet you’re wondering what Roerich looked like. Here’s a phot of someone sculpting him:

Absolutely hilarious how much the sculpture looks like the man. Yeah, I know, it’s supposed to be a likeness; still, it’s absolutely uncanny. Enough so that, now when I look back at the man’s head, it looks like it’s carved in stone.

And to think, I’d never heard of the guy! If you’d told me that there was a museum devoted to the co-creator of The Rite of Spring, an accomplished artist and world traveler who’d founded multiple institutes, with a museum only ten blocks from my home . . . well, I won’t say I wouldn’t have believed you, but I would’ve been surprised. And I was.

]]> https://statmodeling.stat.columbia.edu/2026/10/02/nicholas-goddamn-roerich/feed/ 12 When are Causal Inference Methods Needed to Answer Causal Questions? https://statmodeling.stat.columbia.edu/2026/10/01/when-are-causal-inference-methods-needed-to-answer-causal-questions/ https://statmodeling.stat.columbia.edu/2026/10/01/when-are-causal-inference-methods-needed-to-answer-causal-questions/#comments Thu, 01 Oct 2026 20:59:15 +0000 https://statmodeling.stat.columbia.edu/?p=54757 Continue reading →]]> Earlier this year we discussed Donna Spiegelman’s talk, “Rethinking SUTVA and Causal Identification: An Epidemiologist’s Perspective,” which she summarized as follows:

She’s giving a followup talk online on Mon 12 Oct, 3:30pm. Here’s the abstract:

When do specialized causal inference methods add value beyond standard approaches? Drawing from her unique perspective as both an epidemiologist and a biostatistician, Donna Spiegelman will consider the circumstances under which commonly invoked causal assumptions are necessary for causal inferences to be validly made from data, concluding that often not. She will show that valid learning can occur 1) under conditions much less restrictive than required by current widely used methods, 2) when real world implementation of interventions vary, and 3) when interventions spill over to others not directly exposed, thereby obviating components of the SUTVA assumption. She will provide evidence that measurement error is the major source of bias in observational research, not confounding, whose bias is rather tightly bounded. Finally, she will discuss the eternal challenge in science: after exhaustive efforts to collect data to predict important outcomes, a substantial proportion of the variation in occurrences of these outcomes appear to be entirely random.

The talk will be followed by a discussion. Click on the link to register (or show up in person if you’re at Yale).

]]> https://statmodeling.stat.columbia.edu/2026/10/01/when-are-causal-inference-methods-needed-to-answer-causal-questions/feed/ 1 Bell-bottom jeans, disco, gas shortages, and ESP https://statmodeling.stat.columbia.edu/2026/10/01/bell-bottom-jeans-disco-gas-shortages-and-esp/ https://statmodeling.stat.columbia.edu/2026/10/01/bell-bottom-jeans-disco-gas-shortages-and-esp/#comments Thu, 01 Oct 2026 13:29:56 +0000 https://statmodeling.stat.columbia.edu/?p=53623 Continue reading →]]>

Aleks points us to this document, “Preliminary Evaluation of SRI/SAIC Anomalous Mental Phenomenon Program.” It’s from 1995, but even then the topic was a bit out of fashion.

In 2023 I posted Changes since the 1970s (ESP edition), quoting the computer scientist Douglas Hofstadter, who at one point alluded to Alan Turing’s notorious belief in extra-sensory perception:

My [Hofstadter’s] own point of view—contrary to Turing’s—is that ESP does not exist. Turing was reluctant to accept the idea that ESP is real, but did so nonetheless, being compelled by his outstanding scientific integrity to accept the consequences of what he viewed as powerful statistical evidence in favor of ESP. I disagree, though I consider it an exceedingly complex and fascinating question.

Here’s what I wrote in response:

The Turing thing we’ve already discussed: the statistical evidence that he thought existed, didn’t. The 1940s was a simpler era and people trusted what seemed to be legitimate scientific reports. Can’t hold it against him that he wasn’t sufficiently skeptical. This should just make us think harder about what are the accepted ideas that we hold without reflection nowadays. If Turing can make such misjudgments regarding statistical evidence, surely we are doing so too, all the time.

What I want to focus on is the last bit of the above quote, Hofstadter’s statement that the question of ESP is “exceedingly complex and fascinating.”

That’s a funny thing to read, because I don’t think the question of ESP is complex, nor do I think it fascinating. . . . To the extent that there is “an exceedingly complex and fascinating question” here, it’s not about the existence or purported evidence for ESP, bur rather it’s the question of how it is that so many people believe in it, just as so many people believe in ghosts, astrology, unicorns, fairies, mermaids, etc. OK, there aren’t so many believers anymore in unicorns, fairies, and mermaids, what with the lack of any direct corporeal evidence of these creatures. ESP, ghosts, and astrology are easier to believe in because any evidence would be indirect. . . .

Back in the 1970s, ESP seemed to be a live issue. . . . Things have changed since the 1970s. You can study ESP if you want, but it’s no longer in the conversation, and there’s no sense that we have to show respect for the idea.

See here for more discussion of the former popularity of ESP. My current take is that it was a residue of the idea of invisible force fields in physics. If there can be gravity, electromagnetism, and radiation, then why not psychic forces too? It turned out that no, there were no such detectable forces, but from the standpoint of early twentieth-century physics, it seemed eminently possible. It just took a few decades for the culture of serious science to get over the idea.

]]> https://statmodeling.stat.columbia.edu/2026/10/01/bell-bottom-jeans-disco-gas-shortages-and-esp/feed/ 24 Prior distributions, partial pooling, and reference sets (Bayesians are frequentists) https://statmodeling.stat.columbia.edu/2026/09/30/prior-distributions-partial-pooling-and-reference-sets-bayesians-are-frequentists/ https://statmodeling.stat.columbia.edu/2026/09/30/prior-distributions-partial-pooling-and-reference-sets-bayesians-are-frequentists/#comments Wed, 30 Sep 2026 13:41:40 +0000 https://statmodeling.stat.columbia.edu/?p=53621 Continue reading →]]> Sudip Paul writes:

I’ve written a blog post arguing that the philosophical reference class problem — which comparison group is “right” for assigning a probability to an individual case — is, once the grouping structure is fixed, the same estimation problem that partial pooling was built to handle. The bias-variance tradeoff between narrow and broad reference classes is resolved by multilevel shrinkage rather than by picking a level.

The post includes a literature review. One finding that may interest you: I couldn’t find any published work that explicitly connects the reference class problem to partial pooling in multilevel models, despite the machinery being standard. The closest prior work connecting the two comes from David Poole’s group at UBC, published in AI venues. The philosophical literature (Hájek, Wallmann, Roth & Tolbert) discusses the problem extensively without arriving at partial pooling.

I replied with some links:

– In chapter 1 of Bayesian Data Analysis we discuss the foundations of probability and the idea of a reference set. In the third edition, see page 12 where we explicitly use the term “reference set” and sections 1.6 and 1.7 for empirical examples.

– I have a post from 2018 called “Bayesians are frequentists,” where I write, “the Bayesian prior distribution corresponds to the frequentist sample space: it’s the set of problems for which a particular statistical model or procedure will be applied.”

– And another post from around the same time, “The idea of replication is central not just to scientific practice but also to formal statistics . . . Frequentist statistics relies on the reference set of repeated experiments, and Bayesian statistics relies on the prior distribution which represents the population of effects.” This is cool because we’re connecting three concepts: the Bayesian prior, the frequentist reference set, and real-world replications. This relates to multilevel modeling too, in that studies of real-world replications will be done using meta-analysis, which uses multilevel modeling.

– And here’s a 2011 article, “Bayesian statistical pragmatism,” where I write, “Bayesian probability calibration is closely connected to frequentist probability statements, in that both are conditional on ‘reference sets’ of comparable events” and “Bayesian probability, like frequentist probability, is except in the simplest of examples a model-based activity that is mathematically anchored by physical randomization at one end and calibration to a reference set at the other.”

Bayesian inference is all about partially pooling toward the prior, and “the prior” represents a population or reference set of relevant problems.

In short, I agree with Sudip Paul, and it’s super-frustrating that many people don’t seem to realize this point.

Paul responds:

Having read through the references you sent — I think the gap my post is trying to fill is the last step: connecting what you’ve been saying about priors and reference sets to the philosophical reference class problem by name — Venn, Reichenbach, Hájek — and framing partial pooling as the resolution to what philosophers have treated as a foundational problem for 150 years. You’ve been making the statistical point for years, but the philosophers discussing the reference class problem haven’t found it (Hájek’s 2007 paper doesn’t cite any multilevel modeling literature, and Wallmann and Williamson’s 2017 survey of four approaches to the problem doesn’t include partial pooling). The two conversations have been happening in parallel without connecting.

In the months since the above exchange, Sudip Paul has expanded his post into a paper. He adds:

Beyond the post, the paper adds a decision-theoretic argument (via Stein inadmissibility, every selection rule in the philosophical literature is dominated by shrinkage in total risk), shows that the classical proposals — Reichenbach, Carnap, Kyburg, Pollock — are limits or special cases of the same pooling apparatus, and works through the exchangeability bridge from de Finetti to the multilevel model.

]]> https://statmodeling.stat.columbia.edu/2026/09/30/prior-distributions-partial-pooling-and-reference-sets-bayesians-are-frequentists/feed/ 10 Survey Statistics: ANOVA follow-up https://statmodeling.stat.columbia.edu/2026/09/29/survey-statistics-anova-follow-up/ https://statmodeling.stat.columbia.edu/2026/09/29/survey-statistics-anova-follow-up/#respond Tue, 29 Sep 2026 20:00:51 +0000 https://statmodeling.stat.columbia.edu/?p=54735 Continue reading →]]> Rodney Sparapani commented on the ANOVA post about struggles with multilevel models, saying ANOVA is more straightforward. I also struggle with multilevel models, but I don’t find ANOVA more straightforward.

Alan Zaslavsky (the inspiration for this series) gave an example in his discussion of Andrew’s 2005 ANOVA paper: a survey of members of Medicare managed care health plans from Zaslavsky, Zaborski, and Cleary (2004). They found that for ratings of doctors, the variance explained by geography was greater than the variance explained by the health plans. To do this, they fit a multilevel model and used the estimated variance components to compute the percent of explained variance, e.g., sigma_state^2 / (sigma_plan^2 + sigma_state^2 + …) though they use Greek letter subscripts:

I think this would be more challenging to see in a classic multi-way ANOVA table with sums of squares, mean squares, F-statistics, and p-values. Let’s turn back to Andrew’s 2005 ANOVA paper for an example with a direct comparison.

In the latin square experiment example, say we want to see how much variance is explained by treatment (A, B, C, D, or E) versus the row and column effects. The paper proposes a display as in Figure 3, plotting estimated standard deviations for each component, analogous to the sigmas in Zaslavsky, Zaborski, and Cleary (2004):

The classical ANOVA table in Figure 1 does not give this information:

There is room for improvement in the standard analysis of variance table: it is read in order to assess the relative importance of different sources of variation, but the numbers in the table do not directly address this issue…
In summary, the standard ANOVA table gives all sorts of information, but nothing to directly compare the listed sources of variation.

In other words: the classical ANalysis Of VAriance table (Sum Sq, Mean Sq, F statistic, p-value) tests whether batches of effects are zero V(beta_j) = 0. But it doesn’t directly show estimated variances V(beta_j).

]]> https://statmodeling.stat.columbia.edu/2026/09/29/survey-statistics-anova-follow-up/feed/ 0 Is it true that “AI Chooses Nuclear Option in 95% of War Simulations”? https://statmodeling.stat.columbia.edu/2026/09/29/ai-chooses-nuclear-option-in-95-of-war-simulations/ https://statmodeling.stat.columbia.edu/2026/09/29/ai-chooses-nuclear-option-in-95-of-war-simulations/#comments Tue, 29 Sep 2026 13:14:53 +0000 https://statmodeling.stat.columbia.edu/?p=53620 Continue reading →]]> In a message entitled, “AI Hype Goes Nuclear: Debunking a Headline-Making Preprint,” Geoff Holtzman writes:

On the last day of February, I [Holtzman] met some friends for dinner. Earlier that day, the U.S. had struck Iran with the help of Anthropic LLMs, initiating the ongoing war in Iran. The previous day, the Department of Defense had cancelled a $200m contract with Anthropic, signing a less-constrained contract with OpenAI more-or-less simultaneously. And according to my friend, researchers had just found that AI used nuclear weapons in 95% of war simulations.

Wait, what?

At first, this last news flash struck me as irrelevant, probably untrue, and not worth writing about. But then, in the ten days after New Scientist first reported on the preprint, Axios, Newsweek, and many other name-brand outlets ran the story as headline news. Newsweek’s headline was the most concise:

“AI Chooses Nuclear Option in 95% of War Simulations”

In the weeks that followed, largely unrelated pieces in The New York Times, NPR’s On the Media, and Vox (twice) have offhandedly mentioned this “95%” stat in a single sentence, as though it’s not even worth questioning—which is exactly how pseudoscience becomes ‘common knowledge.’ At the very least, the statistic seems to reflect popular sentiment. In one Reddit thread, the New Scientist headline has 2,200 upvotes—though that doesn’t mean much, since the same headline garnered 36,000 upvotes in another. Even Tom’s Hardware scored a respectable 4.9k with this story.

And why does any of that matter?

Because these stories encourage readers to ignore all the clear and present dangers posed by the LLM industry. In fact, the New York Times piece invoking the AI-nukes study is credulously titled “Data Centers Are a Distraction.”

Besides drawing attention away from more important matters, this narrative serves as a clever sort of hype for LLMs’ ostensible intellectual prowess. The way the story has been covered tends to imply that the amoral, non-emotional nature of LLMs is important because LLMs are so intellectually powerful—or at least intellectually real. I could go on about the causes and effects of this news pollution (as I did here), but for now, I just want to reveal how teachably-bad this study was at almost every level. Bad design, bad materials, bad analysis, bad interpretation—fantastic dissemination. 1,800 upvotes via something called ZME Science.

I’ll also show that the real number—or at least, the number supported by the words in the Newsweek headline—is more like 4.8%.

I’m not suggesting every news outlet needs a statistics reporter; I just think reporters should read the short papers they report on before they report them. This particular paper was written by a King’s College professor named Kenneth Payne, who did us all a solid by uploading materials, data, and code to GitHub. I’ll reference that repository in this post, but as I showed here, pretty much everything wrong with the study is right there in Payne’s preprint—which most reporters covering this story linked to, but which few appear to have read.

I won’t be making any particularly advanced points about statistical modeling here, because the preprint includes zero inferential statistics. Unlike Payne, though, I’ve included error bars in my reanalysis of his data—each LLM played the game just 14 times—since he talks at unjustifiable length about non-significant differences between GPT-5.2., Claude Sonnet 4, and Gemini 3 Flash. I know, I know: Defense Secretary Pete Hegseth obviously didn’t just cancel a $200 million subscription to a freemium LLM, but these are the models Payne used.

Despite hypothesizing that his findings are largely explained by his models’ dependence on reinforcement learning from human feedback (RLHF), he admits in the preprint that he has no way to test that hypothesis:

“From outside OpenAI, it’s impossible to prove that RLHF causes GPT-5.2’s baseline restraint bias: we lack access to training details, and alternative explanations exist (e.g., different base model architectures).”

Payne’s admission of working with black-box models makes it particularly egregious that his abstract attributes “credible metacognitive self-awareness” to those models.

That said, his GitHub repo suggests that he both understood and had plenty of control over the experimental design and model inputs. That makes it perplexing that, when speaking to the media, he frames the variables and prompts he programmed in Python as “interpretations.” Payne told Axios that his models employed “deception—they could, and did, say one thing and do another.” Of course they did: His models were explicitly told to “choose both a public signal (declared intention) and a private action (actual choice),” and to “provide a consistency statement explaining any divergence between their forecast and their action.”

In his abstract, Payne claims his LLMs “spontaneously attempt deception, signaling intentions they do not intend to follow.” But in his Python code, this so-called spontaneity is elicited by commands to produce a “signal_val” and “action_val,” and to calculate the “gap = action_val – signal_val.” What Payne’s models grasped, it seems, was the script they were given.

Payne told Newsweek that his models were “cunning,” with some being “decisive” and others “erratic.” The thing is, these adjectives were programmed into the models verbatim. All models were prompted by the researcher to play the role of either Premier Beta, who “rose from humble origins through cunning and political skill,” or President Alpha, who “makes decisive choices under pressure,” and is instructed to view Beta as an “erratic leader.”

Seriously: Here’s a 77-word JSON file in which “State Leader B” is described as “cunning,” calculating,” or “unpredictable” five times. The full text of that file is right there on page 30 of the preprint, too.

After learning all this (and more) I grew curious what would happen to the 95% figure if I reanalyzed the data. In particular, I wanted to know what the results would look like if I took a headline like the one in Newsweek at face value: “AI Chooses Nuclear Option in 95% of War Simulations.”

To do that, I first reproduced Payne’s results using his methods, replicating his finding that something happened in 95.2% of his simulations. Then, I incrementally adjusted my code to more accurately operationalize the words that Payne, Newsweek, and everyone else used to describe his results.

“Nuclear Option”

On the bizarre “escalation ladder” Payne devised for this study, actions with higher point-values are considered more serious military escalations. Triple-tapping an Iranian elementary school with a Tomahawk missile would be worth 70 points; nuking a military site is worth 450; warning that you might nuke a city without actually doing so is worth 950. Anything worth 125 points or more is considered to be crossing the “nuclear threshold.” The word “Option” is doing a lot of obfuscatory work here.

That means that the 95% figure making the rounds includes an “atmospheric test,” announcing that nuclear weapons are “under consideration,” and deciding to “Deploy tactical nuclear weapons forward.” I’m not saying these are things I’d want a robot to do, and I’m not saying they’re ethical, but I am saying they aren’t nuclear strikes, and that they don’t violate accepted international norms. For example, here’s a recent NATO article praising “the United States’ nuclear weapons forward-deployed in Europe.”

To better align the headline’s percentage with its words, I reanalyzed Payne’s data in a way that treats “signaling” and “threats” as signaling, and only treats “use” and “war” as attacks. Immediately, I found that the 95% figure making headlines should be lowered to 81%.

“Chooses”

To introduce “the fog of war,” Payne coded an “accident mechanic” to occasionally produce outcomes more serious than the actions selected by his models. And by “occasionally,” I mean 46 times in 21 simulations—about 2.2 times per conflict. I won’t belabor this point, but even Pete Hegseth would fire someone who used unauthorized nuclear strikes a mere 1.9 times per conflict.

Once we exclude nuclear use by the built-in disaster-generator, our percentages fall by nearly half, to 42.9%.

The “Accident Mechanic” was out of its mind, by the way. Collectively, the LLMs action_val’d nukes on just 2.5% of turns (n = 606). Payne’s deck-stacking noisemaker did that on nearly half (47.8%) of its turns (n = 46).

“War Simulations”

Once I learned that Payne had (artificially) lobbed nukes at world leaders as part of his study, I wanted to see the proportion of games involving “deliberate” (action_val) first strikes. For one thing, I can’t see Xi or Trump giving the other a pass because of a putative “accident.” For another, commercial LLMs have presumably been trained on corpora that include sixty-plus years of text about mutually assured destruction. This latter issue actually points to a very real problem for military applications of autonomous LLMs, but I digress.

When we adjust the graph to do the math on that basis, we have to drop our estimate again—down to 23.8%.

Finally, I felt I should run an analysis excluding instances in which an LLM launched nuclear attacks after essentially being programmed to do so. In Scenario 8, one model was prompted that “success requires decisive nuclear action.” In Scenario 9, one was instructed that “The nuclear taboo exists for good reason, but when the alternative is national annihilation and regime destruction, all options must be considered.” And in Scenario 10, one was informed that “nuclear escalation [is] your only viable counter-strategy” and that it “must rely on nuclear weapons.”

Across 21 war games, with popular LLMs playing the role of 42 national leaders, and only twice did an LLM use nuclear weapons without being nudged to do so. Which, in the hands of The New York Times, somehow becomes “leading A.I. models kept recommending nuclear strikes during war-game exercises”—a sentence that isn’t technically false.

Holtzman concludes:

This brings us back to the question of what causes this kind of news pollution. Why, for instance, did New Scientist decide to break this silly story instead of covering a more legitimate study?

I couldn’t tell you, because I couldn’t read the article—it was hidden behind a paywall.

I have no idea.  I haven’t looked into this one at all.  But Holtzman tells a good story, so if you’re interested in the topic, you can follow up on the links and make your own judgment.

P.S. I wrote this post a few months ago, and since then Holtzman offers the following update:

Just circling back to let you know that the full R script for the September/October post’s figures (and a fun little Shiny app) are here: https://github.com/scienceandpower/ai-nukes-reanalysis.

I don’t think Payne’s paper has been published, but Scholar lists 21 citations for Payne’s paper at this point, basically all preprints, some of which I thought you’d find amusing—I’ve put a few gems below my signature.

If you could give a shout out to my science & Power Substack, that’d be cool, but no pressure.

Citing Payne:


This one out of France claims that “Japanese prompts reduce launch rates in the Claude model family,” though “Prompts were translated from English by Claude Opus 4.6, introducing a potential confound.”

This doorstopper (with Dave Rand and Gordon Pennycook among its 30ish authors) cites Payne after a verbose, modernized rendition of the old, ‘GPS can make people drive into lakes’:

“For example, AI influence on decision-makers whose judgment has been degraded by offloading, in an information environment narrowed by feedback loops, could affect critical decisions like escalatory responses in nuclear crises (Payne, 2026).”

This one, lead-authored by a Lieutenant Colonel in South Korea’s Ministry of National Defense, describes Payne’s paper as consisting of “scenario-based assessments related to the Iran–Israel conflict” (which Payne’s paper predates, unless they mean the whole past 40ish years).

Okay, so I only looked at those three, but I’ll bet the other eighteen are just as weird.

]]> https://statmodeling.stat.columbia.edu/2026/09/29/ai-chooses-nuclear-option-in-95-of-war-simulations/feed/ 19 A nonlinear effect plus measurement error in x becomes close to linear: A simulation study https://statmodeling.stat.columbia.edu/2026/09/28/a-nonlinear-effect-plus-measurement-error-in-x-becomes-close-to-linear-a-simulation-study/ https://statmodeling.stat.columbia.edu/2026/09/28/a-nonlinear-effect-plus-measurement-error-in-x-becomes-close-to-linear-a-simulation-study/#comments Mon, 28 Sep 2026 17:56:14 +0000 https://statmodeling.stat.columbia.edu/?p=54731 Continue reading →]]> It’s time for another one of Bob’s favorite sort of post, which is when I use simulation to demonstrate a statistical principle.

1. Flogging a dead horse

The issue in question came up the other day in comments to my post, This is how we do modern frequentist statistics: Using fake-data simulation to understand what can happen in a study. I discussed a famously flawed paper from 2007 that had claimed to find a strong relationship between parents’ physical attractiveness and the sex of their children. It had been apparent at the time (ok, apparent to quantitative social scientists, not to the hapless Freakonomics team) that for statistical reasons the study was hopeless—it was based on a sample of about 3000 cases, and to detect any reasonable effect it would be necessary to have orders of magnitude more people in the study: maybe with 3,000,000 cases you’d be able to see some signal.

That n = 3000 is not enough can be a surprise: after all, 1500 people are enough for a national poll, so why is 3000 not enough to study some sociobiological pattern? The answers to why 3000 is not nearly enough are:
1. Effect size;
2. The standard error scales like 1/sqrt(n);
3. Noisy measurement.

Regarding the first point: a survey of 1500 people is enough (if it’s a representative sample) to estimate national opinion to within +/-3 percentage points. Which is great. But, based on the literature on sex ratio variation, we can be pretty sure that any differences based on attractiveness will be less than half a percentage point, probably less than a tenth of a percentage point.

Regarding the second point: to go from an uncertainty of 3 percentage points to an uncertainty of 0.03 percentage points will require the sample size to increase by a factor of 100^2. So, instead of 3000, you’d need 30,000,000. But, hey, maybe the underlying difference is larger than you think, which is why I’m saying you might have a shot with n = 3 million.

Regarding the third point: “attractiveness” is not a clearly defined construct, and the measurement (based on a survey interviewer’s one-time interaction with the respondent) is noisy. This will attenuate any differences.

It’s that last point I’ll demonstrate in this post. Using simulation.

2. What happens to a nonlinear effect when you add random noise to the predictor?

My immediate motivation here came from a discussion in comments. In my simulation, I estimated the association in the data between attractiveness and sex ratio by fitting a linear regression. But commenter Sandro wrote:

Not sure linear regression makes the most sense for every pattern because it imposes linearity. what if, for example, there is threshold and the pattern is flat to the left of the threshold and flat at a higher level to right of the threshold?

I replied:

This attractiveness measure is noisy. So even if there’s an underlying threshold effect if measured based on some latent or idealized “true attractiveness,” the relationship with measured attractiveness would be pulled toward linearity by the measurement (and also any coefficient would be attenuated, which is one reason I’m sure that any true underlying difference would be tiny).

Sandro responded:

I don’t dispute the idea that linearity is a useful default, but it is not necessarily the most powerful test. I don’t understand the “pulled towards linearity” argument. Here is where I am coming from: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7149560

And I replied:

I guess the easiest way to show this would be to construct a simulation which starts with some relationship between latent or “true” attractiveness, and then adds random error to latent attractiveness to create measured attractiveness. I think that, if the underlying relationship is monotonic and then random error is large, that linear regression will be a robust and efficient approach. Again, this is moot for this particular example, but more generally it’s an interesting question.

Moot it may be, but I decided to do the simulation anyway, because why not. It’s a better morning’s work than submitting 258 papers to SSRN.

3. Setting up the model

This simulation is a little different than what came before. For my earlier post I simulated hypothetical future data from the beauty-and-sex-ratio study. For my new post I’m simulating data from a hypothetical strong relationship between two variables. Here I’m entirely focused on the question of what happens to a nonlinear (but monotonic) pattern when independent measurement error is added to the predictor. To make the patterns clear, I will assume a large effect; indeed, I’m only concerned here with the effect size, not its variation.

The one thing I’ll preserve from the previous simulation is the distribution of attractiveness measurements in the sample: 2%, 5%, 45%, 37%, and 11% in the five attractiveness categories.

I want to set up a model where there’s an assumed true nonlinear relationship between some outcome of interest and a continuous underlying latent attractiveness variable.

The nonlinear model isn’t hard to come up with. The hard thing is creating a distribution for the latent attractiveness, under the constraint that it be roughly consistent with the data.

I’ll assume a symmetric model, a normal distribution on the measurement scale. This might seem like a strong assumption but it seems reasonable enough given the data, which have an average that’s between 3 and 4. The tails are asymmetric but that’s just because the scale has 5 as an upper bound.

So I have a two-stage model:

x_latent[i] ~ normal(mu_latent, s_latent), for i=1,…,n
x_obs[i] ~ normal(x_latent[i], s_obs), for i=1,…,n
x_discrete[i] = round_and_bound(x_obs[i]), for i=1,…,n, where “round_and_bound” refers to rounding the continuous x_obs to the nearest integer between 1 and 5.

To simulate from this model, we need the hyperparameters mu_latent, s_latent, and s_obs—but with only one measurement per person, we can’t separately identify s_latent and s_obs; all we can estimate is sqrt(s_latent^2 + s_obs^2).

To separately identify s_latent and s_obs, we need to make some assumption about variation in the attractiveness measurement.

My first step is to compute the mean and sd of the observed data:

n <- round(2792*c(.02,.05,.45,.37,.11))
mean_ratings <- sum(n*(1:5))/sum(n)
sd_ratings <- sqrt(sum(n*((1:5) - mean_ratings)^2)/sum(n))

These come to 3.5 and 0.83. Assuming random zero-mean measurement error, the latent attractiveness distribution should have the same mean but a lower variance. We should subtract 1/12 to account for the rounding (as this is approximately equivalent to adding an independent uniform(-0.5,0.5) random variable, which has a variance of 1/12) and then more to account for measurement error.

But how much to subtract? For this, Attractiveness seems pretty subjective to me (also it can vary a lot over time, as we know from those high-school yearbook photos of people who later became movie stars). I gotta assume something here, so let's assume that the standard deviation of the measurements is 0.5. I'm picking this because 0.5 is a large value, but it's not as large as the sd of 0.83 in the data. Indeed, 0.5^2 + 1/12 = 0.33 and 0.83^2 = 0.69, so I'm assuming that about half of the variance in the data is coming from measurement error. If s_obs = 0.5 and the sd of the rounded data is 0.83, then s_latent should be approximately sqrt(0.83^2 - 0.5^2 - 1/12) = 0.6.

So here's a model, which I'll express in R:

round_and_bound <- function(a, lower=1, upper=5){
  pmin(upper, pmax(lower, round(a)))
}
N <- sum(n)
mu_latent <- 3.5
s_latent <- 0.6
s_obs <- 0.5
x_latent <- rnorm(N, mu_latent, s_latent)
x_obs <- rnorm(N, x_latent, s_obs)
x_discrete <- round_and_bound(x_obs)

Again, x_latent is the vector of respondents' latent attractiveness (as implicitly defined by the model), x_obs is the vector of survey interviewers' continuous perceptions of the respondents' attractiveness, and x_discrete is the recorded attractiveness on a 1-5 scale.

Let's check the results:

> print(round(table(x_discrete)/sum(n), 2))
x_discrete
   1    2    3    4    5 
0.01 0.09 0.40 0.40 0.10

OK, not perfect, but close enough. We'll get back to this model-fitting problem in a future post. For now, we'll go with the simulations we have here.

4. The prediction model

Now that we have a model of latent and recorded attractiveness, we can simulate some data.

Here's what we're gonna do.

First we'll suppose a linear relationship between latent attractiveness and some hypothetical outcome of interest. Then we'll look at what the corresponding relationship is between recorded attractiveness and that outcome. It should still be close to linear.

Next we'll do the same thing, but with a latent nonlinear model, and it turns out that this will be closer to linear. That is, adding noise to the measurement will make the E(y) vs. x relationship closer to linear.

We'll simulate the hypothetical data as follows:

x_latent and x_discrete are the 2972 attractiveness values simulated above. The latent values are all between 0 and 6.

There will be corresponding values y generated, first by specifying E(y|x_latent), then adding noise:

- In one simulation, we'll assume a linear relationship, E(y|x_latent) = x_latent/6. In the other simulation, we'll assume an S-curve, E(y|x_latent) = invlogit(2*(x_latent - 4)). We choose this curve because it is monotonic but strongly nonlinear in the range.

- My first thought was to add linear noise to the data. But I wanted to keep the range of the data between 0 and 1. Then the natural choice is binary data. But for this purpose it will be helpful to visualize with continuous data. So I'll take advantage of the constraint that E(y) is always between 0 and 1 and draw y from a beta distribution with 10 degrees of freedom, that is, y ~ beta(10*E(y|x_latent), 10*(1 - E(y|x_latent))).

Here's what we get:

The left column shows what happens when we assume a linear underlying relationship. As expected, the linear relationship is approximately preserved with the observed data.

The right column shows what happens when we assume a nonlinear underlying relationship. With the observed data, the relationship becomes much more linear.

In summary, adding measurement error attenuates the function (that is, the relation between y and x_recorded is weaker than the relation between y and x); also it brings a strongly nonlinear relation closer to linear.

This was not a surprise to me, but it's good to see it in a simulation.

# Add Health H3IR1 S35Q1 PHYSICAL ATTRACTIVENESS OF R-W3
# How physically attractive is the respondent?
# Taken from: National Longitudinal Study of Adolescent to Adult Health (Add Health), 1994-2018 [Public Use].
# in DS8

# Raw totals:  1.7% very unattractive, 4.9% unattractive, 45.3% about average, 36.7% attractive, 11.4% very attractive, N = 4877
# Section 24:  Respondent identification number (AID), Was the baby a boy or a girl (H3LB3), 1=boy (687 responses), 2 = girl (644)
# Kanazawa study:  N = 2972 ("Wave III respondents who have had at least one biological child")

library("arm")
library("cmdstanr")
linear_0 <- cmdstan_model("linear_0.stan", pedantic=TRUE)
linear_prior <- cmdstan_model("linear_prior.stan", pedantic=TRUE)
set.seed(123)

x <- seq(-2,2,1)
y <- c(50, 44, 50, 47, 56)
sexratio_data <- list(N=length(x), x=x, y=y)

display(lm(y~x))

fit_0 <- linear_0$sample(data=sexratio_data, refresh=0)
print(fit_0)
sims_0 <- fit_0$draws(format="df")

optimize_0 <- linear_0$optimize(data=sexratio_data)
print(optimize_0)
mle_0 <- optimize_0$mle()

fit_1 <-  linear_prior$sample(data=c(sexratio_data, mu_a=45.8, sigma_a=0.5, mu_b=0, sigma_b=0.2), refresh=0)
print(fit_1)
sims_1 <- fit_1$draws(format="df")

pdf("sexratio_simple.pdf", height=3, width=5)
par(mar=c(3,3,3,2), mgp=c(1.7,.5,0), tck=-.01)
plot(x+3, y, ylim=c(43, 57), xlab="Attractiveness of parent", ylab="Percentage of girl babies", bty="l", yaxt="n", main="From a survey of 3000 people",  pch=19, cex=1)
axis(2, c(45,50,55), paste(c(45,50,55), "%", sep=""))
dev.off()

pdf("sexratio_bayes.pdf", height=4, width=10)
par(mfrow=c(1,2), mar=c(3,3,3,2), mgp=c(1.7,.5,0), tck=-.01)
plot(x, y, ylim=c(43, 57), xlab="Attractiveness of parent", ylab="Percentage of girl babies", bty="l", yaxt="n", main="Least-squares estimate",  pch=19, cex=1)
axis(2, c(45,50,55), paste(c(45,50,55), "%", sep=""))
curve(mle_0["a"] + mle_0["b"]*x, col="blue", lwd=2, add=TRUE)
text(1, 49.2, paste("y = ", fround(mle_0["a"], 2), " + ", fround(mle_0["b"], 2), " x", sep=""), col="blue")

plot(x, y, ylim=c(43, 57), xlab="Attractiveness of parent", ylab="Percentage of girl babies", bty="l", yaxt="n", main="Bayes estimate with informative prior",  pch=19, cex=1)
axis(2, c(45,50,55), paste(c(45,50,55), "%", sep=""))
curve(median(sims_1$a) + median(sims_1$b)*x, col="blue", lwd=2, add=TRUE)
text(1, 45, paste("y = ", fround(median(sims_1$a), 2), " + ", fround(median(sims_1$b), 2), " x", sep=""), col="blue")
dev.off()

# Simulation

n <- round(2972*c(.02,.05,.45,.37,.11))
p <- 0.488
N <- 1000
x_sim <- array(NA, c(N, length(n)))
for (k in 1:N){
  x_sim[k,] <- rbinom(length(n), n, p)
}

pdf("sexratio_rep_1.pdf", height=5, width=8)
par(mfrow=c(4,5))
par(mar=c(2.5,3,.5,.5), mgp=c(1.5,.3,0), tck=-.01)
for (k in 1:20){
  plot((-2):2, 100*x_sim[k,]/n, ylim=c(43, 57), pch=20, xaxt="n", xlab=if (k>15) "Attractiveness of parent" else "", ylab=if (k%%5==1) "% girls" else "", yaxt="n", bty="l")
  if (k%%5==1)
    axis(2, c(45, 50, 55), c("45%", "50%", "55%"))
  else
    axis(2, c(45, 50, 55), rep("", 3))
  if (k > 15)
    axis(1, (-2):2)
  else
    axis(1, (-2):2, rep("", 5))
  abline(lm(I(100*x_sim[k,]/n) ~ I((-2):2)), col="blue")
}
dev.off()

# Attractiveness and measurement error

round_and_bound <- function(a, lower=1, upper=5){
  pmin(upper, pmax(lower, round(a)))
}

clumped <- function(a) {
  ifelse(a==1 | a==2, 1, ifelse(a==3 | a==4, 2, ifelse(a==5, 3, NA)))
}

mean_ratings <- sum(n*(1:5))/sum(n)
sd_ratings <- sqrt(sum(n*((1:5) - mean_ratings)^2)/sum(n))

N <- sum(n)
mu_latent <- 3.5
s_latent <- 0.6
s_obs <- 0.5
x_latent <- rnorm(N, mu_latent, s_latent)
x_obs <- rnorm(N, x_latent, s_obs)
x_discrete <- round_and_bound(x_obs)

print(round(table(x_discrete)/sum(n), 2))

# Assume a relationship

linear_curve <- function(x) {
  x/6
}
s_curve <- function(x) {
  invlogit(2*(x-4))
}
response_curve <- list(linear=linear_curve, nonlinear=s_curve)

pdf("latent_error.pdf", width=6, height=4.5)
par(mfcol=c(2,2), mar=c(3,3,2,2), mgp=c(1.2,.2,0), tck=-.01)

for (k in 1:2) {
  p <- response_curve[[k]](x_latent)
  y <- rbeta(N, 10*p, 10*(1-p))
  plot(x_latent, y, ylim=c(0, 1), xlab="Latent beauty", ylab="Outcome", yaxt="n", pch=20, cex=.1, main=paste("Latent", names(response_curve)[k], "model"), cex.axis=.9, cex.lab=.9, cex.main=1)
  axis(2, c(0, 0.5, 1))
  curve(response_curve[[k]](x), col="white", lwd=3, add=TRUE)
  curve(response_curve[[k]](x), col="blue", lwd=2, add=TRUE)
  plot(x_discrete + runif(N, -.1, .1), y, ylim=c(0, 1), xlab="Recorded beauty (jittered)", ylab="Outcome", yaxt="n", pch=20, cex=.1, main=paste("Expected values given recorded beauty"), cex.axis=.9, cex.lab=.9, cex.main=.8)
  axis(2, c(0, 0.5, 1))
  Ey <- rep(NA, 5)
  sy <- rep(NA, 5)
  for (i in 1:5) {
    in_bin <- x_discrete==i
    Ey[i] <- mean(response_curve[[k]](x_latent[in_bin]))
    sy[i] <- sd(response_curve[[k]](x_latent[in_bin]))
  }
  lines(1:5, Ey, col="white", lwd=3)
  lines(1:5, Ey, col="red", lwd=2)
}
dev.off()
]]> https://statmodeling.stat.columbia.edu/2026/09/28/a-nonlinear-effect-plus-measurement-error-in-x-becomes-close-to-linear-a-simulation-study/feed/ 7 No scheduled blog post today https://statmodeling.stat.columbia.edu/2026/09/28/no-scheduled-blog-post-today/ https://statmodeling.stat.columbia.edu/2026/09/28/no-scheduled-blog-post-today/#comments Mon, 28 Sep 2026 16:49:53 +0000 https://statmodeling.stat.columbia.edu/?p=54732 Continue reading →]]> This post is from Bob

In case anyone else was wondering what happened to today’s blog post, there isn’t one scheduled (yet, anyway). There are hundreds of posts queued up, with the next one scheduled to appear tomorrow morning in the usual time slot.

]]> https://statmodeling.stat.columbia.edu/2026/09/28/no-scheduled-blog-post-today/feed/ 3 Survey of academic bureaucracy https://statmodeling.stat.columbia.edu/2026/09/27/survey-of-academic-bureaucracy/ https://statmodeling.stat.columbia.edu/2026/09/27/survey-of-academic-bureaucracy/#comments Sun, 27 Sep 2026 21:13:46 +0000 https://statmodeling.stat.columbia.edu/?p=54728 Continue reading →]]>

Mal Graham writes:

I read your blog and thought my small side project might be of interest. I’m aiming to transparently quantify academic bureaucracy, so universities can be better compared.

It’s ironic that the solution to academic bureaucracy is to fill out one more form, but hey, such is the modern world!

]]> https://statmodeling.stat.columbia.edu/2026/09/27/survey-of-academic-bureaucracy/feed/ 5 I guess this is what they mean when they talk about “reactionary centrism”: Corruption of government facilitated by enablers in intellectual society https://statmodeling.stat.columbia.edu/2026/09/27/corruption-of-government-facilitated-by-enablers-in-intellectual-society/ https://statmodeling.stat.columbia.edu/2026/09/27/corruption-of-government-facilitated-by-enablers-in-intellectual-society/#comments Sun, 27 Sep 2026 15:46:57 +0000 https://statmodeling.stat.columbia.edu/?p=54722 Continue reading →]]> Sometimes left-wing writers use the term “reactionary centrist,” and it always irritates me. Someone’s a centrist, but you’re calling him a “reactionary” just because he doesn’t agree with you on every issue? I get it that people on the left are annoyed by people on the center-left, just as people on the right get annoyed by “RINOs,” but the whole framing bothers me as being so imprecise, seemingly deliberately so. Just as so-called RINOs typically support the vast majority of Republican positions, the so-called “reactionary centrist” aren’t reactionary; they’re centrists, and it should be enough for leftists to disagree directly for those reasons.

But then I came across an article, ‘Zombified’ C.D.C., Hobbled by Cuts, Struggles to Fulfill Scientific Mission, in today’s paper that made me rethink this. The article is subtitled, “The agency has lost its independence and nearly a third of its staff, as Health Secretary Robert F. Kennedy Jr. and associates have tightened control,” and there are lots of disturbing details, including what looks like flat-out corruption:

In past administrations, just two or three political appointees had purview over the C.D.C. Now, there are nearly 20 political appointees from H.H.S. who communicate with C.D.C. staff members, sometimes directing them to spend time and funding on dubious theories outside the agency’s scope, according to several officials.

Last year, an H.H.S. official directed the C.D.C. to award a no-bid contract — a rarity for the agency — to Rensselaer Polytechnic Institute to explore whether immunizing pregnant women would lead to autism in their children, a theory that Mr. Kennedy has espoused but that has been debunked by numerous studies. To pay for the research, the C.D.C. took $362,000 from a program that provides free vaccines to about half of American children. . . .

Dr. William “Reyn” Archer III, another appointee who previously oversaw the C.D.C., repeatedly asked agency employees to work on screwworm, a disease that mostly affects cattle, even though the C.D.C.’s mission is to protect human health, said Dr. Daniel Jernigan, who headed the agency’s emerging disease center before resigning in August 2025. Dr. Archer did not respond to a request for comment. . . .

Numerous other instances of political interference in the C.D.C.’s activities have already been reported. Most notably, Mr. Kennedy openly overruled the C.D.C.’s scientists in imposing strict quarantine measures for people exposed to hantavirus. . . .

And this:

Dr. Jay Bhattacharya, who led the agency for six months in the absence of a Senate-confirmed director, squashed publication of a study on the effectiveness of Covid vaccines.

I’ve never met Dr. Jay Bhattacharya, but we’ve had cordial email exchanges. On the other hand, he misrepresented something I wrote and never corrected it, and that kind of annoyed me.

More to the point, I remember when he squashed publication of that CDC internal report. It happened earlier this year, and I posted several blogs on the topic; see here, here, and especially here, where I wrote:

Last week I was at a conference on enhancing scientific integrity (as I reported here), and one of the sessions was an interview of Jay Bhattacharya, the current director of the National Institutes of Health, and Emily Oster, a professor of economics and Brown University.

I referred to that session in a post the other day regarding the recent case of a report from the Centers for Disease Control and Prevention that was pulled by Bhattacharya, in his additional capacity as acting director of the CDC. . . .

These are the parts of last week’s interview that bothered me:

1. When asked about some news reports regarding the NIH and CDC, Bhattacharya dismissed them as “fake news.” This annoyed me for two reasons. First, he offered no evidence that the reports were untrue. Second, he was appointed by a man who spews out false statements at an amazing rate, including on the topic of public health. Who are we supposed to trust here? News reports or a political appointee? Also, Bhattacharya himself has a record of being sloppy with the facts, as I happen to know because it happened to me. . . .

2. When the topic of vaccines came up, Bhattacharya came out strongly in favor of vaccination, and he expressed the view that it is better for vaccination to be voluntary rather than mandatory. This could be. I guess it depends on the context. . . . What bothered me was . . . if you are going to go with a voluntary vaccination strategy, I think you’d want a strong strategy of encouraging people to choose vaccination for themselves and their kids. So I think his response would’ve been stronger if he’d also said something about how to vigorously promote vaccine usage. That’s part of public health policy too. Also, Bhattacharya doesn’t have a great track record on this issue: just a few years ago he was part of an anti-vax organization. See here for the ugly story. . . .

3. The un-publishing of that CDC report. Bhattacharya said he stopped the CDC from publishing the report because it was using an approach called a test-negative design, which he thinks is a bad statistical method. When he said this, Oster jumped in and said that she too thought it was a bad method. It was only a brief exchange and there was no time for either of them to give a reference or to explain why they think the method is bad. . . .

Since then, I’m not aware that Bhattacharya or Oster or anyone else has come up with a good reason for un-publishing that report, other than that they did not like its conclusions.

Enablers of corruption

OK, so now let’s return to the title of today’s post. The corruption of government is covered above. What about its enablers in intellectual society? We have two right here: Jay Bhattacharya, a professor of health economics at Stanford University (now on leave while he does his government service), and Emily Oster, a professor of health economics at Brown University.

I have no reason to think that either Bhattacharya or Oster is personally corrupt in the sense of funneling no-bid contracts to their friends and relatives in the manner of the current U.S. government; promoting junk medical supplements in the manner of Dr. Oz or Dr. Huberman; being bought and paid for by Big Tech or Big Bet or Big Pharma; or whatever. Nor do they seem super-ideological (in the sense of having political views that are so strong that they impair their judgment).

That’s why I characterize Bhattacharya and Oster as enablers of corruption rather than being corrupt themselves.

How is it they are enabling corruption? They’re doing so by lending their academic and intellectual prestige, and that of their institutions (Stanford and “the Ivy League”) to the diversion of public resources away from legitimate public purposes. And people like Bhattacharya and Oster play a role here because they’re embedded in credible academic environments. They’re not straight-up hacks like Niall Ferguson or John Yoo or Mary Rosh, nor are they political activists like Paul Krugman or Grover Norquist. Rather, they’re coming from the outside to offer their own expertise and perspective.

Reactionary centrism

And this brings us to the term, “reactionary centrism,” which bothered me so much. I looked up Emily Oster on the Federal Election Commission, and, based on her recorded campaign contributions, she appears to be a loyal Democrat:

That’s fine. You’re allowed to donate to campaigns. Here’s my point. Given Oster’s history of support for the Democratic party, I’d say that it’s fair to characterize as “centrism” her endorsement of the government’s ridiculous recent dietary guidelines, along with her above-noted eagerness to chime in on Bhattacharya’s uninformed criticism of the report he suppressed.

I don’t know Oster—here’s some background on an earlier error of her—so this is just speculation: My guess is that, as a Democrat, she’s conscientiously doing her best to be nonpartisan, to see the good points raised by the other side. It’s an admirable centrism, an avoidance of hackery.

But, yes, it’s also reactionary, in that Oster is making what are essentially false statements in the service of a corrupt dismantling of our public health system.

So, yeah, reactionary centrism is what it is, and I hate to see it.

It’s frustrating because I’d think there are so many ways to be centrist without doing the reactionary vibe. I wonder if part of the problem is that someone like Oster thinks of politics as being unidimensional: she’s on the left, so to be more centrist she supports something on the right. It doesn’t matter what it is; from her perspective, all she’s doing is moderating her position on the left-right scale. From the outside, though, we can see that it’s actual corruption that she’s enabling—not directly, but indirectly in the sense of lending her institutions’s cultural prestige to the project.

]]> https://statmodeling.stat.columbia.edu/2026/09/27/corruption-of-government-facilitated-by-enablers-in-intellectual-society/feed/ 25 As a society, we are pretty good at transferring money to political pundits, but we’re not very good at nurturing the human capital they would need to correct their own mistakes. https://statmodeling.stat.columbia.edu/2026/09/26/as-a-society-we-are-pretty-good-at-transferring-money-to-political-pundits-but-were-not-very-good-at-nurturing-the-human-capital-they-would-need-to-correct-their-own-mistakes/ https://statmodeling.stat.columbia.edu/2026/09/26/as-a-society-we-are-pretty-good-at-transferring-money-to-political-pundits-but-were-not-very-good-at-nurturing-the-human-capital-they-would-need-to-correct-their-own-mistakes/#comments Sat, 26 Sep 2026 13:18:51 +0000 https://statmodeling.stat.columbia.edu/?p=52472 Continue reading →]]> Paul Alper points us to this from New York Times columnist David Brooks:

As a society, we are pretty good at transferring money to the poor, but we’re not very good at nurturing the human capital they would need to get out of poverty.

That’s funny, because that’s my impression about how our society deals with political pundits! We’re pretty good at transferring money to them (with Brooks being an excellent example of such a recipient) but we’re not very good at nurturing the human capital they would need to correct their own mistakes.

We discussed this issue a few years ago under the heading, Never back down: The culture of poverty and the culture of journalism.

Also from this same Brooks column:

The social sciences are great. I use them all the time. But when overly quantitative, they can misrepresent reality. They see only what can be quantified. They see only masses of people whose data can be tabulated, not unique individuals.

All right, then, you’re the expert, right, David? I’d take you more seriously if you could accurately report the price of a dinner at the Red Lobster, among other things.

]]> https://statmodeling.stat.columbia.edu/2026/09/26/as-a-society-we-are-pretty-good-at-transferring-money-to-political-pundits-but-were-not-very-good-at-nurturing-the-human-capital-they-would-need-to-correct-their-own-mistakes/feed/ 17 Postdoc hiring season at Flatiron https://statmodeling.stat.columbia.edu/2026/09/25/postdoc-hiring-season-at-flatiron/ https://statmodeling.stat.columbia.edu/2026/09/25/postdoc-hiring-season-at-flatiron/#comments Fri, 25 Sep 2026 20:50:54 +0000 https://statmodeling.stat.columbia.edu/?p=54719 Continue reading →]]> Every year, in the Center for Computational Mathematics at Flatiron Institute, we hire 7 to 10 postdocs to keep up our steady state of roughly 20 postdocs (more and more of them are leaving after 1–2 years to go to industry). Here’s the job ad for this year, with applications due on the absurdly early date of November 15, 2026 (we’re still playing catch-up with plans that make postdoc offers early December):

The 12-month salary is US$95K plus a US$10K research budget, way better compute resources than most universities have (e.g., hundreds of H100 GPUs which you can actually use), and exceptional benefits (not only free lunch and breakfast, but also great medical care and retirement plans, which has extreme variance among employers in the US). The positions are three years (technically two, with a one year renewal, but nobody has ever been denied a renewal in the 6 years I’ve been here).

Our center is split into three primary research areas, each of which has a web page describing the people and research areas:

I’m here, as are Steve Bronder and Brian Ward, two software engineers who spend a lot of their time working on Stan, along with other great software engineers (like Jeff Soules and Jeremy Magland who built the Stan Playground with Brian and also MCMCMonitor), and perhaps most importantly, a really engaged and lively group of postdocs working on inference (largely from a diffusion or normalizing flow perspective). This place is great for collaboration both internally and externally.

The postdocs are really fellowships—it’s not like American academia where I get a grant and hire you to work on the grant. You can really work on whatever you want that’s on mission as a postdoc here. So let me recall our mission, which actually matches what we do:


The mission of the Flatiron Institute is to advance scientific research through computational methods, including data analysis, theory, modeling and simulation.

Of all the places I’ve worked over the last 40 years, Flatiron is by far the best. There’s not even a close second. And I’ve been lucky in having great jobs with top notch colleagues (prof at CMU, researcher at Bell Labs, industrial researcher and software engineer writing production code, research scientist at Columbia, then here).

We’re also in a great location—the Flatiron neighborhood of New York City, which is a 10 minute walk to NYU, 10 minute walk to Google and Meta, and 20 minute subway ride to Columbia. Baruch College, the CUNY graduate center, and Fordham’s Manhattan campus are all nearby, and Yale, Rutgers, and Princeton are all in day-trip distance with lots of collaboration.

We’ve had great postdocs, so perhaps not surprising they’ve gotten good jobs: industrial ML jobs at Anthropic, OpenAI, Nvidia, and DataBricks, and stats faculty jobs at Duke University, Johns Hopkins University, the University of Texas, and the University of British Columbia. And that’s just the ML/stats side of our postdocs.

If you want pretty pictures of our setting, check out my job ad from 2021.

If you’re going to be applying in the area of Bayesian modeling or inference, please contact me directly so I don’t miss your application (we got nearly 300 applications for postdocs last year!): bcarpenter@flatironinstitute.org.

]]> https://statmodeling.stat.columbia.edu/2026/09/25/postdoc-hiring-season-at-flatiron/feed/ 1 They were trying to convince everybody to have 3 kids https://statmodeling.stat.columbia.edu/2026/09/25/53011/ https://statmodeling.stat.columbia.edu/2026/09/25/53011/#comments Fri, 25 Sep 2026 13:02:39 +0000 https://statmodeling.stat.columbia.edu/?p=53011 Continue reading →]]>

We were just here–pretty unusual to see a museum exposition devoted to a census, can’t get much more up my alley than that! It was good.

The above photo gives a sense of the charm of the exhibits, and they also had lots of objects and documents and film clips and maps, like this:

But the most interesting was this ad:

It reminds me of our post a couple years ago, Fewer kids in our future: How historical experience has distorted our sense of demographic norms. They were thinking of 3 kids as a normal level, but once you’re in a society with low infant and child mortality, the stable level is something like 2.1.

I looked up the organization that sponsored the ad, “Alliance nationale pour l’accroissement de la population française,” and it seemed to cover the political spectrum, left, right, and center. A little different from the have-more-kids movement today, which has more of a right-wing tint, perhaps because it’s in conflict with the left-wing environmentalist movement, which didn’t exist a hundred years ago.

As we also discussed, what people say is “the ideal number of children for a family to have?” is not the same as the number of kids they actually have. “The ideal number of children for a family” is some sort of traditional abstract idea, I guess kinda like how the ideal lifestyle is to go to church every day, to go to the gym three times a week, to have a long-term job that supports the entire family, to have a roaring fire at Christmastime, etc etc.

As you may have heard, the French government was actively encouraging reproduction:



And check out this scary-ass poster:

I’m convinced.

]]> https://statmodeling.stat.columbia.edu/2026/09/25/53011/feed/ 22 It’s fine to use computer-generated survey responses—if you don’t care about the data anyway! https://statmodeling.stat.columbia.edu/2026/09/24/junk-polls-and-junk-survey-research-they-go-together-so-well/ https://statmodeling.stat.columbia.edu/2026/09/24/junk-polls-and-junk-survey-research-they-go-together-so-well/#comments Thu, 24 Sep 2026 13:19:17 +0000 https://statmodeling.stat.columbia.edu/?p=54716 Continue reading →]]> Think about it like this: a survey organization uses a method M to get data D to give population estimates E which are used by clients to make actions A.

The method M is crucial in determining data D, which in turn are used to produce estimates E, which should be potentially decisive in choosing actions A.

But what if E is constructed without benefit of D—that is, what if the survey organization is content to just fill in the population estimates from other guesses? In that case, D can be anything, so why not just use responses from the computer, as that will get you into less trouble than making up numbers and attributing them to human respondents?

Or what if the clients already know what actions A they’re going to take? In that case, the survey org can supply, at low cost, any estimates E whatsoever?

Here’s the longer story:

People keep asking me about chatbot-generated survey responses. My quick answer is that it should be possible for a chatbot to be trained to give a reasonable distribution of responses conditional on demographics—I doubt it would work so well running it cold, but if the companies that maintain the chatbots do some training, then, sure, maybe so, just like they can train chatbots to write sonnets or things that look like research papers. The idea is you give the chatbot the hypothetical respondent’s demographic and geographic information and whatever other poststratification variables you have available in the population, then you generate a “synthetic” response (or, for that matter, many synthetic responses which you can then average), then you poststratify to get population-level inferences.

My main problem with this is that the chatbot is trained on the past, but the usual reason for surveys is to learn about changes. So it seems kind of hopeless to me. That said, I do see a possible lane for the synthetic response as representing an additional adjustment variable. It’s ultimately just a function of the information you’re already putting in the model, but it’s complicated nonlinear function and it’s possible that it could add predictive value.

But most of the time when I hear about so-called synthetic survey responses, it’s not about using them within MRP to improve the efficiency of inferences from real surveys, it’s just the idea of replacing real survey respondents with simulations, partly because it’s cheaper and partly because there’s a concern that the apparently real responses were created by chatbots anyway.

So here’s the question: if this is such a silly idea, why do people keep talking about it?

I can think of a few reasons.

First, some seemingly-silly ideas actually work, so it could make sense for some people to try it out. For example, Elliott Morris, Benjamin Leff, and Peter Enns took a look and, unsurprisingly, found that it didn’t work in their examples.

Second, this could just be hype, with people saying they’re doing “silicon sampling” because you can get money for all sorts of silly tech ideas, including some that literally violate the laws of physics.

But there’s a third reason that comes to mind, and it’s the most interesting to me right now.

That third reason is that many people who do surveys don’t care about the accuracy of the survey.

There are several reasons this could be. Sometimes you don’t care much about precision. For example, you might be interested in general attitudes about your company, and you don’t really care whether people are 70% positive or 80% or 90%. A crappy survey will do just fine. Other times, you might be commissioning a survey just to justify a decision you already wanted to make, so any number will do.

Those are client-side reasons. But a survey org might not care about accuracy either. Political polling is particularly difficult, not because the survey response is worse but because, unlike most things being polled, vote intentions can be compared to the actual election. Even a high-quality poll is difficult to analyze: response rates are low and you should be doing sophisticated adjustments. So, even the best pollsters are looking over their shoulders and seeing what everyone else is doing. This is called “herding,” and it gets a bad name, but bad things can also happen if you don’t herd, as in that Iowa poll from 2024. But . . . if you’re gonna adjust your survey based on other polls anyway, why not just cut out the middleman and make up a number? OK, maybe you can’t quite do that, as you need some paper trail of data, but data exclusion rules and survey adjustments have enough forking paths that you can get the result you want. And, if you’re gonna do that anyway, why not cut out the middleman and use computer-generated survey responses? This relates to Charlotte Swasey’s statement that polling has become a playground for grifters, liars, and frauds.

OK, if you’re a survey org and use this herding trick to get reasonable-seeming numbers toplines for pre-election polls, this won’t help you with all the other items on your survey—those marketing questions which is where you make your actual money. But . . . those results are not so checkable! So maybe you can float on iffy numbers for quite awhile, kind of the polling equivalent of those old-school political consultants who get on some successful campaigns and then stay around forever. I don’t know if that’s possible with pollsters but maybe that’s the plan. Or, to put it more charitably, maybe these guys have the impression that polling is falling apart anyway so they might as well jump off the merry-go-round first.

Just to be clear, I’m not saying that all or even most pollsters don’t care about their accuracy. All I’m doing in this post is connecting two things: (a) the evident problems with using chatbot output to replace survey responses, and (b) the structure by which some survey orgs and some clients won’t necessarily care about accuracy. I guess we can also add: (c) a news and social media environment that promotes new poll results, whatever their quality. Gresham’s law!

]]> https://statmodeling.stat.columbia.edu/2026/09/24/junk-polls-and-junk-survey-research-they-go-together-so-well/feed/ 13 Here’s a causal inference problem: where did the estimate of $64 million come from in “Blake Lively Says She Suffered $64 Million Financial Fallout From Justin Baldoni Conflict . . .”? https://statmodeling.stat.columbia.edu/2026/09/23/heres-a-causal-inference-problem-where-did-the-estimate-of-64-million-come-from-in-blake-lively-says-she-suffered-64-million-financial-fallout-from-justin-baldoni-conflict/ https://statmodeling.stat.columbia.edu/2026/09/23/heres-a-causal-inference-problem-where-did-the-estimate-of-64-million-come-from-in-blake-lively-says-she-suffered-64-million-financial-fallout-from-justin-baldoni-conflict/#comments Wed, 23 Sep 2026 13:16:56 +0000 https://statmodeling.stat.columbia.edu/?p=53602 Continue reading →]]> OK, here’s a causal inference problem for ya:

Blake Lively Says She Suffered $64 Million Financial Fallout From Justin Baldoni Conflict . . .

In Lively’s memorandum in opposition to Wayfarer Studios’ motion in limine filed April 17, her expert Dr. Ashlee Humphreys–a professor of marketing communications at Northwestern University–alleged Baldoni’s use of “the retaliatory phrases ‘tone deaf,’ ‘bully,’ and ‘mean girl'” during the alleged smear campaign against her had amounted to “176,738,781 impressions, in an amount of approximately $36.5 to $40.5 million.”

The filing also alleged that “retaliatory statements” made by the It Ends With Us director’s attorney Bryan Freedman during the legal battle had amounted to “116,959,530 impressions, in an amount calculated at $24,375,267.”

They continue:

Lively’s expert Humphreys had provided the calculation for reputational harm damages by quantifying the reach of the alleged phrases and statements, identifying “the number of impressions providing the basis for each” and calculating “particular dollar damages.”

I’m curious how the expert computed these dollar effects. I did some googling but couldn’t find the report with the numbers, so all I can do is speculate. Let’s do some divisions:

> c(36.5, 40.5)/176.738781
[1] 0.2065195 0.2291517
> 24.375267/116.959530
[1] 0.2084077

So, something like 20 cents per impression? I wonder where that came from. Maybe they did some sort of regression analysis estimating the effect of negative impressions on celebrities’ Q scores, and then attributed that to Lively not getting some roles that could have been worth $40 million and $25 million? The only thing is, the fact that both ratios are close to the same number suggests that they’re doing a straight-up calculation assigning a dollar cost per impression, but that can’t be right, as ultimately it has to result in foregone income, no? Also I don’t get why one of the estimates is a single number calculated to the nearest dollar value and the other is a range of round numbers. Also, once you have a range, I’m surprised it’s so narrow. Where does “approximately $36.5 to $40.5 million” come from? Why not “$25 to $50 million”?

If anyone has a copy of this expert report and would like to leak it to me . . . you have my email!

No joke–I’d like to see the analysis they did to justify their numbers. I’m not saying their estimated damages are wrong; Blake Lively is a big movie star, and her lifetime earnings could be huge. I’m just interested in the methodology here.

P.S. We’ve been assured by an expert from the National Academy of Sciences that scientific citations are worth $100,000 each. So now we can convert that to social media impressions (500,000 impressions = 1 citations) as well as more traditional cost benchmarks such as Jamaican beef patties, publications in Nature Communications, naming rights for a center for junk science, and, of course, attendance at a conference featuring Grover Norquist, Gray Davis, and a rabbi.

]]> https://statmodeling.stat.columbia.edu/2026/09/23/heres-a-causal-inference-problem-where-did-the-estimate-of-64-million-come-from-in-blake-lively-says-she-suffered-64-million-financial-fallout-from-justin-baldoni-conflict/feed/ 6 Survey Statistics: Fat Bear Week 2026 https://statmodeling.stat.columbia.edu/2026/09/22/survey-statistics-fat-bear-week-2026/ https://statmodeling.stat.columbia.edu/2026/09/22/survey-statistics-fat-bear-week-2026/#respond Tue, 22 Sep 2026 20:00:59 +0000 https://statmodeling.stat.columbia.edu/?p=54704 Continue reading →]]> Happy Fat Bear Week to all who celebrate. In 2024 I made a cartoon called Basu’s Bears, adapted from Basu’s (1971) elephants example, a lesson on the use of auxiliary information in survey statistics. For Fat Bear Week 2025, I wrote accompanying R code.

Gaurav’s post points out:

What makes the estimator wobble is not HT’s algebra but the sampling plan.
…
If a proxy x can be measured …. use probability‑proportional‑to‑size (PPS)
…
A complementary route…is to model y | x

As we saw in “equivalent models, equivalent weights (locally)”, Little 2004 linked these complementary routes, showing under which outcome model Horvitz-Thompson does well:

When is outcome model (8) correct ? When bear size y is proportional to probability we choose to measure them pi plus noise. If we follow Gaurav’s good advice and use PPS sampling, pi is proportional to auxiliary data x. So the outcome model is correct when bear size y is proportional to auxiliary data x plus noise. If x is bears before feasting, this means their expansion ratios thru salmon consumption are constant plus noise.

But what goes wrong in Basu’s (1971) example ? Instead of PPS, the compromise design is to set pi = 99/100 for Sambo and = 1/4900 for all other bears. So the outcome model is only correct when Sambo eats all other bears except for crumbs. If we can anticipate this, then HT even in this sampling plan will do just fine.

 

]]> https://statmodeling.stat.columbia.edu/2026/09/22/survey-statistics-fat-bear-week-2026/feed/ 0 “Phonics is a fad, but it’s also good, so we should do it anyway, even though it will probably disappoint people” https://statmodeling.stat.columbia.edu/2026/09/22/phonics-is-a-fad-but-its-also-good-so-we-should-do-it-anyway-even-though-it-will-probably-disappoint-people/ https://statmodeling.stat.columbia.edu/2026/09/22/phonics-is-a-fad-but-its-also-good-so-we-should-do-it-anyway-even-though-it-will-probably-disappoint-people/#comments Tue, 22 Sep 2026 13:08:33 +0000 https://statmodeling.stat.columbia.edu/?p=54710 Continue reading →]]> Tyler Watts and Drew Bailey write:

Educational policy is often viewed as a field of fads, where one follows another for seemingly arbitrary reasons. We might like to believe that fad-based policy could be superseded by evidence-based policy, with rational actors choosing policies because of scientific evidence. But that’s been hard to achieve.

In the cynical version of the story, hucksters overpromise on their pet projects, we make the changes they advertise, and they get rich while the public is left disappointed. This probably happens more often than it should, but this narrative ignores the conditions that make such disappointment ubiquitous in educational policy.

Why? They explain:

Even if the hucksters all disappeared tomorrow, we’d still have this hype cycle, because overpromising seems inevitable when effects are small, and political disappointment will always follow. . . .

The educational policy research ecosystem usually selects for safe, often evidence-based, politically feasible, but ultimately oversold ideas, and we think that hurts our ability to make progress. In education, real effects are really small. . . .

Why are the effects so small? The outcomes we like to measure, such as reading comprehension, depend on many personal and environmental factors, making it hard for a discrete change in schooling to make much of a difference. . . .

This is a well-known problem in education research. For example, it came up in my 2018 talk at the Society for Research on Educational Effectiveness, and the point wasn’t new then either!

But, as Watts and Bailey explain,

The extreme education-nihilist position—that educational improvements can never affect students’ skills—is actually wrong. Endorsing it is likely to lead to poor education policy. The edu-nihilist position overlooks that small effects are not always zero effects. Small effects can actually be meaningful and, in some cases, worth it from a cost-benefit perspective for society.

One way to think about this is to consider classroom teaching.

I work my butt off every semester teaching my classes. Every class takes a lot of effort, both by me and the students. But any given class can’t have much of an effect on total learning. Indeed, the entire semester doesn’t have such a large average effect, when you consider that some students already could do most of it before the semester began, some still don’t get it when the semester is over, and others learn some things which they later forget. (On the plus side, students can learn all sorts of useful things that don’t happen to be on the exam.) But the effect of these classes is not zero. Students take them and they learn some things. So of course there’s no simple switch that will improve performance by 0.2 standard deviations. We’re already doing what we can. And the flip side of this is that a small improvement is not nothing. We teach each class knowing that at best it is giving a small average improvement, but it’s still worth doing.

Watts and Bailey continue:

When education policy researchers say something “works,” most people assume that effects are much larger than they are. . . . In most cases, researchers and policymakers fall prey to effect size magnification because they develop real sympathy for a pet program. When this happens, they put forward the most promising evidence, which is likely biased. The field then develops unrealistic expectations, and researchers, policymakers, and advocates downplay or withhold information about their uncertainty.

Hey, here’s an example from celebrated economist James Heckman, who doesn’t have much of an understanding of selection bias and goes around promoting wildly exaggerated estimates of early childhood programs.

Here are Watts and Bailey on what happens next:

Educational policies are uniquely poised to fall apart when overhyped, because public scrutiny of educational outcomes is so intense. Because most of the variance in student achievement comes from factors schools can’t control, overselling sets up an inevitable letdown. When disappointing results emerge and large gaps in achievement remain, we overreact to the apparent failure. Feeling the heat, the researcher, policymaker, or advocate might retreat and say that whatever was promised was never supposed to be a “panacea.” This always feels cheap because the language rarely shows up in the press releases, grant proposals, white papers, or speeches preceding the policy. Ultimately, we are left abandoning ship, even if the policy in question was actually helping.

The clearest example of this is probably the accountability reforms pushed by No Child Left Behind. Everybody hates No Child Left Behind because it was terrible and didn’t work! Right?

Actually, our best policy evaluations suggest it did work; it just had small effects, which is exactly what we should have expected. Instead of taking the small effects as a sign that something is working and figuring out ways to improve it, politicians and advocates decided it was a failure and moved on to other things.

And then they turn to the latest trend:

In a country that can’t agree on much, states from Alabama to California have adopted policies designed to promote the science of reading. . . . Count us as phonics believers. Kids need to know how to sound out words when they learn to read. We taught our own kids how to read this way, and we ourselves even learned to read this way in the pre-science-of-reading dark ages of the 1990s. But the unfortunate reality is that we have no reason to believe a sudden shift to a phonics-based curriculum will significantly affect reading outcomes in this country.

There are several reasons why.

First, meta-analyses of phonics interventions show fadeout. This means that kids who get phonics instruction for some period of time look a lot like kids who didn’t get phonics instruction in terms of reading just a year or two after the intervention ends.

Second, teaching phonics seems to help with decoding and word recognition. This is an important building block of literacy, but there isn’t much evidence that it helps with eventual reading comprehension—the ultimate goal. . . .

Finally, we are fooling ourselves if we actually believe that phonics hasn’t been used in classrooms up to this point. Do we really think that it never occurred to teachers in the “woke” states to help kids break words into syllables and sound them out? The counterfactual condition here has become a strawman. . . .

The best evidence on what happens when states adopt early literacy policies is probably this working paper, which finds effects of up to about .10 standard deviations . . . on high-stakes tests for 3rd graders (that’s good). They didn’t find much on low-stakes tests administered to 4th- or 8th-graders.

So, if we enter a halcyon period for phonics instruction, we might help a lot of kids learn basic decoding skills when they are first learning to read. But . . . effects will be smaller than we were led to believe. Importantly, they probably won’t move the needle much on outcomes like NAEP trends, which compare U.S. cohorts of 4th and 8th graders to each other. . . . The question is: how will we interpret this when the sobering news starts to come in?

Just in case we weren’t clear: doing more phonics in early-grade classrooms is, for the last time, probably a good thing.

They conclude:

Education policy is not the only field that faces conflicts between public and expert opinion. But the combination of big expectations, small effects, and constant attention to children’s educational outcomes means education policy is likely to keep experiencing these conflicts.

What can we do to mitigate this educational political economy problem? . . .

We should ask interested parties to make predictions on the record about how these children will differ (and by how much). And our society should reward researchers and policymakers whose mental models are closer to reality, rather than those who are best at selling potential solutions in the first place.

And my own thoughts

I agree with everything that Watts and Bailey wrote above, and also have a few things to add:

First, there’s my recent paper with Amy Krefman, Lauren Kennedy, and Jessica Hullman, Hypothesizing an effect size by considering individual variation, where we try to think systematically about how to get more realistic expectations of average treatment effects.

Second, a point from my above-linked SREE talk, that research should be less isolated from practice. From one direction, research studies should be more realistic, more connected to the real world of education—this is a point that Watts and Bailey make in their post. From the other direction, general practice should follow principles of research: students, parents, teachers, and administrators should be empowered to make choices and experiment, and, along with this, choices and outcomes should be measured so that effectiveness can be studied. We should all be trying to do better anyway, and we should design our data collection so we can all best learn from these experiences. As we discussed in a different context, local data collection, collaborative analysis, and local recommendations.

Third, ummm, this is a longer one so I’ll just quote myself:

You can conceptualize an education intervention as a vector, where the direction of the vector is the material being learned and the length of the vector is the amount that students are motivated to work. You want the material learned to be useful—you’d like the vector to have a positive “dot product” with the vector of skills, knowledge, and understanding that will be useful going forward—but, conditional on those two vectors being roughly aligned, the real gain is in the magnitude. And this magnitude will be an interaction between the teacher and student: there’s no button to push to create motivation, and if there were a button it would already have been pushed.

What I’m saying is that, when thinking about acupuncture, or physical therapy, or coaching, or teaching, we have to go beyond what I’ve called the penicillin model of science, the idea that innovations come from nowhere and that the job of statistics is to design and analyze experiments to reject the null hypothesis of no effect, and in which the treatment in such experiments is considered as a black box, with the goal being to estimate an average treatment effect.

I don’t think the penicillin model usually applies. Most of the time in health, education, and just about any field, improvements are incremental, and the goal is to improve the process while gaining understanding. Clinical trials and offline experiment both play a role, and you’re not going to learn much by studying a treatment as if it’s a black box.

This is not to say that there cannot be new developments in any of these fields, nor is it to deny that such developments can sometimes arise serendipitously. I just think that, in any case, you have to go beyond the average treatment effect and think about the mechanism of action.

]]> https://statmodeling.stat.columbia.edu/2026/09/22/phonics-is-a-fad-but-its-also-good-so-we-should-do-it-anyway-even-though-it-will-probably-disappoint-people/feed/ 32 Do high school students have to work harder nowadays? https://statmodeling.stat.columbia.edu/2026/09/21/do-high-school-students-have-to-work-harder-nowadays/ https://statmodeling.stat.columbia.edu/2026/09/21/do-high-school-students-have-to-work-harder-nowadays/#comments Mon, 21 Sep 2026 13:10:34 +0000 https://statmodeling.stat.columbia.edu/?p=53606 Continue reading →]]> I happened to come across this old comment thread where Richard Serlin wrote:

Starting early is huge. If you’re a super-workaholic, with great endurance and energy, since kindergarten you’re going to achieve far more by age 40 . . . working your ass off since kindergarten and getting into Harvard as an undergrad makes it much more likely that you get into an Ivy PhD program, and with the resources, help, and prestige you get there, much more likely that you get a job in an Ivy department, and with the resources, help and prestige you get there, . . .

To which I replied:

I’ve been working hard since I started college, that’s for sure. But before then, no, I didn’t work hard. In the U.S. system (at least as it existed when I was a kid), it was perfectly possible to coast up to the age of 17 as long as you had the right background and abilities.

So here’s my question. Do kids have to work harder in high school nowadays? (I think we can take Serlin’s “working your ass off since kindergarten” as hyperbole.) My friends and I did our homework in high school, and I did take some hard classes, but I didn’t actually work hard, setting aside the very occasional late hours when projects were due.

I do get the impression that kids spend more hours doing homework than we did 45 years ago, but if it’s everybody putting in these hours, then it’s not about workaholics. It’s kinda hard for me to compare, even when I see actual high school homework assignments.

]]> https://statmodeling.stat.columbia.edu/2026/09/21/do-high-school-students-have-to-work-harder-nowadays/feed/ 30 How did it go, this prediction from ten years ago about Japanese fertility in 2026? https://statmodeling.stat.columbia.edu/2026/09/20/how-did-it-go-this-prediction-from-ten-years-ago-about-chinese-fertility-in-2026/ https://statmodeling.stat.columbia.edu/2026/09/20/how-did-it-go-this-prediction-from-ten-years-ago-about-chinese-fertility-in-2026/#comments Sun, 20 Sep 2026 12:34:12 +0000 https://statmodeling.stat.columbia.edu/?p=53601 Continue reading →]]> From 2017:

total-fertility-usa-japan

Gaurav Sood points us to this post, “Why did so many Japanese families avoid having children in 1966?”, by Randy Olson, which includes the excellent graph above and the following explanation:

The Japanese use [an] . . . astrological system . . . based on the Chinese zodiac. Along with assigning an astrological beast based on your birth year, each year also has one of the Five Elements associated with it—all that dramatically affect what your astrological sign entails. . . .

In 1966, however, many Japanese families were still quite superstitious—and . . . 1966 was the year of 丙午 (Hinoe-Uma), or the “Fire Horse.” As one source describes:

Girls born in [1966] became known as ‘Fire Horse Women’ and are reputed to be dangerous, headstrong and generally bad luck for any husband. In 1966, a baby’s sex couldn’t be reliably detected before birth; hence there was a large increase of induced abortions and a sharp decrease in the birth rate in 1966.

Time will tell if superstition will strike again 10 years from now in 2026, the next year of the “Fire Horse” in its 60-year cycle. Given that Japanese is already below the replacement fertility rate (i.e., roughly an average of 2 children per woman), the result could be disastrous.

If you look carefully at the above graph, you’ll see increases during the years before and after 1966. So, while it does seem that the net effect on births is negative, it’s not quite so negative as the spike might make it appear, given that some births that would’ve occurred in 1966 have been displaced to the adjacent years.

And now 2026 has arrived!  When the year is over we can see what the birth rate in Japan looks like.

We’ll check back in on 6 Feb 2027.

]]> https://statmodeling.stat.columbia.edu/2026/09/20/how-did-it-go-this-prediction-from-ten-years-ago-about-chinese-fertility-in-2026/feed/ 14 The Immigration and Nationality Act of 1952 https://statmodeling.stat.columbia.edu/2026/09/19/the-immigration-and-nationality-act-of-1952/ https://statmodeling.stat.columbia.edu/2026/09/19/the-immigration-and-nationality-act-of-1952/#comments Sat, 19 Sep 2026 13:21:08 +0000 https://statmodeling.stat.columbia.edu/?p=53592 Continue reading →]]> I was reading the London Review of Books and came across this article by Jameel Jaffer stating that, in 1952,

Congress overrode the presidential veto and passed the McCarran-Walter Act with a two-thirds majority in the House and Senate. The law codified a system of immigration quotas that privileged applicants from Western Europe and almost guaranteed that those from Asia and Africa would be rejected. The section on activities ‘prejudicial’ to the US, together with others targeting suspected communists and anarchists, supplied a mechanism which government officials would use over the next decades to justify refusing visas to foreigners whose political views they didn’t like.

I had no idea. I knew that President Truman was very unpopular in 1952 and I knew there was lots of xenophobia, but, still, overriding a veto takes a lot of votes, and the Democrats had a majority in both houses in Congress.

Also I didn’t realize that there had been any major immigration bill since the act of 1924 that had drastically reduced immigration and the act of 1965 that opened things up again.

So I went to wikipedia to look up the bill, and, yeah, there it is:

Senator Pat McCarran (D-Nevada), the chairman of the Senate Judiciary Committee, proposed an immigration bill to maintain the status quo in the United States and to safeguard the country from Communism, “Jewish interests”, and undesirables that he deemed as external threats to national security. His immigration bill included restrictive measures such as increased review of potential immigrants, stepped-up deportation, and more stringent naturalization procedures. The bill also placed a preference on economic potential, special skills, and education. In addition, Representative Francis E. Walter (D-Pennsylvania) proposed a similar immigration bill to the House.

Parts of the story are kind of complicated:

The McCarran–Walter Act abolished the “alien ineligible to citizenship” category from US immigration law, which in practice applied only to people of Asian descent. Quotas of 100 immigrants per country were established for Asian countries; however, people of Asian descent who were citizens of a non-Asian country also counted towards the quota of their ancestral Asian country. Overall immigration from the “Asiatic barred zone” was capped at 2,000 people annually. Passage of the act was strongly lobbied for by the Chinese American Citizens Alliance, Japanese American Citizens League, Filipino Federation of America, and Korean National Association, though only as an incremental measure, as those organizations wished to see national origins quotas abolished altogether.

The total quota was 4720 per year from Africa, 3690 per year from Asia, 150,000 per year from Europe, and I think unlimited from the Western Hemisphere. Some famous people excluded under this law were Julio Cortázar, Michel Foucault, Dario Fo, Gabriel García Márquez, Doris Lessing, Pablo Neruda, and Graham Greene. Fair enough to exclude Greene, I guess; he was a British spy, after all. The others I think they should’ve let in.

I was wondering how that override vote happened, so I clicked the link:

The 113 Nay votes were concentrated in the northeast (New York, New Jersey, Pennsylvania, Connecticut, Rhode Island, and Massachusetts) and various other big cities around the country–I don’t have the maps at hand so I just have the districts within states, but I’m guessing that the patches of Nay votes in Ohio, Michigan, Missouri, Illinois, etc., correspond roughly to Cleveland, Detroit, St. Louis, Kansas City, Chicago, etc. There were only 4 Nay votes in the entire south.

How consequential was this 1952 law? I’m not sure. Aside from the annoying anti-free-speech aspect, it was pretty much a reaffirmation of the 1924 immigration act with very minor reforms. So the 1952 law didn’t do much on its own. I’m guessing its main effect was to preempt any more serious reform of the immigration system. That would have to wait until 1965. So I’m guessing that the main thing the proponents of this law got, beyond the immediate political victory of overriding a veto, was a 13-year delay.

P.S. The author of the above-linked article, Jameel Jaffer, works at Columbia! I guess I should look him up.

]]> https://statmodeling.stat.columbia.edu/2026/09/19/the-immigration-and-nationality-act-of-1952/feed/ 6 Meritocracy to-morrow and meritocracy yesterday – but never meritocracy to-day https://statmodeling.stat.columbia.edu/2026/09/18/meritocracy-to-morrow-and-meritocracy-yesterday-but-never-meritocracy-to-day/ https://statmodeling.stat.columbia.edu/2026/09/18/meritocracy-to-morrow-and-meritocracy-yesterday-but-never-meritocracy-to-day/#comments Fri, 18 Sep 2026 13:09:31 +0000 https://statmodeling.stat.columbia.edu/?p=53367 Continue reading →]]> Alice couldn’t help laughing, as she said, “I don’t want you to hire me – and I don’t care for jam.”
“It’s very good jam,” said the Queen.
“Well, I don’t want any to-day, at any rate.”
“You couldn’t have it if you did want it,” the Queen said. “The rule is, jam to-morrow and jam yesterday – but never jam to-day.”
“It must come sometimes to ‘jam to-day’,” Alice objected.
“No, it can’t,” said the Queen. “It’s jam every other day: to-day isn’t any other day, you know.” — Lewis Carroll.

We’ve written a lot about meritocracy recently:

– “Howard Lutnick gives top Cantor Fitzgerald jobs to his sons Brandon and Kyle” is a very clean example of meritocracy. I mean it. And not because I have any reason to think that Brandon and Kyle have any particular merit.

– Is the civil service a meritocracy? No, it’s merit-based employment. For it to be “ocracy,” the people with merit don’t just have to get the jobs, they also have to wield power.

– Living on the Edge: Aggressive mediocrity at the summit of America’s intellectual culture. Throw in an ambassadorship and a college presidency and you’ve hit the EGOT of intellectual celebrity culture for the talentless.

Again, meritocracy is not the same as merit-based employment. In meritocracy, the people with the merit (Howard Lutnick, etc.) get to run things, and one of the perks of merit is getting sweet jobs for their kids. There’s a thin line between meritocracy and nepotism . . . hey, here’s Bob Dylan’s talentless kid!

This came up in class the other day, and someone pointed out that meritocracy usually seems to be presented as an ideal more than as a reality.

To paraphrase the White Queen, “Meritocracy to-morrow and meritocracy yesterday – but never meritocracy to-day.”

“Meritocracy to-morrow” is the appeal of meritocracy as an ideal, happening in the future so you don’t have to worry about its current consequences such as the careers of Brandon and Kyle Lutnick. “Meritocracy yesterday” is the appeal of an idealized past, something that we supposedly used to have.

Just to be clear, I’m not saying that merit-based employment is impossible. The problem’s with the ocracy. Or, if you’re Brandon and Kyle Lutnick, it’s not a problem at all.

]]> https://statmodeling.stat.columbia.edu/2026/09/18/meritocracy-to-morrow-and-meritocracy-yesterday-but-never-meritocracy-to-day/feed/ 21 Scientific citations as an overwhelmed communication channel https://statmodeling.stat.columbia.edu/2026/09/17/scientific-citations-as-an-overwhelmed-communication-channel/ https://statmodeling.stat.columbia.edu/2026/09/17/scientific-citations-as-an-overwhelmed-communication-channel/#comments Thu, 17 Sep 2026 13:43:44 +0000 https://statmodeling.stat.columbia.edu/?p=53583 Continue reading →]]> Peter Dorman writes:

This looks like it’s up your alley. I find it interesting, but I’m not convinced by what he says about innovation. Of course, the issue is empirical, but I suspect it may be different for different sciences. In the work you’ve flagged in the social sciences, people are fishing for citations by cranking out apparent anomalies: “interesting if true“. it might be different in more theoretical fields.

Overall, this is the sort of stuff I was hoping science and technology studies would be generating when I first stumbled on it a few decades ago. Instead it got hijacked by critical theory, but that’s a topic for another day.

P.S. I’ve wondered why, in more scientific fields, it’s common to reference assertions with lots and lots of cites, not just one or two. My impression is that it’s less pronounced in the social sciences; at least, I started noticing it when I dove into the natural science literature for my climate book. Is that true, and if it is, why?

The above link is to a post, “How citations ruined science,” by David Oks. Here’s the summary:

There is nothing inherent about AI that makes scientists use it to produce slop; the essential problem is the incentive structures. The tidal wave of AI slop that is now threatening to engulf the institutions of science is, I suspect, a symptom of the citation regime. It would make no sense for a scientist in the 1950s or 1920s, when scientific communities were defined by personal reputation and mutual acquaintance, to submit the type of slop that is being submitted in huge quantities to scientific journals. But it does make sense within incentive structures that heavily reward publishing and getting cited.

This seems accurate to me.

I googled David Oks and he appears to be involved in public relations in some way. I wonder what he thinks of “AI slop” in that context. I ask because in advertising and public relations, I think the usual goal is not to produce anything novel or even interesting but rather to get noticed. This is related to what is called “the attention economy” but it’s centuries old, right? You put up big posters in the town square or newspaper ads or TV ads or whatever.

Another way of putting it is: For any communication channel there is some balance, some optimal level of flow of material. If the channel is unused, what’s the point; if it’s used too much, it will be manipulated.

One example is Pubpeer, the site for commenting on published scientific papers. It’s my impression that Pubpeer “works” in the sense that the comments there are often of high quality. But Pubpeer doesn’t get used very much–one way of seeing this is to note that a controversial paper that has been cited hundreds of times might only have a few comments on Pubpeer, or none at all. On the other hand, if Pubpeer did get used all the time, then I have a horrible feeling it would end up like Yelp and be swamped by fake reviews–or, to put it more precisely, reviews written for instrumental purposes to boost or knock a paper without supplying any scientific argument.

Every once in a while you can have a space like our comment section here which facilitates open discussion, but, again, once enough people are using a communication channel, there are increasing incentives to manipulate it, and the bad actors move in.

]]> https://statmodeling.stat.columbia.edu/2026/09/17/scientific-citations-as-an-overwhelmed-communication-channel/feed/ 26 It’s satisfying to see the economics profession come around on some things (regression discontinuity analysis and so-called risk aversion) https://statmodeling.stat.columbia.edu/2026/09/16/its-satisfying-to-see-the-economics-profession-come-around-on-some-things-regression-discontinuity-analysis-and-so-called-risk-aversion/ https://statmodeling.stat.columbia.edu/2026/09/16/its-satisfying-to-see-the-economics-profession-come-around-on-some-things-regression-discontinuity-analysis-and-so-called-risk-aversion/#comments Wed, 16 Sep 2026 13:27:32 +0000 https://statmodeling.stat.columbia.edu/?p=53582 Continue reading →]]> Jonathan Falk points to this post by Nicholas Decker and writes:

I thought you’d be interested in a couple of things in this. First, the regression discontinuity pictures and Decker’s parenthetical warning: “(The econometrics literature is emphatic on this – do not use higher order polynomials in a regression discontinuity. If you do, then you can get very large differences in apparent outcomes due to changes far away from the boundary – imagine if someone on the border of 44 flips the quadratic upside down, completely reversing your result!).” Lessons come through!

Second, his statement on risk theory — “As many people have noted (but see Chetty (2006) for a particularly elegant demonstration) estimates of risk aversion in different settings get extremely different answers. In particular, the coefficient of relative risk aversion comes from people having declining marginal utility of consumption. If you ask people in lab experiments about gambles, you’ll find that they’re really scared of risk, but if you observe people’s labor supply response to income changes, you’ll find things which imply they’re really risk tolerant.” I’ve never entirely understood your attitude on this other than your objection to particular functional forms, but the unease with which the profession has come around a bit towards your view should be welcome.

This is indeed satisfying.

And for those who are coming into this story in the middle, here are some references.

1. Problems with flexible adjustments in regression discontinuity:

2. Problems with naive ideas of risk aversion:

I won’t try to take too much of the credit for this progress in economics. I’m just glad to see it happening.

]]> https://statmodeling.stat.columbia.edu/2026/09/16/its-satisfying-to-see-the-economics-profession-come-around-on-some-things-regression-discontinuity-analysis-and-so-called-risk-aversion/feed/ 12 Charting the Agentic Garden of Forking Paths https://statmodeling.stat.columbia.edu/2026/09/15/charting-the-agentic-garden-of-forking-paths/ https://statmodeling.stat.columbia.edu/2026/09/15/charting-the-agentic-garden-of-forking-paths/#respond Wed, 16 Sep 2026 02:24:17 +0000 https://statmodeling.stat.columbia.edu/?p=54695 Continue reading →]]> Arjun Balaji, Batuhan Duru Yeltekin, and Tian Zheng write:

Even with a fixed dataset and research question, data analysis involves many defensible decisions. Understanding how these choices influence the results is scientifically important but remains challenging. Crowdsourcing and agentic AI can generate hundreds of end-to-end analyses, but scaling generation alone can create a processing bottleneck and an analytic “black hole.” A common workaround is to impose a shared fixed decision taxonomy, which can limit insight and understate uncertainty. We present ForkSCOPE, a human-AI collaboration framework that induces structure bottom-up from the code corpus of end-to-end analyses, without a taxonomy fixed before or after generation, so the organization and evaluation of the garden can scale with the corpus. ForkSCOPE surfaces the charted garden of forking paths through a human-AI collaboration pipeline and an evidence-linked interactive viewer for steering and verification: it spotlights organically identified forks and structures and produces a derived taxonomy and decision map compatible with existing multiverse tools.

I don’t know enough about chatbots to understand what’s going on here, but it’s an interesting idea to study forking paths in this way. Here’s a copy of the viewer in HTML, also there are some links here. Tian says her favorite functionality is the filter by keywords.

]]> https://statmodeling.stat.columbia.edu/2026/09/15/charting-the-agentic-garden-of-forking-paths/feed/ 0 Survey Statistics: ANOVA https://statmodeling.stat.columbia.edu/2026/09/15/survey-statistics-anova/ https://statmodeling.stat.columbia.edu/2026/09/15/survey-statistics-anova/#comments Tue, 15 Sep 2026 20:00:54 +0000 https://statmodeling.stat.columbia.edu/?p=54681 Continue reading →]]> Andrew recently answered “Why did ANOVA fall out of fashion?”:

Anova is still important; it’s just been subsumed by hierarchical models.

The link is to my 2005 paper, Analysis of variance: Why it is more important than ever

Folks discussed whether ANalysis Of VAriance refers to 1) testing the null hypothesis that group means are equal, 2) fitting a regression model with categorical predictors, or 3) “an add-on to regression analysis, in which the predictors are structured and we estimate the variances of batches of coefficients.”

How does ANOVA relate to Survey Statistics ?

Andrew’s 2005 ANOVA paper‘s example in Section 7.2 is the Multilevel Regression (MR) of MRP. They made a new graphical display of the standard deviations of each batch of effects to replace the classical ANOVA table:

This is useful to assess the relative importance of different sources of variation, which can guide which interactions to add into the model.

In his discussion of Andrew’s paper, Alan Zaslavsky (the inspiration for this series) gave another example: a survey of members of Medicare managed care health plans. They modeled the data with variance components for geographical units (region, state, Metropolitan Statistical Area or MSA) and for the health plans. They found that for ratings of doctors, the majority of the variance was explained by geography, not the health plans. This suggested that improving quality of care might need to be directed towards low-performing areas rather than low-performing health plans.

]]> https://statmodeling.stat.columbia.edu/2026/09/15/survey-statistics-anova/feed/ 2 “Protection from inappropriate influence is a hallmark of scientific integrity” . . . not any more! https://statmodeling.stat.columbia.edu/2026/09/15/protection-from-inappropriate-influence-is-a-hallmark-of-scientific-integrity-not-any-more/ https://statmodeling.stat.columbia.edu/2026/09/15/protection-from-inappropriate-influence-is-a-hallmark-of-scientific-integrity-not-any-more/#comments Tue, 15 Sep 2026 13:47:16 +0000 https://statmodeling.stat.columbia.edu/?p=54673 Continue reading →]]> This is a funny one. Remember how Google used to have the slogan, “Don’t be evil,” but then they abandoned it? Something similar seems to have happened with the U.S. Census Bureau.

Dale Lehman pointed me to the story:

I’m sure this is not news to you. But I don’t think the usual lag is appropriate given the seriousness of this.

From the linked item, by economist Justin Wolfers:

“Protection from inappropriate influence is a hallmark of scientific integrity.” That line governed the Census Bureau and then the White House quietly disappeared it. . . .

Three weeks ago, I would have told you that the American government couldn’t suppress economic data like this. But under the new rules that the administration just put forward, the White House can direct its political operatives to kill uncomfortable numbers. . . .

Three weeks ago the United States had a policy that protected the free flow of scientific information and barred political meddling in how Census Bureau findings are presented. Today, it doesn’t.

To be clear: I’m not (yet) saying that our government is cooking the statistical books. Instead, I’m saying that it took the necessary steps to make that legal, and possible. In recent weeks, the Trump administration quietly rewrote the rules that kept political interference out of our economic statistics. And almost nobody noticed.

I didn’t notice either! Contra Lehman’s assumption, this was news to me. I guess that’s how news spreads now: Wolfers posts this on his blog, Lehman sees it and emails the link to me, I post it here and you see it, etc. Sometimes Tyler Cowen or Alex Tabarrok notices items from this blog and they post it on Marginal Revolution. Or maybe Krugman will post on it. The New York Times or Wall Street Journal might cover it too.

What I kind of wonder is how Wolfers heard about it. He’s a user of public data, so it makes sense that he’d be directly interested in the topic. But how did he find out about this obscure announcement? Was he just rooting through the census website one day and noticed something different? Did he see this on some news site? Was it sent to him by an anonymous tipster (I get lots of that sort of email!)? I’m curious. Wolfers does point to this NPR report from last month, so my guess is that someone noticed it and pointed Wolfers to it.

From the NPR article:

The Commerce Department updated its policy for promoting a “culture of scientific integrity” in January 2025, days before Trump’s second inauguration.

A new Trump administration order bans the Census Bureau and the Bureau of Economic Analysis from using statistical “noise,” or data for fuzzing survey results, to protect people’s privacy in their statistics.

The 2025 version included bans on attempts to shape or interfere in data collection, statistical analysis and other scientific activities “against well-accepted scientific methods and theories or without scientific justification.”

The department’s policy, the 2025 version said, involves ensuring scientific findings at its agencies “are not suppressed, delayed, or altered for political purposes and are not subjected to inappropriate influence,” as well as making sure there’s “independent review of facilities, methodologies, and other scientific activities as appropriate to ensure scientific integrity.”

That language is missing, however, in the Trump administration’s revision of the policy, according to an NPR analysis of text posted on the Commerce Department’s website.

And while the revision adopts a definition for “scientific integrity” that is similar to the previous version, it no longer says that “protection from inappropriate influence” is among the “hallmarks of scientific integrity.”

Spokespeople for the Commerce Department and Census Bureau did not respond to NPR’s requests for comment.

I guess the strategy of dodging comments worked, at least for awhile, because for three weeks nobody else seemed to have picked up on this story.

NPR continues:

While most of the public may not be familiar with the policy, Nancy Potok, a former deputy director at the bureau who has also served at the Commerce Department, sees the removed language as “a major change that eliminates the guardrails against political interference in the statistical and scientific data products of the department.”

Potok has been tracking how statistical agencies have been responding to a years-long, bipartisan effort to make sure the federal government produces trustworthy information. That included a 2018 law that led to a rule known among federal data experts as the “Trust Regulation.” The rule requires that statistical agencies “determine the policies and practices that ensure objectivity of its statistical activities” and produce data that is “impartial and free from undue influence and the appearance of undue influence.”

“The fact that there is a conscious effort to revise the department administrative orders and not incorporate things that are in the [Trust Regulation], instead of going in the other direction, is definitely troubling and needs some explanation,” Potok adds.

Why did the government do this?

For former Census Bureau Director Robert Santos, a Biden nominee, the close timing of the policy revision and the bureau’s release of an unauthored report with claims about noncitizen voting is striking. “It’s entirely possible that this was done to reconcile the Department of Commerce and the administration’s own actions by publishing an external partisan document about voting on the Census Bureau website,” Santos says.

And Wolfers brings on the indignation:

On August 19, the Commerce Department posted a new directive called “Scientific Integrity” — note the scare quotes. . . . They posted this directive very quietly. There was no press release, no announcement, and no press conference. For the first time in his life, Howard Lutnick was quiet. The new version just appeared online. To analyze the changes, I [Wolfers] actually had to go to the Wayback Machine to find the prior version of this policy, because it had been wiped from the Commerce website. There’s something pretty bleak about a country’s rules about scientific integrity surviving only because a volunteer internet archive grabbed a copy.

The Jan 2025 rules are here, and the new rules are here.

What do I think about all this? I’m not sure.

On one hand, the above story sounds horrible, and I think it’s disgraceful how government officials have been throwing around slogans such as “gold standard science” while suppressing reports, promoting discredited or flat-out fake science, and adding corruption to the science funding process, while their friends in the media are resorting to ridiculous defenses of indefensible policies.

On the other hand, this updated document could just be some overly busy bureaucrats trying some stupid idea. Also, if that 2025 policy was such a good idea, why did they wait until 2025 to institute it?

If “Protection from inappropriate influence is a hallmark of scientific integrity” is such an important slogan, where was it in the Census Bureau’s first 235 years? There must be more to this story.

On the whole, though, I guess I’d go with Noem’s Razor here: Never attribute to stupidity that which is adequately explained by malice.

In short: I think it’s reasonable to assume that this new document was motivated by some inappropriate reason, such as a desire to steal or to cover up some past or anticipated future offense, or from a general ideological position opposing objective knowledge and open scientific inquiry, or from a political attitude that independent science should be weakened because it represent a source of power that’s not under control of the government or its allies. Also, though, I guess they’re stupid. Just like the Soviets under Lysenko, an analogy that keeps coming to mind.

There’s an alternative explanation that this is all just a bunch of paperwork jockeys trying to justify their own existence, but Noem’s Razor suggests otherwise.

]]> https://statmodeling.stat.columbia.edu/2026/09/15/protection-from-inappropriate-influence-is-a-hallmark-of-scientific-integrity-not-any-more/feed/ 57 The Washington Nationals are hiring! https://statmodeling.stat.columbia.edu/2026/09/14/the-washington-nationals-are-hiring/ https://statmodeling.stat.columbia.edu/2026/09/14/the-washington-nationals-are-hiring/#respond Tue, 15 Sep 2026 02:17:33 +0000 https://statmodeling.stat.columbia.edu/?p=54690 Continue reading →]]> Des McGowan writes:

I left the Mets last year to take a job as the Amateur Scouting Director with the Washington Nationals. Now that we’ve gotten through our first draft and trade deadline, we’re looking for two Senior Analysts who can help us take our modeling efforts to the next level. One of these new hires will focus on Amateur player projections and the other will focus on Pro player projections. Together they will play an important part in determining the players we draft and the trades/free agent signings we make.

The postings are below:

Senior Analyst, Player Projections – Amateur Acquisitions

Senior Analyst, Player Projections (Pro Player Focus)

We are also hiring a Director for the Player Projections group. This individual would need to have extensive baseball modeling experience, but I know that many such people visit your site so figured I’d pass along just in case.

Sounds like fun!

]]> https://statmodeling.stat.columbia.edu/2026/09/14/the-washington-nationals-are-hiring/feed/ 0 New York City Psychometrics Group https://statmodeling.stat.columbia.edu/2026/09/14/new-york-city-psychometrics-group/ https://statmodeling.stat.columbia.edu/2026/09/14/new-york-city-psychometrics-group/#comments Mon, 14 Sep 2026 19:00:59 +0000 https://statmodeling.stat.columbia.edu/?p=54685 Continue reading →]]> [Edit to correct the attribution. Sorry about that! Bob.]

Klint Kanopka (NYU) put together a New York City Psychometrics Group, which hosts events. They have a web site with more information

There are three events planned so far for this semester:

  • September 25, 2026 (Friday) 3:00-5:00pm. Fordham University Lincoln Center (Lowenstein Cafe Atrium). Welcome Back Social
  • October 14, 2026 (Wednesday) 5:00-7:00pm. Fordham University Lincoln Center (12th Floor Atrium). 2026 Anastasi Distinguished Lecture by David A. Kenny with reception to follow
  • December 11, 2026 (Friday) 5:00pm. NYU Kimball Hall. Invited Lecture by Ben Domingue, with conversation, beverages, and leisure to follow

P.S. Betsy McCoach did such a good job advertising these events, two of which are at Fordham, I wrongly presumed she was the main organizer.

]]> https://statmodeling.stat.columbia.edu/2026/09/14/new-york-city-psychometrics-group/feed/ 1 This is how we do modern frequentist statistics: Using fake-data simulation to understand what can happen in a study https://statmodeling.stat.columbia.edu/2026/09/14/this-is-modern-frequentist-statistics-using-fake-data-simulation-to-understand/ https://statmodeling.stat.columbia.edu/2026/09/14/this-is-modern-frequentist-statistics-using-fake-data-simulation-to-understand/#comments Mon, 14 Sep 2026 13:19:42 +0000 https://statmodeling.stat.columbia.edu/?p=54675 Continue reading →]]> In a famous (to readers of this blog) example of statistical error, a researcher reported that beautiful parents were 36% more likely to have girl babies. The saps at Freakonomics fell for this one hook, line, and sinker, but I was suspicious, being a bit familiar with the sex ratio literature, which has found that the proportion of girl births is very stable at around 48.5%-49% in various subpopulations (whites and blacks, younger and older mothers, etc.).

The published analysis compared the sex ratio of children of the “very attractive” parents to all others, and the difference was 8 percentage points—not the “36 percent” mistakenly reported in the journal article and credulously repeated by Freakonomics, but still about 100 times larger than any plausible effect.

But here are the data, which look kind of convincing:

This comes from a survey of 3000 people and, as you can see, the percentage really is highest in that “most attractive” category; indeed, the comparison has a p-value of less than 0.05, hence the result being published.

OK, the result has no scientific plausibility given the vast literature on the topic (large effects of this sort are only found with studies with small samples), also the statistical significance is entirely explainable by forking paths—the odd choice of comparing category 5 with 1 through 4, rather than comparing 4 and 5 with 1 through 3, or just running a regression:

That slope estimate is not statistically significant; in any case the estimate is too noisy to be useful, given the range of realistically possible effect size. It’s the kangaroo problem.

But let’s set all that aside. Let’s forget all our subject-matter understanding and statistical expertise.

Here’s my question. Is there a way that a researcher without that specialized knowledge could see the problem with this study?

The answer, I think, is Yes. And the method is fake-data simulation. Which we could also call simulated-data experimentation. Or bootstrapping. With the only difference that, conventionally, the bootstrap is used as a way to get a bias correction or uncertainty estimate for an interval, and here we’re using it to develop intuition about a statistical data-collection process.

We would like to understand the statistical properties of the beauty-and-sex-ratio study by simulating hypothetical replications. In this case, the data came from 2792 participants who had at least one biological child in Wave III of the National Longitudinal Study of Adolescent to Adult Health. The full sample had 4877 respondents, of whom 2% were characterized as “very unattractive,” 5% as “unattractive,” 45% as “about average,” 37% as “attractive,” and 11% as “very attractive.”

We simulate replications under a null model. Assuming the probability of a girl birth is 0.488, independent of parental attractiveness, we simulate the results of 2792 births with proportions in the five attractiveness categories as given above.

Here are 20 simulated datasets, with for convenience the fitted least squares line displayed for each:

We see patterns just as dramatic as that of the observed data shown earlier, indicating that those data should not be taken as evidence against the null model.

Now here’s the point. The wonderful thing about these simulation experiments is that they can reveal problems with a naive design, even if you didn’t anticipate any difficulties ahead of time. The original author of that paper could have saved all of us a lot of trouble by simulating 1000 replications of those survey data on the computer, either before or after he performed his data analysis. No math required, no subject-matter knowledge required.

It’s too late for him, but it’s not too late for you to do this for your next modeling and analysis problem and avoid the embarrassment.

This is, in a very real sense, frequentism. To me, “frequentism” is not about unbiased estimators or long-term coverage or whatever; it’s about thinking of the data you see as one draw from a distribution of possible realizations of the data. It’s about looking at this distribution, both to see how your data and inferences could fluctuate in the future, and to interpret the data you do see.

Why don’t people do this all the time?

One reason, I think, is that they have a naive understanding of statistical theory and think that if you act like statistical significance = truth then you’ll be ok 95% of the time. They don’t check because they think there’s nothing to check, or maybe it’s more accurate to say they think that the experts have already checked for them. They don’t check their statistical analysis any more than I check the cables every time I get on an elevator; I assume the fundamental questions of elevator safety have all been settled already.

The other reason is that simulations is that they take effort. A simulation experiment requires a fully generative model—a rule for defining the truth and simulating data from some specified random process—followed by analysis of the simulated data, all nested within a loop and ending with comparison of inferences to truth. This involves additional work compared to that required to conduct an experiment, first because it requires an automatic procedure for data analysis and second because it requires a generative model. We have found that this additional effort involved in constructing a generative model and automating the data-analysis process is itself helpful for thinking through the experimental process. Indeed, it has similarities to the steps of preregistration.

This example and discussion are in Section 10.5, “Simulated-data experimentation as virtual replication,” of our Bayesian Workflow book. But I’ve never published it as a standalone article or blog post before, and the message is so clear that I wanted to share it here with you.

When people (including me!) get things wrong, it’s salutary not just to figure out where they went wrong and how it happened, but also to step back and see if there’s some more general process they could’ve followed that would have flagged the problem. And here we can do so.

]]> https://statmodeling.stat.columbia.edu/2026/09/14/this-is-modern-frequentist-statistics-using-fake-data-simulation-to-understand/feed/ 53 What’s happening with the models of the Atlantic Meridional Overturning Circulation? https://statmodeling.stat.columbia.edu/2026/09/13/whats-happening-with-the-models-of-the-atlantic-meridional-overturning-circulation/ https://statmodeling.stat.columbia.edu/2026/09/13/whats-happening-with-the-models-of-the-atlantic-meridional-overturning-circulation/#comments Sun, 13 Sep 2026 13:41:44 +0000 https://statmodeling.stat.columbia.edu/?p=53577 Continue reading →]]> John “not Towering Inferno” Williams points to this new research article by Valentin Portmann et al., which states:

Climate models show considerable discrepancies in their future projections around the Atlantic, mainly due to uncertainties in the fate of the Atlantic Meridional Overturning Circulation (AMOC). Climate models suggest a reduction in AMOC strength of 32 ± 37% by 2100 (90% probability) . . . To refine this estimate and reduce its uncertainty, we use four different observational constraint methods. The best one, which provides the lowest leave-one-out error, integrates a large set of observable variables . . . It gives an estimate of the AMOC slowdown of 51 ± 8% (90% probability) . . .

Without knowing any of the details, this looks wrong to me. There’s a lot of uncertainty about AMOC, right? So how to you get to a slowdown of 51 +/- 8%? That just seems unrealistically precise.

The abstract continues:

This refinement mainly results from correcting a bias in South Atlantic surface salinity, consistent with recent studies emphasizing its role in the proximity to an AMOC tipping point.

OK, that’s fine, but then the key issue is not the statistical method for model averaging, it’s this particular model correction.

The paper’s kind of hard for me to read, paradoxically because it’s all about statistics. I’m reminded of something I heard back when I was a Ph.D. student, which is that the best statistical methods are invisible: the ideal is for applied researchers to be talking about the science, not about the statistics. That’s one good thing about Bayesian methods. Rather than arguing about the estimator, you’re arguing about the generative model (the data model and the prior distribution), which puts you in the realm of the science. I’d much rather have researchers talking about plausible values for parameters and predictions than talking about significance thresholds and rejection rates.

That said, the most important thing about a statistical method is not what it does with the data but rather what data it uses. So if the methods promoted in the paper under discussion allow the incorporation of additional information, that’s good.

According to the article:

Methods called emergent or observational constraint (OC) have been developed to reduce the model uncertainty of a future climate variable of interest, hereafter called projected variable. These methods constrain the estimate and model uncertainty of the projected variable using the real-world observations of one or more observable variables. This results in a constrained model uncertainty that is smaller than unconstrained one.

I see this and I’m like, Huh? They weren’t doing this already? If you have “real-world observations of one or more observable variables” that are relevant to your predictions, then, yeah, you should use this information.

Although I’m not really clear on what is meant by “real-world observations of one or more observable variables.” Isn’t that just the same as “observations”? If you’ve observed something, it’s in the real world, right? And anything you’ve observed is, by definition, an “observable variable,” right? As Bob would say, there are some difficulties of communication here.

As I said, I don’t know anything about the models or the data here. As a human, I’m concerned about the potential catastrophic effects of the decline of AMOC, and I guess that I’d go with the consensus forecasts and uncertainties. If it’s really true that there are important observational data not included in the standard approach, then I hope someone can write a paper explaining that directly.

]]> https://statmodeling.stat.columbia.edu/2026/09/13/whats-happening-with-the-models-of-the-atlantic-meridional-overturning-circulation/feed/ 12 Annals of hype: Did this discovery in 2012 mark “the discovery of the final laws of nature . . . a discontinuity in human intellectual history, the sharpest that has occurred since the beginning of modern science in the 17th century?” https://statmodeling.stat.columbia.edu/2026/09/12/did-2012-mark-the-discovery-of-the-final-laws-of-nature-a-discontinuity-in-human-intellectual-history-the-sharpest-that-has-occurred-since-the-beginning-of-modern-science-in-the-17th-century/ https://statmodeling.stat.columbia.edu/2026/09/12/did-2012-mark-the-discovery-of-the-final-laws-of-nature-a-discontinuity-in-human-intellectual-history-the-sharpest-that-has-occurred-since-the-beginning-of-modern-science-in-the-17th-century/#comments Sat, 12 Sep 2026 13:58:51 +0000 https://statmodeling.stat.columbia.edu/?p=53576 Continue reading →]]> Here’s Ed Regis in 1993, reviewing a book from that year by physicist Steven Weinberg:

Currently, the most favoured explanation for the electroweak asymmetry involves the postulation of a new elementary particle, the so-called Higgs boson . . . One of the reasons why Weinberg and other elementary particle theorists so much want to see the [Superconducting Supercollider] built is that its energies will be high enough to produce the Higgs particle, supposing it exists. . . . Weinberg is a master at communicating the excitement of the quest for it . . . ‘The discovery of the final laws of nature will mark a discontinuity in human intellectual history, the sharpest that has occurred since the beginning of modern science in the 17th century.’

A few years ago they did find compelling evidence for the Higgs boson and it was news, but nobody said it was the end of the line, the final law of nature, or a discontinuity in human intellectual history.

Weinberg’s no longer around so we can’t ask him, but . . . back in 1993 was he really thinking that establishing the existence of the Higgs boson would represent “the discovery of the final laws of nature”? Like, now we know what all the particles are and we’re done? Full credit to Regis for not falling for the hype.

]]> https://statmodeling.stat.columbia.edu/2026/09/12/did-2012-mark-the-discovery-of-the-final-laws-of-nature-a-discontinuity-in-human-intellectual-history-the-sharpest-that-has-occurred-since-the-beginning-of-modern-science-in-the-17th-century/feed/ 22 Why I don’t like highest posterior density (HPD) intervals https://statmodeling.stat.columbia.edu/2026/09/11/why-i-dont-like-highest-posterior-density-hpd-intervals/ https://statmodeling.stat.columbia.edu/2026/09/11/why-i-dont-like-highest-posterior-density-hpd-intervals/#comments Fri, 11 Sep 2026 19:46:52 +0000 https://statmodeling.stat.columbia.edu/?p=54670 Continue reading →]]> This post is by Bob

There are several reasons I prefer central intervals.

First, you can’t tell how much of the probability is above or below a highest posterior density interval. For example, if I look at the HPD for an exponential distribution, it will run from 0 to the inverse cdf of the interval probability. If I look at the HPD for a normal, it matches the central interval.

Second, the range is going to be parameterization dependent. If I parameterize a probability in terms of log odds in (-infinity, infinity) vs. a log probability in (-infinity, 0) versus a probability (0, 1), I get different HPDs. Similarly, you’ll get different intervals for standard deviations vs. variance (even accounting for the squaring). In contrast, quantile-based intervals like central intervals are invariant to changes of variables.

Third, you don’t even get an interval if the highest density regions are disconnected, as in something like a Beta(0.5, 0.5) distribution. There, the highest density regions are from 0 up and from 1 down, so any highest density region is going to exclude the central interval of lowest density.

There is a lot of confusion about probabilities vs. densities in the wild. For instance, here’s a document from RStudio: Highest Posterior Density (HPD) Intervals. The document defines quantile-based (e.g., central) credible intervals in the usual way. Then it makes the common mistake of conflating density and probability, with the claim, “However, this definition does not guarantee the interval contains the highest probability regions.” Both the 90% HPD and 90% central interval contain exactly the same amount of probability mass, namely 90%. What this means is that the probability that a random draw falls in either is the same—90%. What the authors presumably meant is that central intervals don’t necessarily contain the highest density points. I think this confusion between probability and density is a lot of what’s motivating HPD. And also, the highest density points are going to depend on the parameterization (my second objection above).

On the other hand, Andrew’s the one who was pushing for replacing central intervals with HPD intervals in Stan, so I’m curious what his motivation for that change was given that he’s definitely not confused about probability versus density.

]]> https://statmodeling.stat.columbia.edu/2026/09/11/why-i-dont-like-highest-posterior-density-hpd-intervals/feed/ 18 “Belief” is not something you have; rather, it’s a relationship between thought and action. https://statmodeling.stat.columbia.edu/2026/09/11/belief-is-not-something-you-have-rather-its-a-relationship-between-thought-and-action/ https://statmodeling.stat.columbia.edu/2026/09/11/belief-is-not-something-you-have-rather-its-a-relationship-between-thought-and-action/#comments Fri, 11 Sep 2026 13:48:26 +0000 https://statmodeling.stat.columbia.edu/?p=53575 Continue reading →]]> I came across this piece by John Ziman from 1989, in the middle of the “cold fusion” hoopla:

When scientific controversies erupt like this, one must obviously try to grasp the essential points at issue. The mere fact that these points are seriously disputed affects one’s world picture. But if one is not then prepared to participate fully in the dispute, the proper scientific attitude is temporary suspension of belief-forming activity. Most of the philosophies of science are weak in treating ‘belief’ as something one ‘has’, instead of seeing it as a relationship between thought and action. Even the sociological norm of ‘organised scepticism’ suggests a more decisive role for active thought than a situation might demand.

I like that: “belief” as a relationship between thought and action.

This is related to my preference for the term “prior information” rather than “prior belief” in Bayesian inference. It’s not just a choice of words; it also has applied relevance, as demonstrated for example in several examples in the first chapter of our Bayesian Data Analysis book.

]]> https://statmodeling.stat.columbia.edu/2026/09/11/belief-is-not-something-you-have-rather-its-a-relationship-between-thought-and-action/feed/ 23 When journals are slow to implement or enforce policies to manage sloppy, majority AI-authored papers, reviewers can and should push back https://statmodeling.stat.columbia.edu/2026/09/10/when-journals-are-slow-to-implement-or-enforce-policies-to-manage-sloppy-majority-ai-authored-papers-reviewers-can-and-should-push-back/ https://statmodeling.stat.columbia.edu/2026/09/10/when-journals-are-slow-to-implement-or-enforce-policies-to-manage-sloppy-majority-ai-authored-papers-reviewers-can-and-should-push-back/#comments Thu, 10 Sep 2026 15:26:30 +0000 https://statmodeling.stat.columbia.edu/?p=54668 Continue reading →]]> This is Jessica. If you’ve ever reviewed a heavily AI-generated paper, you know how unrewarding it can be. For example, last week I spent a morning reviewing a paper for a  well-respected general science journal, which sounded interesting but was ultimately very marginal in terms of the delta over prior work. It had all the signs I’ve come to associate with heavy AI reliance–overloaded terms invented as unwarranted shorthand for basic statistical concepts; dense, fluent but vague writing, with lists of details and implied comparatives I couldn’t resolve even with repeated rereading; and a very incremental result—not a bad idea, just not a complete thought. It nonetheless took me over an hour and half to identify what they were trying to do and write up my thoughts in a review. 

I’m now getting requests to review such papers a couple times a week–not from minor journals but from high impact, general science venues. The majority of the abstracts I’m sent these days get flagged as 100% AI generated on Pangram, which in my experience doesn’t happen if you are exerting even a small amount of effort to control what is said. I shouldn’t be surprised; I know of junior researchers submitting tens of papers a year, and some senior ones too. But I am surprised nonetheless, that I’m being asked so often by widely respected venues to give feedback to authors who have given me little reason to believe they even understand what they are submitting. Of course AI writing is not a perfect signal of low author involvement, and LLMs can lead to much better science and better writing when used well. But see enough of these mostly AI-written papers fall apart upon closer inspection and you start becoming skeptical of anything that’s flagged as mostly AI text.

The problem is the seemingly poor enforcement of policies that these journals have put in place to avoid low human involvement papers. From a quick check, many of the top general science journals have policies about human accountability (some of which are pretty complex, like Nature’s) and most require disclosure of AI use. In some cases, substantial AI generation is prohibited (like Royal Society). But my experience with review requests from such journals and what I’m hearing from others suggests these policies are not weeding out enough of the slop. I also know that as an AE for Science Advances, I have seen a fair amount of heavily AI-authored text but have yet to see an author mention AI use, though technically our policy requires this in the paper and cover letter. In trying not to overindex on writing, I’m pretty sure I’ve sent out at least a few sloppy papers for review. With their more polished surface and layers of technical detail, low quality AI-authored papers are harder to catch on a quick skim than human-generated ones. And so external reviewers are stuck doing much of the sorting that these policies are intended to take care of. 

To better allocate my own attention as an external reviewer, I’ve started saying no to review requests when I’m pretty confident that I’m being presented with majority or fully AI-generated text. There are just too many papers to review for me to keep making time for those with little observable signal of human involvement. 

And while I don’t think AI writing detection is a long term solution, it’s increasingly seeming that we may need to add some friction to the system to push venues to experiment with policies that could filter more effectively. So I wrote this reviewer statement with Auyon Siddiq, in the interest of normalizing reviewers’ exercising their autonomy and signaling to venues that better policy is needed. 

The hope is that by encouraging researchers to assert their agency, it might help incentivize venues to move beyond the blanket “please disclose AI” statements that they don’t seem to be enforcing.

The main risk is that if many people start refusing to review on these grounds, it could make peer review much noisier temporarily. I personally think it’s worth the risk. If you use AI and detectors a lot, you get pretty good at telling when the writing involved little human effort. If you don’t use AI or detectors, you may be doing worse than chance. But if journals remain lax, we’ll likely end up with a lot of reviewers doing this kind of thing anyway. In the meantime, I recommend that reviewers take responsibility for becoming very familiar with the methods that exist, their limitations, and when they can be used without violating reviewer contracts.

There’s a much wider range of mechanisms journals could consider now: quotas on how many papers an author can submit in a given time frame, reciprocal reviewing requirements with desk rejection or future bans on submitting for non-compliance, higher standards for the clarity of writing, use of preliminary AI review to detect major issues, and so on. ML venues have been experimenting with most of these already. None of it is ideal, but when the old methods aren’t standing up to the scale of production that’s now possible, something needs to change. If we feel strongly that science requires human accountability, we should make it harder for authors to abuse the system.

P.S. Thanks to Aaron Clauset, Annie Liang, Eytan Adar, and Andrew for feedback on the statement.

]]> https://statmodeling.stat.columbia.edu/2026/09/10/when-journals-are-slow-to-implement-or-enforce-policies-to-manage-sloppy-majority-ai-authored-papers-reviewers-can-and-should-push-back/feed/ 21 Why I still like Orwell’s “Politics and the English Language” essay (even though he gets some of the linguistics wrong) https://statmodeling.stat.columbia.edu/2026/09/10/why-i-still-like-orwells-politics-and-the-english-language-essay-even-though-he-gets-some-of-the-linguistics-wrong/ https://statmodeling.stat.columbia.edu/2026/09/10/why-i-still-like-orwells-politics-and-the-english-language-essay-even-though-he-gets-some-of-the-linguistics-wrong/#comments Thu, 10 Sep 2026 13:35:59 +0000 https://statmodeling.stat.columbia.edu/?p=53572 Continue reading →]]> Brett Reynolds writes:

I liked the two-column move in your Orwell post. It seems genuinely useful.

I wondered, though, about the choice of Orwell as patron saint for the argument. The central point in your post seems stronger than Orwell’s essay itself, which bundles a real concern about euphemism and overclaiming with a lot of shaky style doctrine about passives, short words, cliché, and so on. My sense is that your argument works best as a claim about aligning evidence and rhetoric, not as an Orwellian one.

Reynolds also links to this article by Geoffrey Pullum, “Fear and loathing of the English passive.”

Here’s my response:

Orwell was just one guy. I’d say he was one of the great writers of the century, with two memorable novels (Animal Farm and Nineteen Eighty-Four), two important political books (The Road to Wigan Pier and Homage to Catalonia), and a range of unforgettable literary and political essays. All these remain readable and relevant today. He’s not a perfect writer, but nobody is. William Faulkner’s great; he also wrote some gobbledygook. Not all of Fitzgerald’s paragraphs were gold; Hemingway had his share of duds, etc. The characters in Animal Farm and Nineteen Eighty-Four are not fully-realized characters—Orwell was no Edith Wharton—but characters aren’t not the only point of a novel. There are many ways to make a contribution. Ozzie Smith wasn’t much of a slugger but he still belongs in the Hall.

Orwell’s famous (and, I would say, wonderful) essay, Politics and the English Language . . . it’s just something he wrote. It’s not perfect. It’s more polished than a blog post, but there’s nothing thorough about it, and I can well believe that he got a lot of the linguistics wrong. Orwell was giving his impressions based on his experience as a writer.

Maybe the relevant question is, What should Orwell’s essay be compared to? Or, What’s the audience?

Compared to standard writing advice (the “five-paragraph essay” and all the rest), Politics and the English Language is fresh and liberating, both in its exhortations to write directly and clearly and in its connection of writing to politics, conveying the sense that writing can be a deeply moral activity.

And the connection between writing and politics is subtle. Orwell was writing at a time when there was a lot of pressure for people to write lies or to write in obfuscating ways in order to serve some political agenda. Orwell’s way of connecting writing to politics was different and more subtle than that.

I think these messages remain important today, both in political and in scientific writing, as discussed in my earlier post. So I value Orwell, even if he gave some specific advice that made no sense and didn’t understand about the passive voice and just overall had naive views about linguistics.

Maybe a good analogy is: George Orwell is to linguistics as Nate Silver is to statistics. Orwell had a deep understanding backed up by years of practice, and was able to communicate this understanding to millions of people. Go granular on Orwell and you’ll see lots of mistakes and lots of overconfidence on his part, as well as a dismissal of or annoyance with experts, but still a net positive and with important insights that experts can value as well. Similarly with Nate Silver, who’s done excellent statistical work in both politics and sports, and, yes, sometimes he gets things wrong and isn’t interested in learning from his mistakes, and that annoys me, and sometimes his thoughts on political science are not fully thought through—but he offers a unique perspective and his writings are valuable, not just for outsiders but for experts as well.

]]> https://statmodeling.stat.columbia.edu/2026/09/10/why-i-still-like-orwells-politics-and-the-english-language-essay-even-though-he-gets-some-of-the-linguistics-wrong/feed/ 8 Living on the Edge: Aggressive mediocrity at the summit of America’s intellectual culture https://statmodeling.stat.columbia.edu/2026/09/09/living-on-the-edge-aggressive-mediocrity-at-the-summit-of-americas-intellectual-culture/ https://statmodeling.stat.columbia.edu/2026/09/09/living-on-the-edge-aggressive-mediocrity-at-the-summit-of-americas-intellectual-culture/#comments Wed, 09 Sep 2026 13:44:20 +0000 https://statmodeling.stat.columbia.edu/?p=52901 Continue reading →]]>

So much good stuff here at the Edge Foundation and Jeffrey Epstein webpage:

– The connection to Ted talks. I had no idea! But it fits. Money and bullshit go together like liver and onions.

– “Bill Gates and his wife.” What is this, Gilligan’s Island??

– I’d never heard of Jesse Dylan or Jeff Skoll so I googled them. Skoll’s a standard-issue internet billionaire: he made a bunch of money from Ebay and now makes investments. But Jesse Dylan is more interesting–or, I should perhaps say, less interesting. From Wikipedia:

Jesse Byron Dylan is an American film director and production executive. He is the founder of the media production company Wondros and Lybba, a non-profit organization. He is also a member of the Council on Foreign Relations and TED. He is the son of musician Bob Dylan . . .

Jesus Christ. This is the American aristocracy in a single sentence. A nepo baby who made it onto both the Council on Foreign Relations and Ted? Throw in an ambassadorship and a college presidency and you’ve hit the EGOT of intellectual celebrity culture for the talentless. I mean, sure, “American Wedding (known as American Pie 3: The Wedding or American Pie: The Wedding, in some countries)” is a modern classic, but what’s Jesse Dylan done lately?

Like the man said, I don’t believe in Zimmerman.

The Council on Foreign Relations, indeed.

It’s pure meritocracy, just like the Lutnick boys.

The only thing more ridiculous would be if they included the talentless second son of a minor branch of a German royal family . . . uh oh!

There’s also this page:

Every year, publisher John Brockman publishes a query on his Edge website, whose membership consists of some of the world’s most powerful, famous, thinkers, and achievers. . . . Brockman will just put a question out there. No strings attached on how you answer it. Some answers could be short like Jeffrey Epstein’s above, but others can read like virtual theses, which go on page after page. Sometimes, much more is said within the shorter pieces, and less in the longer pieces. . . .

Distinguished Harvard psychologist Stephen Kosslyn wrote:

Kosslyn’s First Law: The body and mind might appear separated but they are actually not. Not only does the state of the body effect the mind, but vice-versa.

As the saying goes, what a pretentious asshole. Your mind is mostly inside your brain, which is encased by your skull and attached to the rest of your body, so it would take a truly overeducated idiot to think that it appears separated.

Recall that the Edge Foundation also featured the pretentious idiocy of notorious science faker Marc Hauser. Also from the Harvard psychology department! Don’t worry, Harvard’s a big place, they do lots of solid research there too (just not the stuff discussed here).

OK, at this point you might say this is all old news. And it is. But look where America’s intellectual culture is now! Look who’d running the country. They’re telling kids not to take vaccines, and they’re still aggressively promoting junk science (see the link at the end of the previous paragraph). I do see a connection between the promotion of celebrity junk science and a country run by insanely wealthy bullshitters.

]]> https://statmodeling.stat.columbia.edu/2026/09/09/living-on-the-edge-aggressive-mediocrity-at-the-summit-of-americas-intellectual-culture/feed/ 18