Statistical Modeling, Causal Inference, and Social Science

Do high school students have to work harder nowadays?

Andrew — Mon, 21 Sep 2026 13:10:34 +0000

I happened to come across this old comment thread where Richard Serlin wrote:

Starting early is huge. If you’re a super-workaholic, with great endurance and energy, since kindergarten you’re going to achieve far more by age 40 . . . working your ass off since kindergarten and getting into Harvard as an undergrad makes it much more likely that you get into an Ivy PhD program, and with the resources, help, and prestige you get there, much more likely that you get a job in an Ivy department, and with the resources, help and prestige you get there, . . .

To which I replied:

I’ve been working hard since I started college, that’s for sure. But before then, no, I didn’t work hard. In the U.S. system (at least as it existed when I was a kid), it was perfectly possible to coast up to the age of 17 as long as you had the right background and abilities.

So here’s my question. Do kids have to work harder in high school nowadays? (I think we can take Serlin’s “working your ass off since kindergarten” as hyperbole.) My friends and I did our homework in high school, and I did take some hard classes, but I didn’t actually work hard, setting aside the very occasional late hours when projects were due.

I do get the impression that kids spend more hours doing homework than we did 45 years ago, but if it’s everybody putting in these hours, then it’s not about workaholics. It’s kinda hard for me to compare, even when I see actual high school homework assignments.

How did it go, this prediction from ten years ago about Japanese fertility in 2026?

Andrew — Sun, 20 Sep 2026 12:34:12 +0000

From 2017:

Gaurav Sood points us to this post, “Why did so many Japanese families avoid having children in 1966?”, by Randy Olson, which includes the excellent graph above and the following explanation:

The Japanese use [an] . . . astrological system . . . based on the Chinese zodiac. Along with assigning an astrological beast based on your birth year, each year also has one of the Five Elements associated with it—all that dramatically affect what your astrological sign entails. . . .

In 1966, however, many Japanese families were still quite superstitious—and . . . 1966 was the year of 丙午 (Hinoe-Uma), or the “Fire Horse.” As one source describes:

Girls born in [1966] became known as ‘Fire Horse Women’ and are reputed to be dangerous, headstrong and generally bad luck for any husband. In 1966, a baby’s sex couldn’t be reliably detected before birth; hence there was a large increase of induced abortions and a sharp decrease in the birth rate in 1966.

Time will tell if superstition will strike again 10 years from now in 2026, the next year of the “Fire Horse” in its 60-year cycle. Given that Japanese is already below the replacement fertility rate (i.e., roughly an average of 2 children per woman), the result could be disastrous.

If you look carefully at the above graph, you’ll see increases during the years before and after 1966. So, while it does seem that the net effect on births is negative, it’s not quite so negative as the spike might make it appear, given that some births that would’ve occurred in 1966 have been displaced to the adjacent years.

And now 2026 has arrived! When the year is over we can see what the birth rate in Japan looks like.

We’ll check back in on 6 Feb 2027.

The Immigration and Nationality Act of 1952

Andrew — Sat, 19 Sep 2026 13:21:08 +0000

I was reading the London Review of Books and came across this article by Jameel Jaffer stating that, in 1952,

Congress overrode the presidential veto and passed the McCarran-Walter Act with a two-thirds majority in the House and Senate. The law codified a system of immigration quotas that privileged applicants from Western Europe and almost guaranteed that those from Asia and Africa would be rejected. The section on activities ‘prejudicial’ to the US, together with others targeting suspected communists and anarchists, supplied a mechanism which government officials would use over the next decades to justify refusing visas to foreigners whose political views they didn’t like.

I had no idea. I knew that President Truman was very unpopular in 1952 and I knew there was lots of xenophobia, but, still, overriding a veto takes a lot of votes, and the Democrats had a majority in both houses in Congress.

Also I didn’t realize that there had been any major immigration bill since the act of 1924 that had drastically reduced immigration and the act of 1965 that opened things up again.

So I went to wikipedia to look up the bill, and, yeah, there it is:

Senator Pat McCarran (D-Nevada), the chairman of the Senate Judiciary Committee, proposed an immigration bill to maintain the status quo in the United States and to safeguard the country from Communism, “Jewish interests”, and undesirables that he deemed as external threats to national security. His immigration bill included restrictive measures such as increased review of potential immigrants, stepped-up deportation, and more stringent naturalization procedures. The bill also placed a preference on economic potential, special skills, and education. In addition, Representative Francis E. Walter (D-Pennsylvania) proposed a similar immigration bill to the House.

Parts of the story are kind of complicated:

The McCarran–Walter Act abolished the “alien ineligible to citizenship” category from US immigration law, which in practice applied only to people of Asian descent. Quotas of 100 immigrants per country were established for Asian countries; however, people of Asian descent who were citizens of a non-Asian country also counted towards the quota of their ancestral Asian country. Overall immigration from the “Asiatic barred zone” was capped at 2,000 people annually. Passage of the act was strongly lobbied for by the Chinese American Citizens Alliance, Japanese American Citizens League, Filipino Federation of America, and Korean National Association, though only as an incremental measure, as those organizations wished to see national origins quotas abolished altogether.

The total quota was 4720 per year from Africa, 3690 per year from Asia, 150,000 per year from Europe, and I think unlimited from the Western Hemisphere. Some famous people excluded under this law were Julio Cortázar, Michel Foucault, Dario Fo, Gabriel García Márquez, Doris Lessing, Pablo Neruda, and Graham Greene. Fair enough to exclude Greene, I guess; he was a British spy, after all. The others I think they should’ve let in.

I was wondering how that override vote happened, so I clicked the link:

The 113 Nay votes were concentrated in the northeast (New York, New Jersey, Pennsylvania, Connecticut, Rhode Island, and Massachusetts) and various other big cities around the country–I don’t have the maps at hand so I just have the districts within states, but I’m guessing that the patches of Nay votes in Ohio, Michigan, Missouri, Illinois, etc., correspond roughly to Cleveland, Detroit, St. Louis, Kansas City, Chicago, etc. There were only 4 Nay votes in the entire south.

How consequential was this 1952 law? I’m not sure. Aside from the annoying anti-free-speech aspect, it was pretty much a reaffirmation of the 1924 immigration act with very minor reforms. So the 1952 law didn’t do much on its own. I’m guessing its main effect was to preempt any more serious reform of the immigration system. That would have to wait until 1965. So I’m guessing that the main thing the proponents of this law got, beyond the immediate political victory of overriding a veto, was a 13-year delay.

P.S. The author of the above-linked article, Jameel Jaffer, works at Columbia! I guess I should look him up.

Meritocracy to-morrow and meritocracy yesterday – but never meritocracy to-day

Andrew — Fri, 18 Sep 2026 13:09:31 +0000

Alice couldn’t help laughing, as she said, “I don’t want you to hire me – and I don’t care for jam.”
“It’s very good jam,” said the Queen.
“Well, I don’t want any to-day, at any rate.”
“You couldn’t have it if you did want it,” the Queen said. “The rule is, jam to-morrow and jam yesterday – but never jam to-day.”
“It must come sometimes to ‘jam to-day’,” Alice objected.
“No, it can’t,” said the Queen. “It’s jam every other day: to-day isn’t any other day, you know.” — Lewis Carroll.

We’ve written a lot about meritocracy recently:

– “Howard Lutnick gives top Cantor Fitzgerald jobs to his sons Brandon and Kyle” is a very clean example of meritocracy. I mean it. And not because I have any reason to think that Brandon and Kyle have any particular merit.

– Is the civil service a meritocracy? No, it’s merit-based employment. For it to be “ocracy,” the people with merit don’t just have to get the jobs, they also have to wield power.

– Living on the Edge: Aggressive mediocrity at the summit of America’s intellectual culture. Throw in an ambassadorship and a college presidency and you’ve hit the EGOT of intellectual celebrity culture for the talentless.

Again, meritocracy is not the same as merit-based employment. In meritocracy, the people with the merit (Howard Lutnick, etc.) get to run things, and one of the perks of merit is getting sweet jobs for their kids. There’s a thin line between meritocracy and nepotism . . . hey, here’s Bob Dylan’s talentless kid!

This came up in class the other day, and someone pointed out that meritocracy usually seems to be presented as an ideal more than as a reality.

To paraphrase the White Queen, “Meritocracy to-morrow and meritocracy yesterday – but never meritocracy to-day.”

“Meritocracy to-morrow” is the appeal of meritocracy as an ideal, happening in the future so you don’t have to worry about its current consequences such as the careers of Brandon and Kyle Lutnick. “Meritocracy yesterday” is the appeal of an idealized past, something that we supposedly used to have.

Just to be clear, I’m not saying that merit-based employment is impossible. The problem’s with the ocracy. Or, if you’re Brandon and Kyle Lutnick, it’s not a problem at all.

Scientific citations as an overwhelmed communication channel

Andrew — Thu, 17 Sep 2026 13:43:44 +0000

Peter Dorman writes:

This looks like it’s up your alley. I find it interesting, but I’m not convinced by what he says about innovation. Of course, the issue is empirical, but I suspect it may be different for different sciences. In the work you’ve flagged in the social sciences, people are fishing for citations by cranking out apparent anomalies: “interesting if true“. it might be different in more theoretical fields.

Overall, this is the sort of stuff I was hoping science and technology studies would be generating when I first stumbled on it a few decades ago. Instead it got hijacked by critical theory, but that’s a topic for another day.

P.S. I’ve wondered why, in more scientific fields, it’s common to reference assertions with lots and lots of cites, not just one or two. My impression is that it’s less pronounced in the social sciences; at least, I started noticing it when I dove into the natural science literature for my climate book. Is that true, and if it is, why?

The above link is to a post, “How citations ruined science,” by David Oks. Here’s the summary:

There is nothing inherent about AI that makes scientists use it to produce slop; the essential problem is the incentive structures. The tidal wave of AI slop that is now threatening to engulf the institutions of science is, I suspect, a symptom of the citation regime. It would make no sense for a scientist in the 1950s or 1920s, when scientific communities were defined by personal reputation and mutual acquaintance, to submit the type of slop that is being submitted in huge quantities to scientific journals. But it does make sense within incentive structures that heavily reward publishing and getting cited.

This seems accurate to me.

I googled David Oks and he appears to be involved in public relations in some way. I wonder what he thinks of “AI slop” in that context. I ask because in advertising and public relations, I think the usual goal is not to produce anything novel or even interesting but rather to get noticed. This is related to what is called “the attention economy” but it’s centuries old, right? You put up big posters in the town square or newspaper ads or TV ads or whatever.

Another way of putting it is: For any communication channel there is some balance, some optimal level of flow of material. If the channel is unused, what’s the point; if it’s used too much, it will be manipulated.

One example is Pubpeer, the site for commenting on published scientific papers. It’s my impression that Pubpeer “works” in the sense that the comments there are often of high quality. But Pubpeer doesn’t get used very much–one way of seeing this is to note that a controversial paper that has been cited hundreds of times might only have a few comments on Pubpeer, or none at all. On the other hand, if Pubpeer did get used all the time, then I have a horrible feeling it would end up like Yelp and be swamped by fake reviews–or, to put it more precisely, reviews written for instrumental purposes to boost or knock a paper without supplying any scientific argument.

Every once in a while you can have a space like our comment section here which facilitates open discussion, but, again, once enough people are using a communication channel, there are increasing incentives to manipulate it, and the bad actors move in.

It’s satisfying to see the economics profession come around on some things (regression discontinuity analysis and so-called risk aversion)

Andrew — Wed, 16 Sep 2026 13:27:32 +0000

Jonathan Falk points to this post by Nicholas Decker and writes:

I thought you’d be interested in a couple of things in this. First, the regression discontinuity pictures and Decker’s parenthetical warning: “(The econometrics literature is emphatic on this – do not use higher order polynomials in a regression discontinuity. If you do, then you can get very large differences in apparent outcomes due to changes far away from the boundary – imagine if someone on the border of 44 flips the quadratic upside down, completely reversing your result!).” Lessons come through!

Second, his statement on risk theory — “As many people have noted (but see Chetty (2006) for a particularly elegant demonstration) estimates of risk aversion in different settings get extremely different answers. In particular, the coefficient of relative risk aversion comes from people having declining marginal utility of consumption. If you ask people in lab experiments about gambles, you’ll find that they’re really scared of risk, but if you observe people’s labor supply response to income changes, you’ll find things which imply they’re really risk tolerant.” I’ve never entirely understood your attitude on this other than your objection to particular functional forms, but the unease with which the profession has come around a bit towards your view should be welcome.

This is indeed satisfying.

And for those who are coming into this story in the middle, here are some references.

1. Problems with flexible adjustments in regression discontinuity:

[2025] Stable adjustments in regression discontinuity. {\em Journal of Comments and Replications in Economics} {\bf 4}. (Andrew Gelman and Guido Imbens)
[2019] Why high-order polynomials should not be used in regression discontinuity designs. {\em Journal of Business and Economic Statistics} {\bf 37}, 447–456. (Andrew Gelman and Guido Imbens)

2. Problems with naive ideas of risk aversion:

Lots and lots of posts over the years. I hardly know where to start! Here’s one example.

I won’t try to take too much of the credit for this progress in economics. I’m just glad to see it happening.

Charting the Agentic Garden of Forking Paths

Andrew — Wed, 16 Sep 2026 02:24:17 +0000

Arjun Balaji, Batuhan Duru Yeltekin, and Tian Zheng write:

Even with a fixed dataset and research question, data analysis involves many defensible decisions. Understanding how these choices influence the results is scientifically important but remains challenging. Crowdsourcing and agentic AI can generate hundreds of end-to-end analyses, but scaling generation alone can create a processing bottleneck and an analytic “black hole.” A common workaround is to impose a shared fixed decision taxonomy, which can limit insight and understate uncertainty. We present ForkSCOPE, a human-AI collaboration framework that induces structure bottom-up from the code corpus of end-to-end analyses, without a taxonomy fixed before or after generation, so the organization and evaluation of the garden can scale with the corpus. ForkSCOPE surfaces the charted garden of forking paths through a human-AI collaboration pipeline and an evidence-linked interactive viewer for steering and verification: it spotlights organically identified forks and structures and produces a derived taxonomy and decision map compatible with existing multiverse tools.

I don’t know enough about chatbots to understand what’s going on here, but it’s an interesting idea to study forking paths in this way. Here’s a copy of the viewer in HTML, also there are some links here. Tian says her favorite functionality is the filter by keywords.

Survey Statistics: ANOVA

shira — Tue, 15 Sep 2026 20:00:54 +0000

Andrew recently answered “Why did ANOVA fall out of fashion?”:

Anova is still important; it’s just been subsumed by hierarchical models.

The link is to my 2005 paper, Analysis of variance: Why it is more important than ever

Folks discussed whether ANalysis Of VAriance refers to 1) testing the null hypothesis that group means are equal, 2) fitting a regression model with categorical predictors, or 3) “an add-on to regression analysis, in which the predictors are structured and we estimate the variances of batches of coefficients.”

How does ANOVA relate to Survey Statistics ?

Andrew’s 2005 ANOVA paper‘s example in Section 7.2 is the Multilevel Regression (MR) of MRP. They made a new graphical display of the standard deviations of each batch of effects to replace the classical ANOVA table:

This is useful to assess the relative importance of different sources of variation, which can guide which interactions to add into the model.

In his discussion of Andrew’s paper, Alan Zaslavsky (the inspiration for this series) gave another example: a survey of members of Medicare managed care health plans. They modeled the data with variance components for geographical units (region, state, Metropolitan Statistical Area or MSA) and for the health plans. They found that for ratings of doctors, the majority of the variance was explained by geography, not the health plans. This suggested that improving quality of care might need to be directed towards low-performing areas rather than low-performing health plans.

“Protection from inappropriate influence is a hallmark of scientific integrity” . . . not any more!

Andrew — Tue, 15 Sep 2026 13:47:16 +0000

This is a funny one. Remember how Google used to have the slogan, “Don’t be evil,” but then they abandoned it? Something similar seems to have happened with the U.S. Census Bureau.

Dale Lehman pointed me to the story:

I’m sure this is not news to you. But I don’t think the usual lag is appropriate given the seriousness of this.

From the linked item, by economist Justin Wolfers:

“Protection from inappropriate influence is a hallmark of scientific integrity.” That line governed the Census Bureau and then the White House quietly disappeared it. . . .

Three weeks ago, I would have told you that the American government couldn’t suppress economic data like this. But under the new rules that the administration just put forward, the White House can direct its political operatives to kill uncomfortable numbers. . . .

Three weeks ago the United States had a policy that protected the free flow of scientific information and barred political meddling in how Census Bureau findings are presented. Today, it doesn’t.

To be clear: I’m not (yet) saying that our government is cooking the statistical books. Instead, I’m saying that it took the necessary steps to make that legal, and possible. In recent weeks, the Trump administration quietly rewrote the rules that kept political interference out of our economic statistics. And almost nobody noticed.

I didn’t notice either! Contra Lehman’s assumption, this was news to me. I guess that’s how news spreads now: Wolfers posts this on his blog, Lehman sees it and emails the link to me, I post it here and you see it, etc. Sometimes Tyler Cowen or Alex Tabarrok notices items from this blog and they post it on Marginal Revolution. Or maybe Krugman will post on it. The New York Times or Wall Street Journal might cover it too.

What I kind of wonder is how Wolfers heard about it. He’s a user of public data, so it makes sense that he’d be directly interested in the topic. But how did he find out about this obscure announcement? Was he just rooting through the census website one day and noticed something different? Did he see this on some news site? Was it sent to him by an anonymous tipster (I get lots of that sort of email!)? I’m curious. Wolfers does point to this NPR report from last month, so my guess is that someone noticed it and pointed Wolfers to it.

From the NPR article:

The Commerce Department updated its policy for promoting a “culture of scientific integrity” in January 2025, days before Trump’s second inauguration.

A new Trump administration order bans the Census Bureau and the Bureau of Economic Analysis from using statistical “noise,” or data for fuzzing survey results, to protect people’s privacy in their statistics.

The 2025 version included bans on attempts to shape or interfere in data collection, statistical analysis and other scientific activities “against well-accepted scientific methods and theories or without scientific justification.”

The department’s policy, the 2025 version said, involves ensuring scientific findings at its agencies “are not suppressed, delayed, or altered for political purposes and are not subjected to inappropriate influence,” as well as making sure there’s “independent review of facilities, methodologies, and other scientific activities as appropriate to ensure scientific integrity.”

That language is missing, however, in the Trump administration’s revision of the policy, according to an NPR analysis of text posted on the Commerce Department’s website.

And while the revision adopts a definition for “scientific integrity” that is similar to the previous version, it no longer says that “protection from inappropriate influence” is among the “hallmarks of scientific integrity.”

Spokespeople for the Commerce Department and Census Bureau did not respond to NPR’s requests for comment.

I guess the strategy of dodging comments worked, at least for awhile, because for three weeks nobody else seemed to have picked up on this story.

NPR continues:

While most of the public may not be familiar with the policy, Nancy Potok, a former deputy director at the bureau who has also served at the Commerce Department, sees the removed language as “a major change that eliminates the guardrails against political interference in the statistical and scientific data products of the department.”

Potok has been tracking how statistical agencies have been responding to a years-long, bipartisan effort to make sure the federal government produces trustworthy information. That included a 2018 law that led to a rule known among federal data experts as the “Trust Regulation.” The rule requires that statistical agencies “determine the policies and practices that ensure objectivity of its statistical activities” and produce data that is “impartial and free from undue influence and the appearance of undue influence.”

“The fact that there is a conscious effort to revise the department administrative orders and not incorporate things that are in the [Trust Regulation], instead of going in the other direction, is definitely troubling and needs some explanation,” Potok adds.

Why did the government do this?

For former Census Bureau Director Robert Santos, a Biden nominee, the close timing of the policy revision and the bureau’s release of an unauthored report with claims about noncitizen voting is striking. “It’s entirely possible that this was done to reconcile the Department of Commerce and the administration’s own actions by publishing an external partisan document about voting on the Census Bureau website,” Santos says.

And Wolfers brings on the indignation:

On August 19, the Commerce Department posted a new directive called “Scientific Integrity” — note the scare quotes. . . . They posted this directive very quietly. There was no press release, no announcement, and no press conference. For the first time in his life, Howard Lutnick was quiet. The new version just appeared online. To analyze the changes, I [Wolfers] actually had to go to the Wayback Machine to find the prior version of this policy, because it had been wiped from the Commerce website. There’s something pretty bleak about a country’s rules about scientific integrity surviving only because a volunteer internet archive grabbed a copy.

The Jan 2025 rules are here, and the new rules are here.

What do I think about all this? I’m not sure.

On one hand, the above story sounds horrible, and I think it’s disgraceful how government officials have been throwing around slogans such as “gold standard science” while suppressing reports, promoting discredited or flat-out fake science, and adding corruption to the science funding process, while their friends in the media are resorting to ridiculous defenses of indefensible policies.

On the other hand, this updated document could just be some overly busy bureaucrats trying some stupid idea. Also, if that 2025 policy was such a good idea, why did they wait until 2025 to institute it?

If “Protection from inappropriate influence is a hallmark of scientific integrity” is such an important slogan, where was it in the Census Bureau’s first 235 years? There must be more to this story.

On the whole, though, I guess I’d go with Noem’s Razor here: Never attribute to stupidity that which is adequately explained by malice.

In short: I think it’s reasonable to assume that this new document was motivated by some inappropriate reason, such as a desire to steal or to cover up some past or anticipated future offense, or from a general ideological position opposing objective knowledge and open scientific inquiry, or from a political attitude that independent science should be weakened because it represent a source of power that’s not under control of the government or its allies. Also, though, I guess they’re stupid. Just like the Soviets under Lysenko, an analogy that keeps coming to mind.

There’s an alternative explanation that this is all just a bunch of paperwork jockeys trying to justify their own existence, but Noem’s Razor suggests otherwise.

The Washington Nationals are hiring!

Andrew — Tue, 15 Sep 2026 02:17:33 +0000

Des McGowan writes:

I left the Mets last year to take a job as the Amateur Scouting Director with the Washington Nationals. Now that we’ve gotten through our first draft and trade deadline, we’re looking for two Senior Analysts who can help us take our modeling efforts to the next level. One of these new hires will focus on Amateur player projections and the other will focus on Pro player projections. Together they will play an important part in determining the players we draft and the trades/free agent signings we make.

The postings are below:

Senior Analyst, Player Projections – Amateur Acquisitions

Senior Analyst, Player Projections (Pro Player Focus)

We are also hiring a Director for the Player Projections group. This individual would need to have extensive baseball modeling experience, but I know that many such people visit your site so figured I’d pass along just in case.

Sounds like fun!

New York City Psychometrics Group

Bob Carpenter — Mon, 14 Sep 2026 19:00:59 +0000

[Edit to correct the attribution. Sorry about that! Bob.]

Klint Kanopka (NYU) put together a New York City Psychometrics Group, which hosts events. They have a web site with more information

New York City Psychometrics Group

There are three events planned so far for this semester:

September 25, 2026 (Friday) 3:00-5:00pm. Fordham University Lincoln Center (Lowenstein Cafe Atrium). Welcome Back Social
October 14, 2026 (Wednesday) 5:00-7:00pm. Fordham University Lincoln Center (12th Floor Atrium). 2026 Anastasi Distinguished Lecture by David A. Kenny with reception to follow
December 11, 2026 (Friday) 5:00pm. NYU Kimball Hall. Invited Lecture by Ben Domingue, with conversation, beverages, and leisure to follow

P.S. Betsy McCoach did such a good job advertising these events, two of which are at Fordham, I wrongly presumed she was the main organizer.

This is how we do modern frequentist statistics: Using fake-data simulation to understand what can happen in a study

Andrew — Mon, 14 Sep 2026 13:19:42 +0000

In a famous (to readers of this blog) example of statistical error, a researcher reported that beautiful parents were 36% more likely to have girl babies. The saps at Freakonomics fell for this one hook, line, and sinker, but I was suspicious, being a bit familiar with the sex ratio literature, which has found that the proportion of girl births is very stable at around 48.5%-49% in various subpopulations (whites and blacks, younger and older mothers, etc.).

The published analysis compared the sex ratio of children of the “very attractive” parents to all others, and the difference was 8 percentage points—not the “36 percent” mistakenly reported in the journal article and credulously repeated by Freakonomics, but still about 100 times larger than any plausible effect.

But here are the data, which look kind of convincing:

This comes from a survey of 3000 people and, as you can see, the percentage really is highest in that “most attractive” category; indeed, the comparison has a p-value of less than 0.05, hence the result being published.

OK, the result has no scientific plausibility given the vast literature on the topic (large effects of this sort are only found with studies with small samples), also the statistical significance is entirely explainable by forking paths—the odd choice of comparing category 5 with 1 through 4, rather than comparing 4 and 5 with 1 through 3, or just running a regression:

That slope estimate is not statistically significant; in any case the estimate is too noisy to be useful, given the range of realistically possible effect size. It’s the kangaroo problem.

But let’s set all that aside. Let’s forget all our subject-matter understanding and statistical expertise.

Here’s my question. Is there a way that a researcher without that specialized knowledge could see the problem with this study?

The answer, I think, is Yes. And the method is fake-data simulation. Which we could also call simulated-data experimentation. Or bootstrapping. With the only difference that, conventionally, the bootstrap is used as a way to get a bias correction or uncertainty estimate for an interval, and here we’re using it to develop intuition about a statistical data-collection process.

We would like to understand the statistical properties of the beauty-and-sex-ratio study by simulating hypothetical replications. In this case, the data came from 2792 participants who had at least one biological child in Wave III of the National Longitudinal Study of Adolescent to Adult Health. The full sample had 4877 respondents, of whom 2% were characterized as “very unattractive,” 5% as “unattractive,” 45% as “about average,” 37% as “attractive,” and 11% as “very attractive.”

We simulate replications under a null model. Assuming the probability of a girl birth is 0.488, independent of parental attractiveness, we simulate the results of 2792 births with proportions in the five attractiveness categories as given above.

Here are 20 simulated datasets, with for convenience the fitted least squares line displayed for each:

We see patterns just as dramatic as that of the observed data shown earlier, indicating that those data should not be taken as evidence against the null model.

Now here’s the point. The wonderful thing about these simulation experiments is that they can reveal problems with a naive design, even if you didn’t anticipate any difficulties ahead of time. The original author of that paper could have saved all of us a lot of trouble by simulating 1000 replications of those survey data on the computer, either before or after he performed his data analysis. No math required, no subject-matter knowledge required.

It’s too late for him, but it’s not too late for you to do this for your next modeling and analysis problem and avoid the embarrassment.

This is, in a very real sense, frequentism. To me, “frequentism” is not about unbiased estimators or long-term coverage or whatever; it’s about thinking of the data you see as one draw from a distribution of possible realizations of the data. It’s about looking at this distribution, both to see how your data and inferences could fluctuate in the future, and to interpret the data you do see.

Why don’t people do this all the time?

One reason, I think, is that they have a naive understanding of statistical theory and think that if you act like statistical significance = truth then you’ll be ok 95% of the time. They don’t check because they think there’s nothing to check, or maybe it’s more accurate to say they think that the experts have already checked for them. They don’t check their statistical analysis any more than I check the cables every time I get on an elevator; I assume the fundamental questions of elevator safety have all been settled already.

The other reason is that simulations is that they take effort. A simulation experiment requires a fully generative model—a rule for defining the truth and simulating data from some specified random process—followed by analysis of the simulated data, all nested within a loop and ending with comparison of inferences to truth. This involves additional work compared to that required to conduct an experiment, first because it requires an automatic procedure for data analysis and second because it requires a generative model. We have found that this additional effort involved in constructing a generative model and automating the data-analysis process is itself helpful for thinking through the experimental process. Indeed, it has similarities to the steps of preregistration.

This example and discussion are in Section 10.5, “Simulated-data experimentation as virtual replication,” of our Bayesian Workflow book. But I’ve never published it as a standalone article or blog post before, and the message is so clear that I wanted to share it here with you.

When people (including me!) get things wrong, it’s salutary not just to figure out where they went wrong and how it happened, but also to step back and see if there’s some more general process they could’ve followed that would have flagged the problem. And here we can do so.

What’s happening with the models of the Atlantic Meridional Overturning Circulation?

Andrew — Sun, 13 Sep 2026 13:41:44 +0000

John “not Towering Inferno” Williams points to this new research article by Valentin Portmann et al., which states:

Climate models show considerable discrepancies in their future projections around the Atlantic, mainly due to uncertainties in the fate of the Atlantic Meridional Overturning Circulation (AMOC). Climate models suggest a reduction in AMOC strength of 32 ± 37% by 2100 (90% probability) . . . To refine this estimate and reduce its uncertainty, we use four different observational constraint methods. The best one, which provides the lowest leave-one-out error, integrates a large set of observable variables . . . It gives an estimate of the AMOC slowdown of 51 ± 8% (90% probability) . . .

Without knowing any of the details, this looks wrong to me. There’s a lot of uncertainty about AMOC, right? So how to you get to a slowdown of 51 +/- 8%? That just seems unrealistically precise.

The abstract continues:

This refinement mainly results from correcting a bias in South Atlantic surface salinity, consistent with recent studies emphasizing its role in the proximity to an AMOC tipping point.

OK, that’s fine, but then the key issue is not the statistical method for model averaging, it’s this particular model correction.

The paper’s kind of hard for me to read, paradoxically because it’s all about statistics. I’m reminded of something I heard back when I was a Ph.D. student, which is that the best statistical methods are invisible: the ideal is for applied researchers to be talking about the science, not about the statistics. That’s one good thing about Bayesian methods. Rather than arguing about the estimator, you’re arguing about the generative model (the data model and the prior distribution), which puts you in the realm of the science. I’d much rather have researchers talking about plausible values for parameters and predictions than talking about significance thresholds and rejection rates.

That said, the most important thing about a statistical method is not what it does with the data but rather what data it uses. So if the methods promoted in the paper under discussion allow the incorporation of additional information, that’s good.

According to the article:

Methods called emergent or observational constraint (OC) have been developed to reduce the model uncertainty of a future climate variable of interest, hereafter called projected variable. These methods constrain the estimate and model uncertainty of the projected variable using the real-world observations of one or more observable variables. This results in a constrained model uncertainty that is smaller than unconstrained one.

I see this and I’m like, Huh? They weren’t doing this already? If you have “real-world observations of one or more observable variables” that are relevant to your predictions, then, yeah, you should use this information.

Although I’m not really clear on what is meant by “real-world observations of one or more observable variables.” Isn’t that just the same as “observations”? If you’ve observed something, it’s in the real world, right? And anything you’ve observed is, by definition, an “observable variable,” right? As Bob would say, there are some difficulties of communication here.

As I said, I don’t know anything about the models or the data here. As a human, I’m concerned about the potential catastrophic effects of the decline of AMOC, and I guess that I’d go with the consensus forecasts and uncertainties. If it’s really true that there are important observational data not included in the standard approach, then I hope someone can write a paper explaining that directly.

Annals of hype: Did this discovery in 2012 mark “the discovery of the final laws of nature . . . a discontinuity in human intellectual history, the sharpest that has occurred since the beginning of modern science in the 17th century?”

Andrew — Sat, 12 Sep 2026 13:58:51 +0000

Here’s Ed Regis in 1993, reviewing a book from that year by physicist Steven Weinberg:

Currently, the most favoured explanation for the electroweak asymmetry involves the postulation of a new elementary particle, the so-called Higgs boson . . . One of the reasons why Weinberg and other elementary particle theorists so much want to see the [Superconducting Supercollider] built is that its energies will be high enough to produce the Higgs particle, supposing it exists. . . . Weinberg is a master at communicating the excitement of the quest for it . . . ‘The discovery of the final laws of nature will mark a discontinuity in human intellectual history, the sharpest that has occurred since the beginning of modern science in the 17th century.’

A few years ago they did find compelling evidence for the Higgs boson and it was news, but nobody said it was the end of the line, the final law of nature, or a discontinuity in human intellectual history.

Weinberg’s no longer around so we can’t ask him, but . . . back in 1993 was he really thinking that establishing the existence of the Higgs boson would represent “the discovery of the final laws of nature”? Like, now we know what all the particles are and we’re done? Full credit to Regis for not falling for the hype.

Why I don’t like highest posterior density (HPD) intervals

Bob Carpenter — Fri, 11 Sep 2026 19:46:52 +0000

This post is by Bob

There are several reasons I prefer central intervals.

First, you can’t tell how much of the probability is above or below a highest posterior density interval. For example, if I look at the HPD for an exponential distribution, it will run from 0 to the inverse cdf of the interval probability. If I look at the HPD for a normal, it matches the central interval.

Second, the range is going to be parameterization dependent. If I parameterize a probability in terms of log odds in (-infinity, infinity) vs. a log probability in (-infinity, 0) versus a probability (0, 1), I get different HPDs. Similarly, you’ll get different intervals for standard deviations vs. variance (even accounting for the squaring). In contrast, quantile-based intervals like central intervals are invariant to changes of variables.

Third, you don’t even get an interval if the highest density regions are disconnected, as in something like a Beta(0.5, 0.5) distribution. There, the highest density regions are from 0 up and from 1 down, so any highest density region is going to exclude the central interval of lowest density.

There is a lot of confusion about probabilities vs. densities in the wild. For instance, here’s a document from RStudio: Highest Posterior Density (HPD) Intervals. The document defines quantile-based (e.g., central) credible intervals in the usual way. Then it makes the common mistake of conflating density and probability, with the claim, “However, this definition does not guarantee the interval contains the highest probability regions.” Both the 90% HPD and 90% central interval contain exactly the same amount of probability mass, namely 90%. What this means is that the probability that a random draw falls in either is the same—90%. What the authors presumably meant is that central intervals don’t necessarily contain the highest density points. I think this confusion between probability and density is a lot of what’s motivating HPD. And also, the highest density points are going to depend on the parameterization (my second objection above).

On the other hand, Andrew’s the one who was pushing for replacing central intervals with HPD intervals in Stan, so I’m curious what his motivation for that change was given that he’s definitely not confused about probability versus density.

“Belief” is not something you have; rather, it’s a relationship between thought and action.

Andrew — Fri, 11 Sep 2026 13:48:26 +0000

I came across this piece by John Ziman from 1989, in the middle of the “cold fusion” hoopla:

When scientific controversies erupt like this, one must obviously try to grasp the essential points at issue. The mere fact that these points are seriously disputed affects one’s world picture. But if one is not then prepared to participate fully in the dispute, the proper scientific attitude is temporary suspension of belief-forming activity. Most of the philosophies of science are weak in treating ‘belief’ as something one ‘has’, instead of seeing it as a relationship between thought and action. Even the sociological norm of ‘organised scepticism’ suggests a more decisive role for active thought than a situation might demand.

I like that: “belief” as a relationship between thought and action.

This is related to my preference for the term “prior information” rather than “prior belief” in Bayesian inference. It’s not just a choice of words; it also has applied relevance, as demonstrated for example in several examples in the first chapter of our Bayesian Data Analysis book.

When journals are slow to implement or enforce policies to manage sloppy, majority AI-authored papers, reviewers can and should push back

Jessica Hullman — Thu, 10 Sep 2026 15:26:30 +0000

This is Jessica. If you’ve ever reviewed a heavily AI-generated paper, you know how unrewarding it can be. For example, last week I spent a morning reviewing a paper for a well-respected general science journal, which sounded interesting but was ultimately very marginal in terms of the delta over prior work. It had all the signs I’ve come to associate with heavy AI reliance–overloaded terms invented as unwarranted shorthand for basic statistical concepts; dense, fluent but vague writing, with lists of details and implied comparatives I couldn’t resolve even with repeated rereading; and a very incremental result—not a bad idea, just not a complete thought. It nonetheless took me over an hour and half to identify what they were trying to do and write up my thoughts in a review.

I’m now getting requests to review such papers a couple times a week–not from minor journals but from high impact, general science venues. The majority of the abstracts I’m sent these days get flagged as 100% AI generated on Pangram, which in my experience doesn’t happen if you are exerting even a small amount of effort to control what is said. I shouldn’t be surprised; I know of junior researchers submitting tens of papers a year, and some senior ones too. But I am surprised nonetheless, that I’m being asked so often by widely respected venues to give feedback to authors who have given me little reason to believe they even understand what they are submitting. Of course AI writing is not a perfect signal of low author involvement, and LLMs can lead to much better science and better writing when used well. But see enough of these mostly AI-written papers fall apart upon closer inspection and you start becoming skeptical of anything that’s flagged as mostly AI text.

The problem is the seemingly poor enforcement of policies that these journals have put in place to avoid low human involvement papers. From a quick check, many of the top general science journals have policies about human accountability (some of which are pretty complex, like Nature’s) and most require disclosure of AI use. In some cases, substantial AI generation is prohibited (like Royal Society). But my experience with review requests from such journals and what I’m hearing from others suggests these policies are not weeding out enough of the slop. I also know that as an AE for Science Advances, I have seen a fair amount of heavily AI-authored text but have yet to see an author mention AI use, though technically our policy requires this in the paper and cover letter. In trying not to overindex on writing, I’m pretty sure I’ve sent out at least a few sloppy papers for review. With their more polished surface and layers of technical detail, low quality AI-authored papers are harder to catch on a quick skim than human-generated ones. And so external reviewers are stuck doing much of the sorting that these policies are intended to take care of.

To better allocate my own attention as an external reviewer, I’ve started saying no to review requests when I’m pretty confident that I’m being presented with majority or fully AI-generated text. There are just too many papers to review for me to keep making time for those with little observable signal of human involvement.

And while I don’t think AI writing detection is a long term solution, it’s increasingly seeming that we may need to add some friction to the system to push venues to experiment with policies that could filter more effectively. So I wrote this reviewer statement with Auyon Siddiq, in the interest of normalizing reviewers’ exercising their autonomy and signaling to venues that better policy is needed.

The hope is that by encouraging researchers to assert their agency, it might help incentivize venues to move beyond the blanket “please disclose AI” statements that they don’t seem to be enforcing.

The main risk is that if many people start refusing to review on these grounds, it could make peer review much noisier temporarily. I personally think it’s worth the risk. If you use AI and detectors a lot, you get pretty good at telling when the writing involved little human effort. If you don’t use AI or detectors, you may be doing worse than chance. But if journals remain lax, we’ll likely end up with a lot of reviewers doing this kind of thing anyway. In the meantime, I recommend that reviewers take responsibility for becoming very familiar with the methods that exist, their limitations, and when they can be used without violating reviewer contracts.

There’s a much wider range of mechanisms journals could consider now: quotas on how many papers an author can submit in a given time frame, reciprocal reviewing requirements with desk rejection or future bans on submitting for non-compliance, higher standards for the clarity of writing, use of preliminary AI review to detect major issues, and so on. ML venues have been experimenting with most of these already. None of it is ideal, but when the old methods aren’t standing up to the scale of production that’s now possible, something needs to change. If we feel strongly that science requires human accountability, we should make it harder for authors to abuse the system.

P.S. Thanks to Aaron Clauset, Annie Liang, Eytan Adar, and Andrew for feedback on the statement.

Why I still like Orwell’s “Politics and the English Language” essay (even though he gets some of the linguistics wrong)

Andrew — Thu, 10 Sep 2026 13:35:59 +0000

Brett Reynolds writes:

I liked the two-column move in your Orwell post. It seems genuinely useful.

I wondered, though, about the choice of Orwell as patron saint for the argument. The central point in your post seems stronger than Orwell’s essay itself, which bundles a real concern about euphemism and overclaiming with a lot of shaky style doctrine about passives, short words, cliché, and so on. My sense is that your argument works best as a claim about aligning evidence and rhetoric, not as an Orwellian one.

Reynolds also links to this article by Geoffrey Pullum, “Fear and loathing of the English passive.”

Here’s my response:

Orwell was just one guy. I’d say he was one of the great writers of the century, with two memorable novels (Animal Farm and Nineteen Eighty-Four), two important political books (The Road to Wigan Pier and Homage to Catalonia), and a range of unforgettable literary and political essays. All these remain readable and relevant today. He’s not a perfect writer, but nobody is. William Faulkner’s great; he also wrote some gobbledygook. Not all of Fitzgerald’s paragraphs were gold; Hemingway had his share of duds, etc. The characters in Animal Farm and Nineteen Eighty-Four are not fully-realized characters—Orwell was no Edith Wharton—but characters aren’t not the only point of a novel. There are many ways to make a contribution. Ozzie Smith wasn’t much of a slugger but he still belongs in the Hall.

Orwell’s famous (and, I would say, wonderful) essay, Politics and the English Language . . . it’s just something he wrote. It’s not perfect. It’s more polished than a blog post, but there’s nothing thorough about it, and I can well believe that he got a lot of the linguistics wrong. Orwell was giving his impressions based on his experience as a writer.

Maybe the relevant question is, What should Orwell’s essay be compared to? Or, What’s the audience?

Compared to standard writing advice (the “five-paragraph essay” and all the rest), Politics and the English Language is fresh and liberating, both in its exhortations to write directly and clearly and in its connection of writing to politics, conveying the sense that writing can be a deeply moral activity.

And the connection between writing and politics is subtle. Orwell was writing at a time when there was a lot of pressure for people to write lies or to write in obfuscating ways in order to serve some political agenda. Orwell’s way of connecting writing to politics was different and more subtle than that.

I think these messages remain important today, both in political and in scientific writing, as discussed in my earlier post. So I value Orwell, even if he gave some specific advice that made no sense and didn’t understand about the passive voice and just overall had naive views about linguistics.

Maybe a good analogy is: George Orwell is to linguistics as Nate Silver is to statistics. Orwell had a deep understanding backed up by years of practice, and was able to communicate this understanding to millions of people. Go granular on Orwell and you’ll see lots of mistakes and lots of overconfidence on his part, as well as a dismissal of or annoyance with experts, but still a net positive and with important insights that experts can value as well. Similarly with Nate Silver, who’s done excellent statistical work in both politics and sports, and, yes, sometimes he gets things wrong and isn’t interested in learning from his mistakes, and that annoys me, and sometimes his thoughts on political science are not fully thought through—but he offers a unique perspective and his writings are valuable, not just for outsiders but for experts as well.

Living on the Edge: Aggressive mediocrity at the summit of America’s intellectual culture

Andrew — Wed, 09 Sep 2026 13:44:20 +0000

So much good stuff here at the Edge Foundation and Jeffrey Epstein webpage:

– The connection to Ted talks. I had no idea! But it fits. Money and bullshit go together like liver and onions.

– “Bill Gates and his wife.” What is this, Gilligan’s Island??

– I’d never heard of Jesse Dylan or Jeff Skoll so I googled them. Skoll’s a standard-issue internet billionaire: he made a bunch of money from Ebay and now makes investments. But Jesse Dylan is more interesting–or, I should perhaps say, less interesting. From Wikipedia:

Jesse Byron Dylan is an American film director and production executive. He is the founder of the media production company Wondros and Lybba, a non-profit organization. He is also a member of the Council on Foreign Relations and TED. He is the son of musician Bob Dylan . . .

Jesus Christ. This is the American aristocracy in a single sentence. A nepo baby who made it onto both the Council on Foreign Relations and Ted? Throw in an ambassadorship and a college presidency and you’ve hit the EGOT of intellectual celebrity culture for the talentless. I mean, sure, “American Wedding (known as American Pie 3: The Wedding or American Pie: The Wedding, in some countries)” is a modern classic, but what’s Jesse Dylan done lately?

Like the man said, I don’t believe in Zimmerman.

The Council on Foreign Relations, indeed.

It’s pure meritocracy, just like the Lutnick boys.

The only thing more ridiculous would be if they included the talentless second son of a minor branch of a German royal family . . . uh oh!

There’s also this page:

Every year, publisher John Brockman publishes a query on his Edge website, whose membership consists of some of the world’s most powerful, famous, thinkers, and achievers. . . . Brockman will just put a question out there. No strings attached on how you answer it. Some answers could be short like Jeffrey Epstein’s above, but others can read like virtual theses, which go on page after page. Sometimes, much more is said within the shorter pieces, and less in the longer pieces. . . .

Distinguished Harvard psychologist Stephen Kosslyn wrote:

Kosslyn’s First Law: The body and mind might appear separated but they are actually not. Not only does the state of the body effect the mind, but vice-versa.

As the saying goes, what a pretentious asshole. Your mind is mostly inside your brain, which is encased by your skull and attached to the rest of your body, so it would take a truly overeducated idiot to think that it appears separated.

Recall that the Edge Foundation also featured the pretentious idiocy of notorious science faker Marc Hauser. Also from the Harvard psychology department! Don’t worry, Harvard’s a big place, they do lots of solid research there too (just not the stuff discussed here).

OK, at this point you might say this is all old news. And it is. But look where America’s intellectual culture is now! Look who’d running the country. They’re telling kids not to take vaccines, and they’re still aggressively promoting junk science (see the link at the end of the previous paragraph). I do see a connection between the promotion of celebrity junk science and a country run by insanely wealthy bullshitters.

Survey Statistics: Pew Research finds “No Easy Fix for Bogus Respondents in Online Opt-In Polls”

shira — Tue, 08 Sep 2026 20:00:31 +0000

I just read the Pew Research Center’s new report “No Easy Fix for Bogus Respondents in Online Opt-In Polls”. It begins:

One of the most urgent problems in online opt-in polling is bogus (or fraudulent) respondents. These are survey-takers who make no effort to answer questions truthfully and instead are just looking to finish surveys quickly and collect rewards.

To address this, they try 3 screening methods: 1) trap questions, 2) prescreening, and 3) voterfile matching. There is no direct way to know how many bogus respondents are removed with each method. So instead they compare them with 3 data quality measures: 1) “yea-saying”, 2) quality of open-end responses, and 3) response order effects.

The “yea-saying” data quality measure is the % who say “yes” to at least 10 of 15 questions. Without screening, this was 7%. At first this seemed fine to me and I was confused why this is even a data quality measure. But a probability sample estimates only 1% of people say “yes” to this many of these questions. Screening with trap questions helped the opt-in sample match the probability sample. One of their trap questions was asking if folks use a made-up social media platform called Fizzypress.

They made a separate set of calibration weights (see “3 flavors of survey weights”) for the unscreened sample and for the 3 screening method subsamples, calibrating to ACS along these dimensions:

How does screening for bogus respondents affect results for Harris-vs-Trump 2024 vote choice ?

Say we want E(Y), Harris vote choice in the population. But we only observe the opt-in sample, so we have nonresponse error (beyond Pew’s weighting adjustment): E(Y | R_opt_in = 1) – E(Y). Also, we only observe Y* != Y due to measurement error from bogus responses.

Say everyone we screen-in has no measurement error. Then the error with screening is nonresponse: E(Y | R_opt_in = 1, screened_in) – E(Y). This may be bigger than nonresponse error without screening: E(Y | R_opt_in = 1) – E(Y), but this isn’t knowable. We only know overall error including measurement error: E(Y* | R_opt_in = 1) – E(Y). Pew found that overall error was less without screening, screened-in folks overrepresented Harris.

How do opt-in samples compare to probability samples ?

Pew writes that there is “no way for potential bad actors to self-select into probability-based samples.” Of course, a randomly selected person can still give bogus responses (responding with Y* instead of their true Y). But the rate of measurement error in probability samples would match the population: P[Y != Y* | R_probability_sample = 1] = P[Y != Y*]. Whereas the rate of measurement error in opt-in samples is presumably higher due to folks selecting into the survey to earn rewards: P[Y != Y* | R_opt_in = 1] > P[Y != Y*]. In other words, the rate of measurement error is subject to nonresponse error, a mixing of the two sides of the Groves et al. figure below.

In this Survey Statistics series we’ve seen measurement error in an adjustment variable X, e.g. recalled vote: see “is a mismeasured X better than none at all ?”, “more adventures in mismeasured X”, “more on recalled vote”, “it is (still) the people”. We saw that the NYT used to drop adjustment for recalled vote due to its measurement error, which could worsen nonresponse error. Pew is studying the effect of screening out people with a lot of measurement error, which could worsen nonresponse error.

In this series we’ve also seen measurement error in the outcome variable Y, e.g. disease status, see “wanting workflow”. This example modeled the measurement error, rather than screen out mismeasurements.

Also relevant is Andrew’s post “She wants to know what are best practices on flagging bad responses and cleaning survey data and detecting bad responses. Any suggestions from the tidyverse or crunch.io?”.

Hey–here’s an R package for imputing Census data using iterative proportional fitting on available margins.

Andrew — Tue, 08 Sep 2026 13:52:16 +0000

Gustavo pointed us to this project (see also here) by David Dorer to impute data to lower census levels (block/tract) from PUMAs (public use microdata areas, as defined by the U.S. Census).

From a quick glance this looks like the basic method that I would recommend right now. I have dreams of a better, Bayesian, approach, but iterative proportional fitting (IPF) is what I usually recommend to people. So if they’ve programmed this up and set it up with real census data, that looks like it could be useful.

Ben Goodrich adds:

I think it sounds sort of reasonable. In general with the PUMS, the person-weights only depend on race, sex, and age, so I don’t know how great it is for other variables anyway. But if it rakes in such a way that the smaller subareas compose into PUMAs, that is a start. I think if you did table fusion on a bunch of variables including the geographic subarea, then you could loop through each row in the PUMS can use as a “prior” the proportion of the person’s PUMA in each subarea and use as the “likelihood” the probability of having the person’s demographics given that they live in in each subarea with positive prior probability, you could get a “posterior” probability that the live in each subarea and impute from that categorical distribution. I doubt it would matter that much, except when the subareas are Congressional districts, though.

Going forward, I’d like to move to soft constraints, as discussed here and section 2 of this post. I’m thinking of a model in which the specified margins are reals, not integers, with normal errors on the margins with sd’s that are user-specified (I think we could come up with reasonable defaults, too, given that the method would often be used in default settings).

“In the Weights”—an exclusive invitation just for you!

Andrew — Mon, 07 Sep 2026 13:17:43 +0000

We’re usually posting on a one-year delay, but I bumped this one up because this private invitation is “unique to you, not forwardable, and expires in seven days.” So hop to it!

This is your chance to join a forum of “successful people from every walk of life, including artists, athletes, scientists, founders, and others at the top of their fields.”

Say what you want about this one, it seems like a better deal than paying $16,846 to meet Gray Davis, Grover Norquist, and a rabbi.

Just remember, this invitation is not forwardable. It’s just a secret between you and me. As the man said: “Listen carefully. I’m only going to tell you once, because the person who told it to me made me promise not to repeat it.”

Does anyone have a lower Jordan3 number than Evans Boney? (And what about the Ariely-Armstrong-Enron number?)

Andrew — Sun, 06 Sep 2026 13:24:11 +0000

From last year’s post, What’s your Jordan3 number?:

In the discussion of our post, Who has the lowest Erdos-Bacon-Epstein number? (the winner appears to be the mathematician Daniel Kleitman, my freshman-year academic adviser at MIT!), an anonymous commenter asks:

Is there anyone with a finite Michael Jordan^3 number (acting with Michael B Jordan, coauthoring with Michael I Jordan, playing on a team with Michael J Jordan)?

Good question! In the earlier post we discussed the rules for what counts in being in the acting network (IMDB and with a legitimate acting credit, not just being interviewed) and the academic authorship network (scholarly journals).

What about playing on a team? What would it take to be in the Michael J Jordan network? It would be too much to restrict to players on NBA teams. I’d allow any college team—but only varsity would count, not intramurals—but even that is pretty darn restrictive, so I think I’d count high school varsity as well.

Here’s what we got in comments:

Gabe Durazo and Garrett House nominate Evans Boney:

Evans Boney ’06 — MIT Jordan³ = 14

Michael J Jordan = 5:

Boney -> Perkins (**2002 Evans Boney MIT / Ross Perkins Duke University** + **Pre-Draft Team: Duke (Sr)**) -> Redick -> Crawford -> Pippen (**2003–04 Bulls… re-signed Scottie Pippen… Pippen retired and Crawford was traded**) -> Jordan

Michael B Jordan = 4 3:

Boney -> Lockwood -> Teller (**Penn & Teller Tell a Lie | Starring | Penn Jillette Teller**) -> Esai Morales (**Atlas Shrugged II Cast: … Esai Morales**) -> Bacon (**Forty Deuce – Kevin Bacon as Ricky… Esai Morales as Mitchell**)

Boney -> Katie Condidorio (**The PHD Movie 2**) -> Marla Maples (**A Nanny for Christmas**) -> Jordan (**Black and White**)

Michael I Jordan = 5:

Boney -> Marcus (**E. T. D. Boney and R. A. Marcus**) -> Voth (**R. A. Marcus’s co-authors include Gregory A. Voth**) -> Baker -> Jordan (**Ben Blum, Michael I. Jordan, David Baker: Feature Selection Methods… NIPS 2007**)

~~5+4+5 = 14~~ 5 + 3 + 5 = 13

Garrett House comes pretty close to that himself:

Jordan³ = 16

Michael I Jordan = 3

House -> Osung Kwon (co-author) -> Wolpert / Ghahramani -> MI Jordan via *internal model for sensorimotor integration DM Wolpert, Z Ghahramani, MI Jordan*

Michael B Jordan = 5:

House -> Ben Armes (Lightly Ghosted Marcus Riley) -> Adam Johns (Milkboy Pretty Boy, Ben is Milkboy) -> Staci Lynn Fletcher (Poverty Tourism directed by Adam Johns, starring Fletcher) -> Michael Peña (30 Minutes or Less co-star Fletcher + Peña) -> Michael B Jordan (Triple Nine co-star Peña + MBJ)

Michael J Jordan = 8:

House 2013-14 Garrett House | F 6-4 | Fy.** -> Matt Redfield (same MIT roster) -> Will Tashman (Tashman + Redfield double-doubles) -> Bill Johnson (tri-captains Bartolotta, Gampel and Johnson = Johnson teammate of Bartolotta) -> Jimmy Bartolotta (playing professionally in Europe, signed with ÍR Icelandic) -> DeMarcus Nelson (**Bartolotta’s teammates were DeMarcus Nelson**) at Biella -> Jamal Crawford (Warriors teammates 08-09) -> Scottie Pippen (Bulls teammates 03-04) -> Michael Jordan

3+5+8 = 16

And then we have a couple of suggestions which sound promising but with no precise numbers yet.

Daniel Weissman suggests Neil deGrasse Tyson:

Tyson has had cameos in movies and has MBJ number 2, and was a D1 athlete, but it was in wrestling and crew.

He has a bunch of legit coauthored scientific publications (search on Neil D. Tyson) so he surely has a finite Michael I Jordan number through some pathway or another. I’d guess that he has some sports-team link that could get us, in some circuitous way, to pro basketball. Maybe none of his wrestling teammates at Bronx Science lettered in any other sport, but I bet that at some point there have been guys on Harvard’s rowing team or wrestling team who played basketball or baseball or football in high school and had a teammate who played a major sport in college etc.

And commenter Basketballer suggests Bill Bradley:

As he played in the NBA, a connection to MJJ should be easy. He has some publications in business journals, which may tie him to MIJ. Lastly, IMDB credits him as appearing as a “basketball player” in an episode of the Cosby show, so he should be linkable to MBJ.

To which I responded that the basketball connection to Michael J Jordan is obvious, and the Cosby Show is fiction, so even though Bradley was only playing himself, I think this still counts as acting, and then it won’t take many steps to get to Michael B Jordan. Finally, you link to a Harvard Business School article by Bradley and two McKinsey executives (it seems that Bradley was working for McKinsey too at the time), one of whom, Lester P. Silverman, got a Ph.D. in economics in 1973 at CMU and specialized in the energy industry, so I’m pretty sure he was a coauthor of a paper, “Economic costs of energy-related environmental pollution,” with CMU professor Lester B. Lave, who coauthored with all sorts of bigshots such as Kenneth Arrow and Herbert Simon, and we can definitely get from there to Michael I Jordan.

So Bradley might be the winner . . . but I’ll hold off on crowning him until I see his pathways to all three Jordans.

P.S. Now that we’ve covered the Erdos-Bacon-Epstein number (shortest number of coauthored scholarly publications linking you to Paul Erdos, plus the shortest number of movie credits linking you to Kevin Bacon, plus the shortest number of personal connections linking you to Jeffrey Epstein) and the Jordan3 number (shortest number of movie credits linking you to Michael B Jordan, plus the shortest number of coauthored scholarly publications linking you to Michael I Jordan, plus the shortest number of teammates linking you to Michael J Jordan), I think the next goal should be the Ariely-Armstrong-Enron number, defined my adding the number of links in these three chains of connection:

– Coauthorship on papers linking to Dan “Epstein” Ariely, the psychologist who has been involved in many different research projects involving fraudulent or altered data.

– Membership on sports teams linking to Lance Armstrong, the most notorious sports cheater of all time.

– Employment in companies linked to Enron, the world’s most notorious business fraud.

P.P.S. Commenter J reports that Bill Bradley’s Jordan3 number is 12: 2 to Michael B Jordan, 7 to Michael I Jordan, and 3 for Michael J Jordan.

So it looks like Bill Bradley is the winner! And he’ll have a finite Erdos-Bacon-Epstein number too. We can get to Erdos from somewhere in the scholarly link chain, we can get to Bacon from the movie chain, and Bill’s Epstein number is no more than 2 for sure. I expect he’ll have a reasonably low Ariely-Armstrong-Enron number too: Ariely through the social-science papers, Armstrong from some basketball player who played on some high school team with someone who became a professional cyclist, and Enron through McKinsey—yeah, his Enron number is just 2. Bill Bradley was truly the center of the world. And he just missed becoming president of the United States.

OK, here’s a statistics problem for you, ripped directly from the headlines!

Andrew — Sat, 05 Sep 2026 13:29:42 +0000

From the NYT:

Mayor Zohran Mamdani is expected to propose linking Grand Army Plaza with Prospect Park by closing a dangerous stretch of road between them. . . .

The plan would effectively reconnect the oval-shaped plaza’s most prominent feature, the Soldiers’ and Sailors’ Memorial Arch, with Prospect Park, Brooklyn’s emerald jewel, restoring the 80-foot-tall arch as a grand entrance to the park, as its designers intended.

Cars would be banned from the parts of the southern end of the plaza that border the park, from Union Street to Eastern Parkway, which would do away with a forbidding four-lane crossing, according to the city Department of Transportation. . . .

After public feedback, much of which favored more dramatic changes to the area’s roadways, the Mamdani administration selected a plan that would remove cars from the lower tip of the plaza and divert some traffic to other streets.

They quote people on both sides, who raise a number of points:

Lynda Balsama, an organizer with . . . United Neighbors of Prospect and Crown Heights, said the redesign would divert more traffic to nearby residential streets and add travel time for fire trucks and ambulances. . . .

Between 2021 and 2025, there were 219 traffic injuries along the plaza’s central roadways and outer ring, according to the Transportation Department. . . .

In spite of the constant traffic, the plaza remains a popular destination for those who want to admire the work of Frederick Law Olmsted and Calvert Vaux, the designers of the plaza and Prospect Park (as well as Central Park in Manhattan). . . .

“Every time N.Y.C. D.O.T. has provided more space to pedestrians at the park, it’s been an instant success, and it becomes impossible to think of how the space could have functioned before,” Mike Flynn, the transportation commissioner, said in a statement about the plan. . . .

The Transportation Department also expects the plaza redesign to help speed up some of the slowest buses in the borough, including the B41, which serves over 27,000 daily riders, because the reconfiguration should reduce congestion in and around the loop.

Here’s the official announcement from the city government.

Here are the issues raised above, stated more abstractly:
1. Current public opinion,
2. Worse traffic on other streets,
3. More fire damage (because the fire trucks are delayed) and more deaths or serious health problems (because the ambulances are delayed),
4. Fewer traffic injuries at the intersection in question,
5. Happier tourists and locals who want to enjoy the park,
6. Track record that past pedestrian expansions have been popular,
7. Faster buses.
I’m sure you could think of more; this is what I got from the news article. Of these seven issues, five are pro-redesign and two (#2 and #3) are anti. That doesn’t mean the plan is a good idea, as it reflects the balance among who was quoted. That said, I did some googling and it’s hard to find much opposition to the plan, so maybe the Times had to work to get even that one quote.

In any case, the research challenge is to put numbers on all these things. This still doesn’t answer the question of what to do, as it involves balancing questions of public opinion, economics, convenience, and life and death. But it would be a start. It’s hard for me to imagine that item #3 counts for much at all, but it should be possible to give some estimate of the components that go into it. Indeed, it could be that the fire trucks and ambulances would actually come faster.

Anyway, it seems like a good example, to work through the numbers on the seven items above.

Update on U.S. government’s Soviet-style push for “gold standard science”

Andrew — Fri, 04 Sep 2026 13:43:47 +0000

We had a discussion last year:

I got this email the other day from a journalist at a major news organization:

I’m a science reporter from **, wondering if you’d have some time to talk/reflect on the Trump administration’s embrace and use of concepts like “Gold Standard Science” and replicability – and the extent to which this is being used to improve science, or not.

I responded that it’s impossible to take this seriously, given that the same people who claiming to advocate so-called gold standard science have been energetically pushing junk social science such as unsupported claims of widespread election fraud and junk biological science such as, most notoriously, a discredited paper on vaccines and autism. This is the absolute opposite of gold-standard science.

Now comes this news item:

President Trump has opened a new front in his campaign to intimidate political foes, threatening a prominent liberal think tank with a $5 billion defamation lawsuit over a report concluding that his deployment of the National Guard to cities across the country has had little effect on reducing violent crime. . . .

On Monday, one of Mr. Trump’s personal lawyers, Alejandro Brito, wrote a letter to the center warning that he would file the suit if the group did not fully retract the report, which was published on its website on July 13. . . .

The Center for American Progress report accused Mr. Trump of seeking to take credit for a nationwide decline in violent crime that began before he returned to the White House. The report determined that there was “no evidence” that the National Guard deployments had affected the crime rate, adding that they were poised to cost taxpayers more than $1.7 billion if they continued through the end of 2026. . . .

The report is here.

Feel free to disagree with the report’s method or its conclusions, but it seems a lot more “gold standard” than much of the crap being fed to us by the government recently (see link at the top of this post, or here for another example).

This is Soviet-style ideological governance, very fitting for an administration that is now suppressing its own reports when they come to unacceptable conclusions. I hate to see it.

Maybe the NYT could balance the scales on their news reporting here by running an op-ed from some Ivy League economist patiently explaining to us that this latest policy is “not crazy” and “overall very sensible,” maybe even “a very good start for telling people where to go.”

Update on “I don’t understand this paper claiming election fraud in 2024 in Pennsylvania”

Andrew — Thu, 03 Sep 2026 20:11:21 +0000

Last year we briefly discussed a hard-to-follow paper by political scientist Walter Mebane discussing possible “malevolent distortions of electors’ intentions.”

Carter Walker and Jessica Huseman at the election news site Votebeat followed up on this story, and here’s their report.

Updike and statistical writing; more on honesty and precision

Andrew — Thu, 03 Sep 2026 13:59:09 +0000

So, yeah, I’ve been reading more of these Updike stories. They’re really good, and there’s something extra you get from reading a lot of them at once. The different stories have a sameness of themes, settings, and tone, but that’s part of the interest too, to see the same life investigated from so many very slightly different angles—it gives you the full three-dimensional perspective in a way that you wouldn’t get from any single story or even any few stories. And the development over time—50 years!—adds a fourth dimension of time. The ultimate effect is a sort of gradually moving hologram.

Updike has a sort of precision in his writing; I think he could’ve been a scientist or even a statistician. And, when I was a kid I wanted to be a writer when I grew up. I have indeed become a writer, but not of the sort I’d imagined way back then: “writer” to me meant “writer of fiction” or, more precisely, “writer of stories.”

I’m happy I didn’t try to have a career as a creative writer or even as a journalist—I appreciate the job security I have, also in this job I can write whatever I want to write whenever, anyway—, but, could I have become an Updike-style writer of stories had I tried? I don’t think so. I say this not (just) because I strongly doubt I’d have the ability to write his sort of compelling prose, but also because, to write this sort of story, you need to kinda peel off your skin. You need a ruthless honesty and also a sort of precision: a willingness to describe exactly how you feel, and how you felt, and what you see and hear, and what you saw and heard, along with the ability to put it accurately in words, without all those words getting in the way. Avoiding cliches is part of it but only part of it. Updike is mocked for his careful phrasing, but I see his beautiful style not as a frill (perhaps covering up an emptiness of content) but rather what is for him an absolute necessity to get at the truth, which he can only do through unique language. He is a painter with words, not a collagist.

Anyway, no way I could do this. Again, I’m not talking about the perfect sentences (but, sure, I can’t do that either); rather, I don’t have the ability or the willingness to peel off my skin, maybe not even to myself but certainly not to the world.

But, when it comes to statistics . . . that’s another story! I can be both open and precise, and I think that the 10 books and 1000 articles and 10,000 blog posts are presenting that four-dimensional hologram.

One thing I’ve noticed is that it can be hard for researchers to write directly what they’ve done, or to write directly what they’ve thought. I’m not saying it can never be done, but . . . consider this paper, for example: Criticism as asynchronous collaboration: An example from social science research. This is not a complicated paper, and lots of people could’ve written it, but it’s not the sort of paper that you’ll usually see. As with Updike, part of this is that I’ve trained myself to write fluently in a reader-friendly style, and part of it is that I feel the freedom to write how I want to write–but that’s not the whole thing. I do think this combination of honesty and precision is difficult to carry out, in part because, in their research, a lot of people aren’t ready to peel off their skin in that way. They’re committed to their findings, or to their professional relationships, or to their view of the scientific process . . . with openness comes vulnerability. I wouldn’t write about my personal life in that way, so I can see how others wouldn’t want to write professionally in that way.

And then, again, to write honestly and precisely in a way that’s readable and relevant to your intended audience, that can require lots of effort and lots of practice. It’s not like you can just turn on a switch and do it.

And it doesn’t always work. For every John Updike there are ten boastful blowhards who write about their fantasies and call it reality, and in science and technology, too, it can be all to easy to bloviate, to make strong statements in that authoritative in-your-face style that I associate with a certain style of internet writing, whether it comes from the left, right, or center.

So I’m not saying this will work for everyone. Indeed, as I said, I don’t apply Updikean honesty and precision to the writing about my personal life; it just doesn’t seem appropriate to me, and, indeed, Updike himself had what seems to me a very sad life—or, at the very least, a life that I would not have wanted to have. I understand that my approach of writing in an open, precise way—peeling off the skin in my professional writing—may have cost me some career points, but I don’t care. Or, at least, I’m happy with the tradeoff and I think I’m contributing the most I can to the world, which is something I’ve always felt the duty to do. Updike may well have contributed the most he could to the world with his writing; unfortunately that came not at a professional cost but at a personal cost, one that I would not have wanted to pay.

The other thing I haven’t brought in is collaboration. Most of my work is collaborative—indeed, even the above-linked article, which I wrote all by myself and for which I did all the work—is a sort of asynchronous collaboration with others, as indeed is indicated in its title. Most of the time I prefer to collaborate in the research, the structuring of the problem, and the writing too. It just makes it better work.

But Updike wrote all on his own. He had editors—not just editors, but New Yorker editors, they were the best—but it was his writing. An editor is great but it’s not the same as a collaborator. But Updike did have collaborators in the source material—that is, in his life. So I do see some collaboration there, in that the events he describes are all very social.

Updike’s books aren’t about character, and they’re certainly not about plot. They’re about observation, which is coming from Updike alone, but, even more, they’re about relationships and how they develop in four dimensions. (Remember, time is a dimension.) It sounds kinda funny to say that Updike can characterize relationships in four dimensions even though he can’t portray characters even in three, but I actually think that’s correct. He is somehow more sensitive to the changing connections between people, the pulls and pushes and connections and broken connections, than to the people themselves. You can especially see this in his writing about children (remember, Updike, like Roth, is always a son, never much of a father): they’re running around underfoot, and their relationships with each other and with the adults in Updike’s circle are important, but you never get a clear view of any individual child. Each person in an Updike story, child or adult, is defined by their places in the social network more than by their own characteristics.

When describing relationships, Updike works with a rich palette; when describing individuals, he’s a cartoonist, using physical features such as floppy hair or a stooping posture to stand in for deeper observation. You could argue that he’s just reflecting the perspective of his self-obsessed of his narrators, who only see others in relation to themselves, but I don’t think it’s that. Or I don’t think it’s just that. I think Updike is genuinely more interested in the shimmering patterns of interpersonal connections than in the individuals. He’s not gonna write a story about a man on a desert island. Or, if he does, the man will be spending the entire story reflecting on his childhood.

And this brings us back to statistics. Not the desert island thing, but the idea that connections are paramount. And here I’m not talking about links among researchers, important as these are, but rather the idea that a probability isn’t just a number; it’s part of a network of conditional statements. Arguably, statistics—data science!—is fundamentally about connections. Conditional probabilities, the relation between existing data and new data, the links between science and process models, the data models that connect measurements to underlying constructs of interest, the sampling and selection processes that mediate between sample and population . . . the whole thing.

So maybe Updike can be viewed as a sort of statistician of his own life. And remember the idea that the development of a story is a working-out of possibilities, which we’ve likened to posterior predictive checking. Thomas Basbøll and I have expounded on the virtues of true stories which, by being constrained by reality, allow us to learn. The surprises in a true story—what Basbøll and I call “anomalies”—can force us to re-evaluate our perspective, if we’re open to the possibility. But fictional stories have their own virtues, not just as entertainment (valuable as that is), but also because the open-endedness of fiction allows us to work out possibilities, as long as the fiction writing is done with rigor—or, one might say, honesty and precision—which can reveal the implications of our assumptions, our implicit models of the world.

Confirmation bias in software testing, Walnutpie edition

Bob Carpenter — Wed, 02 Sep 2026 19:00:34 +0000

So, those exciting results I presented at StanCon about Walnutpie? Let’s pull back the curtain.

Confirmation bias in testing

Those stellar results were the result of a bug in the way we calculated R-hat and effective sample size in the interfaces. When we plug in the bug fix (as we have for the 0.0.3 release on PyPI), the really stellar results on a collection of three models from Andrew shrinks and in some cases goes away entirely. So why didn’t we catch the bug? Because the results looked good but not unreasonable. Low R-hat and a decent ESS at about a tenth of the number of draws.

Claude’s attention to detail

We should have been more attentive. When I couldn’t replicate the results in our latest interface, I fed the results into Claude. Being much more attuned to detail, it immediately noticed that the table we’d provided was inconsistent. Can you see the inconsistency?

=== hierarchical_matrix_2.stan ===
chains=4  draws/chain=[1000, 1000, 1000, 1000]  total=4000  wall=0.6s
param                  mean        sd     mcse      ess    ess/s    rhat
s_y                  0.5551    0.0316   0.1193    161.7    273.2  1.0003
mu_b.1               0.9542    0.0921   0.0784    279.0    471.3  1.0001
mu_b.2               1.9781    0.0902   0.0750    302.4    510.8  1.0001
mu_b.3               2.1326    1.5192   0.0779    314.9    532.0  1.0003
...

Hint: Look at mu_b.2 and mu_b.3. Or focus on rhat vs. ESS.

Second hint: Look at the MCSE estimates of mu_b.2 and mu_b.3 and compare to the standard deviations and ESS values. All I can say is, “D’oh!” How can R-hat be so close to 1 with such low ESS values? Let’s sing it again together, “D’oh!”

I spend my life bouncing between “D’oh” and “Yatta!“.

What’s wrong with (Wal)nutpie?

It seems that like Nutpie, the use of gradients (specifically the diagonal of a regularized outer product of gradients to initialize the mass matrix) can make the initialization very sensitive. The problem is easy to see even with a multivariate standard normal (i.e., one located at zero with a unit covariance matrix). With any normal, the Hessian (second derivative matrix) is equal to the negative inverse covariance at every point at which it is evaluated. Fixed curvature is relatively straightforward to adjust with a mass matrix (equivalently, a linear preconditioner).

With a standard multivariate normal, the covariance is a unit matrix, so the Hessian’s just a negative identity matrix. This is what we’re trying to estimate as the mass matrix and it’s the default initialization for Stan.

The gradient is a different story. At position x in a multivariate standard normal, the gradient is -x. With a unit step size, a move along the gradient takes us back to the origin in one step (x + 1 * -x = 0). Nutpie regularizes the diagonal of the outer product, diag(x * x'), by taking a geometric average with a unit matrix. This effectively multiplies by one, then applies a square root, giving us an initialization of sqrt(diag(x * x')) = diag(abs(x)). This no longer takes us back to the origin in one unit-step-size step, but it does grow without bound. As the draws move closer to the bulk of the probability mass, this initial mass matrix is a very bad estimate which will then drive down step size.

Fool me twice, shame on me

I’m sorry to say that I’ve done this once before with the evaluation of an information extraction algorithm in NLP. I was working in industry on building named entity finders (e.g., people, places, things) in text that would have very high sensitivity. We allowed specificity to go down accordingly as forced by the ROC curve. This was some really fun n-best algorithm work for hidden Markov models (HMMs) and conditional random fields (CRFs), which was in our LingPipe software. This got us disqualified from the United States National Institute of Standards (NIST) bakeoff because we’d have won the sensitivity evaluation by a landslide. In the mid 2000s when this was going on, NLP and ML couldn’t get themselves away from the first-best-guess paradigm of evaluation. We continued to get funding because we aced the evaluations by real-life intelligence analysts, who were willing to sacrifice specificity for sensitivity.

So what was the bug? I coded what should have been a multiset evaluation as a set. It inflated our sensitivity at an acceptable (to the analysts) level of specificity from something like 99% to 99.999%. Oops. We came clean of course, just as I’m doing here.

A record of getting disqualified

This happened to me before with the United States Defense Advanced Research Projects Agency (DARPA), who disqualified our team at SpeechWorks in 2000 or so, which had built a spoken dialogue system that actually worked in production to book airline flights (it was in production at Delta for internal use). We were supposed to be the industrial “pacing horse” that would show just how much better things could be built using what they called “mixed initiative dialogue” in a research lab. The problem is, they got research lab members to build prototypes of an idea that was flawed linguistically from the get go. The flaw was assuming someone could build or would want a system where you could say “I need to fly to Dallas Tuesday morning, then that night go to Atlanta for two days, then back to Los Angeles for a night, then onto New York City, but I’m not sure how long I’ll need to stay in New York City. And by the way, I prefer aisle seats.” That’s just not how human dialogue works. Also, my advice is, don’t bring an academic prototype to a bakeoff with industry in an area they’ve launched into production.

OpenAI tries to one-shot human dialogue

Ironically, OpenAI made what I consider to be the same mistake with its post-training that focused on one-shot question answering. It trained the system to always try to provide an answer, like a student facing a closed-book exam with no penalty for guessing.

Then every other system just copied ChatGPT’s behavior. Given their corporate nature and the literally billions, if not trillions, of dollars involved, OpenAI and Anthropic have been complaining about IP theft. I’m laughing too hard to cry. To bring it back home to the topic of this post, the problem is an ill-chosen initialization. As an aside, did that initial training set include a hefty admixture of em-dashes, triple-adjective combos, and positive/negative contrasts because someone thought that’s what good writing was? I’m not suggesting hiring “theoretical” linguists, but someone who knew something about writing and the structure of actual human dialogue might have been helpful. Now all the AI companies are opening up the engines to search, which helps with the closed-book part of the exam, and just yesterday, Claude stopped in the middle of “thinking” (see, scare quotes for the haters) and asked a follow-up question before going on. Yay, progress! Let’s hope I can do the same on a much smaller scale with Walnutpie.

Metascience corner: What can we learn from 12 million empirical research results?

Witold Więcek — Wed, 02 Sep 2026 18:04:21 +0000

This is Witold and today we are building a bear.

In our recent paper with Erik van Zwet and Andrew, A statistical case for qualified scientific optimism, we fit metascientific models that use hundreds of thousands of z-values from empirical research. We will be blogging about this paper in the coming months, but the data that we needed for this paper seemed so interesting, that they spun off into a whole separate resource, Benchmarks of Empirical Accuracy in Research. I think readers interested in metascience will find it useful.

BEAR is an open-source compilation of 26 (and counting!) documented metascience datasets. Sometimes it just re-uses data from publications, like Kevin Lang’s work on false positives in economics—and dozen more. (I can’t stress this enough, huge thanks to the legion of researchers who created/maintain these datasets.) For some others I do much more myself, like processing outcomes from tens of thousands trials from clinicaltrials.gov.

And you can access all of that, 12.5 mln results in total, with a few clicks. Just go to https://witold.xyz/BEAR/. BEAR has bespoke datasets from medicine, neuroscience, psychology, ecology, education, economics, political science and more—all standardised and ready to use. Some sets have only z-values, but some have much richer structure: study types, subcategories, effect sizes, type of measures, meta-analytic groupings, etc. Of course quality varies, but I hope the documentation/standardisation will help people pick the right dataset for their research question.

Why does this exist? As I say on the website:

Quantitative metascience drives important debates about research standards. Most of its crucial contributions have been based on analysing individual datasets, often painstakingly constructed by researchers. Thanks to their work we now have better understanding of replicability, publication bias, p-hacking, reporting practices, pre-registration, significance rates, and more. But many canonical findings of metascience are based on individual datasets, each constructed in a specific way. How generalisable are these findings?

In other words, metascience itself can suffer from one of the crucial metascientific concerns: external validity. As someone who works on evidence synthesis I am very concerned with this. To give one example, Erik blogged about the virality of the scary publication bias histogram in biomedical journals. But different metascientific sets give completely different pictures of selection:

I am NOT particularly interested in selection in this post. We could talk about any other metascientific property, e.g. p-hacking, low power. I am also not trying to explain what drives these differences or say that these sets are comparable. (We get into that a little bit in the paper and if people are interested in that particular question, I can write a follow-up post.) The point is that we can learn much, much more when we go beyond one data point. OK, that’s pretty trivial thing to say, but Erik’s example clearly shows that it’s one that hasn’t fully sunk in. So I just hope that BEAR will make it easy to compare metascientific corpora and give people a better picture of the limits of generalisability of various claims. Or generally make metascientific research easier.

For completeness, the GitHub repo also includes the model that Erik, Andrew and I use in the paper I linked earlier. You can use it yourself to e.g. estimate (idealised) expected replication rates, but the model is optional and separate from BEAR data. Big kudos to Erik who got this process started and hunted down many of these sources for the paper. But most importantly the credit goes to teams of researchers who generated and maintain the linked sets.

I also have no doubt that in putting this together I made many mistakes. Even with automated checks, it’s not easy to work with so many data sources in parallel, so I’d be grateful for any corrections!

Yale med school prof tells his postdocs: Work extra hours at my command or you’re no better than “the guy who charges me by the hour to cut my lawn.”

Andrew — Wed, 02 Sep 2026 13:58:27 +0000

I came across this opinion piece, “Postdocs Aren’t Burger Flippers: Why Unionization Is a Bad Idea,” by Yale medical school professor Evan Morris, who writes:

Unfortunately, in 2026, postdoc unions are proliferating. And because of them, a key element of training for young scientists may be doomed. . . .

If we allow unions to commoditize the work of postdocs, we begin to undermine a key facet of their training, and one that is often credited for their future success: mentorship. . . . As a lab director, I may introduce you to my friends, advise you on your career path and write you letters of recommendation. But it also means . . . holding meetings in parks when the offices are closed by COVID, or inviting you for Thanksgiving dinner when you cannot go home for the holiday. . . .

Now, sadly, into the personal milieu between mentor and mentee come the unions. . . . if you’re my postdoc and you declare that you won’t work when we are up against a deadline or won’t help when I’m working over the weekend to support our shared glory, then don’t expect that invitation for Thanksgiving. I don’t invite the guy who charges me by the hour to cut my lawn. I invite my friends.

Hey, you can’t be friends with the guy who cuts your lawn??

The med school professor concludes:

Unions are about treating everyone, every problem and every circumstance exactly the same according to union rules. . . . What is passed from the mentor to the mentee is not something that can be codified or regulated—especially not by groups focused on hours of sick leave or retirement benefits.

This seemed odd to me, because . . . burger flippers aren’t in a union, right?

Or maybe they are? The wikipedia article on the topic has some interesting stories from around the world. In the United States, however:

McDonald’s first opened in California in 1940. It operates 14,300 franchises. None of the restaurants are unionized.

Damn.

The other thing that struck me in the above-quoted op-ed is that his motivation is “our shared glory.” What ever happened to teaching, research, and service?

Anyway, if you’re a postdoc and your boss puts you in the position where you are required to care enough about “the glory” to work on the weekend or else you won’t be “his friend” (which isn’t just about free turkey, given that he also says, “I may [emphasis added] introduce you to my friends, advise you on your career path and write you letters of recommendation”), then, yeah, maybe you do need some codified rules!

I’m not saying you need a union—maybe this could be just in your employment contract that your adviser’s professional duties, including networking, advising, and letters of recommendation, are not contingent on working overtime without pay—but, yeah, some sort of rules would be helpful.

Don’t get me wrong here. I have no problem if postdocs want to work on the weekend, nor am I concerned if they’re chasing glory. I enjoy a bit of that glory myself! But, yeah, here you have a boss openly saying he wants his employees to work extra hours without pay at his whim. This is the sort of power imbalance that motivates codified rules of some sort. I also find it creepy that Morris refers to postdocs as “learning at the feet” of their advisers. If that’s how the bosses at Yale operate, it’s no surprise to me that the postdocs want to join a union!

Again, I have no informed opinion on the merits of the issue. Just because the postdocs at Yale voted overwhelmingly to unionize, that doesn’t mean it’s a good idea. But, if the lab directors at Yale are coming on like J. P. Morgan in the 1930s, then, yeah, I recommend their employees do what they can to get themselves some job protection.

Why write about this?

Why devote valuable blog space (attention is a limited resource!) to a forgettable op-ed by an obscure professor? I have a couple reasons, beyond the usual reason that someone is wrong on the internet.

Most directly, as an Ivy League professor who employs postdocs, I feel some complicity here. It’s a funny system. The postdocs are paid from research grants, so I’m not directly spending the money myself; I’m just directing how it’s spent. The way it goes, we’re not really set up to pay people overtime for extra work, and I don’t ask them to keep a record of their hours. I guess that sometimes they do work on the weekend, not because I require or even ask them to, but more for their own convenience, just as I often work on the weekend when nothing else is going on. I guess things are different in a medical research environment because of the physical equipment: working on the weekend could require going into the lab and operating the machinery.

A postdoctoral fellowship is different from some other sorts of jobs in that it is time-limited, so if you’re a postdoc you’re doing your immediate work but always with one eye toward your application for that next job.

For that reason, I think that Morris’s desire to “support our shared glory” is misplaced. As a well-paid senior researcher in late middle age (like me!), Morris isn’t looking for a new job and he doesn’t need more money or job security. At this point, all that’s left is teaching, research, and service: that is, he wants to make the world a better place, save lives, and help train the next generation. I’ll take “glory” as a shorthand for all that. I looked him up and it says that Yale has given him the Graduate Mentor of the Year award. I take him at his word that he wants the best for his students and postdocs and considers them to be his friends, and that’s great. But being a wonderful mentor is still not the same as being able to put yourself in someone else’s shoes. For a postdoc, priorities can include paying the bills, living in an unfamiliar place, and taking care of a young family, as well as the continuing search for future employment. “Glory” might be the original motivation for going into research, but there will be lots of more immediate goals standing in the way. Rearranging child care so you can go into the lab on Sunday? That’s a real tradeoff. Not such an issue for those of us with secure jobs and grown children, but a real concern for someone working on a short-term contract without clear future plans.

So, when I read the above-discussed op-ed, I was uncomfortably reminded how my perspective, and Morris’s, are so much different from that of the postdocs we hire.

And this made me more bothered by the unsupported and false claims in that piece, along with what I considered an offensive attitude. It’s great that this guy is an award-winning mentor, but does that mean his students should be “learning at his feet”? The whole thing seemed positively feudal to me, the idea that postdocs should be subordinated now in preparation for their future upper-class lifestyle, in comparison to the lowlife “burger flippers” and “the guy who charges me by the hour to cut my lawn.” I was reminded of George Orwell’s classic memoir, Such, Such Were the Joys, which was all about hazing and the subtle distinctions among the upper class.

Survey Statistics: logit shift and raking

shira — Tue, 01 Sep 2026 20:00:59 +0000

We’ve been discussing how to use population margin information about poststratification variables (see Modeling Complex Contingency Tables and SynthMargins & Bayes-Raking). I want to connect two terms for this we’ve mentioned a lot in this series: the logit shift and raking.

In June 2025 we discussed 2 flavors of calibration, including the logit shift: calibrate p(z | x, survey) to match known aggregates p(z).

In June 2025 we discussed 3 flavors of survey weights, including calibrated weights: find weights w such that weighted survey E(wx | survey) matches known totals E(x). When we have at least 2 variables, say x and z, we can calibrate their cross-classification, or one margin at a time. The latter is called raking.

We can think of raking as finding weights w(x,z) or equivalently finding population probabilities p^(x,z) = w(x,z) p(x,z | survey) that match known totals. In the logit shift we want to find the population conditional probability p^(z | x) that matches a known total. These seem really similar ! Let’s compare.

Here’s my setup:

2 variables: binary z, categorical x
Have p(x,z | survey), and p(x), p(z) population margins
Want p^(x,z) = argmin_q KL(q, seed) that matches known margins
For raking: seed = p(x,z | survey)
For logit shift: seed = p(z | x, survey) p(x)

So here’s some homework: show that Lagrange multipliers gives p^(x,z) = seed(x,z) exp(- 1 – d_x – d_z). And so the solution p^ will have the same odds ratio as the seed (which is the same for raking as the logit shift).

We can substitute these solutions into the constraints, and for either seed we get:

sum_x p(x) logit^-1( logit p(z=1 | x, survey) – d_z ) = p(z=1)

This is one equation with one unknown shift d_z. In other words, logit shift and raking give the same solution. This can be solved with a root-find, as Andrew commented that Lei et al 2017 did in this code.

More generally, if z is multinomial, we can’t get such a closed form equation with one unknown, so we can use Iterative Proportional Fitting (IPF), as we discussed in 2nd helpings of the logit shift.

Updike’s kaleidoscope: Telling a life through a set of unconnected stories

Andrew — Tue, 01 Sep 2026 13:43:50 +0000

John Updike wrote around 250 short stories, the vast majority of which are from the perspective of the “Updike” character. As has been well documented, they’re based on himself, his family, and particular events that happened in his life. This corpus of stories includes one series with a single pair of characters—Richard and Joan Maples, modeled after Updike and his first wife, Mary. The Maples stories were written over the period of decades and, collected, are the length of a short book.

The 200 other stories about the “Updike” character mostly feature new characters, new names, and new settings in each story. There stories don’t have continuity; rather, they offer a kaleidoscopic view of this character’s life as he develops from an adolescent to a young man to a young husband and father to a divorcing middle-aged man to a reflective man in late middle age. It’s kind of amazing. Most of the stories have little to no plot, and the characters (other than “Updike”) are mostly not portrayed so vividly, but it’s very moving, living inside Updike’s head and seeing everything he sees: a curated GoPro, as it were.

So here’s my question: Why did Updike do it that way? Why did he tell his life in two hundred mostly unconnected stories, rather than in a series of novels? Or, for that matter, memoir? Updike of course did write many novels, but they’re mostly not about the “Updike” character. He did write a memoir or two, but these are primarily of interest because of what they reflect on the fiction.

A direct answer to why Updike told his story in distinct short fiction rather than novels is that he got into a rhythm with the short stories. They gave him a regular audience and a steady income, he’d figured out a way to transmute his life and thoughts into fiction. For him, novels served a different function.

I can relate to that. I get into a rhythm with blogging, then I also write research articles and books. These different sorts of writing reach different audiences, but, more to the point, they’re different ways for me to express myself. Kind of like if I wrote about different topics using pencil, pen, typewriter, or word processor.

There’s still the question of why introduce new characters and situations for each story (with the notable exception of the Maples series). Why not one long series? Why new names, new cities, new occupations, new minor character quirks for each story?

I can’t say, and I don’t know that any critics have asked this question . . . the answer I’ll venture is that, by fragmenting his perspective in this way, by telling a life from 200 slightly different angles, Updike is giving a kaleidoscopic view that’s special in its own way. As a writer, Updike is hardly modernist, but one could argue that his larger life’s project has a modernist aspect. Each of his sort stories is a coherent whole, but the collection of stories, constructed over decades, give a sort of cubist view of “Updike” and, indeed, of Updike himself.

Also, it wasn’t just Updike. Back in the day, many serious authors of fiction expressed themselves by writing new stories and scenarios with new characters each time: consider Irwin Shaw, John Cheever, John O’Hara, F. Scott Fitzgerald, Ernest Hemingway, etc. Series characters and continuity were more of a genre-writing thing, with Sherlock Holmes, Hercule Poirot, Tarzan, Nancy Drew, and all the rest. So I guess a lot of this was just that Updike was following the literary conventions of his time. But, for whatever reason, the result–the fragmented perspective constructed out of hundreds of different stories–is amazing.

P.S. Also, one minor thing, no big deal but it’s something I noticed after reading all these stories in succession: There are a bunch he wrote around the time when he was going through a divorce, and in just about all of them, the protagonist notices that his first wife has been putting on weight. Not that she’s ever obese, just that she’s getting bigger. If this happened in just a few stories, or if it was just in the context of a novel, I’d say that this my-first-wife-is-getting-fat thing is some hangup of the character. But it happens over and over again—we see it from many angles of the kaleidoscope—enough so that it’s gotta be Updike’s thing. That’s fine—everybody’s got issues, and a big part of what makes Updike, and other authors, great, is that they’re not trying to smooth away their awkward or unpleasant aspects. Still, it’s kinda funny. The purest illustration comes in a story where “Updike”‘s wife is an identical twin, and, during the story, the wife gets fatter, the twin gets thinner, and “Updike” ends up with the twin, commenting on her boniness. The point seems to be that the first and second wife are just two different versions of the same person, just representing different stages in “Updike”‘s life. The other funny thing about this is that, if you look at photos of the couple, Mary Updike doesn’t look fat at all!

P.P.S. If you look at enough of those stories, you will find some recurring names (the Updike stand-in is sometimes named “David Kern,” and his hometown is often called “Olinger”), but these stories still don’t show a continuity. In their totality they’re presenting a fragmented perspective of a life, so I think my “cubist” take still stands.

P.P.P.S. OK, just one more thing. I like almost all of these stories, even the less famous ones, some are absolute masterpieces, and their combined effect is stunning. But there was one story that was really bad, called “Licks of Love in the Heart of the Cold War.” To start with there’s the title, which is a lame pun, with “licks” referring to musical riffs—the protagonist of the story is a banjo player—and also a sex act. Lots of critics are exhausted or annoyed by Updike’s fascination with the memories of sex, his male gaze, etc., but that doesn’t bother me. Updike’s subject is his own life, and the images and emotions he shares seem as real as anything else would be. As an analogy, consider someone like James Jones, who wrote book after book about the army. That was central to his life, and he wrote about it. Writing about one of Updike’s later collections, critic James Wood writes, “If Updike’s earlier work was consumed with wife-swapping, his late work is consumed by nostalgia for it.” That’s accurate. But then Wood follows up with, “The dismaying thing about this collection is that Updike appears to be ascribing a value to adultery which it no longer has. . . . But it is now hard to share Updike’s extreme valuation . . . and the effect is to make Updike seem not only dated, but provincial and minor: the great, central writers do not seem to date in quite this way.” I disagree! Great writers often ascribe high value to their transformative life experiences; that’s how it is. You can feel uncomfortable with Updike’s values and find his perspective “provincial,” but I don’t think that makes the work “minor” or even “dated” except in the way that Jane Austen is dated and Mark Twain is dated and everyone is a product of their time.

Buuuuuut, ok, that one story, “Licks of Love in the Heart of the Cold War”: it’s really terrible. I like the other stories in that collection (sorry, James!) but this one is as bad as Updike’s later novels. It’s a mess, and not in an interesting way. The big problem is, interestingly enough, the writing style, in which the narration oscillates unstably between a homespun countrified style (as befitting a banjo player, I guess) and the traditional Updike diction that usually comes from the mouths of his well-educated white collar set. It just makes no sense at all, really a failure of editing that nobody caught it. The story was rejected by the New Yorker, so that’s something, but then Updike showed poor judgment by not fixing its flaws. Had he maintained perspective (either in the “banjo” voice or by giving the character the usual “Updike” background), I think the story could’ve worked just fine.

But, yeah, that’s just one story. Overall the kaleidoscope works very well, and I’m ok hitting the occasional dud. Contra Wood, I think Updike’s high productivity, at least when it comes to the stories, contributes in a good way to the overall effect. 100 of these stories telling the life from different perspectives would’ve been great; 200 is even better, giving me even more views inside.

I am shocked, shocked to learn that this 2002 paper by Dan Ariely is based on dubious data and doesn’t replicate.

Andrew — Mon, 31 Aug 2026 19:15:37 +0000

Dan Ariely—the much-decorated business school professor, retired Wall Street Journal columnist, Ted-talk star, NPR hero, Jeffrey Epstein contact, Founding Partner of Irrational Capital, insurance agent, inspiration for a hit TV sitcom, and author of the instant-classic children’s book, “The Adventures of Professor D.”—has had pretty much the worst professional luck that any scientist could have.

Over the period of decades, he keeps ending up as coauthor on journal articles don’t replicate and that turn out, through absolutely no fault of his own, to be based on dubious or fake data. Bad collaborators, lazy research assistants, missing computer files . . . who knows how this is all happening, but it’s bad luck for sure.

Uri Simonsohn, Joe Simmons, and Lief Nelson just came across another one:

A new paper in Psychological Science reports a failure to replicate Study 2 of Ariely and Wertenbroch’s influential article entitled, “Procrastination, Deadlines, and Performance: Self-Control by Precommitment.” The original study, published in Psychological Science in 2002, found that people performed better on a set of tasks when each task had its own externally imposed deadline than when people set their own deadlines or faced a single last-day deadline for all tasks. The paper has had a lasting influence. It has been assigned reading in many economics and psychology courses, and has more than 2,100 citations on Google Scholar. . . .

About 20 years ago, on April 20, 2006, one of the authors of the forthcoming replication, Kyle Hyndman, received the original data files in an email sent from ariely@mit.edu . . . .

But then when Hyndman and his coauthor, Aberto Bisin, attempted to reanalyze the original data as part of their replication effort, they report that this happened:

In October 2024, at the request of the editors, we shared with Dan Ariely an analysis of the contents from the file purportedly for their Study 2 and asked for permission to include a summary of it in the paper. Dan Ariely denied our request, arguing, among other things, that the files we received may not be the actual data. He did not subsequently provide us with any additional data from the original paper.

Damn. I hate when that happens.

Simonsohn, Simmons, and Nelson continue:

This motivated us to return to this paper and fully analyze the original data for the two main studies. We conclude that the data in Studies 1 and 2 were tampered with. . . .

Our assessment that the data were tampered with are based entirely on the analyses presented in our posts. Readers can review the evidence and draw their own conclusions. . . .

Now comes the fun stuff.

And, by “fun,” I mean “horrible.”

Here’s one of the graphs from the 2002 paper:

Wow—that looks pretty impressive! Everything’s exactly in order, the standard errors are small, but the gaps between conditions 1, 2, and 3 aren’t exactly equal. They’re slightly irregular: exactly equal could raise suspicion, but these data look like they could be real . . . at least they do, until you look at them more carefully.

Here come Simonsohn, Simmons, and Nelson to rain on the parade:

Red Flag #1: The Effect Is Too Big
As shown in the reprinted figure above, Ariely and Wertenbroch report a perfect pattern of results, for all three dependent variables, with a sample size of only 20 per condition. The effects are also large. Extremely, implausibly large. . . .

Red Flag #2: Duplicate Observations . . .
18 of the 20 participants in the Last Day Deadline condition had a “Corrections Twin”, another participant who found exactly the same number of errors for each of the three proofreading tasks. Interestingly, these twins have ID numbers that are exactly 10 positions apart (e.g., subject S1 and subject S11 are twins; so are S7 and S17; etc.). (There were no error twins in the other two conditions.)
The existence of so many of these twins – and all of them in only one condition – is inconsistent with these data being real. . . .

Red Flag #3: Things That Should Be Very Highly Correlated Aren’t Correlated At All
At the end of their study, Ariely and Wertenbroch purportedly “asked participants to evaluate their overall experience [of the proofreading task] on five attributes . . .
You might expect these judgments to be correlated. For example, if someone says they liked the task, you might also expect them to say that it was interesting.
In the replication, this was (super) true. Controlling for experimental condition, the partial correlation between liking and interest was, quite sensibly, close to perfect:

But in the original data, this relationship was not only imperfect; it was not there at all. Participants who said they liked the task more did not say that they found the task to be more interesting:

The problem is not limited to these subjective measures. Consider the fact that people did three very similar proofreading tasks, each with 100 mistakes. Surely, we’d expect people who do better on one task to also do better on another, nearly identical task. That simple fact should manifest in extremely large correlations between performance on one task and performance on another. And in the replication data it does, as the correlations range from +.74 to +.90. But in the original data it doesn’t, as the correlations range from +.03 to +.27. . . .

Red Flag #4: No Rounding In Self-Reported Minutes
As you’ll recall from a minute ago, Ariely and Wertenbroch (2002) purportedly asked participants to “estimate how much time they had spent on each of the three tasks” (p. 223). When people provide estimates like this, they tend to round. They usually say “20 minutes” or “30 minutes” instead of “17 minutes” or “32 minutes”. And, indeed, when the replicators asked people to report how many minutes they spent on each of the three tasks, 85% of them gave a round number:

This is what we’d expect humans to do.

But in the original data, they did not do that. Only 11.7% of estimated minutes were round, consistent with the 10% you’d expect by chance alone:

Simonsohn, Simmons, and Nelson conclude:

We are unable to generate a benign explanation for all of the anomalies presented here. The original findings are too large and yet they do not replicate; there are duplicated observations; correlations that should be very strong are often non-existent; and values that should be rounded are not rounded. Based on this evidence, we believe the data for Study 2 of Ariely and Wertenbroch (2002) were severely tampered with or fabricated to produce the desired results.

Can you believe the bad luck of Ariely, to have this happen to him over and over again? I imagine he’ll want to launch an investigation to catch the real faker or fakers. What a waste of time, though. This is effort that could otherwise be devoted to designing psychology experiments. Or delivering Ted talks. Or modifying paper shredders. Or something.

I recommend to Ariely that, moving forward, he choose his collaborators and research assistants more carefully, so that he doesn’t again get stuck with fabricated data.

Maybe he could have them sign some sort of honesty pledge?

In Ariely’s own words, Professor D is “a charming character doing his best, struggling, and using social science to his advantage.” That’s one way to put it!

P.S. I employ some humor in these posts, but this sort of story doesn’t make me happy, it makes me sad. It doesn’t surprise me—lots of people enjoy cheating, and some of them will find their way into academic research, and some of those people will be successful at it—indeed, cheating can make it easier to be successful, as you’re no longer bound by the truth—so I’m not surprised, but I hate to see it.

In all seriousness, this sort of fraud is disgraceful, and whoever did it should admit it and pay some restitution to the Association for Psychological Science and the thousands of subsequent researchers who have relied on these fake findings. They should also reimburse the government for funds received in any later research grants whose proposals relied on these results. Ariely’s a nice guy, I get that, but I think that at this point he should just share the names of his collaborators and research assistants who’ve been doing all this faking It’s not right that he gets all the blame and they get off scot-free.

P.P.S. See here for more from Simonsohn et al.

P.P.P.S. I also again want to thank Simonsohn et al. for their efforts here. They got sued for doing this sort of thing! But they’re not intimidated and they keep at it. Good for them.

I hope that, at the very least, Ariely can send a public note to Simonsohn et al. thanking them for uncovering all these problems in his published papers. When people point out problems in my published work, I appreciate it and I thank them.

Do children grow continuously or do they grow in fits and starts?

Andrew — Mon, 31 Aug 2026 13:54:14 +0000

There’s some literature on saltatory growth in children.

From 1992: Saltation and Stasis: A Model of Human Growth, by Lampl, Veldhuis, and Johnson:

From 1993: A case study of daily growth during adolescence: a single spurt or changes in the dynamics of saltatory growth?, by Lampl and Johnson:

On the other hand:

1993: Linear growth in the rabbit is continuous, not saltatory, by Oerter, Bacher, Cutler, and Baron. This paper unfortunately has no graphs, but the abstract is compelling:

A recent report in Science suggests that human growth occurs in brief bursts, up to 1.65 cm in a single day, separated by extended periods of stasis, lasting up to 63 days. Thus, the organism is proposed to alternate between two states, one with a growth velocity of zero, the other with a mean annualized growth velocity greater than 350 cm/yr. These observations, if correct, suggest the existence of a previously unsuspected hormonal mechanism capable of abruptly switching growth plate cell division on and off and of synchronizing cellular growth not only throughout the growth plate, but presumably throughout all the growth plates in the organism. However, the experimental assessment of short-term growth velocity in the human faces the formidable obstacle of a technical error of measurement that exceeds the mean daily growth rate. Accordingly, we tested the saltatory growth hypothesis by measuring proximal tibial growth in the rabbit, a model in which daily growth rate could be measured more than 15 times more accurately than in the human. The model of saltation and stasis predicts a majority of daily growth velocities clustered around zero, and a minority of high growth velocities, that is, a bimodal distribution. The frequency distribution of observed daily growth velocities instead approximated a single Gaussian distribution, indicating continuous growth. We conclude that linear growth, in the most accurate mammalian system yet studied, is continuous, not saltatory.

A similar analysis was published in 1995, No evidence for saltation in human growth, by Hermanussen and Geiger-Benoit. Again, the paper is not publicly available and I could find no graphs.

The skeptical claims were disputed by a paper from 1996, Is growth saltatory? The usefulness and limitations of frequency distributions in analyzing pulsatile data, which reports:

If the FDGV [frequency distribution of daily growth velocities] is highly skewed, then it is consistent with saltatory growth. However, if the FDGV is not highly skewed, then it is consistent with both the saltatory model and a smooth, continuous growth model, and thus, the results are ambiguous. We conclude that FDGV analysis is not a valid method to exclude saltation and stasis growth processes in longitudinal growth studies.

I came across this published exchange from 1995, where Heinrichs, Munson, Counts, Cutler, and Baron share these data:

to which Lampl, Cameron, Veldhuis, and Johnson reply with this:

There’s also a paper from 2011, Infant head circumference growth is saltatory and coupled to length growth, by Lampl and Johnson, but the graph is not very convincing in that the saltations appear to line up with the precision of measurement:

So here’s my question

Whassup with this saltatory growth thing? It shouldn’t be so hard to study if you just get enough data. It’s not even like you’d need to be measuring a kid for months. Taking several measurements per day for a week or two on a bunch of kids and this should settle it, once and for all, no?

But I couldn’t find much literature on the topic. Do people just believe that the finding was an artifact of bad data collection, so nobody’s followed up on it? The original paper from 1992 has 515 citations on Google scholar, but I couldn’t find any review on the saltation question in particular.

What’s the state of knowledge on this one? It seems weird to have such a specific and measurable question that is still unsettled in this way.

Coyote vs. Acme (also, for free, my take on The Odyssey)

Andrew — Sun, 30 Aug 2026 15:30:03 +0000

First, let’s get this one out of the way: my take on The Odyssey movie. It was brilliant, and I especially like how they handled the gods. It was clear that the gods were real—that storm after they kill the cattle (ummm, spoiler alert, sorry!)—but I liked how the gods manifested themselves indirectly. It seems to me that, in that universe, that’s how it would work: the gods would be involved everywhere, and as entities with their own free will (not merely labels for different aspects of the natural world), but they would affect us through nature. The natural world would be the medium through which the gods act. To me, that makes much more sense than gods being some sort of supernatural add-on to nature in the manner of 1940s superheroes.

Come to think of it, that’s kind of what bothers me about those silly studies of ESP, faith healing, intercessory prayer (see section 9.7.3 of this book), etc. The proponents of these ideas present these elusive phenomena as supplements to the usual laws of nature, anomalies that can be isolated using randomized experiments. To the extent these things really exist, I would expect them to work through, rather than on top of, nature. I’m an unbeliever myself, so I’m not saying I’m right on this; it’s just how I feel. I could imagine a god or gods intervening in the world; I would just picture that happening as it does in the Odyssey, in a way that’s clear in the context if you believe in it, but with a sort of plausible deniability for unbelievers, or for believers in a competing deity.

I also thought the scenes with the Trojan horse and the sack of Troy were just amazing. I’d never before thought of that wooden horse (oh, spoiler again!) as being such a corporal object. Seeing them laboriously drag the horse, the scenes with the men inside, and then the horrifying scenes after the gates of Troy are opened . . . the movie opened this up to me in a way that movie adaptations of literature rarely do. The recent Dune movie, for example, was brilliantly executed and made its world seem very real, but I feel like I’d already had that from reading the book. In contrast, the movie of the Odyssey took my understanding to a new level, which surprised me, given how familiar the story is of the Trojan horse.

The other scenes in the movie worked well too, in the same way of mixing myth and physicality. For example, the cyclops and the giant knights were not realistic, but you remember that it’s Odysseus telling the story and exaggerating all the way. The scenes were filmed in a way that looked real but at the same time featured impossible beings, which to me was a perfect way to convey the way that tales would be told in that era. So, yeah, the movie was a revelation.

OK, now on to Coyote vs. Acme.

I’m pretty much the exact target audience for this one, having grown up with the cartoons on endless reruns on TV and also being an Ian Frazier fan from way way back. No surprise we went to see it the weekend it came out.

The movie was excellent: great cinematography, wonderful visual gags, great use of the Warner Bros cartoons in major roles and cameos. The mix of live action and animation was entirely convincing. And they came up with excellent new Road Runner gags. Well done, and very satisfying. The theater wasn’t completely full, but it was mostly full, and the audience got into it, laughing hard at the jokes, cheering when Bugs Bunny came on screen, and so forth.

Just three things bothered me.

1. Road Runner says Meep Meep. They got this right in most of the film, but in the scene where Road Runner shows up to testify, he doesn’t say Meep Meep, he says Beep Beep. I can only assume the people who made the movies were superfans, so I’m surprised that they got this wrong. Sure, you could construct a within-universe explanation—Road Runner’s speech is always written as Beep Beep, and on the stand he’s careful to enunciate—but that seems like a stretch to me. This was no big deal; it just took me out of the movie while it was happening, and it surprised me because they seemed to get just about everything else right.

2. The scenes with the humans were pretty much a direct ripoff of Better Call Saul. I get it: given the physical location of the Road Runner, it makes sense for the action to take place in Arizona or New Mexico, and Bugs Bunny had that line about making a wrong turn at Albequerque, there’s the legal theme, and presumably any lawyer who’d take that case would have to be unusual in some way . . . put that all together and you get Better Call Saul. But the movie really leaned into it. The main human character looked a lot like Bob Odenkirk, and the human sidekicks could’ve fit right into that show too. It all made sense in the context of the movie, but I think they would’ve been better off either explicitly linking up with the Vince Gilligan universe—casting Odenkirk and the gang, etc.—or else doing it differently.

That said, Looney Tunes has a long and glorious history of ripping off pop-culture tropes, from What’s Opera, Doc to that saloon piano, so, in that sense, using a recent TV show to structure a Road Runner movie is canonical and fair.

3. The other thing is . . . it’s hard for me to explain this exactly, but the movie didn’t have the spark of life that makes a work of art special. Part of this may be that it was aimed at kids, so its plot and characters were simplified and kid-friendly, but it’s not quite that. After all, Charlotte’s Web is aimed at kids too, and it’s inspired and brilliant. I hate to even say this because I liked the movie a lot; it was so well done and I appreciate all the care that went into it. I feel that the world is a better place because this movie has been released. But still.

Here, let me take a shot at explaining. Remember that 1970s classic, The Taking of Pelham One Two Three, with Walter Matthau foiling a team of robbers who hijack a subway train? That movie had a lot of flaws, various dead spots, uninspired dialogue, weak characterization, all sorts of things. But it had that spark of life.

Or consider Who Framed Roger Rabbit? I loved that movie (even though it had a weak ending), and one thing I appreciated was that they invented an entirely new character, and he was convincing and appealing (“P-p-p-please”): at first he was annoying but then you get to love him for who he was. Jessica Rabbit was an inspired creation too. In contrast, Coyote vs. Acme play out the string but they don’t create anything new in that way.

Or maybe I’m being a bit too vitalistic here. One specific thing that Coyote vs. Acme lacked was good dialogue. It had wonderful gags, including excellent references to past Looney Tunes classics, but it didn’t have snappy lines or interesting speeches or anything like that. It was well plotted but the speaking roles were under-written. Maybe if the dialogue had been more memorable, that would’ve been enough to take it over the hump. I don’t know.

There’s a larger question here: To what extent is originality required to create a great work of art? I think I’m mostly arguing that Coyote vs. Acme, for all its excellent characteristics, falls short of greatness because its creators did not go to the trouble of creating anything new.

And I’m not saying this from some sort of theoretical perspective on aesthetics. It’s the opposite. I went to this movie and liked it—I was pleasantly surprised at how good it was—but, partway through and continuing to the end, I felt that the whole was less than the sum of its parts. I think I first realized this when I noticed that I was appreciating the gags but not laughing out loud at them.

That all said, I did like the movie and I recommend you see it, at least if, like me, you’re an old-time Looney Tunes fan. I would see The Odyssey first, however.

“Why did ANOVA fall out of fashion?”

Andrew — Sat, 29 Aug 2026 13:07:50 +0000

A student asks the above question.

My response: Anova is still important; it’s just been subsumed by hierarchical models.

The link is to my 2005 paper, Analysis of variance: Why it is more important than ever, which begins:

Analysis of variance (ANOVA) is an extremely important method in exploratory and confirmatory data analysis. Unfortunately, in complex problems (e.g., split-plot designs), it is not always easy to set up an appropriate ANOVA. We propose a hierarchical analysis that automatically gives the correct ANOVA comparisons even in complex scenarios. The inferences for all means and variances are performed under a model with a separate batch of effects for each row of the ANOVA table.

We connect to classical ANOVA by working with finite-sample variance components: fixed and random effects models are characterized by inferences about existing levels of a factor and new levels, respectively. We also introduce a new graphical display showing inferences about the standard deviations of each batch of effects.

We illustrate with two examples from our applied data analysis, first illustrating the usefulness of our hierarchical computations and displays, and second showing how the ideas of ANOVA are helpful in understanding a previously fit hierarchical model.

This is the paper that discusses the five different definitions of fixed and random effects from the literature; scroll down to page 20.

I guess the problem is that Anova got associated with clever-but-ultimately-bad ideas of null hypotheses and F tests. Hierarchical modeling remains very important; I think it’s an underrated topic in statistics.

What stories should we tell about science now?

Jessica Hullman — Fri, 28 Aug 2026 16:11:15 +0000

This is Jessica. Like many academics, I am concerned about what sort of new steady state U.S. universities will find themselves in after the dust settles on recent transitions. Namely, the last few years have brought funding cuts, targeted visa policy, reduced demand for grad degrees, and a general brain drain to industry (particularly noticeable in AI and computer science). It’s disorienting to think that academia has already peaked, and that the prestige ranking of the R1 faculty job over the top industry research positions (at least in computer science) might be inverting. But things feel very different than they did even a year ago. The reality of there being less money available to pay for basic aspects of research really started to hit me in the last six months. Post-covid, working on campus became less lively, but now it also feels like our collective attention is anxiously focused on Silicon Valley or Washington D.C. We hold faculty meetings where we discuss things like, Is there any way we can help local faculty members who were laid off from tenure track jobs? How will we ensure we can fund all of our own PhDs, given that TA quotas stay fixed but faculty are running out of funding runway?

To some, this is an overdue rebalancing. Nate Silver, for example, calls getting a PhD a “much worse value proposition than 20 years ago”, and predicts that elite higher ed will become “~50% less relevant in the new steady state,” which is in his eyes a good recalibration.

But it’s worth reflecting on what is lost exactly, if this dwindling of minds and resources continues. How should we think about the value of what universities provide over industry, like intellectual autonomy, or training on how to think scientifically? As a professor, I could make a list of the things that have kept me in academia–being free to work on the problems I find most important, the diversity of topics I can work on at any given time, grad students who care about doing deep work, having time to think about the best solution to a problem. But at an aggregate level, it’s less clear what the equation is.

As I was puzzling over all this, I attended a metascience conference, where there was a panel on “the social contract for science.” This is the transactional relationship dating back to at least the 1950s, by which scientists receive public funding and autonomy and society gets the benefits of scientific research. Back in the 19th century, scholars began to make a distinction between “pure” science–research unmotivated by any particular application–and applied science. The social contract takes this distinction and further presupposes a dependence relationship: what is confusingly called the “linear model”, the idea that pure science provides the well from which applied science contributions are drawn. Threaten this foundation, e.g., by letting applied science intercept too much of the resources society puts toward science, and we risk running out of useful innovations. Or so the story goes.

The social contract for science was an attempt to cement the importance of scientific understanding to society, making it an interesting counterpart to the current moment. If the actions of the current administration to direct funds away from universities, and the possibility of using AI to produce research output without understanding, are threatening our sense of what science should be, the history of science policy provides some perspective on how our expectations got shaped in the first place.

In search of the mysterious fruits of basic science

I’ve been reading the work of philosopher Heather Douglas, who has traced and critiqued the basic versus applied science distinction, the linear model as justification, and the idea of scientific freedom as limited social responsibility (see, e.g., here and here, or her book on the value-free ideal). Popularized by Vannevar Bush after WWII, in a report prepared for President Roosevelt, basic science is a reframing of pure science, presented as “scientific capital,” providing the principles and conceptions to power new products and processes years into the future. Bush called for deliberate policy to guard against the otherwise inevitable scenario where applied science drives out the pure. One of the eventual outcomes of his report was the creation of the NSF.

But despite the pragmatic nature of basic science espoused by Bush, as a derivation of pure science, it is hard to separate from less tangible values. One is that scientific understanding is a good outside of practical application, at both the individual and societal level. The earliest advocates of pure science associated it with being closer to God. Post-Enlightenment, this view gave way to a more secular superiority complex, which implied the strong character of the pure scientist, who chose to eschew wealth. “The highest occupation of mankind”, Henry Rowland called it in his Gilded Age era essay, “A Plea for Pure Science,” which bemoaned the vulgarity of attributing scientific greatness to the applied scientist rather than the pure.

From a less moralistic point of view, we’ve been encouraged to believe that a society that has rigorous ways of understanding the world is better off over one that doesn’t. Throughout history, understanding the laws of Nature has been portrayed as a good in itself, along with an intellectual life. From this view, by educating people on how to pursue deep understanding of the world, universities provide the general good of scientific thinking to society. If we believe in the intrinsic value of reading, writing, or intellectual discussion, then it would seem we should value the university as a place that provides the kind of timespan and environment needed to develop these skills.

Some argue that the university has come to serve too many conflicting purposes (research engine, job training center, credentialer, incubator of coming-of-age experiences), and should go back to its classical roots: training in oral reasoning and rhetoric, ethics and moral judgment, historical analysis, and the cultivation of taste and discrimination. This may be a useful refocusing, but it offers little consolation for the fact that the elite research infrastructure that helped this country establish and maintain scientific leadership for decades is in the process of being gutted.

If we take our intuitions from the linear model, we might protest that innovation will suffer if universities’ research purposes are deprioritized. The post WWII science-industrial complex expanded the presence of basic research in industry, but studies suggest that the knowledge generating role of corporate R&D has been on the decline for years. To the extent that basic research is the supplier of downstream applications, it would seem we need universities more than ever.

But the distinction between basic and applied science that’s become synonymous with how we envision science has never been airtight. Critics questioned how an institution could be built around a distinction that seemed to amount to little more than a difference in intention, since applied research sometimes produced important new general knowledge, and pure science contributions sometimes had direct applicability.

AI research is a recent example. Not only is serious money being made without necessarily requiring advanced degrees, research positions do not require PhDs. By some accounts, passing 30 years old puts one in the older demographic of researchers at frontier AI companies. Yet much of the visible innovation in frontier model development has been heavily concentrated in industry labs, including transformer models, scaling laws, and AlphaFold.

Of course, AI owes much to academia. The amazing thing about deep learning and LLMs, to anyone who was paying attention to NLP before these developments, is that after many years of AI research contributing interesting questions but lackluster results, the technology finally seemed to work. Would we have had the foundations for deep learning if perceptrons had not been stubbornly pursued by academics like Frank Rosenblatt at Cornell early on, picked up again in the 1980s by Rumelhart and McClelland’s Parallel Distributed Processing group, despite multiple periods during which consensus said connectionist approaches were unlikely to pay off?

The challenge is that arguing that “someday the research will pay off,” without being able to point to any hard evidence that basic research is, on average, worth the investment, is not such a convincing argument. According to Douglas, studies have been attempted to show the payoffs of basic research, but without very impressive results. Uncertainty about what time scale we should expect between discovery and application makes this kind of exercise difficult.

At the same time, it’s hard to dispute that monetary incentives can sometimes discourage exploration that would eventually pay off. In evaluating the role of academic research to AI progress, we should keep in mind the uniquely massive private investments AI companies have received, and be cautious using it as a general example. It would be premature to conclude that because progress (in terms of models’ standalone capabilities as measured by benchmarks) doesn’t seem to depend much on academic research at the moment, cutting off academic research would be immaterial. Particularly unfortunate about the historical contingency of frontier companies defining the direction of the field is that they are focused on a pretty narrow space of methods, evaluations, and design ideas. But the power and resources they hold give newcomers to AI research the impression that ideas outside this narrow space aren’t important.

Indulgence, autonomy, and social responsibility

As suggested above, it’s always been tempting to bring moral judgment to bear on the basic versus applied research divide. The latest moment with AI research is no exception. Does the moral high ground belong to those who are staying in academia, underfunded or not, to preserve university culture and their autonomy from corporate interests, or those who are willing to give up a comfortable job to shape the impact of AI as a product in the world?

From one perspective, the academy, as a haven for basic science, has always been at risk of being seen as indulgent. In practice, building a scientific career is in many ways a process of identity development and fulfillment for the scientist. Historical pure science rhetoric associated the pursuit of scientific truth with self-realization. But talking about personal fulfillment does not go over well when your opponent is promoting the idea that science could do more direct good for the country or humanity. Academic scientists have always been at risk of coming off as being self-indulgent, insular, or dilettante when they defend understanding for understanding’s sake. The current political moment is just rehashing old themes.

Another unfortunate historical association of basic science is with insularity and shirking responsibility. After WWI era advances in chemical warfare and explosives, the social responsibility of the scientist became a much greater concern. Philosopher John Dewey came down sharply on the idea that an autonomous space for pure science, unhindered by societal concerns, was something to strive for. Instead, he argued that this impetus to protect pure science was partly a convenient abdication of moral responsibility for the downstream outcomes of research, a “shirking of responsibility.’’

Dewey’s concerns came at a time where philosophy was itself seeking to be more scientific. According to Douglas, Dewey’s views on how philosophers should approach science–through greater integration of societal concerns–lost out to the argument espoused by Bertrand Russell, who instead valorized the “disinterested intellectual curiosity which characterizes the genuine man of science.” The latter view became the more accepted one, and our definition of scientific freedom arose in tandem with expectations of limited social responsibility. It’s not particularly surprising, then, to encounter beliefs that academia is not the place to go if you want to have impact in the world.

At the same time, it seems hard to deny that at this point of time in AI, where we have a large imbalance of power and resources, there is something to be said for the autonomy afforded by the university or nonprofit. Some beliefs about AGI coming out of Silicon Valley border on religious. My biggest concern if I were to join an AI company at this point in time would be losing my ability to think for myself about what problems deserve priority. Having greater agency and impact are attractive, but not if they come at the expense of one’s internal compass or values. As Brendan McCord said recently, “Autonomy is different from agency. Agency is getting things done…You can be more effective than you’ve ever been, and you can be less the author of your own life than you’ve ever been.” Against the groupthink of Silicon Valley, the value of the intellectual autonomy academia provides does feel real. Though it’s unclear how valuable this autonomy will continue to be if academics and others outside the big labs can’t retain enough funding or visibility into frontier model development to remain relevant.

The problem with defining progress as prediction and control

In a 2014 article called Pure science and the problem of progress, Douglas suggests that if the pure/applied science distinction doesn’t survive scrutiny (which she argues it does not), we’re left with an account of scientific progress based on our ability to predict, intervene, and control our world. But this is not a definition of progress we should be content with:

“Any increase in the capacity to predict or control the thoughts and feelings of human beings would count as scientific progress. An increased capacity to destroy human subpopulations (through, say, targeted pathogens) would count as scientific progress. Developing new heinous capacities would count as scientific progress. Unlike Rowland, we should have no illusions that greater causal efficacy, greater power of intervention, will in fact always provide a better society.” (p. 63)

If misaligned AI, our own creation, changes how we view ourselves and the world, if it convinces some of us it has all of our best interests at heart even as it feeds our insecurities, or pursues its own goals in the background, is that scientific progress?

Douglas argues that judging real progress requires society to weigh in. When it comes to AI, this is happening through pushback against data centers, and the pace of AI progress, and the culture of Silicon Valley. Adoption matters too, but can’t be a substitute for evaluation. We need institutions independent of the companies to help interpret what’s going on. In the midst of changes to so many of our current institutions, we should expect the story we ultimately tell to take time to sort out.

In the meantime, defending academia as a category of research, or a moral standard, is a dead end. What seems more reasonable to advocate is a set of conditions — time, autonomy, training in scientific judgment, the evaluation of new approaches independent of their profitability. The value of these ingredients isn’t easily summarizable in some neat story, because what drives scientific progress is not that simple. But institutions that help society judge what’s been achieved seem worth defending.

Where’s the Evel Knievel blockbuster biopic?

Andrew — Fri, 28 Aug 2026 13:51:13 +0000

A few years ago we discussed Objects of the class Jacques Cousteau: people who are world famous (or at least world famous in the U.S.) but of a category for which there’s only one famous person. Some examples:

• Cousteau was a world-famous underwater photographer . . . actually the only famous underwater photographer.

• Another example, also from the 70s (sorry): Marcel Marceau. A world-famous mime . . . actually the only famous mime.

• The guy who wrote All Creatures Great and Small was a world-famous veterinarian . . . actually the only world-famous veterinarian.

• Jackson Pollock is a world-famous drip painter, and indeed the only famous drip painter. But it’s not clear to me that “drip painter” should count as a category.

• Stradivari: same idea, he works if you’re willing to count “violin maker” as a category.

• Tony Hawk: world-famous skateboarder, the only world-famous skateboarder.

• Luther Burbank: world-famous agriculturalist, the only world-famous agriculturalist.

• John Philip Sousa: world-famous composer of marches, the only world-famous composer of marches.

• Margaret Mead: world-famous anthropologist, the only world-famous anthropologist.

• Alan Turing: world-famous codebreaker, the only world-famous codebreaker.

• Temple Grandin: world-famous slaughterhouse designer, the only world-famous slaughterhouse designer. She’s kind of like Joseph Joanovici, the only world-famous scrap metal dealer, in that she’s famous for her story more than for her occupation; nevertheless, as with Joanovici, the occupation is central to the story.

• Noam Chomsky: world-famous linguist, the only world-famous linguist.

• Jim Henson: world-famous puppeteer, the only world-famous puppeteer. OK, maybe we need to count Frank “Yoda” Oz too.

• Frederick Law Olmsted: world-famous park designer, the only world-famous park designer.

• Oscar Pistorius: world-famous paralympic runner, the only famous paralympic runner. This works even without adding “killer” to his title, but it was the killing that kept him famous.

• Weird Al Yankovic: world-famous writer of novelty songs, the only . . .

• We could also throw in Dr. Demento: world-famous DJ of novelty songs, but that seems like too narrow of a category.

• Anna Wintour: if you’re willing to count “fashion magazine editor” as a category.

• Freddy Mercury is the only world-famous person from Zanzibar, but I don’t think that really counts. We’re talking here about categories defined by what you’ve done, not where you’re from.

And, of course:

• Evel Knievel: world-famous daredevil, the only world-famous daredevil.

There was some discussion of how famous he still is, and commenter Manuel wrote:

Evel Knievel is just one blockbuster biopic away from being world famous.

This made me wonder: why hasn’t there been a blockbuster biopic of Knievel? Wikipedia informs me that there have been three Knievel biopics already, including a biopic back in 1971 starring George Hamilton! and a 2004 TV movie directed by John Badham, who’s not a nobody—he also directed Saturday Night Fever. Also this:

On December 19, 2024, a new biographical film adaptation of Evel’s life was reported to be in the works with La La Land director Damien Chazelle attached to direct, William Monahan set to pen the script, and negotiations with Leonardo DiCaprio to star as Knievel.

That sounds like something.

The incredible shrinking SSRN page

Andrew — Thu, 27 Aug 2026 19:32:10 +0000

I was going through comments on this morning’s post which involved the amazing productivity evidenced by the list of preprints here:

But then when I was checking something, I saw that something happened:

The number of papers dropped from 259 to 99!

Then when preparing this new post, I checked again:

And again:

I guess someone’s running the bot in reverse.

P.S. It’s down to 44. I wonder what the story is.

P.P.S. Down to 1:

The lone remaining paper is “Sequential Bayesian Pricing of AI Infrastructure,” posted 29 May 2026.

P.P.P.S. And now the page is gone. I guess the experiment has concluded, or the impersonation has been stopped, or something.

P.P.P.P.S. According to this commenter, it was SSRN that took the papers down.

This University of Chicago business school professor has authored 258 academic papers in 2026 (so far).

Andrew — Thu, 27 Aug 2026 13:05:47 +0000

Jeremy Horpedahl tells the story:

Nicholas Polson has, by my count using his SSRN page, already written 258 working papers in 2026 alone. He’s already written (or at least published to SSRN), six papers today, August 26, 2026.

OK, but who am I to talk?—I’ve written over 200 blog posts this year. But wait:

These aren’t just short notes. Most of the papers are of normal academic length: 32 pages, 27 pages, 58 pages. . . . Obviously the research productivity of Polson and his co-author Sokolov is aided by AI. . . . I have seen any academic, at least not in economics, that has really pushed it to the limit.

Horpedahl writes:

Read any single paper, and it feels like just a normal academic paper, the kind of thing that an academic might work on for a few months.

I wanted to see if I shared that judgment so I clicked through to the list of Polson’s papers on SSRN and looked for something interesting . . . ok, here’s something. It’s called Theories of Human Connection, and . . . ulp! It’s 80 pages long. The paper’s subtitle is “An Interdisciplinary Synthesis Across Economics, Psychology, Biology, Philosophy, Game Theory, and Spiritual Tradition.”

But let’s take a look. The abstract on SSRN starts like this:

Human connection is the most studied and least integrated phenomenon in the social sciences. Every discipline that examines intimate relationships captures something real that the others miss, yet no existing framework holds all dimensions in simultaneous view. This book synthesises fourteen thinkers across biology, psychology, economics, game theory, communication theory, existential philosophy, and spiritual tradition into a unified account. Morris established that the need for physical touch is an evolutionary drive as fundamental as hunger, with a biologically ordered sequence of escalating vulnerability whose disruption produces intensity without depth. Bowlby showed how early caregiving creates invisible templates governing adult intimacy, encoded in the nervous system before language exists. Becker revealed that partnerships generate value neither partner could produce alone, but assumed partners are interchangeable — an assumption Frankl demolishes by showing that the meaning generated by shared life cannot be transferred, and that meaning multiplies rather than adds to life satisfaction, explaining widespread disconnection in the wealthiest societies in history. Von Neumann’s game theory explains how mutual self-protective withdrawal, individually rational for each partner, produces the disconnection neither intended. Bateson identifies the communication structure that accelerates this collapse: contradictory demands that make any response wrong. Gottman’s laboratory models predict separation with over ninety percent accuracy from the ratio of positive to negative interactions. The Hindu philosophical tradition adds the final dimension: partnership as a laboratory in which selfishness and fear are progressively revealed and surrendered.

We argue that most relationship failures are not failures in one dimension but misidentifications of which dimension is actually in play, and that effective intervention requires the multi-dimensional map this synthesis provides.

From the preface:

The thinkers assembled here — Becker, Bateson, Von Neumann, Schelling, Keynes, Morris, Vaughan, Frankl, Maslow, Gottman, Yogananda, Vivekananda, Maharaj, Polson, Thomas, and Paltrow — did not, for the most part, know each other’s work. They worked in different centuries, different countries, different intellectual traditions. What unites them is that each identified a dimension of human connection that the others left in shadow.

Gottman, huh? The name rings a bell . . . He’s the guy who conned Malcolm Gladwell and various media outlets—maybe he conned himself too—into believing that he could predict divorces with 94% accuracy.

I searched for Gottman in the document and found a whole chapter on that bullshit! You can click through for yourself, it’s chapter 10. Here’s a key bit:

I wonder what prompts were used by Polson and his coauthor to write this article. I guess the prompts did not include, “Evaluate implausible claims skeptically.”

They bring it all together in Chapter 13, “The Cumulative Model: A Unified Theory of Human Connection”:

But let’s not forget “The Master Equation” on page 75:

Jesus Christ. I’ve heard that Cambridge University has an open position in their school of education . . . this kind of thing would fit in very well there, no?

In all seriousness, no, I don’t think this “feels like just a normal academic paper, the kind of thing that an academic might work on for a few months.” At least, not the sort of thing a non-bullshitting academic might write.

I know Nick Polson—he’s a statistician, and he’s done lots of solid work over the years! What happened here?

Here are a few possibilities:

1. It’s an experiment or a joke. But if it were a joke I’d think there’d be some internal clues, no? It’s hard to imagine playing the whole thing straight. And if it were an experiment, I’d expect they’d all read like straight-up statistics papers, nothing so obviously bogus as “Theories of Human Connection.”

2. Someone else is impersonating Polson. Seems unlikely, but it’s possible. It might not even be personal. Maybe all this is an experiment, not by Nick but by someone else who programmed a chatbot to choose the name of a successful academic and then spew the internet with papers attributed to him. If so, how horrible.

3. “Intellectual squatting.” That’s the conjecture of Michael Makowsky, who writes:

The nice version is it’s putting out a series of half-baked papers in the hopes of establishing a property right to the underlying ideas at an earlier stage of the research process than previously possible. The less generous interpretation is it’s dumping a series of haystacks on the plains and laying claim to the needles probabilistically within each.

I guess . . . but what does Nick ultimately get out of it? Invitations to speak at more conferences?? I don’t get it.

Makowsky writes:

Imagine you are a person who has highly esoteric, potentially important ideas every day. Many of those ideas you suspect, based on some combination of experience and ego, are new in at least one dimension. You would like to get credit for that newness. For being first. What’s the problem?

The problem is that scholarship remains more perspiration than inspiration. Having a new idea is great, but it takes years to work through the nuance in sufficient detail that you can convince your peers of the coherence and originality of the contribution. During the minutes each day you are not working on this singular project you have the inspiration for other ideas, sometimes multiple within a single day. How frustrating is the proposition that someone else gets credit for the originality of contribution just because they had time to reveal it to the world while you were embroiled in your investigation of what is only one of your many score ideas!?

Ah, but meta-level inspiration has struck you! What if you took each one of those ideas, spent an hour curating a series of prompts around it, and then let Chat GPT (or another LLM) fabricate an entire research paper around it? . . .

But . . . that’s what blogging’s all about! Often when I have an idea or a reaction, I blog it. No need to pipe it through a chatbot; I’ll just save the cycles and post it right here.

To return to the 256 working papers, I can think of one more motivation:

4. Education. This seems like the most plausible explanation to me. Polson has had an active research and teaching career, and he’d like to share his insights with a broader audience than the readers of his published papers and the students at the University of Chicago business school. And one way to reach people is . . . econ preprints! So Nick picks 258 interesting topics, writes some prompts for each, and produces the articles. I guess he’s programmed a bot to do this. He just feeds it the prompts and the bot writes the paper and posts it directly to SSRN.

That could explain the mystery of how that ridiculous 80-page article with “The Cumulative Model: A Unified Theory of Human Connection” (shades of Stephen Wolfram!) ended up there. Not only can’t you expect an author to write 258 articles of that length in less than a year, you can’t expect him to read all of them too. The content of that bizarre article could be as much a surprise to Polson as it was to me.

This then raises a question: setting aside the motivations of Polson (or his impersonator), do these 258 papers have any value?

It’s hard for me to answer this question, given that I’ve only looked at one of them. My guess is that the net value of the papers is negative, in that the amount of time that people (including me) have wasted going through them outweighs any positive contributions that might have been there.

My suggestion

Here’s what Nick could do on this, which could have value: Take these 258 prompts and write an article (himself, not using the chatbot) explaining why he thinks these ideas are important. Aki and I wrote a paper a few years ago, What are the most important statistical ideas of the past 50 years?. Nick could write something similar: What are the 258 most important things in statistics to learn today? Or something like that. I’m not saying it would be easy—it would take more effort than programming a chatbot to spam SSRN—but valuable products often take work to produce. Nick has tenure and could set aside the time to do it.

Also I’d recommend withdrawing all those papers from SSRN. Withdrawing 258 papers seems like a lot of work, but I’m sure he could easily program a bot to do the job.

P.S. There’s a further twist: there are two accounts for Nicholas or Nick Polson at the University of Chicago business school; see this comment thread. This would seem to be consistent with the “social experiment” hypothesis (if Nick decided to set up a separate account to play around with) or the “impersonation” hypothesis (if the bot that wrote and posted these papers was not created by Nick at all). The whole thing remains a mystery to me.

P.P.S. OK, I did a bit more nosing around.

SSRN allows you to list the papers in time order. If you go to Nick’s SSRN page linked from his website, you’ll see 16 papers, with the first (“The Impact of Jumps in Volatility and Returns”) being posted on 1 Jan 2001, then others through the next two decades, with the most recent being “Deep Learning in Characteristics-Sorted Factor Models,” posted on 23 Sep 2018 and last revised 26 Jun 2023.

If you go to the SSRN page with all the fake papers, it starts with “Kramnik vs Nakamura or Bayes vs p-value,” posted 7 Dec 2023. It’s a badly written paper—I’m guessing not AI, just text by a non-English-speaking author that was not ever checked by native speakers before posting. This rings a bell . . . I actually have a blog post on this paper, scheduled to appear next year. Next on the list is a 25-page paper, “AI and Vivekananda,” posted 5 Mar 2024, then a gap of two years until another AI-related paper appeared on 9 Mar 2026, then on 11 Mar 2026 came the aforementioned “Theories of Human Connection.”

So, yes, Polson has two SSRN pages, but they have no overlap in time. He also has papers on Arxiv, including the intriguingly-titled “Bayes with No Shame: Admissibility Geometries of Predictive Inference,” dated 24 Aug 2026 . . . Hey, that’s just 3 days ago! Oddly enough, I can’t find this one on SSRN.

But what about the article itself? I don’t have the patience to read it, but I did catch that it mentions the martingale property, which is one of my current interests—that’s cool. But, just flipping through, it looks much more substantive—much more like a real scientific paper—than that horrible “Theories of Human Connection” thing. This could be a tribute to the power of modern chatbots to create something so convincing.

P.P.P.S. Update here: The incredible shrinking SSRN page.

P.P.P.P.S. Vadim Sokolov, coauthor of several of the chatbot-assisted papers, comments here. Assuming this comment itself is legitimate, those 258 papers were not an experiment or a hoax, nor was there any impersonator, nor was it intellectual squatting. Rather, Polson and his coauthors just had 258 different things to say, and they felt the best way to do so was using a chatbot to write tens of thousands of words on these topics.

Also, according to Sokolov, they did not program a bot to do this. He reports that at least one of the papers went through multiple rounds of review, so even if a chatbot was involved in that one, the human contribution was much more than inserting a prompt. He says that some of the papers “are much newer and were developed with much more extensive AI assistance. Modern AI made it possible to develop and complete this material at a speed that would previously have been impossible.”

The fallacy in this statement by Sokolov is the idea that using a chatbot to turn a few hundred words of prompts into ten thousand words of text is a way to “complete this material.” Nothing’s being “completed” here in any intellectual sense; the process just adds pages and pages of froth. I think that Josh’s analogy of this to the Sorcerer’s Apprentice is apt.

P.P.P.P.P.S. Polson appears to think he’s proved the Riemann hypothesis (see comments here and here). I would think this would cause some concern among his collaborators.

Postdoc and doctoral student positions in Bayesian workflow at Aalto, Finland

Aki Vehtari — Thu, 27 Aug 2026 07:58:58 +0000

This job ad is by Aki

I’m looking for postdocs and doctoral students to work on Bayesian workflow. The candidates need to have knowledge of Bayesian inference and some experience with building models (for real applications, as part of methods development, or as part of courses). Although we have published Bayesian workflow book, there is still a lot more to do. The focus in the group is in cross-validation, model checking and inference diagnostics (see my publication list).

All positions are fully funded and the salaries at Aalto CS are 53k€-55k€ / year for postdocs and 40k€-45k€ / year for doctoral students. There are occupational healthcare and other benefits. Postdoc positions are typically offered for up to three years and doctoral student positions for four years. Starting dates are flexible and the details of each position will be agreed individually.

You can apply via joint ELLIS Institute Finland call and pick me as your preferred supervisor.

(1) “Do you think the culture of research has genuinely changed since the replication crisis became widely discussed, or has it mostly generated new compliance rituals around pre-registration and open data while leaving the underlying incentive structure intact?, (2) Regarding Columbia University, “is there a statistical or social scientific way of understanding how institutions lose the ability to accurately perceive their own situation?”

Andrew — Wed, 26 Aug 2026 13:40:00 +0000

Luke Ford writes:

[Regarding] the replication crisis, researcher degrees of freedom, and the gap between what statistical methods claim to establish and what they can actually support . . . Looking at the current state of the social sciences, do you think the culture of research has genuinely changed since the replication crisis became widely discussed, or has it mostly generated new compliance rituals around pre-registration and open data while leaving the underlying incentive structure intact?

And a question about Columbia specifically since you are there: the university has had a difficult two years in ways that have played out publicly. From your position as someone who thinks carefully about institutional incentives and measurement, what do you think the administration consistently misread, and is there a statistical or social scientific way of understanding how institutions lose the ability to accurately perceive their own situation?

My reply:

1. I’m loath to give an answer about the changes in the culture of research because I have not studied this systematically. My impression is that, yes, there’s more skepticism and less acceptance of noisy N=38 papers in psychology, etc., and less toleration for unfalsifiable evolutionary psychology and that sort of thing. On the other hand, perhaps this has just shifted from the science establishment to social media. Ten or fifteen years ago, there was a pipeline (partly abetted by Jeffrey Epstein) from researchers at top universities to publication in top journals to books, NPR, Ted, Gladwell, Freakonomics, etc., and lucrative speaking and consulting gigs. So you get people like Marc Hauser or Albert-Laszlo Barabasi or Brian Wansink or Dan Ariely doing the basic research (such as it is), academic middlemen such as Steven Levitt and Cass Sunstein as promulgators, and the universities, journals, and prestige news media as part of this system (as for example here: https://statmodeling.stat.columbia.edu/2023/08/31/the-variation-ignoring-junk-science-thats-promoted-by-association-for-psychological-science-and-related-academic-celebrities-its-like-a-poker-player-thinking-okay-if-push-all/).

Nowadays, though, social media runs on its own steam, and the models for academic junk science are researchers such as Andrew Huberman and Dr. Oz, who cut out the middleman and promote junk science directly, sell supplements, etc. And social media is full of fake news and AI slop. They don’t really need NPR, Ted, Gladwell, Freakonomics, etc., anymore; they can do it on their own. So, in short, yes, I do have the impression that science has reformed from the bad old days of 2010-2015 (about which, see this article with Simine Vazire: https://sites.stat.columbia.edu/gelman/research/published/jmmss-3062-gelman.pdf), but maybe the public intellectuals don’t need academic science anymore; they can just make up whatever they want on their own.

2. My take on Columbia is similar to my take on many institutions, which is that they have an executive function but minimal legislative or judicial functions; I discussed this here: https://statmodeling.stat.columbia.edu/2018/01/19/lesson-charles-armstrong-plagiarism-scandal-separation-judicial-executive-functions/ and here: https://statmodeling.stat.columbia.edu/2025/11/11/from-the-three-branches-of-government-to-the-bidirectional-nature-of-legal-reasoning-in-a-way-that-is-similar-to-how-statistics-works-and-should-work-in-the-real-world/. As a result, their decisions are made on consequentialist rather than proceduralist gounds, and over and over again the administration takes the seemingly reasonable decision to cover up misdeeds.

Ford posted this discussion on his blog. It was kinda weird seeing myself discussed as a sociological object, but, fair enough, I’m a public figure, and people can say what they want as long as they don’t misrepresent my writings or claim that I said something I never said.

Ford’s assessment is accurate that I’m not very good at strategic behavior so often I don’t even try. It’s similar to how I’m a bad negotiator so usually I’ll just try to make my goals clear and not try to optimize, following the “Getting to Yes” principle that the main thing getting in the way of smooth negotiation is ignorance of other people’s goals. I think back to various successful and botched negotiations I’ve been involved with in the past, and almost always the problems come with struggles over details without there being clarity on the goals of the different parties.

Survey Statistics: more on SynthMargins and Bayes-Raking

shira — Tue, 25 Aug 2026 20:00:09 +0000

Last week we discussed SynthMargins, a method from the poster Modeling Complex Contingency Tables that uses partial information (margins) about poststratification variables. On theme for this series (“it is the people”), the authors commented thanking Andrew for introducing them, folks from different fields with a shared goal: Max Goplerud, Shiro Kuriwaki, Jens Wiederspohn, Adam Conner-Sax, and Philip Greengard.

Bob Carpenter shared a Stan example to get a flat prior over tables that match specified margins. I think this prior would imply a prior on what the authors call alpha0, the covariance coefficients for the target geography. The method as described in the Modeling Complex Contingency Tables poster seems to focus on point estimates:

In other news, Shiro responded on Twitter to one of my questions about their Application 1 (ACS): “the density plot shown there is an empirical density of 1700+ TREs, where each error is for a non-Southern county.”

I have 2 remaining questions:

What are the covariates w in this ACS example ?
Suppose you also have survey data in Palo Alto. Would this be added to the training tables ?

Their Application 2 asks if lower postratification table reconstruction error improves the downstream MRP:

In the comments last week, Shiro cited related work by Si and Zhou (2021) who propose a method called Bayes-Raking to incorporate known margins into modeling. They found Bayes-Raking was similar to raking in the overall mean but outperformed raking for subgroups:

It would be interesting to directly compare Bayes-Raking to the poster’s SynthMargins !

Bayesian Workflow free pdf!

Andrew — Tue, 25 Aug 2026 13:00:56 +0000

Our wonderful new Bayesian Workflow book is now available as a free pdf! Just go the link—it’s right there!

I recommend getting the hard copy too because you’ll want to be able to read it while working on the computer, and the cost of the book is trivial compared to the benefit from faster learning that you will get by being able look at the book without taking up valuable screen real estate; also you can see connections when flipping through the pages that might not be apparent by viewing one page at a time on a screen.

Conversely, if you have the hard copy, you should still download the pdf because it fixes a bunch of minor errors that we caught after the book went to press. Also in the printed version we accidentally repeated some of the exercises in chapters 2 and 3. For the pdf we fixed this.

Regarding the content, as I wrote last month, with Bayesian Data Analysis, the big steps forward were:

Going beyond Bayesian inference to also consider Bayesian model building (as a researcher, you construct the model, it isn’t just given to you as in a textbook), model checking (breaking through the absolutely horrible attitude, common to Bayesians in the early 1990s, that the model was “subjective” and thus should not be checked), and model improvement (continuous model expansion, not the misguided idea of assigning posterior probabilities).
Going beyond simple conjugate models. BDA had lots of hierarchical models, also lots of computational tools so that you could fit the models you want by putting them together from understandable components. And I like how we had a clear separation between modeling and computing. The model comes first, then you figure out how to compute it. Or you set up a model that works within your computational constraints.
A Bayesian approach to sampling and causal inference. This was Rubin’s framework in which unobserved units in the population and unobserved causal outcomes are treated as missing data and are part of a joint probability model. We worked this out in chapter 7 of BDA (which became chapter 8 in the third edition of the book).
Lots of live examples. Not just “real-data examples,” but problems we’d directly worked on. This motivated us and I think it gave our readers a sense of how Bayesian methods worked not just in theory but in applied problems.
A pragmatic view of probability as a measurable quantity. That’s right there in chapter 1. Bayesian methods are not the product of a philosophical stance; they’re a way to connect models and data using probability.

I could go on and on, but for that I can refer you to the Bayesian Data Analysis book.

And these are the key innovations of Bayesian Workflow:

Going beyond Bayesian data analysis (model building, inference, model checking, and model expansion) to consider the larger process of statistical modeling, including comparisons of multiple models fit to a single dataset.
A fuller use of informative priors. This is a big deal. In BDA we still had a bit of the Bayesian cringe going on. One reason we’ve moved toward stronger priors is that the replication crisis has taught us that the amount of prior information available in any given problem is often approximately the same as the information coming from an experiment (see here, for example). Informative priors also fit our increased focus on generative modeling, and we’re doing a lot more prior predictive checking to understand the implications of our models.
More integration between modeling, data analysis, and computing. One way to see this is that the Bayesian Workflow webpage has the code to run all our examples. We also have lots of code snippets in the text as a way of demonstrating the way in which coding is central to our statistical workflow.
Lots more live examples. It’s been 30 years since BDA first came out. One reason that Bayesian Workflow has 11 authors is that different collaborators worked on different examples (but the three principal authors read through the entire book, so the general approach should remain coherent).
Simulation-based experimentation. This is something my colleagues have been doing more and more over the years. At its most basic, simulation-based experimentation provides a best-case baseline for statistical methods: if you can’t recover your quantities of interest with sufficient accuracy under ideal conditions (when your data are simulated from the model you’re fitting), then you know you’re in trouble. And often this is the case! Beyond that, we can simulate from one model and fit another, and see what happens. Simulation experiments aren’t always so easy to construct, as they involve specifying the entire data-generation process. But we think this is effort worth expending, as it involves thinking about the problem you’re working on.

I could go on and on, but for that I can refer you to the Bayesian Workflow book.

What contributions can academic statisticians make to sports analytics? (a discussion related to the launch of the new open-access Journal of Statistics and Data Science in Sports)

Andrew — Mon, 24 Aug 2026 13:40:19 +0000

Related to our recent post on the absurdity of open access fees, somebody recently informed me that a group of statisticians that work in sports decided they were sick and tired of this with the Journal of Quantitative Analysis in Sports and launched a new open access journal, the Journal of Statistics and Data Science in Sports:

JSDSS was founded on three core principles.

First, our commitment to open access is more than just lip service. At JSDSS, open access means free to read and free to publish—no exceptions. JSDSS is a Diamond Open Access journal. Research supported by academic institutions, public funding, or personal effort should not require a payment to reach the audience it deserves.

Second, reproducibility is essential and not an afterthought. The credibility of sports analytics research depends on our ability to verify, replicate, and build upon earlier work. JSDSS will actively incentivize transparency in data, code, and methodology, and work that meets our reproducibility standards will receive a special designation recognizing this commitment.

Third, we believe sport is a rich and underutilized laboratory for statistical and data science innovation. From player evaluation and in-game strategy to league design and fan engagement, sports data present compelling, real-world problems that demand rigorous and creative analytical thinking. We intend for JSDSS to be the definitive venue for this work.

Cool! I should send them something. We think about statistics and data science in sports a lot around here.

My only concern is that I get the impression that the cutting-edge work on sports analytics is happening outside of academia. So I hope that this new journal can get useful contributions from people who are working in sports analytics who have material they can share without compromising their competitive advantage.

What contributions can academic statisticians make to sports analytics?

Or we could flip it around and ask, What contributions can academic statisticians (like me!) make to sports analytics? Here are a few things:

– Developing general methods that can then be used in sports analytics (as here);

– Writing textbooks explaining general methods that can then be used in sports analytics (as here);

– Teaching students who can then work in sports analytics, or consulting on sports analytics projects, which can be thought of as a form of intense teaching;

– Doing work in sports analytics which, although it is not cutting edge, can still give valuable insights (as here);

– Evaluation and criticisms of published work in sports-related topics (as here);

– Contributing to the sports analytics community (as here, and indeed as in the present post);

– Collaboration on sports strategy, or sports medicine, or the sociology of sports, or various other places where sports links up with academic research;

– Clearing up confusion on topics related to statistics and sports (as here), along with social-sciency thinking about how these misconceptions persist;

– Sports-related research where it can be helpful to have an outside perspective, something available to academics who aren’t on a deadline (as here).

There are probably some more things I didn’t think to include on this list.

Head to head on 125 St: The Jamaican beef patty battle!

Andrew — Sun, 23 Aug 2026 22:19:49 +0000

My new posts here have a one-year waiting list, but I’m bumping this one up because it’s important.

OK, we went on over and did a head-to-head Jamaican beef patty taste-off. As the above wrappers indicate, we compared the spicy beef.

None of this randomization, tea-tasting crap, we just kept taking bites of each patty until they were done.

Both were good, but the classic Golden Krust was definitely better than the newcomer, Juici Patties. The Golden Krust patty had a more delicious filling which was also better integrated with the crust. By comparison, the Juici Patty had a bit too much crust and was too empty inside—it didn’t work as well as a whole. Also the Golden Krust patty was $4.05 and the Juici was $4.25.

In summary:

1 Golden Krust patty > 1 Juici patty >>>> 1/1381 of a conference featuring Grover Norquist, Gray Davis, and a rabbi.

To the three different people who sent me chatbot emails today

Andrew — Sun, 23 Aug 2026 20:45:36 +0000

Just send me your goddamn prompt.

Failing upward, Norwegian style

Andrew — Sun, 23 Aug 2026 13:10:59 +0000

Wendy Moore, in a review of a book by Oliver Basciano, writes:

Leprosy is caused by a bacterium, Mycobacterium leprae, first identified by a Norwegian doctor, Gerhard Armauer Hansen, in 1873 . . . Convinced, wrongly, that leprosy was hereditary, he spurred Norway to introduce the Seclusion of Lepers Act (1885), which enabled the authorities to remove people from their families to isolated leprosaria.

OK, fine, everybody makes mistakes, better to err of the side of caution bla bla blah.

But then comes this stunner:

After Hansen injected a virulent strain of the disease into the eye of one patient, Kari Nielsdatter Spidsøen, without her consent, she took him to court. He was stripped of his hospital post in 1880, but continued to oversee Norway’s leprosy policy.

Whaaaa?

Further research (i.e., I went to the Hansen’s wikipedia page) yielded this:

Hansen had attempted to infect at least one female patient with the nodular form of leprosy without consent, and although no damage was caused, the case ended up in court and Hansen lost his post at the hospital.

“No damage was caused,” huh? He just injected her in the eye, that’s all. No harm, no foul, I guess.

Just amazing that he continued to run government policy after that. Kinda reminds me of how noted terrorist John Poindexter was tasked by the U.S. government to run a terrorism prediction market. I guess it makes sense—he was a true expert on the topic.

Also reminds me of that unfortunate psychology researcher who keeps coauthoring fraudulent research papers, got disciplined by MIT, and then left for a prestigious chair at Duke University, ran an advice column in a major newspaper, and had a TV show made about his research. Actually two TV shows: one is a highly critical documentary and another is a fictional show where a character based on him is the hero.

Or that political figure, I can’t remember his name now, who keeps citing discredited, fraudulent, fake, and racist research claims, and at the time of this writing he remains in charge of health policy in a major industrialized country.

Some people end up in prison for minor crimes. Other people do things like inject people in the eye with a virulent strain of leprosy, and they get to make government policy.

This is one of the worst scientific papers I’ve ever seen.

Andrew — Sat, 22 Aug 2026 13:32:42 +0000

A frequent commenter pointed me to this paper, “Sport and longevity: an observational study of international athletes.”

All I can say is . . . Wow! This paper is an absolute clinic in bad quantitative social science research.

I’ll leave it as an exercise for the reader to count up all the problems.

The key takeaway: For God’s sake don’t play volleyball. It’ll reduce your life span by 5 years, and the result is statistically significant, with a p-value of 2.4e-11.

I used to play some volleyball. Our team made the B-league playoffs in the MIT intramural league one year. Now I’m scared. Who knows what all that jumping did to our fragile young bodies.

Also this:

Your tax dollars at work!

Too bad the journal doesn’t seem to publish its reviews. It would be hilarious to see the referee reports for this one.

P.S. To those of you who think I’m being mean here, “punching down,” etc., let me just say a few things.

1. It’s not personal. I know nothing about the authors of this paper and I purposely did not include their names in the post. They may be wonderful people, indeed they may do wonderful research in other areas.

2. As noted above, public funds were spent on this project. And the paper was published, i.e. it’s there for anyone to read. If you don’t want to be criticized, don’t publish.

3. Unsupported claims get out there and they don’t go away. For example here’s what happened when I googled *pole vaulting longevity*:

This is just one silly example. But, yeah, I do think we should be bothered by broadcasts of unsupported scientific claims, whether they’re about volleyball, himmicanes, faith healing, air rage, governors’ lifespans, or anything else. It’s bad science and it’s part of our culture of B.S. Even if the authors of this particular paper are completely sincere in their efforts. Remember, honesty and transparency are not enough.

Here are the talks from StanCon 2026!

Andrew — Fri, 21 Aug 2026 13:25:35 +0000

StanCon 2026 just happened!

And here are the talks:

Matthew Kay
Adrian Seyboldt
Paul-Christian Bürkner
Charles Margossian
Javier Enrique Aguilar
Sean Pinkney
Nikolas Siccha
Anna Dreber
Kaitlyn Johnson
Jonas Wallin
Pranav Sanke
Anna Elisabeth Riha
Colling Cademartori
Tim M. Szweczyk
Bob Carpenter
Chandler Ross
Nils Rudi
Fredrik Ronquist
Jakob Torgander
Aleksi Lahtinen
Ville Laitenen
Zeno Romero
Soham Mukherjee
Steve Bronder and Brian Ward
John Ashley Burgoyne

Lots of great stuff here. Check out the titles of the talks—all sorts of different topics.

Time to start planning for StanCon 2027.