I’m a first-year statistics major and wanted to reach out because statistics and political science were my two main options when choosing a major, and I’m still considering law school down the line.

I’m very interested in how statistical thinking intersects with politics, public policy, and legal reasoning, and your career seems to embody that combination. I was hoping to ask whether you have any general advice for a student early in college who is trying to keep these paths open and build a strong foundation.

My response: I think I’m too old and too privileged to offer much useful advice to a young student just starting out. My own experience is that I always loved math but I didn’t want to do pure math–it just seemed pointless to try to prove theorems, knowing that there would be other mathematicians who were better than me, proving better theorems–, I studied physics, but then I took some classes in probability and statistics and the subject really grooved with me. I also took some political science classes, and it was interesting to see the relevance of mathematical and statistical ideas in understanding various aspects of voting and political representation. Back then the state of the art in political analytics was pretty low. There was some good work, but also lots of unthinking applications of inappropriate models, so there were lots of openings for a student to do innovative work. I guess things are even better now, in the sense that you can do innovative work at a much higher level, making use of what’s already out there.

As for advice: ok, yeah, I still think it’s a good idea to “learn to code.” Coding is the most rigorous thing out there, and it’s how we understand our statistical models (as discussed in our Bayesian Workflow book). Work on real applications where you can. And choose your courses more based on the quality of the teachers than on the descriptions of the classes.

And, ummm, anyone else out there have any further advice to offer?

A Study Is Retracted, Renewing Concerns About the Weedkiller Roundup

Problems with a 25-year-old landmark paper on the safety of Roundup’s active ingredient, glyphosate, have led to calls for the E.P.A. to reassess the widely used chemical.

In 2000, a landmark study claimed to set the record straight on glyphosate, a contentious weedkiller used on hundreds of millions of acres of farmland. The paper found that the chemical, the active ingredient in Roundup, wasn’t a human health risk despite evidence of a cancer link.

Last month, the study was retracted by the scientific journal that published it a quarter century ago . . .

The 2000 paper, a scientific review conducted by three independent scientists, was for decades cited by other researchers as evidence of Roundup’s safety. It became the cornerstone of regulations that deemed the weedkiller safe.

But since then, emails uncovered as part of lawsuits against the weedkiller’s manufacturer, Monsanto, have shown that the company’s scientists played a significant role in conceiving and writing the study.

Oh, what was that significant role?

Monsanto employees praised each other for their “hard work” on the paper, which included data collection, writing and review. One Monsanto employee expressed hope that the study would become “‘the’ reference on Roundup and glyphosate safety.” . . .

In retracting the study last month, the journal, Regulatory Toxicology and Pharmacology, cited “serious ethical concerns regarding the independence and accountability of the authors.” Martin van den Berg, the journal’s editor in chief, said the paper had based its conclusions largely on unpublished studies by Monsanto. . . . There was no disclosure of a conflict of interest on the part of the authors beyond a mention in the acknowledgments that Monsanto had provided scientific support.

There seems to be some controversy about the safety of this pesticide:

Dr. Philip J. Landrigan, who is a pediatrician and epidemiologist and the director of the Program in Global Public Health at Boston College . . . recently chaired an advisory committee for a global glyphosate study that found that even low doses of glyphosate-based herbicides caused leukemia in rats. . . .

Laboratory tests first flagged potential risks posed by exposure to glyphosate as far back as the early 1980s, and soon after, studies of Midwestern farmers exposed to herbicides started to show an increase in certain cancers. A U.S.-backed effort to eradicate coca fields in Colombia by spraying glyphosate from planes onto hundreds of thousands of acres of cropland led to widespread reports of illnesses among residents.

The 2000 paper declaring glyphosate safe was published against that backdrop. . . .

Bayer has paid out more than $10 billion to settle approximately 100,000 Roundup claims . . .

And then there’s the bigger picture:

The retraction points to a wider problem of research secretly funded by industries like tobacco and lead, said David Rosner, co-director of the Center for the History and Ethics of Public Health at Columbia University. “Shading the science to favor the corporate interest,” he said, was likely “the rule rather than the exception.” Journals needed to “press scientists more forcefully to identify conflicts of interest,” he said. “Huge financial interests are at stake.”

The most disturbing thing in the linked emails was that the Monsanto people referred to the authors of that paper as “third party experts” and as “independent experts.”

But if they were paid by Monsanto, then it doesn’t seem accurate to characterize them as “third party” or “independent” experts.

The research article appeared in 2000. The emails were released in 2017 in the process of a lawsuit. The article was retracted in 2025 (although the official publication date of the retraction is February, 2026, i.e., a month after the writing of this post).

I don’t know what to think about all this. On one hand, how much can you trust research on a controversial topic that was written, funded, and reviewed by one of the parties to the controversy? They do say this in the paper, “In this effort, the authors have had the cooperation of Monsanto Company that has provided complete access to its database of studies and other documentation,” but it sounds like Monsanto provided more than data access.

I guess I could try to read the original article . . . .OK, let’s take a look:

The paper goes into details on three studies from 1988, 1991, and 1992 of oral doses in rats over 10 or 15 days. Then it looks like there was another study from 1973 on oral doses in rats for 15 days, and then three studies of skin exposure from 1983 and 1991, two on monkeys and one on humans. Then there’s a mouse study from 1992, rat studies from 1987 and 1992, a dog study from 1985, a rat study from 1979, a mouse study from 1983, a rat study from 1981, . . . ok, I’m getting tired now. There’s not really much for me to chew on here as a statistician. It does seem that belief in these results is going to boil down to your trust in the research team, and so the undisclosed conflicts of interest are a big deal.

On the other hand . . . I’ve done research funded by Novartis–they paid my colleagues and they paid me directly too. We published a paper based on that work–two of the authors were Novartis employees and two of the other authors had worked for me at the time (more precisely, they’d worked at Columbia under my supervision). That project used Novartis data, but it was a little different from the above-discussed Roundup article in that its purpose was methods rather than policy.

Also I did some consulting for Monsanto at one point, I think! I can’t remember the details, I think I was on the scientific advisory board of some company that was doing some agricultural stuff, I went to one of their meetings and then I stopped hearing from them, actually I can’t even remember if they paid me. So I’m not gonna get on my high horse and denounce industry-funded or pharma-funded research in general terms.

Last November, Tom Dietterich asked for opinions on social media about what arXiv should do about papers that are bulleted lists, like this:

Some of the impetus to reduce narrative in papers predates LLMs.  E.g., even before the appearance of chatbots in 2022, people were arguing that we should get rid of or restructure Discussion sections in scientific papers, because authors are often tempted to use them to drift into rhetoric (“spin”) and unwarranted speculation. But most papers still include them. Maybe LLMs will be the impetus that actually shifts the norm.  

However, there are lots of ways to contextualize scientific contributions that are not simply rhetoric, and which are already disincentivized more than they should be. Some move in the opposite direction from adding interpretation, such that omitting them is like withholding the information readers need to judge the work. 

For example, one of my pet peeves with the way many AI and machine learning papers get written is that showing examples of the task is de-prioritized in order to fit in more results, especially in the main paper text. It’s very hard to evaluate how much improvements in performance matter if you aren’t given a single concrete example of the problem being solved! You made an LLM reviewer that’s great at finding errors in papers? Show me some examples of the kind of errors it’s detecting, so I can judge how much this moves forward our ability to verify science. You made a benchmark for the fairness of visual language models? Show me what the image and text prompts you’re testing the model on look like, so I can judge whether I agree that you are evaluating something meaningful versus a few people’s conception of what is politically correct. Instead we get high-level verbal descriptions of the kind of task (“detecting errors”, “fairness”, “content moderation”) and/or references to datasets, followed by lists of metrics and comparisons of how different models or algorithms performed. 

What’s weird is how comfortable entire fields can become going through the motions of evaluation without treating the task itself as part of the science. But authors are incentivized to bury the details by tight space limits on the main text, and to avoid giving reviewers more to pick apart. 

For us as human readers, seeing the examples often prompts a kind of common sense judgment about how “real” the problem is, making it harder for authors to pass off research that makes progress on made-up problems. But how much a task is likely to matter in the world is not the kind of thing current AI models are particularly good at evaluating. This makes me think of lots of other “just show me…” guidelines that tend to improve people’s ability to assess science:

Show me the plot: Don’t give me a big table of numbers, plot your coefficients so effect size and uncertainty dominate over significance (though you can still easily get that if you want by seeing which estimates include 0).  

Show me the variance: Don’t just show me the uncertainty in parameter estimates, plot the measurement variation. We found this reduced overestimation of treatment effects by lay people by a fair amount in this paper, and my coauthors also found similar effects with experts.   

Show me the interface: When gathering data from humans (whether ground truth labels for training or aligning a model, or behavioral responses to some experimental task), show me what they saw and how they were asked the questions, so I can judge how hard their task was, what biases might arise, etc. 

Show me the prompt: The LLM version of the above. If it’s long you may not have space in the main body of the paper, but all the prompts you used should appear somewhere.

Show me the failure cases: Seeing what instances throw a model or system off says a lot about how much progress has been made, and how the model may be succeeding in the other cases.

Show me the baselines: Improvements are meaningless if the reader has no idea what’s being improved over. Give the reader a bit of intuition about how the other approaches work, including the dumb ones you should definitely be beating. 

Show me the design analysis: I was on an open science panel last week where someone asked what they should look for as a reviewer and what they should report with their own experiments to help readers evaluate them, beyond open data and code. I said that something I often request in reviewing empirical papers is information on how the authors chose study design and sample size, including what effect estimates they were prioritizing with what intended level of precision or power. It’s easier to make sense of what’s been learned if you know what the authors were attempting.

Show me the raw output: Whenever you are doing qualitative coding of model outputs (or human responses for that matter) show me a couple examples of the original texts for each possible code. 

Some of these may be informative for LLM reviewers as well as humans, in the sense of helping them predict consensus human judgment on the paper, but others (like plots instead of tables of numbers) probably not so much. 

And yet . . . they keep saying some really dumb things.

Whassup with that?

My guess is that they have no editor. Even the best of us make mistakes, even the best of us write some stupid things from time to time. If we’re lucky, though, we can show our writings to some trusted person who can point out where we’re wrong. Or to some complete strangers–like you blog commenters!–who feel free to point out where we’re wrong, or where they think we’re wrong.

I absolutely looove when youall disagree with me. It’s a no-lose situation: either you find a legitimate error and then I can fix it and recalibrate my thinking as needed, or you’re wrong in your correction, but in that case it’s still a useful sign that I’ve failed to communicate clearly.

I’m speaking here of sincere criticism, not trolls or Russian agents or people who are otherwise trying to muddy the waters. But the vast majority of you do seem to be sincere, and even the trolls often have good points, and the agents and equivalents usually go away once it’s clear that we’re not going to give them twitter-style engagement.

Anyway, back to the Nudgelords . . . I think their problem is they’re too successful, so they don’t need to listen to critics. Also, not listening to critics is a contributing factor to their success! One thing that made them Lords rather than just Commoners is their unshakeable confidence.

I thought about this because I happened to come across this 2013 review by Samuel Freeman of Cass Sunstein’s book, “Simpler: The Future of Government.” Here’s Freeman:

Simpler is a follow-up to Nudge. Sunstein draws from his experiences as head of the Office of Information and Regulatory Affairs (OIRA) from 2009 to 2012. . . .

Sunstein contends that “the future of government” largely lies in policies that preserve freedom of choice. Such policies, which he and Thaler dubbed “nudges,” would encourage people to make decisions that benefit rather than harm them. . . .

“To count as a mere nudge,” Sunstein writes, “the intervention must be easy and cheap to avoid. Nudges are not mandates. Putting the fruit at eye level [in a school cafeteria, for example] counts as a nudge. Banning junk food does not.”

Uh oh . . . he’s citing the work of the discredited business-school professor Brian Wansink! As we’ve discussed before, the problem is not that Sunstein got conned by that now-disgraced food researcher, but rather that, after the problems with Wansink’s work came out, they removed all references to it from the second edition of Nudge–without reflection on how they’d been fooled. That’s where the idiocy happened.

But now let me show you the place where Sunstein really brings on the stupid. Here’s Freeman again:

Finally, rather than “Soviet-style” national restrictions on major sources of pollution, they advocate incentive-based approaches that increase freedom of choice, ideally, for example, a cap-and-trade system in which “rights” to pollute could be purchased or given away and then traded on the market.

What an idiot, to refer to environmental protection laws as “Soviet-style”! Hasn’t he heard about the environmental devastation in the Soviet Union? Soviet-style is to let factories pollute because they’re run by well-connected people, and there were no independent executive, legislative, and judicial branches to make and enforce the rules. To think of pollution restrictions as “Soviet” . . . that’s just nuts, it’s both illogical and ahistorical.

It’s really frustrating that this sort of thing is taken seriously.

P.S. You might thing, Sure, but that was 2013, and since then we’ve had the replication crisis, the winds have changed, and nobody takes that crap seriously anymore. But nooooo, here it is in 2023: bullshit nudge numbers in the New York Times in 2023. I’m not blaming Sunstein for that one; my point only is that there are a lot of people who want to believe this stuff.

P.P.S. To be fair, Sunstein can be a thoughtful writer sometimes, for example in this review of a biography of the economist Albert Hirschman. I suspect that Sunstein’s thinking is clearest when it is detached from his personal ambitions, so that instead of trying to stake out some position, he can just step back and tell it like it is. I’d like to think he could do more of that going forward.

P[voter i chooses candidate c from C] = exp(f(X_ic)) / sum_c’ exp(f(X_ic’))

We saw this model implies independence from irrelevant alternatives (IIA):

Another consequence of the multinomial logit model is a simple expression for ranked data:

P[i ranks Other then Left then Right] = exp(f(X_iOther)) / sum_c’ exp(f(X_ic’))   *   exp(f(X_iLeft)) / (exp(f(X_iLeft)) + exp(f(X_iRight)))

Train (2009) Chapter 7 calls this an exploded logit.

To derive the exploded logit:

• Train (2009) Chapter 3 explains that the multinomial logit model is equivalent to latent utilities with a Gumbel distribution.
• Powell (2023)* notes “The exponentials of the negated Gumbel random variables are Exponential random variables” and uses the memoryless property of the Exponential to derive the exploded logit.

The exploded logit form implies that the ranking of 3 alternatives can be expressed as 2 pseudo-observations: 1) choosing Other from C, 2) choosing Left from {Left, Right}.

* I got Powell (2023) from White Rose Research, not to be confused with Blue Rose Research, where I work. The paper’s subtitle “why endurance is better than speed” caught my eye. They study competitions like Backyard Ultras, where the goal is to outlast your competition.

I referred to that session in a post the other day regarding the recent case of a report from the Centers for Disease Control and Prevention that was pulled by Bhattacharya, in his additional capacity as acting director of the CDC. I’ll get back to that story in a bit, but here I wanted to talk about some larger things that bothered me in the interview.

Before getting to my disagreements, let me give my positive take, which is that both the people in the interview had an air of moral seriousness.

This is important. So much of the discourse in politics and social science these days is polluted with cynicism, whether it be from history professor Niall Ferguson decrying the “wokeness” on college campuses when he’s not encouraging college students to do “oppo research” on each other, or Lawrence Summers sleazing around with a sex trafficker and then trying to enlist his rich friends to intimidate student journalists, or Cass Sunstein writing an entire book on a topic he knows nothing about, or Sunstein’s friend Adrian Vermeule promoting election denial, or Mehmet Oz and Andrew Huberman trading off their medical and scientific credentials to hawk dietary supplements, or Steven Levitt promoting dubious claims on mind-body healing and global warming denialism (presumably because they’re cool and transgressive, respectively), or Matthew Walker torturing the data, etc etc. I’m talking about researchers who see science as a path to glory, not to understanding, and politically-minded academics who will happily promote stupid ideas that push their agenda. Beyond that there are straight-up politicians who lie, cheat, and steal, and that’s bad too–but here I’m talking about that nexus between government, policy, and the human sciences.

Anyway, Bhattacharya and Oster weren’t like that. They recognize that we’re talking about serious issues here. When asked about disruptions to NIH funding, Bhattacharya emphasized the larger goal of improving public health, making the point that they want to fund a portfolio of projects to address health challenges. I have no sense of how things are run internally within NIH, so I’m not saying I agree or disagree with his particular administrative directions, but I appreciated that he kept his eye on the ball by emphasizing ultimate goals. For her part, Oster questioned Bhattacharya on a number of issues. She too gave the sense that this is a serious topic, not just a political game.

How to do better is another question! Last month Oster wrote positively about some silly dietary guidelines recently released by the FDA, and if you read her op-ed carefully she doesn’t actually seem to agree with most of those guidelines (the best thing she could say about them was that they were “not crazy”), so I take it that in writing that piece she was making a sort of persuasion calculation that the best way to be effective is to mix the criticism with a gallon of sugar. That’s not my style. So, Oster uses a different approach than I do, and I’m sure we’d have our differences in how to interpret statistical evidence. But, again, I think she’s engaging with moral seriousness.

And it’s possible to be morally serious while still having fun. Consider Nate Silver. Nate’s an entertaining writer–I try to be too!–and I’ve had my disagreements with him regarding statistics and communication, but I think he’s coming from a place of intellectual and moral seriousness that shows respect for the challenges of political analytics and the stakes involved. Indeed, sometimes when he’s disagreed with me, it’s on the implicit grounds that he’s making progress in understanding the real world, doing some analytical engineering that is outpacing the statistical theory. I still think there’s a benefit to interrogating the edge cases where our methods break down . . . anyway, my point is that I’m not just using the term “moral seriousness” to refer to things that I agree with. I’m talking about an attitude that I see in Bhattacharya, Oster, and Silver that I don’t see in, say, Niall Ferguson or Andrew Huberman.

Now, to return to our main thread, these are the parts of last week’s interview that bothered me:

1. When asked about some news reports regarding the NIH and CDC, Bhattacharya dismissed them as “fake news.” This annoyed me for two reasons. First, he offered no evidence that the reports were untrue. Second, he was appointed by a man who spews out false statements at an amazing rate, including on the topic of public health. Who are we supposed to trust here? News reports or a political appointee? Also, Bhattacharya himself has a record of being sloppy with the facts, as I happen to know because it happened to me.

Now, don’t get me wrong, I’m not saying that Bhattacharya was lying or misinformed regarding recent NIH and CDC policies. It could well be that the news items were erroneous or misleading–and, if so, I can see how Bhattacharya would be legitimately annoyed. And he should feel free to express his annoyance! But just dismissing the reports as “fake news” . . . that’s not a serious response.

As I wrote above, I appreciate that Bhattacharya treats the nation’s public health spending with the seriousness it deserves. As a statistician, I think information needs to be treated with respect as well. Which means he should be addressing serious news reports and, for that matter, respecting the institution of journalism. Which he wasn’t doing here.

2. When the topic of vaccines came up, Bhattacharya came out strongly in favor of vaccination, and he expressed the view that it is better for vaccination to be voluntary rather than mandatory. This could be. I guess it depends on the context. For almost all my life, childhood vaccines were mandatory, just about everybody got vaccinated, and just about nobody complained about it. So mandatory vaccination can work just fine–we have decades of experience on this one. The bad news is that in the past few years, vaccination has become politicized and anti-vax attitudes have become embedded in right-wing politics. So it could be that Bhattacharya is right and the mandates will have to go, we’ll just have to accept more sick and dead kids and adults, just the price to pay for this aspect of political dysfunction. I don’t know, but it could be, so I’m not going to criticize Bhattacharya for his hot take on this issue.

What bothered me was . . . if you are going to go with a voluntary vaccination strategy, I think you’d want a strong strategy of encouraging people to choose vaccination for themselves and their kids. So I think his response would’ve been stronger if he’d also said something about how to vigorously promote vaccine usage. That’s part of public health policy too. Also, Bhattacharya doesn’t have a great track record on this issue: just a few years ago he was part of an anti-vax organization. See here for the ugly story. OK, fine, everybody makes mistakes and has lapses in judgment. But then at least he should address that, in the past, he’s been part of the problem. To just say that you want vaccines to be optional but without addressing that history, that’s not right.

3. The un-publishing of that CDC report. Bhattacharya said he stopped the CDC from publishing the report because it was using an approach called a test-negative design, which he thinks is a bad statistical method. When he said this, Oster jumped in and said that she too thought it was a bad method. It was only a brief exchange and there was no time for either of them to give a reference or to explain why they think the method is bad. In the meantime, it seems that the report has been leaked; see here. One of the authors of the report said, “I’m strongly opposed to this kind of censorship . . . It should be out in the world at large for the scientific community to judge it for what it is.”

I think the best next step would be for the CDC to release the report officially, along with a critical response from a statistician explaining how the method is flawed. Bhattacharya said it was common knowledge that the method was terrible; on the other hand, it seems that this “test-negative design” is a standard approach for studying the effect of vaccines in the population after they have been released; see also here. So at the very least it would be a valuable educational opportunity to see this article that was on the verge of publication, and to understand its purported problems. Publishing the report along with a companion article discussing its problems, that could make sense. Canceling the report without explaining why (and, no, just saying you don’t like this method isn’t enough of an explanation) . . . that’s not serious science. Scientific integrity is not being advanced by this sort of behavior.

I was also upset that Oster just jumped into the discussion to say that she, too, hates the test-negative design. Neither Bhattacharya nor Oster are statisticians. They’re health economists. It’s fine for a health economist to have an opinion on a statistical method, but, to be so sure about it, that doesn’t seem right to me. To the extent that Bhattacharya and Oster have legitimate concerns about the statistical method, they can work with a statistician to express these concerns openly and scientifically.

I’m not saying that statisticians or epidemiologists are always right or that other professionals should defer to them. Statisticians can be wrong, really wrong, and the errors can be compounded by a presumption that they know what they’re doing. So question these reports all you want. But then is the time to bring in an expert of your own, not to wing it.

Above I talked about moral seriousness regarding outcomes. There’s also moral seriousness regarding methods, and neither of the two people in that interview were displaying it. Also important is moral seriousness about communication, which has not been displayed by Bhattacharya, who has yet to come to grips with the fact that he was on the board of an anti-vax organization.

P.S. See Dorothy Bishop provides a detailed discussion of this event.

Dan has some thoughts on this, and so do I. I have a post scheduled for October (that’s the current end of the queue) with our thoughts, but he’d like you to try it out now without being influenced by our takes.

Dan did the No Vehicles in the Park survey awhile ago and got some interesting results, so I think you’d be contributing in some small way to the public good by trying out this new survey and giving him some data. Enjoy.

An anonymous statistics student from France sends in the above plots (click twice to see big versions) and writes:

I’m trying to push French pollsters to start doing MRP.

I made a poll agregator and applied it to the last 100 days of the last five french presidential elections.

I did some smoothing using an algorithm from a paper of Aki Vehtari. It is Kalman-RTS with cross-validated levels of noises.

I tested it on some simulated data to confirm it is fitting properly.

I put the data and the code on my blog.

What I shared as “the data” is the smoothed result. I fitted it on the wikipedia pages of the french polls.

On the plots, the same parties (with changed names or fusions) are on the same position horizontally to allow comparisons.

I see some periodic movements in opinion that I think may be coming from a periodic non-response.
Also, the movements seem far too large to me. I can believe 10% increase for a candidate in five years, but not in less than 100 days.

The French polling industry is in profound need of reform. A fun fact: They allow themselves to change the final result by plus or minus one point based on the feelings of the person in charge of the poll. They call that the “pifomètre” or nosemeter. I heard about this in an interview with sociologist Hugo Touzet on his book, “Produire l’Opinion: Une Enquête Sur Le Travail Des Sondeurs.” I trust his descriptions of their methods since he has interviewed their workers.

I think MRP would allow the pollsters to do predictions for the legislative elections and municipal elections, which have been largely ignored because they are too difficult and expensive with quota sampling.

I know next to nothing about French polling, but, yeah, I do think they should be using Mister P (multilevel regression and poststratification; MRP).

P.S. Here’s a fun cranky post from this student.

Poker players have collectively decided it’s not possible to play the game without trolling unless you play for “serious” money. The reasoning is something like, “obviously, people will make stupid plays like going all in every hand unless there’s real money on the line”. Outside of the implicit collective agreement to do so, this is patently absurd — people play all sorts of games where there’s no money on the line and they don’t, in general, purposely make troll moves, so there shouldn’t be an inherent reason poker can’t be played seriously when there isn’t serious money on the line, but since people have agreed to buy into this collective delusion, it seems fairly difficult to find a poker game where people actually want to play well without putting an amount of money up that’s meaningful to the people playing.

As a poker player myself, this rings true to me. OK, I’ve never been serious about the game–in grad school we had a weekly nickel-dime-quarter dealer’s choice game, mostly seven-card stud (this was before the popularity of table stakes Texas hold ’em, and “going all in” wasn’t a possibility in our game), and in the decades since then I’ve only played a few times, most recently over ten years ago. That last game included some political scientists and also some actual politicos who fit the stereotype (they were cynical and cursed a lot). It was pretty stressy, not a pleasant experience. I won a couple hundred bucks, probably more from luck than anything else, and one of the politicos was annoyed at me about that. I still think about the game, though. It’s a point of reference for me, as here, for example.

Anyway, yeah, in grad school we weren’t broke, but throwing $4 into the pot counted for something; it’s not a move we’d do just for laughs. Playing for pennies wouldn’t have been enough. And playing just to win, in the way that you might play a game of Scrabble, or chess, or ping-pong, or Uno . . . Nah, that just doesn’t work in poker.

The question is, why? Luu argues that this is just a convention, just one of the unwritten rules of the game, just as players avoid strategies using grid positions in Codenames. There’s an implicit agreement in poker not to play seriously unless the stakes compel it, and without this convention, people could play happily for low or even zero stakes, just as they do with chess or bridge. Luu:

There’s often some specific argument like “it’s more fun to play than to fold”, but most people would say this about declaring vs. defending in bridge, and yet you don’t see people randomly bidding 7NT (the maximum bid) in bridge all the time so their team is declaring and not defending, the way you see people randomly going all in in poker when money isn’t on the line (or only a very small amount of money is on the line).

I don’t know about that. I mean, yeah, I think Luu is right about people being willing to play serious bridge or Scrabble or whatever for zero stakes but not doing so with poker, but I don’t think it’s just a convention.

Some possible reasons

So let me throw out a few reasons why it’s essentially impossible to play a fun and serious game of poker not for money, even though people have no problem doing this for many other board games:

1. There’s a historical relation between recreational game-playing and gambling. I’m not an expert here, but my impression is that if you went back a hundred years ago, when people played bridge, gin rummy, poker, cribbage pretty much any card game, it was usual to play for money. Not to mention dice games, which are only played for money. Nowadays I don’t think anyone plays gin rummy–it’s just too damn boring, and there are too many other competing leisure activities.

2. Low effort, high risk, high reward strategies (what Luu calls “trolling”) exist in poker more than in other games. What would be the equivalent in Scrabble, for example? Maybe trading in your letters more often in the hope of getting a seven-letter word? But that’s a lot of work, especially if you’re not a top player. (If you are a good player, then trading in can be a legitimate strategy, just as going all-in can be a serious play for a good poker player.) In chess, you can play more wildly, more offense and less defense, sacrificing pieces for a positional advantage—and players are more likely to do these fun plays in a home game with no stakes than in a tournament where rating points are on the line. There is some “trolling” in chess too–for example, goofy openings where you purposely block off your own pieces, just to get to an interesting position unlike anything your opponent is familiar with–but that’s not quite the same as going all-in; the poker equivalent would be more like a strategy of betting in a slightly irrational way to throw off the other players.

Or what about Uno? Uno’s a boring game but it has the pleasant feature that it requires no thought to play; it can be relaxing in the same way that it’s relaxing to watch a baseball game on a sunny afternoon. When you play Uno for no money, I guess you play with less focus than if you’re playing for money, but it’s pretty much the same game.

I guess my point is that, in any game, the lower the stakes, the more opportunity for silly play, but poker is one of the few games where trolling can be exciting. The closest analogy would be ping pong. Slamming it on every point is like going all-in in poker: it’s exciting, you’ll probably miss, but it’s very satisfying when you win.

3. Poker is a multi-player game. In ping-pong you can have a friendly game where both players are slamming every point, or a friendly game where both players are trying their hardest to win, or a friendly game where both players are just hitting it back and forth–any of these are possible. But in zero-stakes or low-stakes poker, it only takes one player to troll and it throws off the whole game.

4. Poker’s a skill game but not completely a skill game. Luu writes:

I would’ve thought that playing in the largest public cash games around would be the equivalent of joining a local open chess tournament, where anyone who started as an adult, let alone as a middle aged adult, will get demolished by IMs/FMs/NMs (I looked up one random local chess tournament, and there was an IM who placed 3rd). But you can play poker for two weeks and sit down at the biggest public games in town and do fine (there are, supposedly, some well-known private games that are a bit bigger than the largest casino games and I have no idea what the level of skill in those games is). Part of that may be down to variance, but part of that seems to be that the local level of play in poker isn’t all that high, at least in the largest public cash games around. . . .

I strongly suspect the best poker players are much better at poker than the best modern board game players. But, for some reason, you don’t see this difference expressed in local games in the same way that you would if you went down to the local chess club.

I just think the range of abilities, from beginner to intermediate to expert, is much wider in chess than in poker. I’ve played poker with some people who are clearly worse than me and some who are clearly better than me–but these differences are nothing like the difference between me and a really bad chess player, or the difference between me and a really good chess player.

5. The structure of the game. Poker’s much more interesting when you play it for money. An 8-hour poker session is commonplace, but people usually would not want to play a board game for 8 hours. And nobody would play 8 hours of poker if not for money (unless, say, you’re trying to get practice for a future money game)–it would just be too boring.

There a scene in Valis, I believe, where Dick is in a mental hospital and they’re playing games like Go Fish. There’s the opportunity to play poker, but not for money, and Dick says that poker is not a card game, it’s a money game. And he’s got a point. Money is central to poker in a way that it’s not in chess or Scrabble or even bridge. In poker, you’re not just playing for money; the game is built around betting. Money is involved at every stage of the game play.

6. In money poker, the goal is not to win; it’s to improve your bank balance. This makes a difference. For example, suppose it’s the end of the night, you’re down by a lot, and you’re in one last big hand. If your only goal was to end up a winner, you might be motivated to risk a big outlay even if it only gave you a small chance of winning that final pot. But it doesn’t work that way with money. Being down $100 is bad, but being down $300 is worse. It’s not like football where you might as well throw that Hail Mary pass because, if you don’t try, you’ll lose, and getting that pass intercepted won’t make things any worse.

All said and done, though, I think Luu is on to something when he talks about the culture of the game. I could imagine a version of poker that’s played for points, just like Scrabble, and the goal is to be the winner at the end of the game. I guess the point is that such a game would be kind of boring, closer to gin rummy than to Scrabble.

I’m a music education researcher working on a book that constructs an international index of K–12 music education systems — indicators, weights, and rankings across 20 developed countries. (Measuring Music: An International Framework for Comparing K–12 Music Education Systems)

I understand your expertise in statistics, and how they intersect with the humanities.

My question is what’s the best way to have the measurement and statistics competently checked? Should I be looking for a particular kind of applied statistician or methodologist, or is there a vetting approach you would recommend for a project like this?

My recommendation is that he should talk with a statistician or methodologist at a school of education.

But if anyone reading this is interested in helping on this project, you can contact him directly: ned.kellenberger@gmail.com

It happens that I was already going to be in the area so I agreed to stop by Wednesday morning.

As the organizer, Ben Pace, describes the program, “We’re bringing 40 writers to Lighthaven to write-and-publish an essay online every day.” So, it’s for people who want to blog. That sounds cool–I’m a big fan of blogging–beyond the evidence of the 12,000 posts and 200,000 comments here, you can see my various posts on the topic, including:

• 2026:  What’s the essence of blogging?
• 2025:  Blogging’s a great way to express your ideas.
• 2024:  The Village Voice in the 1960s/70s and blogging in the early 2000s
• 2024:  Bill James hangs up his hat. Also some general thoughts about book writing vs. blogging. Also I push back against James’s claim about sabermetrics and statistics.
• 2023:  Blogs > Twitter, part the umpteenth
• 2022:  Blogs > Twitter again
• 2020:  Why I Rant
• 2018:  What is “blogging”? Is it different from “writing”?
• 2015:  Another benefit of bloglag
• 2014:  Blogs > Twitter
• 2014:  “What should you talk about?”
• 2014:  What is the appropriate time scale for blogging—the day or the week?
• 2013:  On blogging
• 2013:  Blogging
• 2012:  Blogging, polemical and otherwise
• 2012:  Blogging is “destroying the business model for quality”?
• 2012:  Blogging
• 2010:  A half-decade of blogging
• 2009:  Other voices, other blogs
• 2009:  The U.S. government’s guide to blogging
• 2009:  Little blogs and big blogs
• 2009:  How many blogs do we rip on the daily
• 2008:  Blogs as places?
• 2008:  Little blogs and big blogs
• 2004:  A weblog for research in statistical modeling and applications, especially in social sciences

Traditionally there have been four sorts of blog posts:

1. Annotated lists of links
2. Online diaries
3. Open-format op-eds
4. Tech explainers

My own blogging is a mix of the four:

1. I rarely just supply links, but often my posts are motivated by things that people send in.
2. Each day I write what comes to mind, so it’s a form of online diary, even if I usually don’t write about things that happened that day.
3. I offer lots of opinions on a wide range of topics (voting power, statistical graphics, Bayesian inference, literature), often with the goal of persuading or influencing or just getting people to think about a topic.
4. I write statistics textbooks!  So no surprise that the blog is full of tech explainers, case studies with code, stories of life in the trenches, etc.

Nowadays, I get the impression that blogs (sometimes brand-named as “Substacks”) are focused on op-ed-style posts, without so much annotated lists of links or online diaries.  The lists of links are less valuable now that there’s google, twitter, etc., and the online diaries . . . I don’t know why that’s less of a thing.

To get me prepared, Pace pointed me to the Inkhaven webpage webpage and suggested I read a few posts ahead of time.  So I did so, keeping in mind the traditional forms of blogging as described above.

• I started to read the very first post, Kill Yourself Cave, by Remy, but then halfway through some sort of ad popped up and I couldn’t read the rest–I guess I’d need to buy some sort of subscription?
• The second post is called Against Against Romance, by Viv.  This is traditional blogging in the sense that it’s a response to earlier blogs by others.  The only thing that put me off was that it was framed as advice for the reader (as an “op-ed”) but I think it would be more valuable as an online diary or personal story.
• The third post on the page, Contra Aiella on Status, is by Alexander Wales.  As with the two posts above, I’m struck by a sort of online writing style, with phrases such as “status maneuvering,” “downstream of,” “dominance hierarchy,” etc. Nothing wrong with that–everyone has a style and a vocabulary–; it’s just something I noticed.  I don’t really care so much about status, but that could be a life stage thing, that I’ve aged out of such things, in the same way as it’s hard for me to relate to posts on “securing a husband” or whatever.
• The next post is a poem by Alok called Why homeless people hide drugs from kids.  Oddly enough, this one mentions “status” too.  I guess the concept is in the air.
• Next is Late Pregnancy is Pretty Bizarre, by Viv.  It’s kind of refreshing to see a straight-up online diary!  She’s not telling me anything I hadn’t already heard before, but that’s fine:  an important product of writing is that it helps you work out your own thoughts.
• Next, Kamikaze Dreaming, by Drew Schorno.  This one’s a tech explainer–or, maybe I should say, a business explainer–the sort of post where someone tells you what it’s really like in some corner of the economy.  Kind of like the blogs of Dan Luu and Nikhil Suresh.  Here’s a characteristic passage:
  

  When I said that I worked on pine point and they gave me a standing ovation.
  a fucking STANDING OVATION.
  Little did I know that this was the beginning of the end of my lucky streak.

  There’s something compelling about this sort of story.

• Next, Adventures in Dieting, also by Drew Schorno.  I don’t know why they have multiple posts by the same person on this one page; I guess that tells me that they’re selected by an algorithm, not a person.  Of course there’s nothing stopping the person from updating the algorithm to avoid repeats.  The crazy diet advice reminds me of my late friend Seth Roberts (see here for the sad part of the story).  In a world of online hucksters promoting crazy health ideas, I’m relieved to see Schorno expressing some skepticism!
• Scrolling down a bit, we come to Is Salt Salty, by Itsi Weinstock.  A pure tech explainer post, it seems!  I don’t see this one as being of general interest, but that’s fine; as noted above, writing is a way of working out ideas.

One thing that made me happy about the Inkhaven bloggers is that none of them are writing in that aggressive style that I associate with some business books and internet gurus.

The above comments are not intended to be dismissive, or “patronizing,” or “punching down,” or whatever.  People write for all sorts of reasons, and that’s fine.  As I wrote above, writing is a way to work out what’s latent in your head, to work out the implications of some partly-formed ideas.  This is true for nonfiction and also for fiction, which I see as a sort of posterior predictive checking, a working out of implications.

To put it bluntly:  when you’re blogging, you’re not writing for me, you’re writing for you, which is how it should be.

When my colleagues and started blogging, back in 2004, we were already sending lots of long emails to each other, and it made sense to make these public, for three reasons:

• Many others were interested in these topics too, so we were doing a service by sharing our ideas with anyone out there who wanted to look,
• Commenters provide useful feedback, including correcting our mistakes, revealing where we aren’t expressing ourselves clearly, and pointing us to ideas and literature we hadn’t been aware of.
• Knowing you’re going to post something in public pushes you to be more aware of what you’re saying.  When I blog, I’m usually expressing myself more clearly than in an email, and that’s useful to me as well as to my correspondents.

That last item cuts both ways:  writing a blog post takes more work than writing an email, and for some people it’s not worth the effort.  For me, though, I like it; I’ve gotten used to expressing myself this way.

What’s my advice to the Inkhaven bloggers? Just remember that your first audience is yourself. Write clearly enough that you can come to terms with whatever implicit incoherences are in your thoughts. Don’t worry about being persuasive; just try your best to see through your own fog. And learn from each other.

Also, blogging. Writing a blog is like writing a book: you can aim directly for your audience (first that’s you; after that it’s everyone else who will read it). In contrast, if you write for a newspaper or a magazine or a scientific journal, you’re writing for the editor. I hate writing for the editor. Editors can be great, but it’s such an indirect process. I remember back when I was writing for the Monkey Cage, the political science blog, and it got absorbed by the Washington Post. Which was great–we reached a broader audience–but then we had an editor, who was a nice person but kept telling me that my posts were too bloggy. Which they were–I’m sure the editor was correctly judging what would work for a Washington Post product–but I absolutely hated not being able to write directly. The prospect of more readers wasn’t worth the unpleasantness of having to write in a style that wasn’t mine, and the degradation of the quality of the writing and of the ideas being expressed under such a style.

So, you be you. And, if you find that your blogging style is very similar to that of some other bloggers out there–even me!–I recommend mixing it up a bit. Try to write in the style that’s most transparent for you.

But then today I was at a conference where Jay Bhattacharya, the acting director of the CDC, was asked about this unpublished report, and he said it was because it was using a really bad statistical method. It was only a brief exchange and there was no time for him to give a reference or to explain why he thinks the method is bad. I still haven’t myself seen a copy of the report so it’s hard for me to judge.

I think the best next step would be for the CDC to release the report in question, along with a critical response from a statistician explaining how the method is flawed. Bhattacharya said it was common knowledge that the method was terrible, so at the very least it would be a valuable educational opportunity to see this article that was on the verge of publication, and to understand its fatal problems. As a citizen, as well as in my role as statistician, I find it frustrating to hear about this dispute and not be able to see the controversial document and an explanation for why it’s not to be trusted.

It may be that the CDC is in the process of doing this. There could be a statistician right now writing that document explaining the problems with the almost-published paper.

Or maybe they sent it back to the original researchers to redo using a better analysis.

It’s kinda scary if the CDC was routinely using a terrible statistical method. Or maybe there’s more to the story. I just don’t know, which is why I’d like to see the study and also to see the criticism.

This workshop will bring together researchers, journal editors, publishers, funders, and scientific association leaders to identify practical, forward-looking strategies for strengthening data integrity and transparency in the social and behavioral sciences. Participants will explore innovative tools and frameworks to detect and prevent errors, promote accountability, and reinforce public trust in research. Discussions will also consider how journals, institutions, and professional societies can adopt fair, sustainable practices that support scientific rigor while ensuring accessibility for researchers across many contexts and settings.

I brought up some relevant points, including:

1. The science-reform movement as an awkward alliance between reformers (who anticipated that failed replications will cause people to move away from some bad published ideas) and status-quo people (who anticipated that successful replications would validate various now-controversial studies in the past).

2. It’s not always clear at first that a paper is bad, but then in retrospect its problems jump out at you. The analogy I gave is that Arthur Conan Doyle was fooled by photos of garden fairies that, years later, were obvious fakes. This is one rationale for post-publication review.

3. We should consider variation when hypothesizing effect sizes, and this connects to the point that researchers and the public should be more accepting of uncertainty. This is really the most important point. Later in the conference the health economist Jay Bhattacharya discussed the problem that people don’t know whether published research is true. It’s a good point, and it leads to the next step, to move beyond the expectation that research results should produce certainty. Even the cleanest and best study is only telling you about some set of people under some conditions at some particular time. Future effects on new people in new settings will differ.

4. The Bayesian cringe.

5. The role of the field of statistics in improving science (multilevel modeling!), and at times making science worse (null hypothesis significance testing!). As I said to Uri, I do think we’ve developed a better statistical understanding in the past fifteen years, and this has allowed us to understand and address replication concern in ways that were not done in previous decades.

But there were some other things I meant to talk about but I never got around to saying:

1. My frustration that many of the people who promote bad research don’t seem to even care about the work that they’re promoting. For example, consider the physicist who pushed the ridiculous claim that scientific citations are worth $100,000 each, or the biologist who pushed the ridiculous claim that chess players burn 6000 calories per day. If they really cared about these things, they could try to study them! For example trying to trace where this $100,000 is going to, or studying the variation in the value of paper. Or trying to understand physiologically where those 6000 calories were going. But nooooo . . . they just want these B.S. factoids. Or the people who studied ovulation and voting, but got the dates of ovulation wrong. So often it seems that the critics care more about the topic than the people who are out there pushing these claims.

2. The role of the National Academy of Sciences. The Academy is sponsoring this workshop, and the workshop has gone well, but I also wanted to point out that the National Academy of Sciences is part of the problem too! Their journal has published some notably bad articles (air rage, himmicanes, ages ending in 9, etc etc). I have no reason to believe that PNAS is worse than other journals, but it does get some attention.

3. The problem of junk science as a betrayal of trust. Later at the conference, the political scientist Skip Lupia made the point that research is expensive and it’s the responsibility of the academic community to justify this to the taxpayer, especially in a modern information-rich environment where many people might feel that they don’t need academic institutions at all because they can find everything online. And I agree with this. But, even beyond waste of resources, it seems to me that when credentialed scientists promote junk, this degrades the reputational coin of the realm. As it should be. Every time a researcher at Harvard or Stanford or the University of California or wherever is promoting ridiculous work, every time an academic podcaster plays the promotion game, etc., this does its part to discredit the scientific enterprise. And this makes me mad.

I guess I can see why that last point never came up in the discussion, because I expect that everyone in this meeting is, like me, incensed by that sort of scientific careerism. So I didn’t need to say that.

I do like the idea of replication being a norm.

For example, imagine a world in which, when this psychology professor tells his Stanford class that chess players burn 6000 calories per day, that some student would raise their hand and ask, “Where did that number come from?” And then, if the professor were to supply some reference or rationale, another student could ask, “Is there any outside confirmation about that claim?”

I don’t know how the professor would answer such a question. Maybe he’d give another reference, maybe he’d say he doesn’t know, maybe he’d just ignore the question and move on . . . there are many possible responses. The real point is to set up the expectation that there be a response. The goal is to move beyond the pattern of strong claims supported by vague references. When you look carefully, you’ll often find that the evidence claimed in support isn’t always there.

Beyond the direct value of the replications themselves, there are benefits from thinking about replication, in part because it moves you to think about evidence and to think about how the conditions of an experiment can vary. If, instead of thinking of that $100,000 per citation or those 6000 calories as cool numbers, you think seriously about their variation and you think seriously about replication, the claims themselves will crumble. And then, pushing it back one step, maybe you’d think twice about promoting those sorts of stupid claims in the first place.

So, I guess the thing I’d like to have added to the discussion is a clearer discussion of the links between the procedures of science and science reform (publication, replication, etc.) and the particular claims being made.

– This time there are no “grownups” in cabinet positions to talk the president out of bad ideas. Instead, the government is run by some combination of ideologues and airheads–sometimes both at once!–who actively come up with bad ideas themselves.

– Trump is getting older and more incoherent and delusional, which diminishes his common sense, increases his susceptibility to whatever stupid or criminal idea is proposed to him, and reduces his ability to resist bad ideas or to weigh options.

– Lack of serious consequences for the 6 Jan 2021 insurrection emboldens extremists within the government to break more laws.

– The news media environment is more fragmented, so it’s easier for the government to dodge bad coverage.

– At this point Trump is so unpopular that his party is likely to lose lots of seats in Congress no matter what he does, so they’ve moved to full bust-out mode, just pushing all the buttons they can before they lose access to the public treasury.

I’m framing this in a negative way, but if you’re a Trump fan, the same arguments apply: less of a “deep state” to stop the government from shooting up boats that might harbor terrorists, arresting people without the usual red tape, shooting protesters, starting wars, pardoning patriots who just happened to have committed crimes, etc., the hell with the legacy media, doing whatever it takes to swing the pendulum back to the center after the far left excursions of the Obama-Biden years.

In any case, here’s an important explanation that I don’t think is being mentioned often enough, and that is that the right wing of the Republican party controls all three branches of government.

By “all three branches of government,” I don’t mean the presidency, the House of Representatives, and the Senate. I mean the legislative, executive, and judicial. The last time one wing of one party controlled all three branches of government was the left wing of the Democratic party in the mid-1960s, and they did a lot, indeed they did some things that took decades to roll back.

The Republicans did control all three branches of government in 2017-2018 after Trump’s first election, but Congress and the courts were not dominated by the right wing as they are now. Moderates had some influence. I’m not saying that Congress and the courts always say yes to the president right now–nor, for that matter, did liberal Democrats in the 1960s always get what they wanted–but, despite the occasional conflict, there’s an ideological and partisan uniformity.

In short, I think that if the executive branch had tried to do what it’s doing now, back in 2017 and 2018, it wouldn’t have gone through. The Senate wouldn’t have confirmed Pete Hegseth. The courts would’ve been less forgiving. The president’s office would’ve moved more slowly because of the recognition that they’d have to get congressional approval for wars, tariffs, domestic surveillance, etc.

None of what I’m saying here is new. Everybody knows about the three branches of government. But I think it’s easy to focus on the colorful personalities and crazy doings in the executive branch and forget about the constraints–or, in this case, the relative lack of constraints–that they face.

P.S. Lots of discussion in comments about the political positions of the Democratic and Republican parties–which, don’t forget, can differ a lot from the positions of Democratic and Republican voters, not to mention independents. That’s all fine; these are important topics to discuss.

I just want to emphasize that the point of the above post is not about whether particular policies are good or bad, or even where they stand on the left-right spectrum. Rather, my point is that the unleashed nature of the second Trump administration can, to a large extent, be explained by the lack of constraint resulting from a unified control of all three branches of government. It’s easy to get lost in the details and then to forget this simple point.

On p.124, Davies shares an amusing story of the unraveling of a scam involving counterfeit Portuguese banknotes: “While confirming them to be genuine, the inspector happened to find two notes with the same serial numbers—a genuine one had been stacked next to its twin. Once he knew what to look for, it was not too difficult to find more pairs. . . .”

The birthday problem in the wild!

P.S. I sent this story to John Cook, saying that this was the first time I think I’ve ever seen this particular problem come up in real life. John replied that birthday problems come up all the time in cryptography, e.g. hash collisions, and he pointed to this post from 2017:

Ideally, a secure hash is “indistinguishable from a random mapping.” So if a hash function has a range of size N, how many items can we send through the hash function before we can expect two items to have same hash value? By the pigeon hole principle, we know that if we hash N + 1 items, two of them are certain to have the same hash value. But it’s likely that a much smaller number of inputs will lead to a collision, two items with the same hash value.

The famous birthday problem illustrates this. . . . Variations on the birthday problem come up frequently. For example, in seeding random number generators. And importantly for this post, the birthday problem is the basis for birthday attacks against secure hash functions. . . .

I had no idea! John also points to Pollard’s rho algorithm as another real-world application of the birthday problem.

The acting head of the Centers for Disease Control and Prevention has canceled the publication of a study that found that the Covid vaccine sharply cut the odds of hospitalizations and emergency visits last winter, a Health Department spokesman said. . . .

The study, conducted by C.D.C. scientists, calculated the effectiveness of Covid shots by looking at the vaccination status of people who had sought care at hospitals and emergency rooms. It found that vaccination cut the likelihood of emergency visits due to Covid by 50 percent and of hospitalizations by 55 percent, according to a summary of the study viewed by The New York Times.

It was scheduled to be published on March 19 in The Morbidity and Mortality Weekly Report, the C.D.C.’s flagship journal. News of its cancellation was reported earlier by The Washington Post.

Some former C.D.C. officials said it was unusual for the head of the agency to cancel a scientific publication that had already been cleared by the agency’s staff scientists and had been scheduled for publication.

So what happened?

Andrew Nixon, a spokesman for the Department of Health and Human Services . . . said that assessment “identified concerns regarding the methodological approach to estimating vaccine effectiveness, and the manuscript was not accepted for publication.”

But:

“I’ve never seen a case where an article in the M.M.W.R. that got to that stage was not published,” said Dr. Michael Iademarco, who led the center that included the publication’s operations from 2014 to 2022.

And:

The approach employed in this research has been used for years by scientists at the C.D.C. and elsewhere to gauge the real-world performance of flu and Covid vaccines, said Dr. Fiona Havers, a vaccine expert who resigned from the agency in June.

No link to the report itself. Maybe the authors should anonymously email it to jeevacation@gmail.com and then it can appear in the next file dump.

It must be horrible to be working for CDC right now. They were literally shot at by an anti-vax terrorist, and now the in-house anti-vaxxers are suppressing their reports. Meanwhile the government is releasing health-related reports with fake citations and is releasing dietary guidelines which are so bad that even a supporter of these guidelines can do no better than describing them as “not crazy.”

So, that’s the way it’s going. The report with fake citations is released. The “not crazy” (actually, crazy) advice is promoted. The CDC report is suppressed. I guess it doesn’t meet the government’s high standards. Maybe if they’d thrown in some fake citations and some nutty health advice, it would’ve been approved for publication. That’s how you get “gold standard science,” right?

P.S. More here. I hope that future updates are coming.

And, yes, typing some prompts into a chatbot and producing a paper is fraud, in the same way that publishing textbook excerpts as if it were new research is fraud, or copying from wikipedia as if it were new research is fraud, etc etc. It doesn’t require fake data and it doesn’t require some cackling Snidely Whiplash attitude. It can be some schlub sitting at a computer terminal who wants to get his contract extended or get admitted to a Ph.D. program or whose adviser is pressuring him to get some publications . . . But it’s fraudulent publication, not the same as bad research (which is actually research, it just happens to be useless because bad measurement and kangaroo).

There are clearly some differences between passing off work containing fake evidence you purchased from paper mills and work that you contracted AI to do for you–in one case you are paying money for someone to pass something fake off as real with your name on it, whereas in the other you might be using actual data and think you’re just saving time, especially if you’re reviewing what the AI does. So should we really consider both forms of fraud? 

It struck me how sharply Andrew’s perspective contrasts with the current direction of discussions among ML and other researchers interested in AI for science. There, it’s seen as inevitable and not necessarily morally problematic that the future of science will have humans largely playing the role of curators, who prompt and select among results produced by LLM agents, who do the bulk of the work. It’s worth considering what kinds of ethical lines this crosses exactly. 

Let’s imagine that I give an LLM an initial high level research question related to a topic on which I am knowledgeable. It churns on the idea and ultimately designs an experiment it’s happy with, I review the plan before prompting it to continue, maybe tweaking slightly, like changing a condition or suggesting an additional robustness check. It then gathers data on my behalf (e.g., running an online experiment or downloading existing datasets), conducts the analysis, and presents me with the results. I review these and then give it permission to write up a paper. I read the final paper to make sure I know what it’s saying before I submit. Maybe I change a few things I don’t agree with. I add my name and also credit the AI. In other words, there is a light human touch throughout, but much of what is presented as my work comes from the model.

From an “optimistic” AI-for-science perspective, the strongest argument is probably to cast it as part of the scientist’s job to try to make the most of current technology. If we think AI might help us be more productive, then we should explore how much time it can save us, just like it was a good move for statistics to embrace the computational revolution that made previously intractable models commonplace. Proponents of AI for science argue that it is irresponsible not to use AI given its current capabilities, just like it could be construed as irresponsible for a brilliant researcher to refuse to use calculators if doing so meant they could contribute more useful advances to the field. Of course, this assumes that we won’t be sacrificing anything vital in the process. 

The “pessimistic” view of AI generated science as fraud thinks we are sacrificing something vital in the process. But what is it exactly? If you believe that “the devil is in the details” (or “God is in every leaf of every tree”, depending whose side you want to be on) then whenever you outsource decisions you would otherwise make yourself, you have potentially compromised the work from the perspective of your own expert judgment. So putting your name on it betrays what you know to be true of good science. Of course, you could check everything down to the lowest level, and intervene whenever the agent tries to do something you don’t agree with. Then the AI is really just a means of computation–even if you use it for brainstorming what research questions to ask, you could view it as a way of extending your limited resources but without sacrificing your own scientific judgment. This requires that you are knowledgeable enough to assess everything it does. Assuming you are, then it seems hard to argue that this is fraud. Though admittedly, a lot rests on how careful you are when you check things over.

Part of the concern may be that AI makes it tempting to extend your methods or claims outside of what you know well. Without the option of using it, you would have had to do the research yourself (and presumably gain understanding in the process) in order to apply that method. Relatedly, I suspect most people would agree that when you are in a training context, like taking courses in grad school, turning in work that was largely driven by the AI is a form of fraud because it holds you back from gaining the understanding yourself. 

Another version of the fraud argument focuses on misattribution of ideas. Maybe this is what Andrew had in mind when he wrote “publishing textbook excerpts as if it were new research is fraud, or copying from wikipedia as if it were new research is fraud.” If the AI produced significant parts of the contribution, like the specific hypothesis, the choice of methods, and the framing of the contributions, then those aren’t your novel creations and adding yourself as author is misattribution. But this is tricky because human scientists also recombine existing ideas, methods, and frames constantly, and we often call the resulting combinations original contributions. So the question isn’t whether ideas are derivative per se but when the degree of derivation crosses into infringement. We have copyright law for some creative domains, and we can try to formulate when AI outputs are permissible in light of this (e.g., Annie Liang has some recent work on this). But to say AI-assisted papers are fraudulent on such grounds, we’d need to work out what the scientific analog of substantial similarity is. This is hard because science explicitly values building on prior work. But we can agree on some things, like you shouldn’t do something that is too close to others’ work without citing them.

A final angle on why its fraud might be that it misportrays what science is more broadly. If you think the reasons to do science are fundamentally human–that as scientists we are concerned with producing understanding for ourselves just as much as we’re concerned about improving things in the world–then you could argue that for science to be meaningful we have to be the ones coming up with the ideas and shaping them as we go. From this perspective, automated science isn’t inherently wrong, it’s just missing the point. AI for science arguments often completely overlook the “people production” role of science. In the extreme, they envision AI finding solutions for lots of real world problems and intervening to control outcomes in the world without us understanding how any of it works. In reality, the personal side–including the search for personal fulfillment through science–is a big part of why smart people who could potentially make a lot more money in applied roles end up choosing research careers. And it’s a big part of how we evaluate scientists. How many of your Ph.D. students have gone on to competitive research positions? What does the trajectory of topics you’ve worked on say about your research taste? 

Pushback to this argument might point out that by saying science is entirely a matter of human careers, we contradict claims that we as scientists like to make, about how we are dedicated to improving the state of the art in our field, or producing value for the world. Would it still be science as we know it if we started acknowledging that it’s really about personal fulfillment for scientists? But I think this is a bit of a false dichotomy. The public value of science depends on there being humans who find the work meaningful enough to do it well, including pushing back on sloppy results, exercising their taste to shape the direction in their field, training students worth training, etc. Careless AI use can threaten this by flooding the system with outputs that crowd out careful work, disincentivizing the people who would be intrinsically motivated to do quality work less likely to stick around. It also implicitly reframes science as nothing more than a pipeline for results.

My view is that AI use can go either way, depending on how you approach it. What best determines whether it’s fraud or not is the attitude you bring. It can help you do less fraudulent research if you’re the kind of person who is already very picky about what you send out to the world. But it can help you fool yourself and others if you let competitiveness and obsession with metrics drive how you use it.

As a final comment, there’s some irony in using terms like “optimism” to talk about this. I described the pro-automated science argument above as “optimistic,” because I think that’s how many in this camp see themselves–as fundamentally optimistic about the future of science and our ability to improve it by using AI. But the underlying motivation to figure out how to produce papers with as little human oversight as possible is also often deeply pessimistic. A common narrative is the “review death spiral”: AI production stresses the review system, which increases the noisiness of paper acceptance decisions, which further incentivizes submitting sloppy AI produced papers. The answer is presumed to be putting more AI in place on both sides. The idea that scientists have agency and could continue to shape the meaningfulness of what gets produced starts to seem out of the question.

Increasingly, a lot of the most enthusiastic pro-AI discourse (including for science) strikes me as nihilism masquerading as optimism. We have people who perceive themselves as huge optimists that will reshape science or society for the better simultaneously lacking the imagination to see beyond their own technological determinism. It reminds me a bit of the “optimism” associated with some open science and science reform positions, who also suggest that we just need the right technology to fix the problems (though in this case, its heuristics like replication or preregistration). It’s a fundamentally non-agentic view of human scientific endeavor. 

For some time, I’ve noticed postmodernist views cropping up in the magazine Science, mostly in the book reviews. In the 23 Feb 2024 issue, however, they arrived full blown in a set of opinion pieces about genetics education (e.g., Duncan et al., Vol. 383, Issue 6685, pp. 826-828, excerpt below):

“The methods of conducting genetics research and its outcomes are steeped in, and influenced by, power and privilege dynamics in broader society. The kinds of questions asked, biological differences sought, and how populations are defined and examined are all informed by the respective dominant culture (often Eurocentric, white, economically privileged, masculine, and heteronormative) and its predominant ways of knowing and being (3). Findings from human genetics and genomics research subsequently play into existing sociopolitical dynamics by providing support for claims about putative differences between groups and the prevalence of particular traits in particular groups (3). Historically, such research has been used in support of eugenic movements to legitimize forced sterilization and genocides. Yet it would be a mistake to assume that such research is merely a discredited past relic, a stain on the otherwise objective and rational track record of genetic research. Rather, it was mainstream work conducted by prominent researchers and supported by major professional societies. The reality is that some modern human genetics is still informed by the same racist logic (4).” (Duncan et al., The sociopolitical in human genetics education; (3.) J. A. Hamilton, B. Subramaniam, A. Willey, Fem. Stud. 43, 612 (2017). (4.) A. C. F. Lewis et al., Science 376, 250 (2022).

Well, yes, science is done by people, and is subject to all the silly and sometimes pernicious stuff that goes with that, but this seems overblown, to say the least, and not that well founded. For example, ref. 4 argues against using any geographically based categories for genetic variation among people, by showing that the clustering evident in geographically based genetic samples is not apparent in BioMe data from New York, where almost everyone is from somewhere else – therefore, the geographical clusters are a “by-product of sampling strategy” [see their Fig. 1].

As an old progressive, I more or less agree with a lot of the points Duncan et al. make, but I still find their essay alarming. Duncan et al. do acknowledge that science can produce “knowledge that is credible and valuable,” but postmodernists by and large assume that since scientific knowledge is socially created, like other kinds of knowledge, it should not be “privileged” over other ways of knowing. I see this where I live in NW California, where some involved with environmental restoration want to put “indigenous science” on a par with regular science. However, I have the sneaking suspicion that postmodernists also want to claim that their “way of knowing” is best – this fairly oozes out of the language quoted above.

Perhaps the main thing that bothers me about Duncan et al. is that they say nothing about the role of genetics in undermining the ideas they complain about, such as the notion that races are “real.” There is nothing, for example, about Lewontin’s celebrated 1972 paper showing that, averaged over loci, there is much more genetic diversity within rather than among groups. This gets to the question: given that genetics has been misused in the past, what should people teaching genetics now do about it? My sense is that the most important thing is to teach genetics well; that showing students why the concept of races as biological categories doesn’t make sense is more persuasive than telling them that it doesn’t.

Yeah, I know that the above is about genetics, not statistics, but I expect that statistics have been used to support the “respective dominant culture,” too.

I’m not sure how to think about all this. Postmodernism doesn’t come up in the sort of political science that I do. Or maybe I should say that it’s all postmodern, in the sense that even basic questions such as the value of democracy are questioned; there are no foundations. And postmodernism doesn’t come up in statistics at all. Political issues do arise in statistics, most notoriously with biological measurements, but these involve specific application areas, not the core of statistics. So when I hear about stories such as above, I’m coming from the outside.

P.S. Several commenters say that Duncan et al. make some good points, even if their writing style is annoying and they also have some major lapses. I think that’s kinda Williams’s point! If he thought the arguments had no merits at all, maybe he wouldn’t have thought it even noteworthy that it has problems.

Each dot is a PSU (primary sampling unit), which can be an individual but is often a group/cluster of individuals. Each color is a stratum. dCV is the usual K-fold CV but:

1. keep PSUs together within a fold
2. reject a split if a whole stratum falls into one fold
3. modify the weights so that each subsample replicates the original sample

Let’s return to the problem of using CV to assess Multilevel Regression and Poststratification (MRP) models. We saw that individual-level Loss(y_i, yhat_i) might not be great for assessing MRP models, even when weighted, and that CV noise can swamp model differences.

The dCV method from Iparragirre et al. (2023) is for a probability sample. MRP is usually used for a nonprobability sample (e.g. an online survey). But maybe there’s still something to learn here.

Bayesian Data Analysis Chapter 7 about evaluating predictive accuracy p.169 says “we can imagine replicating new data in existing groups …or new data in new groups”. New data in existing groups is strata-like, while new data in new groups is cluster-like.

Iparragirre et al. (2023) say that splitting clusters between training and test (i.e. not doing #1 above) will underestimate error because we fit models with more information than we should. This “overfits” to the data. So the usual CV chooses unnecessarily complex models. (See ESL Chapter 7 about model assessment.)

What about the reverse, for strata instead of clusters ? Suppose we don’t do #2 above and we have whole strata within a fold. Then we fit models with less information than we should. Will this “underfit”, choosing overly simple models ?

In Multilevel Regression and Poststratification (MRP) does it help to do the CV rejecting a split if a whole stratum falls into one fold (#2 above) ? For example, if all members of an age or education group fall within one fold, we could redo the CV split ? Any references where folks do this ?

In a post entitled “FARCE: FARS Album Release Coincidence Examination,” Gaurav Sood writes:

Replication and extended analysis of Patel, Worsham, Liu & Jena (2026), “Smartphones, Online Music Streaming, and Traffic Fatalities,” NBER Working Paper 34866.

Key Findings

1. The Statistical Effect Is Real

Traffic fatalities are elevated on major album release days:

Estimator	Effect (Tier 1)	SE	t-stat
Local (±10 day)*	+23.0 deaths	5.1	4.5
Donut-global	+16.2 deaths	5.1	3.2
Forecast	+22.8 deaths	4.9	4.6

. . .

2. But The Causal Story Doesn’t Hold Up

No dose-response relationship:

Album	Streams	Effect
Tortured Poets (2024)	313M	-2 deaths
Her Loss (2022)	97M	+63 deaths
Midnights (2022)	185M	+5 deaths

. . .

Out-of-sample replication fails (2023-2024):

The paper analyzed 2017-2022 releases. We tested 7 major 2023-2024 albums as a true out-of-sample test:

Album	Streams	Effect
Tortured Poets	313M	-2.1
UTOPIA	128M	+10.5
For All The Dogs	109M	-12.8
Cowboy Carter	76M	-0.4
Hit Me Hard and Soft	73M	+7.0
SOS	68M	+9.4
One Thing at a Time	52M	-1.5

Average effect: +1.4 deaths (vs. +22.8 for original sample). The biggest streaming day in Spotify history (Tortured Poets, 313M) shows a negative effect. The pattern found in 2017-2022 does not replicate forward.

Single outlier dominates: Her Loss accounts for 34% of the total Tier 1 effect.

3. Methodology Concerns

The ±10 day estimator uses post-treatment days as controls. The paper compares release-day fatalities to the average of the surrounding ±10 days—but this includes days after the release. Standard event studies use only pre-treatment periods. If the effect persists beyond day 0, the control mean is biased upward.

What The Paper Claims

Patel et al. (2026) find:

• 139.1 deaths on release days vs 120.9 on control days (+18.2 deaths, +15%)
• 123.3M streams on release days vs 86.1M control (+43%)
• Proposed mechanism: smartphone distraction from streaming while driving

What We Did

Analysis	Description
Extended data	FARS 2007-2024 (vs. 2017-2022)
Forecast estimator	Train model on non-release days, predict counterfactual
Dose-response	Test if more streams → more deaths
Extended sample	Added 2023-2024 albums (27 total vs. original 10)
Placebo tests	Pre-trends, year permutation, window sensitivity

Results Summary

Finding	Result	Interpretation
In-sample effect	+22.8 deaths/release	Statistically significant (2017-2022)
Out-of-sample	+1.4 deaths/release	Effect vanishes in 2023-2024
Dose-response	r = -0.18	Wrong sign for causal story
Her Loss outlier	34% of total effect	Results driven by one album
Tier 2 ratio	0.80 (expected 0.50)	Effect doesn’t scale with streams

And Matt Thachet writes in with further thoughts:

I was wondering if you saw this paper. I first saw it written up in the New York Times, but it generated a fair number of articles in other outlets, too. The main claim is that the 10 biggest album releases (by Spotify streams) were associated with a 15% increase in fatal car crashes in the US.

I see the logic: higher streaming activity indicates more distracted driving, which causes more car crashes, but something feels flimsy to me. For one thing, it’s not clear to me that streaming music would actually be that distracting. If I wanted to listen to a new album I would put it on and then drive. There’s not much more to it, but maybe I’m underestimating the amount of other smartphone use that comes from this, like posting my reaction on social media.

The other part that sounds challenging is controlling for the day,.Most albums are released on Fridays which will have higher car crashes than other weekdays, but they control for this by comparing the 10 day periods before and after release date, which will include the same day of the week before and after the release date.

They include this list of albums and 5 of them were released within 10 days of another album in the list, which presumably makes the 10 day before and after control trickier. The other thing I wondered about, but they don’t seem to mention is whether the albums in the bottom half of the list have half the fatalities associated with the ones at the top, having half the streams. The average number of traffic fatalities per day is about 100, so maybe this would be too hard to tell.

Anyway I’m curious if you have time to hear your reaction to it. Like I said, the causal mechanism makes sense to me, but 15% is a huge increase and it just seems like controlling for day, season, holidays, etc. would make this almost impossible to be sure about.

I don’t have the energy to look into this myself.  Gaurav and Matt seem to have the right general approach, which is to look at the effect in the context of specific cases and to study variation.  In contrast, the common approach to quantitative research in published social science is to find some statistically significant relationship and hold onto it for dear life.

Or maybe I’m just saying this because I don’t want to believe that musicians are killing people.  I have a soft spot for pop stars, as compared to the culture heroes of today.

That’s a title that doesn’t mess around!

Richardson writes:

1. Editors abuse their positions of authority to collude with authors to publish problematic articles en masse. . . . certain authors seem to have a preference for having their articles handled by these flagged editors. For PLOS One, we identify a network in which these flagged editors were all handling each other’s submissions to the journal.

2. Networks of image duplication can be thousands of articles wide and these articles tend to appear in the same journals at around the same time. . . . articles connected by shared images also tend to get published at around the same time and in the same venue. This suggests that paper mills are capable of both producing articles and getting them published in a highly coordinated fashion–wholesale, not custom-made. Publishers often claim that paper mill products are published because they have slipped through the cracks. This vignette instead suggests a model where paper mills have relatively open pathways into journals–likely facilitated through the knowing cooperation of editors, as suggested in the first vignette.

3. Broker organizations are capable of placing articles in journals on demand and adapt well under adversarial conditions. . . .

5. Publishers understand that systematic fraud underlies the bulk of their integrity issues. We show that most retractions are now issued in batches, alongside ten or more retractions in the same journal on the same day. . . .

6. The integrity measures used to contain systematic scientific fraud are dwarfed in scale by the problem itself. We assemble a corpus of suspected paper mill products . . . this corpus has been growing in size (annual count doubling every 1.5 years) at a rate far eclipsing the growth rates of all scientific articles (doubling every 15 years) . . .

#5 here is particularly interesting to me. My head’s still stuck in the 2010-2015 Psychological Science era, a time when the absolute top journal in the field was routinely publishing junk science. At the peak of the problem, I think that more than half–no joke–of the papers in Psychological Science were crap, just pure combinations of noise mining, hype, and the sort of theory that was so flexible that it could explain any pattern or its opposite.

None of this was from paper mills, and I believe that very little of it was Wansink/Ariely-style fabrication or fraud. It was just bad science, done by highly-connected, highly-credentialed researchers who were working in a sort of unintentional parody of the scientific method. The sort of bad work that led to the terms HARKing, p-hacking, and questionable research practices. We discussed some such papers here, here, and here.

Based on this experience, I’ve been going around for years saying that the big problem in science is not fraud but rather well-intentioned bad work. Honesty and transparency are not enough.

But that was then, this is now. In this new era of paper mills and chatbots, where the marginal effort of writing and publishing a paper is essentially zero, there is more and more motivation for fraud. And, yes, typing some prompts into a chatbot and producing a paper is fraud, in the same way that publishing textbook excerpts as if it were new research is fraud, or copying from wikipedia as if it were new research is fraud, etc etc. It doesn’t require fake data and it doesn’t require some cackling Snidely Whiplash attitude. It can be some schlub sitting at a computer terminal who wants to get his contract extended or get admitted to a Ph.D. program or whose adviser is pressuring him to get some publications . . . But it’s fraudulent publication, not the same as bad research (which is actually research, it just happens to be useless because bad measurement and kangaroo).

I guess it still depends on the journal. But even if the top journals stay mostly immune from fake papers (so, for them, we can be more concerned about the bad research they publish), all this paper mill stuff has an impact, because it affects citation counts.

One option in evaluating research would be to follow the lead of economics, where I’ve been told that pretty much all that matters is the number of publications in “top 5” journals, and it doesn’t really matter how many citations you have or what you’ve published elsewhere. The downside of restricting to the top 5 is that this can enforce conformity (in writing style, research methods, subject matter, and conclusions), which leads to this sort of thing. As an interdisciplinary person, I like to publish in lots of different places.

The other weird thing is how we’re proceeding on multiple tracks.

On one hand, the current system of science is flying apart. Here’s Richardson:

We start our article by conceptualizing the scientific enterprise as one large public goods game . . . everyone makes some contribution to the pot and receives some reward realized through the collective sum of contributions . . . Scientists can earn long, rewarding careers. Private-sector firms can capitalize on new technologies. States collect more taxes from a healthier, wealthier workforce. . . .

Here’s what I’ve come to understand over the last three years . . . the scientific enterprise is now witness to widespread, organized defection from the scientific public goods game. Large swaths of players, among them many scientists, reviewers, editors and publishers, are choosing to no longer make genuine contributions to the pot. Parascientific organizations (like ARDA, paper mills and the groups of collaborating editors) now facilitate and profit from mass-scale defection. Many scientists, especially in countries where the resources for doing genuine science are more scarce, are now trained in contexts where defection is the normative behavior.

He continues:

Some model public goods games integrate a mechanism by which defectors are punished. While this can be effective at mitigating defection under certain circumstances, our study shows that the punitive measures employed to enforce science integrity, like retractions and de-indexing, are currently applied far too infrequently to meaningfully increase the costs of defection.

In the meantime, the resources and incentives to doing good science are declining:

Meanwhile, the United States government is dismantling research support infrastructure and funding wholesale, defecting from their role in our immense public goods game and ensuring that contributions by taxpayers will also wither. In effect, competition for resources will only grow more fierce and it will become more and more difficult to make genuine contributions. . . .

If the model public goods game offers any prognostication, it’s that the current paradigm, where defection is the winning strategy, ensures that genuine contributions will only decay from here. We will all be worse off for it. Anyone that has studied industrialized scientific fraud has seen the future of our scientific enterprise . . .

I have seen the future of science. It is ruled by bitter competition instead of collaboration, pageantry instead of exploration. Bright minds beginning careers in science will be taught to debase their training for drudgerous pursuit of meaningless metrics. Those willing to toil over genuine questions will necessarily lose out to those that can furnish cheap answers.

Many scientists worldwide already inhabit this reality. So, we all may. If this vision comes to pass, humanity will lose its most potent engine for progress and its most abundant source of wonder.

So, yeah.

On the other hand, when doing my own writing and research, I’m still living in the world I grew up in, where we craft our articles one at a time and shepherd each through the reviewing process. I keep doing this. This is weird. I don’t know what to think.

Wow. I’d only known Wigner as a famous theoretical physicist (not that I had any idea what he actually did in that area) and as the author of the classic article, “The Unreasonable Effectiveness of Mathematics in the Natural Sciences.”

According to Wikipedia, Wigner not only supported the Moonies, he also was “credited as a member of the advisory board for the Western Goals Foundation, a private domestic intelligence agency created in the US in 1979 to “fill the critical gap caused by the crippling of the FBI, the disabling of the House Un-American Activities Committee and the destruction of crucial government files.”

Wow.

In the grand scheme of things, this is no big deal. Lots of mid-twentieth-century scientists, from Oppenheimer and Haldane on down, had Communist sympathies, Ronald Fisher had a soft spot for Nazis, and Max Planck signed the notorious Manifesto of the Ninety-Three German Intellectuals.

Scientists are just people, and people have a range of political beliefs, including some on the far extremes. So it’s no surprise that if you look into the actions and writings of a bunch of scientists, or a bunch of physicists, or a bunch of Nobel prizewinning physicists, or whatever, you’ll get some wackos. I’d just never known about Wigner, that’s all. My generic image of a mid-twentieth-century physicist is someone like Enrico Fermi, who I don’t think would’ve endorsed the Unification Church in a million years. But who knows? Maybe I’m wrong about that.

Stephanie Rolin, a mental-health services researcher, found out last month that a journal had accepted her latest paper for publication. But there was an asterisk. Community Mental Health Journal was requiring her to fork over about $4,400 — a fee that she hadn’t budgeted for, and one she says she cannot afford. . . .

Most studies appear in paywalled journals, and critics have long contended that those paywalls enrich publishers while gatekeeping taxpayers from the research they fund. The NIH has been pushing for more openness in the ecosystem into which it pours nearly $48 billion annually, and its biggest move yet took effect on July 1. Under a policy that was approved by the Biden administration to take effect at the end of 2025, and moved up six months by the Trump administration, all agency-funded research must now be made freely and immediately available. The previous policy had allowed papers to stay paywalled for up to a year.

But since July 1, some publishers have only given researchers one way to comply with the NIH’s mandate: paying fees that were previously optional. In a year when federal funding has been exceptionally unreliable, scientists say they are stressed about spending thousands of grant dollars on unexpected and questionable open-access charges.

Things don’t have to be this way, open-science experts say: These fees are imposed entirely by publishers. The most prominent examples are Springer Nature and Elsevier, for-profit enterprises that generate billions in revenue. . . .

When Rolin submitted to Community Mental Health Journal earlier this year, she expected the process to go as it had when she’d published in its pages before. At the time, Springer Nature — which sets policies for the 3,000-plus journals under its umbrella — gave NIH-funded authors a “hybrid” of two choices. They could pay an open-access fee to make their study available right away. Or, for free, they could put their paper behind the journal’s paywall while preparing a second copy that was identical save for formatting changes and copy edits. Within 12 months of journal publication, this author’s version would become openly available on a federal database called PubMed Central, in line with a 2008-era NIH requirement. . . .

In late July, Community Mental Health Journal hit Rolin with a $4,390 bill for article-processing charges. Springer Nature’s website now explains that publishing behind a paywall is “not a viable option” for authors like her because it “conflicts with immediate public access policies, such as NIH’s policy.” . . .

Rolin said she’d been aware that the NIH policy was forthcoming, but was surprised by Springer Nature’s hard-line interpretation. Similarly, Elsevier’s terms and conditions for putting studies on PubMed Central list options that involve either author-paid fees or delayed embargoes that wouldn’t comply with the NIH’s mandate. A page describing how NIH-funded authors can “comply with NIH’s public access requirements” has been deleted. . . .

Not every publisher is responding in kind. The JAMA journals, published by the American Medical Association, say that immediately after publication, authors can post their accepted manuscript in a repository of their choice. . . .

But Springer Nature and Elsevier aren’t the only ones reacting to the NIH’s mandate this way. Melanie J. Scott, an associate professor of surgery at the University of Pittsburgh, had a paper accepted in August by the Journal of Leukocyte Biology, which is published by the Society for Leukocyte Biology and Oxford University Press. . . .

In the meantime, researchers will have to figure out how to foot the bill. . . .

Now I’m wondering exactly what is the government policy. I’d think it would be fine to post the paper on a preprint server such as Arxiv, then it doesn’t matter what’s happening with the journals, right?

The funny thing is, this happened to me just the other day, with this article, I think it was, which is indeed published at a Springer journal. Fortunately for me, this research was not NIH-funded so I did not need to pay, nor did I need to withdraw my submission from the journal. I can’t remember how much they wanted to charge me because I was never going to pay. Maybe $2K? And Theory and Society is not a major journal! I like Theory and Society–I’ve published two papers there in the past year–; I’m just saying that it’s wack to ask someone to pay $2K to publish there.

P.S. It’s good to see a government policy that was pushed by both the Biden and Trump administrations so we can talk about it without getting into a political tangle.

I used to work in natural language semantics, and the following dialogue from Lewis Carroll’s Through the Looking Glass, and What Alice Found There was the most common pull-quote to see at the beginning of a thesis.

“When I use a word,: Humpty Dumpty said in rather a scornful tone, “it means just what I choose it to mean — neither more nor less.”

“The question is,” said Alice, “whether you can make words mean so many different things.”

“The question is,” said Humpty Dumpty, “which is to be master — that’s all.”

Humpty Dumpty came to mind recently after a spate of discussions with ML folks about inference (i.e., what they call “learning”).

What in the world does “empirical Bayes” mean?”

Empirical Bayes came up on Wednesday with some ML folks I was talking to, then I ran into Dave Blei this morning, who told me he’s giving a sequence of talks on empirical Bayes over the next few weeks (at Columbia and at University of Chicago). I asked him what “empirical Bayes” meant to him, because it seems to be used very fluidly in ML. He gave me a new usage, saying he used it for any model that uses data to fit parameters of a prior, including just plain old hierarchical modeling fit with sampling. Dave gave the example of ARD in Gaussian processes (aka, hierarchical models).

I would only use the term in the way described in the Wikipedia entry for “Empirical Bayes”, namely

… empirical Bayes may be viewed as an approximation to a fully Bayesian treatment of a hierarchical model wherein the parameters at the highest level of the hierarchy are set to their most likely values, instead of being integrated out.

I pinged Mark Goldstein, one of our top-notch ML postdocs, and he pretty much reeled off the Wikipedia definition. So there seems to be a lot of variation in how this is used.

Robbins on Empirical Bayes

Dave also pointed me to the following video by Herbert Robbins (yes, that Robbins, who was so far ahead of the computational statistics curve that he introduced stochastic gradient descent and multi-armed bandits in the early 1950s).

• Herbert Robbins. 1990. Origins of empirical Bayes. YouTube!

Terminology drift in ML

I’ve been a bit shocked at how many technical terms have drifted in meaning in ML. I’m not talking about people making honest mistakes or clueless mistakes, I’m talking about true drift in meaning where the ML folks will stand by their definitions.

Likelihood: I’ve seen “likelihood” used for what I’d call the data generating distribution and sometimes just as a synonym for density. Aki has this one covered in his recent post, A data model is not just a likelihood. In stats, the likelihood is defined as the function L(theta) = p(y_obs | theta)—that is, it’s a function of theta for some fixed observed data.

Causal: With the advent of LLMs, anything with an autoregressive structure is now being described as “causal” in a “past causes the future” sense. This is even being extended to arbitrary directed graphical models, which are now being called “causal” even when there’s no explicit causality being modeled. That is, you can now describe a simple regression from an observational experience as “causal” with no extra work.

Estimation: Estimation is almost always called “learning” in ML.

Parameters: These are usually called “weights” for neural networks, which I think now make up 99.9% of all work in ML. But if you tell ML folks that neural networks are parametric models, they’ll most often deny it. A statistician would confusingly call a neural network a “non-parametric model” and then tell you that means it has a lot of parameters.

Inference: In our diffusion model reading group, the ML postdocs tell me that “inference” means what I would call “posterior predictive sampling”. For example, generating output to a query from an LLM would be called “inference.”

Bias: In statistics, this usually means expected error. In ML, it’s heavily overloaded. It can be used to name just about any kind of error measure (e.g., errors in Matt Hoffman’s sampling papers). ML folks also use the term “bias” to mean the intercept in a regression (no, I’m not kidding).

Prior: This is often called an “inductive bias” in ML circles, which can include aspects of data generating distributions as well as priors.

In Bayesian statistics, a prior is the marginal distribution over parameters. In ML and informal presentations of “Bayesian statistics,” it’s just any marginal that gets plugged into Bayes’s rule. For instance, the prevalence of a disease p(disease+) is called the “prior” when I evaluate positive predictive accuracy p(disease=+ | test=+) given a testing sensitivity distribution p(test=+ | disease=+). In a k-way classification, “prior” means the marginal distribution over categories.

Bayesian: I think in ML this term is used very broadly for any situation in which there’s a prior not strictly stated as a penalty function for penalized learning (i.e., regression). I think Andrew’s down with this definition as he also uses Bayesian for anything that looks vaguely Bayesian no matter how inference is performed. For example, Empirical Bayes is just Bayes to Andrew, as is using a Laplace approximation or even a simple maximum likelihood estimate (just think of it as a one-point posterior summary!).

Regression: Note quite on topic, but I think of neural networks as just a GPU-friendly form of non-linear regression.

Uncertainty quantification: This is the primary subject of statistics, though I think the term “uncertainty quantification” is much more prevalent in engineering/signal processing than in ML. There are even journals of that title that look sort of like statistics journals.

I’d find reading ML papers easier if there was less meaning drift from well-established terminology. I’m not saying the ML folks should be up to date with Gelman’s idiomatic statistical lexicon (the concepts are fun, and it can be useful for talking to Andrew and people in his circle like the blog readers, but I wouldn’t recommend using these terms in papers without explanation).

I’m sure there are many more terms have been coined or have drifted one way or another that I’m forgetting about.

So when did the loss of certainty begin? Where did we take a wrong turn?

It began around 1800, and it began with geometry. I usually like to quote Mark Twain about this. He said that man is the only animal that has the one true religion—several of them. And that is just what happened with geometry.

The geometry that came from the Greeks is usually called Euclidean geometry, after Euclid. But suddenly at the beginning of the 19th century other geometries were developed—non-Euclidean geometries. Who gets the credit for this is sometimes disputed among historians, but I would say Carl Friedrich Gauss. He was the man who said flatly that we can no longer be sure that Euclidean geometry describes the physical world correctly. The various geometries conflict, although one of them, according to thousands of years of tradition, should describe the truth. You can see the problem.

Can you give me an example of an alternative geometry?

Well, one can cite as an example the theorem of Euclidean geometry that the sum of the angles of a triangle is one hundred eighty degrees. In one of the non-Euclidean geometries, called hyperbolic geometry, the sum is less than one hundred eighty degrees; in another, called double-elliptic non-Euclidean geometry, the sum is always larger than one hundred eighty degrees. Yet all of these geometries are equally accurate insofar as man can measure the sums of angles of triangles.

I actually disagree with Kline on that point! Draw a (virtual) triangle connecting the North Pole to two points on the Equator that are 90 degrees apart in latitude, and each angle of that triangle will be 90 degrees. 90 + 90 + 90 = 270: we can measure that.

The interview continues:

What did the mathematicians do when the bottom dropped out of geometry so to speak?

Many mathematicians tried to rescue and maintain as truths the portion of mathematics built on arithmetic, which by 1850 was far more extensive and vital for science than the several geometries. Unfortunately, other shattering events were to follow. Arithmetic and algebra were the next to go by the board.

The best example of this I could give in a semi-popular book was the creation of what are called quaternions, in 1843, by the great mathematical physicist William Rowan Hamilton. Now in the algebra of quaternions, a kind of number known as a hyper-number, multiplication is not commutative. In other words, if I were talking quaternions, I could not say that three times four is the same as four times three. Other strange algebras were created, and it made people start to worry about the laws of ordinary arithmetic. (The one I just stated is known as the commutative law of multiplication). And if we can have perfectly good algebras in which the old familiar laws don’t work, then how do we know they work in the case of the real numbers? That’s where a mathematician named Hermann von Helmholtz stepped in and told us we don’t know it at all. They work in some situations, but not in all.

Are there any elementary examples of these sorts of algebras, where 2 + 2 = 6, or where 5 x 7 = 35, but 7 x 5 is only 34?

I can think of several. Take a quart of water at 40 degrees and mix it with another quart of water at 50 degrees. Do you get two quarts at 90 degrees? You do not. It’s more like 45 degrees. So you can’t just say I’m going to add 40 and 50 and automatically get 90. It depends on the physical situation.

Consider music, a simple musical tone with a unique frequency and amplitude, say one hundred cycles per second. Now suppose on top of that you impose another note at two hundred cycles per second. Do you get a note at three hundred cycles? Again you do not. It is a note of two hundred cycles, the first harmonic above the one-hundred-cycle note. It is the highest harmonic that determines the pitch—two hundred cycles. This is an important factor in the design of musical instruments.

Those are excellent examples. And recall that the laws of probability do not apply in real life (that is, quantum mechanics; see section 2 of this article).

This is interesting:

If mathematics has no underlying truth–if it is filled with contradictions and uncertainties, why does it work?

There is no definitive answer to that. It just works. The only test we have that mathematics is reliable–not certain, but reliable–is that one can apply its laws to physical problems and make predictions. If the predictions come through, then we can say that mathematics has some substantial basis, but not certainty. I think people can’t help being impressed by what mathematics achieves. Consider the problem of sending a spaceship to the moon and bringing it back. It is entirely mathematical. Of course, there is a tremendous amount of engineering involved in the production of the ship, but the entire plan for it is mathematical. We have a theory about the sun, the planets, and more distant heavenly bodies. We say that what makes them behave as they do is the force of gravity. But nobody knows whether there is such a thing as gravity. We have no physical understanding of it. The theory is mathematical–gravity is a scientific fiction.

The same could be said about electricity and magnetism, couldn’t it?

That’s exactly right. Everybody today knows what a radio is, and what a TV is, but nobody knows what a radio wave or a TV wave is. You can’t smell one or hear one or taste one. But we do have a wonderful mathematical theory developed in the nineteenth century by the mathematical physicist James Clerk Maxwell. The evidence for this wonderful theory is the performance of our radio and TV sets. So we have to accept the fact that mathematics works, or else abandon our radios and our TV sets.

I pretty much agree but I’d put it slightly differently. There are various mathematical theories that don’t work, and because of that we don’t use them to design radios and TV sets. To put it another way, it’s not quite right to say that “mathematics” works; rather, some branches of mathematics work. For example, you could think of various goofy variants of logic and probability as mathematics, but nobody’s using them to build ships. Or, for another example, physicists don’t use classical (“Boltzmann”) probability in quantum problems, because . . . it doesn’t work. The mathematics that works, that’s the stuff that works. Any bit of mathematics works until it doesn’t, which is the point where people try to push it beyond its bounds of applicability.

I enjoyed this bit:

Are most mathematicians since the loss of certainty now working on these physical problems?

No, they aren’t. Most of the mathematics created today–maybe ninety percent of it–is a waste of time. That is an opinion, but one that authorities who are far more creative and far better known share with me.
Can you give us an example of mathematics you consider a waste of time?

Some problems now being considered in the theory of numbers, for example, are a waste of time. Take pairs of primes, called double primes. These are prime numbers in a sequence, eleven and thirteen, for example. No even numbers, of course, are primes. Are there an infinite number of these pairs? Are there triple primes? Endless papers are written about these subjects. Who cares?

That’s how I feel too! That said, I understand that the sorts of insights required to solve this sort of number theory problem are cognitively similar to the sorts of insights required to solve what I would consider to be interesting and important problems in mathematics and statistics. So, even though I agree on “Who cares?”, I don’t think that research in this area is “a waste of time,” any more than it’s a waste of time if you’re an athlete to do cross-training.

The interview continues:

It makes mathematics sound a lot like playing chess or bridge. Exciting, beautiful, challenging; the same words apply to all three kinds of activity.

That’s right. I’m glad you suggested it because it makes the point sharper. People enjoy playing chess. Some people even devote their lives to it. But no matter how ingenious a man is at playing chess or bridge, it isn’t going to change this world one iota. Now mathematicians may probe deeper problems, but it is the same thing.

Again, I kinda feel that Kline is missing the point. For one thing, the effort spent to build programs that can win at chess and go has led to general improvements in machine learning and AI. For better or worse, the study of chess has changed the world, and by more than one iota.

Overall, I’m a big fan of Kline and I like a lot of what’s in that interview, which is one reason it’s interesting to see where we disagree.

We’re very excited about this book. It’s the result of several years of effort. You can pre-order from the publisher or from Amazon.

Here’s the book’s webpage, which includes the data and code for the book’s examples and case studies, of which there are many.

Here’s the table of contents:

Part 1: From Bayesian inference to Bayesian workflow
1. Bayesian theory and Bayesian practice
2. Statistical modeling and workflow
3. Computational tools
4. Introduction to workflow: Modeling performance on a multiple choice exam

Part 2: Statistical workflow
5. Building statistical models
6. Using simulations to capture uncertainty
7. Prediction, generalization, and causal inference
8. Visualizing and checking fitted models
9. Comparing and improving models
10. Statistical inference and scientific inference

Part 3: Computational workflow
11. Fitting statistical models
12. Diagnosing and fixing problems with fitting
13. Approximate algorithms and approximate models
14. Simulation-based calibration checking
15. Statistical modeling as software development

Part 4. Case studies
16. Coding a series of models: Simulated data of movie ratings
17. Prior specification for regression models: Reanalysis of a sleep study
18. Predictive model checking and comparison: Clinical trial
19. Building up to a hierarchical model: Coronavirus testing
20. Using a fitted model for decision analysis: Classification competition
21. Posterior predictive checking: Stochastic learning in dogs
22. Incremental development and testing: Black cat adoptions
23. Debugging a model: World Cup football
24. Leave-one-out cross validation model checking and comparison: Roaches
25. Model building and expansion: Golf putting
26. Model building with latent variables: Markov models for animal movement
27. Model building: Time-series decomposition for birthdays
28. Models for regression coefficients and variable selection: Student grades
29. Sampling problems with latent variables: No vehicles in the park
30. Challenge of multimodality: Differential equation for planetary motion
31. Simulation-based calibration checking in model development workflow

Appendices
A. Statistical and computational workflow for Bayesians and non-Bayesians
B. How to get the most out of Bayesian Data Analysis

One way to think of the book is that it’s all the things missing from BDA, like how to set up an informative prior, what to do when your computations aren’t converging, how to work through a series of models fit to the same data, how to design and perform simulated-data experiments . . . and all sorts of other things too.

The core of the book–parts 1 through 3–clock in under 200 pages, and then we have another 300 pages full of case studies demonstrating different aspects of Bayesian statistical and computational workflow. The appendices should be useful to you too, first because the workflow ideas in this book apply to non-Bayesian inference too, and second because BDA still has lots of valuable material in it, so it’s good to know where to look.

This new Bayesian Workflow book could change your life (we hope), and I thank my coauthors, Aki Vehtari and Richard McElreath, with Daniel Simpson, Charles C. Margossian, Yuling Yao, Lauren Kennedy, Jonah Gabry, Paul-Christian Bürkner, Martin Modrák, Vianey Leos Barajas, for all their care and effort. We thank our employers and various funding agencies for giving us the resources to be able to write this book as a side project along with all our daily responsibilities. And we thank many people for their input on earlier versions of the book, along with the Stan developers making so much of this work possible and the Stan community of users for supplying a continuing series of challenges that have motivated many of the ideas and methods discussed in the book.

I hope you find the book readable, interesting, and useful.

Hey Professor Gelman,

I’m sure this is a weird email that you probably don’t get often but if you could respond that would be awesome!! My school is having a massive debate right now. In an INFINITE amount of attempts (without the loss or gain of strength) could a 5”7, 140lb Senior hit a home run off a 100mph pitch from Paul Skenes, at PNC park (shortest dimension of 320 ft.). If you could get back to me that would be awesome, thanks!

He has no experience playing the sport of baseball, he is not very athletic, there is no wind.

In my opinion I think he can as the possibilities of infinity would eventually create a scenario where he has the perfect swing, with the perfect launch angle, making perfect contact, in the precise direction.

I replied that it’s hard to speak of infinities but my guess is no, he couldn’t ever do it because he couldn’t swing the bat fast enough. But this is just my quick guess; I haven’t done any analysis on the question lately.

Consider vote choice with candidates C = {Left, Right, Other}. “Other” can be a third party, not voting, or “don’t know” in a survey.

A common choice model is multinomial logit:

P[voter i chooses candidate c from C] = exp(f(X_ic)) / sum_c’ exp(f(X_ic’))

Where X_ic are various chooser and choice covariates. This model implies independence from irrelevant alternatives (IIA): ratios of probabilities don’t depend on choice set. (Homework for the reader !)

For example, in a two-round voting system, folks choose from C in round 1 and then choose from {Left,Right} in the runoff. (In a survey, folks choose from C in question 1 and from {Left,Right} in a “push” question.) IIA says that the ratio of Left-vs-Right preference is the same in round 1 (question 1) as in the runoff (“push” question):

P[i chooses Left from C] / P[i chooses Right from C] = P[i chooses Left from {Left,Right}] / P[i chooses Right from {Left,Right}]

In fact, not only does logit model –> IIA (your homework to show), but IIA –> logit model. The latter direction is harder to show, see Luce (1959). For more, see Train (2009).

We aren’t on the verge of a new millennium, but recent advances in AI make automated evaluation and production of science increasingly possible. This raises the question of what qualities we should most be trying to preserve as processes change. It got me thinking it’s a good time to undertake a parallel exercise to Calvino’s, but for science. 

I enlisted Andrew and Berna, and as a first step we are seeking input on which “epistemic virtues” practicing scientists see as most critical to uphold. By virtues, we mean any durable qualities of scientific character and practice that shape how inquiry is conducted, claims are framed, evidence is evaluated, and disagreement is handled. At the link below, we compiled a set of candidate virtues for you to consider, and ultimately rank your top six. 

The virtues are Accountability, Apoliticalness, Authenticity, Awareness of Contextual Dependence, Coherence Seeking, Consensus Seeking, Curiosity, Discernment, Epistemic Cost-Benefit Awareness, Epistemic Fortitude, Epistemic Humility, Epistemic Pluralism, Impartiality, Indifference, Intellectual Autonomy, Intellectual Humility, Precision, Preference for Generality, Reproducibility Seeking, Reputational Grounding, Responsibility, Seeking Contestability, Seeking Correspondence to Observable Reality, Skepticism, Transparency, Unsettledness.

We provide short descriptions of each, and you can also tell us whether you think we missed any important ones.

We’re hoping for broad participation from researchers on this! If you are a faculty member, research scientist, postdoc, or senior Ph.D. student in any area of science, please take five minutes and fill it out. We’ll share the results widely along with some reflections. 

Survey link:

https://docs.google.com/forms/d/e/1FAIpQLSc_jHOrXpFMF3CKgHXfnUhZzeHLnagh3S1G5Kg8ZCyLPXUgxg/viewform?usp=sharing&ouid=103049774617868167713

Acknowledgments: Thanks to Carl Bergstrom, Pam Reinagel, and Tian Zheng for providing some of the candidate virtues.  

Someone who goes by the handle Concerned Cow writes:

I am writing anonymously to ask whether you might be willing to look at a series of major statistical issues in a recently published Nature paper, “CD8⁺ T cell stemness precedes post-intervention control of HIV viremia,” that appears to contain a textbook unit-of-analysis error.

The central analyses of the manuscript treat epitope-specific T cell measurements as independent biological replicates, even though multiple responses come from the same individual (e.g., 23–26 “responses” from only 7 participants). This pseudo-replication inflates the effective sample size and makes non-significant participant-level differences appear highly significant.

When the data are aggregated properly at the participant level, the reported p-values collapse (for example, p = 0.007 becomes approximately p = 0.14–0.39 in Figure 2c, and removal of a single outlier even further eliminates all claimed effects). This pseudoreplication is evident in several panels (Fig. 1d, 2h, 2i, 4c-f). Moreover, a substantial selection bias in Figure 2h-j further compounds the problem.

The above picture reveals both the pseudoreplication, but also a significant imbalance in epitope-specific responses per participant (e.g., 23–26 responses from 7 individuals, with one individual contributing 5 responses), which substantially inflates the apparent sample size and drives the reported significance. When aggregated properly at the participant level, the differences disappear.

Here is a concise one-page technical summary
outlining the statistical issues and why the reported analyses cannot support the paper’s conclusions (which is also attached).

I don’t know nuthin bout CD8⁺ T cell stemness, but there was something about the name “Concerned Cow” that appealed to me. I have the unreasonable feeling that anyone who uses the handle of Concerned Cow will be a good person.

On the other hand, I have no good reason for that feeling, and, in any case, good people make scientific mistakes all the time–I know I do!–so we shouldn’t jump to any conclusions here.

At this point I could just give up, as I’m not planning to educate myself on the topic of HIV viremia, but the above issue seems purely statistical so I’ll take a look. I have some sympathy for people who see problems with published papers. I guess the Cow should also post these concerns on Pubpeer.

I guess the main concern here is that of generalizing from only 12 people. In a medical study you can learn a lot from just one person, so it’s not like a low sample size is disqualifying.

So maybe the most helpful way to consider this sort of study is not to compare it to a hypothetical study of 1200 people (in which you should be likely to get statistical significance even with an unquestionably legitimate analysis) but to a study of one or two people.

What do you get out of N=12 that you wouldn’t get out of N=1 or 2? Mostly, what you get is some sense of variability. The 12 people in your study will be different in various ways–different bodies, different ages, different stages of the disease, etc. If all 12 people show the some responses, that’s telling you something. To the extent these responses vary, that’s telling you something too.

Can N=12 give you reliable information on population average behavior? Let’s do a quick calculation. Suppose you’re comparing two groups with 6 people each. If the standard deviation of your outcome variable within each group is sigma, then the sd of the difference between the two group means is sqrt(sigma^2/6 + sigma^2/6) = sigma/sqrt(3) = 0.58*sigma. So, if you want your comparison to have a signal-to-noise ratio of 2 (so that you’d have approximately 50% chance of attaining conventional statistical significance in a clean experiment), your underlying mean effect size would have to be at least 1.16*sigma. That would be a huge effect. Not that it can’t happen, just that it will only happen if:
(a) The underlying effect really is large.
(b) The outcome varies very little within each group, or, if it does vary, this variation is explained by pre-treatment predictors included in your model.
(c) The outcome is stable within each person and is measured precisely. Just about any amount of uncontrolled measurement error or variation over time will make it hard for you to get that signal-to-noise ratio down.
(d) The treatment or exposure is measured well. Misclassification or noise in the treatment variable will destroy any chance of keeping that high signal-to-noise ratio.

From the perspective, one of the key roles of an N=12 study is to identify the sources of variation and error in your experiment, so you can figure out how to control these. Or, where that can’t be done, how you can adjust for them.

To paraphrase the famous saying:
God grant me the serenity to adjust for the things which cannot be controlled; The courage to control the things which can be controlled; And the wisdom to know the difference.

Now, to get to the study at hand, the key statistical point is that, unless you’re pretty sure you’ve satisfied conditions (a), (b), (c), and (d) above, you shouldn’t be looking for statistical significance in your data anyway, for three reasons:
1. With so much variability, the fact that an observed difference not reach statistical significance should not be taken that the underlying effect is zero, or even that it is small.
2. Any differences that are statistically significant in the data are likely to be huge overestimates–that’s the well-known problem of type M errors in noisy studies.
3. If you’re under pressure to find statistical significance, there’s a motivation to cheat. That’s the Amstrong principle. I’m not trying to say or imply or insinuate that the authors of this particular paper were “cheating,” just that, by reporting significance levels in this small study, they’re (inadvertently) asking for trouble.

And, indeed, these issues arise here. In addition to reporting some statistically significant comparisons (the ones addressed by Cow above), the paper also reports some lack of associations based on non-statistical-significance.

What should the researchers have done?

Most of the paper under discussion is about the technical details of the experiment and the associated biological processes. There’s also lots of data, including at the individual patient level. I won’t try to evaluate any of this! My guess would be that the value of the paper is in all these data and that these results could be useful in designing future studies. I wouldn’t do that based on statistical significance, that’s all.

You may notice that I never got around to evaluating the particular issues raised by Concerned Cow. That’s because I wouldn’t expect to see statistical significance in this small-sample, high-variance setting, absent some selection on forking paths. On one hand, this means that I would not be surprised if the Cow’s concerns are legitimate; on the other hand, in some sense it doesn’t matter so much anyway, because even if the standard errors aren’t invalidated by clustering in the data, I’d still be concerned.

It’s possible that the authors of the published article will see this post. If they do, my recommendation to them is to think more about how to control and adjust for variation, and to not use statistical significance thresholds to classify your results.

It doesn’t seem that belief in ghosts will be going away any time soon, which on the face of it might seem strange, given that the existence of ghosts violates our current understanding of science, also there’s no good evidence for ghosts–just the usual story with supernatural phenomena of a lot of bad evidence that disintegrates as you look at it too closely. On the other hand, it makes sense to believe in ghosts because the idea is so intuitive: when friends or relatives or pets die, they can remain vivid in our minds–in some way they still feel very alive–so it’s natural to think that this feeling can have some physical manifestation. This seems similar to the way that extra-sensory perception seems like it should be true: at some intuitive level, if seems like if you focus your mind really hard, you should be able to move objects at a distance, perceive things that are far away, etc. And, of course, once you hold a belief or want to hold a belief, it’s not hard to find evidence that is consistent with what you want to be true. So, yeah, ghosts. Again, an interesting example because there’s no organized religion pushing it.

Although belief in ghosts is prevalent, it’s not exactly respectable. There will be the occasional news media feature on ghosts, but it’s more for fun or sociological curiosity than anything else. For example, a quick search led to this Halloween-themed NPR story, “Listen If You Dare: Exploring Our Belief In Ghosts.”

Then there’s extra-sensory perception, which used to be respectable but not anymore. My theory on ESP is that, in the wake of the developments in quantum mechanics in the 1920s and 1930s, it seemed reasonable to believe that “spooky action at a distance” could be occurring in all sorts of ways. Indeed, ESP is not much of a stretch given the discovery of radio waves in the late 1800s. And regular readers of this blog will recall Alan Turing’s statement: “I assume that the reader is familiar with the idea of extra-sensory perception, and the meaning of the four items of it, viz. telepathy, clairvoyance, precognition and psycho-kinesis. These disturbing phenomena seem to deny all our usual scientific ideas. How we should like to discredit them! Unfortunately the statistical evidence, at least for telepathy, is overwhelming.”

Turing was wrong on that one. Not just erroneously believing in phenomena that, as far as we know, don’t exist, but, more importantly to me, wrong in his assessment of the evidence. Even if it were to somehow turn out that ESP is real, it was an unambiguous error to believe that there was overwhelming evidence for ESP in 1950. In any case, ESP was popular with credentialed scientists from the 1940s through the 1970s (see the second-to-last-paragraph of this post), but decades of debunking had their effect, and now it’s not taken seriously. Even NPR seems to have given up on the topic, after getting burned by running a credulous report in 2011.

Somewhere near ESP, but still vague enough to maintain some academic and news-media credibility, are the more extreme claims regarding mind-body healing.

And then there’s the belief that UFOs are space aliens. We’ve talked before about the role of the elite media in promoting this idea.

I’ve discussed before why I characterize UFO’s-as-space-aliens as a supernatural belief. A good analogy here might be unicorns. We can easily imagine a world in which unicorns are real, just as we can imagine a world in which space aliens are exploring our planet. The problem is that there’s no good evidence for space alien UFOs or unicorns, or is there any good indirect evidence or theories (for example, residues of previous alien spacecraft, or unicorn fossils, or whatever).

Lots of people already believe that UFOs are, or might be, space aliens. No assist from the news media is required here. It’s a belief that’s out there. The role of the elite news media here has not been so much to disseminate the belief but rather to legitimize it. The idea that UFOs are space aliens is as ridiculous now as it’s ever been, if not more so. That doesn’t stop a large percentage of people from believing it.

I thought about all of this after receiving this email from Eric Potash (great name for an environmental scientist, by the way!):

Here’s an essay about UFOs by economist Robin Hanson (linked by Tyler Cowen on Marginal Revolution) that starts:

IMHO, the strongest evidence so far that (some) UFOs are aliens just dropped… This evidence is of many brief bright glints of sunlight reflecting off of big surfaces in high orbit around Earth, before humans had put anything up there, and correlated in time with both UFO reports and nuclear tests.

There are a lot of technical details about telescopes and then the correlation claim is elaborated:

These glints also seem to have a significant date correlations with nuclear tests and UFO reports. Glints were 45% more likely (p = 0.008) on dates within one day of nuclear tests, and there was a significant (p<.001) correlation between the number of UFOs reported and number of glints on each date.

That reference is currently under review at Nature scientific reports. Thought this nexus of economists, UFOs, p-values and Nature might be of interest to you.

My interest here is not in Hanson’s arguments, which I find absolutely ridiculous–you can follow the link to that post and read the comment thread to see lots of clear arguments explaining the problems with the purported evidence–; rather, I’m interested in the question posed in the title of this post: Which beliefs are considered acceptable and which are not?

A bunch of elite news media people such as Ezra Klein, Nate Silver, and Tyler Cowen, along with elite-media-adjacent people such as Robin Hanson, are into this whole space aliens thing. (Just to be clear on the labeling here, I consider myself to be elite-media-adjacent as well, so I’m certainly not saying that all or even most elite-media-adjacent people are space-aliens-curious.) We’ve reached a point where a pundit can openly talk about believing in space aliens, in the same way they can talk about believing in Jesus or Moses or whatever, but not in the way that they would talk about ghosts.

I’m not saying Hanson wouldn’t talk about ghosts. In his writings, Hanson doesn’t seem to be constrained by what might be socially acceptable, as evidenced by his notorious post about rape. But if he were to start writing about ghosts, I don’t know that Cowen and the others would pick up on it. The space aliens thing, like belief in the Bible, or belief in the Book of Mormon, or whatever, has been successfully carried to respectability in a way that ghosts haven’t, even though lots of people believe in all these things.

Following up on this discussion from a few years ago, John Kastellec offers some practical advice for producing better graphs:

In this short paper I present a few practical tips for producing better published graphs. These include: making labels big enough to read; avoiding legends and labeling lines directly; using small multiple plots; and using different line types and shapes to draw distinctions.

I agree with these points. Well, not the last one. I much prefer using colors, rather than line types and shapes. Sometimes line types and shapes can work, but I recommend first using colors.

Also, I’d add the recommendation to give each graph a title or embed it in a figure with a caption. Remember, a picture plus a thousand words is better than two pictures or two thousand words. It’s a classic error to think that a graph should be self-explanatory.

In any case, I disagree with this statement from John:

These are suggestions designed for better published graphs; they are not generally applicable for scholars’ own visualizations in the course of their research workflow. Indeed, some of the basic problems arise because what works well as a default during the data exploration phase does not translate well to published graphs.

No no no no no!

The same attributes that make a graph more useful and readable by others will make it more useful and readable by you. You are the first audience for any graph. All too often I’ve seen people make ugly, unreadable default graphs, and this has inhibited their ability to learn from their data and models. Give your graphs titles, label those axes, use small multiples without making each plot too crowded, use colors effectively, etc etc . . . do it for yourself too!

OK, here’s a story for you. Many years ago I knew a couple who were moving and they needed to sell their apartment. To make it more sellable, they had all sorts of work done, fixing everything that was broken, getting new appliances, new floors, etc. It was wonderful! And they should’ve done it earlier, while they were still living there!

Oh, one more thing. I don’t like “comb plots,” which multiplex an x- or y-axis. Here’s an example from John’s paper:

I think a simple dot plot would be better, with a red dot and a blue dot on each row. That said, I appreciate that they listed the issues in decreasing order of frequency rather than alphabetically.

Overall I think that John’s article should be useful to many people. Even if you don’t agree with all the recommendations, it gives you something to think about.

For more on the topic, placing graphical displays in the context of statistical measurement, I recommend chapter 2 of Regression and Other Stories.

Remember what Tufte taught us: All graphs are comparisons.

The FTC has “settled” with OKCupid, which is a dating app owned by the Match group, on giving data to a face recognition company (Clarifai, a competitor of Clearview) without proper notification to its users (link).

I [Kaiser] put “settled” in quotes because throughout the story of the data belonging to Americans, the meaning of many words has been warped beyond recognition. This is, as Ars Technica pointed out in its header, a “settlement” without any financial penalty. . . .

The FTC claimed it knew what happened: based on the passage quoted by Ars Technica, it appeared that they merely repeated the story told by OKCupid, Match, and Clarifai. They claimed that no formal agreement existed but disclosed that the founder of OKCupid and the CEO of Match were both “financially invested” in Clarifai. These parties somehow believed that this cover story gave them a get-out-of-jail card, a rationale to support their use of the word “sharing”. In fact, this is even more troubling than if a straightforward commercial agreement were to exist.

For one, this story proves that user data at tech companies are at the hands of individuals. (We already sort of knew from some past actions e.g. by Elon Musk.) It also shows that these individuals – none of whom face any kind of sanctions – will sell out their users for personal financial gain. When no agreement exists, it’s harder to trace where, when and what data have left the building. . . .

This settlement stemmed from actions that took place in 2014, so it took 12 years for regulators to uphold the law by a friendly handshake with the offenders. . . .

Let’s read the terms of the settlement carefully, shall we? Hold on to your seat belts, this is truly scary:

OKCupid and Match… agreed to a permanent prohibition barring them from misrepresenting how they use and share personal data.

Wait, so businesses are heretofore not prohibited from “misrepresenting how they use and share personal data”? It takes 12 years and a negotiated settlement to confirm to Americans that our businesses are in fact free to misrepresent how they use and share personal data – unless the FTC imposes a specific “permanent” ban from lying?

I’m not the first person to note that white collar crime is basically legal now.

P.S. Way back when, OK Cupid had a blog! I guess that you can make cool graphs (there were more here but the links no longer work) while still misrepresenting how you use and sell, or share, personal data.

From the Australian Bureau of Statistics:

On 17 July 2025 the Australian Bureau of Statistics (ABS) announced that it will not release statistics from the 2023-24 Survey of Income and Housing (SIH), noting that the data did not meet the ABS’ high standards for official statistics.

Wow!

Here’s the background:

The ABS has conducted surveys of household income in various forms since 1974, and the SIH has been conducted in its current form since 2003-04. It gathers data on household income sources and amounts, net worth, housing situations, and both household and personal characteristics.

While SIH has been conducted on a 2 yearly basis since 2003-04, the last annual results were released from the 2019-20 cycle. The scheduled 2021-22 cycle was cancelled in December 2021 due to disruption and statistical impacts associated with the COVID-19 pandemic. . . .

Across its survey program the ABS evolves its collection approach, wherever possible, to improve efficiency and reduce cost, reduce demands on the Australian public to provide their data, improve respondent experiences and take advantage of methodological and technological innovations. While these innovations are not always visible to data users, they are critical to maintaining the quality of official statistics and have been a feature of the SIH program since its inception.

They continue:

Given an increasing public preference for digital engagement, ABS made changes to the SIH 2023-24 sample design with the intention of delivering high quality data while making it easier for households to complete the survey. A new approach was implemented whereby the sample was divided into two groups to reduce any potential statistical impact arising from households not responding to a digital survey.

One group was randomly selected for ‘self-enumeration only’ using a digital (web or phone) approach, with no interview follow-up of non-responding households. In the second group, those households that did not complete a web or phone interview were assigned for field follow-up by an interviewer. This sample design allows the ABS to apply statistical adjustments to address potential under-representation of groups that were less likely to provide a digital survey response.

Unfortunately, when implemented, system and business process limitations resulted in some households in the second group not receiving the field follow-up required to support this design. This left the survey results open to bias arising from systematic differences between those who responded to the survey and those who did not. . . .

The ABS had planned to process SIH 2023-24 data progressively from the commencement of data collection, allowing an early assessment of data quality. Unfortunately, initial data extraction was significantly delayed by system problems, which in turn delayed the commencement of data processing. SIH 2023-24 data collection ceased on 30 June 2024. The inability to transfer the collected data from the data collection system to the statistical compilation system was not resolved until December 2024. . . . Detailed, aggregated data comparisons were made against previous iterations of the SIH, the Australian System of National Accounts (ASNA), the Person Level Integrated Data Asset (PLIDA), and non-ABS sources such as the Household, Income and Labour Dynamics (HILDA) survey.

These assessments identified inconsistencies, including:

Estimates of personal and household income were much lower than expected when compared to previous iterations of SIH or information in the National Accounts and other sources.
Aggregate liabilities and loan values were lower than expected, and exhibited lower coverage compared to the National Accounts than previous SIH cycles.
An observed decrease in superannuation income since SIH 2019-20 was not consistent with other sources and population trends.
Falls in outstanding mortgage values since SIH 2019-20 were also inconsistent with trends observed in other sources.

So:

Taken together, these results suggested the SIH 2023-24 results were not fit-for-purpose and led ABS to initiate an internal Quality Incident Response Plan (QIRP) process. . . .

Following review, the QIRP process confirmed that the observed data quality issues stemmed from two key statistical limitations discussed above, namely:

• A pattern of survey non-response that reduced the representativeness of the results.

• Pervasive question level non-response and response error, largely caused by ‘skippable’ questions in the survey form.

The QIRP panel found that the combined effect of these errors rendered the data unsuitable for its intended purpose.

That’s a big step. The report discusses the plans of the Australian Bureau of Statistics for this survey going forward.

Who are these customers who would pay $600 + $6.99/month to have toilet cam photos uploaded to a consumer-products company?

I mean, I get it that Google’s grabbing all sorts of information on me right now, but at least I’m not directly paying for it. Are there really people who’d shell out all this money to share their poop-cam data?

I guess maybe the same sorts of people who fall for the government’s junk science, or who buy the supplements advertised by Dr. Oz, Andrew Huberman, etc. Maybe they upload their poop photos after taking their Stanford-recommended cold showers.

A year of this service would cost $683.88. For this price you could afford 440 Jamaican beef patties, one-tenth of a paper in Nature Communications, or 1/27th of an invitation to a conference featuring Grover Norquist, Gray Davis, and a rabbi.

I’d take the Jamaican beef patties . . . but maybe after eating 440 of them I’d actually need the poop cam!

Jordan Anaya (of Pizzagate fame) writes:

For the last year I [Anaya] have been working on a web application that implements how I trained deep learning models at Johns Hopkins entirely in the browser. It’s available at https://aleaaxis.net/

I’m not sure if you have any experience with building deep learning models, but there’s a data generation component which you might find interesting. The data generation wasn’t meant to be very complex but I’d be interested if you have suggestions for improvement.

I think it’s going to be really useful for introducing students to deep learning. There’s a video at the learn page if you want to quickly see some examples. If you run into any issues let me know (I haven’t tested on different hardware / operating systems).

I don’t know enough about deep learning to be able to evaluate this, but it could be of interest to some of you. I like the idea that it can simulate from a generative model.

Yesterday I read an Updike story from the early 2000s called The Varieties of Religious Experience. The title reminded me of the classic Irwin Shaw story, Main Currents of American Thought. The stories have nothing in common other than their titles, and Shaw is pretty much forgotten nowadays . . . in some way, Updike is similar to Philip K. Dick as a writer in that he takes the same few characters and recycles them through lots of similar situations. Dick has the pathetic lead character (“Joe Chip”), the reliable older man (“Runciter”), and the nagging wife (the sister in Confessions of a Crap Artist). Updike has the “Updike” character (a student or young man in the early stories, then a young husband and father, then a divorcing middle-aged man, then a rueful man in late middle age, drifting among his own thoughts), along with various supporting characters. I like lots of individual Updike stories but I’ve gotta say that the best are the Maples stories because these are the only ones where the woman character is as strong as the man. Joan gives as good as she gets.

The Varieties of Religious Experience is an unusual Updike story in that it centers on a public event–the 9/11 attacks–and it offers multiple perspectives, including an elderly “Updike” visiting his adult (divorced, of course) daughter in New York, and then also a scene with two of the hijackers and another on one of the planes. Many of Updike’s novels are deeply concerned with current events, but his stories usually focus on internal or local interpersonal drama, with the outside world being only implied. And I don’t think this was one of Updike’s most successful stories or even one of his better late stories.

Still, the story moved me. Reading it reminded me of how it felt in New York when the towers went down. We were all completely numb for about two weeks–it was hard to think about anything else at all–, and even for years later it was in the front of my mind. All my thoughts revolved around it. Maybe the best analogy is how, in the first two years of the 2020s, the pandemic seemed to be all that mattered. But the World Trade Center attack was different in that this special effect was concentrated locally. Everybody was shocked by the event, but it’s my impression that people living in New York were particularly shaken, even those of us not living or working close to that neighborhood.

Reading the story now, my thoughts immediately turned to those residents of Kiev, Gaza, Beirut, and Tehran whose buildings were destroyed by sudden strikes from the sky. I imagine that this has to feel a lot like the World Trade Center attacks, but much more so, as the bombs kept coming and coming. After 9/11 we were waiting for the other shoe to drop, but it didn’t–or, at least, not locally.

I wonder what Updike would think. He had mixed feelings about the Vietnam war at a time when many American writers and intellectuals opposed it (in 1966, he wrote, “I am for our intervention if it does some good–specifically, if it enables the people of South Vietnam to seek their own political future. . . .”, but then a year later he wrote, “The bombing of the North seems futile as well as brutal and should be stopped” and “I differ, perhaps, from my unanimously dovish confrères in crediting the Johnson administration with good faith and some good sense”). So I don’t know if, in writing about the feelings of Americans watching their towers come down, he thought about the Vietnamese seeing that happening to them, decades before. When I saw the attack on live TV back in September 2001, I remember thinking about the historical context, but it still shook me in a way that a comparable event in another country, or even another city, would not have. I don’t know anyone in Kiev, Gaza, Beirut, and Tehran (I have ancestors who were from “near Kiev” but that was such a different time that I can’t really make the connection in my mind), so it’s hard for me to say how the residents of those cities might feel; indeed, even if I knew people living in these places, it would be a stretch of imagination to picture their emotions. Which I guess is one reason we have fiction.

Let’s first review Poststratification:

• We want the population mean E(Y)
• We have Y,X in sample, X in population.
• So we calibrate our estimates of E(Y) to population distribution of X.

Gao et al. 2021‘s example:

• Y = support for gay marriage
• X = sex, race, income, state, age, education
• sample data on Y, X from National Annenberg Election Survey 2008
• population data on X from the American Community Survey (ACS)

By the law of total expectation: E(Y) = E(E(Y|X)). When our estimate of E(Y|X) is the sample mean of Y for folks with that X, the aggregate estimate is classical Poststratification (no honorific). When our estimate of E(Y|X) is based on a model that regularizes across X, the aggregate estimate is Mr. P.

How to regularize across X ? Suppose age is one of the X variables. Gao et al. 2021 consider 3 priors for the coefficients a_j of age groups j = 1,…,J:

1. Independent Normal: a_j ~ N(0, sigma)
2. Autoregressive: a_j | a_{j-1} ~ N(rho a_{j-1}, sigma) with rho in (-1,1)
3. Random Walk: a_j | a_{j-1} ~ N(a_{j-1}, sigma)

The first is often used in Mr. P. The next two belong to Ms. P, as they use the ordinal structure of age, where age group j is closer to age group j + 1 than age group j + 5. Using this structure can help Ms. P regularize more, with smaller sigma and more borrowing information across ages.

Gao et al. 2021 simulate data from E(Y|X) = logit^-1(… f(X_age[j])…) where the function f(X_age[j]) is how support varies by age. They consider 3 smooth functions f(x). For data simulated from each, they fit models with the 3 priors above. Ms. P out-performs Mr. P, with the Random Walk structure doing best:

This footnote in the Stan documentation might explain why Random Walk outperforms Autoregressive:

In practice, it can be useful to remove the constraint to test whether a non-stationary set of coefficients provides a better fit to the data. It can also be useful to add a trend term to the model, because an unfitted trend will manifest as non-stationarity.

Gao et al. 2021 also consider spatial priors, where neighboring PUMAs are more correlated. They simulate data both with spatial smoothness and from independent Normals, to confirm that the prior “does not force spatial structure when it’s not present”. Did they skip this for the age model ?

Along with the simulation studies, Gao et al. 2021 also apply Ms. P to the National Annenberg Election Survey 2008 and ACS example above.

What to do when your estimate is 1 standard error away from 0?

Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University

We provide a new answer to this simple yet very important question. Thinking clearly about this problem leads us to bring in many ideas in statistical analysis and computing, including causal identification, meta-analysis, Mister P, expectation propagation, decision analysis, experimental design, and the fundamental unity of Bayesian and frequentist statistics. We demonstrate our approach in examples from many applications, including medicine, social science, business, sports, and public policy.

This work is joint with Witold Więcek and Erik van Zwet.

In addition to all the above, I’ll probably drift into some related general topics such as the role of experimentation in science and engineering and the limitations of thinking about policy analysis in terms of causal inference.

The link to Bump’s longer post is here. I’m not questioning Moynihan’s or Bump’s numbers–I haven’t checked them myself, but I have no reason to believe that they’re wrong–but I think they’re both kinda missing the point regarding Bezos’s motivations.

Here’s my story (which, as noted in the above title, is entirely speculation):

Jeff Bezos is a rich guy with money to spare. He can afford to buy a major newspaper if it is for sale, or to start his own newspaper if he wants to do so. He could’ve done this in Seattle, for example. But that’s not just true of Bezos, it’s true of many other rich people, and most of them are not buying or starting up news organizations. There are reasons for this: first, running a newspaper costs money, and even if you’re rich, you don’t want to throw money away. Second, fewer and fewer people read newspapers or even watch TV news. If you buy or start a newspaper, you’re not getting in on the ground floor; it’s more like you’re stepping into an elevator that’s in free fall. The dream in tech is to find the next big thing with unlimited potential, not to invest in a mature and declining industry.

Why, then, did Bezos buy the Washington Post in the first place? Well, it is a unique property with influence and a storied history, so maybe it’s just that it became available and he grabbed it. He might have had some vague goal of supporting journalism, also owning a newspaper provides some protection against biased reporting. Even if Bezos is owning the newspaper in a hands-off way, and its reporters are willing to report bad news against him, presumably they’ll give him a fair shake and not be actively biased against him. Later he was hassled by the National Enquirer, so it’s not like this sort of thing couldn’t happen. If you think the media might do you harm, it can be good to have your own media outlet. So, some mix of public spiritedness, the idea of being a leading citizen, also with possible defensive value. And, who knows, maybe the possibility of a business success.

There’s a logic to it. Supposing you can afford the initial outlay and the ongoing cost, if you buy or start a newspaper, staff it with serious journalists and editors, and let them do their thing, you’ll have bought yourself some good press and some protection against biased reporting and political intimidation, you can feel you’re making some contribution to civil society, and it’s a toy to play with; nothing wrong with that!

But then what happened between 2013 and 2026? My guess is that it all just became too much hassle. Instead of being a source of good press, owning the Washington Post became a public relations hassle. When it reports stories or run op-eds that make Republicans look bad, Bezos gets attacked from the right. When he tells his staff to go easy on Republicans, he gets attacked from the left. Dude’s getting slammed from both sides. If he lets the Post continue to be run by its editors, he gets no credit from the center or left but he has to deal with an ongoing stream of annoyance from the right. But when he decides to solve that problem by bringing in a compliant outside editorial team, he’s suddenly the man who killed the news.

Meanwhile, it’s not clear what benefit Bezos is getting from owning the newspaper. In theory he could use his ownership of the Post as a political tool, for example telling Trump that if the government doesn’t give Amazon some juicy contract, he’ll start filling up the front pages with Jeffrey Epstein stories. But in practice it doesn’t seem that this is the sort of hardball that Bezos likes to play. Avoiding sales tax is one thing, lobbying is fine, but maybe outright threats would be an escalation too far.

So, instead of being fun, good publicity, and a sort of political insurance, owning the Post has become the opposite: it’s a pain in the ass, invites attacks form all sides, and entangles him in politics more than ever. Also it keeps draining money.

So, from that perspective, it makes sense for Bezos to wash his hands of the whole thing. The point is not the cost of running the newspaper as compared to his unimaginable wealth is irrelevant.

At this point you might wonder why Bezos doesn’t sell what remains of the newspaper or just shut it down. It could be a business decision on his part, or maybe an assessment that it could be useful to have the Washington Post around if you’re in some fight involving public opinion.

And, again Bezos isn’t the only zillionaire out there who can afford to buy a big-city newspaper. The fact that zillionaires (whether politically-minded or just motivated to protect their business interests) are not queuing to buy newspapers or set up their own alternatives suggests that they don’t see much value in media properties. There’s also something about the nature of the business: buy a news or organizations and it comes with all these reporters who want to report the news without political interference, then you have to suck it up and get bitten by the people you’re feeding, or you have to fire a bunch of people and replace them with compliant substitutes. Maybe this will be less of an issue going forward as the supply of conservative journalists increases.

P.S. Sam in comments offers another plausible explanation, which is that Bezos saw this as a business opportunity and thought that he’d be able to turn the Washington Post into a profitable newspaper.

It’s my impression that, up until the early 1900s, partisan newspapers were the standard in this country, but then in the early part of the last century, the newspapers became less nakedly partisan. I guess this was because there was so much money to be made by attracting mass audiences, and mass audiences are less interested in reading one-sided presentations of the news.

In recent decades, there have been some partisan newspapers created from scratch, but they’ve mostly been representing extremist political views and funded by foreigners. The first of these was the Washington Times which was founded by the Moonies in the early 1980s and continues to push a right-wing agenda. Also of course there’s Fox News, which is pretty much part of the Republican party and has been very successful, both as a propaganda outlet in its own right and also in doing its part to set the national news media agenda. The Republican party is not a fringe group like the Unification Church; the point is that these news organizations are ideological and partisan in a way that is common with political magazines (the Nation, National Review, etc.) and websites, but different from major twentieth-century U.S. media, which even when they had strong political slants (for example, the Chicago Tribune’s isolationism or various New York newspapers’ internationalism in the 1940s) were still mostly reporting straight news. With the Washington Times and Fox News, the Moonies and the Republican Party are going back to an earlier tradition, more like the partisan-aligned newspapers of the eighteenth and nineteenth centuries. Which is not to say that they’re bad guys for doing this. Running a partisan newspaper or TV network is a legitimate strategy, and if the Democrats have not gotten their act together to do this, that’s on them.

I’ve written earlier about the asymmetry of media bias, with Democratic reporters–a survey awhile ago found that twice as many journalists identify as Democrats than as Republicans–biasing their reporting by choosing which topics to focus on, and Republican news organizations (notably Fox News and other Murdoch organizations) biasing in the other direction by flat-out attacks. I’ve never been clear on which sort of bias is more effective. On one hand, Fox can create a media buzz out of nothing at all; on the other hand, perhaps there’s something more insidious about nonpartisan news organizations indirectly creating bias by their choice of what to report.

Moving aside from major newspapers and TV news outlets, there were some more partisan, or ideological, outlets on the fringes of the U.S. media from the 1960s through the early 2000s. I’m thinking here of talk radio, which mostly fell in the spectrum from sports talk to far-right ranters, with a lot of religious programming thrown in, and of weekly city newspapers that featured entertainment listings, sex ads, and mostly left-wing journalism of the muckraking variety. Some of these programs and newspapers must have been created primarily for ideological reasons, but they were also a way to make money, and I guess there was a natural market segmentation, with weekly newspapers in liberal cities and conservative talk radio in other parts of the country.

Nowadays it’s all the internet, so you don’t get this sort of geographic segmentation. Maybe that will be good news in the sense of reducing geographic political polarization. Nowadays you can get your left-wing or right-wing take on anything, from anywhere with an internet connection.

But, to return to the 1900s for a moment, somewhere along the way, the model switched from creating news organizations to buying ads. In the mid-twentieth-century, political advertising was seen as somewhat disreputable, but by the end of the century, it was accepted that the way to run a mass campaign was to raise tons of money and then spend lots of it on TV ads.

The basic setup was this: The mass media–newspapers, radio, local and network TV–exist. You can pressure them on the margin (“working the refs,” etc.) but mostly you just accept them as part of the system, you pay for ads and you do your best to get free media time via press releases or whatever.

One thing that you didn’t see was political organizations buying or trying to take control of the mass media. The most notable exception may have been Henry Ford buying the Dearborn Independent and turning it into a racist rag, but the Dearborn Independent was not a major newspaper before that purchase. It’s not like he tried to take over the Detroit News. Similarly, political organizations would start new magazines and new think tanks, only rarely taking over existing institutions.

Recently, though, we’ve been seeing more instances of political shutdowns or takeovers of existing media. The big step might well have been the shutdown of the left-wing gossip site Gawker by the right-wing investor Peter Theil. More recently there was the alignment of CBS News with the Trump administration and threats against the New York Times.

So the far right has been operating on four tracks: (a) following the Fox News template with increasingly extreme fringe media outlets (Newsmax, One America News Network, etc.), (b) giving arms-length encouragement to extremist provocateurs such as Alex Jones, (c) taking over formerly independent outlets (CBS, Twitter, etc.), and (d) attacking or trying to neutralize remaining mass media (New York Times, Washington Post, etc).

And one question is why this wasn’t being done earlier. Back in the 1970s, say, why didn’t a bunch of rich guys get together and buy one of the big 3 TV networks, or some major metropolitan dailies? Or, for that matter, why didn’t some rich left-wingers do this? I can give a few answers:

1. There were some super-rich men in the oil industry etc., but it’s my impression that industrial money at the time was more tied up in the companies and not so much at the discretion of a few tycoons. So, for example, Gulf + Western acquired Paramount Pictures, but this was for their investment portfolio, not so they could get Paramount to release corporate propaganda.

2. There were restrictions on what you could do on TV, politically. The Fairness Doctrine said that you had to present both sides of any issue, so it wouldn’t have been possible, before 1980, to have the TV equivalent of a partisan newspaper.

3. There was less partisan polarization, so even if you were a right-wing Republican tycoon or, less likely, a left-wing Democratic tycoon, you wouldn’t just support your party straight-up on almost every issue. There could be a logit to a conservative tycoon buying a newspaper and giving it a conservative line, but it might just seem weird for it to follow a partisan Republican line. Such a thing would look too much like Pravda, which in the mid-to-late 1900s would be an embarrassment. And there were some newspapers back then that took pretty hard-line free-market, anti-communist lines, so in lots of cities there’d be no need for any action in that direction.

4. I guess the other thing is that major newspapers were mostly not for sale. I don’t know the full story on this one . . . before Jeff Bezos bought the Washington Post in 2013, did any other rich guys make a bid for it? What about the Miami Herald or other influential papers?

5. It wouldn’t make business sense. It’s my impression that, in the mid-to-late twentieth century, city newspapers mostly represented the local business elite. This again ties into the relative lack of partisan polarization in that earlier era. Suppose you were a rich guy 50 or 80 years ago with a strong attachment to the Republican party. Would it have made sense for you to buy a major newspaper and turn it into a partisan rag? Maybe not. The paper was probably already supporting your business interests already, and by making it partisan you might just antagonize a lot of people and hurt your bottom line. It would make more sense to let the mass media do its thing and exert your influence through donations to campaigns and to political magazines, think tanks, etc.

There’s also the question of how things are different for the left and the right. In the late twentieth-century, the news media had a center-left tilt on national politics–for example, surveys found that about twice as many journalists identified as Democrats than as Republicans–so this perhaps explains some of the Democratic party’s relative lack of aggressiveness in media strategy. Nowadays, though, there’s a lot less of traditional independent media, and web platforms are more important. Twitter’s gone in the Nazi direction but still has lots of left-wing stuff on it too; platforms such as Facebook and Google seem to be trying to keep all sides happy; I don’t know how this all shakes out.

There’s also an interaction with political polarization in the activities that the parties might choose to do, or not do, because of the fear of backlash, either in their party or among the electorate.

This all seems worth studying from a political science perspective. Back in 2012, Justin Gross, Cosma Shalizi, and I reviewed a book on media bias by political scientist Tim Groseclose. We wrote:

In Left Turn, Groseclose concludes that, in a world without media bias, the average American voter would be positioned at around 25 on a 0-100 scale, where 0 is a right-wing Republican and 100 is a left-wing Democrat. In this world, a balanced media might include some TV networks promoting the view that abortion should be illegal under all circumstances and subject to criminal penalties, whereas others might merely hold that Roe v. Wade is unconstitutional; some media outlets might support outright discrimination against gays whereas others might be neutral on civil unions but oppose gay marriage; and on general politics there might be some newspapers that endorse hard-right Republican candidates (0 on Groseclose’s 0-100 scale) whereas those on the left would endorse positions near those currently held by Senator Olympia Snowe. But instead of this, Groseclose must endure a world where he estimates the average voter is around 50, with all that follows from this, and he attributes this difference to media bias.

With a dozen years of hindsight, we can say that some of this happened and some of it didn’t. The news media landscape really has changed to the extent that far-right views are in the mainstream much more than before. The median opinion hasn’t changed so much, though. It’s just that extreme opinions are more respectable. More recently, political scientist Eunji Kim has argued that reality TV is a major influence on politics and has its own slant. In a study of data from 1869-1928, Gentzkow et al. estimate that “newspapers have a robust positive effect on political participation, with one additional newspaper increasing both presidential and congressional turnout by approximately 0.3 percentage points.” A lot has happened since 1928: newspapers were supplanted by TV which in turn is being taken over by the internet which in turn is being colonized by AI slop. But it makes sense that a steady stream of political news will make it more likely that people will vote. A third of a percentage point is pretty close to zero, which I guess is telling us that, even in the period from 1869-1928, there were many other information sources out there. Including having a friend or neighbor who is politically connected.

Finally, let me emphasize that the above discussion, while focusing on Republican strategy, is not intended to take a partisan stand in either direction. The Republicans have had a more active media strategy than the Democrats in recent years, but I don’t think this implies that the Democrats are more moral or principled on the issue; they’ve just been more constrained in being more vulnerable to criticism from the independent news media.

For the liberals out there who are annoyed at the flood of right-wing propaganda being spewed by Fox, Twitter, etc., or, earlier, right-wing talk shows on the radio, or “copaganda” in popular entertainment, just think of how frustrating it was for urban conservatives to have to wade through left-wing political takes every time they picked up the local free paper to check the band listings. Or to watch big-budget movies and TV shows with a liberal agenda. But I guess that will be changing with these TV networks and movie studios being bought by partisans on the right.

But my point here isn’t really about the story of Jeffrey Epstein, Bill Clinton, Donald Trump, Steve Bannon, Larry Summers, etc etc etc, but rather the role of the press.

A few years ago, I wrote about the decline of journalism and the rise of public relations and, unsurprisingly, it’s got a lot worse since then.

Thirty years ago, newspapers weren’t what they had been–lots of major cities were one-newspaper towns–but there must have been dozens of major independent news organizations in the country, starting with the three TV networks and the big-city dailies, but also including lots of local newspapers and TV stations, alternative newspapers such as the Village Voice which would break stories and cover topics the press would otherwise avoid, news magazines, and various other media outlets that would sometimes break news stories.

Now, there’s the New York Times and . . . not a lot else. The Times does a lot! Almost every day it seems they’re running an investigative piece that required a lot of reporting effort. But the rest of the journalism business seems hollowed out. There’s a lot more opinion writing than there used to be, but lot less regular news. You can get a lot of sports news, but I think that’s more a national aggregation than anything else. Compared to 30 or 50 years ago, you can get updates on sports all over the country; I don’t think there’s more total coverage, it’s just more remotely accessible. But when it comes to news, it just seems that there’s less.

That all impacts the Times and other remaining active news organizations: they’re aware of their role, which gives them an awkward responsibility–if they don’t cover a story, it might not get covered at all–and also they’re not so much pushed by the competition. It’s just a thinner market. You might expect the Washington Post to be all over the Epstein story (although without so much interest in Larry Summers), but instead the Post columnist is reacting to opinions about Summers on Twitter.

And this brings us to campus newspapers. As I discussed last week, the Harvard Crimson has a very good reason to go after Larry Summers: just a few weeks ago he publicly announced his intention to cut them off at the knees for what he called their “moral bankruptcy.” This is a former president of their university who remains well connected, indeed enough so that his disapproval could harm these student reporters seeking careers in business, government, or the news media. Summers’s threat was real! And, to their credit, the Crimson reporters handled it in the way that we would all hope that journalist would respond: by throwing the facts right back in Summers’s face. Student journalists can be fearless.

The scary thing is, student journalism may be one of the few remaining bastions of the independent press. The Columbia Spectator does a lot of reporting too.

Juxtaposing the Washington Post columnist telling people not to report on a major story, with the students at the Harvard Crimson just doing it . . . it’s a stunning contrast. The Crimson is aggressively running story after story, while the columnist doesn’t seem bothered at all that a morally compromised bigshot was trying to intimidate student journalists.

P.S. One positive sign is that sometimes professional news organizations will follow up on student reporting. For example, on 17 Feb 2026 the student newspaper at UC San Diego reported, “Emails released by the Department of Justice indicate that Jeffrey Epstein provided funding for a UC San Diego lab led by Vilayanur Subramanian Ramachandran, director of UCSD’s department of psychology’s Center for Brain and Cognition and emeritus distinguished professor. The DOJ released more than 3 million additional pages of the Epstein files on Jan. 30, in which Ramachandran is named by Deepak Chopra, a lifestyle guru with ties to UCSD. . . . On Sept. 25, 2017, Ramachandran replied to Chopra in an email regarding a study the lab was conducting on an ‘autistic savant who displays telepathy.’ Ramachandran wrote that he does not ‘have problem with [his] lab being funded by Epstein.’ . . . Ramanchandran further wrote that if Chopra’s ‘pal [Epstein] is serious about setting in motion a lab for the study of extraordinary brain potential . . . something like 500,000 to 3 million would get the administrators excited.'”

Then on 8 Mar 2026 the San Diego Union Tribune ran an article, “UCSD professors wanted money to research telepathy. They turned to Jeffrey Epstein,” with subtitle, “‘I don’t have a problem with my lab funded by Epstein,’ one wrote to another.” The article is paywalled so I can’t see if they credited the student newspaper for reporting on this first.

Recently I found this album in the library, Un Avion Sans Elle, by Fred Duval and Nicolaï Pinheiro, d’après Michel Bussi. It was good! I was wondering about Michel Bussi so I googled: he wrote the novel on which the BD was based. Bussi’s a popular writer and, according to wikipedia, il est “Professeur de géographie à l’université de Rouen . . . Il est spécialiste de géographie électorale.”

Wow! That’s kinda close to what I do–except for the bit about writing compelling mystery novels. I don’t think anyone is gonna adapt Red State Blue State into a bande dessinée.

I love the library–you never know what you’re gonna find there.

P.S. Also, since I’m giving out recommendations, Le Château des Animaux is excellent. It will be especially interesting to the political scientists in the audience. We’re currently waiting for the fourth and final volume to come out. Also, I finished The Nice House on the Lake: it was compelling but it was hard for me to keep all the characters in mind and remember who was who. Also a good one by Hervé Bourhis about Paul McCartney. And De Cape et de Mots, which we read a couple years ago. And a bunch of others.

I too often see people saying their model is converging or not converging. Sure, if you are doing iterative model building as part of your Bayesian workflow you could say that that iterative process eventually converges to the final model, but it seems people are actually talking about whether the inference algorithm is converging.

A Bayesian model describes a joint distribution of data and parameters. If we condition on observed data, we get the posterior distribution. We often use iterative inference algorithms to make posterior inference. If the inference algorithm doesn’t converge, the convergence problems don’t depend only on the model, but on the model, parameterisation, and the data, which together determine the geometry of the posterior. The same model and different parameterisation or data lead to different posterior geometry. For the same posterior, different iterative algorithms or algorithm choices can also lead to different convergence problems. (We have several exmples of iterative inference algorithm convergence problems in the soon to appear Bayesian workflow book)

If you want someone to help with possible inference convergence problems, it is not sufficient to tell which model you have, but you also need to tell about the parameterisation, data, and algorithm. Stop talking about models (not) converging (unless doing iterative model building) and talk about the inference algorithm (not) converging, as it is more accurate and implies dependency on the posterior and algorithm.

This post is by Lizzie, on behalf of my colleague Will Pearse in the UK about a cool forecasting competition. It’s different in some ways than the cherry competition, but the same in other ways, predict blooms, predict beds… Only predict! (Only connect!)

Every four hours of delay in admitting patients from an emergency department adds up to roughly 25 potentially avoidable deaths per month (Howlett et al. 2026). We’re running a contest where you can forecast those risks 1–10 days ahead so UK hospitals can take action.

The data in the contest are all real healthcare data. The data come from the Bristol NHS system, with 220 variables ranging from daily counts to 15-minute feeds (e.g., bed occupancy, ambulance waiting times). You build a model in R or Python, submit by June 5, and forecasts are judged by mean squared error over short- and medium-term horizons.

The winning model(s) will be implemented in Bristol’s live system to flag emerging system pressure and so this is, literally, a chance for you to save lives with your stats know-how.

If you like this sort of thing (…and of course you do, you’re reading this blog…) then please sign up for the SPHERE-PPL mailing list. We (SPHERE-PPL) are organising a series of forecasting challenges just like this one, and we can tell you more about them.

Press release: https://www.imperial.ac.uk/news/articles/natural-sciences/life-sciences/2026/researchers-launch-forecasting-challenge-to-help-predict-severe-patient-harm-in-nhs-hospitals/

GitHub repo to enter: https://github.com/SPHERE-PPL/NHS-EAD-forecast

Paul Alper points us to an online video promoting something called Lotto Champ, which describes itself as “a cutting-edge tool designed to help lottery players make more informed choices. By analyzing data and patterns, LottoChamp offers personalized number suggestions, providing a smarter approach to playing. It takes the guesswork out of the lottery experience, enabling a more confident and strategic play.” It provides “tailored sets of numbers” and costs a mere $197.

As a statistician, this claim of effectiveness surprised me. Most obviously, if the lottery is run well, the numbers are random so there’s no way to predict them, and even if there’s some flaw in the randomization, the vig of the lottery is huge that it’s highly implausible that any edge would be enough to make you money in expectation. The next obvious point is that the lottery numbers are the same for everyone, so the concept of a “personalized number suggestion” makes no sense.

And, yeah, yeah, I know what you’re gonna say: if it’s the powerball then at least you can avoid commonly picked numbers (hint: avoid numbers between 1 and 31) so that, if you do win, you’re less likely to split the prize. I was still under the impression that it was still a losing bet, and that it would be bad news bad news if you’re such an addict that you’d even consider spending $197 on this. And, again, I thought that “tailored sets of numbers” made no sense.

But what do I know? I’m just a simple country statistician, I’m no lottery expert. The real experts are on the internet.

So I googled *Lotto Champ* to see what came up, and I saw some things like this:

and this:

OK, so some ads and some reviews. The reviews are unusual in that they use advertising promotional language and are full of links to the site, encouraging you to buy. I guess these reviewers are really excited about the product! There’s no way that these could be fake reviews, inserted just to suck customers in. Doing such a thing would be immoral, and there’s no reason to think that someone selling lottery tips for $197 would be immoral. On the contrary, they’re trying to help people!

Then this:

Hey, thanks, Google, for providing this valuable information!

Ahhh, but scroll down on the page and you’ll find some warnings:

Now I’m concerned. I’ll first click on the site from “ACCESS newswire.” This seems like a legitimate source, and here’s what they say:

Lotto Champ Reviews & Complaints 2025: What You Need to Know Before Buying This AI Software

AUSTIN, TX / ACCESS Newswire / June 9, 2025 / Lottery games are always considered a game of chance that cannot be rigged. But have you considered the fact that since it is a game made by humans, it will have a set pattern that can be cracked? Well, this realization was what led to the creation of the Lotto Champ lottery prediction tool. This is an AI-powered software built to eliminate the guesswork from the lottery games.

Whew! For a moment there I was concerned that this lottery numbers thing might be a scam. It’s a relief to learn from this independent news source that it will “eliminate the guesswork from the lottery games.” huh? Sounds like a great deal–my only question is why they only charge $197 for this wonderful innovation? My guess is that the people at Lotto Champ are just very nice, and they want ordinary people to be able to make money too. The news story continues:

The basis of the program is that it is powered by artificial intelligence tools that have access to a huge collection of publicly available data on the lottery games of the country. This huge dataset that has historical winning patterns, ongoing lottery games, and future games will help the AI choose the best game with the biggest winning odds and payouts for the customer.

Wow–cool!

The article continues:

Understanding how the Lotto Champ software works is simple. Before getting deep into that . . .

The article goes on for a few zillion more paragraphs without ever “getting into that.” I guess the authors of the review were so excited that they forgot to put in their simple-yet-deep explanation of how it works. But, don’t worry, I’m already convinced that it works. And for only $197, I don’t need to know how it works, I can just live off the steady stream of money it will provide me:

The main aim of this Lotto Champ review was to present a comprehensive analysis of this new lottery prediction tool that has garnered all the attention. The lottery game is one of the most sought-after and celebrated games in the gambling world. People bet their chances on the belief that they are lucky and will get a bigger win.

Even though this is the basis of the lottery game, this lottery prediction tool is introduced for players who want to make their wins more consistent and get a stable income from this game.

It’s good that the Lotto Champ system really works. Because if it didn’t–if lottery numbers really were indistinguishable from random and unpredictable in any useful way, then it would just be evil to feed the deluded fantasies of gambling addicts. It would have the potential to ruin people’s lives. But we don’t have to worry about that.

Oh, and what about the other link above? It’s from www.msn.com and it’s entitled, “Lotto Champ Reviews (SCAM WARNING!!).” SCAM Warning . . . that’s pretty scary? And msn.com is a legitimate news site. From wikipedia:

MSN is a web portal and related collection of Internet services and apps provided by Microsoft. The main home page provides news, weather, sports, finance and other content curated from hundreds of different sources that Microsoft has partnered with.

OK, you may not be the biggest fan of Word and Excel, but Microsoft is a mainstream institution, and it doesn’t seem unreasonable that they’d have a webpage warning you off some internet scam.

So I better click on the link and go to msn.com to see the full story, whose full title is “Lotto Champ Reviews (SCAM WARNING!!) Can This AI-Powered Software Help To Win Lottery Multiple Times?” Here’s what msn.com reports:

Lotto Champ is an advanced AI-powered software that is specifically developed to increase the chances of winning lotteries. It provides a more strategic approach and leverages AI-powered technology to generate numbers that have a higher probability of winning.

It analyses past lottery results based on data-driven insights and tends to predict the best possible combinations. It optimizes your selections and enhances your odds compared to traditional random choices. Lotto Champ provides an intelligent analysis of the historical data and takes the guesswork out of the equation.

Hey, thanks, Bill Gates! I was worried that Lotto Champ was a scam. I’m glad you cleared this up. Now that a reputable source has confirmed that it’s cool, I can confidently send them my $197.

It’s a good thing that Lotto Champ is providing a legitimate and valuable service, otherwise Microsoft would be promoting a scam on its own branded website (no, this article is not labeled as an advertisement, and its url begins with innocuous root, https://www.msn.com/en-in/news/techandscience/).

But now I’m still kind of concerned so I google “lotto champ scam,” which reveals a pile of videos and text links saying how wonderful it all is.

These Lotto Champ people must have done an awesome job at search engine optimization. Good for them! They’re providing a valuable service for a mere $197. It’s the least they can do to spread the work on the internet, especially to those skeptics who might naively think that a lottery-promotion system is a scam.

And good job, Google! You’re not just promoting a wonderful scheme to win the lottery, I guess you also made some money selling slots on your search pages. I guess that’s why your motto is “Don’t be evil”: you’re helping people and making money at the same time! What could be moral than that.

On the other hand, if you’re a potential customer who’s lucky enough to google “Is Lotto Champ a scam,” the first link is this no-holds-barred youtube video by Jordan Liles shooting them down. What a party pooper! C’mon, Jordan Liles, you’re just jealous of all those people who live a comfortable life playing the lottery–and for a mere $197 investment! You can make all the videos you want; I don’t care.

I also came across a review titled “Lotto Champ Reviews (EXPOSED)” at morningstar.com. Hey, Morningstar’s a reputable company too! So I was scared about the exposé. But, not to worry, click through and read the article and it’s all about how great the system is. It even includes links so you can go buy it directly! Good for you, Morningstar! Like Google, you’re helping the ordinary Joe and you’re also taking sweet sweet advertising dollars. Again, I breathe a sigh of relief that Lotto Champ really does what it says. Otherwise mighty Morningstar would be enabling gambling addiction.

As with the other cases, the Morningstar review is not labeled as an advertisement. Indeed, it’s in the “Market News – Accesswire” section of their website. Market News from Morningstar . . . that sounds pretty legit!

Google also provides an “AI overview” informing us that Lotto Champ is “Backed by data: The software uses AI to analyze decades of past draw data to identify patterns and trends. This gives users a strategic approach instead of relying on luck or superstition.” Bafflingly, the AI review also says “The software cannot alter the fundamental randomness of a lottery drawing, and the ultimate outcome still depends on chance.”

That sounds like a contradiction to me! First it says it does not rely on luck, then it says the outcome depends on chance. Hey, that’s the Markov model for ya.

So, yeah, thanks Google!

And some other sites popped up associated with Apple and Yahoo, two more recognizable brand names have positive reviews (including helpful links to where you can spend your $197) on their webpages.

Before seeing all these entirely neutral third-party reviews, I was suspicious of the idea of an AI that could pick personal lottery numbers for you. But given that Google, Apple, Yahoo, Morningstar, and Microsoft all endorse it, I’m convinced.

Also, if Lotto Champ were really a scam, I’m sure the government would’ve already cracked down on them, just as they’ve already prosecuted cryptocurrency frauds, promoters of dangerous anti-vax misinformation, the mayor from some city, I can’t remember where, who was allegedly taking bribes from a middle eastern government, etc. One thing we know about the U.S. government is that they have no tolerance for crime and corruption.

In better times I’d say the government should crack down on this. Not just the lottery crap but the corruption of Google, Apple, Yahoo, Morningstar, MSN, etc etc, which are either actively promoting it or else are passively letting themselves be manipulated.

Selling lottery numbers is already a scam. But setting up a network of fake reviews with the implicit complicity of some of the world’s richest corporations, that takes it to the next level of evil.

I’m sure the internet is full of such things. I just hadn’t been aware.

In the meantime, remember that Reputation is a two-way street. If I were foolish enough to believe that Lotto Champ is a scam, I don’t think I’d ever trust anything on msn.com or Morningstar Market News or whatever. Or Yahoo, either, but I actually hadn’t been aware that Yahoo still exists. Fortunately, I have full trust in Lotto Champ, msn.com, and Morningstar. Lotto Champ deserves my $197, and Google, Microsoft, and Morningstar deserve every dollar that is given to them to run these valuable and informative reviews, and Google is wise to run its server farms 24/7 and burn up whatever remaining coal we have in the world in order to produce these very helpful AI overviews.

What I’d really like is for some rich guys to buy Reddit, Stack Overflow, and Wikipedia and convert them to sites that are as useful as those provided by Google, Microsoft, and Morningstar.

P.S. The above post is not intended to provide any financial advice. Spend any $197 at your own risk. Remember that $197 can be converted into 137 Jamaican beef patties, 1/85 of a conference featuring Gray Davis, Grover Norquist, and a rabbi, or 2 1/2 dinners of a soggy burger, sad-looking fries, and a quart of airport whisky. Spend your money wisely, kids!

Aki explains options here:

Thomas Lumley’s blog post and coauthored paper Iparragirre et al. (2023) explore CV using “replicate weights” ideas:

Replicate weight methods split the sample into partially independent subsamples, and modify the weights so that each subsample replicates the original sample. They are usually used for variance estimation, but Iparragirre et al. (2023) consider them for out-of-sample error estimation.

Although I was rooting for BRR (Balanced Repeated Replication) because I work at Blue Rose Research, a better method seems to be design-based cross validation (dCV), depicted in their Figure 1(d):

Each dot is a PSU (primary sampling unit), which can be an individual but is often a group/cluster of individuals. Each color is a stratum. dCV is the usual K-fold CV but:

1. keep PSUs together within a fold
2. reject a split if a whole stratum falls into one fold
3. modify the weights so that each subsample replicates the original sample

The first mirrors Aki’s LOGO (leave-one-group-out) CV above. The third is an idea from replicate weights.

The first two seem useful for nonprobability samples as well ? Suppose there is structure in the data and our predictive task is to predict for new schools (PSU-like) but existing states (strata-like). Is there a good reference for this ?

This post is by Lizzie

Recent reports of the toxic workplace culture of the Uber-fancy Copenhagen restaurant Noma confused me, initially. I thought we already knew this place and its lead chef, Redzepi, was toxic and it had closed. I eventually figured it out–these were new reports and involved more than reported before—and was surprised to find that Noma in Copenhagen closed but Redzepi came back and was ushering in folks in black SUVs past inflatable mushrooms to dine in LA (and other pop-ups).

This got me thinking of some of the parallels with academia. I think/hope we don’t have quite the level of bullying/abuse described in the recent NY Times investigation but we do seem to have:

• A culture of entree via unpaid internships (volunteering) where you get the worst job.
• A tension between what level of bullying is `part of the process’ versus too much.
• Some media-savvy enough folks who may try to control the narrative.

I somehow have never done a post on how lame I find our ‘volunteer’ culture so I should do that. For now I will just say it’s a great way to reduce equity and has repeatedly been called out in conservation  (one of the worst offenders).

For the last one, I turn to the case of Tom Crowther (which also touches on the second point) that has had me wondering for some time: how much media outreach is too much for a lab? As you may or may not know, Tom Crowther was a assistant professor at ETH Zurich who published a series of high profile papers (including perhaps the most famous one suggesting we could nip anthropogenic climate change in the bud by just planting a lot more trees, including in lots of places that don’t have trees for good reason, like the savannas of Southern Africa where the carbon-rich grasslands are actually an excellent carbon sink, let alone a bit of a biodiversity hotspot) and was covered breathlessly in news and related bits of the journals of Science and Nature for his work (ETH went full Lance Armstrong effect on him).

What was less covered, was that recently he did not receive tenure from ETH after a series of bullying and harassment complaints were filed against him. I was fascinated by how little media there was on this and still wondering why. Given how much he was covered in the news, given the movies he was making, this seemed a story worth covering. I have a couple hypotheses for why.

Hypothesis one I am calling `isn’t it all always so confusing?’ It seemed messy (but isn’t that always the case? Hence my hypothesis name). The reports came some time after they reportedly happened, they appeared via a news story, not through formal ETH processes, and a number of lab members said they were very happy in the lab and urged anyone discussing this to stop.

Another (perhaps related) hypothesis, is that Swiss laws seemed to squash part of the story. The first I heard of the reports were from a story ran in Tages Anzeiger about a rising star professor at ETH who had bullied members of his lab. The newspaper had reports from multiple lab members they planned to publish, but were blocked because — apparently (as best I understand) — in Switzerland if you hear something will be published about you, you can try to block it (like a preemptive libel law, I gather). Why the accusers did not just go to a US, UK or some other nationality media outlet I never understood (that would get around the law, no?). Anyway, this all led to the story dripping out, to where everyone seemed to know one of the accusations included Crowther unzipping at a party and accosting one of his lab members with his privates, and it’s all filmed so we don’t have to wonder if this happened (later this was all detailed in the pro-Crowther version of the story in NZZ).

[Can I just take a short moment here to say that I am still confused by the number of colleagues who have said to me any of the following: Oh, but it was probably all in good fun! Can’t people take a prank these days? Maybe ETH should have told him not to do stuff like that; he needed more guidance. But should you really lose your job for this [at a government funded university where you have massive control, support and responsibility]?]

And then my last hypothesis (definitely perhaps related) is that the Crowther lab’s media branch semi-successfully controlled the story. The lab had an entire media branch and ETH found in its report that funds were used “not in accordance with ETH regulations. Specifically, funds were allegedly used for services such as crisis communication, legal advice and marketing.” It others words, he used ETH funds to combat the media reports surrounding the allegations against him. He admits this in the report and “explains that this was part of routine research communications before the recent media crisis caused by the media coverage at the end of August 2024. When the recent media crisis arose, he regarded the crisis communication services as a standard aspect of public relations.”

This last hypothesis has really made wonder how much media is too much media for an academic lab? As a climate change scientist, I have been raised on the dogma that we need to communicate our research. We need to help counter disinformation, get the word out. Fossil fuel companies have been pouring money into the disinformation campaign for as long as they have been accurately forecasting rising temperatures due to their fossil fuels (or pushing us to work on our own personal carbon footprints). Isn’t the massive branch of your lab doing media just taking this to the appropriate level? Or, is the outcome then that you control the media and you control the world? You become a mini-Rupert Murdoch and believe you deserve all that control?

And what happens then? Perhaps there is one more parallel between this story and Redzepi at Noma. Redzepi stepped down from Noma and many funders pulled out. But already wonderings abound about whether he is just lying low, waiting for the `squall’ to break. How long do you have to lay low? Crowther has resurfaced at KAUST University, which has hired him as a professor and has launched a new institute.

There’s an opportunity with post-Redzepi Noma to create something new. There’s also a chance he just comes back and nothing changes. The funders have a role in that outcome, but so do the diners. Do they come back? Or do they perhaps also want something new?

In general there are three ways of attacking a statistical problem:

1. Probability calculation. Set up a probability model and crank it through. This will require a bunch of assumptions, and you’ll also need to set parameters in your model to reasonable values.

2. Direct empirical calculation. This will work if you have enough data, and if these data are not subject to selection.

3. Statistical modeling. Kind of like method 1 above, except that you fit (“learn”) the parameters from the data; as a result you can fit a more complicated model. I include machine learning in this category too.

In statistics classes, we focus on method 3. No surprise, right? Statistical modeling encompasses probability calculation and direct empirical calculation; indeed methods 1 and 3 can be viewed as special cases of method 3. Method 1 is method 3 but with a simple model and a crude method of setting the parameters. Method 2 is method 3 but with a simple model assuming stationarity in all directions.

So, yeah, statistical modeling. There’s a reason my colleagues and I have written multiple books on the topic, spent innumerable person-hours developing and using Stan, etc.

But . . . it’s good to know about methods 1 and 2 as well.

Why? Four reasons.

First, methods 1 and 2 are simpler, and sometimes they work just fine.

Second, even beyond simplicity, methods 1 and 2 have fewer requirements. Method 1 does not require the data (which is how we were able to get a good answer to that football question in the first place), and method 2 does not require a data-generation model. In contrast, method 3 requires data and a model.

Third, even when estimates based on methods 1 and 2 are seriously flawed, they can be useful starting points and comparison points to better approaches. Indeed, sometimes when a probability calculation gives a ridiculous result, this can be useful in developing intuition. For example, the notorious calculation of a probability of a tied election as 10^-90 came from an inappropriate application of a binomial-distribution model, which motivates the development of models for statistical dependence among voters, while the failure of straight-up empirical estimates motivates models that combine probability modeling and empirics.

Fourth, when people are informally estimating things, they’re often using some version of method 1 or 2. Which is fine! But then I think it’s important to be aware of what you’re doing and to ask, What is the probability model you are assuming, or What is the frequency calculation you are making?

Those four reasons–that’s the point of yesterday’s post.

I’m looking for a postdoc to help organize Bayesian Data Analysis course (200 students) and to do research on Bayesian workflow at Aalto University, Finland. Background in Bayesian topics needed. Up to five year contract possible.

Job is at Aalto University, Espoo (15min Metro ride from Helsinki center), Finland.

Starting time is flexible. There is no specific deadline, but it is better to apply soon. Apply by email (see contact information). Include CV and explanation of your skills and experience related to teaching and research.

Salary and occupational benefits are better than in academia in many other countries, and living costs are moderate (not the cheapest country but also less expensive than big cities). Finland is the world happiest country (9th time in a row)