Will AI make science's reproducibility crisis worse?

Obviously — but is there room for optimism in the long run?

The question “Will AI make the reproducibility crisis worse?” has an obvious answer: absolutely.

This isn’t a daring prediction so much as a foregone conclusion. Consider the analogous question of whether AI will make fake news worse. Duh.

Science is no different — it has it’s unique quirks but the general forces at play are the same.

So the better question is how to stem the tide in the short term and whether we’ll land somewhere far better on the other side. Those answers are far less obvious.

Science works backwards

One thing non-scientists often fail to realize is that science works from belief to data, not the other way around. People imagine scientists surfacing data in a totally unbiased way, without any preconceived notions of what it will show them.

With few exceptions, it is the opposite. A scientist comes up with a hypothesis and then works to prove it.

That should give you a feeling for why LLMs are such a powerful potential accelerant of irreproducible science. LLMs are notoriously easy to bias — frequently making up citations or entirely inventing the data underlying their claims.

Lowering the friction to write papers is also dangerous. I wrote an article for the Scholarly Kitchen during the peak of the COVID preprint “infodemic” about what we saw on the front lines at ResearchGate. Especially when it comes to lit review articles, the ability for scientists to build reputation by creating AI-generated papers could quickly drown any fact checking systems we have.

Incentives, Incentives, Incentives

While difficult to fix, the reproducibility crisis has never been hard to understand — scientists need to build proof of work every few years to achieve the next milestone, but it takes a few years to run a proper experiment.

When I was in the lab, I spent 2 years pretty much full time on a single experiment. I was lucky to get some good data and a few publications out of it, but suppose I hadn’t.

That, in essence, is the root cause of the reproducibility crisis.

Being a junior scientist is like getting a coin to flip and being told that you only get to flip it once or twice before deciding whether to publish which way it’s weighted. If you don’t publish anything, your career is shot. And every few years, you repeat the process.

Replacing scientists

At the risk of ruffling some features, I will say the obvious out loud – junior scientists who spend 95% of their time doing grunt work are going to be replaced by robots.

This may actually be the most optimistic take. For one thing, being a junior researcher kinda sucks. I spent basically two years dissecting mouse brains and developing back problems from hunching over the cryostat all day (being tall in the lab is not fun).

And for reproducibility, it seems obviously better if a given scientist more closely resembles a principle investigator (managing a lot of experiments in parallel) than a PhD student. To go back to the coin analogy, if you get to flip the coin a hundred times, then the pressure to publish half truths is much lower.

In the distant future, a given scientist might even resemble an organization like Arcadia — not just doing a lot of experiments in parallel but bringing discoveries closer to market. In that world the incentives to publish false data is basically zero because it only makes it harder for you to produce impactful products. Why spoil your own soup?

Whether there will be more or fewer total scientists in that world is very similar to arguing about whether there will be more or fewer total engineers as AI gets much better at code. If individual leverage goes up, do we get fewer individuals or a much larger pie? It’s an interesting question that smart people are asking about society in general, which is also why it is out of scope of this article.

Perhaps academic publishing can’t save itself

If you follow my logic here, I think it’s reasonable to conclude that AI can solve the reproducibility crisis but only when it can dramatically increase the leverage of any given scientist (unless all scientists can go work for organizations like Arcadia, but that would require a large increase in billionaires looking to fund basic research).

Massive increases in speed and efficiency aren’t coming to academia via a lot of the AI for academia we have so far, such as publication search or literature review — though those products are awesome and solve real problems for scientists.

The simple fact is that scientists spend the bulk of their time in the lab, and making that 100X more efficient is much more dependent on laboratory automation. And even that is a fairly wicked problem especially in biology — you can’t change the time it takes to raise a mouse, for example.

So in the short term, I’d expect the situation to get worse. We can build bandaids — better peer review, LLMs for science with strong safeguards on source data, and so on. And as a society we should celebrate and encourage many more experiments like Arcadia that aim to offer alternatives to the traditional academic path. But it’s really in the long term as we manage to automate the grunt work out of science that we can truly address the incentive issue.

As always, I’d love to hear if you see things differently. And I’m always happy to connect with others working to use AI to make the system of science better.