How the LLM finally killed off science

And why this (unfortunately) is (kind of) a good thing

I was a rookie when I got into academia. Unjaded and wide-eyed. Had real high expectations of the people around me, I thought I was in the company of a select few passionate idealists commited to the truth no matter what the cost.

That didn’t turn out to be the case.

Granted, some of them were indeed stellar, and I probably got a somewhat skewed perspective from first being raised up in this old-school little sub-department where people actually did philosophy for God’s and philosophy’s sake and just preferred that the rest of the world stayed out of their way.

But when I was set to roam free in the outside world of academia and came up against this massive rot of ideology production, status-jockeying and cronyism, where the few who actually persisted in spite of all this were finally absorbed by the private sector (often enough the military-industrial complex), let’s just say my perspectives had to shift somewhat.

I mean, I knew how Feyerabend for good reason associated academia with the Nazi death camps, and I was well aware of the lines Foucault drew between the university, the panopticon and the mental asylum, but I’d thought those were just extreme examples. Illustrations of important associations between the key ideas of the modern project and authoritarianism, imperialism and colonization, I didn’t think that this same atmosphere literally thrived in the midst of our contemporary institutions.

But I was quite wrong about that.

Accurate scholarship can
Unearth the whole offence
From Luther until now
That has driven a culture mad,
Find what occurred at Linz,
What huge imago made
A psychopathic god:
I and the public know
What all schoolchildren learn,
Those to whom evil is done
Do evil in return.

(Auden)

So things were evidently quite bad way back in 1960s, which prompted some disaffected voices like those above to emphasize the crucial connections between academia and the prison, as well as to point out that science as an institution had somehow been turned into an overarching ideological apparatus that ought immediately to be separated from the state. An even more perfect one than could be provided through the political appropriation of religion in post-Reformation era, since there’s always something indomitable and unpredictable about the spiritual.

Science, nonetheless modelled after sola scriptura and the secularized Divine Right of Kings (human reason provides political legitimacy through objective knowledge of the world, but is also the unique prerogative of Science and cannot be questioned), had become more important as a fount of authority than anything else, which naturally also impeded its own legitimacy as an organized quest for knowledge, truth and meaning.

This was obvious to any discerning onlooker almost three-quarters of a century ago. Modern science had been set up as the foundation of imperial political legitimacy and social cohesion which was not only an incredibly dangerous mix, but also immediately undermined any merits and legitimate function of science as a collective endeavour.

When it’s about maintaining a power structure, evidence and theoretical coherence become secondary factors at best — and are even likely to be suppressed since they can always in principle provide leverage for challenging the established paradigm.

Modern orthodoxy.

And now, in the brave new world of the large language model, we don’t have to argue about these things anymore. Science is dead and buried and gone and there’s just nothing more to say about it.

Independent leverage for challenging the established paradigm? Forget it.

You can just toss out any sort of self-correcting capability of this legacy Western institution for finding truth and improving knowledge, that’s now a thing of the past.

Because as I’m sure many of you have already heard, just last week we got a violently clear indication of just how deep the damage has already gotten from the widespread use of these glorified nonsense generators.

A survey revealed that researchers at a number of prestigious institutions systematically gamed their papers by hiding secret prompts directed towards a prospective chatGPT “peer review” to render the outcome favorable. Technically a kind of prompt injection attack. A really dumb one.

And this is of course only the tip of the iceberg. These are just examples of the stupidest midwit crop of “researchers” that just stick a fucking white-tinted all-caps message right in the middle of their body text like abject morons.

Because an LLM prompt is essentially a command fed into a program, there are literally hundreds of ways of encoding hidden directives that are undetectable to a casual human observer into something like a research paper. And I guarantee that this has been an established practice in STEM-related publications for the last three years to such an extent that this amounts to a new reproducibility crisis all on its own.

Just look at this garbage.

And how do we know this? Because everyone and his dog are now “writing” their papers using an LLM, just like many of us predicted would happen several years ago.

The cleverly entitled article “Delving into LLM-assisted writing in biomedical publications through excess vocabulary” (yes, we see what you did there) from last week showcases a bit of this massive, oozing rot beneath the surface of academic publishing, indicating that millions of papers from the last couple of years are generated through the use of LLM tools. And this is certainly a low-ball figure, because their approach is only likely to flag the laziest of the perpetrators who make absolutely no effort to hide their tracks.

We’re at a place where I’m genuinely impressed by the select few students who actually make the effort to clean up their AI-generated slop so it at least looks somewhat like human output, even though both of us know it’s fake.

And now, why is all of this in any way a good thing?

Because it clearly shows that the LLM only killed what was already basically a corpse. This could never have happened unless quality of content output in academic publishing for a long time has been an afterthought at best. Those without blinders have known this at least since the wonderful shitshow of the covid debacle with the masking studies probably taking the cake, of course, but all of this really brings home the point that quality really doesn’t matter very much in the framework of academic research.

Imagine a wastewater engineer trying to pull something like this off, using the “Gemini co-pilot” to design the sewage system of a housing block. He can’t, because shit would literally start flooding people’s apartments, and nobody would have the slightest idea what the problem was or how to even begin fixing it.

“Scientists” and “academic researchers” can, however — because the whole fucking enterprise is now predominantly ideology production and marketing.

And let me reiterate the whole map of how violently stupid all of this is. First of all, you have a whole generation of students right now whose ability to think independently and critically are being rotted away by these tools that help them get through university by pretending to learn and to produce meaningful work. These people will then join the ranks of “scientists” and “researchers” who are already happily forging their own research output by using LLMs to simulate something that kind of looks and feels genuine if you squint, but whose quality at best will be that of derivative garbage.

What’s more, these midwits are already flooding the journal ecosystem with their mass-produced counterfeit research, which even before all of this was in really bad shape in terms of peer review and quality control, so journals will be massively incentivized to simulate the reviews as well.

Academia.org recently sent us this marketing e-mail happily letting us know how this wonderful AI had reviewed a recent paper of ours, and since it was so fantastically helpful according to the fake testimonials, why shouldn’t we also avail ourselves of this magical “productivity” tool?

You see the feedback loop? You end up with a system where scholars progressively lose their ability to do independent research, and where academic output becomes dominated by increasingly better-looking but ultimately useless slop, with an LLM in charge of quality control.

And since both generative AI and contemporary academia are tools designed for marketing and ideology production, this unholy alliance may well persist for much longer than I’d care to hope. I wish I could say that this obvious collapse of academic and scientific legitimacy would render the whole project useless in terms of the propaganda it’s now more or less reduced to, but that’s far from certain.

As a complex institution design to produce and sustain myth, supported by the spectacular AI and it’s emotionally gratifying relational avatar, it could very well endure for a long time, surviving off of its vast libraries of accumulated genuine knowledge even if it no longer adds much.

I see these students of mine coming into the university for the first time, some enthusiastic as I once was, eager to learn or to make some kind of lasting positive change in the world. This one kid looked me up for advice because he wants to do philosophy and has the intention to really get the basics right from the start.

He’s in trouble. I guess I could try the Fight Club approach. Have him stand on the porch for two nights, and if he’s still there the next morning (and has two pairs of black shirts and two pairs of black boots), then I guess I’m going to have to let him in.

But I don’t want to tell him to just get a marketable degree and then stay the fuck away from academia. That it’s most likely going to be a waste of his time and his talents. I want to be able to look him in the eye and tell him that there is a genuine quest for meaning, truth and beauty here that can be found nowhere else, and which he can be a part of if he works hard at it and keeps on asking questions no matter what.

But that would be a lie. This just isn’t the case any longer.

We’re going to have to start over. All the way from the beginning.

To be sure, it’s only after we’ve lost everything that we’re free to do anything.