Big Brother is having a baby
I’m never gonna learn Cantonese, but I’ve kind of picked up a bit of the prosody. The typical cadence, rhythm, intonation and that stuff. And I sometimes blurt out random nonsense attempting to channel somebody’s Hong Kong uncle smoking a cigarette while angrily frying eggs in the kitchen for the sole purpose of getting a rise out of this unfortunate woman I’m sharing a space with.
Surprisingly often, I manage to say something weird in Chinese, and this morning, I apparently woke her up successfully ranting about “big brother giving birth” (and "shrimp who overthink”).
This struck me as an appropriate coincidence, since these last few days, I’ve been preoccupied with the ideological impact of this increasingly ubiquitous “AI co-pilot” that now apparently has the gall to pretend to criticize any halfway unorthodox google search I happen to produce.
So this “AI overview”, the retarded magic 8-ball nonsense generator, is now popping up with some inane politically correct soundbite to steer you in the preferred direction like an annoying baby brother who’s memorized the rulebook without having any idea what any of it actually means.
And it’s genius. You recall how the panopticon was not actually about surveillance, but about producing the experience of being watched whether or not there was a real guard in the observation room?
Foreign information operations on social media platforms pose significant risks to democratic societies. With the rise of Artificial Intelligence (AI), this threat is likely to intensify, potentially overwhelming human defenders. To achieve the necessary scale and tempo to defend against these threats, utilizing AI as part of the solution seems inevitable. Although there has been a significant debate on AI in Lethal Autonomous Weapon Systems (LAWS), it is equally likely that AI will be widely used in information operations for defensive and offensive objectives.
van Diggelen, J. et al. (2025). “Designing AI-Enabled Countermeasures to Cognitive Warfare”. https://arxiv.org/pdf/2504.11486
This hyperreal simulacrum of an intelligent response to your queries is just the upgraded version of all this.
It hijacks our deeply human response to the seeming presence of another agent in the social arena, in a way that eliminates the need for actual surveillance, to produce micro-nudging in terms of politically or ideologically “problematic” ideas. The system doesn’t need an omnipresent network of cameras or a detailed analysis of your every move or utterance to reliably produce not only the desired behaviours, but to promote orthodox thinking.
Think about how we feel obligated to thank chatGPT for its helpful assistance.
Even somebody like myself who loathes the technology, would like it uprooted, pelted with garbage and burned at the stake, and who knows more than most how absurd the notion of real artificial agency is, I still feel an almost irrepressible urge to be polite and friendly to the gpt simulation since, well, it's kinda nice to me and it would be rude not to.
Our human nature interprets this thing as another agent in the social sphere, and when the AI dismisses the implied unorthodoxy in our google searches, we perceive this as peer rejection.
There's no need for actual surveillance if you can establish an omnipresent experience of being reviewed or scrutinized by a digital pseudo-agent that triggers a social and relational response in the human beings subjected to it.
You don't need a social credit system or a global network of surveillance equipment if you can infiltrate people's basic interface to the digital with this co-pilot bullshit. You can totally dominate the emerging cognitive shadow war by poisoning the well like this, by wedging your propaganda filter between the human being and the digital sphere, which by now pretty much is the predominant source of information for anything whatsoever.
As part of the NATO Warfighting Capstone Concept and using the Warfare Development Agenda as a framework for the delivery, Headquarters Supreme Allied Commander Transformation (HQ SACT) initiated the development of a Cognitive Warfare Concept in 2021. The concept is part of the Warfare Development Imperative of cognitive superiority. The aim of the concept is to seize the initiative in the cognitive dimension through enabling a shared understanding and appreciation of the dimension looking 10 to 20 years into the future. This needs to be accomplished through defensive and proactive measures that ensure the Alliance's protection and enhance our cognitive processes.
…
Unlike psychological operations, cognitive activities are not directed at our conscious mind, but at our subconscious mind, the main drivers of our behaviour: emotions. This takes place through hyper-personalized targeting integrating and exploiting neuroscience, biotechnology, information and cognitive techniques (NBIC), mainly using social media and digital networks for neuro-profiling and targeting individuals.
We need to realize that individuals are at the centre of all military operations and strategic-political decision-making. Although they often sound like ideas from a science-fiction film, cognitive attacks are not science fiction anymore. They are taking place already now, and these attacks will continue to become more sophisticated. Several countries are developing NBIC capabilities and collecting data for use in targeting the cognitive dimension. These activities are supported by aspects such as datamining and data analytics, and are further combined with artificial intelligence.
van der Klauuw, C. (2023). “Cognitive Warfare”. The Three Swords. https://www.jwc.nato.int/wp-content/uploads/2024/12/selects-klaauw.pdf
It’s being marketed as this valiant effort towards “psychological defence” against shady foreign influence, but we all know it’s probably not the Yemeni rebels or the PRC that have the wherewithal to effectively leverage this technology against the unwashed masses of the West. It’s not like the Big Five tech companies are Russian, or the dominant mass media companies are controlled by embattled socialist regimes in Africa.
The cognitive warfare that both of these NATO publications I’m quoting here are promoting is about targeting their own population, just like how propaganda always mainly was about maintaining the consent of the human resources in your own immediate territory.
So Big Brother is having a baby. And people are lining up to pay for his inane nudging wedging itself into every fucking tool they use to access and reproduce information, because this apparently helps secure “productivity” and “qualified traffic”.
“Overthinking shrimp”, concludes my internal random Cantonese generator.
The amount of money that's gone into this, into 'how to nudge the people into a mental construct and have them stay there' is probably more than some nations annual GDP. As you noted, just the idea of being watched is enough to shift behavior. I've seen it in others and I've watched myself metaphorically shrink when surrounded by untold and unseen cameras in downtown London. I've even edited my own speech knowing Alexa is in the room. And I feel a coward for doing so. Yet we, collectively, must not submit to this bullshit, as you so aptly put it.
I don't know the answer, or at least an easy answer. It's going to take a tremendous amount of personal agency and courage to overcome. Watching people around me submit to the ascendancy of Artificial Ignorance is both horrific and chilling, that people, many of my friends, have allowed their minds to be captured and knowing that I can't do a thing about it. Then hearing them repeat orthodox headlines and narrative summaries like some rote catechism puts it into the religious sphere of a savior.
I'd love to read your thoughts on the AI as Savior in its pseudo-religious context.
I try to be polite to Alexa so as not to fall out of practice.