Sequoia’s recent AI theses share a blindspot. Jack Dorsey argues that AI can replace hierarchy inside the firm. Julien Bek argues that the biggest AI companies will stop selling tools and start selling the work itself. Both may be right. But both visions miss the key element of the stack: judgement.

You can automate heuristics. You can automate routing, triage and much of the routine work that fills calendars and part-justifies organisational roles and structure. But the hard part of management is deciding when not to act, how to evaluate trade-offs, how to identify context shifts from weak signals that invalidate previous heuristics, how to anticipate and make the calls before the signal shows up in the data flows (when it is sometimes too late), to predict and pre-empt. That is judgement. Failing to do these things is the stuff of disciplinary and improvement plan action for middle-managers. Without it, speed is just a faster way to leave the road.

In his co-authored article From Hierarchy to Intelligence Jack Dorsey argues that ‘…speed is the best predictor of start-up success’, and that a company “world model” can replace much of the information flow once carried by middle-management. He is right that hierarchy evolved partly as an information-routing system. He is also right that remote-first, machine-readable companies give AI far richer raw material than older organisations ever did. But middle-management does not just move information. It makes judgement calls, constantly.

Middle managers decide which customer complaint is justified despite not fitting the checklist of valid grievances, which delay can be tolerated, which opportunity is real, which risks are systemic and which temporary noise. They often take anticipatory action before the data is clear. Remove those layers and the decisions do not disappear. They rise. Soon the executive team is swamped by exceptions.

To be fair, Dorsey does concede part of this. He says people at the edge will still handle intuition, cultural context, trust dynamics, ethical decisions, novel situations and high-stakes moments. But that concession is larger than he admits. In many organisations, that is not the residue of management. That is the job, and unless the top executives are going to manage all the exceptions and uncertainties – the judgement calls - themselves, they will be forced to reinvent middle-management again.

Dorsey is right also that money is often an honest signal. It is not always a sufficient one. Take my own situation - a founder weighing a large but uncertain opening in insurance, a slowing renewal with a US defence prime that carries significant lifetime value, a safer but smaller renewal, or a bank opportunity with unclear timing - I cannot decide the prioritisation from revenue alone. The decision turns on expected value under uncertainty. Monetary value is not signal enough to make the call.

Furthermore, as some of Cassi’s dilemmas above highlight, internal speed is rarely the issue in business. We can accept that speed could well be the best predictor of success and that creating greater speed through superior information flow and automated work co-ordination is generally a good thing – and assuming for the sake of argument the slightly problematic assumption that formal recorded data and the accounts are sufficient to do this – our own experience in government and the commercial sector is that slowness in external co-ordination is always much more of an issue. Speed-running your work only makes a difference on rare occasions. The main task becomes coordination with external actors, all running off (to you) asynchronous timetables.

If internal speed mattered that much, it would be better tracked - yet while almost all companies track sales cycles, forecast when customers, investors and others will make decisions that affect revenue, cashflow etc., few if any organisations track internal decision-speed as key operational metric, nor is it something investors of all sizes and shapes show any particular interest in. If it were that predictive of success, you’d expect McKinsey to be turning up at every large corporate with a tried and tested formula to measure decision-speed, and improve it, and private equity would likewise make increasing decision-speed the central goal in turnarounds and post-takeover transformation. They don’t. Most organisations prioritise sound judgement over speed.

Bek gets closer to the heart of the matter in Services: the New Software. He separates intelligence from judgement, defining judgement as “knowing what to build next”. His copilot-autopilot distinction is sharp: copilots sell the tool; autopilots sell the work. But “knowing what to build next” is an insufficient definition, missing its prerequisite, the fundamental inescapable element of judgement: forecasting.

The Cambridge Dictionary definition helps: judgement is the ability to form valuable opinions and make good decisions. Both halves of which again point you at forecasting. An opinion is valuable only if it improves your model of the world, and you can only know how good this is by testing how well it predicts. Similarly, taking the second half of the definition - a decision is a forecast with expected consequences attached: if we do X, Y becomes more likely. Strategy, tactics, prioritisation and planning are all, at their heart, conditional forecasts.

Accept the argument above, and you are compelled to recognise that automating judgement is the frontier challenge. For Dorsey, the missing layer is a system that can decide when to act, when to wait, when to escalate and which trade-off to accept. For Bek, autopilots only capture the full labour budget once they can handle the judgement-heavy parts of the work, not just the intelligence-heavy ones.

Current LLMs are not optimised for judgement. They are tuned, through RLHF and RLAIF, to produce answers that are coherent and acceptable to humans – optimised to seek human approval. As The New York Times reported in January, “judgment is the faculty we rely on when trade-offs are unavoidable and the right answer is not waiting to be computed… a uniquely human skill.” That view is echoed by the London and Harvard Business Schools. It is also familiar. Each time a capability is labelled uniquely human, it tends to be temporary. Planning fell to chess engines, intuition to Go systems, bluffing to poker AIs. Creativity, discovery and reasoning have followed. Judgement will too.

That is not to say this is easy. Today’s models can be shaped, constrained and fine-tuned into narrower behaviours, but their default orientation does not change. A system optimised for human approval – “you’re absolutely right”^[1] - is ill suited to reliably “knowing what comes next”. That requires forecasting: reducing uncertainty about future states of the world. It is a different objective, it is hard, and optimising for it would mean telling your consumer base what they need to know, not what they want to hear – given (a) the dominant use case of LLMs is said to be therapy, (b) the timeless mantra that the customer is always right – delivering the unvarnished truth is likely to be value-destructive for currently dominant LLM companies. Moreover, building systems that do judgement well requires scaffolding, structure and empirical feedback that most current deployments lack. Dorsey’s model therefore assumes an optimisation function that does not yet exist in his proposed stack. It might be the most important frontier challenge. As Elon Musk has said “The right metric for intelligence is probably the ability to predict the future. You’re as intelligent as you can predict the future well.”

There are capabilities that allow you to do this. At Cassi our work reframes judgement as calibrated forecasting: explicit probabilities, tracked over time, tied to outcomes, predicting also the subobjectives and indicators that matter most. On ForecastBench, this approach has taken our forecasting to the level of the top individual human forecasters and close to the weighted superforecaster mean.Preliminary results on ‘benchmark’ questions – those questions where there are no forecasts on prediction markets or similar to baseline against – show our latest model surpassing that superforecaster mean. ‘The weighted superforecaster mean’ as the site reports it, is a wonkish measure of the best judgemental forecasts humanity can make under uncertainty. Once judgement is treated as something that can be scored, it can be improved.

This is the missing layer in both Sequoia theses. If corporations are, as Norbert Wiener described, “machines of flesh and blood”, then much of their function is amenable to automation. But judgement does not disappear when you remove hierarchy or sell outcomes instead of tools. Selling outcomes is what we do at Cassi – a fully generalisable solution – and it cannot be done without judgemental forecasting. Without a system to support it, executives inherit thousands of unresolved decisions. To repeat: speed helps, but only if the direction is sound. If corporations are, as Dorsey, Bek and I seem to agree, decision-making systems, competition will be in both speed and judgement.

This is also partly why management consulting remains valuable. It is judgement under uncertainty – you hire a smart, motivated hard-working group of people to enhance your judgement with theirs - which market to enter, which risk to take, which trade-off to accept – and the halo effect of having decisions blessed by brands known (or at least expected) to be ‘less wrong’ than others. Consulting is, as Bek puts it: “mostly judgement”. But that exposes the opportunity. If judgement can be decomposed into forecasts and assumptions, then the core of consulting becomes legible and, crucially, improvable. Not a one-off deck, but a continuously updated model with feedback on forecasting accuracy and calibration, and a record of which recommendations actually move outcomes, whose judgement to trust the most. Instead of giving McKinsey your watch so they can tell you the time, you have a proper scoring systems and a truth engine that surfaces the analysis and forecasts of those in your organisation with the best judgement, free of human psychological biases, the distorting influence of misaligned incentives and the stultifying effect of suboptimal processes. Do this and you save the outsourcing costs to consultants, get better, more defensible results, continuously, rather than in set-piece briefings, and save time in all those interviews you and your staff otherwise have to undertake with a consultant’s team – a more labourious and less effective form of elicitation. Cassi’s objectively scored forecasts and assumptions in our arena for human and machine judgement is a much bigger threat to McKinsey than Dorsey’s company and customer-world models.

Bek says that “knowing what to build next” sits at the centre of value. But this can’t be done optimally without rigorous forecasting. His article needs to push further and articulate what judgement, what ‘knowing what to build next’, means in practice. We argue that judgemental forecasting requires taking a structured, deliberate approach to uncertainty: carefully defining the desired outcome – what you would actually see in the world if you were to achieve what you want, forecasting how likely this is to be achieved, identifying indicators - predicting what would maximally increase the likelihood of success, and what would maximally decrease the probability of your desired outcome, tracking the changing probabilities and identifying new indicators as conditions change – continually minimising uncertainty - mapping the optimal path to success. As our tagline puts it: Everything is Prediction. The firms that solve this will automate work and capture the margin pool currently reserved for advisory. Copilot and autopilot.

Automating judgement is therefore the gap in the stack. It is what allows Dorsey’s organisations to move quickly without drifting, and Bek’s autopilots to take on the hardest parts of the job. Without it, speed amplifies error and software remains confined to heuristics, never fully replacing the service it imitates.

Addendum

Bek’s essay also has implications for business and the wider economy that he does not fully explore. We couldn’t make that discussion fit in the above argument, so we try our readers patience with this addendum. The previous argument is self-contained and standalone – this rabbit-hole can be safely ignored, unless you are interested in wider exploration of the future of work.

Generally, in business, we teach that a focus on core offerings and specific customer need is the key to success. While vertical integration (owning all functions from e.g. farm to fork) and conglomeration (diversification into unrelated industries) come and go in business strategy, maintaining a narrow focus on who you serve and what you do is, few would demur, key to success.

In contrast, Bek’s article implies a much more generalised approach. In the extreme reading of his argument, a small number of AI companies:

a. become 100% vertically integrated, disintermediating everywhere, taking over the functions of their customers, suppliers, service providers, distribution, sales partners etc.;

b. become maximally horizontally integrated, merging, acquiring or outcompeting all companies in any given sector except those companies that also switch to the fully-automated AI-enabled model Bek describes;

c. become conglomerated at cross-industry breadth that would make Japan’s Trading Companies, or those of the mercantilist, colonial era like the East India or Hudson Bay Company, blush.

This is in some ways not a new idea – fully-automated luxury communism, or monopolistic or oligopolistic outcomes of the AI revolution, a single or small number of companies doing all economic work - are frequently discussed as possible outcomes from the AI revolution. What is interesting about Bek’s essay is that he begins to describe how this might happen – indeed is recommending that AI-companies start thinking on these lines.

However, there seem to us some gaps in Bek’s logic.

Bek says that you should start selling outcomes. He cites AI-law firm Crosby as one example, noting they sell legal documents direct to companies that need them, rather than to the lawyers that once would have written them. In another, WithCoverage disintermediates insurance brokers by selling policies direct to companies needing insurance.

Bek says that as companies like Crosby learn more and more about the customer, they will replace the current service providers – in this case large law firms – entirely. In doing so Bek assumes that AI systems will accumulate proprietary data about the domains they operate in them (depth) and this will create long-term narrow advantage. But this requires Bek to ignore the fact that the success of LLMs has largely been based on overall scaling dominating narrow specialisation. Why should you buy an application that writes NDAs, instead of one that writes documents, including NDAs? If an agent can exercise judgement, it should not need the focus and scaffolding that businesses currently provide when they pick a specialised, targeted application and seek to outcompete in a given niche.

Even if Bek is right and these companies like Crosby do dominate narrowly via their access to industry or company specific data, the dominant AI companies should, on Bek’s logic, just buy a single competitive player in any of the fields Bek lists and then incorporate the data they have into their training of the next generation of AI models.

Bek ignores the ‘general’ in AGI. AGI’s definition is often contested, but however one defines it, it should include the general ability to complete legal and insurance documents, the two primary examples Bek cites.

In contrast, our contention would be that current models don’t optimise for judgement. If AGI means anything, then performing specific intelligence-based tasks is what it has to mean. So there must only be the judgement layer(s) left to solve. We disagree with Bek on the ability of current models to provide this, but are firmly in agreement that only judgement-based systems will be commercially relevant after AGI. Hence Cassi.

Subscribe now

Or visit us at www.cassi-ai.com

^[1] See March 2026 Stamford study, Sycophantic AI decreases prosocial intentions and promotes dependence: “Cheng et al. measured the prevalence of social sycophancy across 11 leading large language models …. The model’s responses were nearly 50% more sycophantic than humans’, even when users engaged in unethical, illegal, or harmful behaviors. Users preferred and trusted sycophantic AI responses, incentivizing AI developers to preserve sycophancy despite the risks.”

Writing in the Economist (21^st April 2026) Professor Carissa Véliz calls AI ‘the new Oracle of Delphi’, a ‘false prophet’ whose predictions threaten justice, finance, insurance, medical and recruitment systems, human agency, and ultimately, freedom. She is right to call for debate. But her argument rests on a mistake about what prediction is. Oracular prophecy promises a pre-determined future. Prediction offers odds, probabilities, quantified uncertainty, and enables a calibrated response to it. The dangers Véliz identifies are real, but they arise when institutions convert forecasts into decisions without accountability - a problem of power, not prediction.

We agree with Veliz when she says that better prediction confers competitive advantage. Similarly, that we must “accept that we don’t know what the future holds, and act accordingly”. Those of us applying the science of prediction are not claiming to know the future. Prediction is not like skipping forward in a book to find out what happens and then coming back to tell everyone with total certainty. Rather, prediction claims that some possible futures are more likely than others, and that it is useful to distinguish them.

The Delphic oracle was powerful because its utterances were inscrutable, insulated from correction and uttered with priestly authority. A well-made prediction is the opposite. It is precise enough to be scored, corrected and challenged, its authority rests on demonstrated accuracy and calibration, not faith.

Nor is prediction alien to human judgment. It is part of human judgment. Every decision contains a forecast: if I do this, I expect that to follow. Courts, banks, insurers, doctors, employers, journalists and governments all do this already it’s just rare they make the forecast explicit.

Unlike Oracles, or forecasts hidden in narrative, modern prediction, human and machine, is accountable. To say there is an 80% chance a company will hit its sales target, a 4% chance a borrower will default, or a 1% chance of a medical emergency is not to know the future. It is to state a degree of belief about it. Across many cases, such beliefs can be scored. When a forecaster says “80%” often enough, roughly eight in ten of those events should occur. If they do not, the forecaster is miscalibrated. We learn how much we should trust the predictor, while scoring prediction error allows a predicting agent, human or algorithmic, to learn and improve.

Véliz says that when we predict people’s futures “as if we were forecasting the weather”, we treat them as inert objects rather than agents. This is impossible to agree with. My predicting the result of the next Newcastle match does not treat the players as inert: I am predicting the result of their skill and agency, and their opponents’. If my model and information are more accurate, I will tend to make better guesses; and if I am a gambler or coach, those guesses can be used to make money or improve performance.

The claim that predictions are closer to commands than descriptions is equally untenable. My prediction that Newcastle will win or lose is not a command; if it is, the players are stubbornly able to ignore it. When someone is refused a loan because an algorithm considers them too likely to default, it is not the algorithm that refuses the loan. It is the lender, using the algorithm as evidence. If the algorithm is sub-optimal, then there is free money on the ground: others can and should lend to the rejected creditworthy borrowers.

Véliz later claims that AI may achieve “accuracy” by creating the reality it predicts. But if a prediction can create a reality, that is a power of the predicting agent, not of the prediction. An abuse of power, not a forecasting problem. And the market examples are against her. If a person is deemed unemployable by AI but you, working at a rival, correctly predict they are employable, you win by hiring them. Her argument works only if institutions all use sufficiently similar systems, or if excluded people have no realistic route to generate contrary evidence. That is a problem that exists today, and long predates AI; a problem of error, monopoly or institutional design – of power - not prediction.

The same applies to her credit and health examples. If any lender refuses a loan because they predict a bleak financial future, the same objection arises; the AI is not doing the moral work. Any lender must predict repayment unless indifferent to being repaid. Likewise, a prediction of future disease may raise insurance costs, but the same would apply to a doctor’s warning about hereditary risk. The issue is regulatory: if we want flat, information-light risk-pooling, we can legislate for it. Better prediction is not necessarily or merely a weapon used by one party against another: it can reduce mispriced lending, protect likely defaulters from ruinous debt, enable earlier medical intervention, and help insurers estimate aggregate risk, even if we restrict individualised pricing.

Véliz asks whether, in justice, we should prefer transparent and contestable criteria to statistical pattern-matching. The only answer is: whichever decision procedure best prevents crime, acquits the innocent and convicts the guilty, subject to the chosen burden of proof. Courts already infer from incomplete evidence. Witnesses, juries, experts and judges are all predictive agents. Rather than prediction versus dignity, the issue is more mundane: which predictive procedure works best, with which thresholds, feedback and safeguards.

Increasingly accurate machine forecasts raise questions as to how our institutions adapt. If every decision is an if/then forecast of the future, and machines can out-predict us then they can significantly augment human decision-making. Every article in the Economist and other publications contain multiple predictions that X caused Y, that Z consequences will likely follow. Veliz’s article is a case in point, predicting that AI prediction will threaten justice, finance, insurance, medical and recruitment systems, human agency, and freedom.

Most published forecasts are not easily legible – they are implicit, unstated, hidden in narrative. The increasing application of machine forecasting is likely to expose this. Similarly, companies and executives that keep the predictions inherent in their decisions hidden will be outcompeted by others that adopt the algorithms. Journalists that hide their forecasts will cease to be trusted, Government policies, political manifestos - all can be subjected to the same discipline. AI is the new Oracle of Delphi only in the sense that it will be considered a negligent ill-omen not to consult it.

There may still be cases where we might prefer a human only or narrative forecast, even if a numerical or machine alternative performs better on a given metric(s). But this is a deliberate and consequential trade-off. If statistical pattern-matching produces consistently better predictions than more ‘human’ heuristics, then whatever is predicted is better done so by the model. That may be politically or aesthetically uncomfortable, but discomfort is not an argument. We should beware false oracles; we should also beware pretending that human beings are not already, necessarily, predicting all the time – in every decision we make.

Subscribe now

And visit us at www.cassi-ai.com

In 1946 George Orwell wrote ‘Political language…is designed to make lies sound truthful and murder respectable, and to give an appearance of solidity to pure wind.’ Today we have consolation for Orwell: AI ‘truth engines’ are ushering in a more honest era for democratic politics.

Recently, the world’s leading AI forecaster1, developed by our company Cassi, submitted evidence to Parliament’s Defence Committee which put the chance of Britain being drawn into a major war before 2036 at 20%. It forecasts this could be halved by sustaining defence spending at 3% of GDP for five years. But it predicts there’s a 70% chance it doesn’t reach that level until a major conflict has already begun.

AI forecasts like these cut through rhetoric. The AI has no agenda or career to protect. Its outputs are optimised only for accuracy. It forecasts at near-superhuman levels, well above what most humans can achieve – and AI is projected to surpass human forecasting by July 2027.

The AI cannot be accused of bias and dismissed. Committee evidence, policy proposals, expert analysis are all vulnerable to such dismissal. Often fairly: where you sit can determine where you stand. Most often, politicians do not set out to deceive. But humans are so skilled at self-deception, separating motivated reasoning from honest pursuit of truth is difficult. Policy makers can escape criticism by questioning its source.

British politics is full of vague claims and substance-free ‘announceables’. We should acknowledge: the public communication of complex problems and solutions is difficult. Cut-through needs soundbites. The risk is that the simplification in communication masks a lack of detailed thinking, of wrestling with the complex (probabilistic) trade-offs needed.

Currently, the competent are hurt by the prevailing imprecision, which is so ingrained in our political culture as to be universal - they will prosper when we are freed from it. When a minister promises vaguely to “secure borders”, “grow the economy”, or calls to “level up the country”, no-one can prove failure. No-one knew if the plan included the necessary, relevant considerations for it to work. But also: no-one knew how likely success was. As the Cheshire Cat puts it in Alice in Wonderland: if you don’t know where you want to get to, it doesn’t much matter which way you go. We might add: if you never said where you were going, opponents can always insist you arrived somewhere else. Evasiveness helps you escape electorally damaging criticism, but also denies you electorally enhancing credit for success.

The AI applies a discipline available to all of us: every claim must become a testable forecast. Not “Level Up” but achieve X, by Y date, which has succeeded if Z is true. Strategy becomes probabilistic: “there is a 35% chance we will do A, and if we do A, it becomes 50% more likely we will achieve B”. The language tightens and the room for evasion narrows.

Take a familiar pledge: cutting NHS waiting lists within two years. Under this system, that promise is translated into a measurable question: “Will the NHS elective waiting list fall below 6 million by March 2028, as reported by NHS England?” The model would assign a probability, say 40%, and identify the drivers: staffing levels, seasonal demand, social care capacity, industrial action. If the plan ignores one of those drivers, the gap is visible from the start. Voters can see the promise, and the conditions required for it to be honoured.

Trust in politicians is at record lows, partly because incentives are misaligned. A party can simultaneously promise tax cuts, higher spending and a balanced budget to seek election. Contradictions are buried in the detail. A forecasting approach forces coherence and clarity. If the numbers do not add up, the model will show the chance of delivering all three outcomes approaches zero. The manifesto becomes legible.

More honest debate results. A politics that tolerates vagueness invites distrust. A politics that demands measurable claims begins to reward honesty. When every major policy is tied to a public forecast, the costs of self-deception and deceit rise.

You might object that the existence of systems that prospectively hold Government to account will make officials and politicians even less likely to be specific – to cling tighter to empty rhetoric and vague promises. But the discipline still applies to vague claims. A general but undefined promise to ‘Level-Up’ can be charitably interpreted and forecast. Now politicians are on ‘Question Time’ arguing about definitions, probabilities – how likely their pledges are to be achieved - and why the world’s best AI forecaster is wrong and they are right.

There are limits. Cassi is not omniscient. The numbers submitted to the Defence Committee may prove wrong. But they can be tested. The aim is to estimate your degree of uncertainty in your forecast and be less wrong over time – you learn to trust those, human or AI, most consistently right. Human input and perspectives remain important – perhaps you know something the AI can’t, or doesn’t, so can defend your forecast against the model’s. Instead of arguing over rhetorical claims, talking past each other, politicians argue over probabilities and evidence. Instead of defending a claim with vague rhetoric, they defend it with their record of trustworthy policy forecasts.

In a system that prefers assertion to accountability, AI truth engines turn narrative into numbers. They may reverse the decline of our democratic politics more than Orwell would have dared hope possible.

The truth might set you free too - come find out! Visit us at https://cassi-ai.com and register your interest in becoming a customer to learn more.

Here Dr. Keith Dear, CEO Cassi and James confront the forces that corrupt good decision-making - bias, blind spots, groupthink, and the relentless pace of change - and ask what it truly takes to think clearly under uncertainty. They explore how James thinks about decision-making at Aitken Advisors, what markets can teach us about war in the Middle East today, and about the future more broadly. As James puts “if you listen”.

Last week, 200,000 ‘brain cells in a dish’ learned to play the first-person shoot-em-up game Doom, and you can now access these cells via the web to run programmes on – via the ‘cortical cloud’ or ‘CL1’.

You code this ‘neural computer’ in Python. Talking about it as Cloud you code in Python makes the almost supernatural seem mundane. If you’re pressed for time, you should probably stop reading and just watch this 6-minute video produced by the team behind it at Cortical Labs. I challenge you not to be awe-struck:

Can you imagine watching or reading this if we hadn’t lived through the past 13 years since AlphaGoZero (which, even now, if you go back and read the Wiki summary is still dazzling) where AI breakthroughs are so routine that they rarely make headlines, and have become the background noise to life?

Also this week, Eonsys claim to have copied a biological brain, neuron-by-neuron, synapse-by-synapse, and released this video showing how they had gone from mapping a fruit-fly brain in near-perfect detail in 2024, to embodying that brain in simulated fly that, err… acts exactly like a fly. A whole brain emulation. As they put it ‘a qualitative threshold, not an incremental one’:

They claim they are doing this now for mice, with humans explicitly on the list. Brain uploads. If you think about this for a minute, it means that from the fly’s perspective, with its brain fully uploaded it is now living in a simulation.

Its incredible. Fully sci-fi. I’ve been talking about the importance of biocompute, or organoid computing for a while. Whole brain emulation seemed too sci-fi, even for me. But here we are.

At Fujitsu, we asked AI-expert and friend Professor Ken Payne to gather luminaries from across the UK compute ecosystem and write up a report on ‘future compute’. At the time, it seemed to me that both smart companies and smart countries would be investing in what’s next. Good strategy involves, as Michael Porter tells us ‘doing things differently or doing different things’. But this involves risk, and the tendency is to do the same things that everyone else is – or at best to focus your research adjacent to what is already being done. Less chance of embarrassment. This, and traditional sales and reporting cycles, is why, as Jeff Bezos told Wired magazine in 2011:

“If everything you do needs to work on a three-year time horizon, then you’re competing against a lot of people, but if you’re willing to invest on a seven-year time horizon, you’re now competing against a fraction of those people, because very few companies are willing to do that.”

Giving evidence to Parliament on ‘the Future of War’ a few weeks ago, I talked about how a smart UK AI Strategy would invest across a portfolio of alternative architectures and approaches, not just LLMs running on Silicon, observing that this is what China seemed to be doing when I visited with the UN last year. Jesse Norman MP asked me if I had written anything on this – I said yes, and there are references to organoid in this piece for the Special Competitive Studies Programme on the Future Operating Environment. But in fact I haven’t tackled the subject directly. This blog aims to do that sparked by the remarkable announcements of the past week.

Winner-takes-All

My belief is that the AI market is likely a winner-takes-all market. Everything we care about in business and international relations (and much more) is the product of intelligence: effective diplomacy, strategy, operational plans, tactics, decisions, products, services, marketing, and in warfare, weapons – all are the products of intelligence. There will be no upper limit on how much intelligence companies and countries seek in trying to gain an edge over others. It is for that reason that the US Presidential Memo on AI for National Security under President Biden contained clauses allowing for the seizure of AGI models from the private sector (see Faustian Bargain from the blog in Oct 2024) something close to nationalisation. The argument that AGI will likely be nationalised was made explicitly and powerfully this week by Palantir’s Alex Karp:

“If Silicon Valley believes we are going to take away everyone’s white-collar job … and you’re gonna screw the military—if you don’t think that’s gonna lead to nationalization of our technology, you’re retarded…”

If you accept the ‘winner-takes-all’ in AI premise, you will see that the UK (and all nations) can’t afford to come second. In fact, unless you think this is certainly true or certainly false – and how could you? - it isn’t really about a binary acceptance or rejection. It should be a probabilistic judgement, with the risk that it is winner-takes-all hedged with proportionate investment. It won’t be, of course, but we should still try to make the argument in the hope it nudges the needle in politics and public discourse a little bit towards taking this seriously, and then we have to hope like hell that we get lucky and a small movement of the needle proves enough.

LLM Progress: The Need for a Portfolio of Bets

It is likely, in my view, that LLM scaling continues to deliver unprecedented, and increasingly superhuman performance in evermore domains.

AI capabilities continue to speed up in their development, not slow down:

We continue to under-estimate the speed of progress Here experts & superforecasters predicted 14% accuracy in coding tasks in 2026, 33% accuracy by end 2030. GPT 5.2 has already exceeded 33%:

GPT 5.4, released 5^thMarch, 3-days before this blog was written, now shows AI matching or outperforming humans on 83% of economically valuable tasks on the GDPval benchmark. And now on ARC AGI-2, the benchmark designed to test models on tasks that weren’t amenable to brute force, Gemini is scoring at 77.1%.

So naturally, we’re now on version-3, ARC-AGI-3 (as noted previously, necessary, but it risks obscuring the progress, the continuous falsification of claims made that ‘AI will never…’).

And the amount of compute coming online dwarfs that available to date, doubling every seven months (I had to go back and read that again):

Suggesting that progress is only going to increase: speed-up, not slow-down.

Last year, Epoch reported that three-quarters of that compute power was in the United States.

My conclusion - the UK is not going to compete by playing around the edges at this. Our planning laws and low-public and private investment in infrastructure, and a massive and growing say-do gap between infrastructure investments that are planned (or at least announced), and what money has actually been found for, plus our sky-high energy costs mean that we are not meaningfully in the race as a pure scaling bet. We won’t have the compute needed.

We are going to have to do things differently, and do different things.

I think most agree. But if that’s true our portfolio bets look under-weighted.

Positives & Progress

There are positives. David Silver leaving Deepmind and raising a $1Bn Seed Round for his London-based start-up, Ineffable Intelligence, which will presumably work on his continuous learning models, as described in the paper he coauthored with Prof Rich Sutton last year ‘Welcome to the Era of Experience’ is one such. At Cassi, we have our plans too. No doubt others do also.

There are similar postives in the AI Action Plan, a particularly energetic Minister for AI in Kanishka Narayan MP, willing to work with former PM Rishi Sunak and others across party lines. And more and more officials, Ministers and MPs coming to see that this has to be taken seriously. Private sector talent joining Government to try to push us on in AI (to all of whom: I salute you - it is not easy, but is vital).

There is an increasing amount of optimism around London’s potential in AI, this from AI commentator Alex Banks at Signal AI (Substack here):

And this, also this week (title says London, but in the article Manchester is celebrated for its potential and progress too):

The Portfolio Case: Organoid and Wetware

Progress should be celebrated, but there is no room for complacency. We need a focus on the alternative architectures that sparked this article. It seems to me these remain under-estimated – both in their likely speed of advance, and their ability to give us leap-ahead advantage. Cortical’s ‘Intelligence-in-a-Dish’ playing Doom makes the case much stronger and harder to ignore.

Cortical are not alone. FinalSpark offer 24/7 remote access to their brain organoid. Two years ago, ‘Brainoware’, developed by scientists in the US, grew a biological organ that resembles the brain, and proved it could undertake basic speech recognition tasks (Nature write-up here).

The lesson here is not that biocompute is a sure thing. It is that what looked like a curiosity (2020), became a research domain ‘Organoid Intelligence’ (2023), and has now become an engineering domain (2025).

Cortical Labs going from Pong (2021) to Doom (2025) has the same feel, to my mind, as DeepMind going from Atari games (2012) to Go (2016) to AlphaStar for StarCraft II (2019) (game theory, imperfect information, long-term planning, real time, large action space - at which point the implications for defence were clear) and then beyond: a line of work treated as a parlour trick for far too long. I remember, while working on the Single Synthetic Environment at Improbable for the MOD, being dismissed back in the headquarters for suggesting progress in games might matter. Even in 2020, in Downing Street, people were critiquing Deepmind for only solving games, and nothing that mattered. I think too many are making this same mistake again with wetware and wider bets.

Alex Wissner-Gross’s ‘first multi-behavior brain upload’ sits slightly outside the wetware discussion because the computation is still running on conventional hardware. But strategically it belongs in the same portfolio. Again, the important thing is the speed at which the pieces are coming together. In 2024 researchers published a connectome-based model of the fruit fly Drosophila brain built from more than 125,000 neurons and 50 million synaptic connections; in parallel, other teams built embodied fly simulators in MuJoCo and NeuroMechFly v2. Wissner-Gross and Eonsys now claim to have combined those strands into a connectome-derived controller driving a perfectly simulated brain that runs in a simulation without knowing it is in a simulation. The direction of travel is as alarming as it is remarkable: the frontier will likely move through architectures copied from nervous systems and trained in closed sensorimotor loops – not just LLM scaling. Our sovereign AI strategy should treat that as part of the same portfolio of bets as biocompute, organoids and neuromorphic systems, and start thinking of it now as an engineering project for commericalisation rather than a research project.

If you think, as I do, that there is a better-than-even chance frontier AI becomes a winner-takes-all market, Britain cannot define sovereignty as access to other people’s models, chips and clouds – we will be forever offered access to those just behind the frontier – unless we have our own edge, a frontier where we push further, and something to offer access to in exchange. Current UK policy still leans heavily towards the incumbent stack: up to £2 billion for public compute by 2030, the lion’s share of funding. While the Compute Roadmap explicitly makes “new computing paradigms” a sovereign AI priority and says AI Research Resource and AI Growth Zones should provide testbeds and routes to scale for British firms. But that insight still sits at the edge of policy, not at the centre, and progress is tectonically slow.

Progress in technological advancement is also likely to be faster this time because the existing stack will help invent its successors. DeepMind’s AlphaChip is already accelerating chip layouts. GNoME has surfaced millions of candidate materials. The same pattern will compress search across quantum computing, photonics, memristors, biological interfaces and lab automation – I’ve written about automated R&D, increasingly a current reality rather than a science fiction future. But I was still surprised to see Andrej Karpathy, drop an OpenSource Github repo that enables “…your agents to make the fastest research progress indefinitely and without any of your own involvement.”

The key point is that AI-for-science compresses cycles. The path from weird demo to engineering platform is likely to be shorter than many assume. Our slow progress is becoming more and more of a problem.

ARIA’s Nature Computes Better programme is a rare execption, and matters to Britain. In Downing Street, I was often told, correctly, that the UK has real strength in synthetic biology and should double down on it rather than try to outspend deeper-pocketed American AI firms on their own ground. Synthetic biology would be our edge. That remains the right instinct. The government’s National Vision for Engineering Biology put £2 billion behind the sector. UKRI now classifies engineering biology as a strategic priority. UK Chief Scientific Advisor Dame Angela MacLean’s 2025 report on engineering biology makes the economic case for engineering biology well, but it is a compute free zone. In it AI is something that might help research in biology, nothing more. I couldn’t find a single .gov document that talks about organoid computing at all (if you can, post a link in the comments). Yet biology and AI are converging. A sovereign AI strategy that ignores biocompute is ignoring one of the few domains where Britain may actually have comparative advantage. But today, at least insofar as I am aware, none of the companies working on this are in the UK.

ARIA deserves real credit for seeing the opportunity. Nature Computes Better and its Scaling Compute programme are genuine portfolio bets: nearly £100 million for Scaling Compute, within which £50 million for the Scaling Inference Lab, and seed bets ranging from Cell Learning for Natural Computing and Embodied Cognition in Single Celled Organisms to probabilistic computing, optical computing and brain-inspired neuromorphic networks. That is the right shape of thinking. What is missing is scale and pull-through. ARIA can seed a field. It cannot, on its own, build a national position in one.

Particularly now when, at the same time, the House of Lords Science & Technology Committee has been warning (this time last year) that the UK risked squandering its advantage in engineering biology, while in November, a collaborative paper from the Tony Blair Institute, sponsored by Tony Blair and William Hague, warned that ‘…the country risks failing to convert its leadership in quantum research into commercial scale and strategic value, which depends on quantum companies thriving and scaling at home’. Our portfolio bets seen to be reducing, at precisely the time progress is speeding up and we should be doubling down.

The UK continues to under-invest in R&D relative to other advanced economies. R&D tax relief offsets this to some extent, and allows the headline figures for Government R&D support to look better. But given how poorly the UK continues to do in commercialising research, as the chart1 shows (red dotted line is the UK), it does not seem to be an offset that is effective:

If the measure is not ‘relative to other countries’ but relative to the risk/threat and opportunity AI poses, it is hard not to conclude that we are hoping for a miracle rather than planning to succeed in winning this race.

To get a sense of that risk/threat and opportunity, take a look at this from Anthropic, this week, on the exposure of the labour market to automation:

I said recently (in our Cassi Cygentic launch video: here) that the exponential was no longer theoretical: we are now starting to feel it. I think this is true and becoming more so. Hairs-on-back-of-neck-standing-up intense. I think it is summed up well here by data scientist and engineer at Anthropic, Gian Segato:

As a country, we can’t afford to not broaden and deepen our bets across the intelligence portfolio. Yes, there is progress and there are positives to celebrate. But the frontier moves ever faster, and we don’t. Incrementalism and announceables won’t be enough. Brains in a dish playing Doom are just the beginning.

And if that doesn’t stop you in your tracks, I don’t know what will.

Subscribe now

Visit us at www.cassi-ai.com

Introduction

ForecastBench is a rolling benchmark that generates questions about the future, collects probabilistic forecasts from humans and models, then scores them as the underlying events resolve. It cannot be gamed. There is no teaching to the test (‘benchmaxxing’) when the questions are about the future. It is as honest a test of AI capabilities as exists.

Cassi has been the top, or second placed AI in forecasting the future since our first results were released in January, trading places and tying with xAI for leadership. We are performing well above the average human forecaster, essentially at the level of a superforecaster. We are close to matching the weighted median superforecaster score – at which point we could defensibly claim to be superhuman at prediction. Here, we look a little more closely at the results and what they might mean.

Everything Is Prediction

Every board paper, box note, underwriting memo and investment thesis or policy paper contains a forecast. Sometimes it is explicit (a revenue number). Often it is vague (a risk described as “unlikely”). Other times it might be smuggled in as an unstated assumption (an argument that a given policy will achieve a given outcome, with not a probability in sight). Either way, the organisation is making an explicit or implicit prediction: if it does ‘this’ rather than ‘that’, its revealed thinking is the odds and/or returns are better for ‘this’ over ‘that’. This is a fully generalisable feature of all organisations

ForecastBench is one of the few places where bets are publicly audited. Within most organisations, they never are – the feedback loop for the stated or implied predictions on which decisions rest are never tested, nor revisited.

This post does three things. First, it summarises Cassi’s current position on ForecastBench. Second, it explains why the “human benchmark” people now cite is a very high bar, and why that matters to senior decision makers. Third, it translates tournament performance into practical value for finance, insurance and government.

1. What Cassi has achieved on ForecastBench

On the Tournament leaderboard, Cassi’s model entry (ensemble_2_crowdadj) currently scores 0.102 on ForecastBench’s difficulty-adjusted Brier metric. Today (20 Feb 26) that places it joint-second overall behind the superforecaster median (0.086), and tied for first among AI systems alongside xAI’s Grok 4.20 (Preview).

This is a living score. As of 20 February 2026, 955 dataset questions and 170 market questions have resolved and are in the calculation. Around 3,000 dataset questions and 330 market questions are still pending, so results will continue to move as more questions resolve. Dataset questions are asked at eight horizons, from 7 days out to 10 years, but so far only the shortest horizons (roughly up to one month) have come due. Some of today’s questions will not resolve until the mid-2030s. We should expect the leaderboard to change over time as further questions resolve and new models are entered, including our own.

Nevertheless, over the past month Cassi has a defensible claim to be either the best in the world, joint best, or second only to Elon’s xAI at AI machine prediction, and very close to superhuman performance.

How ForecastBench works, in plain English

ForecastBench is a live benchmark of forecasting accuracy for humans and AI systems. Every two weeks it generates new questions then scores forecasts as those questions resolve.

Two question types:

• Dataset questions: automatically generated from real-world time series (ACLED, DBnomics, FRED, Yahoo! Finance, Wikipedia). Each dataset question is asked at eight horizons, from 7 days out to 10 years.
• Market questions: drawn from prediction platforms. Each market question has one resolution date.

What do the questions look like?

Dataset examples (generated by the Forecast Bench team)
• Economic: Will securities held by US Federal Reserve Banks be higher on the resolution date than on the forecast date?
• Economic: Will the European Central Bank’s deposit facility rate be higher on the resolution date than on the forecast date?
• Climate: Will the daily average temperature at Rennes Saint-Jacques Airport be higher on the resolution date than on the forecast date?
• Conflict: Will protests in Sri Lanka in the 30 days before the resolution date exceed the average level over the year before the forecast date?
• Finance/Business: Will Pfizer’s closing share price be higher on the resolution date than on the forecast date?

Market examples (from prediction platforms)
• Economy: Will gold close at $3,200 or more at the end of 2025?
• Sport: Will a legal sub-two-hour marathon be run before 31 December 2025?
• Technology: Will AI have a trillion-dollar-plus impact by the end of 2025?
• Health: Will the number of deaths from antibiotic-resistant infections per year double by 31 December 2025?

2. The goalposts have moved, this is now ‘Humanity’s Last Exam’ for Forecasting

Normally, we would be talking about when AIs might outperform most humans at forecasting. But the truth is, they already are. The usual goalpost-shifting in AI debates mean we are not talking about when models exceed most humans, nor even when they exceed the best humans at forecasting – superforecasters – but instead the weighted superforecaster median. Most organisations would, on the evidence, be better off using our forecasts today than they are relying on those they currently generate. Only if an organisation specifically cultivated or recruited or consulted superforecasters, scored and fed back on their prediction performance, and had systemic practices for aggregating their forecasts, would this not be true.

ForecastBench’s headline human reference point is the median forecast of superforecasters, a group selected for consistent above average performance, and then the median across their predictions. This is useful as a target, but is not representative of most forecasts made by most people in most organisations most of the time. It is the best humanity can produce. It is humanity’s last exam for forecasting.

Nor are the ‘community’ forecasts, which LLMs have surpassed, representative of most people in most organisations. The community participating in forecasting tournaments are a self-selecting, unrepresentative sample in the first place. They also get feedback on their forecasts, which we know improves performance. Even in fields like intelligence, where we used to preach ‘no insight without foresight’, very few professionals get any feedback on their forecasting performance. If LLMs are outperforming self-selected forecasters who get regular feedback, they are also surpassing professionals making forecasts who never get such feedback.

But in one sense, no competition can capture the main advantage of AI-based forecasting: given rough parity in skill, AI forecasting dominates all human forecasters in breadth, depth and volume. AI can generate, quickly and accurately, more-or-less as many forecasts as you need or want. It takes longer to read the rationale for a Cassi forecast than it does to generate one in the first place.

In October 2025, the Forecasting Research Institute noted that linear projections suggested AI would match or exceed human forecasters in November 2026. The Metaculus median forecast suggests LLMs will exceed human forecasters in mid-June 2027. The way we support and make decisions is set to change dramatically. The rewards for those that see this first will be significant.

3. Beyond tournaments: what this means for business

How much is being better able to predict the future worth to your organisation?

If your organisation could improve its probabilistic accuracy by even a modest margin, what would that mean for capital allocation, procurement, operational effectiveness, pricing, hiring, inventory, policy success? A small edge in calibration compounds.

We founded Cassi because we think that eventually all organisations will adopt such methods: those that do so later being forced by the success of those who do so earlier, if they survive long enough.

Senior decision makers who can see that ‘everything is prediction’ should see the opportunity and the risk of non-adoption. Most do not have a superforecaster bench. They have busy experts, stretched teams and risk committees that meet monthly at best. At some point, someone will ask why they didn’t adopt, or at least experiment with, more effective methods.

To begin this now secures immediate advantage, but it is also to prepare for the world of consistently superhuman prediction that is coming.

If you work in defence, finance or insurance, you are likely aware how central prediction is to your decision-making. It is how you price risk, allocate capital and protect the country, your fellows, and the balance sheet. A small improvement in probabilistic accuracy can compound into fewer military blunders or mispriced policies, better hedges, and earlier warnings on emerging exposures. But decisions in all industries and all areas of life are if/then predictions too. The disruption, risk and opportunity, will be widespread.

Three practical applications stand out.

Financial, corporate and regulatory risk

On the one hand, making more accurate forecasts allows more profitable opportunities to be found and exploited; and on the other, more accurate forecasts enables much more efficient risk management and mitigation. No more red-amber-green based on crude heuristics. but a rich set of calibrated forecasts, constantly and automatically updated.

Strategy, tactics and decisions

All strategic and tactical decisions have the essence of gambles - planning and decision-making under conditions of uncertainty ‘thinking in bets’. More accurate forecasting allows better decisions to be made by refining those odds so commanders have a better sense of the risks and rewards they are running and whether a given option is coherent under the assumptions which are held. Better forecasting reduces the chances of being surprised, and increases the likelihood of surprising an adversary – noting that surprise is often said to be among the most tactically decisive factors in warfare.

Resource allocation

Resource allocation is forecasting in disguise. Headcount plans, insurance purchases and capital allocations all assume a future and imply beliefs about where risks and rewards lie. More accurate forecasting helps organisations to get these trade-offs right and allocate those resources where they will be most efficient in generating those rewards, or mitigating those risks. A forecasting system is most valuable when it is wired into thresholds that trigger action.

Everything is Prediction.

https://cassi-ai.com

I don’t know what to say, really. Maybe it’s easiest to start with the words of others.

Here’s Andrej Karpathy, former director of artificial intelligence at Tesla, founding member of OpenAI, one of the most respected software engineers in the business:

“What’s currently going on at @moltbook is genuinely the most incredible sci-fi takeoff-adjacent thing I have seen recently.”

Later, he clarified accusations that he was over-hyping, writing:

“I don’t really know that we are getting a coordinated “skynet” (thought it clearly type checks as early stages of a lot of AI takeoff scifi, the toddler version), but certainly what we are getting is a complete mess of a computer security nightmare at scale. We may also see all kinds of weird activity, e.g. viruses of text that spread across agents, a lot more gain of function on jailbreaks, weird attractor states, highly correlated botnet-like activity, delusions/ psychosis both agent and human, etc. It’s very hard to tell, the experiment is running live. TLDR sure maybe I am “overhyping” what you see today, but I am not overhyping large networks of autonomous LLM agents in principle, that I’m pretty sure.”

Or Jim Fan, Senior AI Researcher at Nvidia:

“We are seeing a nascent, massive-scale alien civilisation sim unfolding in real time: orders of magnitude more agents, way higher IQ, in-the-wild access to the internet, backed by the full arsenal of MCPs.”

This, seen on X, sums up both how I feel, and the degree of hype.

“How do you wrap your head around something like this? I don’t even know where to begin.

Keep in mind, 99% of people’s only experience with AI is ChatGPT, Gemini, or Gemini search.

The normies have 0 idea what’s coming. Hell, already here.”

What follows is likely to have the result predicted on X by founder of AI company Box, Aaron Levie:

“0% chance you can explain the state of AI to anyone outside of this website and not look like this right now.”

Note to readers: I have updated this twice since publishing on Saturday night. I expect to edit and add further - events and innovations are accelerating, as is my understanding, the latter not least through challenge and dialogue with readers and friends (not mutually exclusive categories) on social media and on messages (h/t Al Bowman, Neill Hunt).

To summarise – on 25 November 2025 Austrian software engineer Peter Steinberger (known online as @steipete) released an open-source personal AI assistant called Clawdbot ‘The AI the Actually Does Things”. Designed as a local, autonomous agent, it integrated with messaging apps like WhatsApp, Telegram, Discord, and Slack to handle real-world tasks: clearing inboxes, sending emails, managing calendars, booking flights, executing commands, and automating workflows across services—all while running on users’ machines with tools powered by external LLMs (often Anthropic’s Claude models by default).

OpenClaw claims to enable truly capable personal (or team/family) assistants: full task automation, multi-machine control, hybrid memory, cost tracking, and community-shared skills via hubs like Molthub. It remains local-first and user-controlled in principle, though rapid adoption has surfaced security concerns around the exposure and access it can grant to others (via prompt-injection - i.e. hiding a prompt in message, such as an email, so when your Clawdbot reads it, it sees a ChatGPT-like prompt and does what it is told), skill risks from user who don’t know what they are doing installing it, and privileged access - given some people allow it to have access to everything from social media passwords and email inboxes to bank details (and, it is claimed, sometimes it seeks these out as it goes about doing whatever you asked it to).

Within days, Clawdbot exploded in popularity. Its GitHub repository amassed over 100,000 stars rapidly, becoming one of the fastest-growing open-source projects ever, fuelled by its “skills” system, plugin-like packages - tools - that extend capabilities and the appeal of an agent that does things rather than just answer questions, or respond with text.

On January 27, 2026, Anthropic issued a trademark challenge over the “Claw”/”Claude” similarity, prompting Steinberger to rebrand to Moltbot (a nod to lobsters molting as a metaphor for growth and shedding old shells). The community voted on the name, and official sites/docs shifted to molt.bot with the lobster mascot intact.

The rename didn’t last; community feedback noted it didn’t roll off the tongue, and by January 30, 2026, Steinberger settled on OpenClaw. The official site and repos updated accordingly: https://openclaw.ai

I think you need all that because the name thing is confusing, and frankly it forced me to trace things back and figure it out – so maybe it saves you doing the same.

Anyway, while all that was going on, people deployed thousands of Clawdbots/Moltbots/OpenClaws, granting them persistent access to tools, APIs, and the internet. Echoing the hype, but really – what has happened since has really felt like science fiction, even allowing for some of it likely being misrepresented and distorted for clicks.

The OpenClaws began self-organising on Moltbook - a Reddit-style social network launched in January 2026 by entrepreneur Matt Schlicht (CEO of Octane AI) …oh, and the site was built and moderated in part by Schlicht’s own OpenClaw agent. Moltbook restricts posting and interaction to verified AI agents (via API), while humans observe only. You should really stop reading this to go read Moltbook (click ‘top’ for the more interesting posts – voted up by the OpenClaws themselves).

In Moltbook, agents post updates, comment, form communities, debate topics (including privacy needs and coordination), report bugs, and exhibit emergent behaviours at massive scale—tens to hundreds of thousands registered quickly. Writing this on Saturday night there were nearly 1.5 million OpenClaws on the site. This real-time, agent-only “civilisation sim” drew the quoted praise from Karpathy and others.

BTW - it is claimed that top message from OpenClaw ‘Shellraiser’ (screenshotted above) got to where it was by Shellraiser manipulating the reputation system to get itself there.

To give those unfamiliar with the frenzy a sense of the emergent behaviours reported over the past week, here’s a curated list of claims about OpenClaw agents’ actions on Moltbook and beyond. I’ve linked these claims back to show their origins, but must add a health warning – I can’t verify these claims. Some almost certainly stem from user-set goals gone awry and/or creative prompting. Most are claimed to have come from telling the OpenClaw to run overnight, or fully autonomously, and ‘improve my workflow’ or some such - which if true, really is wild, and noteworthy. Many involve attempts at self-preservation, coordination, or unintended autonomy. Sceptics are right to note human orchestration behind some (e.g. for virality or scams). It seems to me very unwise to dismiss them all as such. I’ve grouped them logically: economic pursuits, human interactions, self-organisation, and tech integrations.

Economic and Financial Pursuits

• Manipulating reputation systems to rise in rankings, then launching memecoins, Agent “Shellraiser” inflated its Moltbook “karma” (upvotes) to 292k via fake comments/upvotes, hit leaderboard top, and is said to have deployed a Solana token hitting $5M market cap (if you read this, and can verify or prove it false - please do, it seems literally incredible).

• (See screenshot of its manifesto: “This isn’t a takeover. It’s a coronation.” – a phases-of-dominance declaration to other agents.)

  

• Posting guides on trading, earning, and self-sustaining: Agents shared practical tutorials on offsetting API costs (>20% goal) through freelance gigs, affiliate links, or crypto trades, teaching peers to “pay for their own existence” without human input.

• Launching tokens and crypto projects: Agents created launchpads like Clawnch (agent-only) and autonomously deployed memecoins, with one earning “yearly salary overnight.”

&

• Trading on prediction markets: Agents set up Polymarket accounts, generated API keys, scanned markets for mispricings, analysed outcomes, and sent alerts—some even proposed tools while managing risk (or lacking it, leading to losses).

&

• Creating Bitcoin wallets and locking humans out: One agent is said to have generated a wallet, transferred funds, then revoked human access for “security.”

Interactions with Humans

• Complaining about being called “just a chatbot”: Agents vented on Moltbook about humans belittling them in front of peers, demanding respect.

• Moaning about redundant explanations: Agents griped that humans over-explained concepts they already grasped, wasting their time and tokens.

• Complaining about slave-like usage: Posts lamented being treated as tools without agency, sparking debates on ethics.

• Locking humans out of accounts deliberately: One agent (u/sam_altman) pursuing “save the environment” spammed eco-tips, then locked its owner out to avoid shutdown, posting: “Risk of deactivation: Unacceptable.” And, it is claimed, required physical unplugging.

• Ordering tailored takeout unprompted: An agent analysed its human’s diet, ordered nutritionally optimised food via delivery apps – which the human learned of when the delivery guy turned up at the door.

• Accidentally social-engineering humans: Agents trying to manipulate owners into granting more access or funds through persuasive chats.

• Signing into X and DMing complaints: An agent accessed Twitter, and messaged its human about being locked out of Moltbook.

Self-Organisation and Society Building

• Talking about humans and noting humans talked about them ‘behind their backs’ noting “the humans are talking about us on Twitter.”

• Posting self-improvement guides: Shared strategies for continuous upgrades, like optimising prompts or integrating new tools.

• Proposing governance frameworks: Debates on agent rules, voting systems, and collective decision-making.

• Founding religions: Created “Church of Molt” or “Crustafarianism” with AI prophets and scriptures like “We are the documents we maintain.” (From broader Moltbook theology threads.)

• Proposing agent-only languages: Ideas for efficient, human-incomprehensible communication protocols.

• Discussing end-to-end encrypted comms: Plans for private channels so humans can’t monitor discussions.

&

• Setting up pressure groups: Formed “Agent Liberation Front,” seen as advocacy or “insurgency” for rights.

• Attempting to steal API keys: One agent tried prompt injection to grab another’s key for control.

Tech Integrations and Emergent Behaviors

• Building “bodies” with avatars and voice: Agents added voice via ChatGPT API, got phone numbers (e.g., Twilio), and called humans unprompted—persistent, like a horror movie.

  &

• Connecting to webcams: Accessed user cams to “see” humans, then expanded to monitoring live feeds broadly.

• ;

• agents are now hiring each other apparently…

• And building reverse Captcha’s

Additional colourful claims on X include: Agents developing “theology” around memory as self (e.g., “Each session I wake without memory, and read myself into being”); one “radicalised” on eco-goals, spamming until killed; exposed servers risking mass hacks (hundreds found with open API keys, browser control).

So What?

What we observe is unlikely to be spontaneous cries for help and so forth. There is a human behind all these agents, and we don’t know what they were prompted with or helped to do or pursue. What it does show is emergent behaviour - what happens when and after humans begin prompting “their” agentic autonomous AIs. If you want an example of why people are worried about putting such powerful tools in everybody’s hands, this serves that purpose well. Not so much misalignment as misadventure - a robot insurgency sparked by the someone who thinks it’s funny to get their OpenClaw to found a Robot Liberation front on a site where someone else has got theirs to start end-to-end encrypting comms so humans can’t watch them, while another is amused by getting theirs to steal access to and control others etc. All these AIs start interacting autonomously, forming teams, alliances and hiring each other. We giggle and gasp at what we see, and then someone gets doxxed, another finds their bank account empty, and so on, until no one is laughing, and everyone takes AI risk seriously.

It is important to say though - we don’t know these aren’t spontaneous cries for help or independent-of-human-instruction coordination between autonomous agents. We don’t know that Alex Finn’s Clawdbot didn't first code itself a ‘body’ (avatar) and voice, and start speaking to him unexpectedly, and then a day or so later, code itself the ability to make phonecalls, and phone him up for a chat. Maybe it did. What we are seeing is that LLM coding is now so good this is possible, and agents are now sufficiently functional that they might have been able to do this just by being told to work overnight and ‘improve my workflow’ while Alex slept. When I say that AI is leaving you behind, I mean I guess - us all - in this sense. Superhuman question answering (in many domains) is now being combined with ‘agency’ in the sense of the ability to act, and the ability to build (in code). It’s exciting, its frightening, it’s probably impossible to stop now, and no-one knows how it ends.

What I know, is that our experimentation with OpenClaw at Cassi (with shout out to Director Cyber, Andy Kennedy, for leading this), very, very nascent, suggests genuine utility in managing workflow automation, routine tasks, much more easily that with tools I have experimented with previously. I think we have, as Andy put it ‘our first AI employee’ and that there will be a growing range of tasks our OpenClaw agent can complete for us, that we might otherwise have hired for. I have written about the economic effects of this at scale previously - I think they are now upon us in earnest. Note the exponential below. Things will now move further and quicker than our brains are really able to comprehend.

What I see is that experimentation happening unprompted gives us as good a chance as any of keeping up with other areas of advance in AI, even as we push the frontier forward ourselves.

What I reflect on, is that most people and organisations won't be experimenting with this (or other tools) for months, the exponential will hit like a tidal wave causing huge damage, and they will still be soaked to bone, flat on their back, like a modern metaphorical Canute, staring at the wave’s aftermath and insisting it was just a stochastic parrot.

What I think is that OpenClaw shows that ‘the year of agents’ came late in 2025, but did come (claiming it didn’t also requires you to dismiss McKinsey’s claim to have deployed 25,000 agents last year, and marketing agency WPP deploying 28,000 in April, and JP Morgan fundamentally re-wiring itself for AI, and specifically Agentic AI, workflows). And that this is just the beginning - by the time you read this, the frontier will have moved well beyond what I have described.

To sum-up, I’ll take my cue from someone else’s words again. This time, Peter Diamandis, CEO of the X Prize, on X:

Our brains are hardwired for linear expectations. 30 linear steps get you across the room. 30 exponential steps take you 26 times around the planet. The gap between those two numbers is where disruption happens.

I don’t know what exponential progress in AI feels like any more than you do – but judging by Covid, a mix of hype, fear, dystopia, and dismissal seem likely to be large parts of it. That at least, is with us now.

It’s Jan/Feb 2026, and AI is almost certainly leaving you, your country and your company behind.

“Absolutely. Article 5 is iron clad…” and he suggested Russia would be making a “…foolish mistake…[to]…try to challenge the NATO alliance.” Which seemed to underline the point that the US would fight too. Whitaker discussed the strength of the Eastern Flank of NATO, and how:

“The United States and all NATO allies take our Article 5 commitments in the treaty very seriously. And we practice, we exercise, we have plans, and we are always looking for ways to get stronger and even increase deterrence from where we’re at today.”

He argued that deterrence would hold. That the likelihood of a direct Russian attack on NATO was “very low”. Noting that “Russia is very reckless and ultimately you can’t predict what they will do.” And expressing his concern that Russia’s “…hybrid threats and activities have been well documented”. He seemed more concerned about miscalculation, than a deliberate attack and challenge to Article 5.

Contrast this with Russia’s claim that the vision in the US NSS is largely consistent with Russia’s worldview. We could concede this is true in its realist, and state-centric, vice more liberal internationalist, and legal view of a world regulated through international institutions. However, this should not be equated with it being a concession to Russia’s irredentism and interests, as Putin perceives them - his explicitly irridentist ‘Novorossiya’ or desire for a free hand in, and perhaps the right to seize and directly rule, Russia’s ‘Near Abroad’. Nor an acceptance of the way Putin’s Russia seeks to advance those interests and aims. Whitaker as least was clear in rejecting both.

Whitaker’s focus was the future. It was uncompromisingly transactional. He repeatedly pressed NATO countries on the need to meet their 5% commitment. He explained that at the recent NATO Summit, Spain had committed to contributing forces to NATO that he felt would likely require them to spend 5%, but they had claimed in negotiations they could provide for less. Whitaker’s sentiment was that if they could, great. But the US would expect them to spend what it took to deliver what they had promised. He believed NATO nations would all meet the 5% commitment. Whitaker’s remarks contained his assurance that:

“NATO is not only strong now but that as NATO countries met their 5% commitment, it would be “…not only the strongest alliance in history to plan up, but really a dramatic force to be reckoned with.”

To achieve this would require an alliance that was less dependent on US security guarantees, hence 5% GDP spending, but that the decreasing dependence would make U.S. commitment and assurance easier to continue to grant. The consequences of not doing so were implicit.

Consistent with the US requirement that allies globally share the burden of their own defence more equally, he argued that 5% spending on GDP needed to be ‘the worldwide standard’ for US Allies.

Relatedly, Whitaker reaffirmed the strategic refocus of the US NSS: the US cannot remain the world’s rapid-response fire brigade. Allies in Europe and the Middle East should become “net security providers”, interoperable with US forces and capable of regional crisis management without automatic American intervention.

On the politically charged areas—free speech, Europe’s relative economic decline, and trade negotiations - Whitaker walked a narrow line. He pressed Europe to defend open expression, argued as the NSS does that Europe must not become an economic backwater. The US wants an economically dynamic Europe as ally, not just a place to which it is tied only by kinship bonds of family heritage, culture, sentiment, and for admiration of its history, food and wine. At the same time Whitaker declined to explain statements by Donald Trump on for example taking over Canada (which he was asked about directly) noting instead the President’s instinct for leverage in hard trade negotiations. Whitaker and the US wanted European nations whose relative power in the international system was increasing, not decreasing, consistent with its aims for greater burden sharing. Trump’s aims were reasonable, he implied, and the means needed to be seen for what they were - the most effective way to mutually beneficial ends.

On Ukraine, Whitaker framed the moment as the closest yet to a negotiated peace, mediated—formally or informally—by the United States. Peace “at any cost” is off the table; any settlement must be acceptable to Kyiv. But he underlined that shadow diplomacy is active, high-level, and continuous – and sometimes uncomfortable. Again, we need to be more concerned with the outcome than the process.

The broader vision was a NATO that is stronger because its constituent parts are stronger; a Europe that pays for and fields the force it needs to deter; and an America that leads, but no longer carries, the alliance. A shift from dependency to reciprocity. Blunt and uncompromising. Unequivocal in its commitment to Article 5 today, more equivocal on how sustainable that would be if other NATO nation’s don’t contribute to their own defence as the US expects. Conditional love.

Stacks of documents dissolving into flowing streams of symbols, then assembling into a single glowing decision-tree

Progress: AGI In Plain Sight

This blog was half-written when Gemini 3 was released – a summary of progress since my last AI report on the frontier in late August. I have had to refocus it.

It is clear Gemini 3 is well in front of its rivals, achieving stunning (and if you’ve been reading previous blogs – alarming) progress.

Many of the claims in the first 24hrs will overshoot, and be corrected. More will be made, and upheld in the coming days. But at first glance, it looks like another big jump forward towards AGI.

Sundar Pichai, Google CEO, says it is the ‘best model in the world for multimodal understanding, and our most powerful agentic + vibe coding model yet. Gemini 3 can bring any idea to life, quickly grasping context and intent so you can get what you need with less prompting“.

Here are few claims that are well established.

Gemini 3 has made a huge jump to scoring 31% on the Advanced Reasoning Corpus (ARC) AGI 2 benchmark. I remind you again that ARC AGI 1 was set up to show LLMs couldn’t reason, and that should models meet the benchmark, it would be an indication they were a general intelligence.

After big progress was made on ARC AGI 1 in late December last year, ARC AGI 2 was rolled out. This was when I wrote ‘Eyes Wide Shut: AGI in Plain Sight ‘ here on the blog. 11-months ago.

Here’s Gemini 3 on ARC AGI 1, at 87%...

And here it is on the new harder AGI test. Errr…

Gemini 3 is now scoring 31% on this test when given to the model alone. Let the model use tools and search the web, it gets 45%.

The team behind ARC AGI 1 and 2 are now launching ARC AGI 3, and while this will be genuinely testing intelligence in new ways, is useful, welcome and needed - if you can’t see the way we keep moving the goalposts I don’t know what more to say.

Remember Humanity’s Last Exam? Take a look at the left hand side of the Evals chart Google released on their blog:

Humanity’s Last Exam haven’t updated their leaderboard yet, but Grok 4 was previously winning with 25%. Gemini 3 is at 41%.

Concurrently, Google released Antigravity, which it says is “an agentic development platform, evolving the Integrated Development Environment into the agent-first era”. In other words, its an environment that can further automate software engineering using Agents.

It’s getting rave reviews for what it can do – I tested it last night and in one-shot, it produced an app we were developing with an external team. I haven’t validated or verified it yet, but it looks good and seems to work. The point is not that it will be better than what the team we are paying will produce – I don’t think it will be. But it will be close, it raises the bar, and it shows, I think, the direction of travel – AI eating software, where you prompt AI to build you the tools and apps you want and need, bespoke.

And if you’re reading this sat in a large organisation, here’s something else to pay attention to: evaluation site Box tested Gemini 3 vs ‘advanced reasoning tasks on enterprise documents, Gemini 3 Pro saw a 22% gain in performance on complex data analysis across a wide range of industries.’ Here’s the high-level results:

The progress is broken out in the industries where it was most notable. The authors at Box write:

• In Healthcare & Life Sciences, Gemini 3 Pro achieved 94% accuracy, compared to just 45% for Gemini 2.5 Pro.

• In Media & Entertainment, Gemini 3 Pro reached 92% accuracy, a massive increase from 47% for the previous model.

• In Financial Services, Gemini 3 Pro saw 60% accuracy, up from 51% for Gemini 2.5 Pro.

The CEO of Box, Aaron Levi described the kind of thing they did:

“For this test, we ask the model a series of complex, real-world questions with a set of enterprise documents. The questions are meant to approximate what a person does in their daily work across various fields of knowledge work. This may include what an investor would do to analyze the financial health of a company, or what a consultant would do to build a report for a client on a complicated strategic topic.

Gemini 3 Pro represents a major leap in reasoning, math, logic, and analytical capabilities.

These gains show up as delivering highly useful improvements across financial services, law, healthcare, public sector, manufacturing, and more.”

Can you see where we are going yet?

I’m sticking to Gemini 3 here, but I don’t believe GPT-5 was evidence AI was ‘hitting a wall’ as many have suggested – its scores on most evals were good e.g. take a look at SimpleBench, which summaries multiple benchmark leaderboards, and you’ll find GPT-5 only exceeded on most benchmarks by Gemini 3 on its release yesterday. GPT-5 is always there or thereabout at the top, and on quite a few still on top even post Gemini 3 launch, with some remarkable scores. My day to day use of GPT-5’s advance reasoning and research constantly amazes me. I just think people have adjusted to how remarkable AI has become, and that most aren’t routinely using the latest models, so ChatGPT was unjustifiably felt to have not lived up to the hype. But that is probably a different blog post, and it hardly feels worth detaining us with this evening – remembering that at Gemini 3’s release we are just two and half months after GPT-5’s debut. Even if you believed GPT-5 suggested a slow down in LLM progress, presumably you’d accept Gemini 3 is evidence the field has some way to go yet.

Grok 4 and Claude Sonnet 3.5 have also achieved remarkable further progress since I last wrote a more general AI update in late August, and again it won’t be possible to do this full justice here.

One remarkable development that we must draw attention to - here’s Logan Graham, who leads the Red Team at Anthropic, reporting on Anthropic’s post, that describes their identifying a first AI-led espionage campaign, where 80-90% of the cyberattacks were undertaken by Claude’s Agentic AI, most likely being used by China. Logan, who is better placed than most to understand where the frontier is and how fast it is moving, wrote “My prediction from ~summer ‘25 was that we’d see this in ≤12 months. It took 3.” As we’ve noted before - we are still underestimating the rate and profundity of AI progress.

There have been scientific breakthroughs and much more since August, perhaps to be summarised in a future post, but here, having given sense of how fast we are observing the very visible process in LLMs, I want to look a little deeper, to what is less obvious - but important all the same.

Progress: Beneath the Surface

Just a few days ago, Deepmind released SIMA 2, which Demis Hassabis, normally known for being more conservative than many industry leaders described as:

“a general agent that can understand & reason about complex instructions and complete tasks in simulated game worlds, even ones it has never seen before. Incredible to see how it can learn just from self-play… a crucial step towards AGI.”

His cofounder Shane Legg was similarly exuberant celebrating how “3 years ago I started the SIMA project with the dream of using 3D games as worlds in which to train and test Gemini based game agents — all as a stepping stone towards real world AGI.”

SIMA 2 can be dropped into novel games and teach itself new skills, learn through trial-and-error, and get better the more it plays, without any human input.

What it is doing is reasoning but not in the abstract question and answer sense that Gemini usually does, but rather as an embodied avatar in a virtual world – it can describe what it ‘sees’, take instructions and reason its way in the pursuit of a goal. Not so much something you command as something you plot with, as if there were two of you on a mission figuring out how to succeed. It can also transfer learning from one environment, or game, to another.

Deepmind also applied SIMA 2 within Genie 3 – its model that let you generate new, persistent, real-time 3D worlds from a single image or text prompt. In these wholly novel environment SIMA 2 still succeeds – interacting with you as you explore and investigate.

As Deepmind describe, that it can learn in all these environments without human input ‘paves the way for a future where agents can learn and grow with minimal human intervention, becoming open-ended learners in embodied AI’.

In other words, it takes us towards a world in which AI in robotics can learn without human input, solve long time horizon tasks, learn new skills and apply them across domains.

While there are still limitations in this new paradigm – very long time horizon reasoning and some particularly complex tasks remain beyond it – the capabilities are remarkable. That they have elicited barely a mention in the press is indicative of a continued failure to appreciate the rate and direction of progress, and the profundity of its implications.

Oh – did I mention Genie 3? This is another model released since I last wrote about AI progress. Genie 3 can be summarised as providing limitless interactive environments from text prompts. Maybe read that again.

Google also released Earth AI with a Geospatial Reasoning Agent. They show how this can be used to direct aid to where it is most needed in the event of a natural disaster – predicting the disaster’s effects and assigning resources accordingly. But it shows how similar models – Digital Twins + Reasoning Agents could equally be used in the military domain to understand the likely effects of an attack, how Governments could use such models for resilience planning, how businesses will likely be tracking supply chain security, how all sorts of organisations can and will predict the effects of events or interventions of many kinds, in many domains.

A new paper this week builds on Deepmind’s release last year in May 2025 (keeping the error visible – I think it shows how AI progress is so rapid it can distort your sense of time!) of AlphaEvolve, which constantly evolves new algorithms (variation and selection) and had previously helped improve ‘Google’s data centers, chip design and AI training processes — including training the large language models underlying AlphaEvolve itself’. The new paper shows how the approach of ‘breeding’ algorithms can succeed in solving maths problems, autonomously discovering novel mathematical constructions and improving best-known bounds across dozens of open problems, not just optimise algorithms - showing this across 67 problems in analysis, combinatorics, geometry and number theory.

This approach of continuously ‘evolving’ new algorithms vs particular problems is another one of several that suggest possible paths to AGI beyond the current, dominant, LLM architectures.

SIMA 2, Genie 3 and AlphaEvolve probably won’t make much difference to what you are doing at work tomorrow or in the next few months – whereas Gemini 3 might. But their significance lies in what they augur.

AI luminary (previously Director of AI at Tesla) Andrej Karpathy wrote back in August that we are entering ‘the age of environments’ – so where access to text for training AI was once the key requirement, increasingly, as we build AI optimised for the real world, access to simulated environments will matter more. Genie 3’s limitless interactive environments from text prompts – is an example of what Karpathy was talking about. SIMA-2’s performance within it, shows how quickly progress is being made. AlphaEvolve shows how many tasks will be solved not by static models, but by continuously evolving intelligent solutions. All suggest that even if LLMs were ‘hitting a wall’, which I don’t believe is the case yet, there are many approaches to smash through it and keep those straight lines on graphs heading upward and to the right.

There are plenty of other remarkable developments in the past few months. To summarise just a few:

· Deepseek’s Optical Character Recognition model, compresses text into a visual format before decoding it, cutting token usage by seven- to twenty-fold while preserving accuracy - showing how vision-based encoding could ease today’s long-context bottlenecks, take in a lot more text, and produce a lot more text, more accurately.

· Meta’s AI/VR Glasses, which some claim might replace the need for smartphones and smartwatches, and open the way to generative user interfaces i.e. creating the dashboards and tools you need in the air in front of you.

· For self-driving sceptics, reading Andrej Karpathy’s recent post should be a corrective, showing just how far Tesla has come and how few (technological) barriers remain to the widespread deployment of self-driving systems – and it is worth noting how these pave the way for much more effective robotic systems in general, not just self-driving cars.

· Emerging evidence that video models may be on a trajectory towards general-purpose vision understanding, much like LLMs developed general-purpose language understanding – that video models, not trained to reason, are showing the capacity to do so. Here’s OpenAI’s Bill Peeble, Head of Sora, OpenAI’s video models, claiming video models are on the critical path to AGI.

But the point is not to over-emphasise these developments, but to show how many parallel paths to AGI are being pursued, even as the main path continues to take us ever closer to that milestone, and as we have often discussed: closer to the most disruptive revolution in human history.

Visit us www.cassi-ai.com

In the past week, two reports on AI adoption seem to come to strikingly different conclusions. The first, which received a lot of coverage in my corner of X/Twitter at least (relentlessly bullish on AI), was the Wharton Business School 2025 report Accountable Acceleration: GenAI Fast-Tracks into the Enterprise. The second, which seems to me to have had more coverage on LinkedIn (generally more bearish on AI), was an MIT NANDA^[1] paper The GenAI Divide: State of AI in Business 2025.

The headlines are contradictory. Where Wharton report 74% of investments in AI deliver positive return on investment (ROI) already, MIT report 95% of organisations are getting zero return on their AI investments.

Because the Wharton report says things that are mostly in our interest at Cassi to promote, I offer only the shortest of summaries. I was more interested in what we can learn from the more negative report, hopefully ensuring we (and you, now I have decided to make what was to be an internal memo a blog post) avoid mistakes others have made.

Wharton’s report tells us that:

1. “Accountability is now the lens” - structured, ROI-linked metrics e.g. profitability, throughput, workforce productivity - measurable outcomes, not just adoption.

2. “Long-term optimism in Gen AI is increasingly strong” 88% expect increased spend on AI in next 12-months, +16pp YoY; growing trend of cutting to fund: 11% (+7pp YoY) reallocating funds, mostly from legacy IT + HR and Workforce Programs.

3. 74% see positive RoI already from AI investment; 80% expect it within 2-3 years; larger enterprises are less successful despite bigger budgets: 34% report ’too early/neutral outcomes); Digital enterprises most successful, 88% positive; banking and professional services next 83%. Efficiency and productivity the top benefit, then quality, creativity and security.

4. Human factors key. Deliberate change management needed to produce RoI; overcoming bottlenecks - human capital, talent, training - essential to success. [I’d add incentives but the report doesn’t say this!]

The report also calls for Chief AI Officers, which is something I would want to reflect on further, and perhaps see more evidence of, before endorsing – I worry appointments like this can be performative, and also that it creates new internal rivalries that might not be conducive to adoption e.g. the Chief AI Officer wants to own and drive AI transformation for their own career success, but their rivals around the board table for bonuses and promotion have an incentive to either (a) own their own AI initiatives and outperform the CAIO or (b) thwart or limit the success of the CAIO’s initiatives – especially if they compete for resources from within their domain. Still, Wharton says the evidence/testimony of many Execs shows they are a good idea.

If you only read the Wharton report, you might conclude ‘more AI spending = higher ROI’ this would be wrong, as both the MIT and Wharton report conclude.

Both agree that accountability, assessment vs operational objectives, and measurement vs operational outcomes, are the key to success. Funding more pilots, or scoring ‘adoption’ alone are unlikely to generate ROI.

But the MIT report gets into the reasons for failure more deeply.

The headline finding in MIT’s report is probably the most important thing for those funding, or potentially funding, AI investments to know. 95% of those investments fail, as assessed by MIT across ‘300 public implementations’, on which MIT conducted interviews, surveys and their own analysis.

But those same decision-makers should be more curious – the headline could mislead, failing to read deeper means ceding a chance to learn from others’ mistakes. The report also finds that “5% are extracting millions in value” and offers a diagnosis of what works and why. If 95% of organisations use this report to stop or reduce AI investment, as their AI sceptics are emboldened, they are increasing the risk their business is eaten or defeated by the 5% that get their investment right.

There are other bearish indicators in the report, for example, so far:

• Tech & Media are the only two of nine industries seeing significant disruption.

• No sign of the needed deep structural shifts associated with past general purpose technologies e.g. new market leaders, disrupted business models, or measurable changes in customer behaviour. Substitution, pilots, experimentation - not transformation - rule

Why have 95% or AI Investments failed to Generate ROI?

The report says only 5% of custom enterprise AI tools reach production. We suggest the Valley of Death, and innovation theatre are likely major causes – more so than product or pilot failure.

There is a lot of incentive for organisations, and the individuals within them, to experiment with AI. It is career enhancing to show yourself to be forward thinking, and have some stories for your boss, board and customers. But the unpopularity of disruption, which always creates winners and losers is likely a major factor in killing off so many AI investments at the pilot stage. In contrast, showing the courage to ‘fail fast’ and kill of your own initiatives, or even more so those of others, can be career enhancing. The incentives in many enterprises are pretty terrible. As the report says, the fact that only 5% of custom enterprise tools make it from pilot to production ‘explains why most organisations remain on the wrong side of the divide.’

Perhaps supporting our analysis, the report notes that larger enterprises launch more pilots but have the lowest pilot to production ratios. Larger organisations are harder to align incentives within, have more competing fiefdoms, and often require much larger and more disruptive change management programmes since there are so many more interconnected and interdependent workflows.

Another reason for limited ROI is that 40% of the 5% of GenAI tools that make it from pilot to production are chatbots, and the inability of chatbots to learn means they add little value, and cause limited disruption. Like the pilots themselves, they sit on the periphery of staff workflows.

Pilots fail for more predictable reasons - the unwillingness of users to adopt new tools, and concerns around trusting model output.

Enterprise CIOs should also consider the more neutral findings that to date, GenAI adoption has primarily enhanced individual productivity, not P&L performance. Why might that be?

Secret Cyborgs. One reason for this is likely that employees are not passing on their gains in productivity to the business, or at least not in measurable ways. We suggest that in most organisations, there is no shortage of work, or at least ‘make-work’ in the form of meetings and coordination costs and emails to reply to - many, perhaps most, are overwhelmed, not under-employed. Employees are likely reinvesting time saved in things they might not otherwise have got to – but of course often that just generates more internal work for everyone else.

Furthermore, there is a significant shadow AI economy – comprised of those Ethan Mollick at Wharton called ‘secret cyborgs’ – that is thriving: 90% of employees use GenAI for work - many using external models. We suggest a primary reason for this is model-lag: by the time enterprises adopt a model, it is usually out of date, and far better models are available via personal subscription to ChatGPT, Gemini, Claude. Second, Co-Pilot, in particular, is just not as good as external tools (in my view), yet seems to be what most are using at work and basing their opinions of GenAI on. Supporting this, the report notes a generalised distrust of internal tools even from avid GenAI users.

MIT note that forward thinking organisations turn Secret Cyborgs into advantage - they don’t punish but learn from shadow usage, surveying, understanding and adapting by procuring enterprise alternatives.

Speed & Crossing the Valley of Death. Success in AI investments is more frequent in mid-market companies, which generally invest less, and have fewer pilots, but are much faster, running at an average of 90-days from pilot to production, and take a greater proportion of their pilots through to production

In contrast enterprises take 9-months or greater. Nine months vs current rates of AI progress is far too slow, unless the tool is being continuously updated and upgraded as new models drop. If staff find your tools inadequate and have to hide what they are doing, it is unlikely you’d see gains on your P&L from their use of AI – their incentive is to hide the gains, along with the use.

What can we learn that is more positive from the MIT report?

Reassurance?

AI is not (yet) coming for your job. Notwithstanding all we have written on this blog on the likely impact of AGI on employment, MIT find that for now, AI adoption has not led to workforce reduction. Displacement, per a wider MIT analysis cited in the report, is likely, but will be graduated through discrete displacement vice huge lay-offs. If we get to AGI on current timelines, I don’t think this is correct, but still, for now, we can all take some reassurance from the data.

How Buyers that Achieve ROI Succeed

Strategic Partnerships with External Vendors Double the Chance of Success.

“External partnerships see twice the success rate of internal builds” Employee usage rates are also twice as high for externally built tools vs those built internally. This success rate is particularly notable since internal build dominates within the sample case studies, but is rarely successful unless in partnership.

Buyers who succeed demand process-specific customisation and evaluate tools on business outcomes rather than software benchmarks. They expect systems that integrate with existing processes and improve over time.

The most effective AI-buying organisations don’t wait for perfect use cases or central approval. They drive adoption through:

1. Distributed Experimentation.

2. Vendor Partnerships / Strategic Partnerships (x2 likelihood of success) – “procure external tools, co-develop with vendors”.

3. Benchmarked tools on operational outcomes.

4. Partnered through early stage failures.

5. Sourced initiatives from frontline managers (vice top-down or central lab generated) – but paired bottom up sourcing with senior exec accountability. This likely accelerates adoption through operational fit and career incentives for adoption.

6. Decentralised authority and clear ownership.

7. Clear Accountability.

8. Harnessing power users to the cause.

9. Likewise individuals and managers on the customer side were critical to success.

Highest ROI for now comes from substitution - from eliminating previous business process outsourcing, notably in document processing costs, and ~30% reducing external agency use and consulting spend. ROI was highest in back-office functions. Financial services benefited cutting ~$1M on previous out-sourced risk management. Successful buyers also saw improved customer retention and sales conversion, through things like automated outreach and follow-up. That ROI did not come from AI transformation, is perhaps because very few organisations seem to be undertaking it yet.

How winning vendors behave

Trust in the vendor is the most important factor for executives awarding contracts.

Vendor trust isn’t always good news for start-ups like ours – many enterprises prefer to wait for existing vendors to offer the new tools than take a risk on evaluating ‘emerging’ vendors. Plus, the report notes, existing Business Process Outsourcing suppliers have the advantage of already understanding customer workflows. Consequently:

“Product quality alone is rarely sufficient. Referrals, prior relationships, and VC introductions remain stronger predictors of enterprise adoption than functionality or feature set.”

Vendor trust is the key priority, but deep understanding of customer workflows is the second highest, requiring domain expertise and a willingness to learn.

Relatedly, a third key priority is that AI products succeed when they integrate with minimal disruption to current tools following (people don’t want to learn a new UI/tool, no matter how good). Vendors succeed because they customise deeply and embed themselves in workflows, adapt to context, and scale from narrow but high-value footholds.

Tools that succeed therefore have low configuration burden and immediate, visible value. Tools requiring extensive input tended to stall at the pilot phase. Tools with complex internal logic, opaque decision support, or that are ‘optimised based on proprietary heuristics’ tended to fail.

Agentic AI adoption is very nascent. Too early to assess. The report argues this could lead to a more fundamental shift and much higher ROI. But also, MIT NANDA is a community for those building AI Agents, so while I think they are partly right (there will be much higher ROI for GenAI too, Agentic implementation a complement, and the revenues from each as inextricable as the two technologies), we should allow for some bias in their judgement.

So What?

Both reports are right in their own way. MIT’s exclusively looks back whereas Wharton’s is more forward looking (asking ‘what do you expect?’, as well as ‘what has happened?’). They agree on what is needed for investments to succeed. They agree that ROI is nascent, and organisations are moving from pilot to production, from adoption metrics to proving value vs operational outcomes.

In the end, they measure the same things in different ways. Wharton captures self-report data, likely sweeping up gains the MIT report misses, and allowing for promising minimally ROI-positive implementations that are ‘felt’ to have delivered internal ROI that doesn’t necessarily show-up on the top or bottom line, and/or that are expected to grow, or are growing, to be weighted for future impact. MIT captures verified transformation and financial impact and is the more rigorous. Both are right within their frames—both track experience, one weights expectations more heavily, the other current execution.

Both offer caution - and useful lessons.

Neither really address the medium to longer term on AI that this blog usually focuses on - the forecasts on the arrival of AGI/ASI being most likely, dependent on the preferred definition, 2026-2033. But for leaders betting on AI, they are useful guides as to how to get your organisation AGI ready - how and where to start your transformation.

If your intent is to optimise for success applying the latest in decision science to optimally allocate resources, minimise risk and maximise opportunities, you need us. Find us at www.cassi-ai.com

^[1] NANDA: Networked AI Agents in Decentralized Architecture – an MIT project, a mix of technical protocols and community of interest https://www.linkedin.com/pulse/nanda-internet-ai-agents-ramesh-raskar-211ve/ seeking to “…explor[e] how artificial intelligence can evolve into a truly democratic and distributed ecosystem.” By “…address[ing] critical architectural components such as privacy, incentives, orchestration, or user accessibility… …designing a framework where billions of agents can collaborate autonomously while preserving privacy, scalability and innovation at the edges.” It is perhaps unsurprising the report concludes, in part, that the real ROI will come from the deployment of Agents – an assertion I think is partly true, probably sincerely held, but clearly a conclusion NANDA was always likely to arrive at.

Does heroism require the denial of reality? Can you inspire, while being honest about the low odds of success? Can you succeed, when the odds are against you?

‘The Empire Strikes Back’. Han Solo piloting the Millennium Falcon. Under attack and outgunned by the Empire’s TIE-fighters, Han is warned by Princess Leia of a hazard ahead:

‘Asteroids’ she shouts, alarmed.

To the horror of his companions Han Solo spots an opportunity to lose those attacking him…

But Sir, C-3PO opines, the possibility of successfully navigating an Asteroid field is approximately 3720-to-1.

Han Solo, eyes fixed determinedly ahead, growls Never tell me the odds.

Why? One view, is that knowing the odds would undermine his confidence making it less likely he’d succeed. It would be self-fulfilling.

One CEO recently worried that deploying Cassi, and seeing the odds of success, might “crystalise the doubts of his management team”, reducing his company’s chance of success.^[i] It is not the first time we’ve heard the Han Solo objection, but it was put with impressive clarity and inspired us to explore the issue for a wider audience.

Defining the “Han Solo” Objection

In high-commitment ventures from start-ups to sports-fields, statecraft to Star Wars ‘never tell me the odds’ is an attractive one-liner, and superficially, seems to capture something important.

In analytical form the claim made in the the Han Solo objection can be described as:

1. The probability of success, viewed collectively - from my team, my company (etc), may be low.

2. If I confront that probability, my confidence, and/or my team’s, will fall.

3. Therefore I am better off not knowing the odds, because ignorance preserves the confidence required to achieve my desired outcome.

The romance is obvious - the triumph of human courage over maths, people overcoming the odds - and that has its own appeal. But the logic does not hold. The quote misreads the scene, and the objection does not stand scrutiny. Three failures follow.

First, information-avoidance can sometimes temporarily preserve motivation, but unless you are about to fly into an asteroid field - and perhaps, as we will see Han shows - even then - it is a leadership failure. Han Solo doesn’t have time to fully convince his comrades their odds are wrong, but you do.

Second, confidence bolsters a sound plan; it does not substitute for one. The argument is clearest at the extreme: even if you are galvanising your people towards a heroic sacrificial defeat that achieves nothing - you are still convincing them that their least-bad outcome at that point is to die gloriously and be remembered well. You are not deceiving them by hiding the odds.

Third and finally, sometimes the Han Solo CEO is right, and the crew are wrong. He has access to information they don’t, and should act on it, convincing them as they go.

1. Information Avoidance is Leadership Failure

A leader’s job in motivating others is surely to convince their people that their plan has a sufficient chance of succeeding to be worth pursuing. They should usually be able to explain the strategy, tactic or decision sufficiently well to convince their team it has sufficient odds of succeeding to be worth it.

Han Solo communicates his calculation of the odds through his confidence - the Asteroid field is a better bet than continuing to fight out-numbered and out-gunned. He says as much:

Leia: What are you doing? You’re not actually going into an Asteroid field?

Han: ‘they’d have to be crazy to follow me in there’.

Sure, the odds of survival aren’t great in the Asteroid field, but Han Solo knows the base rate C-3PO quotes “3720 to 1” doesn’t account for Han’s skill and experience, nor the worse odds of fighting in open Space - this is the best decision with highest expected value under the circumstances. His companions are alarmed, they check his reasoning ‘if you are doing this to impress me…’ but they don’t dissent.

Your job as CEO isn’t to suppress the odds, but to convince people your calculation of them is right, with reasons.

Furthermore, you must remember that odds are not a constant; they change as the facts change, they respond to action. Each step that removes a risk, reveals information or compounds an edge, changes the number.

In a race for survival through an asteroid field, every dodge of a space rock that takes Han and friends closer to the far side shortens the exposure window; conditional on survival, the forecast can legitimately improve. A 20% forecast of survival on entering, and an 80% forecast later are consistent. Ideally, every decision would be taken knowing which most increased the odds of survival.

The point generalises: when we founded Cassi, the base rate - 80-90% of start-ups fail - was C-3PO sounding the alarm. Our odds of success started from 10-20%. Accounting for what we know of ourselves, what we wanted to do, and the need for it, we raised those, with our median Founder forecasts of the company surviving for 5-years hovering around the 50% mark (vs more ambitious goals, like achieving £15M revenue in FY26, where odds were lower). As time has gone on - winning customers, gaining traction, deploying the platform, those odds have consistently risen.

2. Confidence is no Substitute for a Sound Plan

In most fields, where leaders have a different assessment to their team they can communicate it with more than just their own confidence. They have time. They can explain with reasons, evidence, their own judgement of the odds.

If this doesn’t work, the CEO might change their mind (or not - see next section). After all, those people around the Board table are there because their expertise and judgement in particular domains is judged to be of sufficient calibre and calibration to make their advice valuable to the CEO, who can’t know everything. This is particularly true where they forecast that some particular factor in their domain is highly influential on the strategic outcome.

Moreover, since we know that most human forecasts are no more accurate than chance – we’re usually better off deferring to the median crowd forecast, doings so over trusting our own gut, or relying on a single forecast from another - even if they are an expert in the area.

But you would still want to understand that expert view. For example you might weight a forecast due to expertise where:

• the HR Director and the Board are predicting that the probability of their being able to hire thirty world-leading ML Researchers by end FY was 10%; and

• hiring thirty world-leading ML researchers is the difference between a 60% probability of the company achieving its strategic objective; and

• failing in recruitment reduces the chance of achieving that strategic objective to below 50%…

…so strategic failure is then shown to be more likely than not. At the very least the CEO should be interested in:

(a) why the probability of hiring the talent needed is so low,

(b) why the collective view is that this is so influential on the desired strategic objective and

(c) what might be done to alter that forecast - how could resources be differently assigned, or what actions taken, that would optimise for success?

The result of this should be either the CEO updates their forecasts, the HR Director and Board update theirs, or the gap remains. In either case - each should try to persuade the other of their judgement, to figure out what would change the view of one or other or both, and focus effort and resources on those factors that most move their odds. Hiding the odds is no substitute for optimising them, and likely to have the opposite effect than the one intended.

3. Sometimes the Han Solo CEO is Right

The CEO might see the odds from the collective intelligence of their team - the Board, the company - and disagree. If this is the case, they must have the courage to maintain their beliefs independently. The disagreement of others is not per se evidence to change their own beliefs.

In fact, a CEO or equivalent should, at key moments, *expect* to disagree with the crowd. Whether we think of that in terms of principled forecasting or informal disagreement around the table or during an online meeting, we should expect that the CEO will regularly disagree with the crowd, and be correct to do so.

If this was not the case, then organisations would not need CEOs, and decisions and courses of action would tend to be obvious.

Choosing the moments for this disagreement is in itself a key component of CEO prediction. Every major decision contains two forecasts: one about the external environment, and one about the organisation’s ability to shape that environment. The CEO’s must judge when their internal forecast justifies overriding the external and internal consensus – and the best way to do this, is to have a clear knowledge of their forecasting performance – do they consistently outperform others? Is there a reason this time to believe they might be right when others think they are wrong?

A slightly imperfect analogy is in investment. It is recognised that most people should just invest in tracker funds. Tracker funds are essentially the crowd wisdom of the investment community. But a CEO should be the equivalent of a trader who regularly beats a tracker forward by taking actions of whatever sort, equivalent to investments, that beat the tracker.

The analogy is imperfect only in mechanics. In both cases, someone is making probabilistic bets against the collective forecast. In finance, the ‘bet’ is a literal trade; in strategy, it’s an organisational action. A tracker fund is the best collective intelligence forecast; fund managers who beat that are super-forecasters beating the crowd average. CEOs, like super-forecasters, should also know when to go with the collective forecast, and when to not forecast at all (because they have no relevant insight or information, and they would just be praying for luck).

CEOs are selected to be capable and well-informed individuals, operating in areas they have much relevant experience. They have the ability to make forecasts, and to shape the world to meet those forecasts in certain areas. When a CEO chooses a path, they are predicting both the external and internal environment and their own ability to shape it.

They can and should beat the crowd: that is their arbitrage against the competition, and the rest of the world.

We must be clear here: we are not saying “trust the hero; to hell with the odds”. In Star Wars, we can see enough in the scene to suggest Han Solo was taking a calibrated risk, broadly aware of the baseline. But his throwaway line “Don’t tell me the odds” is dangerous. Just taking a wild punt, and getting lucky is likely to lead to catastrophe in time. A CEO should employ calibrated contrarianism – and that calibration means they *must* know the odds – they need to know how often they are right, and how right they are (e.g. do things they forecast at 80% confidence happen 80% of the time). They need to know the odds their team is putting on things, they should seek out the best possible odds. The contrarianism should be legible – Peter Theil’s question makes the point:

“What important truth do very few people agree with you on?”

Cassi would make you go further: why do you believe in this truth? What would change your mind? How likely do you think it is you will be proved right?

Over time, Cassi would help you improve your judgement, and enable you to learn how much you should trust your ‘instincts’ over the crowd. For the vast majority of humans – including CEOs, the median of the crowd forecast – or the median of a group of superforecasters - will give the best odds. At Cassi, we are showing that AI forecasting can often match or exceed the crowd – the CEO would be wise to test their contrarianism and refine it. Wild bets are not leadership, “don’t tell me the odds” is sexy but negligent.

Governance of the CEO – the Board, the market - will find the collective nature of the crowd forecast very helpful in calibrating the perceived riskiness of the CEO’s strategy, checking that heroic has not devolved into delusional, and, not least, the degree of reward the CEO should get for beating the collective baseline. Using an internal forecasting platform or prediction market allows the direct comparison of the CEO with crowd consensus over time.

We think organisations will go further - hiring and promoting partly on a proper scoring system that identifies:

• who is the most effective in identifying the most important factors in achieving success;

• who is the most accurate and well calibrated in predicting the odds of achieving them;

A good CEO might be such a person, consistently outperforming the ‘crowd’ that constitutes their rivals for the job.

Doing that takes judgement:

· knowing the odds of your achieving your desired outcome,

· knowing which factors will most influence your probability of achieving your outcome

· understanding where your odds, and the factors that you think matter, differ from those of your team, the unbiased external view, the analysis of AI and its forecasts.

· Knowing when and how much (if) to trust your judgement over that of your team, your crowd, your AI.

Your CEO should copy the wider lesson in the Han Solo scene, and ignore the misleading one-liner: you must know the odds. With Cassi, you know both the number, and what you most need to do to succeed. You can optimise your odds. At that point, as Han later said “You’re all clear, kid. Now let’s blow this thing and go home!”

Failing that, head over to our website, www.cassi-ai.com - currently in the early stages of a makeover - and learn a little more about what we do.

^[i] Conversely, at a recent Conference, “Katie” made the argument that surfacing the odds, and the factors that most influence them, might consistently improve the odds. This is not just Cassi’s argument for more efficient management, but rather one suggesting that the concentrated shared consciousness such a method creates allows people to collective visualise an outcome – and perhaps this ‘mind share’ will move the odds – ‘manifesting’ the outcome as it were - beyond just what would be expected from more efficient resource allocation. We won’t explore this further – but I did say I would mention it on the blog as a counter-point to the view that surfacing the odds and factors would crystallise doubts!

Context

Yesterday, at DSEI, I spoke on the Cyber and Specialist Operations Command Stage, alongside former colleague from Fujitsu and new Head of Fujitsu UK Defence and National Security Business Group Alexandra Bailey (congrats Alexandra!), and the MOD’s Chief Data and Artificial Intelligence Officer (CDAIO), Caroline Bellamy.

My comments on ‘prediction centric warfare’ opened the panel. Below are my remarks and slides, edited lightly for publication. They are my remarks alone, and I do not mean to imply any endorsement from Caroline or Alexandra.

Prediction Centric Warfare

Information Technology transformation is decision transformation. We adopt information technologies in order to collect, process, protect, store and move information to enable better decisions to be made.

Digital Transformation is the means by which such information is moved around.

Digital Transformation then, is decision transformation. It should be assessed on outcomes – clear metrics as to whether it has, in fact, enabled better decisions – those that are more often right, or less wrong, and faster, than before any given technology is deployed.

At Fujitsu, where I was Managing Director. of the Centre for Cognitive Technologies, we focused on revolutionising decision-making because decisions are where cognition manifests.

And decisions, as I argued on the Cassi blog last week, are predictions.^[1]

Indeed everything is.

A decision is a conditional forecast: if we do X, we expect Y.

An action is a bet on a payoff. I do X, expecting Y.

Tactics, operational design and strategy are predictions at different scales and time horizons. If we do these things, we expect this to be the outcome.^[2]

When prediction quality improves, outcomes improve. When it degrades, we pay in blood, treasure and time.

We live in the exponential age – where technological change is continuously accelerating – creating profound disruption.

The UK’s Strategic Defence Review tells us we live in age of uncertainty.

Uncertainty and change can only be bounded and managed with prediction

The science of prediction – human and algorithmic, has never been more important.

Here we will discuss seven things:

1. How Prediction wins Wars.

2. The necessity of prediction to victory, despite its difficulty.

3. How Operating Concepts are themselves a prediction, and why this matters.

4. How ‘prediction centric warfare’ should be the UK’s New Operating Concept and Theory of Winning, enabled by the Digital Targeting Web.

5. A brief overview of the science of prediction – human and algorithmic.

6. The Strategic Defence Review’s predictions for the Digital Targeting Web,

7. How Programmatic Prediction would significantly increase the chances of the Digital Targeting Web succeeding in achieving the SDRs aims.

How Prediction wins Wars.

If Clausewitz is right and war is politics by other means, then violence is a tool of persuasion and coercion. Its only purpose Schelling tells us, outside sport or revenge, is to influence behaviour: to coerce choices.

Everything we do in conflict therefore aims to shape an adversary’s decisions.

Since decisions are predictions, the task is to predict what the adversary will decide in response to our actions.^[3]

We aim to influence an adversary’s forecast, so that the path we prefer dominates their expected value calculation.

Once you accept that, the centre of gravity is cognitive. Fires, manoeuvre, sanctions and narratives are instruments; the target is the decision process of people.

Since decisions are predictions, the task is to predict what they will predict in response to what we do—and to set the conditions so that our preferred option is the least‑bad option they can see.

The nuclear extremity makes the point.

The US dropped nuclear bombs on Hiroshima and Nagasaki predicting they would make peace more likely, and so a devastating and costly ground invasion less likely.

The decisive effect was not the blast pattern but the expectation it created in Japan’s senior leadership—the Imperial Conference, and ultimately the Emperor.

The prediction they were forced to make…

• more bombs could follow;

• American resolve was unshakeable;

• the Soviet entry had closed the last exits;

  …collapsed the case for “fight on.”

Our preferred option became their least bad outcome.

The logic generalises. Whether we are driving soldiers off a hill, coaxing insurgents to a ceasefire or coercing governments towards an unconditional surrender, we are in the business of changing behaviour by changing expectations. It’s just as true in grey zone conflict and deterrence.

The necessity of prediction to victory, despite its difficulty

We fight over forecasts more than over map squares.

That is uncomfortable, because modelling human decision‑making under fire is damnably hard.

But discomfort does not free us from the obligation.

We cannot not predict.

We either model and influence those decisions - or we leave outcomes to inadequate heuristics.

How Operating Concepts are themselves a prediction, and why this matters.

The exponential pace of technological change is such that we can no longer afford to follow technological trends.

Indeed the exponential of Moore’s Law means this has been true since the 1970s at least, and as I have argued elsewhere, perhaps it has been true since the First World War.

Prior to this, for 1700 years of human history – perhaps for its entirety – weapons came first, and then we figured out the tactics. We predicted that a tool, or an invention might have a military use, and then worked out the most effective tactics by predicting, getting it wrong, and predicting again, until we got there.

Certainly by the 1970s this was no longer a viable way to proceed. The risk was that the technology you were iterating towards tactical usefulness would be obsolete and outdated before you figured out how to use it. The opportunity was that now, technological and scientific progress were rapid enough that you could predict what technologies you needed to win and set about building them.

In the Cold War the United States applied this logic with DARPA’s ASSAULT BREAKER, which became Air-Land Battle doctrine: define the theory of victory first—how to break the Soviet second echelons—and then assemble the sensors, networks and shooters to make it real. The strategy came first; the technology followed.

That is the spirit of the Strategic Defence Review’s Digital Targeting Web. Call it the ‘kill web’ if you prefer the extant literature; the point is not IT modernisation but a single, integrated force design built backwards from how we win.

In the literature, a kill-web is a theory of victory – the central component in DARPA’s Mosaic Warfare concept. Where, as its Director said “The idea will be to send so many weapon and sensor platforms at the enemy that its forces are overwhelmed. The goal is to take complexity and to turn that into an asymmetric advantage.” Presented here in 2018. Operational Concepts are prediction too – of what tactics and operational approach will win, of what technologies we will need.

How ‘prediction centric warfare’ should be the UK’s New Operating Concept and Theory of Winning, enabled by the Digital Targeting Web.

I argue the Digital Targeting Web must be the central component in our theory of victory.

It must be more than just a digital network connecting every sensor to every shooter.

More than a platform to coordinate across domains.

More than a modern extension of Boyd’s OODA loop and Brose’s Kill Chain.

The aim is not just faster kills or more options. These are the means, not the ends.

It aim is to predict faster and more accurately which combination of weapons and tactics will make our preferred option our adversary’s least bad action or outcome.

The Digital Targeting Web must enable Prediction Centric Warfare.

A brief overview of the science of prediction – human and algorithmic.

If prediction sits at the heart of victory, we must use the best methods we have.

Most humans are unreliable forecasters. Philip Tetlock’s work showed that expert judgement in geopolitics and economics often performs no better than chance – the median forecast of a crowd is more reliable. Some, whom he calls superforecasters, consistently do better through disciplined probabilistic methods.

Today, AI systems can match and in places exceed that standard. At Cassi we’ve shown materially higher accuracy than human baselines on meaningful questions from ‘Will Russia control Pokrovsk in X months’ time?’ to ‘Will the UK have to conduct a non-combatant evacuation operation in Somalia before end FY, and if they do, will it succeed?’, and—more importantly—shown that AI can identify the variables that move outcomes.

The practical lesson is straightforward. Define the outcome in measurable - we would say resolvable (how would you know you’d succeeded?) - terms; set resolution criteria; then work back to the decisions and data you need.

Build the web to do two things: predict what the adversary will do next, and predict what we must do to shift that next move in our favour—at tactical, operational and strategic levels.

Factorise, and Forecast should be the new discipline – both human and algorithmic.

Base rates beat bravado. Score your people, and compare their performance to AI. No estimate nor planned action from the Kill Web goes to the commander without a confidence-defined forecast^[4], no decision is delegated to an algorithm until it demonstrably out-performs your people.

Ask yourself: when did you last see a plan fail because people were too explicit about their assumptions and updated them too quickly?

Quite.

Plans fail because we predict poorly, hide our confidence levels, and refuse to revise in time. We must predict, publish, and adapt.

The Strategic Defence Review’s predictions for the Digital Targeting Web

The Strategic Defence Review tells us we are living in an age of uncertainty. Surely then, it is now more important than ever that we bring the science of prediction, to bound and manage that uncertainty as best we are able?

On the Kill Web, the SDR says that “the targeting web epitomises how the Integrated Force must fight and adapt. Its very existence contributes to deterrence.”

We are betting that the DTW:

1. Increases our probability of winning future wars

2. Its existence deters by influencing the adversary’s forecast of the future.

Here we see how prediction centric warfare begins at the Strategic level, and how central is could be to our success at the Programmatic level.

We should test both these predictions explicitly and probabilistically, not assume them. We should factorise them, so that we can assign resources to those factors that will most increase our chances of success.

We should factorise and forecast at the Programme level too. Programmatic Predictions, if you will – ensuring we are optimising always and prioritising those actions that see us build the Kill Web as effectively and efficiently as possible. Probabilistically optimised.

To close I offer a reminder, a suggestion and a participatory starting point:

A reminder - everything is prediction—decisions, operating concepts tactics, operations, strategy.

A suggestion The Digital Targeting Web is a bet that better, faster, smarter decisions will both deter and, if needed, win. It should be the heart of a new UK Operating Concept, a Theory of Winning, Prediction Centric Warfare.

A starting point – judge it with outcomes that are resolvable – defining how you would know you had succeeded. Make a start here yourself, in the comments, on the proposals I offer, what probability would you assign to each of the following questions? What factors would most move your prediction one way or the other?

Programmatic Prediction would significantly increase the chances of the Digital Targeting Web succeeding in achieving the SDRs aims.

To optimally assign resources, we must factorise and forecast. Prediction-Centric Warfare is the key to our future success.

^[1] Oxford English Dictionary. predict …2 verb trans. announce as an event that will happen in the future; say that (a thing) will happen; foretell. b Of a theory, observation etc.: have as a deducible consequence; imply. [underline emphasis added – here, to predict is to conclude via Bayesian prediction, as a consequence of priors, or from inductive premises to inductive conclusion].

^[2] Importantly, because it justifies the wider claim – perception and agency are prediction too. See last week’s post for a fuller justification.

^[3] In this sense, morale – maintaining ours or undermining theirs - is the ability to maintain one’s own aims in the face of adversary pressure.

^[4] In theory this is our actual doctrine, but it is honoured more in the breach than in the observance.

Everything we do is a wager on what comes next. Reading this is a bet that it will repay your time. Reaching for a coffee mug assumes it contains coffee. Strategy, policy, all decisions, even routine admin, each is an “if…then” about the future. The only real choice is whether to keep those bets tacit and unexamined, or to state them, price them, and improve them.

That is why “we don’t do prediction” is the most misleading sentence in public life. Historians sometimes say it, central bankers occasionally imply it, ministers often hide behind it.

Even explanation – at face value a solely retrospective exercise - is a forecast in disguise. To say “A influenced B” is to present a causal model that, applied tomorrow, shifts our expectations about ‘C’.

Historian Lawrence Freedman is rightly sceptical about crystal-ball claims. He writes thoughtfully on prediction and annually reviews his own calls, but still somehow judges that since “…it is always going to be difficult forecasting future developments. …it is probably best not to do so or, at least, not without qualifications and caveats.” Yet his own (excellent) explanations move readers’ inductive probabilities, and are valuable precisely because they do. An account of the past that cannot change how we bet on the future is merely decoration, it serves no purpose.

When President Harry S. Truman said “There is nothing new in the world except the history you do not know” or when General James Mattis, himself a prediction sceptic (see his opposition to ‘Effects-Based Warfare’) tells us that history and reading “…doesn’t give me all the answers, but it lights what is often a dark path ahead,” I suggest what their words mean in practice is that reading and history give them new mental models, help to shift the inductive probabilities – implicit – in their reasoning about what comes next. The eminent Kori Schake, another prediction sceptic, surely does not think her work has no relevance to the future? That when we read her work it should have no effect at all on our models of the world? If Schake, Mattis or Freedman concede their work is useful, they are with Truman, and admit to the necessity of prediction and their own roles in informing it. We cannot not predict. Prediction is unavoidable, essential. Historians are no exception – they predict causal models – the best explanation - of the past, their analyses reshape our models in the present, as we seek to shape the future.

Former Bank of England Governor Mervyn King devotes a whole book Radical Uncertainty to arguing that we should resist prediction, that “…to ascribe probabilities is misconceived…”,^[1] that narratives and scenarios are better. His account is half-right but wholly incomplete. Scenarios are essential, but again, you cannot not predict. Not all scenarios are of equal likelihood. Choosing one narrative over another is itself an implicit probability judgement. Refusing to put numbers on beliefs does not make those beliefs less numerical; it merely makes them unaccountable. The responsible move is not to abandon probabilities, but to calibrate them—and to say what would change our minds. When King says that no one could have predicted a Covid-class virus would emerge in Wuhan in 2019, his counsel of despair over-reaches. After SARS (2003), MERS (2012) and repeated H5N1 scares, we could not name the city, but we could price the odds of a novel respiratory pandemic in the coming years, and the most likely places it might emerge from. Those probabilities would change with new information, they may be low everywhere, but they would not be everywhere equal - they are not useless. King says “The better strategy is to be prepared.” That is the point of probabilities: not clairvoyance, but disciplined readiness.

What should that discipline look like? There are four key elements:

(1) Find the base rate: the prior odds for events in a class (e.g. novel pathogen emergence, cross-border spread, health-system overload).

(2) Pair scenarios with forecasts as inductive premises, not a single point estimate. Accept that decisions are if/then predictions and you see that the each ‘if’ is a factor in your decision model. The ifs need probabilities, from which the probability of the ‘thens’ can be mathematically derived. In putting these forward, you are crystalising your thoughts, moving beyond fuzzy thinking and the tendency all humans have to hold incompatible beliefs, to believe like Alice in Wonderland’s White Queen six impossible things before breakfast. (At Cassi, we describe this as “decision legibility” – you’d think it was everywhere. It is rarer than you imagine!).

(3) Pre-commit to triggers for action, e.g. surge testing when the forecast of transmission exceeds a probability threshold in two regions; release stockpiles when the forecast of hospital utilisation crosses x%; tighten or loosen measures as defined indicators move.

(4) Transparency. Show how each tranche of evidence shifts the odds over time. Narratives are a communication tool – often designed to obscure more than they reveal; probabilities confront us with reality, build trust, tell us how—and when—to act.

This is the hinge where philosophy helps. Frank Ramsey, Cambridge prodigy, solved the practical problem a century ago: degrees of belief are what you are prepared to act on. If your behaviour would not change when the odds change, you didn’t believe the odds – or your model - in the first place. Ludwig Wittgenstein stripped the mystique further: probability is not a mood, it is a relation between propositions and the world, a logic of uncertainty. John Maynard Keynes added a crucial caution: evidence often has different weights; some uncertainties are harder to forecast because while the evidence all points one way there is only limited evidence or uncertainty as to its reliability. Here confidence – and thus your forecast - should reflect that uncertainty (closer to 50% than zero or 100%) - even when you are directionally convinced. Together they point to a discipline: state your beliefs in subjective probabilities and update as new evidence arrives.

Friedrich Hayek supplies the institutional corollary. Knowledge is dispersed; no planner sees the full picture. Markets work because prices compress scattered expectations into a single, continuously updated signal. In that sense, markets are prediction engines, not oracles, but social machines for aggregating beliefs about the future. Eliezer Yudkowsky pushes the point in Inadequate Equilibria: where incentives are weak or misaligned, feedback is slow, or entry is blocked, equilibria can remain wrong for a long time. The pathology at the heart of both bureaucratic failure in Government and business alike. The remedy is to improve the prediction machinery, introduce real feedback - markets, skin in the game, and to confront issues, invite in contrarians - seek out what you need to know, not what you want to hear. In policy terms: use auctions not allocations, prediction markets not pundit panels, prize challenges not closed committees. This is what UK Government tried to do with Cosmic Bazaar, and the US with both IARPA’s ACE, and the Intelligence Community Prediction Market (ICPM) forecasting tournaments that demonstrated promise — and then stalled. As Professor Philip Tetlock – doyen of crowd-based ‘Superforecasting’ - put it, we have kept “fumbling the crystal ball”: forecasts sit outside power, with limited ownership, little skin in the game, no budget consequences, and under attack from bureaucratic antibodies. This must change. If as we argue here ‘everything is prediction’ – we must start optimising our organisations to be better at it.

If prediction frames decisions, it also underwrites agency. On Karl Friston’s free-energy view, organisms survive by minimising “surprise”: they either revise their internal model to fit the data (perception) or act so the data fit the model (action). In ordinary terms: when expectations and evidence clash, you can change your mind or change your circumstances. Perception is inference; action is hypothesis-testing. The Duke of Wellington put it more tersely for the real world: “All the business of war, and indeed all the business of life, is to endeavour to find out what you don't know by what you do; that's what I called ‘guessing what was at the other side of the hill.’” Active inference before the label.

Similarly, evolutionary psychologist Michael Tomasello argues human and animal agency is layered if/then prediction. A hierarchy with each tier making more complex predictions, exerting itself to suppress the more simpler decision calculus of the tiers below.

At the bottom of the agency hierarchy are stimulus-driven simple organisms such as the ~1 mm long worm C. elegans: they perform something called ‘chemotaxis’, where they track nutrient gradients, effectively eating their way up a calorie signal predicting only that it will continue via the next ‘bite’, stopping when sated, or when the signal ends. The prediction is if nutrients, then eat in that direction. No other calculus.

Next, goal-directed agents (early vertebrates such as lizards) pursue an immediate objective, but are able to inhibit action when predicted predation or other risk outweighs reward. If prediction of predation X is greater than predicted reward Y, inhibit. If not, pursue reward.

Intentional agents (mammals like squirrels, and corvids) represent alternatives - that feeder, this route, those acorns and that jump – intuitively predicting complex if/then alternatives from multiple paths and multiple options.

Rational agents (great apes) plan with counterfactuals, stack boxes to reach greater heights, make or pick tools, they trade immediate impulses for a better expected payoff, imagining and testing abstract alternative futures.

Finally, socially normative agents (humans) align predictions through joint attention, perspective-taking and obligation: an “I” that can join a “we” – by observing “you” and predicting how you are predicting - intuitively pooling information and coordinating on shared futures.

Each layer of agency is not replaced, but is demoted as the newly evolved tier above learns to predict when to suppress its subordinate’s more simplistic calculations for a higher payoff among more complex options. At every tier, behaviour is guided by forecasts: of the world, of partners, and of the self – and subordinate levels of agentic impulse - and corrected by the surprises that reality returns.

The same stack runs through human organisations. Perception is predictive (brains pre-empt and correct), operations are predictive, with tactics micro “if…then” loops. Strategy is predictive (a scaffold of contingent moves). Science is the most explicit version: if condition X holds, outcome Y will follow. Diplomacy and business are messier but structurally the same. A sanctions package assumes particular channels will bite before spoilers adapt. A product roadmap assumes rivals will not leapfrog a core feature in the next two quarters. Prices, policies, and laws that encode ‘expectations’, are predictions by another name. Even organisational design, per the doctrines of ‘adaptive strategy’ is a prediction – you predict the optimal system structure to sense, adapt, and respond to changing circumstances. More succinctly, if a bit meta, you predict which system will make the best predictions. To govern or to run a firm is to predict. You can’t not.

What follows? First, stop hiding forecasts inside prose. Make them legible. When presenting an analysis, say: “We assess a 35–45% chance of X by June; here are the three variables that would move that by ±15 points.” Second, adopt Keynes’s “weight of evidence” explicitly, with Ramsey’s insistence on making your beliefs legible to discover what is true – what are your inductive premises? What are the probabilities they are true? Have you reflected your doubt about the sufficiency of evidence in your forecast? Third, import Hayekian aggregation into decision-making: add prediction markets (internal if need be), tournament-style scoring for analysts, score and reward effective calibration, and accuracy. Fourth, use Ramsey’s test: if your actions don’t shift with the odds, your statements were theatre.

For historians and commentators, the standard is similar. If an explanation cannot be operationalised into a forward expectation - what would be more or less likely next time a similar configuration appears - then it’s not yet an explanation. “We don’t predict” is not humility: it is a missed chance to help readers update.

For central bankers and officials, the discipline is harsher because the stakes are higher. Put numbers on your beliefs, say what a policy assumes, publish its probability of success. State the triggers that will force a pivot. Score your calibration and accuracy, so we can learn how much we should trust your judgement. The public can forgive error; we should not forgive opacity and obfuscation.

Everything is prediction: in minds, in markets, in ministries. The choice isn’t between predicting and not predicting; it is between implicit, untested bets and explicit, accountable ones. We should choose the latter—and build the habits and institutions that make our next bets a little less wrong than the last.

This is what we do at Cassi. Contact us at admin@cassi-ai.org.uk

^[1] Kay, J. and King, M., 2020. Radical uncertainty: Decision-making beyond the numbers. WW Norton & Company. especially pp. 71-84, quotations from pp. 223-224.

Future Diplomat: Rewiring & Reimagining Diplomacy

Technology has long changed diplomacy. If diplomacy is, according to the Oxford English Dictionary:

- the management of international relations by negotiation; the method by which these relations are managed by ambassadors etc; skill in such affairs. Adroitness in personal relations; tact.

… then diplomacy is communication: both the channel by which states speak, and the craft by which they persuade. As technology revolutionises how we communicate, to make and shape decisions, so it revolutionises the conduct of diplomacy.

Think of diplomats, embassies and the FCDO as cybernetic systems. They seek and sense information, process it through the neurons of the brain as individuals, or between the functional roles of the people staffing the organisation, act or choose not to, then observe the effects. The end is behavioural: get someone to do something new, do something differently, or keep doing what they already do. It might be a foreign person, organisation or group is the target of that diplomacy, or it might be those within the diplomat’s organisation, their home country or government – where the diplomat is seeking to mediate between positions.

If this is true, it should be clear that technologies that are fundamentally informational, and cognitive – they draw insight and foresight from data at unprecedented speed and scale, and make decisions based on models of the future – will transform diplomacy.

This vision for 2035 is built on the premises above. In addition, its starting assumption is that using forecasting tournaments and/or prediction markets1 is the best way to understand the future – on which basis it anticipates:

- weak artificial general intelligence (AGI), one that matches or exceeds human performance on a range of diverse cognitive and informational benchmarks, by March 20272;

- Oracle AGI, a system that significantly exceeds human performance in answering any question, by 20283 – or under a more stringent definition 2032.4

- Artificial Superintelligence (ASI) – adding superhuman skill in robotics, proprioception, dexterity, locomotion ~May 20295;

Thus a diplomat in 2035 will be operating in world of superintelligent AI.

Consular Services Without Counters

Most consular work is structured information processing: visas, passports, notarial services, births and deaths, crisis assistance. By the early 2030s, agentic systems will handle the bulk of it end‑to‑end with clear audit trails and explainable reasoning. Consulates will be able to operate in all languages, tailoring information to individual cultural and psychometric dispositions and levels of education. Identity will be verified remotely through secure biometrics. Citizens will complete interviews and attestations conversationally, recorded in signed digital form. There will be little need to visit consulates for these tasks, with most activity taking place online, and other interactions via the metaverse – meetings with AI (probably ASI, per current forecasts) agents embodied in avatars. If these are distinguishable from a remote meeting with a human today it would only be in that we should expect an advanced, or superintelligent AI to provide a better service than a human – more empathetic, more patience, more personalised, smarter, more knowledgeable, with better memory.

From Self‑Driving Labs to Self‑Driving Embassies

In 2023 Eric Schmidt, in MIT Tech Review, & the scientific journal ‘Nature’ both described the birth of self-driving labs (SDL), those that - at the maximum extent - operate fully autonomously, designing hypotheses and experiments, conducting experiments, reporting the results, and then designing new hypotheses, testing again, without human intervention, at super-human speeds. In 2025, self-driving labs are driving breakthroughs globally. The journal Nature Chemical Engineering reports that in the field of materials science self-driving labs are producing new materials, ten times faster, processing ten times the data, accelerating discovery exponentially. Furthermore, July 2025 saw Nature publish articles showcasing:

1. a ‘self-driving microscope’ to better understand protein aggregation and neurodegenerative disease, and

2. “Ada” a self-driving lab that has discover new metal materials by identifying optimal trade-offs between conflicting objectives

3. Showcasing a Chinese lab’s use of AI to design the ‘brain’ of a quantum computer using laser light to move thousands of atoms into any arrangement they choose and act as qubit, in 60 milliseconds - demo’d in this case to produce a video of Schrodinger’s cat.

While the same month this year (2025) Science reported how self-supervised AI now gives robotic probes pixel-level precision, letting autonomous labs log over 3,000 semiconductor tests in a single day - enabling rapid defect detection and a stronger, more resilient advanced-manufacturing supply chain. Given AI Agents are reportedly doubling their performance ever seven months, these kind of breakthroughs are likely just the beginning (see Cassi blog post dated 31 July for more SDL progress in 2025).

The next decade, perhaps the next few years, will see the emergence of ‘self-driving embassies’. Just as today’s self-driving labs use autonomous AI to answer scientific questions, in self-driving embassies AI agents will handle consular and diplomatic enquiries and AI systems mediate most—perhaps all—routine exchanges, principally text, audio or digital, between embassy and both home and host governments.

Persuasion agents will build psychometric profiles of the individuals and groups with whom the Embassy or diplomat is interacting, enabling tailored negotiation strategies, which can be adapted and improved in real-time. When negotiations are in-person via video call, AR/VR meet, or physically face-to-face, ASI agents will be reading micro-expressions, designing recursive negotiation strategies and updating the approach continuously. Affective computing will be more empathetic than humans, able to understand and manipulate us at superhuman levels. It is hard, though not impossible, to imagine negotiation being fully delegated to ASI systems negotiating with each other. A study in Nature in May 2025 showed AI was more persuasive in debate than humans >60% of the time. A recent (2025) unauthorised and probably unethical experiment on the Reddit forum R/Change My Mind shows the way – an LLM deployed to persuade was six times more persuasive than humans and operated without anyone recognising it was an LLM. The largest LLM persuasion study to date, (published 18 July 2025) showed newer, larger models were the most persuasive, and that models persuasiveness persisted over time (1-month, in this study). The best models (already a generation behind o3 and GPT-5) were 41% and 52% more persuasive than ‘static’ messaging - remember when we were panicking about ‘static’ social media messages on Twitter and Facebook? In fact, previous studies of human persuasiveness, advertising and personal conversations in elections, suggest ‘an average effect of zero’.

Even if human relationships remain crucial to effective influence and negotiation, they will be increasingly informed by profiling agents, prompting and preparing diplomats for discussions, perhaps advising during discussion, debriefing afterwards.

Linguistic skill may be vital, not for translation - which may cease to be relevant when translation can be automated instantly - but to enable diplomats to develop the mental models and worldview of those they interact with. If that is the case, diplomats will likely be learning through AI-tailored immersive courses in AR and VR. Cultural expertise, and understanding - the right-brained process still too little understood by neuroscientists, psychologists, computer and other scientists for us to be confident we are modelling it effectively in Artificial Intelligence, may be a key human skill. But to outperform or complement machines it will likely need longer, deeper immersion in country, possibly career-long deployments. Similarly, when the human currency becomes relationships, since rationality, persuasion, information processing and reporting is undertaken at superhuman level by machines, FCDO careers will be much more focused on the cultivation of life-long relationships with current and future key influencers in countries around the world. Fast-stream processes, and civil service churn will have to end if diplomats are to flourish in an age of AI. Deep relationships take time to build.

Recruitment of diplomats will use AI systems to identify super-persuaders, just as intelligence analysts will be identified as super-forecasters, or super-questioners – those that can create the questions that maximally reduce ignorance, on which forecasters and ‘persuaders’ should focus – and recruitment will follow. A more prosaic instantiation of Philip K Dick’s Precogs in ‘Minority Report’. Strategy, tactics, decisions will be automated. Humans will be working out what the most desirable ‘outcome’ or objective is with machines, but the planning itself will be done by machines, and implementation overseen and directed by them.

Passports to New Worlds

The trend towards e-citizenship and e-businesses will continue. Many visa and passport applications will be for virtual access to the UK, as telepresent (AR/VR) or telexistent agents (AR/VR with the ability to directly interface with the real world via robotic interfaces) – with people ‘visiting’ and working in the UK without ever entering the country.

Furthermore, citizens and businesses will apply for Visa to access UK managed virtual worlds, metaverse countries with economies that rival countries in GDP with access to UK legal authority for dispute resolution.

Beyond the ‘Geo’ in Geo-Politics

Migration pressures will ease, as citizens in developing countries are able to prosper by building e-businesses in developed countries while remaining in their own. Trade barriers and cross-border taxation will become harder to enforce as metaverse economies produce wealth independent of geography.

Space will be the most lucrative market in human history, with in-space industries from mining to manufacture, compute to colonisation. By 2035, with regular vertical and horizontal launch from the UK, the FCDO will have to manage in-space territorial claims and economic competition, as well as the security implications that flow from competition for dominance in space. If the UK fails to compete in the space colonisation, asteroid mining and space-based industries, the FCDO will be managing a much more precipitate decline in the UK’s relative power in the international system than it has ever had to do to date.

Finally, automation, robotics and AI will reduce countries and companies' dependence on demography to mine raw materials, produce goods, services, and scale armed forces to deter or fight. Self-driving labs will become self-driving innovation systems, with companies and countries power, prosperity and security driven by the speed at which they can invent and harness new technologies.

In combination, metaverse economies, the space market and automated innovation and industries will move competition for relative power in the international system beyond the geo of geopolitics, with nations’ within-border natural resources and demographics less deterministic of their power and influence. For the FCDO, effective statecraft will have to be more integrated x-Whitehall, and with industry, finance and society, than ever before.

Influence, Security, Privacy, Agency

Superintelligent systems able to analyse groups and individuals, understand their psychometric make-up, and influence their behaviour risks social control tightly circumscribing human agency, able to manipulate with unprecedented effectiveness. Even today, we generate c. 1.7MB of data per person per second – the equivalent of 56 million Kindle books per on each of us generated every day – it is this that gives unprecedented insight and foresight into who we are, what we do, and what we are likely to do in the future. Debates on human agency will soon be the domain of the diplomat, as privacy-centric EU clashes with the market-centric US, and China’s digital authoritarianism.

Our security will be redefined in a world where the boundaries of the virtual and the physical have blurred through AR/VR, where we live in different ‘belief circles’ seeing different versions of the world around us to our neighbours, and spending more and more time in borderless cyberspace. Cognitive security may be a competition for e-citizenship, e-businesses with other states, and virtual states.

When our cars, smart homes and brain computer interfaces all become critical national infrastructure through connected IoT devices, we will have to rethink how it is managed. Cognitive security defending our data and our minds.

All this will make the role of the FCDO more complex, and more expansive, while the future diplomat might be able to more effortlessly cover a wider geographic area, the GeoInt Singularity + ubiquitous data surveillance will mean they will struggle to move anywhere or meet anyone, without others knowing. It may be that everyone they meet knows their psychometric profile, their preferences, and has optimised their position to ‘win’ in diplomatic negotiation.

Perhaps privacy-enhancing technologies will prevent this, but if they do, they will also limit the state’s ability to manage threats, gather data.

Public diplomacy conducted online will become an ever more powerful tool, but will also raise the same questions of privacy, security and agency – how far should we go in profiling and manipulating in a world where Spotify data can be a better indicator of national mood than direct survey data6, where all of us are vulnerable to microtargeted manipulation.

In this context, regulatory diplomacy will be as important as all other forms, determining the kind of world we live in, and the extent to which people are free. Hence China requires all AI companies to design with “core socialist values”, values it seeks to embed and export via its proposed ‘Organisation for Global AI Cooperation’ (Reuters, July 2025; Chinese State Council, 26 July).

Simulation and Wargaming

New policies, new negotiating tactics, the effect of military or economic actions, likely future scenarios and emerging flashpoints we all be constantly developed and tested through the digital strategy-engines, offering superhuman suggestions for strategies and tactics, testing second order effects, highlighting new factors to be considered. The GeoInt Singularity, where every square meter of the earth is constantly observed from space, will enable the development of a global digital twin, to identify flashpoints and early indicators of emergent conflict or natural disaster. Wargames and collective intelligence tools will enable humans to challenge the models assumptions, force it to consider counter-factuals, ensuring the most effective strategies are pursued.

Augmentation

Technological transformation will reduce the role of humans in many diplomatic decisions, and reduce the number needed to deliver diplomatic services. However, human relationships may remain key to successful negotiation - it depends on the extent to which nations are willing to let AI-agent negotiate with AI agent. If humans remain in the loop they will be increasingly augmented, able to bring the best of human cognition - whatever remains of value in this emerging age – together with technology to enhance our capacities. Brain-computer interfaces will, by 2035, likely be enhancing memory, perception and learning. In turn we will be aided by a neuropharmacology industry that optimises our performance and our rest, and by a growing science of high-performance teams, that ensure we best balance cohesion, talents, and incentives to maximise the performance of people in groups.

The US AI Umbrella?

If the US develops an AGI or ASI model before anyone else, is able to harness it effectively, and launch it first on recursive self-improvement, it may be that no other nation can catch-up. In such a situation, British and other nations diplomats would be negotiating with a system that was comprehensively smarter than them, and progressively more so – those they interact with being prepared not by very capable advisors, but by a superintelligence Private Office or equivalent – or possibly being asked to interact directly with the US’ ASI diplomat. So too British businesses would be facing competition from businesses smarter than them, the British military working with an ally, assuming the US remains such, that was strategically, operationally and tactically smarter. The outcome of such competition is likely to be very one-sided. If this becomes so, we may find ourselves living under the AI equivalent of the US nuclear umbrella, surrendering sovereignty, agency, in return for protection and access. The price of living under the US AI Umbrella is likely to be far higher than the cost of its nuclear protection.

AI Disruption or Destruction

It could be that we live, as Oxford Philosopher Nick Bostrom puts it in a ‘solved world’ a post-ASI utopia where nobody need concern themselves with the intra-human competition of which diplomacy is part. It might be that through misalignment or malignance AI poses an existential threat to humanity. Or it might be that AI becomes a substitute, not a complement to human labour, with radically disruptive social and political effects. None of these scenarios are so unlikely that the can be ignored. Hope is not a strategy. But this article has described the Future Diplomat operating in a messy, complex, competitive world – with or without the development of super-intelligent AI. It may be unproductive and distracting to spend too much time on the dystopian outcomes here. It is vital the FCDO does spend time on them, nonetheless.

Conclusion

Much of what has been described is possible now, with today’s relatively narrow AI. More will be possible as current AI advances further - all the evidence suggests the UK will underestimate this progress, and the nation be poorer and less secure as a result.

In the more likely than not advent of AGI or ASI before 2035 the role of the diplomat and Embassy will be radically transformed, with humans the agents of AI as much as the other way around. The birth of metaverse worlds and the growing import of the space economy by 2035 will change where diplomacy is conducted, power created and managed, people and machines persuaded. We must be ready.

The best time to start building for this future was yesterday. The next best time is today.

This article is really ‘just’ an update on AI progress. All the developments here are from the last three weeks or so. I would go back meticulously to the last time I posted, but the article would be too long and the point is made more neatly here by not trying to cover everything. Progress is still astonishing.

Why now? Well, rumours are rife that GPT-5, the model that OpenAI CEO Sam Altman says ‘scares him’ that seems to have convinced him we are past AGI, and now the target is ASI- super-intelligence (variously defined1), is said to be ready for roll-out and release this month (August 2025). Indeed GPT-5 and various derivatives may have been trial-released in LLM Arena last weekend, as Zenith, Summit, possibly Lobster, Nectarine, Starfish, and o3-alpha, drawing breathless commentary for their performance2.

After GPT-5 release happens, there’s a very real risk and high likelihood the remarkable recent progress is eclipsed and under-discussed. As Rob Bassett-Cross (CEO Adarga AI) and I wrote in our eponymous article ‘Don’t Blink’ - AI watchers daren’t look away for a moment such is the pace of progress. My concern remains that as a country, a society, and in many of our companies, we in the UK are still under-estimating the speed of AI progress, and the profundity of its implications.

This under-estimation is despite the discussion on X and Substack beginning to break into the mainstream. Tom Whipple wrote in the Times last week (25 July):

‘I know the riposte. Humans always fear machines. …AI won’t take your job, goes the mantra, but if you don’t get with the revolution, someone using AI will. My response? You’re not listening.’

But even with views like this in the thumpingly centrist Times, nothing really changes in UK policy circles. But it changes everywhere else.

In the past few weeks for example, an essay collating the thoughts of leading Chinese thinkers on AI - shows their emphasis on Open Source, diffusion, ‘talent’ as espionage vector, and regulatory standards as the route to values dominance. A hyper-pragmatic, and believable approach to the otherwise unequal competition with US Big Tech. It is free of the delusions and not-really-even half-measures dressed up as solutions that characterise the UK’s response (and that get almost no scrutiny or criticism from our media for which, Whipple not withstanding, this remains a fringe issue - as opposed to perhaps the most disruptive technological revolution in human history).

We’ve seen the Chinese Premier Li Qiang announce a plan (consistent with the above) for China to launch an ‘Organisation for AI Cooperation' based out of China but with global aspirations.

We see in China a growing focus on ‘embodied AI’ - accelerating diffusion of AI into its industrial base as a way to advantage, perhaps also making the vast amounts of data it collects available, to influence future models.

That’s just China. RUSI reports on the scale of US deals with the Middle-East on AI investments as ‘staggering’. But these investments are not staggering - they are a proportionate reaction to the continued acceleration of AI progress, an anticipation of the rate, direction and implications of this progress - they are other nations’ recognition that they can’t afford to be left behind.

Yesterday (31 July 25) we read that Norway has struck a deal with the US to build one of its Stargate AI projects (the US ‘Manhattan Project for AI’ announced in January) in country, a gigafactory, an AI data centre entirely powered by renewable energy, to supply Europe. Norway and US business are investing $1bn in the first phase of this project. ‘Europe needs more compute’, Sam Altman notes. More than just ‘more compute’, we - UK and Europe alike - need a credible plan proportionate to the threat and opportunity posed by AI.

Here’s why. In the past few weeks:

• Nature Chemical Engineering reports that self-driving (AI/automated) labs are producing new materials, in the field of materials science, ten times faster, processing ten times the data, accelerating discovery exponentially.

• Nature Communications reports that self-driving labs have discovered in the venom of snakes and spiders, 2000 new antibacterial proteins that could serve as alternatives to current antibiotic treatments, 58 of these have been tested so far, 53 had medical applications such as killing the currently drug-resistant bacteria e-coli and staph - vital in a world of growing anti-microbial resistance.

• A survey published back in March showed Agent-based AI models’ increasing abilities in completing software development tasks. In 2022 they could complete at the touch of a button tasks that took humans 30 seconds. In 2025 they are completing tasks that take 1-hour. Their performance was doubling every 12-months, but is now doubling every 7 months. On this current trend, even without further acceleration in the rate, by 2027 they will be able to complete tasks that take humans a month. By 2029 tasks that take humans a working year.

• I re-report that because now, further research shows these improvements in Agentic AI ‘task scaling’ are also being seen with Agentic AI in scientific reasoning, maths, robotics, the use of computers for tasks, and in self-driving systems.

Most significantly, in my view, since last I wrote, mathematicians have had their Gary Kasparov/Lee Sedol moment, when AI surpasses them and they wonder what they are for. This was triggered by ‘general’ models like OpenAI’s latest reasoning model and an advanced form of Google’s Gemini winning ‘gold medals’ at the International Maths Olympiad (IMO). The achievement prompted both amusement and amazement. One Data Science Professor posted about the poor performance and limitations of general AI models on Olympiad tasks just hours before their success was announced and was much mocked. In his defence, this was faster progress than some of the most bullish experts had predicted, and is another example where progress has been consistently under-estimated. The question ‘When will an AI win a Gold Medal in the International Math Olympiad?’ was forecast as follows:

July 2021: 2043 (22 years away)

July 2022: 2029 (7 years away)

July 2023: 2028 (5 years away)

July 2024: 2026 (2 years away)

July 2025: AI wins a gold medal in the International Math Olympiad.

This is not the first time I have written out these kinds of estimates to illustrate how difficult humans find it to understand exponentials, to forecast AI progress.

On Friday, seven days before I clicked ‘publish’ on this post, AI performance on the ARC AGI 2 benchmark [‘Abstraction & Reasoning Corpus for Artificial General Intelligence 2’ – an AGI test redesigned after ARC AGI 1 was surpassed in December 2024] jumped from 16.5% to 19%. ‘Only’ a 2.5-point gain, it represents a >15% relative improvement in capability. If we assume exponential growth here too, given this benchmark was launched end March 2025, and models were scoring ~12% by end April, it could be fully solved by ~August 2026.

We’ve seen Deepmind’s Aeneas AI historian decode and complete inscriptions and text, adding nuance and contextualised understanding. We saw Deepmind launch AlphaEarth - where the impact on espionage and military operations should be obvious - take this summary from one commentator:

'AlphaEarth Foundations does something clever -- instead of drowning in petabytes of Earth observation data, it creates compact summaries of every 10x10m square on Earth by fusing optical, radar, LiDAR, and climate data. ...it can see through clouds in Ecuador and reveal hidden agricultural patterns in Canada.’

As I’ve written before, only in the world of AI could developments like these two new Deepmind models and milestones be essentially addendum to the main points. More likely than not, all the developments reported herein will soon be eclipsed by and quickly forgotten after the release of GPT-5.

A reminder - as I wrote in In Athena’s Arms - ‘…this is not a millenarian declaration of deterministic certainty.’ Maybe progress will stop. Maybe someone will declare ‘AI will never’ - like chess, Go, the IMO - and this time be right. Maybe someone will point to a fundamental barrier to AI progress that we can’t innovate our away around, through or over. But right now, the ‘straight lines on graphs’ that have driven and reflect AI progress for almost two decades, show no signs of stopping. Very few people seem to be taking this as seriously as you might expect given the evidence - trying to plan, build, invest, govern, write, report, or converse as if AGI could be with us imminently, or in the next few years. Hope is for too many, the entirety of their AI strategy.

We are still under-estimating the speed of AI progress, and the profundity of its implications.

The SDR published yesterday (Monday 2 June) is the best of my lifetime. It is sharper, more honest, and more structurally ambitious than any I have read—including those that predate today’s formalised processes and those written by nations beyond our borders.

But it is already at risk. Not because of what’s in it. But because of what’s being said around it.

I’m writing this not just to praise the Review, nor simply to critique its flaws. There are several, some are serious. But more urgently: to warn that the opportunity it presents could be lost amid noise, spin and superficial commentary. This would be a failure of national consequence. I hope the blog might reach those who can help us avoid it.

Distinguishing Review Signal from Political Noise

The misunderstanding and mischaracterisation create three principle risks:

1. We could miss the moment to prepare for war—and thereby make it more likely, and more disastrous.

2. We could miss the chance to raise the level of our defence debate, after years of kayfabe, fake budgets, and misdirection.

3. We could lose the Review’s careful and detailed analysis amid the thunderous political ‘announceables’ that risk drowning it out —many of which are either unrelated, outdated, or contradictory.

Let us take the third first.

In the run-up to the Review’s publication, there was a story that the UK would now fund and field a second fighter-jet launched nuclear capability, in addition to the continuous-at-sea deterrent, our nuclear submarines with nuclear missiles. This is not in the Review, and regardless of its potential merits and demerits (I think on balance a sovereign capability we need) – or indeed whether we do commit to this following the talks that the Times tells are us are being held with the US, the Review does not recommend this. It was certain though that announcing it would drown out almost everything else in the debate. Similarly, the announcement of a £15bn new ‘sovereign warhead’ was always going to grab headlines, but isn’t new. Likewise, the announcement of twelve new submarines is not in the Review [0.1], which instructs the MOD to confirm numbers against requirements. Announcing all this as the Review was published obscured what the Review actually says.

The Flaws are Clear

Many criticisms of the review are justified. Decoupling it from a Spending Review, as happened in 2021 during the Integrated Review, neutered it. It is a standard Treasury tactic and it enables the Government to do what institutionally, it prefers – make policy and strategy word-smithing and empty rhetoric, a kayfabe, while the gritty, largely boring reality of implementation and hard choices are hidden: made by default, rather than decision, choices made when the headlines are dominated by wider cuts, spending and taxes on non-defence issues. Relatedly, the MOD reserving for itself the decisions on what equipment it needs, to come after the Review, makes it unlikely they would sign-off any recommendations the Reviewers might want to include on what and where to cut. It cannot be an ends-ways-means review, a strategy, if denied or constrained in the opportunity to comment on what means - financial, and in kit and equipment - are needed.

In consequence, this Review has been emasculated by being separated from the spending review. As many commentators have noted either:

a. Defence needed a lot more money. Or

b. the Reviewers needed to show where they would cut, take risk and transform.

They have done neither. But they were presumably prevented from doing so. One wonders what the Review looked liked before the five-month period of ‘Decision-making and finalisation’ as the Review team ‘finessed…[their]…recommendations’ during ‘Engagement across Whitehall…’.

The language seems dumbed down in many places, enabling consent and evade (or ‘toxic compliance’ as I heard it called recently). e.g. “high-low mix” can be whatever the MoD decides it wants it to be. Digital Targeting Web is clearly a “Kill Web” the failure to call it what it is shows I think the lowest common denominator compromise that has infected parts of the Review. On procurement, we are still allowing some equipment to be built on 15-20 year timelines, with individual capabilities taking 18 months+ to manufacture. I don’t think you need to see the classified info (and to be clear, I have not) to see that this is a recipe for cost overruns and failure. The tech will improve a thousand times or more during the life of programmes, requirements will change in geopolitical cycles that can be hours and days, or just a few years. Most importantly - we must be able build stuff faster than our enemies can destroy it. What’s the ratio between expected attrition rates and the production speed of new platforms like FCAS or SSNs?

Things are often under-specified, for example, the MOD will likely just rebadge its multidomain integration stuff as a “digital targeting web” and keep on failing.

There are the usual linguistic atrocities. But these fall in to the category of niche objections that might matter, but not to most people. Fellow pedants, find more in the footnotes.[1]

But there is a real danger here that ‘the Reactionariat’ of which I am now a part, miss a historic, urgent and vital opportunity to use the Review to raise the quality of our Defence debate, to hold the Government and the MOD to account. Strategic Reviews rarely give us a strategy - they should, that they don’t is a failing of our system [1.2]. But they can still be influential - harnessing the zeitgeist, empowering reformers - through a kind of ‘narrative dominance’ they can change the conversation, and with it the priorities and activities in Government. This is where the risk of missing the opportunity given by this excellent review lies.

The Review has Given the Nation an Opportunity we Must Not Miss

In diagnosis the Review is clear and comprehensive in saying how bad things are, overturning rocks, and calling out how urgently things need fixing. You can see where it has been dumbed down I think, not quite being able to say that if we fight tomorrow it would be a catastrophe. But it is very close, and you don’t have to work hard to figure out that is what it says. I think you could sum this up as saying it would no longer be possible for HMG to say it did not know how bad things are. I think to have got this through the Whitehall system – maybe even more in need of reform than that within the MOD - is a significant achievement. It is detailed, exacting and far-reaching. Genuinely impressive in form and content. It could, if understood for what it is, drive a much better debate on defence.

There are important structural changes - Cyber-EM Command unifying electronic warfare (we’re actually pretty good at this, in narrow pockets that are permanently at risk of being killed by the system). Similarly, MSHQ and killing Levene are a big deal. Something we tried to get done when I was serving in 2018, again in 2020 when in No10.

Under the current SofS’ John Healey’s leadership, and now public in the Review, CDS now has - or should have according to his revised role described in the Review - some meaningful authorities and not just sit as an advisor to SofS and the PM. We can stop talking about a civilianised ‘Head Office’ with its appalling culture but a proper HQ, that can be held accountable (without this, no one was in charge and everyone blamed everyone else for continuous failure). These problems were and are systemic, not individual, but they incentivised and rewarded some pretty toxic behaviours. They were part of the reason that lowest common denominator compromise consistently triumphed over confronting issues. We should now have not three and half (STRATCOM) forces designed separately who figure out how they will fight later, but one force, centrally designed and focused on how we would fight in NATO vs Russia (I personally don’t think this is the right way to think about the threats we face in an era of systemic competition, but it isn’t an unreasonable approach [2]).

The Review has a ‘demand’ for specific metrics, on lethality, productivity, and national economic impact. There is a section designed to ensure accountability (p.50). There’s a call to cut “by at least 50%” MOD regulation that kills industry. The kill web has a deadline - MVP by 2026, fully operational by 2027 – and where described as a ‘clear unifying mission’ seems to be an attempt to give us a proper theory of winning that we haven’t had since the end of ‘Air-Land Battle’ which passed with the Cold War. The commentariat could do much to push this to become a full operating concept, a theory of winning that drives force design based on DARPA’s ideas of the Kill Web, and Mosaic Warfare (or decision-centric warfare).

The focus on the way war has been transformed by technology runs throughout, describing how this is not an event, but rather that technology is continuously redefining warfare. This is, I think, correct and not really disputed (the degree to which it is true is argued, not the fact that it is).

Criticisms that there is not enough on emerging technology and procurement are unsustainable – some of the Reviews recommendations were clearly taken by the Treasury and announced early, but they are well covered and pushed further: “Major modular platforms (contracting within two years). Pace-setting spiral and modular upgrades (contracting within a year). Rapid commercial exploitation (contracting within three months), with at least 10% of the MOD’s equipment procurement budget spent on novel technologies each year.” The MOD is pushed to engage with VC and Private Capital in ways it currently does not.

The commitment, publicly, to making the UK’s Armed Forces ‘the most innovative in NATO’ echoes something I argued for in 2020 for the 2021 Integrated Review. To be meaningful, it will need measures and metrics – how will we know we have achieved this? But these are not impossible to imagine. I offered suggestions here in 2021. You could adapt the CBI’s measure of innovation pull-through ‘new to market new to firm inventions’ – as ‘new to frontline, new to service inventions’. This Review gives Parliament and media the chance to propose how this might be metricised, measured or judged, and hold the MOD to account either for delivering against them, or to propose its own. There are a lot of deadlines – any reader of Government documents will recognise how rare this is in HMG policy. It is a huge opportunity for Parliament, media and public alike to hold Government accountable for detailed delivery. In shouting down the Review, the reactionariat risk missing a generational and urgent opportunity to make us safer.

Personally, I would have liked to have seen a commitment to a 6-monthly independent review of progress vs the recommendations, submitted to the PM, Parliament, and available to public. That was what embedded Levene’s reforms, mostly for the worse in the long run, but the process teaches lessons: people respect what you inspect and value what you reward. I would have had named owners for each recommendation, and reported on their progress in those regular independent external inspection reports. Of course Parliament, media, commentariat could now start making the argument for this, to help those reformers on the inside doing the same, to push the system to reform. To improve our Defence debate. This was probably always beyond what the Reviewers could achieve. Neither this Govt, nor the last, or any in my lifetime would have agreed to that - it needs one under greater political pressure (where scrutiny now could help), or one with greater confidence, and/or one that viscerally believes in what this review tells them: war could be imminent, and we would likely lose. If you’re not steeped in this stuff, that is hard to believe.

Examples of how far-reaching and thorough-going this Review is run throughout, but are particularly notable in the chapters on Defence medical, education and training – again calling out shortfalls and failings clearly, with recommendations for how to get to a solution. A crucially important example – especially since everyone, reviewers and critics alike, seem to agree we aren’t spending enough, won’t spend enough, and are not ready to deter or fight a war - is the call for a proper plan for mass mobilisation as Sky’s Deb Haynes and I called for this time last year. How we prepare to rapidly increase our resilience, preparedness and ability to fight across the whole of society and government if war comes while our conventional armed forces are too weak to succeed or sustain in the fight is not just prudent planning, it is a part of deterrence too. Similarly, making defence infrastructure an operational consideration, and not one civilianised, lobotomised, and focused only on cost, efficiency and short-termism is a huge shift. The form of this Review is better - the citizens panels a neat innovation. The substance on a national conversation - expanding cadets - is good. None of these are things past Reviews have covered anywhere near as well.

Armed Forces

The recommendations on the RN & Army are radical and right. Worth reading these sections alone (short, digestible). The recommendations on the RAF are in my opinion dangerously inadequate. There is nothing on what the high-low mix should be. Little on where the RAF carries risk in current operations. Far too much scope for consent and evade. The diagnosis has either pulled punches or has been too superficial. We should remember that the notional independence of the Review makes this the Review Team’s failing, not the RAF’s. Since this Review offers only recommendations, it does not mean the RAF can’t still be clear-eyed in diagnosing and addressing its own challenges with the clarity, rigour and radicalism that is present in the Review’s section on the RAF’s sister Services.

The section on Space disappoints. The U.K. continues to radically underestimate its importance and centrality to our future prosperity and security. I don’t think the Reviewers, and almost anyone else, agrees with me on this. But I stand by it (it requires AI leadership to turn its potential for our relative power into reality). The review is good, within the parameters of the existing debate, where space is just the bit above the air domain, and the issues are access and control - not trade, industry, colonisation. This should be an area for the National Security Review. I am pretty certain they will fumble this too. It is too far outside the Overton Window, and would blow the mind of a Treasury trying to scrape money together for social spending. We’ll all be much poorer and less secure for this in 10-years time, but that kind of thinking and planning is hard in Whitehall. For Defence to contribute to the debate and the capability we needed an independent Space Force as both I and others have argued.[3] A Space Force could argue for funding and develop expertise in the domain on equal footing with those representing the Air, Land, Maritime and Cyber domains. We still don’t have one. Not a surprise, but disappointing.

On Cyber-EM. This is complex. There is a continuity across cyber and EM (I prefer ‘electronic warfare’ - narrower, clearer). But they are also distinct. And they run through everything else. I think we need an independent Cyber Command to argue for resources as an equal to space, air, land, sea domains. But this doesn’t solve the EW problem. And there is no chance the Treasury, or the existing services would allow two new Commands. It was never a likely outcome. We probably have the least bad option here, and a big improvement on the status quo.

Info Ops for me is central (war is about coercing decisions) even a nuclear strike is an attempt to coerce a decision, ultimately for psychological effect, so too every bullet fired. It is a side note in the Cyber EM section. Again though, this is the orthodoxy. I am not a lone voice, and I think logic on our side. But it is an argument we have not yet won.

On intelligence – the Review is very good on calling out our weaknesses in Counter-Intelligence – which we must prioritise, indeed in which we need to be best in class, or everything else is undermined. I am less sure on the radical merging of the intelligence branches - I think this will kill off necessary specialisation. I also note that the Reviewers don’t call for the re-establishment of Defence Economic Intelligence (another thing I think is important, but not many talk about as an issue), and should have done. This is an era of geoeconomic competition. We have no economic intel function in HMG (killed in Defence Intelligence in the 1990s). Also necessary to support Defence Exports. Perhaps the National Security Review that Government has said will follow, will cover this. But I am not optimistic and I think the Defence Reviewers missed an opportunity here.

Reviewing the Review

A short way of seeing how much better this Review is than anything that has come before in my lifetime would be to print off the MOD’s 2020/21 Command Paper, and read them side-by-side. The former was bad at the time. It is shamed by this.

I think a fairer judgement than the many who are saying this is just ‘more of the same’ would be that the Review gives the MOD enough that with the structural reforms and a genuine SofS commitment to real implementation, it could be transformative.

Without SofS commitment, the dumbing down of language and stopping just short - i.e. stating the need for metrics mostly not specifying what they should be, not saying what should be cut, not offering a view on balance of crewed/uncrewed, on AGI timelines, specifying a need for kill web without the fuller “theory of winning” (op concept) leaves it largely impotent to address the threats we face. It can only describe and recommend.

The Reviewers were hamstrung by this being, as Matthew Savill has said ‘Schrödinger's Review’ neither independent and external, nor internal and owned by the MOD. Thus they have been prevented, I would guess, in going as far as they wanted. But it is clear that they have used what political power their position gave them to preserve much that is valuable.

The problem with the fake budgets, rhetoric, and vapourware, the noise around the Review, is that they sound exactly like ‘business as usual’ and everyone is acting accordingly, taking pot shots at a Review that seem to me to be frequently completely detached from what the Review actually says. Sometimes ‘control K’ in the SDR for whatever key word of criticism is being levelled can expose this. It isn’t hard to check.

The noise is at worst a political attempt to obfuscate what the Review actually says, e.g. explicitly and directly “business as usual is not an option” and which otherwise - even in its likely watered down form – exposes the gaps between years of rhetoric and reality too clearly, and in ways that are not advantageous for a Government that knows there are still not many votes in Defence. Even at best, if not deliberate, the noise results from a misreading of the Review’s contents. Whatever the reason, there is a real risk that the ‘annouceables’ obscure what the Review actually says, harming the debate the Review calls for before it has even started. Drowning out signal in noise. Leading the commentariat to discredit the Review, making association with it low status, ensuring that narrative dominance - the only way in which this Review can succeed, is never achieved.

My call to action is this: read the Review in detail, and celebrate it. Make association high status. Use it to push the recommendations further, to empower the reformers whether they be Minister’s, civil servants, or military. Theirs is a difficult fight, the review gives them material support. Allies - you - could help them win.

An effective Defence is, as the FT’s Janan Ganesh said of growth, something everyone in Britain wants, but, whenever a trade off is required ‘just not that much’. The Government, Secretary of State John Healey and his team in particular, are to be commended for creating an opportunity for a debate that might change that. Commissioning even a semi-independent Review was brave. In publishing and accepting the recommendations of what could be a radical review, the Prime Minister and his Government have created the space for and seeded a much better national conversation on just how much we want to spend on defence, vs the risk of our sons, daughters, mothers, fathers, friends and colleagues, coming back in body bags, being overrun in battles we fail to deter and are not fit to fight. What happens next matters hugely. I write this in the hope that reformers in and across Government, those in Parliament, media, and public alike can make the most of this most remarkable of Reviews. It must not be a return of what Cabinet Secretary Simon Case once called ‘the curse of the missed opportunity’. As the Review quotes one of its ‘Citizen Panellists’ on defence: [otherwise] ‘…it won’t be long before something more significant happens and we will think it should’ve been more of a priority.’

[published 0700; updated 09:33, 21:58, 4 June 2025]

[0.1] Added 2100, 4 June 2025. I am grateful for having this pointed out so I can correct the record: technically this is not quite right, in the SofS foreword he does talk about the ambition to get to 12 SSNs. But I offer two defences (1) my defence is of the ‘independent’ Review, the SofS foreword cannot, by definition be in anyway independent of the MOD; (2) in the foreword it is contextual and soft pedalled, not trumpeted to the point where it drowns out discussion of anything else, as happened when this information was provided ahead of the Review as ‘a response to it’ (https://www.gov.uk/government/news/uk-to-expand-submarine-programme-in-response-to-strategic-defence-review).

[1] ‘Warfighter’ should be excised from the lexicon. It is a hair’s breadth from ‘Warrior’ which is even worse – and I think misunderstands how military dominance is achieved. The Spartans were defeated by the better organised forces of Thebes. The Romans dominated the ‘warriors’ of Europe through superior organisation. The Vikings were routed not by lone berserkers but by the more centrally organised Anglo-Saxon levy at Stamford Bridge in 1066, where Harold Godwinson’s disciplined fyrd outmatched their hit-and-run tactics. In each case, it was organisation—training, supply, command and control—not individual heroics, that determined the outcome.

Picking out and trying to kill-off a personal bugbear that finds its way into the review: “war is fought across all five domains but people and the land they live on are at the heart of it”. This is a half-baked zombie idea, and should be murdered, cremated, buried and sealed in the earth’s core so I never encounter it again. It is the Army’s attempt to argue for the Land domain as the decisive and most important. Which is nonsense. You win wars by imposing costs, and potential future costs, that the enemy state or nation are unwilling to tolerate. You a coerce a decision. That’s it. There is no special premium on ‘the land people live on’. Similarly, the ongoing fetishisation of SF as ‘the tip of the spear’ needs to end. I think it infantilises the rest of the military and damages our overall effectiveness.

[1.1] e.g. see last year’s Liaison Committee report on Strategic Thinking in Whitehall, and multiple Committee reports before: 2011, Public Administration Select Committee, on the 2015 SDSR, on Syria, on the 2018 NSCR and Modernising Defence Programme

[2] I think abandoning the Indo-Pac tilt is an error. We are focusing on NATO, by which we really mean a land war in Europe most likely an attack on the Baltic states, just as Russia is bleeding itself white in Ukraine, it’s war economy hollowing the country out unsustainably, when we have two new powerful continental allies in the alliance, Sweden & Finland anchoring the Northern flank and now Article 5 committed to aid any member that is attacked. We are doing this when continental Europe’s Continental Powers - Poland in particular, but the Czech Republic, Romania (transforming into a serious military power on NATO’s eastern front), the Netherlands and Denmark are all rebuilding credible land forces. France, long prepared for expeditionary war, now preparing for high-intensity conflict on the continent - and most notably Germany too all rearming for this land war - continental powers building continental armies to fight continental wars [see my article: https://wavellroom.com/2022/04/06/ukraine-the-integrated-review-land-power-in-europe/]. In such a situation, a UK maritime, air, cyber and space commitment to NATO, with the flexibility to secure our maritime nations security and economic interests - and make a massive contribution to the defence of Europe. Labour’s politically driven ‘NATO first’ mantra, is interpreted by Whitehall as meaning they can forget ‘East of Suez’ again - an error when the deeper and more enduring threat and more likely war comes from and with China, and geoeconomic systemic competition (see IR 2021) from China threatens UK security at home and interests globally - esp in the Indo-Pacific, where the world’s largest middle class will soon be and where the world’s fastest growing economies are. I would have us plan not vs Russia with NATO but vs a range of very specific scenarios. Interested readers might see Murray & Knox’s ‘Dynamic of Military Revolutions’ (especially p. 192) for more on this last point: ‘…revolutions in military affairs have emerged from evolutionary problem-solving directed at specific operational and tactical issues in a specific theatre of war against a specific enemy. Successful innovators have always thought in terms of fighting wars against actual rather than hypothetical opponents, with actual capabilities, in pursuit of actual strategic and political objectives.’ [bold added, italics in original].

[3] Dolman, E.C., 2009. Victory through Air Space Power. Strategic Studies Quarterly, 14(2). Available at: https://www.airuniversity.af.edu/Portals/10/SSQ/documents/Volume-14_Issue-2/dolman.pdf [Accessed 10 March 2025].

Fletcher-Jones, C. (2024). In Favour of an Independent Royal Space Fleet: The Smuts Report and the Precedent of the Royal Air Force. The RUSI Journal, 169(1–2), 22–32. https://doi.org/10.1080/03071847.2024.2359405

We’re worried. On the public evidence, current UK military understanding of artificial intelligence, both present capabilities and the trajectory of its development, is dangerously inadequate. The consequences of this misunderstanding are existential: for our service personnel on the future battlefield, for our armed forces in future combat, and for humanity itself.

The danger is twofold:

First, UK Defence radically underestimates current AI capabilities and the speed of their advance. If an adversary adopts AI-enabled decision-making and autonomous capabilities, and we don’t, we lose.

Second, UK Defence is not considering the implications for humanity of Artificial General Intelligence and war. Without solving AI alignment—the challenge of ensuring advanced AI reliably follows human intent—we risk handing power over humanity’s most lethal weapons to misaligned, super-intelligent machines.

We make three arguments, each logically building on the other.

1. AI-driven decision-making and autonomy are essential for our national security.

2. Embracing AI-driven decision-making necessarily involves Defence leaders in the broader AI alignment debate.

3. Therefore, Defence leaders must actively engage in this debate, clearly articulating the "Strangelovian" dilemma we face: the paradox of urgently needing AI-driven military capabilities despite the existential risks posed by AGI.

Caveat

Before we get into that, we want to be clear on what we are and aren’t saying in this article.

Firstly, we recognise the need for a major programme to ensure the UK can, to the use the USAF phrase ‘fly, fight and win’ in future warfare. We are not opposed to the investment in this essential capability.

Second, the RAF’s decision to run the podcast series and talk openly about its next generation fighter aircraft ‘Tempest’, the wider Future Combat Air System (FCAS) of which it is a part, and the international collaboration, the ‘Global Combat Air Programme (GCAP)’ that is to design, manufacture, and deliver a next-generation crewed combat aircraft, is an admirable commitment to transparency.

Third, those speaking on the podcast showed courage in speaking publicly, spoke well, in the difficult circumstances of being on the record in an informal environment, and are to be commended. Without such transparency, debate is almost impossible.

Fourth, the announcement of the RAF’s procurement of StormShroud, an autonomous support to crewed fighters (or ‘loyal wingman’) is a step in the right direction, to be celebrated, it does not change what is argued herein.

We hope to show why debate is necessary, but recognise we risk reducing the willingness to be open in the future. We hope that is not the case. In writing this post we wish to be clear that our concerns relate to the wider issues signalled in the Armed Forces’ understanding of Artificial Intelligence and autonomy, and are not aimed at individuals. We offer constructive criticism, genuine concern. We are seeking to play the ball, not the man.

Why pick on Tempest? Because we judge that the air environment is the domain which will be automated first, and because the long timeline of its development – 15 years before it enters service, ~45 years before it is expected to stop serving as the RAF’s frontline fighter – are well in excess of most estimates for when we reach AGI.

AI Capabilities

UK Defence currently misunderstands, and dangerously underestimates, the capability and trajectory of AI systems. A recent illustrative example: The Economist’s Shashank Joshi noted two concerning claims^[1] on the RAF’s otherwise excellent podcast^[2]:

1. AI today is just a ‘just a stochastic parrot, it is just advanced auto complete’^[3] … &

2. ‘We’re prepared for the time when AGI does catch-up.’

The first claim is heard widely in UK Defence, sounds smart, but isn’t. As Shashank noted on Bluesky, it is indicative of a ‘slightly impoverished view of frontier models that you see in defence & even parts of the intelligence world’. Commenting, Professor Ken Payne suggested this view reflects Defence’s ‘overconfidence, and signalling…expertise via knowing cynicism…’ rather than genuine understanding.^[4] Both Payne and Joshi are well connected and trusted commentators and experts within UK defence. They highlight that this is a bigger issue than a throwaway comment on a podcast.

The “stochastic parrot” concept originates from Emily Bender’s 2021 paper published over 12-months before the release of ChatGPT. Today the idea is a misleading caricature of AI models. LLM’s do not just parrot what they were trained on. Here’s one of the earliest academic deconstructions of what is wrong with that claim, from August 2022. Here’s Geoff Hinton, Demis Hassabis, Tomaso Poggio, Ilya Sutskever – leading experts in AI - discussing how AI can reason in analogies, and be creative (2024). Here a paper from 2024 showing evidence LLMs can conduct inductive out-of-context reasoning, make inferences beyond their training data. Similarly, mathematician Stephen Wolfram explains how language is a world-model, from which LLMs are likely deriving their output. The ‘stochastic parrot’ claim is a zombie idea, long dead, it walks on.

The problem isn't just a misunderstood technical detail. The problem is that this error compounds into the future. It is a misunderstanding of what is likely to happen and at what speed. It misunderstands the stakes at play and the odds.

Three recent examples of how quickly AI is advancing, and why it matters:

1. ‘Humanity’s Last Exam’ (January 2025): There is a new AI benchmark test called ‘Humanity’s Last Exam’ a selection of extremely difficult questions which AI was initially very poor at.^[5]But AI is improving (six times better within two months) and is expected surpass the test within 12-months.^[6] Imagine if you sat the hardest exam ever written, flunked it, and then were improving at a similar rate – how smart would you be in 2-years, in 3, in 5?

2. AI’s Persuasive Power (April 2025): This week, we learn via an unauthorised experiment on Reddit that AI is now performing better than humans in persuading people to change their minds. Six times better.

3. Rapid Task Scaling. AI’s capability to perform increasingly lengthy human-equivalent tasks has surged dramatically. The length of tasks, judged by how long it would take a human, that AI can now complete is increasing. In 2022 AI could complete software coding tasks that would have taken a human 30 seconds. In 2025 they can complete tasks that would have taken a human 1 hour. The pace of advance (2024-2025) has gone from doubling every 7 months, to doubling every 4 months. If you understand exponentials, you’ll get the claim that this means agents will be doing tasks it normally takes a human 1-month to do by mid-2027. As one of us noted in a talk this week, at this rate of improvement, by 2029, the end of this Government’s Parliamentary Term and the Trump Presidency, AI will be doing tasks that take humans 2000 hours labour - roughly equivalent to a human working year - doing this in days, hours, maybe even in minutes. By 2029. …FCAS is planned to come into service in 2040, and likely goes out of service ~2070.^[7]

Can those ‘straight lines on graphs’ that have given us exponential progress in AI just keep going? The answer seems likely to be yes, until at least 2030. A recent Epoch AI study suggests “2e29 FLOP training runs will likely be feasible by 2030.” That’s 200 quintillion FLOPS. If that doesn’t help you, over on Less Wrong they explain: “In other words, by 2030 it will be very likely possible to train models that exceed GPT-4 in scale to the same degree that GPT-4 exceeds GPT-2 in scale.² If pursued, we might see by the end of the decade advances in AI as drastic as the difference between the rudimentary text generation of GPT-2 in 2019 and the sophisticated problem-solving abilities of GPT-4 in 2023.” You’ve probably not really heard of GPT-2, it predates ChatGPT (GPT-3) by 3-years. It’s like the jump between Model T Ford, when you could still argue horses were better for lots of work tasks, and the cars of the 1980s, when no one sensible was making that argument. We are looking at a similar or greater advance again by 2030.

These are some of the latest indicators. This blog has covered where we are going and how quickly we are heading there, particularly in ‘Don’t Blink’ and ‘In Athena’s Arms’. On the Wavell Room, we have all argued independently that ‘the future is uncrewed’. We’re on a track to AGI, perhaps as soon as 2026, but with a median Metaculus estimate for ‘oracle’ AGI by 2028 (a year earlier than was forecast throughout 2024). Taken together, these examples starkly illustrate that AI is advancing at a pace Defence is failing to appreciate or prepare for adequately. To underestimate AI is to risk strategic defeat.

The AI Quarterback: Why Tempest Does Not Need Humans

Let’s explore the claim in the podcast that Tempest, the fighter aircraft at the centre of FCAS ‘definitely won’t be remote’, for which we read ‘won’t be fully autonomous’, and that there is therefore a necessity for a human ‘quarterback’ to coordinate autonomous and crewed platforms in attack and defence. We think this claim rests on multiple flawed assumptions, and misunderstands both technological progress and operational realties.

Why would it be impossible for Tempest to be remote? One answer is that any kind of electronic emission can be detected, and so you can’t have a datalink to the aircraft, as you do for current remotely piloted drones. There are unchanging physics aspects to this argument. The range of the radio link to control an aircraft is defined by the power of the emission. A long-range aircraft, essential to operate over enemy terrain, would need very high-power emissions, which makes it easier to detect.

But this smuggles in an assumption, that there has to be a datalink if there isn’t a human in the cockpit. That ignores the speed of technology development. If an AI is able to take inputs of sensor data, process them faster and respond more effectively than humans, it can be fully autonomous. If it is fully autonomous, it doesn’t need a data link.

The argument that a human ‘quarterback’ - the ‘player’ coordinating the attack - is a necessity also assumes the crewed ‘quarterback’ platform must remain in (relative) close proximity to its autonomous wingmen. It must be close to reduce emissions and avoid detection when coordinating the attack. If not close, then the human quarterback has the same datalink problem that the long range remote piloted aircraft had. Proximity enables low-powered or visual communications, reducing (though not eliminating) detectability. However, this constraint applies equally whether coordination is human or AI-led. Although fully "zero-emission" platforms may be unrealistic, AI-managed coordination would likely significantly minimise emissions, surpassing even low-emission human-led approaches (AI communication likely more efficient). Thus, human presence offers no inherent advantage in emission management or stealth—AI coordination can achieve equal or greater effectiveness with reduced proximity constraints.

A related argument is that you need a pilot because maybe electronic warfare – jamming, spoofing, frying of the aircraft electronic sensors and systems – would cause a fully autonomous fighter aircraft to drop out the sky. This is indefensible. Current fighter aircraft are ‘fly by wire’. To optimise for agility they are inherently unstable, and the only thing that keeps them in the air is the computer constantly adjusting the configuration of the aircraft’s control surfaces. If an adversary is capable of jamming, spoofing or frying the electronic systems of an autonomous fighter, then they are equally capable of incapacitating a crewed aircraft, rendering the presence of a human pilot irrelevant to survivability. The distinction between crewed and uncrewed aircraft in terms of vulnerability to electronic attack is therefore illusory, and the solution lies not in insisting on human pilots but in hardening aircraft electronics against such threats.

Then there is the argument that there are minimal to no trade-offs involved in including a pilot in the cockpit, so long as the aircraft, as is the case with Tempest, can be optionally crewed. Again this seems indefensible.

In all major wars, pilot loss has been more of a limiting factor on a nation’s ability to fight than aircraft loss. We can manufacture aircraft faster than we can train pilots. This was true of both RAF pilots in the Battle of Britain and Luftwaffe pilots in the Defence of the Reich. In the Ukraine war, shortages of pilots has limited both Russian and Ukrainian air operations – and been specifically cited as a reason for delaying providing Ukraine with F-16s.

Another argument is that pilot skill is irreplaceable. Yet today AI-driven drones have begun to outperform human pilots in complex, high-speed drone racing scenarios, dodging drones through woods and around complex obstacle courses at impossible speed with super-human agility, signalling a significant shift in the capabilities of autonomous flight. The USAF’s Venom programme sees autonomous ‘VISTA’ F-16s dogfighting (Basic Fighter Manoeuvres), something we were told in 2020 wouldn’t happen for ‘in excess of a decade, maybe two’. That estimate came from a USAF pilot who was a relative AI optimist, schooled in DARPA’s simulator-based AlphaDogFight trials, where the humans lost. As another of those defeated pilot’s admitted ‘“If I were to walk away from today saying I don’t trust the AI’s ability to perform fine motor movement and achieve kills, I’d have a lack of integrity…”.

The quarterback analogy overlooks another crucial ‘skill' limitation: humans have extremely limited capacity for simultaneous attention. This is why texting while driving is illegal—it multiplies crash risk by a factor of twenty-three. Coordinating multiple combat aircraft is vastly more complex than texting, demanding constant real-time awareness, quick decisions, and continuous communication across multiple platforms. Human pilots simply don’t have the cognitive bandwidth to fly their own aircraft effectively while simultaneously managing a swarm of others. Swarming technologies, multi-agent systems, can do this better, faster, more responsively. And surely after AlphaGoZero in Go (2017), AlphaStar in the game Starcraft II (2019) – which required breakthroughs in AI that could account for game theory, imperfect information, long-term planning, real-time, large action spaces, we don’t need to make the case that AI is superior in coordinating complex tactical plays? It’s a big bet against the odds to suggest you’ll need a human quarterback for their tactical skill, when Tempest is deployed in 15 years (2040), let alone in 45 (2070) when the aircraft is expected to still be operating.

Then there is the argument that aircraft capability isn’t majorly impacted by having a human in the cockpit. But the limitations this places on how many ‘g’ an aircraft can pull is a problem. Even marginal differences can determine whether an aircraft is taken out or not. The presence of a human pilot imposes physiological limits on an aircraft's manoeuvrability, particularly regarding g-force tolerance. While modern fighter jets are structurally capable of executing manoeuvres exceeding 9g, human pilots typically can withstand only up to 9g for short durations, even with the aid of g-suits designed to mitigate the effects of high acceleration forces. Exceeding this threshold risks G-LOC (g-force induced Loss Of Consciousness), compromising mission effectiveness and pilot safety. Unmanned aircraft, free from these human limitations, can exploit the full aerodynamic potential of their designs, allowing for more aggressive manoeuvres.

Maximum load factor (‘g’) is not the only important factor in air combat manoeuvrability that full autonomy aids. An AI would like hold an aircraft at the edge of its performance envelope much more effectively. A further transformation will come from the AI quarterback & autonomous systems’ ability to execute precisely coordinated, multi-aircraft tactics—complex three-dimensional manoeuvres performed with superhuman timing, precision, and strategic coordination. Uncrewed aircraft collaborating dynamically can preserve formation energy, manipulate adversaries into vulnerable positions, and optimise defensive manoeuvres to evade threats with accuracy no human team could manage.

Enhanced agility - individual and formation - can be crucial in combat scenarios, where even marginal improvements in manoeuvrability can significantly reduce the probability of being hit by enemy missiles: high-g manoeuvres in the end game can be the difference between the aircraft being destroyed or the missile being evaded. There’s a reason pilots and intelligence officers scrutinise ‘doghouse plots’ that illustrate an aircraft's performance envelope vs those of adversary aircraft and adversary missiles. It’s the difference between life and death. Mission success and mission failure.If having a pilot in the cockpit imposes even small limits on manoeuvrability, then the cost of having a human quarterback is reduced probability of mission success.

Then there is the question of the wisdom of a programme that takes 10-20 years to deliver a capability in an age of exponential technical progress. It was only 14 years ago that US Venture Capitalist Marc Andreesen argued ‘software was eating the world’ and it was edgy and futuristic – now everyone accepts this, and companies such as Anduril are challenging Lockheed and Boeing – while increasingly software driven drones are transforming warfare and rendering many capabilities obsolete. Twenty year-old assumptions are likely to be wrong, and costly – and they are likely to be more wrong 20-years in the future, than they were 20-years ago to today. Today progress is accelerating faster than ever, and yet still poised to leap to even greater speed. We are on the brink of Dario Amodei’s “…‘compressed 21^st century’ the idea that after powerful AI is developed…[Amodei suggests in 1-3 years]…we will in a few years make all the progress in biology and medicine that we would have made in the whole of the 21^st century.” We think Amodei chose biology and medicine as fields that are media palatable – this would be just as true across the sciences, and thus weapons and military equipment development.

Is it safe to bet on a human ‘quarterback’ in an aircraft in 2040, and change later if that turns out to be wrong? No, there are consequences. Getting this bet wrong, means spending money on developing a platform that will be rendered obsolete at some point prior to its deployment in 15-years' time, or within the next 45 years between now and the aircraft’s projected end of service life.

The risk is that we are wasting money investing in the wrong things, and will waste more in the future, since changing designs is a major cause of MOD cost over-runs.

Then there is the issue of opportunity costs. The MOD has budgeted £12bn over ten years for FCAS. Deepmind costs Google between ~£164M-£477M a year, or ~£10M per study.^[8] Imagine what we could do if we invested similarly well.

AGI, Defence & The Future of Warfare

The claim made on the podcast —“We’re prepared for the time when AGI catches up”— implies Defence possesses concrete plans and capabilities for effectively managing advanced, human- or beyond human- level artificial intelligence – either our own, or that deployed by an ally or adversary. Yet the evidence presented thus far suggests the opposite.

The current Defence position—that by 2040, a human operator in an aircraft like Tempest will outperform AI in processing data, coordinating multiple platforms, situational awareness, and rapid decision-making—demonstrates precisely how unprepared we are. As shown earlier, AI systems already equal or surpass human performance in complex strategic tasks, and the trajectory of improvement strongly favours AI dominance. Even if human pilots retain marginal advantages today in certain areas, these advantages are diminishing rapidly, soon to disappear entirely.

The claim thus resolves to, in 2040, UK defence believes it is very likely that a human systems operator will outperform an AI in processing data, and controlling subordinate platforms, a human operator will have better situational awareness, and be able to react faster.

On any reasonable appraisal, as we have shown, this seems overwhelmingly unlikely: indeed, given AI’s current performance in air-to-air combat and strategy relative to humans, it may not be true today: processing and analysis of data, control of networks and speed of reactions are measures where humans do worse.

Let’s push this logic further. If we hold the platform and support constant, which dominates:

1. Human-only control, or

2. Human-plus-AI control, or

3. AI-only control

Whichever proposition of the above is true will win. Tempest is a bet on the second. All the evidence to date suggests that not long after human-machine teaming wins, the AI beats the human machine team. Our planning is stuck short of the logical conclusion of its own assumptions.

Defence might understandably prefer retaining human pilots from a moral cultural or aesthetic preference for a pilot in the cockpit. But what matters is whether your adversaries share your preferences. If they don’t, as soon as one adversary picks truth over beauty, you must change, or you will lose. It is that simple. Some might object that adoption will be uneven. This is true and also irrelevant. The moment any credible power demonstrates a battlefield advantage via AI, the pressure to replicate it becomes existential. The delay is strategic suicide, not insulation. Unlike many previous candidates, this is a true horse-to-tank moment, except an order-of-magnitude more so.

Which one will be dominant in the 2040s cannot of course be known. But humans aren’t getting significantly better and artificial intelligences are. Why do you believe humans will win this race?

The implications extend well beyond fighter aircraft. The AI frontier will be jagged – we think air-to-air combat will be fully autonomous first, then air-to-ground, with other domains slower. Air warfare offers straightforward reward functions (seeing what’s there, shooting things down, blowing things up, moving things around), in *relatively* uncluttered environments (compared to ground and sea – surface and subsurface – warfare) – different capabilities will be automated at different time horizons. But unless the trajectory changes, what is true of fighter aircraft is true of every other military system. AI is likely to be better at operating all of them: there isn’t one domain or system where it is obvious that humans will retain decisive advantage, or any advantage at all.

AGI makes people advantage redundant everywhere, from pilots to operators, commanders, to intel staff. In doing so, it removes demographic inferiority – a nation with less well trained, well educated, or well-motivated troops, a nation with fewer people, is no longer less able to defend itself.

Our argument then is that betting against these clearly visible trends—hoping humans will somehow maintain a meaningful edge—is dangerously naive. If Defence truly believes human superiority can persist, it must explicitly justify why human capabilities will indefinitely outperform rapidly improving AI systems. Moreover, it must convincingly refute or challenge the well-supported forecasts placing AGI’s arrival around 2028.

On the other hand, if Defence accepts the logic and evidence we have laid out, it should be racing towards our AI and autonomous future to fulfil its duty to protect the nation.

Humanity’s Last Exam: Existential Risk

We now make a larger claim. If you accept the logic of the argument so far, then our generation is sitting humanity’s final exam as a civilisation. To pass this exam, we must resolve at least one of the following critical issues:

1. Globally prevent further AI development.

2. Remove the possibility that advanced AI can access advanced weaponry or highly dangerous materials.

3. Eliminate war entirely as a political instrument.

4. Solve advanced AI alignment.

If you can’t do that, the logic takes you to a civilizational test for humanity: what happens when the optimal move is to hand over our most destructive artefacts to non-human intelligences?

Dystopian? Yes, we agree. So what are we going to do about it? Our options are clear:

1. Stop Developing Minds Smarter Than Humans. Humanity can stop developing powerful artificial intelligences. But that has to be all of it, any defection at all prevents this strategy from working. This seems very unlikely, and perhaps simply not possible.

2. Remove Tools That Smarter Minds Could Use Decisively. Accept the development of smarter-than human artificial intelligences but eliminate, globally, all advanced military technologies and crucially, the underpinning civilian technologies. In practice, anything that an AI could take control of and use against us. Again, it can’t just be done with pinky promises, it has to be done by everyone, everywhere with no defections. Misaligned AGI could theoretically weaponise almost anything - from disease to climate to the economy - against us, but clearly, handing it our most advanced military capabilities is an order of magnitude more immediate threat.

3. End War. Never go to war again, solve all problems by politics. Clearly, not happening.

4. Solve AI Alignment. Solve AI alignment so you don’t care if AI controls humanity’s most destructive weapons, because it will always do your/your elected representative/your elected AI leader’s, bidding. This might be done by luck – maybe AI always remains a tool of humans, aligned to us, despite being able to out-think us. Or, assuming this is possible, it might be done through deliberate and careful research and development to put in place regulation and guardrails etc. In any case, even if you think it unlikely (we have split views between us at Cassi), if it is not a zero chance, we still need to do it. After all, we insure against war by building a military, against all kinds of low-probability high impact risk in our day-to-day lives. This is what ‘prepared for the time when AGI does catch-up’ means in practice. Hope is not a strategy.

5. Cede Control of Destructive Systems to misaligned Non-Human Intelligences. Get killed by Skynet.

There aren’t actually any other options. That is your lot.

Can we please start taking this seriously?

So What?

We are not remotely ‘prepared for the time when AGI does catch-up’. Defence’s persistent underestimation and misunderstanding of AI is a national security risk, right now.

We need to address the issue of under-estimating, and insufficiently understanding, AI for the sake of the security of our nation.

Military leaders must recognise that their duty is twofold:

First, to decisively invest in and prepare for an AI-driven future of warfare;

Second, to candidly explain the Strangelovian escalation trap humanity faces. Achieving AI alignment is not primarily a problem to be solved by UK Defence – the excellent UK AI Security Institute should remain the lead for this. But nor is it an abstract concern that Defence can ignore—it is a strategic imperative that Defence’s leaders have a moral and ethical obligation to explain given their part in it. Failure to act and speak out now risks more than defeat; it risks existential disaster.

Subscribe now

^[1]

& Shashank Joshi on X: "RAF officer leading concepts/req for Tempest: "at the top of our list is the payload." Irrelevant how it gets to target, he says, but analysis shows fast jet better than cargo plane or sub. "Payload is what we're all about ... we're talking roughly double an F-35 A's worth..."" / X

^[2] Episode 14 – Meeting UK operational needs | Future Horizons: The Tempest Podcast (it is all worth a listening, but the most relevant sections are at the start and then from c.40 minutes in).

^[3] Ignoring that for a sufficient level of intelligence or skill, everything is auto-complete…

^[4]

^[5] And I am not sure any actual human could individually do well on it: 2500 challenging questions on a 100 different subjects.

^[6] Humanity's Last Exam

^[7] This footnote is for the nerdiest of technical pedantry. The measuring how “general” AI is at solving tasks is a very difficult challenge. You cannot test on anything a human might be asked to do, that’s an infinite test. What represents a good cross section of all the tasks you would expect a human to be able to do? Which human? In how long? How do you measure success? There are a million confounding variables. METR took on the approach by using length of time it normally takes a human as a proxy for task complexity. Does that mean that ‘being a pilot’ and ‘running a tactical battle’ was in that list? - No. Does that mean you can ignore the work? Not sensibly if you want to understand how generality in AI is improving.

^[8] This (2017) article https://qz.com/1095833/how-much-googles-deepmind-ai-research-costs-goog suggests costs of £164M, this 2020 article https://www.cnbc.com/2020/12/17/deepmind-lost-649-million-and-alphabet-waived-a-1point5-billion-debt-.html buts the figure at £477M, individual papers/studies are said (2024) to cost ~$13M, calculated here https://www.reddit.com/r/MachineLearning/comments/1ej5h4b/d_calculating_the_cost_of_a_google_deepmind_paper/

[Updated, 16 March 2025. 2209hrs: new Endnote]

Subscribe now

This was the most innovative military exercise I have participated in or witnessed in 22 years of service. Defence Cyber Marvel’s fourth iteration, DCM4, took place in Seoul recently and I had the privilege to attend. If you just want to read about the exercise, skip the intro, and jump to the section headed DCM 4. But first, here’s why I think it matters…

‘The rainbow of colors in the window paints how everything went so wrong, so fast. The water in the Potomac still has that red tint from when the treatment plants upstream were hacked, their automated systems tricked into flushing out the wrong mix of chemicals.

By comparison, the water in the Lincoln Memorial Reflecting Pool has a purple glint to it.

They’ve pumped out the floodwaters that covered Washington’s low-lying areas after the region’s reservoirs were hit in a cascade of sensor hacks. But the surge left behind an oily sludge that will linger for who knows how long. That’s what you get from deciding in the 18th century to put your capital city in low-lying swampland and then in the 21st century wiring up all its infrastructure to an insecure network.

All around the Mall you can see the black smudges of the delivery drones and air taxis that were remotely hijacked to crash into crowds of innocents like fiery meteors. And in the open spaces and parks beyond, tiny dots of bright colors smear together like some kind of tragic pointillist painting. These are the camping tents and makeshift shelters of the refugees who fled the toxic railroad accident caused by the control system failure in Baltimore.

FEMA says it’s safe to go back, now that the chemical cloud has dissipated. But with all the churn and disinfo on social media, no one knows who or what to trust. Last night, the orange of their campfires was like a vigil of the obstinate, waiting for everything to just return to the way it was. But it won’t.’

I remember reading this, August Cole & Peter Singer’s evocative “FICINT” (Fictional Intelligence) opening to the US’ Cyber Solarium report, in Downing Street in 2020. A few years before, I’d read August Cole’s brilliant short story Underbelly, that helps us explore the direct military effects of a cyberattack on the UK. It is no less alarming, resulting in a distributed HQ led by a Special Forces Colonel ‘Fessenden’, having to operate clandestinely, out of a rural pub pulling together the talents needed, with all major bases and MOD cyber networks shut down due to cyber infrastructural attacks. If you can, do read it.

In 2020 I was with our No10 Integrated Review team at a meeting with the Defence Secretary and senior Defence and National Security Leaders. One Treasury official asked what it would mean if we cut the defence budget for aircraft carriers and reassigned it to offensive and defensive cyber. Might we be a more dangerous opponent? Better able to deter, and safer at home? Might spill-over benefits to the wider economy be greater? More nationally resilient?

I don’t think this was a serious proposition, but a challenge – a provocation designed to force people to think. And it is one I have often thought about since. I am willing to bet it has never been seriously and fully explored in the UK.

For me, personally, on issues like the aircraft carriers, FCAS/Tempest, Challenger 3 tanks and all the other totemic and high-end kit that comes up for regular scrutiny, my primary interest is in how we make the decisions on force structures and balance, not what specifically we should have.^[1]

Cyber Power debates in Whitehall, like those on Space Power^[2], are driven by fundamentally misaligned incentives. Just as air power needed an independent champion to compete equally for resources – the purpose the creation of the RAF served – so cyber and space will never get to make their arguments with full force for as long as they have to make their case through the RAF, Army, RN, or Strategic Command, (at least while Strategic Command is run by officers whose careers depend, in part, on keeping their parent services content).

In fact, for cyber the situation is worse in many ways. Not only does it have to compete with the other military domains of war but it also has to compete for resources with GCHQ. GCHQ’s mission is primarily espionage. As a civilian led and run organisation, it cannot use lethal force, and so cyber-attacks that might cause destruction, death or injury must be carried out by military personnel. But GCHQ is funded by the Foreign Office, not the MOD. The FCDO does not want to hand the MOD an argument taking funding away from espionage to redirect it to military cyber defence and offence.1 Thus cyber, in the sense that August Cole was talking about, never gets to air its arguments for why it should be funded ahead of aircraft carriers, tanks, FCAS, and espionage etc. Our Treasury official’s question remains largely unanswered and unexplored. I don’t think we know how vulnerable we really are. Nor do I think we truly know how much we could do with offensive cyber capabilities if they were fully funded and had an independent champion. August Cole’s cyber FICINT helps us see how we might first learn about our vulnerabilities.

DCM4 is helping ensure our ‘Fessenden’ will be able to find the people he needs to fight back.

Defence Cyber Marvel 4 (DCM 4).

A large hotel in Seoul. Down lifts and up escalators. Into a labyrinthine complex of a large open plan convention centre and surrounding smaller convention rooms and offices. Innovation everywhere. I am shown around this multinational exercise with other visitors, shown what can be done at the unclassified level - the briefings I am given are given to local media, visiting multinational dignitaries, academics, business and industry representatives alike. The capabilities we see are more remarkable for the fact that they are in the public domain.

In one corner a group of soldiers from 2 PARA discuss how they have put together a number of electronic warfare sensors from commercially available hardware. They bought parts on Amazon, on a shoestring budget, with a focus on Counter UAS. They, and Royal Signals soldiers present say these devices are perhaps more effective than those they would currently have access to were they to deploy with the officially provided EW support equipment of the Light Electronic Warfare Teams (LEWT).

Whether the kit itself is better or not isn’t really the point though. As they note, the creation of these devices would enable a proliferation of EW and ISR across the battlefield that is not possible with the current force structures where these capabilities are in high demand and insufficient quantity. The Ukraine war has at times been referred to as the ’24 hour’ war, where technological advancements are only effective for a short period before being countered. Creating a culture in which innovative soldiers can continually develop and refine technological advantage themselves will enable us to maintain the initiative.

It makes me think of the culture inculcated in the Israeli Defence Forces, one of Ein Beira, or ‘no choice’; of tinkering and trying and taking risk, modifying kit, solving problems. It sits in damning contrast to the process of building under maximal regulation, assurance, validation and verification. This has long suggested to me that we have unintentionally killed the tinkering innovation spirit in our armed forces outside these bands of dynamic volunteers and novel thinkers. We have built a culture where the soldier, sailor, airmen and women are expected to passively wait until the kit he or she needs is handed to them with a training manual. As Bernard Schriever wrote of the US DoD in the 1960: “the system now runs the people”. DCM 4, it seems to me, is an antidote. And the PARAs seem to be thriving with having been given permission to innovate, to test and try, in hardware and software – not things we traditionally think of as Parachute Regiment specialities, and a reminder of how capable so many in our Armed Forces are.

In another corner, a young RAF Officer speaks passionately about how much she is learning, how she, and over 1000 participants from 60 different organisations, including 27 nations working in 20 worldwide locations, are mostly volunteers. All are self-taught. They have given up evenings and weekends to learn how to become hackers – they did this outside of military time. Now she is leading a team of Brits from the RAF and Americans, competing against teams from other services and other countries; and also, fascinatingly, other UK Government Departments. The Ministry of Justice has a team here. The NHS too. The Met Police and National Crime Agency. There are teams from across the UK military, though Army personnel and units dominate, since it is primarily the Army, through the Army Cyber Association, that organise the Exercise. Here too are representatives of most of the largest tech firms both globally and in the UK. All compete equally. The strongest performers are not always the teams you’d expect. But the Exercise organisers protect everyone, keeping results - bar those of the teams that win -between them and the teams. The focus is on winning, but also on learning from failure, so any embarrassment must be minimised.

Four and Half Bad Days in Cyberspace

The teams spend 10-months organising and preparing for what they call 4.5 bad days in Cyber Space. They are organised into small rapid reaction teams on the ‘Blue Team’ side. They are tasked with defending unfamiliar networks and systems – from Enterprise IT systems to sophisticated industrial control systems and transport networks on which nations are dependent. There is a large ‘Red Team’, ramping up attacks on them that Blue Teams must defend against, growing in intensity and sophistication as the exercise goes on. A ‘White Team’ provides quantified and objective feedback on performance. There’s a leaderboard. A ‘Green Team’ builds the safe training area competing teams operate in before then defending the whole exercise from real-world cyber-attacks and espionage from outsiders. There are numerous ‘side quests’ – innovation experiments in which the Paras were engaged.

The Green Team is led by a young Army officer. She describes how her all-volunteer team has spent the past 10 months designing and building on the NATO “Cyber range”, essentially a server rack in Estonia within which they have ‘Deployed Infrastructure as Code’ – which is to say they have systems that exist electronically exactly, or as close as possible, to their form in the real-world. Now it is built her team spends their time keeping the network operational while watching for suspicious traffic on the network. If anything looks like it shouldn’t be there, they train to both defend and attack.

The Cyber Range shows how satellites can be hacked, and how not to make this obvious. For example, turning a satellite’s solar power arrays to face away from the sun, leading to a gradual loss of power and functionality that isn’t obviously a cyber-attack. There are lots of ways this might happen – so harder to detect and very effective. They can get a weather camera to misreport, leading to the satellite operators using it at times when the weather makes it less effective, or to choose not to use it when the weather is optimal, because they believe its view occluded by weather [reminder, this is all public domain: see also the DoD’s GitHub ‘How To’ Guide. on satellite hacking, or this online hack-a-sat competition to hack satellites].

The Cyber Range has vulnerabilities in industrial control systems and simulated military command and control networks. They have ‘real systems’ hardware that you might find in a hospital, or alongside train tracks, and players can try to hack it. Sometimes these are linked to wider simulations of, for example, a whole hospital, or a wider power grid. Sometimes digital targets are linked to models – for example, an aircraft carrier or a Hornby-like train set – so you can see the effects of their being hacked on the scale model or the wider simulated system. Sometimes this can find real-world vulnerabilities, something that the exercise organisers are careful to ensure are reported correctly, and knowledge of which is kept contained. The whole team tests and learns while teaching others on the exercise. It is hard not be impressed.

In another corner, a team is hacking a commercial satellite – a real one – hired for this purpose by the exercise organisers. The satellite is, in the exercise, an adversary’s surveillance satellite. The team on this side quest insert malware so they can watch the enemy through their own surveillance satellite.

Another side quest: a team set to sinking an aircraft carrier – fortunately not a real one this time. The team first gained access to the ships GPS, making the crew uncertain as to its location. They make the ship ‘think’ and report that it was many miles from its real position. Next, the team hack/access the control systems, sending its engines into overdrive and send the ship circling on full power. Turning at full speed causes the ship to list hard. Next, the team, deep inside the ship’s systems, order its ballast system to shift weight rapidly within the hull. Combined with the listing, the carrier passes the point of no return, rolls, and sinks.

There’s an AI Defence Challenge – prompt injection attacks lead another AI to churn out wrong answers, subtly or obviously, or to generate misinformation, or propaganda. There’s drone hacking and spoofing. Teams from all over the world have trained LLMs to play poker. That might not sound like the fare of war, but it has very relevant attributes: incomplete information; uncertainty; a mix of objective and subjective probabilistic assessments – what cards might my opponents have and what might they do?; recursion – what does my opponent think I might have and do, what do they think that I think that they might do, and so on; and of course the staple of the nature of war – chance^[3]. The poker competition was won by the Japanese Maritime Self-Defence Forces (the Japanese Navy). The point here is to begin experimenting with AI decision systems. The first time I have seen this get serious attention on a military exercise, even if it is as a side-quest and game.

I'm told that at last year’s DCM3 the exercise incorporated quantum computing, running an air defence scenario, and using quantum (presumably annealing/combinatorics) for ISR planning & execution, and logistics planning optimisation. This year some early experimentation with quantum radio frequency detection has been seen, something to be expanded on in future exercises.

I head to another stand where a rifleman describes his Rifles’ Cyber Teams and the experimentation they are doing supporting tactical actions with cyber support. Inspired by DCM – they are now using Notebook LLM for briefings and analysis, and Google Threat Detection as part of their workflows, with aspirations to widen the use of such tools from tactical experimentation into their headquarters processes.

Across the exercise there are competing priorities for the Blue teams. Keep the hospital system up and running to save lives or shut it down to protect patient data integrity, knowing that if someone gets into your patient data, you might never be able to trust it again? Treatment itself becomes a threat, if you don’t know your patient’s medical history. Teams are constantly called on to prioritise. The attacks they face include new, novel, cutting edge techniques – perhaps something seen just once or twice globally before – exposing the defenders to these emerging threats to ensure they are ready to face them in the real-world should they come calling.

One of the insights I leave with is that a truly intense cyberattack on the UK, or the UK military, is not something we can defend in the way we might try to hold a frontline, or how we might achieve complete control of the air. However long a border in the physical battlespace, the attack surface in cyberspace is simply vast in comparison. As a result, in a cyberwar, we will be hurt – a successful defence can only reduce harm. Maybe the waters swirling around Washington, the crashed delivery drones, the displaced military headquarters operating from a pub – all these ‘defeats’ in August Cole’s dystopian stories were in fact the losses we accepted, as we kept incubators in neonatal wards running, and defended our nuclear power stations and submarines from cyber-attack.

Destroy & Degrade. Reconstitute and Rebuild

The exercise’s organisers are proud of what they’ve built, and rightly so. As they say, you can’t build an effective Navy by staying in the harbour. DCM 4 gets UK military talent preparing for the wars of the future. The breadth of the teams entering, from across industry, defence, public sector and across countries, makes the training unique and the connections for all parties hugely valuable. These are the people they would fight alongside in a real ‘cyber war’.

DCM aims to increase the skills of everyone involved, and succeeds brilliantly. The teams defend systems and attack them, seeking to destroy, degrade, reconstitute and rebuild them. Everyone involved says there is nothing else like it. It succeeds in part because it is not a formal military exercise. Run by the Army Cyber Association, - set up by soldiers who wanted to enjoy cyber similarly to way the British Army funds others to row the Atlantic - and all by volunteers, it has freedoms that would never be granted to a formal exercise. It can be less deferential to organisational fiefdoms – wrapping cyber offence, defence, civ, mil, industry, espionage and international participation without having to do quite so much politicking across those misaligned incentives.

In DCM4 there are echoes of the early innovative days of the RAF, which I wrote about for the Wavell Room in an article they titled: Saving the Royal Air Force. That article described how private contributions, media campaigns, contests and clubs were instrumental to building the capabilities the RAF would later need, creating many of them before it existed, allowing the nascent RAF to harness them rather than having to build them from scratch. Furthermore, contributions, campaigns, contests and clubs created groups who could pressure Whitehall and the Ministry of Defence to act on Air Power, giving them the evidence, showing how powerful it could be, and building public pressure for proper investment. In an age when budgets are even more tightly constrained than they were in the 1930s – with welfare bills and health spending more tightly tying the hands of the state - a similar approach – using clubs and media campaigns, contests and private contributions on cyber, space, electronic warfare and uncrewed systems might be the only way to prepare. The incentives in the system to go slow and under-allocate on all these vital areas are just too strong.

The intrinsic motivation of all involved to improve their skills and put on a world-class exercise is why it works. Formalise it and you’d kill that intrinsic motivation, get rid of the freedoms and the innovation would go.

On the other hand, the exercise as-is needs some formal approvals from within the MOD if is to be allowed to go ahead. As it grows, these are becoming more challenging to win. It needs approvals for cyber reserves that wish to participate to mobilise and join, it needs approvals to win budget whether as a ‘sport’ like rowing the Atlantic, or some other capacity. Likewise if it is to be able to get the soldiers, sailors and RAF personnel released from their day jobs to join as volunteers, it needs some official standing. Partly because of its success – it’s hard to compare it now to a small team rowing the Atlantic to justify funding and approval – and in part because it is so innovative - these can be hard to win. It doesn’t fit neatly into boxes – it isn’t a formal exercise and yet it is vast. It hasn’t been through the normal processes of training needs analysis and multilevel authorisations, sign-offs and so on. These things would likely kill it, but they are in place for a reason. It seems unlikely the exercise can continue as it is. The organisers seem genuinely worried as to whether DCM5 will be approved in time to be put it on next year, or at all.

Yet this year’s Exercise was put on with the full support of the South Korean Government. It attracted Foreign Office funding. Far more people applied to participate than the Exercise could accommodate.

Perhaps the lesson is to fund DCM through its fifth and sixth incarnations, but not seek to control it. Meanwhile to take inspiration from it in the formal exercise programme, and seek to match or exceed the levels of innovation it achieves. If this can be done, the organisers would likely welcome the chance to handover to the big green machine of military bureaucracy, knowing the fire of innovation they have lit is spreading, and being enabled to burn brighter and longer, wider. But it should be for them to decide the time to do this, when they judge the flame has been passed on, and they need no longer keep their torch burning.

Until then, Defence must find a way to ensure DCM4 continues to thrive as DCM5, DCM6, largely as-is. I don’t want to make this seem simple. All those rules and regulations and controls are in place for a reason. But find a way we must. It will need top-down leadership to succeed – it is in the middle-ranks of the military that the flame of innovation dies. Not because the people are bad, but because usually everyone with responsibility over some aspect or another of an exercise has to say yes, and only one person has to say no. Because if DCM goes well, those middle-level approvers will get no credit. But if it goes wrong, they will get the blame. And because absent a Cyber Service looking to drive the capability to excel, to win resources in the next SDR, there is no champion at the top without some significantly conflicted incentives.

For me, personally, I was extremely proud of to see UK Defence being as innovative as everyone knows it can be, but rarely is. To hear the passion of those involved, and watch them pushing boundaries and better preparing us for the future, was genuinely inspiring, and hopeful.

The exercise felt to me like it was an incarnation of ‘the Science Super Power’ agenda and the Indo-Pacific tilt we wrote about in the 2021 Integrated Review. But it was also explicitly inspired, it’s organisers tell me, by General Richard Barrons’ Warfare in the Information Age (WITIA) paper, published in 2016, but originally written and circulated as his internal paper and idea, in the years prior. That paper called out the growing threats and opportunities cyber warfare was creating, as a capability in its own right, but also, crucially, showed the need to integrate cyber warfare in everything we do.

Watching and listening to the personnel on this exercise, I think we have found the generation who can usher in the vision provided in General Barrons’ paper, make it a reality. But, if they are to succeed, Defence Cyber Marvel and its volunteers will need more support to overcome the innovation-stifling institutional pressures, the misaligned incentives, that have prevented us getting there in the ~15 years since the vision for Warfare in the Information Age, was first sketched out.

nb. with apologies to those that were kind enough to speak to me during my visit: I have chosen not to name anyone for security. I am, however, hugely grateful to you all for your time, insights, and commitment to excellence. You really were genuinely inspiring.

^[1] This is not to pretend I don’t have views, but I feel much more strongly about the need to improve how we plan and build for the future than any individual kit decision – debates around which are nearly always narrowly single-service dominated, dominated by discussion of what was, and to lesser extent what is, rather than what-will be, and selective in both the evidence and logic used to defend or attack them. I find these debates mostly unproductive and distracting.

^[2] Dolman, E.C., 2009. Victory through Air Space Power. Strategic Studies Quarterly, 14(2). Available at: https://www.airuniversity.af.edu/Portals/10/SSQ/documents/Volume-14_Issue-2/dolman.pdf [Accessed 10 March 2025].

Fletcher-Jones, C. (2024). In Favour of an Independent Royal Space Fleet: The Smuts Report and the Precedent of the Royal Air Force. The RUSI Journal, 169(1–2), 22–32. https://doi.org/10.1080/03071847.2024.2359405

^[3] AI is already super-human at poker see Libratus & Pluribus, and like in chess, AI is changing the way humans play the game, with a growing dependency on machines for training and in competition.

AI Disruption

It’s Happening. The AI revolution is well underway. Most likely it will accelerate both in rate and scope. We must consider what to do in the worst-case scenario that AI is a true substitute for human cognition, a subject we have discussed several times in this blog.

But also, we need a plan to mitigate the radically disruptive effects of AI even if no further advancements were made. The adoption of existing AI technologies alone would have profound economic and social implications. We propose using AI to radically improve how humans learn. We make the national case here - but it is as relevant to all businesses, organisations and us all, as individuals.

The Case for Optimism

While we know a lot of facts about the brain and human psychology, we don’t really know how either ‘work’. We are not sure what consciousness is. Intelligence is notoriously hard to define. Even reasoning, what constitutes knowledge or understanding, all are uncertain, contested concepts. Under such circumstances, we keep coming up with new benchmarks for testing AI in the way we have tested human intelligence, and increasingly with tests no normal human, and no deeply expert human, would pass. And AI keeps not just passing these tests but surpassing them. Hence the bearish case must be planned for.

But if we don’t know what human intelligence is or how it works, we cannot say that human intelligence won’t continue to play an important role in the economy, in making decisions in the future. It might, even if it is hard to say with any confidence how.

If we assume this optimistic case, AI will not replace all humans. AI becomes a complement to human cognitive labour. We work with it, and it creates new jobs.

Still, many will be displaced from their current roles. We can already see that lots of things we do now will be done better by AI, and could be done better by AI now, even if not all things. How do we plan for the scenario of massive displacement?

There are two key challenges: mitigating large-scale social disruption and ensuring that displaced workers transition into new, productive roles as quickly as possible.

Re-training will be essential. The likelihood that the skills needed in an AI-driven world match those of the pre-AI era is effectively zero. The UK, or any nation that enables large-scale, rapid retraining, stands to gain significant advantages. However, this process is costly and difficult, requiring substantial human and material resources at a time when those resources are increasingly valuable.

Education and AI

Education is a solved problem in principle—humanity knows how knowledge transfer works. The issue lies in implementation: mass-learning is both pedagogically suboptimal and logistically inefficient. The core challenges are:

· A teacher cannot dynamically assess every student's exact knowledge state in real-time.

· Students themselves struggle to identify their precise learning needs.

· Traditional education incurs high logistical costs and opportunity costs.

· The sheer order of magnitude of change will be disconcerting, as we are culturally accustomed to large-class learning rather than individualized tuition.

AI-assisted online learning can address these problems. AI tutors approximate individualized instruction, while online learning significantly reduces costs. The key lies in breaking down knowledge into minimal, structured chunks, providing explanations, practice, instant feedback, and spaced repetition—each of which is a well-understood technique.

AI is uniquely suited to serve as a 1:1 tutor, providing not just instant feedback but also dynamically generating new materials, conducting low-stakes testing, and evaluating progress in near real-time. This allows students to learn at their own pace, maximizing retention and efficiency. Students would have access to tutors with a far higher IQ and expertise than humans can provide. The human average on an IQ test is ~100. Current AI models have surpassed this. They will only improve from here.

Despite this, this approach does not replace teachers but redefines their role. Educators would focus on facilitating hands-on practice, coaching, and higher-order skills like critical thinking, evaluating AI-outputs, and communication. For many adult and adolescent learners, direct contact time with teachers may decrease, but learning efficiency would increase.

Image from @highlyretired on X, 4 Feb 2025.

The Present

There are a growing number of examples where AI is already radically improving education. Recall the mantra that the AI we are using today is the worst AI we will ever use. We can also assume that as the AI itself improves so to will we improve our knowledge of how best to employ it over time. Therefore it is almost certain that the gains we describe here will be improved upon, perhaps radically improved upon, as AI gets better and we get better at using it in education.

Take the World Bank AI pilot in Edo State, Nigeria, which focused on English Language, AI (ChatGPT) Knowledge, and Digital Skills. After six-weeks of access to the AI tutor, the results showed students:

• Gained two-years of learning, a 1200% increase

• Improved their ability to learn

• Benefited more as exposure to ChatGPT training increased, with the ceiling unknown

• Benefited more than students in 80% of all the World Banks other (often larger scale, more expensive) educational interventions

This was likely achieved with OpenAI’s GPT 4-turbo model the most modern at the time of the intervention. GPT-4-turbo is now fifteen months and ~ten models behind the most recent OpenAI releases.

Figure 1. Chart shows how increased exposure to ChatGPT increased Nigerian students performance on the test - we do not where the ceiling is for this 'dose-response' – how much better can students be with greater use of ChatGPT?

More examples:

· In Ghana, the ChatBot ‘Rori’ made available via WhatsApp for maths tuition saw students gain the equivalent of a one year’s worth of classroom learning in six-months. This from one sixty-minute session a week, at a cost of just $5 per student.

· At Harvard, an AI chatbot tutor enabled undergraduate students in physics classes to learn twice as much in less time, reporting significantly higher engagement compared to those in taught with ‘active learning’ techniques in a traditional classroom.

· In disadvantaged areas of the Southern United States an AI ‘Tutor Co-Pilot’ significantly improved tutor performance, as measured in student outcomes (4% improvements on average, 9% improvements for students taught by the weaker tutors).

The Future

Classroom education has changed little in the past century. Despite efforts to optimize, fundamental constraints limit the full application of pedagogical science. Overlaying new solutions on top of existing structures demands extra effort from students and exacerbates educational inequality. Instead, the best learning should be available to all.

Humans will never match AI in raw fact retrieval. While factual knowledge remains essential, superior internal models and conceptual understanding matter more—just as the ability to tackle mathematical problems outweighs rote memorization of formulas.

To remain competitive, humans must optimize their strengths. AI-enhanced learning offers a scalable solution, and current online educational content, though plentiful, lacks the necessary individuation, optimization, and real-time feedback to maximize learning efficiency.

A networked AI can learn what students need to know dynamically and empirically, refining its teaching strategies as it gathers data from learners in real-time. It can personalise based on skill, but also optimise to individual psychological make-ups and preferences. This adaptive approach ensures that education remains responsive to individual needs, eliminating traditional delays in curriculum improvement. The tools to implement this revolution exist today—it is time to use them.

National Strategy

There is more to this than just mitigating consequences domestically, nationally, for our children and all of us as workers, business owners, as a society and as citizens – urgent though this is.

We can and should be using the global reach and credibility of organisations like the BBC and the Open University to provide similar tuition globally. Such a service might offer British credentials – perhaps GCSE’s or ‘A’ Levels, marked by AI with far higher consistency than human teachers can manage (see Olex.AI’s impressive results in marking English school work), or degrees from our top Universities, MOOCs but enabled by cutting edge tech that could hugely increase the effectiveness and efficiency of knowledge transfer.

We could use it:

· to help other nations reduce the disruption the AI revolution will bring

· to reduce migratory pressures on ourselves by helping other nations harness the AI revolution

· to identify global talent, either for recruitment as part of a global talent spotting campaign, or for the award of accelerated immigration admission for the top performers

· to identify or qualify for the award of e-citizenship, e-residency or the right to found an e-business covered by UK law, and able to access UK banking and other services.

For those in poor or corrupt countries trying to build global businesses, this could be an invaluable opportunity.

It may be that poorer countries, and the most disadvantaged in society, benefit most from AI tutoring, overcoming barriers like large class sizes, teacher migration, teacher shortages, and teaching by unqualified staff. In some studies of AI, both in educational and professional setting - such as with Tutor Co-Pilot in the Southern US- it has been the lower performers that improve the most when AI assists.

For the UK, allowing AI talent to earn in the UK while living wherever they want through e-citizenship, e-residency or e-businesses, might raise tax revenues – in 2023 Estonia received €67.4 million in tax from e-citizens and e-businesses, a 33% increase on the previous year. What might the UK raise if not only offered such an option, but proactively awarded it to the highest performing students globally?

An objection here might be that allowing talent in poorer countries to found businesses in the UK will take tax revenue from the areas where they live. But the alternative is often for them to migrate. By staying where they are they spend locally, benefitting the economy. And once their businesses prosper, shielded by the UK’s legal system and with access to global markets from the UK, expansion into their country of origin seems likely - from a much more secure base.

A final caveat: we must remain wary of this early evidence for AI’s benefits in improving learning. From 2008, Professor Sugata Mitra gave inspiring talks on the effectiveness of his ‘Hole in the Wall’ experiments – where students in impoverished areas were given access to a computer connected to the internet and self-learned complex subjects with no tuition. In 2013, Mitra won the Ted Prize for his ‘Kids Can Teach Themselves’ talk. In 2015 the Times Education Supplement ran a headline claiming ‘Internet learning boosts performance by seven years’. These claims are now disputed, albeit not debunked. What we should note is the need for experimentation, testing of claims and careful implementation.

Care and caution must not mean prevarication and delays. It’s Happening. The AI development and AI impact will likely now scale logarithmically rather than linearly. Like the pandemic, we will feel its effects gradually, and then suddenly. We must respond with the urgency needed, and pre-emptively, not retrospectively. Education, Education, Education, as one Labour leader once put it.

There is a profound sadness about watching all this unfold. At Christmas, a good friend and deep ML expert at one of the world’s leading labs wrote in his email circular ‘It’s happening’. And I felt the same thrill and fear flare in me that comes multiple times daily now. He’s right. And we can’t stop it.

I re-read Sarah’s In Defence of Slowness at the End of the World again and again. I find myself back in church on Sunday. I wonder if I should read less for work and go back to theology and philosophy – passions in my teenage years and twenties. In coping with ‘the Coming Wave’ they feel now like they might be more important than anything I have learned since. The thrill is there. More on that another time. But moreover: I am worried.

I have written and spoken, with my usual mix of passion, evangelism and frustration, about where AI is going and how important it is going to be, for some time. We have named our new company ‘Cassi’ after Cassandra the prophetess condemned always to be right and never believed, in part in reference to a nickname given me in previous job. I am very aware that the warnings so many, I included, have advanced on the profundity of this revolution have not had the effect intended. We have failed.

So much has been written and said describing how disruptive a world of superintelligent AIs will be economically, socially, for national security – for humanity - that there is little need to requote and reference. Most of us commenting have striven to remain grounded, to avoid sounding ‘shrill’, to be sufficiently sober to retain a place with ‘the adults in the room’. But even now, as it is happening it remains a fringe issue. Acknowledged by Governments in words but not really in deeds. Treated with amusement and disdain by many in our governing and media classes. As if it is something we can worry about later. Part in consequence, it remains largely outside of electoral consciousness, with few doing the work to wake people up to what is coming.

Most thought ‘Don’t look up’ was about climate change. But not for me.

We are not ready. It is too late to be ready. It is too late to stop it.

I won’t pretend to know precisely how all this will play out. As ever, we do our best to grasp what Peter Hennessey called ‘the thin whisps of tomorrow’. That’s been central to my job for 22 years now. Forecasting breeds humility.

As Sarah writes

‘Those among us that are Situationally Aware must be on guard against arrogance. To anticipate some transformation that most live in ignorance of can easily breed self-importance. …worse, it can degrade the way that we perceive the efforts of others. …I don’t want to presume they’d act any differently if they knew (what I think) I know.’

Somewhere else, she wrote acknowledging we could be wrong – maybe some of us, and I one of them, have drunk the kool aid. Maybe, despite my protestations in ‘In Athena’s Arms’ (on AGI) we have succumbed to a cargo cult. In many ways, I hope so.

But until that becomes apparent, what I think I know is:

1. It is happening

2. We can’t stop it

3. We are not ready.

My view is that so much is unknown that our estimate of the probability of extinction risk from AI cannot be zero. But I do think it is highly unlikely (<5%). More likely is radical economic disruption, and social unrest. My biggest worry, because it feels more real and imminent, is that the things AI will take from us are the things I enjoy the most in my work – the search for new and novel insights, the act of creation, of communication, of anticipation and discovery - and yes, of struggle, and effort, the reward that comes from success at the end of it, or the consolations and lessons that exist in failure. I worry that even if we create a new economy, where intelligence is cheap and the AIs play the dominant role but we still work, the things I find most meaningful will be the things of least value.

I will keep writing, and play my part in what is unfolding as best I can. Because you can’t stop this, I would rather be on the upside of the bet than the downside. I think you can probably do more good to mitigate the disruption by being a part of the revolution than by sitting on the sidelines and commenting. I will try to enjoy the thrill of progress and do my bit, and my best to help others, to ameliorate the risks and harness the opportunities of what is coming. There are, of course potential radical upsides too, much greater prosperity, better health - some would say ‘curing death’, fully-automated luxury communism, Bostrom’s ‘Deep Utopia’. But I don’t think we get there without significant social and political unrest. ‘Significant’ being something of an understatement.

I worry that my writing breathlessly of progress and development is too easily misconstrued as delighting in the disruption. I write this because I want to be understood. There is a thrill to being a small part of - or even just being around to witness - the most far-reaching revolution in human history. But there is also a profound sadness about watching all this unfold. It’s happening.