Artificial intelligence is transforming how organisations generate analysis, model scenarios and build internal consensus. For those of us working in research and insight, much of this is rightly exciting.

Yet one question has stayed with me over the past year: as organisations become better at generating internal answers, are they also becoming better at knowing whether those answers reflect external reality?

I am not convinced they are.

I have spent the last decade working with senior leaders in financial services, energy and other regulated environments. In that time, I have seen a pattern repeat itself often enough to feel structural rather than incidental. Organisations reach a point of high internal alignment while remaining surprisingly distant from how the people that matter are actually responding.

Customers interpret a proposition differently from how leadership framed it. Regulators prioritise risks that internal teams had discounted. Operational staff encounter barriers that strategy teams never modelled. Markets react in ways that nobody in the room anticipated.

These situations rarely happened because organisations lacked intelligent people, good intentions or sufficient information. More often, they happened because internal agreement had been mistaken for external readiness, and coherence had been confused with accuracy.

This problem predates AI by some distance. What feels different now is the speed at which coherence can be manufactured, and the authority that data-rich internal outputs tend to carry in the room.

The Klarna case is instructive. In early 2024, the Swedish fintech replaced much of its customer service operation with an AI assistant, publicly claiming it could handle the work of 700 agents. Internally, the metrics looked compelling: faster response times, lower costs, measurable efficiency gains. By May 2025, chief executive Sebastian Siemiatkowski told Bloomberg: "We focused too much on efficiency and cost. The result was lower quality, and that’s not sustainable." Customer satisfaction had fallen and the company began rehiring human agents.

The internal story had been coherent. It had not been accurate. And critically, the gap only became visible after the external reality had already made itself felt.

This is the specific risk I think the research and insight community should be paying attention to. Not AI failure in a technical sense, but AI-accelerated confidence in conclusions that have not been adequately tested against the people those conclusions affect.

Research published last October by the British Standards Institution, drawing on analysis of over 100 multinational annual reports and surveys of more than 850 senior leaders, found that many organisations are, in the BSI’s own words, "sleepwalking toward significant governance failures" Fewer than a quarter had a formal AI governance programme in place. The report flagged overconfidence specifically as the mechanism most likely to produce avoidable harm. (BSI, October 2025 )

Large language models can generate analysis, surface patterns, compare options, and help teams build persuasive internal narratives at unprecedented pace. What they cannot do is independently determine whether customers will trust a decision, whether regulators will view it as proportionate, whether employees will adopt it, or whether markets will interpret it as intended. Those questions require direct contact with the people they concern.

This is, of course, precisely what the research and insight function exists to provide. And yet I wonder whether the current conversation in our industry is focused enough on this specific risk. Much of the debate around AI centres on productivity, on what insight teams can now do faster or cheaper. Less attention is being paid to whether the organisations we serve are using that speed in ways that make external reality harder, not easier, to access.

The organisations that have historically navigated complexity most effectively were not always those with the most information. They were often those that remained closest to the people their decisions affected, even when that proximity was inconvenient and the feedback was unwelcome.

As AI becomes embedded within strategy and insight functions, one of our most important responsibilities may be making that case clearly, and loudly, to the leaders we work with. Rather than being about generating answers faster, perhaps the real competitive advantage is staying closest to reality while everyone around you is becoming increasingly confident in their assumptions.

James Tattersfield is chief executive of Polar Insight