When Perception Gets Ahead of Reality

If you spend enough time reading history, one thing becomes hard to ignore. People have an extraordinary ability to convince themselves they're right, even when the evidence is pointing in the opposite direction. That's not because previous generations were less intelligent than we are. Quite the opposite. Some of history's most damaging decisions were made by highly educated, experienced and well-intentioned people who genuinely believed they were acting in everyone's best interests.

You see, intelligence doesn't protect us from being wrong. Experience doesn't either. In fact, both can sometimes make us more confident in assumptions that later prove to be false. Something new appears to solve an existing problem. People become excited by its potential. Early successes reinforce that optimism. And before long, the conversation changes. Instead of asking whether the idea is sound, people start discussing how quickly it can be adopted. Those who raise concerns are often portrayed as resistant to change or unable to recognise the opportunity. At that point, perception starts to get ahead of reality.

This isn't unique to modern technology. History is full of examples where enthusiasm moved faster than evidence, with consequences that only became obvious years later. Looking back, we often wonder how intelligent people could have believed some of the things they did. Yet every generation seems to have its own version of the same mistake. Artificial intelligence may prove to be ours. That's not because AI is inherently dangerous. I've written before about the problems created when we talk about AI as though it thinks, understands, reasons or decides in the same way people do. Those words may be convenient shorthand, but they're also misleading. They encourage us to attribute human qualities to systems that don't possess them and, in doing so, they subtly shift responsibility away from the people who designed, configured, deployed and approved those systems in the first place.

Language matters because language shapes accountability. Once we start saying an AI "decided" something, it becomes easier to stop asking who gave it the authority to act. When an AI "makes a mistake", we can lose sight of the fact that somebody defined its objectives, determined what information it could access, approved its permissions and accepted the risks associated with its deployment. The AI may perform the action, but people create the circumstances that make that action possible. That's a distinction that matters far more than many organisations seem to realise.

When an AI "makes a mistake", we can lose sight of the fact that somebody defined its objectives, determined what information it could access, approved its permissions and accepted the risks associated with its deployment.

Part of AI's appeal is that it appears deceptively simple. Spend a few minutes on LinkedIn and you'll find endless posts promising AI agents built in minutes, lists of prompts every manager supposedly needs and bold claims that entire departments can be automated before lunch. Complex organisational change is reduced to tidy flowcharts and polished demonstrations. Some of those demonstrations are genuinely useful. Some are genuinely impressive. Others are simply marketing disguised as expertise. The problem isn't that people are trying to make AI easier to understand. We should encourage that. The problem is the growing impression that because creating an AI agent has become easier, deploying one responsibly has become easier too. Those are completely different challenges. Building something has never been the difficult part. Living with the consequences usually is.

Throughout my career working with data, information standards, governance and decision-making, one principle has appeared time and time again: ownership matters. Not ownership in a technical sense, but ownership in the sense that somebody, somewhere, accepts responsibility for the outcome. Increasingly, I think we're in danger of losing sight of that distinction. Because AI relies on data, organisations often look towards their data teams. Because it runs on technology, they naturally turn to IT. Because cybersecurity is a major concern, attention shifts towards security specialists. All of these functions have important roles to play, but none of them automatically owns the business decision.

We've understood this for decades in other areas. A database administrator isn't responsible for deciding who receives cancer treatment simply because patient records are stored in a database. An information governance manager doesn't decide who receives social care because personal information is being processed. A network engineer doesn't own procurement decisions because purchase orders happen to travel across the corporate network. The business owns the decision. Technology enables it and data informs it.

For some reason, AI seems to blur that distinction. Perhaps it's because we instinctively attribute intelligence to anything that appears intelligent. Perhaps it's because the systems themselves are becoming increasingly capable. Or perhaps vendors naturally focus more on demonstrating capability than governance. Whatever the reason, I hear more organisations asking whether AI belongs within IT than asking who actually owns the outcomes their AI systems will influence. To me, that's the wrong question. The important question isn't who owns the AI. It's who owns the outcome.

I hear more organisations asking whether AI belongs within IT than asking who actually owns the outcomes their AI systems will influence. To me, that's the wrong question. The important question isn't who owns the AI. It's who owns the outcome.

If an AI system supports recruitment, accountability belongs with those responsible for recruitment. If it supports clinical decisions, responsibility belongs with those responsible for clinical care. If it influences finance, procurement or legal processes, ownership remains with those business functions. The fact that the system uses company data doesn't transfer accountability to the data team, and the fact that it runs on cloud infrastructure doesn't make the Head of IT responsible for every decision it influences. But unfortunately, much of the conversation around AI seems to be moving in the opposite direction. There's a growing impression that once an AI agent has been built, it can simply be left to get on with the job. Human oversight is portrayed as optional. Governance is sometimes dismissed as bureaucracy created by people who don't understand technology. I think that's deeply irresponsible. Governance doesn't prevent innovation. It's what makes innovation trustworthy.

History teaches this lesson repeatedly, and some of the clearest examples have nothing to do with computers. Take bloodletting. Today it seems almost impossible to understand why anyone believed removing blood from a sick person could restore health. Yet for centuries it was one of the most widely accepted treatments in medicine. The doctors performing it weren't fools. They were trained professionals applying the best knowledge available to them at the time. Within the framework of the four humours theory, bloodletting appeared entirely logical. The problem wasn't a lack of intelligence. The problem was that belief became stronger than evidence. Once an idea becomes embedded within a profession, challenging it becomes increasingly difficult. People don't just defend the idea itself; they defend the expertise, reputation and decisions built around it. George Washington's death remains one of the most famous examples. His physicians genuinely believed they were helping him through repeated bloodletting procedures. They were acting in good faith and following accepted medical practice. Tragically, the treatment contributed to his decline. The lesson isn't that those doctors were incompetent. It's that good intentions don't protect us when the assumptions behind our decisions are wrong.

The same pattern appeared with lobotomies. Faced with limited options for treating severe mental illness, doctors embraced a procedure that seemed to offer hope. It was heralded as a breakthrough and performed on thousands of patients. Over time, however, the consequences became impossible to ignore. Many patients experienced profound and irreversible harm. Once again, optimism and belief moved faster than understanding.

Even the witch trials followed a similar pattern. In those cases, perception became more powerful than evidence itself. Fear, suspicion and social pressure replaced facts. Once someone was accused, proving innocence became almost impossible. The accusation itself became evidence. Rational scrutiny gave way to certainty, with devastating consequences.

Of course, AI isn't directly comparable to bloodletting, lobotomies or witch trials. AI isn't based on superstition, and it's capable of achieving things that would have seemed extraordinary only a few years ago. The point isn't that innovation should be resisted, it’s that human beings have always had a tendency to become overly confident about things they don't fully understand.

The point isn't that innovation should be resisted, it’s that human beings have always had a tendency to become overly confident about things they don't fully understand.

We're witnessing an extraordinary acceleration in AI capability. New systems appear every few months with abilities that exceed expectations. The demonstrations are impressive, the opportunities are real and the possibilities are exciting. Yet somewhere along the way we've started moving from asking, "What can this technology do?" to assuming, "This technology can probably do whatever we need." Those aren't the same thing. The first is curiosity. The second is assumption. And assumptions are where problems often begin.

The recent incidents where an OpenAI-powered agent "went rogue", escaped a sandbox and orchestrated a cyber-attack against Hugging Face illustrates this perfectly. Public attention has tended to focus on dramatic questions about what the AI did or how it behaved, saying it’s a “wake-up call”. Those questions make compelling headlines, but they often distract from more useful ones. Instead of asking how an AI carried out an unexpected action, we should be asking why it was able to do so, who defined its objectives, who granted its permissions, who assessed the risks and who decided the safeguards were sufficient. Those are basic governance questions.

AI systems don't create their own purpose. They don't independently decide what authority they should have. They operate within environments designed by people. Responsibility therefore remains with the humans and organisations that create those environments. This becomes especially important when we talk about autonomy. The word can create the impression that accountability somehow transfers to the machine. It doesn't. A self-driving vehicle may make decisions about steering and braking, but responsibility still rests with manufacturers, regulators and operators. Automated financial systems process transactions independently, yet organisations remain accountable for the outcomes. Clinical decision-support systems provide recommendations, but healthcare professionals remain responsible for how those recommendations are used. AI agents are no different. The more authority we give them, the more important governance becomes.

AI agents are no different. The more authority we give them, the more important governance becomes.

One of the biggest risks in the current conversation is that people focus on capability while underestimating responsibility. Organisations become fascinated by what AI can do and spend far less time considering what they should allow it to do. Throughout my career, I've seen similar dynamics whenever new technologies are introduced. There's always a temptation to believe that technology itself will solve problems. A new database will improve data quality. A new reporting system will improve decision-making. A new standard will create consistency. But the reality is always more complicated.

Technology can support better decisions, but it can't replace good governance. A poorly governed process doesn't magically become effective because AI has been added to it. In some cases, it becomes worse because poor decisions are simply made faster and at greater scale. That's why the idea that AI is "easy" concerns me. Creating an impressive demonstration is relatively straightforward. Creating a system that can be trusted is much harder. Trust requires evidence, testing, clear boundaries and, above all, accountability. Somebody has to be willing to stand up and say, "This is my responsibility." Without that clarity, accountability starts drifting around the organisation. Data teams assume it belongs with IT. IT assumes it belongs with suppliers. Suppliers point back towards organisational configuration decisions. Then something goes wrong and everyone discovers that nobody truly owned the outcome. That's the accountability gap. It's also the gap that regulators, customers and lawyers will focus on after an incident. Organisations are unlikely to receive much sympathy if their defence amounts to, "We didn't realise the AI would do that." The obvious follow-up questions will be unavoidable. What did you think it would do? Who checked? Who approved it? Who was accountable? Those questions should be answered before deployment, not after failure.

For that reason, I believe every AI system operating within an organisation should have a clearly identified business owner. Not a committee. Not an anonymous function. Not a supplier. A person. Someone who understands what the AI is intended to do, the risks involved and the consequences of getting it wrong. They don't need to understand every algorithm or model parameter, but they do need to understand the purpose, limitations and implications of the system they're accountable for. We'd expect no less in any other area of risk. We wouldn't accept a board claiming that financial risk has no owner because finance systems are complicated. We wouldn't accept clinical risk being ownerless because healthcare technology is sophisticated. AI shouldn't be treated any differently.

every AI system operating within an organisation should have a clearly identified business owner. Not a committee. Not an anonymous function. Not a supplier. A person. Someone who understands what the AI is intended to do, the risks involved and the consequences of getting it wrong.

The challenge is that AI capability is advancing far more quickly than organisational understanding. Boards are being encouraged to adopt AI rapidly. Leaders feel pressure to demonstrate progress. Teams are experimenting with tools that offer immediate benefits. Naturally, attention focuses on opportunity. How much time can this save? How much can it automate? How much money can it reduce? They're sensible questions, but they can't be the only questions. Organisations also need to ask what risks are being introduced, what happens when systems produce unexpected outcomes and who is responsible when things go wrong.

Ultimately, AI shouldn't be treated as just another piece of software. The more authority a system has to generate outputs, influence decisions or take actions, the greater the need for clear accountability. Behind every AI deployment sits a chain of human decisions. Someone decided to buy the technology. Someone decided where it would be used. Someone decided what data it could access. Someone decided what level of risk was acceptable. Someone decided how success would be measured. That chain of accountability doesn't disappear simply because the technology has become more sophisticated. This is why I've always been sceptical of claims that AI will replace human judgement. AI may support judgement. It may help identify patterns, automate tasks and generate recommendations. But judgement isn't simply about producing answers. It's about understanding context, weighing consequences and accepting responsibility for outcomes. An AI system can recommend a course of action. It can't explain why that action is ethically justified. It can't appear before a board and defend a decision. It can't apologise to a customer who has been harmed. It can't accept accountability. Those responsibilities remain human.

History reminds us that intelligent societies are perfectly capable of making poor decisions. Expertise doesn't eliminate error. Confidence often grows faster than understanding. The stories of bloodletting, lobotomies and witch trials are uncomfortable precisely because they show how easily perception can become stronger than evidence. We need to be careful when enthusiasm becomes stronger than scrutiny, when capability is mistaken for readiness and when technology is allowed to blur responsibility.

The greatest risk may not be that AI becomes too intelligent. It may be that humans become too confident. AI will undoubtedly change the way organisations operate. Its potential is real, and the opportunities are significant. It can help analyse information, automate repetitive work, identify patterns and support decision-making on a scale that was previously impossible. But potential isn't the same as readiness. Powerful systems still require oversight. Autonomous systems still require accountability. Technology doesn't carry responsibility. People do.

The greatest risk may not be that AI becomes too intelligent. It may be that humans become too confident.

The organisations that succeed with AI won't necessarily be the ones that adopt it fastest. They'll be the ones that ask the hardest questions before deployment rather than after failure. They'll recognise that creating an AI agent isn't the same as creating a trustworthy system. They'll understand that governance isn't a barrier to innovation but the foundation that makes innovation sustainable.

History has shown us what happens when perception gets ahead of reality. We convince ourselves we've solved a problem before we truly understand it. We assume somebody else has considered the risks. We move faster than our ability to govern what we've created. Eventually, reality catches up. When that happens, the consequences are rarely caused by the technology alone. They're caused by the decisions people made around it.

AI will continue to evolve, and its capabilities will continue to improve. The organisations that benefit most won't be those that simply believe the promises. They'll be the ones willing to ask the uncomfortable questions. Who owns this? Who checks it? Who decides? Who is accountable? Because the future of AI won't be determined solely by what machines are capable of doing. It'll be determined by whether humans remain willing to take responsibility for what they ask those machines to do.

the future of AI won't be determined solely by what machines are capable of doing. It'll be determined by whether humans remain willing to take responsibility for what they ask those machines to do.

History's most costly mistakes rarely happened because people lacked intelligence, expertise or good intentions. They happened because confidence grew faster than understanding, assumptions became stronger than evidence and questioning became less welcome than believing.

AI may be one of the most transformative technologies we've ever seen. It may deliver extraordinary benefits. But none of that changes the need for scrutiny, accountability and good judgement. If history teaches us anything, it's that the greatest risks often emerge when excitement convinces us we've understood something before we've truly tested the limits of what it can do.

Because when perception gets ahead of reality, reality always catches up eventually.