Appetite For Destruction

There are some album titles that stay with you, even when you haven't listened to the album for years. Appetite for Destruction is certainly one of them. Released by Guns N' Roses in 1987, it became one of those albums that seemed to define an era, with Welcome to the Jungle and Sweet Child o' Mine becoming instantly recognisable songs in their own right.

I suspect most people of my generation can remember the first time they heard that opening guitar line from Slash on Sweet Child o' Mine. I certainly can. I was 17 and only just starting to delve into the world of heavy rock and metal. That album was formative for me. It opened the door to a whole range of bands I might otherwise have missed, including Bon Jovi, Giant and Van Halen, and sent me off exploring a much wider musical landscape.

My musical tastes have expanded considerably since then. These days I'll listen to all sorts of things, and my musical journey has taken me a long way beyond the heavy rock and metal that first caught my attention as a teenager. Even so, I've still got a soft spot for Appetite for Destruction. Perhaps that's partly because of the music, but it's also because albums become attached to particular periods of our lives. They remind us of where we were, what we were discovering and, sometimes, who we were becoming.

At the time, of course, the title Appetite for Destruction had nothing to do with artificial intelligence, technology or governance. If you'd told that 17 year old that nearly forty years later we'd be having conversations about machines that can write essays, produce computer code, analyse information, create images and hold something resembling a conversation with us, I'd have assumed you were inebriated. Yet here we are. And the more I think about that album title in the context of artificial intelligence, the more appropriate it begins to seem, because we're developing an enormous appetite for what technology can do while we're still working out whether our safeguards, our governance and even our wisdom can keep up with it.

To some extent it’s not surprising. There's something very human about the desire for advancement. Humankind has always been fascinated by what it can achieve, and throughout history our ability to solve one problem has usually led us to creating another. We discover how to create and control fire and suddenly we can cook food, keep warm and work after dark, but we also discover that fire burns and destroys things. We develop industrial machinery and discover that we can produce goods on a scale previous generations could barely imagine, but we also discover that machinery can maim and kill. We learn how to harness electricity and transform almost every aspect of daily life, while also creating an entirely new set of risks. The answer hasn't been to abandon fire, machinery or electricity. We've had to develop ways of controlling them because most of us have learned that capability brings responsibility with it.

Artificial Intelligence is beginning to present us with a similar problem, although in a way that we haven't seen before. The AI systems that have been built can process amazing quantities of information, generate plausible answers (not necessarily correct) and perform tasks in seconds that would have taken hours, and in some cases would have been impossible to complete by humans. So it's easy to see why so many people are excited by this. If a machine can take something tedious off my desk and give me more time to do something useful, why wouldn't I be interested? If it can help somebody analyse information, identify patterns or explore possibilities that they might otherwise have missed, there could be enormous benefits. The difficulty comes when the excitement about what the technology can do starts to crowd out the questions we ought to be asking about how we're going to use it.

One of the oldest stories in the Bible gives a useful way of thinking about this. In the book of Genesis, the people of Babel decide to build a city and a tower reaching towards heaven. They’ve discovered that by working together they can accomplish something remarkable, and so they decide to push that capability as far as they can. Genesis describes them as having one language and a common purpose, and they appear to recognise that this gives them a particular strength. They can communicate easily, share their knowledge and organise themselves around a common goal. So they develop the means to construct the city and begin building a tower whose top will reach towards heaven. They’re not content just to survive or to build somewhere to live. No, they want to make a name for themselves, create something that demonstrates what they are capable of achieving, and reach beyond their current limits.

But God sees that there is effectively no limit to what they might attempt, and that’s dangerous for them. So their language is confused, communication consequently breaks down and the people are scattered, bringing the construction of the city and tower to an end. There are, of course, different ways of interpreting what the story is trying to tell us. I think it would be too simplistic to reduce it to a warning that human beings should not be ambitious or inventive. The people of Babel are not condemned simply because they know how to make bricks or because they have learned how to organise themselves. What seems to be the issue is the combination of their capability, their ambition and the assumption that their growing ability to accomplish things somehow gives them the authority to pursue them without limit. The tower represents an attempt to reach beyond the boundaries that previously existed, driven by the confidence that, if they have the means to do it, they can simply keep going.

What becomes clear is the recurring human tendency to assume that because we have discovered how to do something, we have also acquired the wisdom to decide that it should be done. Those are two very different things. Knowing how to build something doesn't tell us whether we should build it, where it should be used, who might be affected by it or what happens when it doesn't behave in the way we expected. The people of Babel had the ability to build their tower, but the story asks us to look beyond the physical achievement and think about the ambition behind it. There is a difference between being able to reach higher and understanding whether reaching higher is actually a good idea, and that difference between capability and wisdom is perhaps one of the most enduring questions contained within the story.

And, of course, we've seen this throughout human history. Civilisations don't generally decide one morning that they're going to destroy themselves. People make decisions that appear reasonable in the circumstances in which they're living, and those decisions interact with other decisions made by other people over years, decades or even centuries. Resources become depleted, political systems become strained, populations move, economies change, environmental conditions deteriorate and institutions become less capable of responding to new problems. The eventual collapse can look obvious when we examine it afterwards, but the people living through it weren't necessarily walking around thinking, "This is the bit where our civilisation starts to fall apart."

The Maya are an interesting example because their civilisation achieved extraordinary things in architecture, mathematics, astronomy, agriculture and administration, yet parts of the Maya world experienced major disruption and decline. The Roman Empire offers another example of a society that developed extraordinary systems of government, engineering, transport, trade and military organisation, but eventually became increasingly difficult to maintain. The thing is that there isn't a single explanation for the decline of either civilisation, and anyone who claims otherwise is simplifying a complicated historical story. Despite this both remind us that sophistication doesn't make a society immune to failure. Sometimes the very systems that allow a society to become successful can also create dependencies and vulnerabilities that aren't obvious while everything is working.

I think that’s worth remembering as we build increasingly complex systems around AI. I’ve written about this before, but we could become so good at automating particular tasks that we gradually lose the knowledge of how those tasks were done before automation. There is a danger that we become so accustomed to accepting machine generated information that questioning it starts to feel unnecessary. We might build organisations around systems that are understood by relatively few people, then discover that those people have moved on when something goes wrong. And if we create a dependency upon technology because it saves time, we may discover that we haven't retained enough human capability to cope when the technology isn't available.

None of this requires an evil machine. It requires ordinary human behaviour.

I've spent the majority of my adult life dealing with data, statistics and information.  In this time I’ve learned that it’s not the producing of information that’s the difficult part. It’s deciding whether the information is meaningful, whether it answers the question we wanted to ask, whether the data behind it is appropriate and whether somebody understands the consequences of acting upon it. A spreadsheet can contain perfectly calculated numbers and still lead us to the wrong conclusion. A dashboard can present accurate information and still answer the wrong question. A statistical model can be technically sound and still be inappropriate for the decision we're trying to make.

We need to realise that AI doesn't remove any of those problems. In some circumstances it makes them more difficult because it can produce an answer that sounds as though it knows what it's talking about. That's a characteristic of AI that easily deceives us. We associate fluent language with understanding, and when a system gives us a well written explanation in seconds, there's a natural temptation to assume that the quality of the writing tells us something about the quality of the underlying answer.

That creates a particular problem for safeguards. We can put a human in the loop, but the important question is what that human is actually doing. If they're checking the output against reliable evidence, questioning assumptions, understanding the limitations of the system and having the authority to reject the answer, then there's meaningful oversight. If they're simply expected to look at the screen and confirm that everything appears reasonable, we've created something that looks like oversight without necessarily providing much protection at all.

I've seen versions of this problem in organisations repeatedly. A policy exists, so everyone assumes the issue has been dealt with. A procedure has been published, so everyone assumes people are following it. A committee has been established, so somebody assumes the governance is in place. But when an unusual situation arises, it becomes remarkably difficult to establish who actually has the authority to make the decision and who will be accountable for the consequences. AI will expose that weakness very quickly because the technology can operate across organisational boundaries and produce information that influences decisions without necessarily fitting neatly into the existing structures of responsibility.

That question of accountability is, I think, more important to address than the technology itself. If an AI system contributes to a decision that affects a real person, somebody has to own that decision. The software can't do it. The algorithm can't explain itself to a board, a regulator, a patient, a customer or an employee. Eventually a human being has to be able to say what happened, why the system was used, what checks were carried out and why the final decision was made in the way it was.

That’s particularly important as we move away from harmless assistance and into areas such as recruitment, healthcare, education, financial services, benefits or public services. If an AI system helps me draft an email and gets something wrong, I can correct it. If an AI system contributes to a decision that determines whether someone gets a job, receives support, obtains treatment or is investigated, the consequences are rather different. The more significant the decision, the greater the need for people who understand both the technology and the subject in which it is being used.

There's another biblical story that comes to mind. In Genesis, Adam and Eve are created and placed in the garden of Eden. They can eat from any tree in the garden, with only one exception. They’re told that they shouldn’t eat from the tree of the knowledge of good and evil, with a clear warning about the consequences if they do.

Then comes the temptation. Eve is challenged by a serpent about what she’s been told and it’s suggested that eating the fruit will open their eyes, give them knowledge they do not currently possess and make them more like God. The attraction is obvious. Why would you accept a limitation on what you can know when there is a possibility of knowing more? Why remain within the boundaries that have been set for you when there’s an opportunity to understand, and be, something that has previously been beyond your reach? So Eve eats the fruit and gives some to Adam, and they have their eyes opened. But the knowledge they have gained is not quite what they expected. They’ve acquired something they didn’t previously have, but with that knowledge comes shame, fear, separation and a recognition of consequences that they’d not understood when they made their choice.

What’s important about the story is that the fruit itself isn't the problem. The problem is that Adam and Eve chose to ignore the boundary they had been given in order to acquire knowledge.  But they weren't ready for what came with the knowledge they received. They wanted the knowledge before they had the wisdom to understand what possessing it would mean. The consequence was far greater than simply discovering something new. Their relationship with each other changed, their relationship with God changed and they were eventually removed from the garden. A decision that may initially have looked like a relatively small act of curiosity and disobedience had consequences that extended far beyond the moment in which it was made.

Whether you believe this literally, or just take it as a storytelling metaphor, there’s something remarkably persistent in the idea it presents. Human beings have always wanted to know more, to see further and to gain greater control over the world around them. And knowledge can, of course, be wonderful, liberating and transformative. The question is whether having the opportunity to acquire more knowledge automatically means that we are ready to deal with what comes with it. Adam and Eve didn't simply discover something new. They discovered something before they were ready for it, having been warned that there would be consequences, and chose to proceed anyway. The story therefore raises a question that is much more difficult than whether knowledge is good or bad. What happens when our ability to acquire knowledge moves faster than our wisdom to understand how, when and why we should use it?

This inherent desire for more means that we've created systems that can give us access to information on an unprecedented scale. We can ask questions and receive answers almost instantly, sometimes questions that couldn’t be answered previously. But more information doesn't automatically make us wiser. In fact, when information becomes abundant, the difficult question becomes deciding what to trust and what to ignore. AI adds another layer because it can generate information as well as process it. We're entering a world where the production of plausible information can happen much faster than our ability to establish whether that information deserves to be believed.

And I think that's why I'm uneasy about the language being used around AI development, that we’re in “an AI race.” We talk about countries competing with one another, organisations competing with competitors and individuals being told that they need to learn AI skills or risk being left behind. There's an unstated assumption in that language that speed itself is a virtue. But if we're running towards something without knowing exactly where we're going, being the fastest person to get there doesn't necessarily make much difference.

One of the lessons from the story of the Tower of Babel is that capability doesn't automatically come with wisdom, and that collective enthusiasm can sometimes make it harder for anyone to stop and ask whether we're heading in the right direction.

To me, that’s remarkably close to where we are with AI. Every new demonstration produces another reason to believe that the next development will be even more impressive. Each new capability creates another potential use. Repeated successful experiments makes wider adoption easier to justify. The momentum builds, and eventually questioning the direction can start to look like opposition to progress itself.

We need to be careful not to fall into this way of thinking. Asking questions isn't opposition to progress. It's how we decide whether progress is actually taking us somewhere worth going.

In my view, we need to become more interested in the foundations than the height of the tower. Before we celebrate another impressive AI capability, we should be asking what data it depends upon, how it has been tested, what happens when it produces an incorrect answer, who checks the output, what happens when the system changes and who has the authority to stop it being used. We should be asking what skills the people responsible for using it need, because putting someone in front of a sophisticated system doesn't magically give them the expertise required to understand it.

We should also be asking what happens to our own abilities when we hand more and more of our thinking to machines. If an organisation becomes dependent upon AI for analysis, writing, coding, decision support and research, does it still have enough people who understand those things without the technology? If the answer is no, we've created a dangerous vulnerability. We've made ourselves more efficient while potentially making ourselves less resilient.

Civilisations have fallen for many reasons, and we shouldn't turn complicated histories into convenient morality tales. But there is a recurring lesson in looking at societies that became increasingly complex and increasingly dependent upon systems they could no longer easily control. Complexity can create strength, but it can also create fragility. The more interconnected the system becomes, the more difficult it can be to understand what happens when one part fails.

We're creating increasingly interconnected systems around AI while still working out how the technology behaves. That doesn't mean we should abandon it. It means we need to be much more serious about the safeguards surrounding it, particularly where the consequences of failure extend beyond an incorrect paragraph or a disappointing piece of computer code.

What I find interesting about the title Appetite for Destruction is that humans don't usually begin with a desire to destroy.  We begin with an appetite for something else, something better (in our own minds at least). We want efficiency, convenience, competitive advantage, better services, lower costs and more productivity. Each individual decision can look perfectly reasonable, and yet the combined effect can easily take us somewhere we hadn't intended to go.

We're now building something that could become one of the most influential technologies in human history, and we're doing it at breakneck speed. The question is whether we're wise enough to recognise when the thing we're building has become more powerful than our ability to control it.

Because if our appetite for AI continues to grow faster than our safeguards, our governance and our willingness to question what we're doing, we may eventually discover that the greatest danger wasn't that the machines wanted to destroy us. It was that we were so hungry to see what they could do that we forgot to ask where they might take us.

And that would certainly be an appetite for destruction.