Figure 1 — From AI skills to institutional judgement. AI literacy extends beyond the ability to use AI effectively. As AI moves closer to consequential decisions, understanding context, challenging outputs and governing their consequences become increasingly important organisational capabilities.

Interestingly, there is something of this idea in the European approach to AI literacy. Article 4 of the AI Act does not define literacy simply as the completion of a standard course. It asks organisations to take account of factors such as technical knowledge, experience, education and training, as well as the context in which AI systems are used and the people affected by them.

I find that word – context – much more interesting than it may initially appear.

Because context changes everything.

A software engineer, a doctor, a teacher, a public official and a senior manager do not need the same understanding of artificial intelligence. Nor should they. The question is not whether everyone can explain transformer architecture. The question is what each person needs to understand in order to exercise responsibility in the presence of AI.

And responsibility begins with something very simple: knowing what the machine cannot know.

When we receive a convincing AI-generated answer, we naturally focus on what is there. But professional judgement has always depended just as much on recognising what is absent.

The unusual case. The missing document. The political context. The organisational history. The tacit knowledge that was never entered into a database. The exception that twenty years of experience teaches someone to recognise immediately. The apparently minor detail that completely changes the interpretation of a situation.

These are not merely data-quality problems.

They are part of what it means to know something inside an organisation.

This is why I am increasingly interested in the relationship between artificial intelligence and organisational knowledge. Organisations know much more than what they store. Their knowledge exists in systems and databases, certainly, but also in procedures, relationships, memories, professional communities, habits and people.

AI gives us extraordinary new ways of accessing and recombining the first category.

I am less certain that we fully understand what happens to the second.

There is a temptation, particularly when AI performs well, to assume that human expertise becomes progressively less important. I suspect that in many situations the opposite will be true. The better the machine becomes at producing plausible answers, the more valuable the human capacity to recognise the exceptional answer that should not be accepted.

And this leads to a paradox.

Poor AI is relatively easy to supervise.

Good AI is much harder.

A system that is wrong half the time keeps us attentive. A system that is right ninety-nine times out of a hundred encourages us to trust the hundredth answer.

At first, the professional uses AI as an assistant. The expert reads everything, checks the evidence and makes the decision.

Then the system improves.

The professional discovers that most recommendations are good. Checking becomes faster. The organisation notices the productivity gain and begins reorganising the process around it.

Eventually, the human role changes almost imperceptibly. The person who once made the decision now supervises the recommendation.

And after enough time, an uncomfortable question appears:

Can the supervisor still do what the system is now doing?

This is not a hypothetical concern about humans being “replaced by AI”. I find that framing too simplistic. It is a question about what happens to expertise when the exercise of expertise is gradually delegated.

Skills that are not exercised change.

Judgement that is rarely required becomes harder to mobilise.

Knowledge that is no longer transmitted because “the system knows it” may eventually disappear from the organisation altogether.

We could therefore create organisations in which artificial intelligence becomes progressively more capable while organisational intelligence quietly declines.

Figure 2 — The paradox of AI capability and human judgement. As AI systems become more capable and organisations rely on them more extensively, opportunities to exercise human expertise may diminish. The challenge is not simply to keep a human in the loop, but to preserve the knowledge, experience and authority required to question the system when it matters.

That possibility interests me far more than the familiar question of whether AI will replace particular professions.

Because an organisation can retain every employee and still lose knowledge.

It can keep humans formally “in the loop” while making their presence largely ceremonial.

Imagine an employee who must approve an AI recommendation but has five minutes to do so. The underlying evidence is difficult to access. Challenging the recommendation creates additional work. Performance indicators reward throughput. The system has historically been reliable. And, over time, the employee has had fewer opportunities to exercise the professional judgement that the approval supposedly represents.

• There is still a human approval button.
• There is still accountability on the organisational chart.
• There is still a person in the loop.

But is there still judgement?

This, for me, is where AI literacy becomes inseparable from organisational design.

We can train somebody to understand that AI can be wrong. But if we do not give that person the time to investigate, the evidence to challenge, the expertise to interpret and the authority to disagree, their literacy has little practical value.

Meaningful human oversight cannot therefore be created simply by inserting a human checkpoint into an automated process.

The organisation has to preserve the conditions under which disagreement is possible.

And perhaps that is the deepest level of AI literacy: not knowing how to operate the system, but knowing when the organisation should refuse its recommendation — or even refuse to use it.

There will be many decisions for which extensive automation is entirely reasonable. Not every activity needs philosophical reflection or elaborate governance. Organisations should use AI to eliminate unnecessary work, accelerate analysis and augment people’s capabilities.

But consequential decisions are different.

At some point, someone has to ask questions that are not technical.

Not only: Can AI do this? But: What changes if it does?

Not only: How accurate is the model? But: What happens when it is wrong?

Not only: Is there a human in the loop? But: Is that human still capable of disagreeing?

And finally:

Should this be an AI-mediated decision at all?

These are questions of governance, but they are also questions of intelligence.

I have spent much of my professional life watching new technologies enter organisations. From expert systems and distributed computing to the Internet, mobile services, digital payments, cloud platforms and now generative AI, the technologies have changed dramatically.

Yet there is a pattern I recognise.

We initially introduce technology into the organisation.

Then we redesign processes around the technology.

And eventually, often without noticing it, the technology begins to influence how the organisation itself understands the problem.

With AI, that progression may happen faster because the technology is no longer confined to processing transactions or moving information. It increasingly participates in interpretation.

That is why I think the discussion about AI literacy deserves to become much more ambitious.

Yes, people need to know how to use the tools.

They need to understand their limitations.

They need to recognise missing context.

Experts need to retain the ability to challenge their outputs.

Managers need to design processes in which that challenge remains possible.

And leaders need to decide where the boundary between machine capability and institutional responsibility should lie.

Perhaps, then, the most useful way to think about AI literacy is not as a curriculum but as a series of increasingly difficult questions.

• Can I use the tool?
• Can I understand the context around its answer?
• Can I recognise when it should be challenged?
• Am I actually able to challenge it?
• And do we, as an organisation, still know when to say no?

The first question is about skills.

The others are about judgement.

And that distinction may become increasingly important as AI improves.

Because the ultimate measure of an intelligent organisation will not be how much artificial intelligence it manages to deploy.

It will be whether, after deploying it, the organisation has preserved something much harder to automate:

its capacity to judge.