3. The next digital divide may be between those who delegate and those who are delegated to

Here is a less frequently discussed possibility.

Imagine two people negotiating a service.

One uses an AI agent capable of comparing thousands of offers, remembering every contractual detail, negotiating continuously and optimising the outcome.

The other relies on personal experience, limited time and ordinary human attention.

The asymmetry is immediate.

Now imagine both parties are represented by AI agents.

The interaction becomes machine-to-machine, with humans potentially involved only at the beginning and end.

Who defines the negotiation objectives?

Who determines what constitutes a fair outcome?

And what happens when optimisation on both sides produces results that are technically acceptable but socially undesirable?

We could be approaching an economy in which access to capable AI agents becomes a new form of negotiating power.

The consequences for consumers, businesses, workers and institutions deserve much more attention.

4. When agents negotiate with agents, what happens to human relationships?

Much of our economic and social infrastructure assumes that interactions ultimately involve people.

We call a business, speak with someone, negotiate, complain, persuade, compromise or build trust.

But imagine a future in which your AI assistant calls a company and is answered by another AI assistant.

Two systems exchange information, verify constraints, negotiate terms and complete a transaction.

No human conversation is required.

This may be extraordinarily efficient.

But efficiency is not the only purpose of human interaction.

Conversations also create understanding, relationships, opportunities for empathy and the possibility of unexpected outcomes.

What happens to society when a growing proportion of interactions is optimised for transaction completion rather than human connection?

We might gain enormous amounts of time while gradually eliminating interactions that have value beyond their immediate economic purpose.

That is not necessarily an argument against automation.

It is an argument for recognising what we are automating.

5. Identity, accountability and the problem of artificial representation

There is another fundamental issue.

When an AI agent contacts a company on my behalf, how does the company know that the agent genuinely represents me?

What authority does it possess?

Can that authority be limited, verified, transferred or revoked?

And if the agent makes a mistake, who is responsible?

The user? The provider? The developer? The organisation operating the system?

We have spent decades developing digital identity, authentication, payment authorisation and security infrastructures.

Agentic AI introduces a new challenge: we must establish not only who an actor is, but whom it represents, what it is authorised to do and under which conditions.

Identity without verifiable authority is insufficient.

Autonomy without accountability is dangerous.

And trust without transparency is fragile.

These are not merely technical implementation details.

They are architectural questions about how an increasingly automated society should function.

6. The paradox of human-in-the-loop

Fo also highlights an interesting design choice: the possibility of involving human assistants when AI cannot complete a task independently.

This is important because it challenges the simplistic assumption that technological progress necessarily means removing humans from the process.

But it also raises a question.

Are humans becoming supervisors of intelligent systems, or are they becoming the fallback mechanism for tasks that machines cannot yet handle?

The difference matters.

In the first case, human intelligence retains authority, interpretation and responsibility.

In the second, humans risk becoming an operational exception within an increasingly machine-centred architecture.

The future of human–AI collaboration will depend on how we distribute not only tasks, but also agency, judgement and control.

A human-in-the-loop architecture is not automatically human-centred.

It becomes human-centred only when the human role is meaningful.

7. From artificial intelligence to institutional intelligence

This is where I believe the discussion becomes particularly interesting.

We tend to evaluate AI agents by measuring their individual capabilities.

• Can they complete a task?
• Can they reason?
• Can they use tools?
• Can they interact with external systems?

But as agents become interconnected, the more important question may concern the intelligence of the overall system.

An organisation with hundreds of autonomous agents is not necessarily an intelligent organisation.

It may simply be an organisation with hundreds of independently optimising systems.

The real challenge is to design architectures in which human intelligence, artificial intelligence, organisational knowledge and institutional rules operate coherently together.

This requires governance, shared context, traceability, accountability and mechanisms for resolving conflicting objectives.

It also requires something we often underestimate: a clear understanding of which decisions should remain human.

The future is not simply agentic. It is relational.

Fo is an interesting demonstration of what is becoming possible.

But the significance of such technologies cannot be measured only by the number of telephone calls they can make, appointments they can arrange or transactions they can complete.

Their real significance lies in how they may transform the relationship between intelligence, authority and action.

For centuries, we have designed institutions around human actors who possess identities, responsibilities, intentions and varying degrees of authority.

We are now introducing artificial actors into those same environments.

Our technological capacity to delegate action may be advancing faster than our collective capacity to govern delegation.

That, to me, is one of the defining challenges of the agentic AI era.

The question is no longer simply:

Can AI do this for me?

It is becoming:

What should AI be allowed to do for me, under whose authority, and with what consequences for everyone else?

And perhaps the most important question of all:

Are we building AI agents to make human life more meaningful, or merely to make human activity more efficient?

The answers will shape much more than the next generation of software.

They will shape the architecture of our future society.

🎬 The accompanying video demonstrates Fo by Wajo. Thanks to Shivani and the Wajo team for allowing me to share it.

🔭 Future with Frankie is my new Instagram initiative exploring AI, robotics and technologies that could transform our world.

https://www.instagram.com/futurewithfrankie/

I would be interested in your perspective: when AI agents begin representing us in the real world, where should the boundary between delegated autonomy and human authority be drawn?