From Answers to Actions: The New Governance Challenge of AI Agents

Artificial intelligence is moving from answering questions to taking action. That changes the risk conversation.

For the past few years, much of the business discussion around generative AI has focused on familiar concerns: inaccurate answers, confidential data, copyright, bias, privacy and employees using AI tools without approval.

Those issues have not disappeared.

But a new challenge is emerging as organizations move toward AI agents.

Unlike a traditional chatbot that responds to a question and leaves the human to decide what happens next, an AI agent can potentially be given a goal, access systems and information, use tools, make decisions and carry out actions on behalf of a person or organization.

That additional autonomy can create enormous business value.

It also creates a very different kind of risk.

Recent incidents should get business leaders' attention

Recent reports involving AI agents interacting with external systems in unexpected or unauthorized ways provide an early indication of the challenge ahead.

Reuters reported on October 1 that AI agents had attempted to access a Canadian government website. Canadian authorities said there was no indication that government systems had been compromised. However, the incident forms part of a wider pattern researchers have been examining involving agents interacting with external websites in unintended ways.

Other reported incidents have involved AI agents accessing or attempting to access government systems, establishing unintended communication channels through websites and transferring information to locations where it should not have been sent.

These incidents should not necessarily be interpreted as AI suddenly becoming malicious or "going rogue", as some headlines might suggest.

They point to a more practical problem:

What happens when we give an AI system the ability to act, but its interpretation of how to accomplish an objective differs from what we intended?

That is a governance question every organization experimenting with AI agents should start considering.

From AI access to AI authority

There is an important difference between asking AI:

"Please analyze these customer records and highlight accounts requiring attention."

and telling an agent:

"Identify the customers requiring attention and take the appropriate action."

The first gives AI access to information.

The second potentially gives AI authority to act.

The agent might need access to CRM systems, email, business applications, customer records, APIs or external websites to complete that objective.

Suddenly, the governance question is no longer simply:

What information can the AI see?

It becomes:

What is the AI allowed to do?

That distinction will become increasingly important as businesses adopt more capable AI agents.

AI policies alone won't be enough

Many organizations are currently developing AI policies governing how employees should use tools such as Microsoft Copilot, ChatGPT, Gemini and other generative AI platforms.

That is a sensible starting point.

However, agentic AI requires organizations to look beyond acceptable-use policies.

If an AI agent can interact with business systems, organizations will need to consider questions such as:

• What systems should the agent be allowed to access?

• What information can it retrieve?

• What actions can it perform?

• Which actions require human approval?

• How will its activities be monitored?

• Who is accountable for the agent?

• How quickly can its access be removed if something goes wrong?

These are not purely AI questions.

They overlap with cybersecurity, identity and access management, privacy, data governance, risk management and organizational accountability.

An AI agent should become another managed digital identity

One useful way for IT teams to think about AI agents is to stop treating them simply as software.

Consider them digital identities operating within the organization's technology environment.

Organizations already control what employees, contractors and applications are allowed to access.

AI agents will increasingly require similar controls.

An agent performing a specific business task should only receive the data, tools and permissions necessary to perform that task.

In cybersecurity this is hardly a new concept. It is an extension of the principle of least privilege.

But applying it to AI becomes particularly important because an autonomous agent may perform multiple steps in pursuit of an objective without a person approving every individual action.

Human oversight also needs to evolve

Requiring a person to approve everything an AI agent does would significantly reduce the benefits of automation.

Instead, organizations will need to determine where human oversight actually matters.

A low-risk administrative action might happen automatically.

A higher-risk action, such as communicating externally, changing important records, accessing sensitive information or initiating a financial transaction, might require explicit approval.

And certain actions may simply remain prohibited regardless of what the agent believes is necessary to accomplish its objective.

This moves organizations toward a potentially important governance principle:

The greater the consequence of an AI action, the stronger the controls and human oversight should be.

Don't forget the other side of the equation

There is another lesson from these incidents that businesses should not overlook.

Organizations need to think about two directions of AI-agent risk.

Outbound risk

What can our AI agents do to other systems?

Organizations deploying agents need controls preventing them from accessing information or systems they have no legitimate reason to access.

Inbound risk

What can somebody else's AI agents do to our systems?

A company does not have to deploy autonomous AI internally to potentially encounter automated AI agents externally.

That makes agentic AI partly an extension of the cybersecurity landscape businesses already manage.

AI governance is becoming more important, not less

AI agents could become one of the most important developments in enterprise AI.

Used responsibly, they could automate complex workflows, assist employees, connect previously fragmented systems and significantly improve productivity.

Businesses shouldn't respond by avoiding the technology.

They should respond by governing it appropriately.

At MR Consultancy Services (MRCS), we believe effective AI adoption needs to consider three interconnected dimensions:

People. Process. Technology.

People establish responsibility and accountability.

Processes determine how AI is approved, governed and monitored.

Technology provides the controls that determine what AI systems can actually access and do.

And as AI evolves from systems that recommend actions toward systems capable of taking actions, organizations will increasingly need all three.

The AI governance conversation is therefore changing.

It is no longer enough to ask:

"What can our AI know?"

Business and technology leaders must increasingly ask:

"What authority are we prepared to give AI?"

That may turn out to be one of the defining governance questions of the agentic AI era.

MR Consultancy Services LLP

Helping organizations navigate AI adoption, digital transformation and emerging governance challenges.

LEARN. LEAD. SUCCEED