Agentic AI and AI Bias in 2026: Why Your Governance Framework Is Already Outdated
Two themes dominated the data protection and AI conference I attended this week, and they are connected in ways that matter for every business deploying AI: the rise of agentic AI systems that can act without stepwise human approval, and the uncomfortable reality that bias is embedded in the mathematical foundations of the large language models underpinning those systems.
This article draws on the presentations and panel discussions at the PDP Data Protection and AI Conference 2026 and applies them to the practical governance challenges facing GCs, CTOs, and in-house counsel.
What Is Agentic AI and Why Does It Break Existing Governance?
Most AI governance frameworks were designed around a simple assumption: AI produces outputs, humans decide whether to act on them, and the risk sits at the point of human decision making. Agentic AI breaks that assumption.
Agentic AI systems do not just draft, summarise, or recommend. They act. They place orders. They move money. They screen candidates. They negotiate with suppliers and trigger downstream systems. The distinction matters because we are no longer delegating execution. We are delegating a degree of judgment and discretion to systems that can operate with minimal oversight.
OpenAI’s white paper on governing agentic AI systems identifies the key insight: agentic risk is not about the model. It emerges from the system design and how it is deployed. The same model can produce radically different risk profiles depending on the scaffolding around it: the permissions it is granted, what it can access, what it can change, and what it can execute.
This means enterprise-level governance, the policies, principles, and committees, is necessary but not sufficient. You also need system-level governance that answers specific questions: Does this agent have permission to execute a refund? Is it logging every external action? Do we require approval above a certain transaction threshold?
What Are the Legal Risks of Agentic AI?
The legal exposure sits across at least four frameworks, all of which already exist and are enforceable.
Consumer protection: Under the Digital Markets, Competition and Consumers Act 2024, you cannot contract out of statutory protections. If an AI shopping agent selects the wrong item, fails to respect allergies, or triggers a sequence leading to harm, the trader’s conduct remains in scope even if the experience is mediated by an AI agent. The CMA has made clear that protecting consumers from AI-related harms is a priority area for enforcement.
Product liability: The new EU Product Liability Directive treats software and AI as products for strict liability purposes. In an agentic context, harm can be triggered by a chain of autonomous actions, not a single human decision.
Negligence: The Hedley Byrne v Heller doctrine on assumption of responsibility maps onto AI systems framed as tailored, authoritative, and designed to be relied upon. The risk increases where the system is performing the advisory or decision-making role itself, exactly the posture that negligence misstatement was built around.
Data protection: The revised Article 22 UK GDPR, as amended by the Data Use and Access Act, converges with these liability frameworks. Agentic systems are more likely to trigger both business liability and Article 22 enforcement because reliance is structural rather than discretionary.
What Should Businesses Do About Agentic AI Governance?
Four practical governance primitives, applicable now:
1. Clear accountability assignment. At least one human entity must be accountable for harms caused by an agentic system. AI agents have no legal personality. Suppliers often disclaim meaningful responsibility. If accountability is unclear internally, it will be assigned externally by regulators, courts, and the press.
2. Action ledgers. Lightweight, chronological records of what the agent actually did. This goes beyond technical logging. When something goes wrong, organisations need to explain behaviour they did not fully anticipate. Governance needs to capture behaviour, not only design intent.
3. Capability boundaries and approval gates. Not every action needs human approval, but consequential actions do. The legal difference between humans approving outcomes and humans auditing after the fact changes the risk profile dramatically.
4. Reversibility and shutdown. Design actions to be undone wherever possible, and ensure there is a reliable way to stop agent operations. Irreversibility magnifies harm.
Why Can’t AI Bias Be Eliminated from Large Language Models?
The panel discussion on AI bias at the conference was striking for its candour. The short answer: LLMs are trained on everything we have ever written, and everything we have ever written is biased.
Here is a concrete example from the panel. In large language models, words are converted into numerical vectors, hundreds of thousands of dimensions. If you take the vector for "King," subtract "man," and add "woman," you get "Queen." That is logical. But if you take "doctor," subtract "man," and add "woman," you get "nurse." That is not a bug in the model. It is a statistical reflection of the text the model was trained on.
The mathematics are doing exactly what they are supposed to do: identifying patterns in human language. The problem is that the patterns in human language contain centuries of bias.
As one panellist observed, the performance pressure on foundation model providers makes de-biasing even harder. Every attempt to retrain or constrain the model to reduce bias damages performance benchmarks. When OpenAI tried to reduce sycophantic behaviour before GPT-5, users complained the model was less creative and less capable. The commercial incentives push toward performance, not fairness.
If Bias Cannot Be Eliminated, What Can Be Done?
The answer from practitioners is consistent: build governance structures around the outputs, not inside the model.
Santander’s approach illustrates best practice across the lifecycle. Before training: collect balanced data, clean for bias, avoid demographic features. During training: penalise biased outputs, add fairness constraints. After deployment: monitor for performance drift, maintain human review, retrain when triggered by drift thresholds or human feedback.
For organisations using third-party LLMs (which is most organisations), the practical toolkit is different but no less important. You cannot retrain the model. You can implement output guardrails, measure bias in results, design governance frameworks that treat bias as an ongoing operational risk, and build human review into consequential decision chains.
The broader lesson is structural. We have built functioning institutions out of biased human beings who try not to be biased. The approach to AI should be the same: accept that the models are biased, measure the bias, and build systems that mitigate the effects before they reach people.
How Does This Connect to the EU AI Act?
The provisional agreement reached on 7 May 2026 includes a notable expansion: sensitive personal data (ethnicity, race, etc.) may now be processed when strictly necessary for bias detection and correction in relation to any AI system, not just high-risk systems. This reflects regulatory recognition that bias detection requires access to the very categories of data that are otherwise restricted.
The EU AI Act also provides the structural framework that many organisations, including US companies, are using as a governance handrail even where they are not legally obliged to comply. As one panellist observed, when 1,000 people have spent years reaching consensus on a risk framework, it would be unusual not to use it as a starting point.
Rory O’Keeffe is the founder of RMOK Legal, a City of London commercial law practice specialising in AI governance, technology contracts, and fractional general counsel services. He is an SCL-accredited Leading IT Lawyer, AI Committee member of the Society for Computers and Law, and author of AI Advantage (2025).
FAQs
-
AI systems that can act autonomously, placing orders, moving money, screening candidates, or negotiating with suppliers, rather than simply producing recommendations for human review.
-
No. LLMs are trained on human-generated text that contains historical bias. De-biasing the models themselves damages performance. The recommended approach is building governance structures, guardrails, and monitoring around the outputs.
-
Four practical primitives: clear human accountability for harms, action ledgers capturing agent behaviour, capability boundaries with approval gates for consequential actions, and reversibility with shutdown mechanisms.