Good AI governance may not be glamorous, but it could determine which organisations make the most of AI. That was one of the key messages from Kate Jones, CEO of the Digital Regulation Cooperation Forum (DRCF), in her keynote speech at our recent AI Agenda conference.
Jones likened good governance to plumbing: when it works properly, you barely notice it and may not fully understand how it operates. It is when something goes wrong that its importance suddenly becomes apparent.
That message is particularly pertinent as concerns about the risks associated with AI continue to grow. Jones pointed to recent examples of AI going wrong, including the controversy surrounding West Midlands Police's assessment of whether an Israeli football team could safely play in Birmingham, as well as widely puclicised examples of AI "hallucinations" in the legal sector.
Her central message was that the organisations that do best with AI will be those that combine good governance with strong strategic leadership.
AI as a digital workforce
Agentic AI presents a particular challenge. Jones gave the everyday example of an AI agent managing your gym membership as part of a plan to get you "beach ready". But the ability of agents to undertake multiple actions on an individual's behalf raises more fundamental regulatory questions. As actions become bundled together, it can become harder to apply safeguards such as consent at the appropriate point.
Jones suggested that businesses should increasingly think about AI as a "digital workforce", and govern it accordingly. AI may perform a task autonomously, but that does not remove the responsibility of the organisation deploying it.
This means businesses need to understand who is accountable for AI systems, establish effective shared ownership and have the capability to oversee what those systems are actually doing. Importantly, Jones cautioned against assuming that someone who can perform a task themselves will necessarily be capable of supervising AI performing the same task.
The DRCF CEO recommended reading the Mills Review – a landmark report from the Financial Conduct Authority, a DRCF member regulator. It sets out how AI could reshape the retail financial services sector and, according to Jones, provides a particularly clear indication of the regulatory direction of travel for AI.
What are regulators seeing?
Jones identified three significant changes regulators are observing:
- AI is becoming embedded in operational workflows.
- Consumer use is increasing, alongside greater confidence in allowing AI to act autonomously.
- Trust is becoming decisive in determining whether AI deployment succeeds.
For Jones, good regulation and innovation are not opposing objectives. Smart regulation can enable innovation, as experience in areas such as cybersecurity and autonomous vehicles demonstrates.
She also highlighted the DRCF's foresight work on emerging technologies, including agentic AI and consumer robotics, and its work to help organisations navigate the regulatory landscape.
Governance as a strategic enabler
Perhaps the most important takeaway was that responsibility does not disappear simply because AI becomes more autonomous. Organisations need clear accountability, effective oversight and, crucially, people with the skills needed to challenge and supervise AI systems.
For lawyers and compliance teams, that may require a shift in mindset. Rather than acting solely as risk managers, Jones encouraged them to become strategic enablers, helping their organisations capture the benefits of AI while putting appropriate guardrails in place.
The stakes are long term. The governance decisions organisations make now could shape their behaviour for decades. The lesson, therefore, is not simply to be the fastest adopter of AI. It is to make sure the governance is there to support it.
Why this matters
As AI moves from answering questions to taking actions, traditional governance models will come under increasing pressure. For businesses, the practical challenge is to establish accountability, oversight and organisational capability before autonomous systems become deeply embedded in everyday workflows. Good governance should not be treated as something that slows AI adoption down. Done well, it is what makes sustainable adoption possible.
