Against that backdrop, the recent AI Agenda panel discussion offered a timely exploration of exactly these issues. Bryony Long, Partner and Co-Head of the Data, Privacy and Cyber Practice at Lewis Silkin, brought together three leading voices to examine where AI regulation stands today, what responsible adoption looks like in practice, and what lies ahead. The panellists were Sophia Ignatidou, Head of AI at the ICO; Emre Kazim, Co-Founder and Co-CEO of Holistic AI; and Hector de Rivoire, Director, Responsible AI Public Policy at Microsoft.
Start with the fundamentals - but don't stop there
One of the clearest messages from the session was that regulators expect organisations to get the basics right. The ICO expects to see a Data Protection Impact Assessment in place - or, at the very least, a documented decision explaining why one was not carried out and why the processing was not considered high risk. A robust AI policy governing internal use of AI is also a baseline expectation.
Sophia Ignatidou acknowledged that the ICO recognises the practical challenges organisations face in meeting compliance requirements. Nevertheless, in her experience, many still treat data protection principles as an afterthought. Transparency is a case in point: complying with this principle is critical when deploying large language models, which process data at scale across multidimensional datasets. She also cautioned against the assumption that organisations can simply "press play" on an AI system: appropriate governance requires data quality, appropriate technical and organisational measures, and employees who are upskilled to review AI output and make informed judgement calls.
AI governance is a category in itself
A key takeaway was the acknowledgement that AI governance cannot be treated as a bolt-on to existing compliance frameworks. Emre Kazim reflected that Holistic AI had initially assumed AI governance would simply be a continuation of data governance, but experience has shown it is a distinct discipline requiring its own methodology.
He also shared that the assumption that the EU AI Act would become the de facto global standard in the way that the GDPR became for data privacy has not been proven true in their experience. AI carries different strategic value for organisations, and we are seeing a regionalisation of compliance, with the US in particular taking a markedly innovation-friendly approach. For multinational businesses, this means navigating an increasingly fragmented landscape - Hector de Rivoire noted that there are currently over 2,000 AI-related bills at US state level alone.
What leading organisations are doing differently
Microsoft's approach offered a useful case study. The company has created a responsible AI standard - essentially a rulebook for product teams - along with a dedicated testing team. With 500,000 AI agents deployed, Microsoft has prioritised foundational governance such as identity management, human approval workflows and containment practices to prevent agents going rogue. It is increasingly important for organisations not only to build assurance around AI through internal governance, but also to engage independent third parties to verify that such governance effectively mitigates AI-related risks. Hector de Rivoire highlighted that transparency is critical in two dimensions: 1. organisational transparency and 2. product transparency, including the use of model cards and system cards, and noted that Microsoft is partnering with the UK AI Safety Institute to review its governance and produce benchmarks for testing. Hector emphasised that organisations need to go deeper than ISO/IEC 42001, pointing to best-practice initiatives such as the C2PA standard - an open technical specification that provides cryptographically verifiable "Content Credentials" to authenticate the origin, history and integrity of digital media. He also encouraged businesses to look to industries like nuclear and pharmaceuticals for models of achieving compliance at a global scale.
Compliance needs to evolve
A recurring thread throughout the discussion was that traditional compliance models are not fit for the pace of AI development. Static compliance forms do not enable real-time monitoring, and the prevalence of shadow AI within organisations creates significant accountability gaps.
Sophia Ignatidou called for a more radical approach to compliance - one that goes beyond organisational measures and legislative change and instead brings all disciplines into the room: legal, risk, technical, finance and beyond. Each has the expertise to contribute to solving the problem, and we need to develop technical measures to govern AI rather than relying solely on process-based controls.
Bryony Long noted that the concept of the "human in the loop" will become increasingly irrelevant as systems grow more autonomous, reinforcing the need for technical standards. Emre Kazim went further, observing that we may end up in a situation where AI is governing AI, and that the current period - in which there is a gap between capital investment and financial gains - will be a critical window for establishing best practice.
What's next from the ICO
The panel concluded with a valuable update on the ICO's upcoming agenda. The regulator plans to publish two pieces of work in the first week of October: a conclusion on its work programme on foundational models, and a call for evidence on agentic AI. An update to its guidance on automated decision-making is expected by the end of the year, alongside a Code of Practice on Automated-Decision Making and AI, which will be reviewed by a panel of experts before publication.
Key takeaways for organisations
The panel left attendees with a clear message: governance is not just a risk-mitigation exercise; it should be part of organisational strategy. Organisations that treat AI governance as a strategic function, invest in cross-disciplinary collaboration, and stay ahead of an evolving regulatory landscape will be best positioned as AI matures.
For now, the practical priorities are clear: ensure DPIAs are in place and that appropriate risk assessments have been carried out before processing begins, maintain robust AI usage policies, invest in employee upskilling, and prioritise privacy by design and by default from the outset. To close, Sophia Ignatidou noted that organisations must also be alive to what happens post-deployment; for example, some software developers have launched new AI features in products without informing their customers in advance, presenting an additional governance challenge. Hector also noted that the speed of technology development makes it hard for organisations to crystallise a risk taxonomy.
AI governance is therefore not just a one-and-done compliance exercise but an ongoing commitment to monitoring, adaptation, and accountability that must be embedded in organisational culture to be truly effective.
