Davies, a professional services firm specializing in insurance compliance, argues in its new white paper that autonomous AI agents are creating a governance gap that existing regulatory frameworks weren’t designed to close. The paper, part of Davies’s AI Impact Series, examines how rules like the FCA’s Consumer Duty and the Senior Managers and Certification Regime already technically apply to AI-driven decisions, but fail to account for systems that adapt, drift, and act continuously<\/a> without human sign-off at each step. The core prescription is that AI agents must be treated as living systems requiring ongoing oversight, not one-time compliance approvals.<\/p>\n The governance gap Davies identifies isn’t about missing regulations. It’s about a structural mismatch between how compliance was built and how autonomous agents actually behave. If your organization cleared an AI deployment through a pre-launch review and then handed it to operations, you’re already in the gap. The insurers most exposed are those who inherited AI governance from their traditional model-risk playbooks and haven’t revisited them since agentic architectures arrived.<\/p>\n The harder problem Davies surfaces is accountability diffusion. Traditional AI systems have a decision point you can audit after the fact. An autonomous agent running a continuous decision loop, learning from new data, and triggering downstream actions through third-party APIs doesn’t have a clean moment of accountability to attach to a named senior manager. The SM&CR, which assigns individual liability to executives for regulated activities, was written for humans making decisions, not for agents making thousands of micro-decisions per hour. That gap is where conduct risk quietly accumulates.<\/p>\n The EU AI Act comparison is the tell in this paper. Davies notes the Act is becoming a de facto benchmark even for UK insurers operating entirely within FCA jurisdiction, and that’s a leading indicator worth tracking. When a governance standard from a different regulatory regime starts shaping internal policy voluntarily, it usually means the home regulator’s guidance is lagging behind what the industry knows it needs. CISOs at insurers who are waiting for the FCA to prescribe agentic AI controls specifically are betting on a timeline that the market isn’t honoring. The budget conversation worth having isn’t whether to fund continuous monitoring infrastructure, but whether the cost of building it now is smaller than the cost of an SM&CR enforcement action later.<\/p>\n Model drift is what happens when an AI system’s real-world environment changes after deployment, causing its outputs to quietly degrade or shift without any code change triggering the problem. Think of it as a compass that was calibrated in one location and slowly becomes less accurate as you move without anyone noticing. For insurers running autonomous agents, drift means a system approved for compliant behavior in January may be making materially different decisions by July, with no human review in between.<\/p>\nWhat this means for your business<\/h2>\n
Concept deep-dive: Model drift<\/h2>\n