AI Agents in the era of Responsible AI

Over the last few months, AI agents have become the buzzword of choice within the AI community. Unlike a standard AI model, an agent autonomously plans and executes multi-step tasks with minimal human involvement in each step, pursuing a goal across a sequence of decisions. By combining tools and chaining tasks, the possibilities of what AI can do is expanded greatly through the agent framework.

This autonomy is also what makes agents genuinely difficult to govern. Most existing Responsible AI frameworks were not designed with agentic AI in mind, and as companies begin to adopt agents, the gap becomes harder to ignore. To navigate this challenge, organisations need to examine what responsible AI and AI agents actually require, and why they can pull in different directions.

What AI agents actually do

AI agents are autonomous systems designed to perceive their environment, make decisions, and take actions to achieve specific goals with minimal human intervention. Unlike traditional AI tools that provide recommendations for human review, agents are built specifically to act independently across extended periods.  

In practice, modern AI agents combine large language models with memory systems and external tools, enabling them to handle complex, multi-step processes. They can schedule meetings across multiple calendars, optimise supply chain decisions in response to changing conditions, or manage customer enquiries from initial contact through resolution. Their adaptive capabilities mean they learn from experience and adjust their behaviour over time. 

This autonomy creates genuine business value. Agents can operate continuously, handle routine tasks that consume human resources, and respond to situations faster than human-mediated systems. They excel at managing complexity at scale in ways that would be impractical with traditional human-in-the-loop approaches. However, this same autonomy makes traditional responsible AI oversight challenging. 

When an agent makes hundreds of decisions per hour, continuous human review becomes impractical. Many AI agents are also built to adapt their behaviour based on new data. While this is a useful feature, this also makes it more difficult to predict its future actions. When an AI agent operates independently, maintaining transparency and explainability becomes complex. 

While AI agents offer significant business value through their autonomy, this very autonomy creates challenges for maintaining responsible AI practices. 

The ethical dilemma of AI agents

Integrating AI agents into a Responsible AI framework is not just a technical challenge; it's also an ethical one. Traditional Responsible AI principles emphasise human oversight and accountability. However, AI agents are designed to operate autonomously, making it difficult to maintain this oversight.  

To address this, organisations are adopting various strategies: 

• Embedding Responsible AI principles into agent systems through monitoring and escalation protocols. 
• Implementing hybrid systems where agents handle routine tasks and escalate complex decisions to humans. 
• Prioritising efficiency over explainability, with transparency about these trade-offs. 

These approaches aim to balance the benefits of AI agents with the need for ethical governance. However, it's crucial to recognise that the effectiveness of these strategies depends on the specific use case and the level of risk involved.  

A risk-based approach to AI agents

Organisations can adopt a risk-based approach to evaluate where AI agents align with responsible AI principles. The EU AI Act's risk framework provides a useful model for this type of evaluation. 

While AI agents share core technologies with traditional systems, their increased autonomy makes them riskier. Higher autonomy complicates oversight, prediction, and intervention, amplifying the risks involved. Thus, mitigating these risks is essential for responsible AI deployment. 

The level of risk depends on the use case:

• High-risk applications like employment decisions, financial services, and healthcare require more transparency, explainability, and human oversight. 
• Medium-risk applications, such as customer service or content generation, still need transparency but can operate effectively with minimal human oversight as long as users are aware they're interacting with AI. 
• Low-risk applications such as internal process automation and data analysis offer the greatest opportunity for autonomous operation without significant tension with responsible AI principles. 

A framework for conscious choice-making helps guide decisions:

• What are the consequences if the agent makes an error? 
• How important is human oversight for this use case? 
• What level of autonomy aligns with responsible AI commitments? 
• When does efficiency justify reduced oversight? 

Ultimately, the goal is not to eliminate all risk but to make informed decisions about where AI agents can add value while maintaining responsible AI principles. By mapping use cases according to risk level and transparency needs, organisations can deploy agents effectively while ensuring ethical governance.