AI Audits: How do you implement the EU AI Act?
Reading Time: 6 minutes
Authors:
Mariano Martín Zamorano Barrios | Senior Research Analyst
Date: 24 July 2024
This article explores the role of AI audits in supporting the objectives laid out in The EU AI Act and briefly discusses their potential desirability and suitability in ensuring adherence to the Act’s requirements.
On July 12th, 2024, the new regulation establishing harmonised rules for artificial intelligence (the "EU AI Act") was published in the Official Journal of the EU. This comprehensive legal framework governs the development, market-entry, deployment, and everyday use of AI systems. Its core aim is to ensure these systems operate transparently, accountably, and in respect of fundamental rights. The EU AI Act adopts a risk-based approach, as follows:
- Unacceptable risk systems, generally prohibited under Article 5, include those with high potential for harm, such as applications manipulating cognitive behaviour or real-time biometric identification, but with exemptions such as those specified in Art. 5, 3 (i.e. use of real-time remote biometric identification in public for law enforcement with prior judicial approval, or post-authorisation within 24 hours in urgent cases).
- High-risk systems (Art. 6), include, amongst others, those used in financial services, healthcare, incl. medical devices, employment, law enforcement, or critical infrastructure.
- Limited risk systems refers to the risk related to transparency and requires that humans are informed about the use of AI when necessary. The EU AI act allows the free use of minimal risk AI systems.
No matter which type of system is being used, organisations need to identify what category the system falls within and ensure compliance with the specific requirements of that risk-level. Furthermore, the Act mandates organisations embed compliance mechanisms before systems are placed on the market to ensure their AI technologies meet legal standards from inception.
AI audits, while not specifically mandated for providers and deployers under the EU AI Act, provide a robust foundation for ensuring compliance with many of its provisions. The Act emphasises principles of fairness (Rec. 74, 110, 27), human oversight (Art. 14), accuracy (Art. 15), and transparency (Art. 50). These principles can be effectively framed, assessed, and monitored through rigorous auditing procedures aiming for preventing bias and protecting fundamental rights. Moreover, the above risk-based approach is translated into provisions such as Article 9, which mandates that high-risk AI systems must be subjected to continuous and documented risk management throughout their lifecycle. This procedure should involve tasks typically falling under AI audits, such as testing AI systems’ impacts on vulnerable groups or estimating risks from intended use and misuse. Additionally, Audits can be the source and outcome of Technical documentation (Art. 11) tasks requested by the Regulation. This article explores the role of AI audits in supporting these objectives and briefly discusses their potential desirability and suitability in ensuring adherence to the EU AI Act's requirements.
What about AI Audits?
AI audits have various and complex legal definitions and conceptualisations. They are described as “structured evaluations designed to identify risks and improve transparency by assessing how well models and methods meet specific desiderata.” According to Raji et al. (2020), audits, especially algorithmic ones, provide external validation of the principle of "doing no harm" by revealing biases or flaws in AI systems. Definitions generally agree that AI audits are critical tools for examining the complexities of AI systems against ethical and legal standards, and they help providers demonstrate due diligence.
Formally, AI audits encompass several types:
- Internal audits, performed internally by company personnel or IT teams with ample access to system information.
- External and/or adversarial audits offer an opportunity for public and scrutiny and transparency, enabling AI systems to be evaluated from an independent perspective. These audits provide an extra and different layer of assurance by challenging internal assumptions and potentially enhancing trust in system integrity. Additionally, the so-called external cooperative audits involve external parties who have controlled access to system features, fostering cooperation and more thorough assessment while respecting confidentiality.
These audits may be conducted either before deployment, during the design and development phases, or after deployment when the system is operational. Each stage addresses flaws at different points in the AI lifecycle and may focus on sources of biases specific to that stage.
AI auditing is becoming more specialised. It includes various types with different scopes and focuses, such as AI governance audits, ethics-based audits and model or data-focused audits. Technical assessments can provide information on accuracy and fairness using practical tools and methods from systems engineering and computer science.
Additionally, legal approaches may evolve, making AI auditing an integral component of compliance with specific regulations or standards. For instance, New York City's Bias Audit Law requires mandatory third-party audits of AI hiring and employment systems.
Visualising the AI Act- AI audit link
Although the EU AI Act does not mandate AI audits, incorporating them can be highly beneficial for many reasons. Annex VII and Article 74 regulate AI Audits for Notified Bodies and Market Surveillance Authorities, respectively. These provisions indicate that, under specific risk and monitoring conditions, these bodies have the right to access technical documentation, the AI system's code, and its training and testing datasets upon request. Ongoing discussions are being held concerning their Member State designation and policies. In this scenario, providers and deployers of high-risk systems should prepare a solid technical foundation not only to facilitate audits by official bodies but also as a best practice for ensuring system reliability and compliance. In this way, in addition to the focus on preparatory actions for official audits, robust audit policies are also valuable for ensuring adherence to the specific principles and requirements of the EU AI Act, including:
| Requirement | Assessing these within AI Audits |
| Non-discrimination and fairness - Rec. 74, 110, 27 | Auditors assess the diversity and representativeness of training datasets to identify and prevent biases that could lead to unjust outcomes. They also evaluate the public outcomes of AI applications across different demographic groups to promote fairness and ethical deployment. |
| Transparency and accountability - Art. 50 | Auditors review internal documentation and system architecture to ensure transparency standards are met, enabling users to understand AI operations. They also verify that disclosed information aligns with AI system operations and decision-making processes. |
| Accuracy and Reliability - Art. 15 | Auditors conduct robustness testing to evaluate performance against adversarial attacks and unforeseen scenarios. They also simulate diverse conditions and perform stress tests to validate reliability claims. |
| Data privacy and security - Art. 10 | Auditors evaluate data handling practices to ensure personal data is adequately protected in automated processing and review user-accessible data, such as Cookies policies or public Data Management Plans, to assess compliance with AI processing security standards. |
| Human Oversight - Art 14 | Auditors assess whether procedures are in place for informed human intervention to override automated decisions or intervene in critical situations. They review documentation and interview relevant stakeholders and end users to ensure compliance. |
Known and emerging challenges of AI Audits
Implementing AI audits encounters various challenges. Auditing teams require significant resources and a combination of technical, sociological, legal and ethical expertise to address aspects such as AI governance or changing contextual scenarios influencing historical bias.
Challenges also include the complex process of defining audit criteria, the need for advanced audit tools, the complexity of specific AI systems (i.e., some NLP tools), or the need for better explainability. Another vital aspect is sufficient documentation or data access. So-called white- and outside-the-box audits provide developers with increased credibility and deeper problem-solving insights. However, developers often resist granting access for open box audits due to concerns about information leaks and reputational risks. This reinforces a tendency to produce black-box audits, which can limit the effectiveness and transparency of AI evaluations. Along these lines, Casper et al. (2024) argue that outside-the-box audits offer significant benefits over black-box audits by providing deeper insights, enabling targeted interventions for AI systems and allowing for more precise diagnoses. Recent research confirms the importance of AI auditability measures in evaluating AI ethics. Advanced audit tools, combined with clearer regulatory guidance and enhanced collaboration between internal and external auditors, further highlight the superior value of open-box audits in ensuring responsible AI.
Navigating compliance and auditing in the EU
The introduction of the AI Act represents a step towards regulating AI technologies within the EU. By conducting rigorous audits, both providers and deployers may ensure their AI technologies meet regulatory standards, ethical principles, and relevant societal values.
However, the EU AI Act does not outline any specific auditing framework for AI: it does not define a specific approach, set of methodological standards or required system characteristics across different applications. Instead, industry actors, standards organisations and other stakeholders will gradually need to establish or adapt governance, tools, and operational strategies to pave the way for (efficient, transparent, and robust) auditing activities. In this process, these actors are also expected to rely on experts who already have these strategies, skills, and resources in place.
Nevertheless, as the space matures some key risks and issues need to be managed effectively. Typical existing issues include tensions between providers’-commercial interest and IP barriers that may prevent data access within AI systems. Additionally, ensuring the validity and reliability of audit results while adhering to fairness and transparency principles can be further complicated by data and commercial restrictions. Emerging issues include the need to establish industry-wide norms and protocols that define the scope of audits (what will be audited and how) while also developing policies for "auditing the auditors" (who will monitor and ensure auditors’ ethical and legal compliance). “Ethics by design” in AI auditing, namely establishing standards and protocols that ensure ethical guidelines are adhered to throughout the auditing process, will therefore be key.
The Trilateral Ethics Innovation Team has extensive experience and expertise in assisting organisations with responsible and ethical use of AI and compliance with the EU AI Act. For your AI governance, algorithmic auditing, or EU AI Act compliance needs, please contact us by email dcs@trilateralresearch.com to discuss your requirements. Our team would be happy to help.
