Integrating AI Governance within the Data Privacy Function

This article explores scenarios where the Data Protection (DP) function, including the Data Protection Officer (DPO), may be best positioned to oversee AI governance in accordance with the EU GDPR (the GDPR) and the EU AI Act (the AI Act). DPOs are well-positioned to lead or support AI governance given the substantial overlap between the two frameworks: both require impact assessments, documented accountability, transparency to individuals, and controls over automated decision-making. It provides a starting point by showing how an existing DP programme can be leveraged and adapted to support AI governance. The article also considers potential conflicts of interest under the GDPR when the DPO takes on this role, and highlights the need for upskilling to meet evolving AI systems and regulatory demands.

Artificial Intelligence (AI) is reshaping industries and becoming integral to business operations and strategic planning, especially among larger organisations, as noted in Eurostat’s 2024 report. This rapid adoption underscores the critical need for strong governance frameworks that address not only AI’s technical aspects but also ethical and regulatory compliance. 

From a data protection perspective, AI is particularly relevant because AI systems are often trained on vast amounts of data, much of which can be personal information. While it offers significant benefits, it also presents notable data protection and privacy challenges, such as unclear data sources, data collected without proper consent or used in ways individuals might not expect, challenges in handling data subject rights requests, breaches, purpose creep, etc. 

Given these challenges, a key question for organisations is determining who should lead AI governance. In many cases, this responsibility may fall under the DP function. 

AI Officers vs. Data Protection Officers (and the broader DP function)

Unlike the DPO, whose designation and responsibilities are clearly defined in Articles 37, 38 and 39 of the GDPR, there is currently no universally accepted definition of an 'AI Officer'. The EU AI Act neither defines this role nor mandates the appointment of an AI Officer. Nonetheless, as AI adoption grows with many high-risk AI systems in use, it may be necessary to assign dedicated leadership, such as an AI Officer, to manage risks and ensure oversight, ensuring compliance with the AI Act and other rapidly evolving global regulations. 

The responsibilities commonly associated with an AI Officer include: 

• Developing AI governance frameworks,  
• Developing AI policies,  
• Leading risk and impact assessments,  
• Coordinating across business functions, 
• Monitoring AI system lifecycle, and  
• Staff AI literacy and training, 
• Engaging with regulators.  
• Many of these tasks naturally overlap with the DPO’s existing remit, including policy development, DPIAs, risk assessments, cross-functional collaboration, etc. These responsibilities may expand further to support AI risk management and oversight across regulatory frameworks. 

Integrating AI governance within the existing data protection framework offers several benefits: it minimises duplication of effort, reduces human resource costs, and leverages established structures to ensure consistent oversight aligned with compliance and ethical standards. 

This overlap suggests the practical starting point of integrating AI governance into the existing data protection structure. 

Where to start: Aligning AI governance with GDPR for EU AI Act

Organisations can build on existing GDPR compliance infrastructure to meet EU AI Act obligations. Key actions include: 

1. Leveraging GDPR tools for AI risk assessments:  Adapt existing DP platforms and DPIAs templates to assess AI systems, ensuring they also address AI-specific risks related to accuracy, ensuring fairness, non-discrimination, handling data subject rights, transparency and explainability, etc

2. Mapping AI systems & determine applicability: Inventory AI systems across development, deployment, and planning stages. Determine if the AI Act applies and define your role under both frameworks: Under GDPR: if Controller or Processor and under the AI Act: if Provider, Deployer, Importer, or Distributor.

3. Clarifying AI system functionality & risk classification: Clearly define each system’s purpose and classify its risk level under the AI Act if: prohibited, high-risk, or low-risk systems.

Based on this classification, conduct: 

• Data Protection Impact Assessments (DPIAs) (if high-risk processing activity under GDPR) 
• Fundamental Rights Impact Assessments (FRIAs) (if a deployer of a high-risk AI system under AI Act) 
• Conformity Assessments (CAs) (if a provider of a high-risk AI systems under AI Act)

4. Identifying risks, mitigate, and monitor continuously:  Identify and assess both AI and privacy risks, implement additional technical and organisational safeguards such as human oversight, bias detection, robustness checks, etc. Ensure ongoing monitoring and periodic reviews to address emerging risks and maintain compliance.

These foundational steps not only align with existing DP practices but also highlight the increasing overlap between AI governance and the responsibilities of the DP function. This convergence raises important considerations around role clarity, upskilling, and how to manage potential conflicts when the DPO is expected to engage with AI strategy and oversight. 

Managing potential conflicts and upskilling needs

According to the insights from The Confederation of European Data Protection Organisations (CEDPO) the allocation of AI governance responsibilities often varies depending on the size and maturity of the organisation. In smaller organisations, limited resources may require the DPO to handle both data protection and AI compliance duties, especially for tools like chatbots or generative AI, which means they will need to develop relevant AI governance skills – a topic further explored in this article. Larger organisations are more likely to separate these functions, with the DPO focusing on areas that overlap with data protection such as bias detection and human oversight. In organisations that develop and deploy AI systems, where regulatory obligations are more complex under the EU AI Act, DPOs need specialised technical knowledge to keep pace with evolving AI systems. 

Regarding potential conflicts, the European Data Protection Supervisor’s guidance on the position of a DPO in an organisation, clarifies that the position requires independence and impartiality to ensure objective oversight of data protection compliance. There must be no conflict of interest between the DPO’s responsibilities and any other duties they may hold. AI governance, particularly at the strategic level, often demands active involvement in AI system design and deployment decisions. This may create potential conflicts of interest if the DPO is deeply involved in operational AI strategies i.e., if they are responsible for both implementing data processing activities and ensuring compliance with data protection legislation.  For example, if a DPO helps design an AI system that processes personal data and also conducts a DPIA for that system, they will face a conflict between supporting the project and enforcing the rules. To address this, some organisations may consider appointing a separate AI Officer. 

If the DPO (and the broader DP function) is to take on AI governance responsibilities, the following should be considered: 

• Clearly define boundaries and responsibilities to preserve the DPO’s compliance oversight role while enabling operational leadership in AI governance. 
• Ensure the DPO is adequately supported with sufficient resources and technical expertise to effectively take on AI governance responsibilities 

In addition to clear roles and proper resourcing, continuous upskilling is essential for all data protection professionals involved in AI governance. The EU AI Act explicitly requires organisations to ensure AI literacy, emphasising the importance of understanding AI risks and governance frameworks. Training should be tailored depending on their backgrounds, where it is legal, technical, or otherwise, to ensure it is relevant and practical. Examples of training areas include, but are not limited to: 

• Assessing the DPO’s existing expertise (e.g. legal, technical, policy, HR) and provide background specific training. For DPOs without a technical background, training should focus on: 
• Understanding the basics of AI and machine learning, 
• Recognising and managing algorithmic risks such as bias, discrimination, algorithmic opacity, etc and end-to-end AI lifecycle. 
• Developing AI policies and familiarising with AI governance frameworks, like the NIST AI Risk Management Framework. 
• Keeping up to date with evolving AI systems and global AI regulations. 

By building these skills, DPOs are better equipped to oversee AI governance throughout the AI lifecycle, ensuring regulatory compliance and ethical accountability. Even if a DPO is not directly responsible for AI governance, it is essential they upskill to understand and manage associated risks. Without proper knowledge, organisations risk overlooking critical issues such as bias, discrimination, and lack of transparency - potentially causing reputational damage and loss of stakeholder trust. Moreover, the EU AI Act imposes significant fines: up to €35 million or 7% of global turnover for major violations, €15 million or 3% for other non-compliances, and €7.5 million or 1.5% for providing false or incomplete information highlighting the critical need for strong and effective AI practices. 

Conclusion 

In conclusion, involving the DP function in AI governance is a practical approach due to the natural overlap in their responsibilities. By building on existing tools, frameworks, and established trust with stakeholders, they can effectively strengthen AI oversight. However, clear role definitions are crucial - particularly for DPOs, who must balance maintaining independence with addressing strategic AI challenges. How responsibilities are allocated will vary based on an organisation’s size and its role under the AI Act, especially for those developing and deploying AI systems that require specialised expertise. To ensure success, organisations should not only define roles clearly but also allocate sufficient resources and prioritise tailored AI literacy and upskilling for the DP functions. 

At Trilateral Research, our AI, Data Protection, and Cybersecurity (DCS) teams bring deep expertise in helping organisations navigate the complex compliance landscapes of Data Protection and Responsible AI Governance. We offer a comprehensive range of AI services, including tailored literacy and training, risk management, policy development, and overall, AI governance support. 

Feel free to contact our advisors to discuss your specific needs and explore how to elevate your data protection and AI strategy. Learn more about our Responsible AI Services here. We look forward to partnering with you to build trust, transparency, and resilience in your digital operations.