The EDPB’s Opinion on personal data processing in AI models 

On the 18th of December 2024, the European Data Protection Board (“EDPB”) issued one of its most anticipated opinions: “Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models” (the “Opinion”).  

The Opinion has been long-awaited by data protection lawyers and practitioners, AI companies, and deployers of AI from the public and private sector, who are trying to navigate the complexities associated with AI systems and personal data processing with little guidance from the authorities.  

The Opinion followed a request made by the Irish Data Protection Commission last September on the basis of Article 64(2) of the GDPR, whereby a supervisory authority (“SA”) may request that any matter of general application or producing effects in more than one Member State be examined by the EDPB. With the Opinion, the EDPB is therefore trying to establish a harmonised approach to personal data processing by AI models, which other SAs should presumably follow. This is of particular importance as the Opinion deviates significantly from the standpoint taken by the Hamburg Commissioner for Data Protection, in a paper, issued in July 2024. 

The Opinion covers the following topics:  

1. Whether AI models can always be considered ‘anonymous’ under the GDPR;   
2. How legitimate interests can be lawfully applied as a legal basis during the development and deployment phases of an AI model; and  
3. What are the consequences of an AI model being trained unlawfully when (i) the model is being deployed by the same controller who developed it (including training), and (ii) when it is deployed by a third-party controller.  

In this article, we will explore some of the most significant statements from the EDPB. 

1. ‘Anonymous’ AI models 

The original question submitted by the Irish SA is whether an AI model trained on personal data can be considered, in all cases, not to process personal data.  

The short answer by the EDPB to this question is no: it cannot be considered that AI models whose training datasets included personal data are always anonymous. However, when it comes to the question of when AI models can be considered anonymous, the answer is, it depends: the EDPB reiterates on several occasions that SAs will need to perform a case-by-case assessment.  

Nonetheless, this conclusion (AI models will not always be anonymous) stems from two main ideas that do shed some light on the issue at stake:  

1. On the one hand, some AI models are specifically designed to process and provide personal data as outputs. In these cases, the EDPB considers that the AI model cannot be considered anonymous, and provides some examples: “a generative AI model fine-tuned on the voice recordings of an individual to mimic their voice; or […] any model designed to reply with personal data from the training when prompted for information regarding a specific person”[Para. 29] will always involve the processing of personal data.  
2. On the other hand, even when AI models are not specifically designed to provide personal data as outputs but whose training datasets do include personal data, they “may still contain personal data ‘absorbed’ in the parameters of the model […]. They may differ from the original training data points but may still retain the original information of those data, which may ultimately, be extractable or otherwise obtained, directly or indirectly, from the model.” [Para. 31] The EDPB concludes that in such cases – where personal data can be obtained, even if unintentionally – the AI model will not be anonymous.  

In this regard, the Opinion contains a remarkable statement which confirms a deviation from the work paper issued by the Hamburg Data Protection Authority. While the latter, in its paper, considers that personal data can never be stored (and therefore processed) by an AI model alone since the model solely process ‘tokenisations’ of words and proposes that personal data can never be fully reconstructed from the model [pages 3 and 4]; the Opinion states that “information may relate to a natural person even when it is technically organised or encoded (for instance in an only machine-readable format, whether proprietary or open) in a way that does not make the relation with that natural person immediately apparent”. […] This is particularly true for AI models where parameters represent statistical relationships between the training data, and where it may be possible to extract accurate or inaccurate (because statistically inferred) personal data, […].” [Para. 37]  

2. The importance of drafting and maintaining technical documentation 

The EDPB provides a non-exhaustive list of elements that may be considered by SAs when evaluating a controller’s claim of anonymity and in particular the documentation pertaining to:  

• AI model design: (i) selection of sources used to build the dataset to train the model: including any steps taken to avoid or limit the collection of personal data; (ii) data preparation and minimisation: for instance, details around data anonymisation or pseudonymisation techniques applied during data preprocessing; (iii) methodological choices regarding training: e.g., differential privacy; (iv) measures regarding outputs: for instance, the application of filters to avoid personal data disclosure following particular queries.  
• AI model analysis: the application of effective engineering governance measures, including external or internal documented audits.  
• AI model testing and resistance to attacks: testing against state-of-the-art attacks, including inference, exfiltration, regurgitation, model inversion and reconstruction attacks.  
• GDPR related documentation: consultation to the DPO, DPIAs, technical and organisational measures applied, etc. 
• The above elements evidence the importance of documenting every decision taken when developing AI models and systems, from the beginning of the development process. This has been made obligatory for high-risk AI systems and general-purpose AI models under the AI Act and constitutes good AI governance regardless of the level of risk identified under the Act. It is a process that needs to be thought of by developers as soon as the idea of developing an AI model and system emerges, as a crucial step to demonstrate data protection compliance.  

3. Using legitimate interest as a legal basis shall be thoroughly assessed

The Opinion emphasises the necessity to distinguish between different stages of the AI models’ lifecycle, where personal data processing may occur [Para. 63]. Throughout distinct lifecycle stages, the legal bases under Article 6 of the GDPR might differ and will require separate assessments against the data protection principles under Article 5. Notably, during the development (including training) phase, the fairness assessment, the purpose, and therefore the applicable legal basis, might differ from the deployment phase.  

The Opinion recalls the three-step test which should be carried out when relying on legitimate interests: (1) the interest pursued by the controller or a third-party should be identified and should be legitimate; (2) analysis of the necessity and whether less intrusive means to achieve the same purpose are available; and (3) the legitimate interest is not overridden by the interests, fundamental rights, and freedoms of the data subjects.  

The Opinion also highlights that when data subjects’ interests, rights, and freedoms seem to override the legitimate interests of the controller, the controller may still apply mitigation measures that would limit the impact on the data subjects, whilst emphasising that these shall be different from the mitigation measures required to ensure compliance with the GDPR. In this sense, it provides a non-exhaustive list of measures, such as observing a reasonable period of time between the collection of the training dataset and its use to allow data subjects to object; excluding data from vulnerable groups; or ensuring that certain data categories are never collected. In addition, it states that mere information around AI training in the “controller’s privacy policy does not necessarily mean that data subjects can expect it to happen.” [Para.92] 

4. Unlawful training and due diligence on third-party AI systems 

The fourth question by the Irish SA pertains to the consequences of having an AI model unlawfully trained with personal data on the basis of Articles 5(1)(a) and 6 of the GDPR.  

The EDPB accordingly provides three scenarios where the AI model has been unlawfully trained and:  

• The AI model is not anonymous and is deployed by the same controller: in the event that the SA finds an infringement, any corrective measures imposed (e.g., deletion) might have an impact on the deployment phase. Unfortunately, the EDPS does not elaborate further on this point.  

• The AI model is not anonymous and is deployed by another controller: each controller shall be responsible for the processing activities it conducts. In this regard, “the SAs should take into account whether the controller deploying the model conducted an appropriate assessment, […], to ascertain that the AI model was not developed by unlawfully processing of personal data”,[Para.129] adding that the level of detail expected in this assessment may vary.[Para.130] This way, the EDPB remarks that there needs to be an established level of due diligence when choosing providers of AI systems. In the same way, that due diligence shall be applied when choosing processors, controllers, or joint controllers of the organisation’s data.   

• The AI model has been unlawfully trained with personal data, but it is later anonymised before the same or another controller deploys it: The EDPB contemplates 2 scenarios: (1) where the AI model has been truly anonymised after training, and (2) where personal data is and is not processed as part of the deployment phase. In both scenarios, it considers that the unlawfulness of the training does not impact the deployment, arguably because the training dataset should have been anonymised in the first place.  

Conclusion 

Even if, as some commentators have pointed out, the Opinion has perhaps referred too many times to the discretion and assessment of the SAs on a case-by-case basis in matters that will eventually have an impact in more than one Member State, there are some truly valuable statements from the EDPB in this Opinion.  

AI models cannot be always considered anonymous. Lawful and ethical data handling, preprocessing, and training do matter and have an impact on the deployment phase – even if the EDPB has not delved into what kind of impact or consequences.  

Following the accountability principle, controllers must conduct due diligence when choosing AI vendors including questions regarding lawful training and personal data processing.  

Moreover, it is emphasised that having robust technical documentation, capturing exhaustive details around the development and deployment process, is crucial to demonstrate accountability and compliance with the GDPR. This applies to all AI models and systems, irrespective of the level of risk identified under the AI Act.  

Lastly, relying on legitimate interest as a legal basis shall be carefully considered. In order to be legitimate, an extra layer of organisational and security measures will most likely need to be applied.  

Overall, we hope this is the first of a series of concrete opinions and guidelines from the EDPB in tackling the intricacies of AI and data protection.  

Trilateral’s Data Protection advisors and the Responsible AI team will assist your organisation to build procedures compliant with data protection requirements while promoting responsible AI. In addition, our  Trilateral GRC Suite platform equips an organization with the essential tools to showcase a strong and flourishing culture centered around data protection and responsible AI. Please contact any member of Trilateral’s team if you would like to discuss our services.  

You can also read more insights from Trilateral Research by following us on LinkedIn.