An algorithm is a set of instructions for solving a problem or accomplishing a task. Common examples of algorithms are a set of directions to get to a particular destination, instructions to build a toy and a recipe to make a biryani. Consequently, computers are only as good as the algorithms they are given. In this article, we consider how poorly implemented algorithms may result in breaches of the General Data Protection Regulation (GDPR).
The Deliveroo Italy s.r.l and Foodinho cases
Deliveroo Italy s.r.l (Deliveroo) and Foodinho are providers of on-demand food delivery services in Italy. Each provides their own mobile application and website to enable their respective users to place food orders for delivery. Both companies implemented their own algorithmic systems to evaluate the riders making their respective deliveries in a manner which the Italian data protection authority, the Garante per la protezione dei dati personali (GPDP), found to contravene multiple provisions of the GDPR.
Neither Deliveroo nor Foodinho adequately informed the riders in regard to the functioning of the relevant algorithmic systems, contrary to the lawfulness, fairness and transparency requirement under Article 5(1)(a) and the right to be informed under Article 13, of the GDPR. Each company failed to guarantee the correctness of the results of their algorithmic systems as well, contrary to the accuracy requirement under Article 5(1)(d) of the GDPR and which could lead to the limitation of deliveries assigned to each rider or exclusion of some riders from the platforms themselves. Both companies also did not ensure that there were robust procedures to protect the right of the riders under Article 22 of the GDPR to obtain human intervention in respect of the decisions based solely upon automated processing, including in relation to the aforementioned limitations and / or exclusions.
In addition, Deliveroo and Foodinho did not comply with the requirements for data minimisation under Article 5(1)(c), data protection by design and default under Article 25, record of processing activity (ROPA) under Article 30, security of processing under Article 32 and data protection impact assessment (DPIA) under Article 35, of the GDPR.
On 5 July 2021, the GPDP announced that it would fine Foodinho €2.6 million. On 2 August 2021, the GPDP declared that it would also fine Deliveroo €2.5 million. The GPDP highlighted the considerable number of riders involved in both instances – 19,000 for Foodinho and 8,000 for Deliveroo.
The GPDP ordered Deliveroo and Foodinho to take several of the following corrective measures:
- to identify measures to protect the rights and freedoms of riders in the face of automated decisions;
- to prepare documents containing relevant privacy information, ROPA and DPIA;
- to verify the accuracy and relevance of the data used by and / or results of the algorithmic systems (including chats, emails and phone calls between riders and customer care, geolocation, estimated and actual delivery times, details about the management of the order in progress and those already made, feedback from customers and partners, the remaining battery level of the devices used by the riders, etc.);
- to identify measures that prevent improper or discriminatory use of reputational mechanisms based on feedback from customers and business partners;
- to identify the individuals authorised to access the relevant systems, the purposes for which necessitate such access and to adopt measures to verify access; and
- to identify retention periods for the relevant data.
The GPDP gave each company 60 days to initiate the necessary measures to correct “serious violations” and an additional 90 days to complete “interventions” on the algorithms.
The GPDP further outlined that it had initiated, for the first time, a joint operation with the Spanish data protection authority, the Agencia Española de Protección de Datos (AEPD), under the GDPR, in order to investigate Foodinho’s Spanish parent company, GlovoApp23.
Recommendations
In light of the above, organisations should ensure that they:
- adopt a data protection by design and default approach;
- conduct a DPIA in respect of the implementation of algorithms which entail systematic and extensive evaluation and upon which decisions are based that produce legal or significant effects concerning the relevant data subjects;
- provide adequate privacy information in respect of such algorithms; and
- maintain the accuracy of the relevant personal data.
Trilateral’s Data Protection and Cyber-Risk Team has significant experience supporting organisations in conducting DPIAs and implementing privacy information. For more information please feel free to contact our advisers, who would be more than happy to help.
