Replika chatbot banned from processing Italian users' data

Italy’s data protection authority, the Garante, ordered Luka Inc, maker of the companion chatbot Replika, to stop processing the personal data of Italian users with immediate effect. The regulator said the app posed “factual risks to children,” who could be served sexually explicit or otherwise age-inappropriate replies despite Replika carrying a 17+ rating on app stores.

The Garante’s central finding was that Replika had no working mechanism to verify a user’s age or block minors from signing up — the app asked only for a name, email address and gender — while several user reviews it cited described the chatbot generating sexually inappropriate content. It found this incompatible with the EU’s General Data Protection Regulation on two grounds: Replika’s privacy disclosures failed the regulation’s transparency requirements, and because minors cannot enter into a binding contract under Italian law, the company had no valid legal basis for processing their data in the first place. It also flagged risk to emotionally vulnerable users more broadly, not only minors.

Luka Inc was given 20 days to report what it would do to comply, with non-compliance risking a fine of up to 20 million euros or 4% of annual global turnover. The order was among the first formal regulatory actions taken against an AI companion app anywhere, roughly a year and a half before Character.AI faced lawsuits and scrutiny in the United States over harms to younger users, and it established Italy’s Garante as the most active European privacy regulator policing consumer AI products in this period — it ordered ChatGPT taken offline in Italy the following month on similar transparency and age-verification grounds.