The sweet voices of robots – cloning voices with AI

March 2025  |  SPECIAL REPORT: DATA PRIVACY & CYBER SECURITY

Financier Worldwide Magazine

March 2025 Issue

As Lord Byron wrote in Don Juan: “The devil hath not, in all his quiver’s choice, An arrow for the heart like a sweet voice”. A truth well-remembered even 187 years after its publication, when OpenAI reportedly approached Scarlett Johansson to lend her voice to ChatGPT.

Despite not reaching a deal with the actress, OpenAI soon released ‘Sky’ with one of its voices sounding, to many, eerily like Ms Johansson’s. While this voice is no longer available, the incident highlights the trust-building qualities of voices, especially celebrities’.

It is no wonder, then, that the copying of voices has become one of the more commercially promising use cases for generative artificial intelligence (AI). Beyond monetisation of novelty videos on social media (e.g., a buff Harry Potter/Daniel Radcliffe giving motivational speeches in a gym) and scams (e.g., a fake Tom Hanks advertising dental plans), the technical ability to mimic voices creates many commercial possibilities.

Instead of celebrities intoning marketing messages in person, such endorsements could be produced by an AI clone, saving both the celebrity’s time and (some of) the advertiser’s money. With voice assistants in anything from AI chatbots to navigation systems, there is a broad market for voice products which do not require the nuance customers may expect in art.

That said, the development and use of artificial voices are not without legal complications. With the European Union’s (EU’s) AI Act coming into effect partially in February 2025 and further in August 2025 and 2026, the legal landscape will become even more complex. Companies looking to capitalise on the commercial opportunities presented by this technology should be mindful of the legal challenges they will face.

Privacy law

Almost all methods of voice cloning, relying on machine learning, require data to be trained on. This includes voice recordings, which are analysed to identify various pieces of information about a person’s voice and speech, including the so-called zero crossing rate, spectrograms (with data on spectral centre, roll-off, bandwidth, flow and density) and chroma.

In many jurisdictions, and certainly the EU, this information qualifies as speakers’ ‘personal data’, the processing of which requires a legal basis and triggers organisational obligations.

Anonymisation is often at odds with the business model, as (pseudo-)recognition of the voice is an explicit part of the value proposition. But even where it is not (e.g., public transit announcements or news reading), many EU data protection authorities’ standards for anonymisation may make it legally unfeasible. Like a person’s face, a voice may be regarded as a direct identifier, which is related to the speaker even if the listener has no idea of the speaker’s name, address or other circumstances.

Thankfully, privacy law generally allows for the processing of personal data that is necessary to perform a contract with the person to whom the data relates. Like using actors’ data for creating movies or promotional material is traditionally justified on such a contractual basis, this is possible for commercial voice cloning.

At least under the General Data Protection Regulation (GDPR), such processing is typically not subject to the restrictions regarding “biometric data for the purpose of uniquely identifying a natural person”, as this is not the purpose. However, companies should nevertheless review whether their individual use case is subject to such restrictions, which also exist in many non-EU jurisdictions.

Moreover, in order to provide a specific legal basis, care should be taken to contractually specify the use of the voice as far as possible given the business model.

In any case, companies should consider carefully whether to obtain the speaker’s consent. While this will be required in some countries, consent is conceptualised differently across jurisdictions. Under the GDPR in particular, the revocability of consent and the strict requirements on ‘bundling’ render it less than ideal as a legal basis.

Finally, while this article cannot go into detail regarding the organisational obligations imposed by privacy law, this does not mean that these obligations can be ignored. On the contrary, enforcement often directly targets the lack of required documentation, which is easier to prove than material violations, and can still result in fines up to 4 percent of a corporate group’s annual turnover.

AI Act

While the AI Act is an EU regulation, non-EU companies should be aware that it applies extraterritorially whenever AI systems are placed on the EU market and even when the output produced by the AI system is used in the EU (irrespective of where the company is established). In the present case, this means that any use of cloned voices in the EU will trigger its territorial applicability.

The AI Act aims to protect against the harmful effects of AI through a multifaceted approach. The first aspect of this approach, which takes effect in February 2025, is unlikely to impact any reputable company’s voice cloning projects. However, companies should be aware that their voice cloning systems must not be used to manipulate individuals into significantly harmful behaviour, exploit vulnerable people to materially distort their behaviour with likely significantly harmful effects, or serve other particularly questionable purposes (refer to the full list in article 5(1) of the AI Act).

On its own, a voice cloning system is also unlikely to qualify as a high-risk AI system. However, voice cloning systems are typically subject to transparency requirements. When voice clones are used to interact directly with individuals (e.g., in conjunction with a chatbot), providers must inform these individuals that they are interacting with an AI, unless it is already obvious.

Additionally, deployers of realistic voice clones, which often qualify as ‘deepfakes’, must disclose the artificial nature of the content no later than the first interaction or exposure. This disclosure needs to be managed carefully to avoid diminishing the intended effect of the voice clone. In areas that are ‘evidently’ artistic, creative, satirical, fictional or analogous, appropriate limitations on disclosure are permitted to ensure that the display or enjoyment of the content is not hampered.

IP protection and personality rights

Another key legal aspect of voice cloning involves intellectual property (IP) and personal rights, which are central to all stages of developing and using a voice clone, starting with its training on original recordings. Typically, the speaker holds the copyright to these recordings, so using them requires a grant of rights.

The contract should specify the permitted uses as precisely as possible. At a minimum, these uses should include retrieval, reproduction, storage, editing, segmentation, analysis and input into the AI model. While an exclusive licence might be the most lucrative for the AI tool provider, celebrities are likely to agree only to a non-exclusive licence.

However, the voice clone itself and the content it generates are not subject to the speaker’s copyright, which protects the recordings only in their specific form and any recognisable modifications. This does not mean that the voice clone and associated content are free of the speaker’s rights.

The speaker’s voice, inherent not only in recordings but also in the voice clone and the generated works, is protected as part of the general right of personality. Therefore, the speaker’s agreement is required for AI training with existing recordings, for the storage, use and editing of the voice clone, and for the generation, editing, use and performance of the generated content.

Similar to consent in data protection law, there is a concern – at least in legal systems that do not clearly distinguish between the economic and intangible parts of the right of personality – about preventing a ‘sell-out’ of personality to such an extent that the applicable law deems the consent invalid. In this context, very far-reaching and irrevocable consent is likely to be problematic.

To make the agreement ‘bulletproof’, the scope of the agreement should be defined as precisely and narrowly as possible in light of the business model, for example by agreeing on adjustable ‘blacklists’ of prohibited uses and end customers.

Finally, all rights and agreements (under data protection, IP and personality rights) should generally be limited in time to prevent invalidity due to unfairness.

Additional considerations

In addition to the abovementioned regulations, a voice cloning provider may also have to comply with cyber security requirements under the EU’s Network and Information Security 2 Directive when it qualifies as a cloud computing provider. Furthermore, a voice cloning tool could be classified as a ‘product with digital elements’, making it subject to the EU Cyber Resilience Act’s product safety obligations starting in September 2026.

Conclusion

Overall, considering the intimate relationships people have with voices, both their own and others’, the legal challenges for the commercial use of voice clones are manageable. However, it requires careful legal manoeuvring to minimise liability risks and avoid scrutiny from supervisory authorities, both now and in the future.

Managing these risks will become even more relevant once not only voices are cloned, but whole likenesses. As shown by the performance of Peter Cushing’s ‘clone’ in ‘Rogue One: A Star Wars Story’, the appetite is there to digitally recreate what (or rather, who) is no longer physically available (although Mr Cushing’s clone was not created by AI, but traditional – if highly advanced – VFX).

That is not even considering the virtual clone army that would be necessary to populate the metaverse. With the seemingly inexhaustible investment into these technologies, the issues presented here will only become more important.

 

Alexander Molle is a partner and Simon Clemens Wegmann is an associated partner at Gleiss Lutz. Mr Molle can be contacted on +49 30 800 979 161 or by email: alexander.molle@gleisslutz.com. Mr Wegmann can be contacted on +49 30 800 979 213 or by email: simon.wegmann@gleisslutz.com.

© Financier Worldwide

BY

Alexander Molle and Simon Clemens Wegmann

Gleiss Lutz

©2001-2025 Financier Worldwide Ltd. All rights reserved. Any statements expressed on this website are understood to be general opinions and should not be relied upon as legal, financial or any other form of professional advice. Opinions expressed do not necessarily represent the views of the authors’ current or previous employers, or clients. The publisher, authors and authors' firms are not responsible for any loss third parties may suffer in connection with information or materials presented on this website, or use of any such information or materials by any third parties.