AI voice cloning is an advanced technology that employs machine learning algorithms to replicate individual voices with remarkable accuracy.

By analyzing a brief sample—sometimes as short as 3 to 15 seconds—these systems can learn the distinct tone, accent, and inflection of a speaker, allowing them to generate speech that sounds convincingly authentic, even in languages not present in the original sample.

Recent advancements, exemplified by tools like OpenAI’s Voice Engine and ElevenLabs, have further improved the technology’s ability to retain emotional nuance and personality in synthetic voices, making outputs suitable for both creative and practical uses .

Development of AI voice cloning relies on both proprietary and open-source models trained on vast quantities of speech data.

Innovations like the Tortoise model have enabled machines to learn not just pronunciation but also the expressive qualities of speech, prompting industry players to formalize consent and licensing agreements for using real voices .

The technology’s versatility has led to a variety of applications, including creating virtual assistants, generating localized or accessible audio learning materials, producing entertainment content, and—on the negative side—facilitating scams through synthetic impersonations .

Real-world uses range widely.

Educational institutions are employing cloned voices to localize language content, and technology companies are using them to increase accessibility for speakers of underrepresented languages.

However, the same sophistication has enabled malicious actors to perpetrate scams and extortion schemes by convincingly mimicking family members or authority figures, sometimes using as little as three seconds of audio .

Notably, controversy arose when OpenAI released a chatbot voice resembling actress Scarlett Johansson, raising concerns about voice likeness and consent .

Legal responses are emerging to address misuse.

AI voice cloning is an advanced technology that employs machine learning algorithms to replicate individual voices with remarkable accuracy.

Title:

Content: Language No Barrier: New AI tool can clone voice and make it say anything in multiple languages

Imagine former US President Donald Trump speaking in flawless Hindi. A new AI voice cloning tool by ElevenLabs can clone voices and make them say anything in a bunch of different languages. All it needs is a three-second sample clip and a bit of text as the prompt.

Mehul Reuben Das May 03, 2023 13:42:20 IST

Imagine former US President Donald Trump speaking in flawless Hindi. A new AI voice cloning tool by ElevenLabs can clone voices and make them say anything in a bunch of different languages. All it needs is a three-second sample clip and a bit of text as the prompt.

As advanced as AI-based voice cloning has been, there were a few limitations to it. For example, voice generators were mainly bound by the language of the source they were trained in or rather the language of the prompt that was given to them. Now, ElevenLabs, an American AI software developing company has come up with a tool that does away with that limitation.

ElevenLabs’ new AI tool can clone the voice of a person and make it say anything in a bunch of different languages. All it needs is a clear voice sample of about 3 seconds, although it also suggests that the longer the sample, the better will be the results. The new tool even manages to maintain the speaker’s original tone of voice.

Limitless possibilities
The creators expect that this would ‘extend the possibilities’ in a variety of industries such as publishing, gaming creation, and media. “Introducing Eleven Multilingual v1: Our New Speech Synthesis Model! We’re thrilled to launch our new model supporting seven new languages: French, German, Hindi, Italian, Polish, Portuguese, and Spanish.”

“This advancement expands the horizons for creators, game developers, and publishers, and it paves the way for using generative media to create more localized, accessible, and imaginative content,” the company said.

Following the announcement, Twitter was flooded with comments, with users testing the software on various famous faces.

To demonstrate just how capable the new voice cloning AI tool is, Eluna AI, a platform for AI-content creators experimented with a sample clip of former US President Donald Trump.

In their demo, Eluna AI showcased a small clip of Donald Trump, speaking in his native tongue, English, followed by French, Spanish, German, Hindi and Italian, almost perfectly. The generated samples even gets the accent right, almost if he was a native speaker of the new target language.

Another user changed David Attenborough’s voice to German while the BBC broadcaster discussed the prospects of AI. “With this tool, anyone can do it,” stated the Twitter user. “It’s also worth noting that fraudsters will undoubtedly exploit this one.”

ElevenLabs says that its model was designed to understand the emotion and reasoning underlying words in order to produce natural-sounding and persuasive audio. ElevenLabs has said that some numerals and symbols are currently spoken wrongly, but that it is striving to rectify this.

Who is this new tool meant to help?
The US-based company believes its technology will be valuable in a variety of areas, including education and journalism.

Game developers and publishers can build engaging, localised experiences for worldwide audiences, breaking through language barriers and connecting with players and listeners to maximise engagement and efficiency while maintaining quality and accuracy.

“Educational institutions now have the ability to develop audio content for varied users in their target languages, enhancing language comprehension and even pronunciation abilities, as well as responding to different teaching methods and learning demands,” says the report.

Because of the model’s capacity to produce unique character-like voices, audiobook and content developers may also find it beneficial.

The conundrum of AI voice genertors
Despite its benefits, ElevenLabs’ launch comes amid widespread concern about the wave of new voice cloning technology. An unidentified TikTok user went popular last month after sharing an AI-generated Drake song starring The Weeknd.

It sparked concerns about the grey area of AI-created music and whether it constitutes a copyright violation. Nonetheless, ElevenLabs claims to be ‘fully committed’ to respecting creators’ rights in the midst of technological advancements.

“At Eleven, we think that we should seek to make the best use of emerging technology, but not at any expense,” it stated.

“As we create them, we make every effort to put suitable controls in place that reduce the possibility of detrimental abuse. With this in mind, we are entirely committed to both protecting intellectual property rights and taking action against infringement.”

Read all the Latest NewsTrending NewsCricket NewsBollywood News,
India News and Entertainment News here. Follow us on FacebookTwitter and Instagram.

Updated Date:

also read

World

Music labels are worried because of AI, thanks to a surprising new song ft. Drake X The Weeknd

Someone used AI to create a whole new song featuring Drake and The Weeknd. But because the singers never sang the song, artists and music labels are worried, how AI voice modulation and AI-generated songs can disrupt the industry.

World

Artificial Imposter: 47% of Indian phone users have experienced AI voice scams, highest in the world

About half of Indian adults, or 47%, have experienced AI voice scams, nearly double the global average of 25%. Furthermore, nearly 69% of adults say they can't differentiate between a real and an AI-generated voice.

By analyzing a brief sample—sometimes as short as 3 to 15 seconds—these systems can learn the distinct tone, accent, and inflection of a speaker, allowing them to generate speech that sounds convincingly authentic, even in languages not present in the original sample.

Title:

Content: You can copy your voice in 15 seconds!
OpenAI Announces New Voice Engine Tool
▲ This technology is at risk of abuse. OpenAI is only available to partners at this stage and is not yet open to the public. (Graph / CFP)
Journalist Chan Ya-ting / General Report
American artificial intelligence development company OpenAI on Friday (29) revealed that the voice AI tool "Voice Engine", just 15 seconds of speaking sound samples, can copy the voice of the person, and the result is not limited by the sound sample language. If you speak English, this technology can use your voice to generate results in Spanish, French and other languages.
However, due to the risk of abuse, OpenAI is only available to partners at this stage and is not yet available to the public.
The New York Times and the Associated Press reported that OpenAI entered the field of speech technology, sharing test results after applying for the trademark Voice Engine, but considering the possibility of synthetic speech abuse, especially worried that the technology could be used to destroy voice authenticators such as online bank accounts, OpenAI is now taking a cautious attitude towards wider release.
OpenAI pointed out that there are significant risks in producing voice-like speech, especially in an election year when Voice Engine testing partners have agreed to regulations such as disclosing to audiences that sound is generated by AI and that the original voice must have explicit and informed consent.
In addition, OpenAI said that the company has implemented a number of security measures, including the use of digital watermarks to track the source of all the audio generated by the Voice Engine and actively monitor its usage.
Share with friends:
※ This article is copyrighted and cannot be reproduced without permission. [ETtoday Copyright Notice]
Related News
Hot News
The best video
And more.
Hot News
Back to the top.

Deepdub, a film dubbing company, uses an AI to process voice and translate into a different language and promises to preserve what?

What has the US Federal Trade Commission warned about the use of voice cloning tools?

Difficult to distinguish from human voice

What is necessary for voice cloning to work?

Recent advancements, exemplified by tools like OpenAI’s Voice Engine and ElevenLabs, have further improved the technology’s ability to retain emotional nuance and personality in synthetic voices, making outputs suitable for both creative and practical uses .

Title:

Content: There were 107 press releases posted in the last 24 hours and 446,378 in the last 365 days.
VoiceAI, technology for local dialects, transforming global customer service experiences
NeuralSpace launches AI voices in local languages
NeuralSpace logo
NeuralSpace's VoiceAI platform
LONDON, UNITED KINGDOM, December 10, 2023 / EINPresswire.com / --
NeuralSpace, a leader in linguistic artificial intelligence, has launched a transformative voice technology feature within its VoiceAI platform, offering realistic artificial sounds in Arabic, Hindi and English.
The innovation, unveiled on December 11, marks a major development in sound technology, offering speech formulation with unparalleled precision, especially in local dialects such as Saudi Arabia.
Three main advantages characterize NeuralSpace ’ s offerings:
Cultural Interaction: Artificial sounds capture the essence of local dialects, fostering deep communication between users and technology in regions such as Saudi Arabia, India and the English-speaking world.
Instantaneous Interaction: VoiceAI technology provides instantaneous, natural-sounding voice feedback, which is essential for dynamic speech generation in virtual assistants and voice response systems.
Ease of integration: The technology is sophisticated but easy to use, allowing developers to effortlessly optimize their applications.
VoiceAI for NeuralSpace meets the growing demand for accurate transcription, speech analytics and authentic synthetic sounds.
These AI sounds are finely tuned to real-world interactions, providing engaging and realistic user experiences.
Felix Le Mans, CEO and co-founder of NeuralSpace, said:
“NeuralSpace was founded with a vision to make the technology globally available in any language.
Today, with the release of our natural AI sounds, we are one step closer to turning this dream into reality.
Our human-quality Saudi, Indian and English voices in AI are not just revolutionizing technological innovation; they are gateways to make dialectical interactions with technology a tangible experience for everyone. ”
About VoiceAI: This platform and application programming interface (API) allow companies to copy, translate and analyze voice data with exceptional accuracy, especially in Arabic and Hindi.
Key features include up to 91% copy accuracy for these languages, surpassing major tech brands.
The platform also ensures high-level data security through in-house deployment, NeuralSpace ’ s ISO 27001 certification and compliance with the General Data Protection Law.
NeuralSpace invites you to integrate VoiceAI into your systems, and promises to change customer experiences.
The platform can be accessed via a code-free web platform, REST APIs, and Python SDK, ensuring easy integration.
About NeuralSpace: At the forefront of linguistic artificial intelligence, NeuralSpace is committed to technologies that enhance human-machine communication.
VoiceAI embodies this vision, reflecting the diversity and richness of human languages.
For more information, demos or enquiries, visit www.neuralspace.ai or contact sales @ neuralspace.ai.
NeuralSpace ’ s latest innovation is more than just a technological advance; it ’ s a step towards a future where you not only understand technology but also speak with a voice that resonates with the cultural richness of human languages.
Melanie Swamy-Brown
NeuralSpace
melanie @ neuralspace.ai
Visit us on social media:
Twitter
LinkedIn
Youtube

Deepdub, a film dubbing company, uses an AI to process voice and translate into a different language and promises to preserve what?

Development of AI voice cloning relies on both proprietary and open-source models trained on vast quantities of speech data.

Title:

Content: OpenAI's Johansson gaffe pushes voice cloning into spotlight

Law enforcement warned that AI clones could be used to extort money from loved ones over the phone

By AFP

  • Follow us on

Scarlett Johansson voiced an AI character in the film "Her". — Reuters file

Published: Mon 27 May 2024, 7:51 PM

Last updated: Mon 27 May 2024, 7:52 PM

OpenAI was forced to apologise to actor Scarlett Johansson last week for using her voice -- or something very similar -- on its latest chatbot, throwing the spotlight on to voice-cloning tech.

Although OpenAI denied the voice they used was Johansson's, their case was not helped by CEO Sam Altman flagging the new model with a one-word message on social media -- "Her".

Johansson voiced an AI character in the film "Her", which Altman has previously said is his favourite film about the technology.

Right from the start, AI voice cloning has proved problematic.

Last year, British firm Elevenlabs went viral for all the wrong reasons when it released its voice-cloning software.

Internet pranksters immediately began pushing out deepfaked celebrities -- Harry Potter star Emma Watson was shown reading Hitler's Mein Kampf.

Law enforcement warned that AI clones could be used to extort money from loved ones over the phone.

The technology has developed rapidly in the past year becoming far more realistic and nuanced.

Danish entrepreneur Victor Riparbelli, CEO of British AI firm Synthesia, told AFP it was largely down to a program called Tortoise that was released two years ago.

The program's developers threw thousands of hours of voice data into their model in an unstructured way and discovered it not only learnt what to say but how to say it.

"That was a pretty big paradigm shift," Riparbelli said on the sidelines of last week's VivaTech conference in Paris.

Tortoise was an open source program and Elevenlabs was the first to go to market using it.

OpenAI uses similar systems though they do not release any details.

Much of the controversy around voice cloning has focused on concerns over people misusing the software.

But the claim against OpenAI is unusual because it is the company itself accused of playing fast and loose.

"It was very unfortunate that OpenAI did that -- really not very good," Katya Laine, CEO of TALKR.ai, told AFP at VivaTech.

"If they actually cloned her voice without her knowing then I think that's very very bad," said Riparbelli.

The two entrepreneurs are among hundreds harnessing AI voice programs for uses that they argue will make companies more efficient.

Laine's firm provides virtual voice assistants -- essentially AI customer service agents.

She said her firm's system could now resolve 25 to 30 percent of calls without any human involvement.

Synthesia specialises in video avatars, which Riparbelli said allowed any office worker to turn text or slides into a video performed by a realistic AI.

Both Riparbelli and Laine allow their clients to use their own avatars, off-the-shelf products or those supplied by the likes of OpenAI and Elevenlabs.

Riparbelli said Synthesia used actors whose likenesses and voices were licensed for two years with an option to renew after the initial period.

The problems arise if actors' voices are used without their consent.

The fiasco overshadowed a developer conference in Paris last week when OpenAI was showing off a suite of new tools.

In front of a big screen in an auditorium, Romain Huet, OpenAI's Head of Developer Experience, breezily chatted into his phone.

Seconds later, his short voice sample had been processed and could be heard commentating over a generated video -- in five languages.

The demonstration showed how quickly the field is moving, but the headlines had already been written.

The Washington Post asked in a newsletter "How dumb is OpenAI?", other commentators were suggesting wunderkind Altman was nothing more than a huckster.

Nonetheless, Riparbelli was open to OpenAI's argument that they had used another actor who just sounded liked Johansson.

"If it's not her but someone who sounds a lot like her... where do you draw that line," Riparbelli asked.

"If they're not allowed to use someone who sounds a lot like her, then it sets a very odd precedent."

More news from World

Innovations like the Tortoise model have enabled machines to learn not just pronunciation but also the expressive qualities of speech, prompting industry players to formalize consent and licensing agreements for using real voices .

Deepdub, a film dubbing company, uses an AI to process voice and translate into a different language and promises to preserve what?

The technology’s versatility has led to a variety of applications, including creating virtual assistants, generating localized or accessible audio learning materials, producing entertainment content, and—on the negative side—facilitating scams through synthetic impersonations .

Title:

Content: The scary side of artificial intelligence

The pros of AI outweigh the Cons or threats in our day-to-day lives, and there’s no magic pill to protect us from the scary threats of AI.

  • Exclusive

It's a free Content, simply login/signup to unlock

Get in-depth Industry Insights and Analysis through our “Exclusive” content, presented to you by our esteemed panel of writers, for free

By continuing, you agree to the Terms & Conditions and acknowledge our Privacy Policy. This same account can be used across all Economic Times B2B portals.

  • Updated On Oct 3, 2023 at 03:02 PM IST

Artificial intelligence (AI), as we all know, is the simulation of human intelligence leveraging computer systems and machines. AI programs can continuously learn and evolve with experience. In technical terms, AI systems render the output by churning and analyzing the data for correlations and patterns and help make future predictions using these patterns.

Advt
Today AI is being used extensively in all facets of our lives starting from lifestyle, e-commerce, home appliances, automobiles, navigation, transport, healthcare, agriculture, education, financial planning, chatbots, marketing, cyber security, virtual assistant, industrial automation and many more.

Clearly, we have and are leveraging the strengths of AI to the fullest to automate the routine tasks and make it auto-learn over a period. There’s no doubt that AI has revolutionized the lives of all – organizations, consumers etc. But as they say, “Everything comes at a price”. This analogy is quite true for AI as well, there have been a lot of threats that have emerged using AI and are being utilized by the Cyber criminals to dupe the ignorant users, cause mental stress and extort money.

continued below

Executive Workshop

I’m not going to delve into all of threats but would like to elucidate some of the real ones that have come to the surface recently:

Voice cloning or mimicry

As the name suggests, it uses machine learning algorithms to simulate anyone’s voice. It requires certain voice data to analyze and learn the specific patterns of any individual. There are certain advantages of cloning a voice that can be used in voice assistants, chatbots, navigation systems and other areas. Voice cloning has existed for a while but was restricted to experts who possess the technical knowledge to be able to use the same. But now AI Voice cloning tools are available on the open Internet that anyone can leverage with literally no technical knowledge or cost to clone anyone’s voice. All you need to do is just Google and it will pop up and get a bunch of both free and paid AI Voice cloning tools.

Advt
In fact, recently one of the global leaders in online protection published a report stating how AI is fueling a rise in online voice scams, with just three seconds of audio required to clone a person’s voice. This is extremely creepy. Just imagine, if a Cyber-criminal clones your voice and tricks your near ones by making them believe that they are talking to you, and request for money due to any distress reasons. In fact, these are not assumptions but real-life incidents that are happening across the world every minute.

Deepfakes AI

Deepfakes AI are being used to impersonate an individual by manipulating videos, images, audios, and other content available on the social media platforms using deep learning techniques. It mainly transforms existing content from one person to another. This is another area that has been exploited by the Cyber criminals in the recent past, and there are a few deepfake cases that have made waves on the Internet. One of the most famous ones being Deepfake created by a VFX artist of one of Hollywood’s most famous male actors. This piece was so well made by rendering thousands of pictures using artificial versions of the actor. There are similar Deepfakes incidents involving other global celebrities and leaders. The damage and misuse of the deepfakes videos could be extremely dangerous for the human society and can be used by terrorists to incite riots and create nuisance.

Creating virtual people

This is another area where AI is being used to create people that do not exist. There are a lot of websites that allow you to generate a random image by just specifying the age group, gender & ethnicity, and you can continue to refresh these images at the click of a button. Be mindful of the fact that these images pertain to people that do not exist in this world. On the similar lines, there’s an emerging trend of AI news anchors that possess a human-like appearance and are fed data that they can read using text-to-speech technology. Recently, there's been a surge of AI-Driven robotic new anchors in multiple language channels. While the main objective of this initiative is to provide royalty free or cost-effective human images for branding and other purposes. However, the overuse of these technologies causes Fear of missing out (FOMO), Feeling of inadequacy, dissatisfaction, and isolation may potentially lead to psychological and emotional vulnerabilities.

One of the Hollywood movies, ‘HER’ portrayed an introverted and lonely writer (Joaquin Phoenix) who purchases an AI virtual assistant personified through a female voice to assist him in his writing job. However, when he finds out about its ability to learn and adapt, he falls in love with it. The film delves into emotional vulnerability and the impact of AI on human relationships.

Needless to say, that the pros of AI outweigh the Cons or threats in our day-to-day lives, and there’s no magic pill to protect us from the scary threats of AI. However, there are certain preventive tips that one can follow to mitigate any such risks. Do spread awareness with your family and friends:

  • Set some code words with family, and friends to verify the identity during emergency circumstances
  • Optimize social media presence by limiting your connections with only known contacts and avoid connecting with any unknown ones
  • Avoid saying ‘Yes’ as a greeting while receiving unknown calls as it can be used to authorize credit card charges or other purposes
  • Continuously update privacy settings on social media to allow viewing by friends and family only
  • Avoid clicking on any suspicious links or calls received without proper due diligence
  • Do not share your passwords and other sensitive information with anyone
  • Verify the credentials of any online information before circulating it through various mediums
  • Avoid public Wi-Fi connections offered at coffee shops or airports, which requires to enter password
  • Closely monitor the time spent by younger generation on social media and online gaming
  • Finally, in case of any suspicion, double check and report it to the host site or concerned authorities
The author is the CIO of MOVIN

Disclaimer: The views expressed are solely of the author and ETCIO does not necessarily subscribe to it. ETCIO shall not be responsible for any damage caused to any person/organization directly or indirectly.

  • Published On Oct 3, 2023 at 03:02 PM IST

Most Read in Artificial Intelligence

Join the community of 2M+ industry professionals

Subscribe to our newsletter to get latest insights & analysis.

Download ETCIO App

  • Get Realtime updates
  • Save your favourite articles

Real-world uses range widely.

Educational institutions are employing cloned voices to localize language content, and technology companies are using them to increase accessibility for speakers of underrepresented languages.

How will AI Voice help educators with building courses?

However, the same sophistication has enabled malicious actors to perpetrate scams and extortion schemes by convincingly mimicking family members or authority figures, sometimes using as little as three seconds of audio .

Title:

Content: Moving at rapid speed, the Federal Communications Commission (FCC) has just announced its unanimous adoption of a new Declaratory Ruling finding that voice calls using artificial intelligence (AI)-generated voices fall under the Telephone Consumer Protection Act (TCPA).

The ruling takes effect immediately and gives state attorneys general powerful new tools to go after voice cloning scams. Under FCC rules, telemarketers that use robocalls subject to the TCPA are required to obtain prior express written consent from the consumer unless an exemption applies. The TCPA has always prohibited the use of both prerecorded and artificial voices but advances in AI-generated voices have prompted the FCC to specifically address their use.

In recent years, scammers and other parties have begun using AI to create fake and even "cloned" artificial voices, including those of celebrities, politicians, and even a call recipient's family member. In this election season, there has been increasing concern about the use of voice clones to engage in voter suppression schemes.

Concerned about the potential malevolent uses of these AI-generated voice calls, the FCC issued a Notice of Inquiry (NOI) in November on the topic of "Implications of Artificial Intelligence Technologies on Protecting Consumers from Unwanted Robocalls and Robotexts." In the NOI, the FCC asked questions regarding the use of "AI-influenced" technologies in robocalling. A coalition of 26 state attorneys general, as well as a telecommunication carrier trade group, strongly urged the FCC to take action to give the attorneys general clear authority to act against AI-generated robocall violators.

The FCC now has made clear that the use of voice cloning in telephone calls and texts falls within the FCC's statutory authority under the TCPA. A major intent of the ruling is to communicate that the TCPA "does not allow for any carve-out of technologies that purport to provide the equivalent of a live agent, thus preventing unscrupulous businesses from attempting to exploit any perceived ambiguity in our TCPA rules."

As FCC Chairwoman Jessica Rosenworcel explained, "with this Declaratory Ruling, we will have another tool to go after voice cloning scams and get this junk off the line."

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.

What is necessary for voice cloning to work?

Notably, controversy arose when OpenAI released a chatbot voice resembling actress Scarlett Johansson, raising concerns about voice likeness and consent .

Legal responses are emerging to address misuse.


AI voice cloning technology employs advanced machine learning algorithms to replicate the unique patterns, intonations, and emotional nuances of human speech.

By analyzing brief samples—sometimes only three to fifteen seconds—these systems can accurately generate highly realistic copies of an individual's voice.

This process has become increasingly refined, requiring minimal input data, and is now widely accessible through both free and commercial platforms, allowing even those without technical expertise to produce convincing synthetic voices .

The underlying technical development of voice cloning relies on vast datasets of human speech and sophisticated neural networks.

Systems like OpenAI’s voice engine, Tortoise, and products from ElevenLabs use thousands of hours of voice recordings to train models that learn not just the words being spoken, but also how they are articulated.

These models can mimic vocal accents, inflections, and emotions in multiple languages, creating speech that, to human listeners, is often indistinguishable from genuine recordings .

The uses for AI voice cloning are expanding rapidly across various industries.

In entertainment, cloned voices have been deployed to recreate performances by actors unable to work, such as using a synthetic James Earl Jones voice for Darth Vader.

The technology is also used in media dubbing and in video games to reproduce celebrity voices.

For accessibility, it enables personalized voice assistants, customer service bots, and audiobooks, and allows people who have lost their voices to communicate in their own or a synthesized likeness.

In education, voice cloning supports production of localized and leveled audio materials for language learning.

However, the same technology is also exploited for nefarious purposes.

High-profile scam cases have used synthetic voices to impersonate trusted individuals for fraudulent activities such as fake kidnappings and calls requesting money.

Deepfake audio content, including false statements attributed to public figures and celebrities, further exemplifies its potential for misuse and societal disruption .

The ethical implications of AI voice cloning are profound.

The unauthorized use and replication of a person’s voice raise critical concerns about privacy and consent.

There is serious risk of identity theft, extortion, and reputational damage when voices are cloned and used without the individual's permission.

The ability to fabricate convincing audio for scams, political manipulation, or the diffusion of misinformation erodes public trust in audio recordings and complicates efforts to verify authentic communication.

The ease and realism of voice cloning increase both the scale and subtlety of these risks, magnifying the potential for harm .

Legal responses to voice cloning are still evolving but are becoming increasingly robust.

In the United States, the Federal Communications Commission has banned robocalls utilizing AI-synthesized voices in response to their misuse in telecom scams.

The European Union’s “AI Act” includes guidelines specifically targeting the responsible development and use of artificial intelligence, encompassing synthetic voice technologies.

Additionally, several US states have enacted or proposed legislation aimed at restricting deepfake content, including voice clones.

These measures often seek to safeguard individuals from unauthorized use of their voices and increase requirements for content identification and transparency.

Concurrently, research and regulatory attention are being directed at developing detection tools and ensuring platforms provide methods to label or identify synthetic audio .

Nonetheless, the legal framework still faces notable limitations.

Many regulations lag behind the technical capabilities and varied applications of voice cloning, leaving gaps in protection around intellectual property, impersonation, and the commercial or political exploitation of an individual’s voice.

The law continues to adapt as new uses and abuses of this technology emerge, demanding ongoing vigilance from professionals considering its adoption .

In summary, AI voice cloning stands at the intersection of remarkable innovation and significant societal risk.

Its ability to generate authentic-sounding audio offers opportunities for entertainment, accessibility, and personalized content creation, but also enables new forms of deception and harm.

Awareness of the technical function, potential, ethical hazards, and evolving legal context is essential for anyone contemplating its use .

AI voice cloning technology employs advanced machine learning algorithms to replicate the unique patterns, intonations, and emotional nuances of human speech.

Title:

Content: Language No Barrier: New AI tool can clone voice and make it say anything in multiple languages

Imagine former US President Donald Trump speaking in flawless Hindi. A new AI voice cloning tool by ElevenLabs can clone voices and make them say anything in a bunch of different languages. All it needs is a three-second sample clip and a bit of text as the prompt.

Mehul Reuben Das May 03, 2023 13:42:20 IST

Imagine former US President Donald Trump speaking in flawless Hindi. A new AI voice cloning tool by ElevenLabs can clone voices and make them say anything in a bunch of different languages. All it needs is a three-second sample clip and a bit of text as the prompt.

As advanced as AI-based voice cloning has been, there were a few limitations to it. For example, voice generators were mainly bound by the language of the source they were trained in or rather the language of the prompt that was given to them. Now, ElevenLabs, an American AI software developing company has come up with a tool that does away with that limitation.

ElevenLabs’ new AI tool can clone the voice of a person and make it say anything in a bunch of different languages. All it needs is a clear voice sample of about 3 seconds, although it also suggests that the longer the sample, the better will be the results. The new tool even manages to maintain the speaker’s original tone of voice.

Limitless possibilities
The creators expect that this would ‘extend the possibilities’ in a variety of industries such as publishing, gaming creation, and media. “Introducing Eleven Multilingual v1: Our New Speech Synthesis Model! We’re thrilled to launch our new model supporting seven new languages: French, German, Hindi, Italian, Polish, Portuguese, and Spanish.”

“This advancement expands the horizons for creators, game developers, and publishers, and it paves the way for using generative media to create more localized, accessible, and imaginative content,” the company said.

Following the announcement, Twitter was flooded with comments, with users testing the software on various famous faces.

To demonstrate just how capable the new voice cloning AI tool is, Eluna AI, a platform for AI-content creators experimented with a sample clip of former US President Donald Trump.

In their demo, Eluna AI showcased a small clip of Donald Trump, speaking in his native tongue, English, followed by French, Spanish, German, Hindi and Italian, almost perfectly. The generated samples even gets the accent right, almost if he was a native speaker of the new target language.

Another user changed David Attenborough’s voice to German while the BBC broadcaster discussed the prospects of AI. “With this tool, anyone can do it,” stated the Twitter user. “It’s also worth noting that fraudsters will undoubtedly exploit this one.”

ElevenLabs says that its model was designed to understand the emotion and reasoning underlying words in order to produce natural-sounding and persuasive audio. ElevenLabs has said that some numerals and symbols are currently spoken wrongly, but that it is striving to rectify this.

Who is this new tool meant to help?
The US-based company believes its technology will be valuable in a variety of areas, including education and journalism.

Game developers and publishers can build engaging, localised experiences for worldwide audiences, breaking through language barriers and connecting with players and listeners to maximise engagement and efficiency while maintaining quality and accuracy.

“Educational institutions now have the ability to develop audio content for varied users in their target languages, enhancing language comprehension and even pronunciation abilities, as well as responding to different teaching methods and learning demands,” says the report.

Because of the model’s capacity to produce unique character-like voices, audiobook and content developers may also find it beneficial.

The conundrum of AI voice genertors
Despite its benefits, ElevenLabs’ launch comes amid widespread concern about the wave of new voice cloning technology. An unidentified TikTok user went popular last month after sharing an AI-generated Drake song starring The Weeknd.

It sparked concerns about the grey area of AI-created music and whether it constitutes a copyright violation. Nonetheless, ElevenLabs claims to be ‘fully committed’ to respecting creators’ rights in the midst of technological advancements.

“At Eleven, we think that we should seek to make the best use of emerging technology, but not at any expense,” it stated.

“As we create them, we make every effort to put suitable controls in place that reduce the possibility of detrimental abuse. With this in mind, we are entirely committed to both protecting intellectual property rights and taking action against infringement.”

Read all the Latest NewsTrending NewsCricket NewsBollywood News,
India News and Entertainment News here. Follow us on FacebookTwitter and Instagram.

Updated Date:

also read

World

Music labels are worried because of AI, thanks to a surprising new song ft. Drake X The Weeknd

Someone used AI to create a whole new song featuring Drake and The Weeknd. But because the singers never sang the song, artists and music labels are worried, how AI voice modulation and AI-generated songs can disrupt the industry.

World

Artificial Imposter: 47% of Indian phone users have experienced AI voice scams, highest in the world

About half of Indian adults, or 47%, have experienced AI voice scams, nearly double the global average of 25%. Furthermore, nearly 69% of adults say they can't differentiate between a real and an AI-generated voice.

Deepdub, a film dubbing company, uses an AI to process voice and translate into a different language and promises to preserve what?

By analyzing brief samples—sometimes only three to fifteen seconds—these systems can accurately generate highly realistic copies of an individual's voice.

Title:

Content: Setup
• Day and night
with the system.
Light color
Dark color
• Theme Colors
Speech cloning raises concerns, OpenAI again explains its text-to-speech tool
2024 / 6 / 10 9: 57: 21 Source: IT House Author: Yuanyang Editor: Yuanyang
OpenAI explained its text-to-speech tool for the second time in a few months, and again stressed that the tool is not currently widely available and may not be available in the future.
“Whether or not we end up deploying this technology at scale, it ’ s important for people around the world to understand where it ’ s going, ” OpenAI said in a statement on its website Friday.“ That ’ s why we want to explain how the model works, how we use it for research and education, and how we implement security measures around the technology. ”
At the end of last year, OpenAI shared its “voice engine ” with a small number of external users.
The engine uses text input and a 15-second clip of human voices to “generate a natural sounding human voice that is very similar to the original speaker. ”
The tool could create fake, multilingual human voices, and the company said at the time that it had chosen to preview the technology rather than launch it at scale to “enhance social resilience ” to the threat posed by“ increasingly realistic artificial intelligence-generated models. ”
As part of these efforts, OpenAI says it is actively phasing out ways to use voice recognition for bank account verification, exploring policies to protect individual voices for use in AI, educating the public about the risks of AI, and accelerating the development of technologies to track audiovisual content so users can identify whether they are interacting with real people or synthetic content.
However, despite these efforts, concerns about the technology remain.
Bruce Reed, the head of artificial intelligence for US President Joe Biden, has said that voice cloning is one of the reasons he can't sleep at night.
The Federal Trade Commission said in March that scammers were using AI technology to increase the credibility of their fraud, using voice cloning tools that make it harder for people to distinguish between AI-generated voices and human voices.
“We continue to engage with U.S. and international partners from government, media, entertainment, education, civil society, and more to ensure we incorporate their feedback in the build process, ” the company also noted, once the“ Voice Engine ” is equipped with its latest model, GPT-4o, it will also bring new threats.
The company says it is actively “conducting ‘ red team testing ’ on GPT-4o to identify and address known and unknown risks from areas such as social psychology, bias and fairness, and misinformation. ”
Advertisement Statement: The external redirect links contained in the text (including not limited to hyperlinks, QR codes, passwords, etc.) are used to convey more information and save selection time. The results are for reference only. All articles of IT House contain this statement.
Related articles
Soft media 's website: IT home hot products - Value shopping guide, coupons most buy - rebate coupons iPhone home Win7 home Win10 home Win11 home
Soft media software: Cube cool point desktop Win7 optimization master Win10 optimization master soft media mobile APP application

Deepdub, a film dubbing company, uses an AI to process voice and translate into a different language and promises to preserve what?

What has the US Federal Trade Commission warned about the use of voice cloning tools?

Difficult to distinguish from human voice

What is necessary for voice cloning to work?

This process has become increasingly refined, requiring minimal input data, and is now widely accessible through both free and commercial platforms, allowing even those without technical expertise to produce convincing synthetic voices .

Title:

Content: The scary side of artificial intelligence

The pros of AI outweigh the Cons or threats in our day-to-day lives, and there’s no magic pill to protect us from the scary threats of AI.

  • Exclusive

It's a free Content, simply login/signup to unlock

Get in-depth Industry Insights and Analysis through our “Exclusive” content, presented to you by our esteemed panel of writers, for free

By continuing, you agree to the Terms & Conditions and acknowledge our Privacy Policy. This same account can be used across all Economic Times B2B portals.

  • Updated On Oct 3, 2023 at 03:02 PM IST

Artificial intelligence (AI), as we all know, is the simulation of human intelligence leveraging computer systems and machines. AI programs can continuously learn and evolve with experience. In technical terms, AI systems render the output by churning and analyzing the data for correlations and patterns and help make future predictions using these patterns.

Advt
Today AI is being used extensively in all facets of our lives starting from lifestyle, e-commerce, home appliances, automobiles, navigation, transport, healthcare, agriculture, education, financial planning, chatbots, marketing, cyber security, virtual assistant, industrial automation and many more.

Clearly, we have and are leveraging the strengths of AI to the fullest to automate the routine tasks and make it auto-learn over a period. There’s no doubt that AI has revolutionized the lives of all – organizations, consumers etc. But as they say, “Everything comes at a price”. This analogy is quite true for AI as well, there have been a lot of threats that have emerged using AI and are being utilized by the Cyber criminals to dupe the ignorant users, cause mental stress and extort money.

continued below

Executive Workshop

I’m not going to delve into all of threats but would like to elucidate some of the real ones that have come to the surface recently:

Voice cloning or mimicry

As the name suggests, it uses machine learning algorithms to simulate anyone’s voice. It requires certain voice data to analyze and learn the specific patterns of any individual. There are certain advantages of cloning a voice that can be used in voice assistants, chatbots, navigation systems and other areas. Voice cloning has existed for a while but was restricted to experts who possess the technical knowledge to be able to use the same. But now AI Voice cloning tools are available on the open Internet that anyone can leverage with literally no technical knowledge or cost to clone anyone’s voice. All you need to do is just Google and it will pop up and get a bunch of both free and paid AI Voice cloning tools.

Advt
In fact, recently one of the global leaders in online protection published a report stating how AI is fueling a rise in online voice scams, with just three seconds of audio required to clone a person’s voice. This is extremely creepy. Just imagine, if a Cyber-criminal clones your voice and tricks your near ones by making them believe that they are talking to you, and request for money due to any distress reasons. In fact, these are not assumptions but real-life incidents that are happening across the world every minute.

Deepfakes AI

Deepfakes AI are being used to impersonate an individual by manipulating videos, images, audios, and other content available on the social media platforms using deep learning techniques. It mainly transforms existing content from one person to another. This is another area that has been exploited by the Cyber criminals in the recent past, and there are a few deepfake cases that have made waves on the Internet. One of the most famous ones being Deepfake created by a VFX artist of one of Hollywood’s most famous male actors. This piece was so well made by rendering thousands of pictures using artificial versions of the actor. There are similar Deepfakes incidents involving other global celebrities and leaders. The damage and misuse of the deepfakes videos could be extremely dangerous for the human society and can be used by terrorists to incite riots and create nuisance.

Creating virtual people

This is another area where AI is being used to create people that do not exist. There are a lot of websites that allow you to generate a random image by just specifying the age group, gender & ethnicity, and you can continue to refresh these images at the click of a button. Be mindful of the fact that these images pertain to people that do not exist in this world. On the similar lines, there’s an emerging trend of AI news anchors that possess a human-like appearance and are fed data that they can read using text-to-speech technology. Recently, there's been a surge of AI-Driven robotic new anchors in multiple language channels. While the main objective of this initiative is to provide royalty free or cost-effective human images for branding and other purposes. However, the overuse of these technologies causes Fear of missing out (FOMO), Feeling of inadequacy, dissatisfaction, and isolation may potentially lead to psychological and emotional vulnerabilities.

One of the Hollywood movies, ‘HER’ portrayed an introverted and lonely writer (Joaquin Phoenix) who purchases an AI virtual assistant personified through a female voice to assist him in his writing job. However, when he finds out about its ability to learn and adapt, he falls in love with it. The film delves into emotional vulnerability and the impact of AI on human relationships.

Needless to say, that the pros of AI outweigh the Cons or threats in our day-to-day lives, and there’s no magic pill to protect us from the scary threats of AI. However, there are certain preventive tips that one can follow to mitigate any such risks. Do spread awareness with your family and friends:

  • Set some code words with family, and friends to verify the identity during emergency circumstances
  • Optimize social media presence by limiting your connections with only known contacts and avoid connecting with any unknown ones
  • Avoid saying ‘Yes’ as a greeting while receiving unknown calls as it can be used to authorize credit card charges or other purposes
  • Continuously update privacy settings on social media to allow viewing by friends and family only
  • Avoid clicking on any suspicious links or calls received without proper due diligence
  • Do not share your passwords and other sensitive information with anyone
  • Verify the credentials of any online information before circulating it through various mediums
  • Avoid public Wi-Fi connections offered at coffee shops or airports, which requires to enter password
  • Closely monitor the time spent by younger generation on social media and online gaming
  • Finally, in case of any suspicion, double check and report it to the host site or concerned authorities
The author is the CIO of MOVIN

Disclaimer: The views expressed are solely of the author and ETCIO does not necessarily subscribe to it. ETCIO shall not be responsible for any damage caused to any person/organization directly or indirectly.

  • Published On Oct 3, 2023 at 03:02 PM IST

Most Read in Artificial Intelligence

Join the community of 2M+ industry professionals

Subscribe to our newsletter to get latest insights & analysis.

Download ETCIO App

  • Get Realtime updates
  • Save your favourite articles

The underlying technical development of voice cloning relies on vast datasets of human speech and sophisticated neural networks.

Systems like OpenAI’s voice engine, Tortoise, and products from ElevenLabs use thousands of hours of voice recordings to train models that learn not just the words being spoken, but also how they are articulated.

Title:

Content: OpenAI's Johansson gaffe pushes voice cloning into spotlight

Law enforcement warned that AI clones could be used to extort money from loved ones over the phone

By AFP

  • Follow us on

Scarlett Johansson voiced an AI character in the film "Her". — Reuters file

Published: Mon 27 May 2024, 7:51 PM

Last updated: Mon 27 May 2024, 7:52 PM

OpenAI was forced to apologise to actor Scarlett Johansson last week for using her voice -- or something very similar -- on its latest chatbot, throwing the spotlight on to voice-cloning tech.

Although OpenAI denied the voice they used was Johansson's, their case was not helped by CEO Sam Altman flagging the new model with a one-word message on social media -- "Her".

Johansson voiced an AI character in the film "Her", which Altman has previously said is his favourite film about the technology.

Right from the start, AI voice cloning has proved problematic.

Last year, British firm Elevenlabs went viral for all the wrong reasons when it released its voice-cloning software.

Internet pranksters immediately began pushing out deepfaked celebrities -- Harry Potter star Emma Watson was shown reading Hitler's Mein Kampf.

Law enforcement warned that AI clones could be used to extort money from loved ones over the phone.

The technology has developed rapidly in the past year becoming far more realistic and nuanced.

Danish entrepreneur Victor Riparbelli, CEO of British AI firm Synthesia, told AFP it was largely down to a program called Tortoise that was released two years ago.

The program's developers threw thousands of hours of voice data into their model in an unstructured way and discovered it not only learnt what to say but how to say it.

"That was a pretty big paradigm shift," Riparbelli said on the sidelines of last week's VivaTech conference in Paris.

Tortoise was an open source program and Elevenlabs was the first to go to market using it.

OpenAI uses similar systems though they do not release any details.

Much of the controversy around voice cloning has focused on concerns over people misusing the software.

But the claim against OpenAI is unusual because it is the company itself accused of playing fast and loose.

"It was very unfortunate that OpenAI did that -- really not very good," Katya Laine, CEO of TALKR.ai, told AFP at VivaTech.

"If they actually cloned her voice without her knowing then I think that's very very bad," said Riparbelli.

The two entrepreneurs are among hundreds harnessing AI voice programs for uses that they argue will make companies more efficient.

Laine's firm provides virtual voice assistants -- essentially AI customer service agents.

She said her firm's system could now resolve 25 to 30 percent of calls without any human involvement.

Synthesia specialises in video avatars, which Riparbelli said allowed any office worker to turn text or slides into a video performed by a realistic AI.

Both Riparbelli and Laine allow their clients to use their own avatars, off-the-shelf products or those supplied by the likes of OpenAI and Elevenlabs.

Riparbelli said Synthesia used actors whose likenesses and voices were licensed for two years with an option to renew after the initial period.

The problems arise if actors' voices are used without their consent.

The fiasco overshadowed a developer conference in Paris last week when OpenAI was showing off a suite of new tools.

In front of a big screen in an auditorium, Romain Huet, OpenAI's Head of Developer Experience, breezily chatted into his phone.

Seconds later, his short voice sample had been processed and could be heard commentating over a generated video -- in five languages.

The demonstration showed how quickly the field is moving, but the headlines had already been written.

The Washington Post asked in a newsletter "How dumb is OpenAI?", other commentators were suggesting wunderkind Altman was nothing more than a huckster.

Nonetheless, Riparbelli was open to OpenAI's argument that they had used another actor who just sounded liked Johansson.

"If it's not her but someone who sounds a lot like her... where do you draw that line," Riparbelli asked.

"If they're not allowed to use someone who sounds a lot like her, then it sets a very odd precedent."

More news from World

Deepdub, a film dubbing company, uses an AI to process voice and translate into a different language and promises to preserve what?

What is necessary for voice cloning to work?

These models can mimic vocal accents, inflections, and emotions in multiple languages, creating speech that, to human listeners, is often indistinguishable from genuine recordings .

Deepdub, a film dubbing company, uses an AI to process voice and translate into a different language and promises to preserve what?

What has the US Federal Trade Commission warned about the use of voice cloning tools?

Difficult to distinguish from human voice

The uses for AI voice cloning are expanding rapidly across various industries.

In entertainment, cloned voices have been deployed to recreate performances by actors unable to work, such as using a synthetic James Earl Jones voice for Darth Vader.

Title:

Content: AI clones child’s voice in fake kidnapping scam

‘I never doubted for one second it was her,’ mother says

Anthony Cuthbertson
Thursday 13 April 2023 12:34
Comments

A mother has said she was “100 per cent” convinced by an AI voice clone of her daughter that scammers used in a faked kidnapping attempt.

Jennifer DeStefano from Arizona picked up a call from an unknown number and heard what she believed to be her 15-year-old daughter Brie “sobbing”.

The voice on the other end of the line said, “mom, I messed up”, before a male voice took over and made threatening demands.

“This man gets on the phone, and he’s like, ‘listen here, I’ve got your daughter’,” Ms DeStafano told local news show WKYT.

The apparent kidnapper told the mother: “You call the police, you call anybody, I’m going to pop her so full of drugs, I’m going to have my way with her, and I’m going to drop her off in Mexico.”

In the background, Ms DeStefano said she could hear her daughter saying “help me, mom, please help me” and crying.

“It was 100 per cent her voice,” Ms DeStefano said.

“It was never a question of who is this? It was completely her voice, it was her inflection, it was the way she would have cried – I never doubted for one second it was her. That was the freaky part that really got me to my core.”

The apparent kidnapper demanded $1 million for the daughter to be released, before lowering the figure to $50,000. Ms DeStefano only realised that her daughter was safe after a friend called her husband and confirmed that she was safe.

The scammer, who police are still investigating, appears to have used artificial intelligence voice clone technology, which has become increasingly competent in recent years at mimicking people’s voices.

It is also relatively easy to access and use, with AI tools freely available on the internet.

AI-generated voices have already been used in films to replicate actors like James Earl Jones who voiced the original Darth Vader character in the Star Wars franchise.

It has also been touted as a way for authors to produce audiobooks without spending hours in a studio reading their work.

The technology has also raised fears that it could be misused by scammers or people attempting to create deepfake videos, with one voice clone startup warning that it could be “harmful in the wrong hands”.

In a press release last year, the firm Respeecher said: “The widespread attention and adoption of deepfake technologies will accelerate the need to regulate the use of the technology.”

Thank you for registering

Please refresh the page or navigate to another page on the site to be automatically logged inPlease refresh your browser to be logged in

The technology is also used in media dubbing and in video games to reproduce celebrity voices.

For accessibility, it enables personalized voice assistants, customer service bots, and audiobooks, and allows people who have lost their voices to communicate in their own or a synthesized likeness.

Eleven Labs is extending its voice cloning services for which industries?

In education, voice cloning supports production of localized and leveled audio materials for language learning.

How will educational institutions benefit from developing AI voice audio content for languages learners?

However, the same technology is also exploited for nefarious purposes.

What are some risks regarding AI voice clning?

What do cybersecurity experts believe about the use of synthesized voice?

could become a popular method of attacking banks and financial services

High-profile scam cases have used synthetic voices to impersonate trusted individuals for fraudulent activities such as fake kidnappings and calls requesting money.

Deepfake audio content, including false statements attributed to public figures and celebrities, further exemplifies its potential for misuse and societal disruption .

What are some risks regarding AI voice clning?

The ethical implications of AI voice cloning are profound.

The unauthorized use and replication of a person’s voice raise critical concerns about privacy and consent.

There is serious risk of identity theft, extortion, and reputational damage when voices are cloned and used without the individual's permission.

The ability to fabricate convincing audio for scams, political manipulation, or the diffusion of misinformation erodes public trust in audio recordings and complicates efforts to verify authentic communication.

What are some risks regarding AI voice clning?

The ease and realism of voice cloning increase both the scale and subtlety of these risks, magnifying the potential for harm .

What are some risks regarding AI voice clning?

What do cybersecurity experts believe about the use of synthesized voice?

could become a popular method of attacking banks and financial services

Legal responses to voice cloning are still evolving but are becoming increasingly robust.

Title:

Content: That’s how easily your vote can be stolen – and used for AI fraud

The person on the phone sounded exactly like Ruth Card’s grandson Brandon.

He needed money fast and Ruth, recognizing his voice, went to the bank.

Benjamin Perkins’ parents received a call from a man who said he was a lawyer. Benjamin was alleged to have killed another man and needed bail money. The lawyer handed the phone to someone – who sounded exactly like Benjamin.

In both cases, as Washington Post reported on, it involved a new form of fraud – voices cloned using artificial intelligence.

“Impossible to imagine a year ago”

In the past, large amounts of audio material were required to be able to reliably clone someone’s voice, but AI development is going at a breakneck pace – now it’s faster, easier and more accessible than before.

– What was impossible to imagine a year ago is free today. The companies that make and sell AI want a lot of data – that’s why they give away their services, says Tobias Falk, lecturer at the department of computer and systems science at Stockholm University, to SVT Nyheter.

“Having reached a point where the machines can imitate us”

– Algorithms are trained to imitate how the human brain works. It’s going really fast now, but it’s taken decades to get to the point where the machines can mimic us as well as they do today.

The police’s national fraud center writes in an email to SVT Nyheter that they are “aware of the phenomenon” but that they have not seen any cases in Sweden at the moment. “We monitor this and have communication with international contacts,” writes the police spokesperson.

How many seconds long clip is enough to be able to clone someone’s voice? And how well does it actually work in Swedish?

Watch the video above for our test.

In the United States, the Federal Communications Commission has banned robocalls utilizing AI-synthesized voices in response to their misuse in telecom scams.

Title:

Content: © 2024
NPR for Northern Colorado
Play Live Radio
Next Up:
0:00
0:00
0:00 0:00
Available On Air Stations

The government announced winners of a contest to tell real voices from deepfake audio

The federal government is trying to promote efforts to counter so-called audio deepfakes by awarding four organizations prizes for developing technologies to distinguish between authentic human speech and audio generated by artificial intelligence.

The prizes, awarded Monday by the Federal Trade Commission, come as the agency has issued warnings to consumers about scams using AI-generated voices and as concerns grow about the influence of deepfakes on this year's elections.

One winner, OriginStory, created by people including researchers at Arizona State University, uses sensors to detect things that humans do such as breathing, motion and heartbeat information.

"We have this very unique speech production mechanism. And so by sensing it in parallel with the acoustic signal, you can verify that the speech is coming from a human" Visar Berisha, a professor at ASU's College of Health Solutions who was part of the team, told NPR in an interview.

OriginStory's technology leverages sensors in existing hardware, though it's not available in all recording equipment - newer Android phones and iPhones don't have them. Berisha says they are focusing on hardware that already can use it.

Another, DeFake, made by Ning Zhang, a researcher at Washington University in St. Louis, injects data into recordings of real voices so AI-generated voice clones don't sound like the real person.

Zhang says his technology was inspired from earlier tools developed by University of Chicago researchers that place hidden changes in images so that artificial intelligence algorithms that try to train themselves on the images cannot mimic them.

A third, AI Detect, uses AI to catch AI. It's made by startup Omni Speech, and its CEO David Przygoda says their machine learning algorithm extracts features from audio clips like inflection and uses that to teach the models to tell the difference between real and fake audio.

"They have a really hard time transitioning between different emotions, whereas a human being can be amused in one sentence and distraught in the next." says Przygoda.

In an FTC video, the company says its tool is over 99.9% accurate, but Przygoda says it's not yet ready for independent testing, and the contest judges did not test the submissions under real world conditions.

The three winners will split a cash prize of $35,000. Another company, Pindrop Security, received recognition for its work but will not receive cash because it's a larger organization.

In a statement, the company says the FTC's award, "recognizes and validates that a decade of work Pindrop has put in to study the problem of voice clones and other deepfakes."

There are a number of existing commercial detection tools that rely on machine learning, but factors such as sound quality and media format can make them less reliable, and detectors need to constantly be trained on new deepfake generators to catch the audio they produce.

Berisha says this is a losing game in the long term, referencing the challenges that detectors of AI-generated text faced. OpenAI took down its own detector months after it launched, citing poor results.

"I think we're headed in exactly the same direction for voices. And so if AI-based detectors work now, I think they'll work less well six months from now and less well a year from now, two years from now. And at some point, they will stop working altogether."

This drove him to develop a process that authenticates human voices as words are being spoken.

Earlier this year, the Federal Communications Commission bannedAI-enabled robocalls. Synthetic audio company ElevenLabs has a tool to detect its own product and is calling on other producers of synthetic voices to do the same. The European Union recently passed and "AI act' and state legislatures in the U.S.are passing lawsthat regulate deepfakes. Researchers, detection software makers and the FTC say that technology like this can't be the only solution to the challenges posed by generative AI tools.

Copyright 2024 NPR. To see more, visit https://www.npr.org.

Huo Jingnan
Huo Jingnan (she/her) is an assistant producer on NPR's investigations team.

The European Union’s “AI Act” includes guidelines specifically targeting the responsible development and use of artificial intelligence, encompassing synthetic voice technologies.

Additionally, several US states have enacted or proposed legislation aimed at restricting deepfake content, including voice clones.

These measures often seek to safeguard individuals from unauthorized use of their voices and increase requirements for content identification and transparency.

Concurrently, research and regulatory attention are being directed at developing detection tools and ensuring platforms provide methods to label or identify synthetic audio .

Nonetheless, the legal framework still faces notable limitations.

Many regulations lag behind the technical capabilities and varied applications of voice cloning, leaving gaps in protection around intellectual property, impersonation, and the commercial or political exploitation of an individual’s voice.

The law continues to adapt as new uses and abuses of this technology emerge, demanding ongoing vigilance from professionals considering its adoption .

In summary, AI voice cloning stands at the intersection of remarkable innovation and significant societal risk.

What are some risks regarding AI voice clning?

Its ability to generate authentic-sounding audio offers opportunities for entertainment, accessibility, and personalized content creation, but also enables new forms of deception and harm.

What are some risks regarding AI voice clning?

Awareness of the technical function, potential, ethical hazards, and evolving legal context is essential for anyone contemplating its use .