"Voice cloning technology has the potential to be used in various fields, including entertainment, accessibility, and education, but it also raises concerns about online scams and ethical implications."

Voice cloning technology has raised concerns due to its potential for misuse in online voice scams, where a person's voice can be cloned with just three seconds of audio, leading to financial loss and emotional distress.

Voice cloning technology is being used for malicious purposes, such as online voice scams, where a person's voice can be cloned with just three seconds of audio.

"Realistic voice cloning is enabled by machine learning algorithms that analyze and learn individual voice patterns from voice data, allowing for simulation of anyone's voice."

"Voice cloning technology poses a risk of being used for malicious purposes, including scams and extortion, by cyber criminals who can leverage its increasing realism and accessibility to deceive and manipulate users."

The technology used to create synthetic voices is AI platforms such as Murf, Ressemble, and ElevenLabs, which can turn text into speech in a desired voice.

Individuals can protect themselves from voice cloning scams by being cautious of unusual audio content, verifying authenticity, and understanding the capabilities and limitations of voice cloning technology.

"Voice mimicry is achieved through machine learning algorithms that simulate a person's voice by analyzing and learning patterns from their voice data."

"Voice cloning tools utilize machine learning to replicate voices, offering advantages in various applications, including voice assistants and chatbots, and are now accessible to anyone with minimal technical expertise."

The AI voice cloning tool by ElevenLabs can clone voices with just a three-second sample clip.

A potential misuse of AI voice clones, as warned by law enforcement, is using them to extort money from loved ones over the phone.

"Voice cloning technology has the potential to be used in various fields, including entertainment, accessibility, and education, but it also raises concerns about online scams and ethical implications."

Title:

Content: The scary side of artificial intelligence

The pros of AI outweigh the Cons or threats in our day-to-day lives, and there’s no magic pill to protect us from the scary threats of AI.

  • Exclusive

It's a free Content, simply login/signup to unlock

Get in-depth Industry Insights and Analysis through our “Exclusive” content, presented to you by our esteemed panel of writers, for free

By continuing, you agree to the Terms & Conditions and acknowledge our Privacy Policy. This same account can be used across all Economic Times B2B portals.

  • Updated On Oct 3, 2023 at 03:02 PM IST

Artificial intelligence (AI), as we all know, is the simulation of human intelligence leveraging computer systems and machines. AI programs can continuously learn and evolve with experience. In technical terms, AI systems render the output by churning and analyzing the data for correlations and patterns and help make future predictions using these patterns.

Advt
Today AI is being used extensively in all facets of our lives starting from lifestyle, e-commerce, home appliances, automobiles, navigation, transport, healthcare, agriculture, education, financial planning, chatbots, marketing, cyber security, virtual assistant, industrial automation and many more.

Clearly, we have and are leveraging the strengths of AI to the fullest to automate the routine tasks and make it auto-learn over a period. There’s no doubt that AI has revolutionized the lives of all – organizations, consumers etc. But as they say, “Everything comes at a price”. This analogy is quite true for AI as well, there have been a lot of threats that have emerged using AI and are being utilized by the Cyber criminals to dupe the ignorant users, cause mental stress and extort money.

continued below

Executive Workshop

I’m not going to delve into all of threats but would like to elucidate some of the real ones that have come to the surface recently:

Voice cloning or mimicry

As the name suggests, it uses machine learning algorithms to simulate anyone’s voice. It requires certain voice data to analyze and learn the specific patterns of any individual. There are certain advantages of cloning a voice that can be used in voice assistants, chatbots, navigation systems and other areas. Voice cloning has existed for a while but was restricted to experts who possess the technical knowledge to be able to use the same. But now AI Voice cloning tools are available on the open Internet that anyone can leverage with literally no technical knowledge or cost to clone anyone’s voice. All you need to do is just Google and it will pop up and get a bunch of both free and paid AI Voice cloning tools.

Advt
In fact, recently one of the global leaders in online protection published a report stating how AI is fueling a rise in online voice scams, with just three seconds of audio required to clone a person’s voice. This is extremely creepy. Just imagine, if a Cyber-criminal clones your voice and tricks your near ones by making them believe that they are talking to you, and request for money due to any distress reasons. In fact, these are not assumptions but real-life incidents that are happening across the world every minute.

Deepfakes AI

Deepfakes AI are being used to impersonate an individual by manipulating videos, images, audios, and other content available on the social media platforms using deep learning techniques. It mainly transforms existing content from one person to another. This is another area that has been exploited by the Cyber criminals in the recent past, and there are a few deepfake cases that have made waves on the Internet. One of the most famous ones being Deepfake created by a VFX artist of one of Hollywood’s most famous male actors. This piece was so well made by rendering thousands of pictures using artificial versions of the actor. There are similar Deepfakes incidents involving other global celebrities and leaders. The damage and misuse of the deepfakes videos could be extremely dangerous for the human society and can be used by terrorists to incite riots and create nuisance.

Creating virtual people

This is another area where AI is being used to create people that do not exist. There are a lot of websites that allow you to generate a random image by just specifying the age group, gender & ethnicity, and you can continue to refresh these images at the click of a button. Be mindful of the fact that these images pertain to people that do not exist in this world. On the similar lines, there’s an emerging trend of AI news anchors that possess a human-like appearance and are fed data that they can read using text-to-speech technology. Recently, there's been a surge of AI-Driven robotic new anchors in multiple language channels. While the main objective of this initiative is to provide royalty free or cost-effective human images for branding and other purposes. However, the overuse of these technologies causes Fear of missing out (FOMO), Feeling of inadequacy, dissatisfaction, and isolation may potentially lead to psychological and emotional vulnerabilities.

One of the Hollywood movies, ‘HER’ portrayed an introverted and lonely writer (Joaquin Phoenix) who purchases an AI virtual assistant personified through a female voice to assist him in his writing job. However, when he finds out about its ability to learn and adapt, he falls in love with it. The film delves into emotional vulnerability and the impact of AI on human relationships.

Needless to say, that the pros of AI outweigh the Cons or threats in our day-to-day lives, and there’s no magic pill to protect us from the scary threats of AI. However, there are certain preventive tips that one can follow to mitigate any such risks. Do spread awareness with your family and friends:

  • Set some code words with family, and friends to verify the identity during emergency circumstances
  • Optimize social media presence by limiting your connections with only known contacts and avoid connecting with any unknown ones
  • Avoid saying ‘Yes’ as a greeting while receiving unknown calls as it can be used to authorize credit card charges or other purposes
  • Continuously update privacy settings on social media to allow viewing by friends and family only
  • Avoid clicking on any suspicious links or calls received without proper due diligence
  • Do not share your passwords and other sensitive information with anyone
  • Verify the credentials of any online information before circulating it through various mediums
  • Avoid public Wi-Fi connections offered at coffee shops or airports, which requires to enter password
  • Closely monitor the time spent by younger generation on social media and online gaming
  • Finally, in case of any suspicion, double check and report it to the host site or concerned authorities
The author is the CIO of MOVIN

Disclaimer: The views expressed are solely of the author and ETCIO does not necessarily subscribe to it. ETCIO shall not be responsible for any damage caused to any person/organization directly or indirectly.

  • Published On Oct 3, 2023 at 03:02 PM IST

Most Read in Artificial Intelligence

Join the community of 2M+ industry professionals

Subscribe to our newsletter to get latest insights & analysis.

Download ETCIO App

  • Get Realtime updates
  • Save your favourite articles

What are some risks regarding AI voice clning?

Voice cloning technology has raised concerns due to its potential for misuse in online voice scams, where a person's voice can be cloned with just three seconds of audio, leading to financial loss and emotional distress.

What do cybersecurity experts believe about the use of synthesized voice?

could become a popular method of attacking banks and financial services

What is necessary for voice cloning to work?

Voice cloning technology is being used for malicious purposes, such as online voice scams, where a person's voice can be cloned with just three seconds of audio.

What do cybersecurity experts believe about the use of synthesized voice?

could become a popular method of attacking banks and financial services

What is necessary for voice cloning to work?

"Realistic voice cloning is enabled by machine learning algorithms that analyze and learn individual voice patterns from voice data, allowing for simulation of anyone's voice."

What is necessary for voice cloning to work?

"Voice cloning technology poses a risk of being used for malicious purposes, including scams and extortion, by cyber criminals who can leverage its increasing realism and accessibility to deceive and manipulate users."

What are some risks regarding AI voice clning?

What do cybersecurity experts believe about the use of synthesized voice?

could become a popular method of attacking banks and financial services

The technology used to create synthetic voices is AI platforms such as Murf, Ressemble, and ElevenLabs, which can turn text into speech in a desired voice.

Title:

Content: Advertisement
Foreign Press
Urgent warning.
This is how artificial intelligence is used to steal your money.
Translated from "Lebanon 24"
|
28-06-2023 | 11: 55
A-
A +
photos 0
A +
A-
There are many forms of fraud, but it usually works the same way: the fraudster pretends to be someone you trust to convince you to send money.
According to the Federal Trade Commission (FTC) website, fraudsters pretend to be an IRS official or a family member among other examples.
Advertisement
According to Insider, “Most scams usually happen over the phone, but they can also happen via social media, text messages or email.
A good example of this is what happened with Google software engineer Richard Mendelstein.
The latter received a call, which he believed was from his daughter Stella, screaming on the phone for help from her father, asking him to withdraw $4,000 in cash as a ransom payment.
It was only after he transferred the money that he realized that he had been scammed and that his daughter was safe at school the whole time and had not been kidnapped.
"Vocal techniques are often used to mysteriously align with a child's age and gender, but using artificial intelligence, the sound on the other end of the phone can now look eerily similar to the sound of a real person."
“Although the AI technology that makes these scams possible is not new, it is becoming better, cheaper and easier to access.
Steve Grubman, chief technology officer at global security software company McAfee, said: “One of the most important things to note this year is that advances in artificial intelligence have made these technologies accessible to many people.
“Artificial intelligence for fraud is easier to use today by cybercriminals than it used to be. ”
Previous phone-call scams relied on the fraudster ’ s acting skills or the victim ’ s level of naivety, but now AI does most of the work.
Through AI platforms such as Murf, Resemble, and ElevenLabs, users can turn text into speech in the voice of the person they want.
“He just has to upload an audio file of the person who wants to use his or her voice to one of these sites (with a small audio clip, the AI can match 95% of the target ’ s voice) and then just write the text he or she wants to use in the scam. ”
"It is difficult to arrest those who commit such fraud.
The victim often has little information to give to the police.
To make matters worse are the logistical and judicial challenges faced by legal agencies, especially as fraudsters are present all over the world.
According to Grubman, if you feel like you ’ re being scammed, you don ’ t have to worry.
All it takes is asking a few questions that only the person whose voice is being used knows the answer to.
The FTC also recommended that if a loved one tells you they need the money, put the call on hold and try calling your family members separately to verify the story.
Security experts also recommend agreeing on a safe word and using it to distinguish between a real emergency and a scam.
“With the spread of artificial intelligence, these kinds of scams are putting the reliability of even our close family members at stake. ”
Advertisement
Our news via your email
SIGN UP
Translated from "Lebanon 24"

Individuals can protect themselves from voice cloning scams by being cautious of unusual audio content, verifying authenticity, and understanding the capabilities and limitations of voice cloning technology.

Title:

Content: Technology

AI-enabled scams will proliferate

Ultra-realistic digital trickery will help fraudsters deceive people on an entirely new scale

Jeff Clune is an associate professor of computer science at the University of British Columbia and Canada CIFAR AI Chair at the Vector Institute.

(This illustration was created by Maclean’s art director Anna Minzhulina using the generative AI image program Imagine. Minzhulina spent weeks feeding prompts into the program, inspired by the essay.)

Money-grabbing scams have been happening for longer than we can remember: con artistry, Ponzi schemes, online phishing, even snail mail gambits. But the advent of more sophisticated artificial intelligence will soon make these tricks far more common, complex and convincing. Knowing who or what to trust online is about to get much more challenging—and we had better be prepared.

Photographs and videos used to be one way of verifying if things are true. Now, AI programs have opened the door to forms of scamming that are much harder to detect. Criminals can simply use software like Midjourney or DALL-E 2 to generate any kind of realistic photos. AI can also mimic someone’s voice, and soon AI-generated videos will be able to clone their appearances and facial expressions too. There are already deepfakes of Elon Musk telling people to invest their money in crypto, Tucker Carlson promoting new trading platforms and Ron DeSantis appearing in an episode of The Office as Michael Scott. We are losing our core methods of verification in ways most of us could not even have imagined just three years ago.

We have to prepare to face elaborate scams that take advantage of our trust. Picture this: in the near future, you might receive a video call from a family member who is asking for money because they are in a tight spot. It probably would not seem possible that you are talking to a deepfake, but this type of trickery—though still rare—is already spreading. Scammers can easily find existing videos or podcasts of somebody online and train an AI on that voice and facial expressions to create an uncanny resemblance. Because it is already possible to imitate my voice on a voice call, I have created a secret password for my family so we can verify each others’ identity if we ever get suspicious of each other over the phone.

The quantity of AI scams will also increase. Back in the ’80s, people would occasionally receive spam letters in snail mail, but those attempts were few and far between because con artists did not want to pay for paper and postage. Email removed the cost of sending mail, and the number of attempted scams skyrocketed. Just like email did a few decades ago, AI is making scamming far easier and cheaper. This could enable disruptive levels of manipulation in the near future. Rather than spending hours trying to defraud one person or paying 1,000 scam employees to work 40-hour weeks, a single scammer could train an AI model to deceive 100 million people simultaneously.

AI-generated, human-sounding robo-calls will contact millions of people at once. Instead of listening to a recorded message, the person will carry on a back-and-forth conversation without realizing the person on the other end of the line is not human. AI scammers will impersonate police officers or civil servants and ask for donations to certain charities, for example. Such scams will also be politically motivated—like an AI calling every Canadian to feed them false information about a political figure, or dictators flooding information channels with fake videos, comments, likes and emojis that subtly, yet relentlessly, repeat the party line. Criminals will wield AI to disrupt finance markets by pumping up worthless penny stocks or driving down the value of other stocks with fake news.

For now, scammers still need to be tech-savvy to create powerful AI-driven schemes. But in technology, what is hard now will be easy soon. As we develop better and more user-friendly AI software, a 12-year-old kid might be able to orchestrate a worldwide phishing attempt from his or her bedroom. Once it becomes cheap, quick and easy to program something that is convincingly human, we are in trouble. It will become incredibly difficult to know which information is trustworthy. Unless we invent technological and sociological solutions to these problems, I am afraid that we will soon not be able to trust most of what we see online, and that makes me deeply concerned about the future of our information landscape.

As we prepare for AI scams to proliferate, the best advice I can offer is for people to seek out and hold onto the sources they trust most—whether that is the New York Times or a particular reporter. But even then they must make sure they are in fact getting information from that source. We can also protect ourselves from scams by better understanding what technology can and cannot do. From now on, we should always second-guess what we see in text, websites and photos. Deepfakes are still rare, and most videos are trustworthy, but that will likely change. If a video seems surprising, do an independent search to see if people are reporting that it is fake. And if you have been scammed, tell others how it happened, so that they do not also fall prey.

Scams work best when they pull on heartstrings—and humans tend to relate and respond to other humans, especially those they know and trust. The better AI can impersonate a flesh-and-blood person, the more turmoil we can expect. As the cost of deception slides towards zero, we will need to build our individual and societal immune systems to handle an army of new threats. As in the real world, an ounce of prevention is worth a pound of cure. The best vaccination for scams is knowledge and a sizable dose of skepticism.

We reached out to Canada’s top AI thinkers in fields like ethics, health and computer science and asked them to predict where AI will take us in the coming years, for better or worse. The results may sound like science fiction—but they’re coming at you sooner than you think. To stay ahead of it all, read the other essays that make up our AI cover story, which was published in the November 2023 issue of Maclean’s.

Looking for more?

Get the Best of Maclean's sent straight to your inbox. Sign up for news, commentary and analysis.
FILED UNDER:

"Voice mimicry is achieved through machine learning algorithms that simulate a person's voice by analyzing and learning patterns from their voice data."

What is necessary for voice cloning to work?

"Voice cloning tools utilize machine learning to replicate voices, offering advantages in various applications, including voice assistants and chatbots, and are now accessible to anyone with minimal technical expertise."

The AI voice cloning tool by ElevenLabs can clone voices with just a three-second sample clip.

Title:

Content: Language No Barrier: New AI tool can clone voice and make it say anything in multiple languages

Imagine former US President Donald Trump speaking in flawless Hindi. A new AI voice cloning tool by ElevenLabs can clone voices and make them say anything in a bunch of different languages. All it needs is a three-second sample clip and a bit of text as the prompt.

Mehul Reuben Das May 03, 2023 13:42:20 IST

Imagine former US President Donald Trump speaking in flawless Hindi. A new AI voice cloning tool by ElevenLabs can clone voices and make them say anything in a bunch of different languages. All it needs is a three-second sample clip and a bit of text as the prompt.

As advanced as AI-based voice cloning has been, there were a few limitations to it. For example, voice generators were mainly bound by the language of the source they were trained in or rather the language of the prompt that was given to them. Now, ElevenLabs, an American AI software developing company has come up with a tool that does away with that limitation.

ElevenLabs’ new AI tool can clone the voice of a person and make it say anything in a bunch of different languages. All it needs is a clear voice sample of about 3 seconds, although it also suggests that the longer the sample, the better will be the results. The new tool even manages to maintain the speaker’s original tone of voice.

Limitless possibilities
The creators expect that this would ‘extend the possibilities’ in a variety of industries such as publishing, gaming creation, and media. “Introducing Eleven Multilingual v1: Our New Speech Synthesis Model! We’re thrilled to launch our new model supporting seven new languages: French, German, Hindi, Italian, Polish, Portuguese, and Spanish.”

“This advancement expands the horizons for creators, game developers, and publishers, and it paves the way for using generative media to create more localized, accessible, and imaginative content,” the company said.

Following the announcement, Twitter was flooded with comments, with users testing the software on various famous faces.

To demonstrate just how capable the new voice cloning AI tool is, Eluna AI, a platform for AI-content creators experimented with a sample clip of former US President Donald Trump.

In their demo, Eluna AI showcased a small clip of Donald Trump, speaking in his native tongue, English, followed by French, Spanish, German, Hindi and Italian, almost perfectly. The generated samples even gets the accent right, almost if he was a native speaker of the new target language.

Another user changed David Attenborough’s voice to German while the BBC broadcaster discussed the prospects of AI. “With this tool, anyone can do it,” stated the Twitter user. “It’s also worth noting that fraudsters will undoubtedly exploit this one.”

ElevenLabs says that its model was designed to understand the emotion and reasoning underlying words in order to produce natural-sounding and persuasive audio. ElevenLabs has said that some numerals and symbols are currently spoken wrongly, but that it is striving to rectify this.

Who is this new tool meant to help?
The US-based company believes its technology will be valuable in a variety of areas, including education and journalism.

Game developers and publishers can build engaging, localised experiences for worldwide audiences, breaking through language barriers and connecting with players and listeners to maximise engagement and efficiency while maintaining quality and accuracy.

“Educational institutions now have the ability to develop audio content for varied users in their target languages, enhancing language comprehension and even pronunciation abilities, as well as responding to different teaching methods and learning demands,” says the report.

Because of the model’s capacity to produce unique character-like voices, audiobook and content developers may also find it beneficial.

The conundrum of AI voice genertors
Despite its benefits, ElevenLabs’ launch comes amid widespread concern about the wave of new voice cloning technology. An unidentified TikTok user went popular last month after sharing an AI-generated Drake song starring The Weeknd.

It sparked concerns about the grey area of AI-created music and whether it constitutes a copyright violation. Nonetheless, ElevenLabs claims to be ‘fully committed’ to respecting creators’ rights in the midst of technological advancements.

“At Eleven, we think that we should seek to make the best use of emerging technology, but not at any expense,” it stated.

“As we create them, we make every effort to put suitable controls in place that reduce the possibility of detrimental abuse. With this in mind, we are entirely committed to both protecting intellectual property rights and taking action against infringement.”

Read all the Latest NewsTrending NewsCricket NewsBollywood News,
India News and Entertainment News here. Follow us on FacebookTwitter and Instagram.

Updated Date:

also read

World

Music labels are worried because of AI, thanks to a surprising new song ft. Drake X The Weeknd

Someone used AI to create a whole new song featuring Drake and The Weeknd. But because the singers never sang the song, artists and music labels are worried, how AI voice modulation and AI-generated songs can disrupt the industry.

World

Artificial Imposter: 47% of Indian phone users have experienced AI voice scams, highest in the world

About half of Indian adults, or 47%, have experienced AI voice scams, nearly double the global average of 25%. Furthermore, nearly 69% of adults say they can't differentiate between a real and an AI-generated voice.

What company created an AI tool that could mimic the voices of deceased movie stars?

What is necessary for voice cloning to work?

A potential misuse of AI voice clones, as warned by law enforcement, is using them to extort money from loved ones over the phone.

Title:

Content: OpenAI's Johansson gaffe pushes voice cloning into spotlight

Law enforcement warned that AI clones could be used to extort money from loved ones over the phone

By AFP

  • Follow us on

Scarlett Johansson voiced an AI character in the film "Her". — Reuters file

Published: Mon 27 May 2024, 7:51 PM

Last updated: Mon 27 May 2024, 7:52 PM

OpenAI was forced to apologise to actor Scarlett Johansson last week for using her voice -- or something very similar -- on its latest chatbot, throwing the spotlight on to voice-cloning tech.

Although OpenAI denied the voice they used was Johansson's, their case was not helped by CEO Sam Altman flagging the new model with a one-word message on social media -- "Her".

Johansson voiced an AI character in the film "Her", which Altman has previously said is his favourite film about the technology.

Right from the start, AI voice cloning has proved problematic.

Last year, British firm Elevenlabs went viral for all the wrong reasons when it released its voice-cloning software.

Internet pranksters immediately began pushing out deepfaked celebrities -- Harry Potter star Emma Watson was shown reading Hitler's Mein Kampf.

Law enforcement warned that AI clones could be used to extort money from loved ones over the phone.

The technology has developed rapidly in the past year becoming far more realistic and nuanced.

Danish entrepreneur Victor Riparbelli, CEO of British AI firm Synthesia, told AFP it was largely down to a program called Tortoise that was released two years ago.

The program's developers threw thousands of hours of voice data into their model in an unstructured way and discovered it not only learnt what to say but how to say it.

"That was a pretty big paradigm shift," Riparbelli said on the sidelines of last week's VivaTech conference in Paris.

Tortoise was an open source program and Elevenlabs was the first to go to market using it.

OpenAI uses similar systems though they do not release any details.

Much of the controversy around voice cloning has focused on concerns over people misusing the software.

But the claim against OpenAI is unusual because it is the company itself accused of playing fast and loose.

"It was very unfortunate that OpenAI did that -- really not very good," Katya Laine, CEO of TALKR.ai, told AFP at VivaTech.

"If they actually cloned her voice without her knowing then I think that's very very bad," said Riparbelli.

The two entrepreneurs are among hundreds harnessing AI voice programs for uses that they argue will make companies more efficient.

Laine's firm provides virtual voice assistants -- essentially AI customer service agents.

She said her firm's system could now resolve 25 to 30 percent of calls without any human involvement.

Synthesia specialises in video avatars, which Riparbelli said allowed any office worker to turn text or slides into a video performed by a realistic AI.

Both Riparbelli and Laine allow their clients to use their own avatars, off-the-shelf products or those supplied by the likes of OpenAI and Elevenlabs.

Riparbelli said Synthesia used actors whose likenesses and voices were licensed for two years with an option to renew after the initial period.

The problems arise if actors' voices are used without their consent.

The fiasco overshadowed a developer conference in Paris last week when OpenAI was showing off a suite of new tools.

In front of a big screen in an auditorium, Romain Huet, OpenAI's Head of Developer Experience, breezily chatted into his phone.

Seconds later, his short voice sample had been processed and could be heard commentating over a generated video -- in five languages.

The demonstration showed how quickly the field is moving, but the headlines had already been written.

The Washington Post asked in a newsletter "How dumb is OpenAI?", other commentators were suggesting wunderkind Altman was nothing more than a huckster.

Nonetheless, Riparbelli was open to OpenAI's argument that they had used another actor who just sounded liked Johansson.

"If it's not her but someone who sounds a lot like her... where do you draw that line," Riparbelli asked.

"If they're not allowed to use someone who sounds a lot like her, then it sets a very odd precedent."

More news from World