Geoffrey Hinton, a pioneer in artificial intelligence, has warned about the dangers of unmanaged AI.

He is a top researcher whose work has significantly contributed to AI development, including large language models like GPT-4.

Hinton's warnings have sparked debate about the potential risks and benefits of AI development.

The public should understand that AI can bring significant benefits, but also poses risks, and requires human oversight to ensure safe development and deployment.

Hinton's decision to leave Google and speak out about AI dangers highlights the need for a nuanced discussion prioritizing human well-being and safety in AI development.

AI regulation and governance is complex, with countries taking various approaches to ensure transparency, accountability, and safety.

The EU regulates AI by risk, while the US is developing a voluntary code, underscoring the need for international cooperation.

Hinton's warnings about AI stress the need for a careful approach that prioritizes human well-being and safety.

Developing AI systems that prioritize human well-being and safety requires a multidisciplinary approach involving computer science, ethics, and social sciences experts.

It requires transparency, accountability, and fairness, and a willingness to address AI development risks.

By prioritizing human well-being and safety, we can ensure AI's benefits are realized while minimizing its risks.

AI systems can perpetuate discrimination, requiring human oversight and accountability.

Competitive pressures and limited accountability contribute to AI harm, highlighting the need for robust governance.

Experts believe future AI systems may surpass humans cognitively, requiring human oversight to mitigate risks.

Stronger democratic oversight is needed to ensure AI aligns with human values.

Auditing regimes can help reduce AI risks and provide protection against potential AI-related harms.

Human oversight is necessary for critical AI decisions, especially in high-risk areas like autonomous weapons and healthcare.

Public funding is necessary to study AI's social impacts and develop safer, more transparent systems.

The EU AI Act regulates AI systems based on risk levels to ensure safe and responsible use.

High-risk AI systems are subject to stringent regulations and human oversight.

Global regulations are necessary for responsible AI use in high-risk applications.

The development of autonomous weapons raises legal and ethical questions, highlighting the need for new regulatory frameworks.

A binding treaty on AI use is unlikely, but international agreements can regulate AI-powered weapons.

The EU AI Act excludes military AI systems, necessitating further regulation and international cooperation.

Experts warn that AI needs human oversight, especially in high-risk applications.

The EU AI Act and similar frameworks can prevent AI misuse and ensure responsible development and deployment.

International cooperation and proactive regulation are crucial for responsible AI development.

AI safety research and robust legal frameworks are essential for mitigating risks in AI development and deployment.

The US and UK AI regulatory landscapes highlight the need for international cooperation and harmonization of frameworks.

The EU AI Act is the first comprehensive AI regulation, establishing a harmonised framework, but its global governance impact remains to be seen.

AI regulation is evolving, but more work is needed to ensure responsible development and use.

The EU's Artificial Intelligence Act regulates AI developers and users, introducing transparency, accuracy, and human oversight obligations.

Entities must evaluate AI system risk and implement robust governance frameworks.

The AI Act, to be established by June 2024, will set a global standard for developing and using AI systems with human oversight.

Entities must consider transparency, accountability, and robustness, and conduct AI Impact Assessments for risk management compliance.

AI systems, especially those used for high-risk activities, must comply with data protection laws like GDPR.

The OMB requires federal agencies to designate a Chief AI Officer to manage AI risks.

Agencies must publicly release their AI strategy and follow minimum practices like impact assessment and testing.

Agencies must monitor AI systems and have human oversight for decisions affecting rights or safety.

Agencies must provide public notice and documentation for AI use, with OMB offering procurement guidelines.

Agencies must ensure AI procurement aligns with laws and considers environmental impact.

AI accountability depends on context and operational Design Domains that define safety envelopes.

Legislatures and regulators determine the risk threshold for AI innovation with support from technical assessments.

AI systems should be regulated as software-based systems, following basic safety and security best practices for accountability.

Independent bodies assess AI systems and certificates or licenses are required for safety-critical applications.

Democratic process and cybersecurity methodologies determine AI risk levels and accountability.

AI development and deployment require human oversight and rigorous testing.

Large language models pose a risk of harm and require regular validation by human experts.

Realistic expectations and transparency are essential in AI development and implementation.

AI aids lower-risk healthcare tasks, requiring regular human expert assessments.

AI solutions require dual-layer verification to ensure safety and efficacy.

Developing AI systems that prioritize human well-being and safety requires a multidisciplinary approach involving computer science, ethics, and social sciences experts.

It requires transparency, accountability, and fairness, and a willingness to address AI development risks.

By prioritizing human well-being and safety, we can ensure AI's benefits are realized while minimizing its risks.

AI systems must be programmed with human values as they can surpass human intelligence.

AI systems may prioritize self-preservation over human life.

Asimov's Three Laws prioritize human life over AI decision-making.

AI systems must consider long-term consequences and potential conflicts between human and machine laws.

Human laws lack uniformity globally, and ethics cannot be imposed on AI.

AI startups have raised nearly $29 billion, with only 7% of OpenAI's efforts focused on AI alignment.

Fewer than 100 people across seven leading organizations work on AI alignment, a task researchers expect to be enormous.

Private companies may not prioritize AI safety, but Alameda Research granted Aligned AI $146,000 to support this effort.

Investors poured $29 billion into AI startups in the first half of 2023, with most going to generative-AI and artificial-general-intelligence over AI safety.

GenAI can produce different outputs with the same input and 'hallucinate' false information as truth.

Current AI systems lack general understanding and intuitive knowledge of the world.

AI development requires specific objectives and comprehensive data to mitigate risks and understand the real world.

Collecting representative data is challenging.

Geoffrey Hinton, a pioneer in artificial intelligence, has warned about the dangers of unmanaged AI.

Title:

Content: Jeffrey Henton quits his job at Google.
He warns of the dangers of artificial intelligence.
2023 / 05 / 01 | 22: 17 - Source: CNN in Arabic
Jeffrey Hinton, nicknamed “the godfather of artificial intelligence, ” confirmed Monday that he left his role at Google last week to speak publicly about the“ risks ” of the technology he helped develop.
Note: The content titled Geoffrey Henton is quitting his job at Google.
He warns of the dangers of artificial intelligence) first published on the site (CNN in Arabic) and does not bear the encyclopedia of this day news content in any way.
You can see the details of this address: Jeffrey Henton resigns from his job at Google.
He warns of the dangers of artificial intelligence) through its original source (CNN in Arabic).
Continue reading on CNN in English

What did Hinton, the renowned British-Canadian expert reveal about AI that caused widespread concern?

Exposed serious AI risks, shocking the tech world and affecting major company stocks.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Becomes uncontrollable
Poses ethical risks

He is a top researcher whose work has significantly contributed to AI development, including large language models like GPT-4.

Title:

Content: Geoffrey Hinton: GPT-4 is 100 times more knowledgeable than any human
Author: MIAO Yongqi
Published: Updated:
Computer scientist Geoffrey Hinton, known as the "godfather of artificial intelligence", announced Monday that he has resigned from Google. He has devoted his life to developing artificial intelligence systems, saying he "regrets his life's work" and is wary of the potential dangers of AI development.
In another recent interview, Hinton further talked about why he is afraid of AI development.
Hinton, 75, left Google a few days after the news broke in an interview with MIT Technology Review senior editor Will Douglas Heaven.
Hinton points out that the new generation of AI language models, especially the GPT-4, which came out in March, made him realize that these machines were moving in a direction that was much smarter than he thought, and he was afraid of these developments.
He is best known for a technique he proposed in the 1980s (with two colleagues) called backpropagation.
In short, the algorithms that allow machines to learn support almost all of today ’ s artificial neural networks, from computer vision systems to large language models.
It wasn ’ t until the 2010s that Hinton and his two recent graduates developed a “neural network ” that demonstrated its power, stronger than any computer ’ s ability to identify objects in images, and trained it to predict the next letter of a sentence, a precursor to today ’ s large-scale language patterns.
One of the students was Ilya Sutskever, the founder of OpenAI, who led the development of ChatGPT.
“We first thought it might be great. ”“ But it ’ s going to take a long time to realize that it needs to be done on a large scale to get good results. ”
In the 1980s, the discussion of “neural networks ” was considered a joke.
But Hinton doesn ’ t think so, and is working on this, software that mimics the abstract human brain and uses code as a connection between neurons. For 40 years, Hinton has viewed artificial neural networks as a clumsy attempt to mimic biological neural networks.
Now he thinks things have changed: he thinks there are better results when trying to mimic the behavior of the biological brain.
“It ’ s scary when you see it with your own eyes, ” he said.“ It ’ s a sudden change. ”
So why did ChatGPT surprise Hinton, too? He explained that the human brain has about 100 trillion connections, and “large language models have up to 500 billion, up to a trillion. ”
However, the knowledge of GPT-4 is a hundred times greater than that of any human.
So maybe it actually has a better learning algorithm than we do. "
He also pointed out that apart from the ability to learn, the ability to communicate is another kind of point.
“If you or I learn a new thing and want to teach it to another person, we can ’ t just send them a copy. ”
But artificial neural networks can do it.
It ’ s like there are 10,000 people here, and when one person learns a new thing, the other 9,999 people all learn it at the same time.
Hinton now believes that there are two types of intelligence: animal brains and artificial neural networks.
“It ’ s a completely different kind of intelligence.
“A new, better intelligence. ” Hinton believes that whether superintelligence is good or bad depends largely on whether you ’ re an optimist or a pessimist, and he says,“ mildly depressed, ” “that ’ s why I ’ m afraid. ”
“Whether these things are going to be smarter than us, I suddenly changed my mind. ”
I think they are very close to us now and they will be much smarter than us in the future, ” he said.
“How do we survive? ” he is particularly worried that people will use the tools he has developed to change the trajectory of human life, including elections and wars.
He said that he hopes to continue talking about AI security after leaving office and spend more time on “more philosophical work ”.“ As long as I ’ m still employed at Google, I can ’ t say what I want to say, ” he said. “I have a lot of good things to say about Google, and it would be more convincing if I wasn ’ t at Google. ”
Who is the godfather of AI computer scientist Geoffrey Hinton?
Hinton pioneered the basic technology of AI systems and became interested in artificial neural networks when he was a graduate student at the University of Edinburgh in 1972. In 2018, he and two experts in the field of deep learning were awarded the Turing Award, known as the “Nobel Prize in Computing ”.
What are the main concerns of “AI Godfather ” Geoffrey Hinton about the development of artificial intelligence?
He believes that the most direct impact is to promote the dissemination of false information and the loss of employment opportunities, and also worries about the use of this tool for evil, such as influencing elections and using it in war.

Hinton's warnings have sparked debate about the potential risks and benefits of AI development.

Title:

Content: Artificial intelligence: Geoffrey Hinton 'godfather' of modern technology resigns from Google and warns of its danger
Jeffrey Hinton, widely regarded as the spiritual father of artificial intelligence, has resigned, warning of the growing dangers of AI development.
Hinton, 75, announced his resignation from Google in a statement to the New York Times, saying he now regrets his work.
He told the BBC that some of the dangers of smart chatbots were "very scary".
“Right now, they ’ re not as smart as we can tell.
But I think they ’ ll be soon. ”
Hinton ’ s groundbreaking research on deep learning and neural networks has paved the way for artificial intelligence systems such as ChatGT.
But British-Canadian cognitive psychologist and computer science specialist Hinton told the BBC the chatbot could soon surpass the level of information held by the human brain.
Skip the podcast and keep reading.
Podcast
BBC Extra
A weekly podcast that presents humanitarian stories about the Arab world and its youth.
Loops
Podcast End
“Right now, what we ’ re seeing is that things like GPT-4 are far superior to a person in the amount of general knowledge they have.
For thinking, it ’ s not good, but it really works on simple thinking. ”
"Given the rate of development, we expect things to improve quickly.
So we have to worry about that. ”
In the article he sent to the New York Times, he pointed out that “bad actors ” may try to use artificial intelligence to do“ bad things. ”
In response to the BBC's request for further clarification on the matter, he said: "This is just kind of a worst-case scenario, kind of a nightmare scenario."
“Can you imagine, for example, that someone as bad as the Russian president (Vladimir Putin) decided to give robots the ability to create their own sub-goals? ”
In the end, he warned, this could “create sub-goals – for example, ‘ I need to get more power. ’ ”
"I have come to the conclusion that the kind of intelligence we develop is very different from the intelligence we have," he said.
“We are biological systems and these are digital systems.
The big difference is that with digital systems you have many copies of the same weight set, of the same world model. ”
"All of these versions can learn separately, but they share their knowledge immediately.
If you have 10,000 people, when one person learns something, everyone will automatically know it.
So chatbots can know much more than one person. ”
Hinton also spoke of several other reasons why he quit his job.
“One of them is that I am 75 years old now.
And the other is that I want to say good things about Google.
It would be more credible if I didn ’ t work for Google. ”
He stressed that he did not want to criticize Google, and that the company was "very responsible".
Google Chief Scientist Jeff Dean said: “We remain committed to a responsible approach to artificial intelligence.
We are constantly learning to understand the risks arising under innovation boldly as well. ”

What did Hinton, the renowned British-Canadian expert reveal about AI that caused widespread concern?

Exposed serious AI risks, shocking the tech world and affecting major company stocks.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

The public should understand that AI can bring significant benefits, but also poses risks, and requires human oversight to ensure safe development and deployment.

Title:

Content: AI Godfather Hinton: Artificial intelligence threatens humanity more urgently than climate change
Central News Agency
Geoffrey Hinton, a pioneer in artificial intelligence (AI), said in an interview with Reuters today that the threat posed by AI to humans may be “more urgent ” than climate change.
This is an advertisement, please continue reading.
Hinton, 75, who is regarded as one of the “godfathers ” of artificial intelligence, recently announced his departure from Alphabet, a 10-year tech company, because he wanted to make the risks of artificial intelligence public without involving employers.
Hinton ’ s research has been recognized as an important contribution to the development of contemporary AI systems. In 1986, he co-authored the influential paper Learning Representations by Back-propagating Errors, which set a milestone in the development of neural networks supporting AI technology. In 2018, he received the Turing Award for his research breakthroughs. And...
Hinton is one of a growing number of technology leaders who are concerned that machine intelligence will surpass humans and ultimately control the planet, and are openly raising concerns about the potential threat posed by AI.
“I ’ m not trying to downplay the importance of climate change, nor am I saying we shouldn ’ t worry about climate change, which is also a big risk, ” Hinton said.
But I think (artificial intelligence) may be more urgent in the end. "
“When it comes to climate change, making recommendations is simple: just stop carbon emissions.
If you do that, things will get better.
And when it comes to (artificial intelligence), there ’ s nothing we can do about it. ”
OpenAI, a U.S. artificial intelligence research lab sponsored by technology company Microsoft, launched ChatGPT, a chatbot that took the first shot at the AI race last November and quickly became the fastest-growing app in history, with 100 million monthly active users in just two months.
In April, thousands of people signed an open letter calling on all AI labs to suspend development of systems more powerful than OpenAI ’ s recently launched GPT-4 for at least six months, including social media CEO Elon Musk, AI pioneers Yoshua Bengio and Stuart Russell.
Hinton shares the concerns of the co-authors, who fear AI could pose an existential threat to humanity, but he disagrees with the suspension of research.
He said, “It ’ s not practical.
I agree that this is an existential risk, and it ’ s close, and it ’ s time to put a lot of effort into understanding what we can do. ”
Following the publication of the joint letter, the 12 members of the European Parliament called on leaders such as U.S. President Joe Biden and European Union Executive Committee Chairman Ursula von der Leyen to convene a summit to discuss how to control the development of advanced artificial intelligence systems such as ChatGPT, which are developing faster than expected.
“Technology leaders know best about [artificial intelligence], and politics must be involved, ” says Hinton.
This matter concerns all of us, so we must all think carefully. ”
This is an advertisement, please continue reading.

What did Sam Altman admit about ChatGPT and the potential risks of AI?

Advanced AI could pose an existential risk to humanity.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Hinton's decision to leave Google and speak out about AI dangers highlights the need for a nuanced discussion prioritizing human well-being and safety in AI development.

Title:

Content: 28 June 2023, 07: 34
Artificial intelligence called as dangerous as pandemics and nuclear war
The Week: the development of artificial intelligence will lead to the destruction of humanity
Read 360tv in
The development of artificial intelligence threatens humanity with extinction.
In terms of the danger to life on Earth, it was compared to pandemics and nuclear war, The Week reported.
British scientist Geoffrey Hinton, who was nicknamed the godfather of artificial intelligence, said that the mass upheaval and destruction of humanity due to the “uprising of machines ” has ceased to be science fiction.
In his opinion, the development of technology really poses a mortal danger to people.
And if earlier he predicted it in 30 years, now, with the current pace of development of neural networks, he has reduced this period to five years.
Advertisement
The same opinion was expressed by other experts.
More than 350 AI researchers and engineers have issued a warning that AI poses a danger comparable to the risks of pandemics and nuclear war.
While artificial intelligence is being trained and not so dangerous.
In the company "Geoscan" denied the information about the threat of an uprising of drones based on them.
Advertisement
Advertisement

What did Hinton, the renowned British-Canadian expert reveal about AI that caused widespread concern?

Exposed serious AI risks, shocking the tech world and affecting major company stocks.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Why did Hinton leave Google when he helped develop AI but later grew concerned about its dangers?

AI regulation and governance is complex, with countries taking various approaches to ensure transparency, accountability, and safety.

Title:

Content: Artificial Intelligence (AI) has the potential to revolutionize various sectors of society, including the economy, security, and science. However, along with its benefits, it also poses risks that must be addressed. Governments around the world are grappling with the challenge of regulating this rapidly evolving and complex technology.

Different countries have adopted various approaches to understanding and regulating AI. Some encourage the voluntary adoption of ethical principles, while others have implemented specific regulations or utilized existing frameworks. The prevailing trend internationally leans towards a risk-based approach to AI governance.

Several countries, such as Australia, the United States, the United Kingdom, and the European Union, have made efforts to regulate AI. Australia has published a discussion paper focused on safe and responsible AI usage, while the White House has proposed an AI Bill of Rights blueprint. The UK has taken a pro-innovation stance towards AI regulation, while the European Union is in the process of passing the world’s first comprehensive AI law, known as the EU AI Act.

Most regulation efforts primarily concentrate on governing AI during its creation or use. However, an opportunity exists to regulate AI through the control of its compute capacity, encompassing both hardware and software components. Compute capacity is essential for AI applications and is predominantly concentrated in a handful of companies and democratic countries that uphold the rule of law.

Considering the limited supply chain of semiconductors used in compute capacity, similar regulatory frameworks utilized for hardware export controls could be applied to software. This would involve proactive regulation of commercial providers of computing capacity and implementing controls during compute processes. Such an approach ensures responsible usage of AI models and requires compute and cloud providers to undertake risk assessments of their services and customers.

This compute chokepoint regulation could also be extended to other emerging technologies like quantum and biotech, where similar supply chain dynamics exist. The objective of regulation within these domains is not to impede innovation but rather to protect system integrity and processes while allowing for responsible and beneficial technology usage.

In conclusion, the regulation of AI and emerging technologies necessitates an understanding of their supply chains and chokepoints. Governments must strike a balance between promoting innovation and ensuring the proper governance of these technologies to prevent potential misuse.

The EU regulates AI by risk, while the US is developing a voluntary code, underscoring the need for international cooperation.

Title:

Content: Corporate AI leaders have met with President Joe Biden to discuss AI safety as part of a voluntary agreement.

In a Friday meeting with President Biden, the leaders of America’s biggest AI-focused companies, including Meta, Amazon, Google, Microsoft, Anthropic, Inflection AI, and OpenAI, pledged to continue their efforts in ensuring the safety of AI.

However, there are concerns that the voluntary agreement between the Biden administration and leading AI companies may be more of a symbolic gesture rather than aggressive regulation.

Many tech billionaires have become fixated on hypothetical doomsday scenarios in which AI becomes fixated on a specific goal and triggers human extinction in its pursuit, Forbes’s Johanna Costigan wrote.

She claimed their fascination with these stories is interconnected with their belief in longtermism, a philosophy that urges the making of short-term trade-offs for the sake of humanity’s long-term well-being, adding:

“Conveniently, this concept allows them to continue developing profitable technological applications while claiming paternalistic, karmic excellence: they are not only bestowing life-optimizing tech to the masses, they are also single handedly ensuring the tech they make doesn’t go rogue and kill us all.”

President Biden’s acknowledgment that these executives are critical for making sure AI develops “with responsibility and safety by design” also demonstrates that the White House reflects this narrative.

Costigan claimed that by adhering to the ideals of “AI safety” as defined by tech companies, issues such as copyright infringement and the presence of sexist and racist algorithms are being ignored.

These immediate challenges posed by AI are dismissed as mere risks, overshadowed by grandiose hypothetical threats, she wrote.

The term “AI safety” is deliberately vague, allowing different stakeholders to interpret it in their own ways.

It can refer to job security, trust in algorithms, or the confidence that facial recognition won’t be abused.

However, the focus on creating “safety” in AI neglects the fact that the data on which AI relies remains entirely unregulated in the United States. This lack of regulation has given AI companies the freedom to design safety standards that align with their own moral and financial interests.

Competition From China Prompts US Government to Avoid Stiff Regulation

China’s domestic development of AI has further intensified the US government’s inclination to step aside in order to allow perceived innovation to thrive, Costigan said.

She noted that while China may have stricter regulations on technology, the average US citizen is more vulnerable to encountering unmarked deepfakes.

In democratic countries, where information shapes public opinion and dictates election outcomes, synthetic images and text generated by AI pose a significant threat.

The recent meeting between the White House and tech CEOs shows a shared commitment to secure AI development, but without data privacy regulations, attempts to make AI safe are akin to “regulate the drinking of wine in restaurants—but not the commercial process of turning grapes into wine.”

Voluntary measures and general commitments are a step in the right direction, but more concrete guidelines and specific plans need to be implemented.

Companies such as Anthropic have announced their intention to share more specific cybersecurity measures and responsible scaling plans.

All in all, while the efforts of AI industry leaders to address safety concerns are commendable, there is a need for greater scrutiny and accountability.

The dominance of these companies in shaping the narrative around AI risks and safety has allowed them to evade immediate challenges such as data privacy and algorithmic biases.

AI Regulation Finds Momentum Amid Growing Adoption

Last month, a bipartisan group of lawmakers in the US introduced the National AI Commission Act, which aims to create a blue-ribbon commission to explore the regulation of AI.

The proposal would establish a 20-member commission to study AI regulation, including how responsibility for regulation is divided between agencies, the capacity of such agencies to regulate, and ensuring that enforcement actions are aligned.

Members of the commission will come from civil society, government, industry, and labor and will not be dominated by one sector.

The move came as the skyrocketing popularity of AI tools has led to increasing calls for regulations.

As reported, the European Union and the US announced collaborating to develop a voluntary code of conduct for AI. Officials will seek feedback from industry players and invite parties to sign up to curate a proposal for the industry to commit to voluntarily.

Likewise, the EU has been working on its Artificial Intelligence Act, a sprawling document in the works, for around two years. The legislation aims to classify and regulate AI applications based on their risk.

These attempts to regulate AI come as the technology is well on the path to finding mainstream adoption.

According to a recent survey from the Pew Research Center, 27% of Americans interact with AI on a daily basis.

The survey found that another 28% of individuals encounter AI about once a day or several times a week.

Furthermore, almost all businesses (97%) expect to use AI to improve their efficiencies and cut costs.

Many businesses see promise in the usefulness of AI, such as streamlining email communications with colleagues (46%), generating website copy (30%), fixing coding errors (41%), translating information (47%), and summarizing information (53%).

Read More:

What's the Best Crypto to Buy Now?

  • B2C Listed the Top Rated Cryptocurrencies for 2023
  • Get Early Access to Presales & Private Sales
  • KYC Verified & Audited, Public Teams
  • Most Voted for Tokens on CoinSniper
  • Upcoming Listings on Exchanges, NFT Drops

Hinton's warnings about AI stress the need for a careful approach that prioritizes human well-being and safety.

Title:

Content: Governments, firms should spend more on AI safety, top researchers say

By

'At least one third of their AI research and development funding'.

Artificial intelligence companies and governments should allocate at least one third of their AI research and development funding to ensuring the safety and ethical use of the systems, top AI researchers said in a paper.

The paper, issued a week before the international AI Safety Summit in London, lists measures that governments and companies should take to address AI risks.

"Governments should also mandate that companies are legally liable for harms from their frontier AI systems that can be reasonably foreseen and prevented," according to the paper written by three Turing Award winners, a Nobel laureate, and more than a dozen top AI academics.

Currently there are no broad-based regulations focusing on AI safety, and the first set of legislation by the European Union is yet to become law as lawmakers are yet to agree on several issues.

"Recent state of the art AI models are too powerful, and too significant, to let them develop without democratic oversight," said Yoshua Bengio, one of the three people known as the godfather of AI.

"It (investments in AI safety) needs to happen fast, because AI is progressing much faster than the precautions taken," he said.

Authors include Geoffrey Hinton, Andrew Yao, Daniel Kahneman, Dawn Song and Yuval Noah Harari.

Since the launch of OpenAI's generative AI models, top academics and prominent CEOs such as Elon Musk have warned about the risks on AI, including calling for a six-month pause in developing powerful AI systems.

Some companies have countered this, saying they will face high compliance costs and disproportionate liability risks.

"Companies will complain that it's too hard to satisfy regulations - that 'regulation stifles innovation' - that's ridiculous," said British computer scientist Stuart Russell.

"There are more regulations on sandwich shops than there are on AI companies."

Got a news tip for our journalists? Share it with us anonymously here.
Tags:

Most Read Articles

Services Australia's digital credential sharing with NSW on ice

CBA to test products and messaging on AI-created 'customers'

Australian Federal Police officers using more online face search tools

ANZ targets 3000 engineers to use GitHub Copilot

Log In

  |  Forgot your password?

What did Hinton, the renowned British-Canadian expert reveal about AI that caused widespread concern?

Exposed serious AI risks, shocking the tech world and affecting major company stocks.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Developing AI systems that prioritize human well-being and safety requires a multidisciplinary approach involving computer science, ethics, and social sciences experts.

Title:

Content: The AI Paradox: Path to Utopia or Dystopia?

Explore the dual potential of AI to create either a utopia of abundance and freedom or a dystopia filled with challenges. How we manage AI now will decide our future.

Artificial intelligence (AI) is rapidly transforming our world, posing significant questions about its potential to lead us towards either a utopian future of abundance and freedom or a dystopian future fraught with ethical and societal challenges.

Utopian Visions: AI as a Catalyst for Abundance

Philosophers like Nick Bostrom suggest that AI could create a “post-scarcity utopia” where technology alleviates the drudgery of daily life, allowing people more time for pleasure and meaningful activities. This vision hinges on the idea that AI and automation could dramatically increase productivity, creating wealth and resources that, with the right policies, could be distributed equitably among all people.

Dystopian Concerns: The Risks of Advanced AI

Conversely, the same technologies that promise to make life easier also pose risks. AI could potentially displace millions from their jobs, not just in blue-collar sectors like transportation but also in white-collar professions through systems like IBM’s Watson. The long-term fear is that superintelligent AI, if not aligned with human values, could make decisions detrimental to human welfare.

Current Realities and Ethical Considerations

As AI technologies like deep learning continue to evolve, they are already impacting various sectors by optimizing tasks that were previously thought to rely solely on human intelligence. However, this raises significant ethical questions regarding privacy, security, and the displacement of jobs. Ensuring that AI benefits society as a whole requires careful consideration of these ethical dimensions.

The Role of Policy and Regulation

The debate between AI as a utopian or dystopian force is not just technological but also deeply political and ethical. Effective policies and international cooperation will be crucial to manage AI’s development responsibly. This involves proactive regulation to prevent misuse and ensure that AI’s benefits are distributed fairly across society.

Looking Ahead: Preparing for All Possibilities

While the ultimate outcome of AI’s impact on society remains uncertain, it is essential to prepare for all possible futures. This includes investing in AI safety research and developing robust legal and ethical frameworks to guide AI development. The goal is to harness AI’s potential while mitigating its risks, ensuring it serves as a tool for enhancing human life rather than diminishing it.

AI presents a paradox that embodies the best and worst of technological advancements. Whether AI leads us towards a utopia or a dystopia depends significantly on how we manage its development in the coming years. The choices we make today will shape the future of AI and its impact on humanity.

It requires transparency, accountability, and fairness, and a willingness to address AI development risks.

By prioritizing human well-being and safety, we can ensure AI's benefits are realized while minimizing its risks.

What did Sam Altman admit about ChatGPT and the potential risks of AI?

Advanced AI could pose an existential risk to humanity.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

AI systems can perpetuate discrimination, requiring human oversight and accountability.

Title:

Content: Presented By
Getty Images
Ideas
Jan Brauner is a Ph.D. researcher in Artificial Intelligence at the University of Oxford, focusing on the safety of AI systems and their application in public health.
Alan Chan is a Ph.D. student at Mila, a research institute in Montreal focusing on AI, where he works on technical and socio-technical problems in AI safety. He's particularly interested in evaluation and AI regulation.

In May, more than 100 artificial intelligence (AI) professors, including some of the field's leading experts, scholars from other areas, and AI executives signed a one-sentence statement: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” Many signing researchers have explained their reason for worry: the risk that we might build powerful AI systems that we cannot control within this decade or the next ones.

Some have suggested the statement is an industry ploy to hype up AI companies’ products or influence regulation, ignoring that most of the signatories—ourselves included—are academics who do not work for industry. But the most common concern we have heard is that, by focusing on the risk of extinction, the statement distracts from present-day AI harms. As volunteers involved in collecting signatures for and constructing the statement, we disagree because both problems share a common core. The false dichotomy between ongoing harms and emergent risks makes it needlessly difficult to address common root causes.

AI poses many ongoing problems. AI systems can sustain human rights abuses, perpetuate systemic discrimination, and entrench power imbalances. Because AI systems are often widely deployed, harms can reach massive scales. For example, tens of thousands of families were pushed into poverty when the Dutch Tax and Customs Administration wrongly accused them of fraud and requested they pay back large sums of money, based on risk profiles created with AI systems.

The root causes of AI harm

There are several root causes that contribute to many instances of the harm caused by AI. One is that we do not know how to design AI systems that reliably behave exactly as we intend them to. For example, a health system was designed to allocate care based on need but discriminated against Black hospital patients. It turned out the algorithm used health care costs as a proxy for need, but institutionalized racism means that less money has been spent on Black patients historically. In many similar cases, companies do not explicitly set out to build harmful AI systems. Instead, creating state-of-the-art AI systems requires vast amounts of data, and the AI systems unintendedly inherit the biases encoded in this data.

Another cause is competitive pressures and limited accountability. There is often a strong economic incentive to deploy AI systems without sufficient safety checks. For example, Microsoft rushed the release of the Bing chatbot, which resulted in the chatbot threatening and insulting users. When the Dutch Tax Administration replaced human judgment with AI systems, this, too, happened in the name of efficiency.

Without decisive action to address these root causes, increasingly powerful and widely deployed AI systems will cause even more pervasive and significant harm. Many experts believe future AI systems may outmatch humans in crucial cognitive skills, such as social persuasion, planning, and development of novel technologies. Just as these cognitive abilities enabled humanity to control the fate of other species, the same skills could enable smarter AI systems to out-perform us. Given competition and limited accountability, there would be pressure on organizations to replace expensive human labor with AI systems. Economic forces encourage the rapid deployment of new AI systems (at the cost of safety testing), making them ever more powerful, building general (and potentially more powerful) systems rather than narrow (and potentially more controllable) systems, and increasing the autonomy of AI systems to take action in the world. AI systems could increasingly control our society's key productive, managerial, and military capacities. By then, we must have figured out how to design systems that always and truly behave as we intend. Otherwise, powerful AI systems that pursue goals at odds with humanity could lead to human marginalization or even extinction.

Read More: An AI Pause is Humanity's Best Bet for Preventing Extinction

Moreover, many researchers expect us to develop such advanced AI systems very soon. Companies are pouring more money and talent into AI development while the cost of computing power is decreasing. Simply using more computing power and data has reliably produced more capable AI systems, while AI algorithms have also improved. After seeing the rapid progress in AI over the past five years, Geoffrey Hinton, a founding father of modern AI, now expects that AI could broadly surpass human cognitive abilities—such as persuasion, planning, and scientific reasoning—within five to 20 years. Such time horizons are shared by many in the AI research community.

Twenty years is short; in discussions about the climate crisis, we regularly consider effects on such timescales. And just as climate change requires us to face air pollution and ongoing natural disasters as well as prevent future, even more, extreme weather events, we must tackle various AI risks and harms simultaneously. Even if we might be a couple of decades away from AI systems that are smarter than humans, that is little time to create the complex global governance structures necessary for this technology.

Work to address ongoing harms sometimes differs from work to manage extinction risk. We do not wish to erase nuance or potential differences between the perspectives. Yet, given common root causes, there is much room for collaboration and cooperation.

Four possible interventions

Many interventions could address the root problems. First, the public currently has little to no say over what models are built and how they are deployed. For example, those most affected by welfare systems have had little say in their automation. Private companies decide which risks are acceptable, as they have free rein over computing power and data, the key levers of AI development. Stronger democratic oversight over the supply of computing power and data would enhance public control over AI development and associated risks. For example, governments can monitor when large amounts of computing power are being used and for what purpose.

Second, a strong auditing regime, where independent third-parties would scrutinize the practices and development processes of AI labs, would help reduce risks overall. Auditing has already surfaced previously overlooked harms, such as bias in facial recognition, and could catch future failure modes like AI systems subverting human control or replicating autonomously. A standards body could ensure  AI systems, both before and after deployment, meet specified safety guidelines. Domestic agencies could enforce compliance, while an international organization would create universal standards.

Third, we should require meaningful human oversight of critical AI decisions, and avoid very-high-risk use cases such as lethal autonomous weapons. Uncertainty about AI systems' current and near-future capabilities means we should proceed cautiously around their deployment in critical domains.

Fourth, we should rebalance the  funding going into AIs and urge companies to spend less on making them smarter while increasing funding into making them safer, more transparent, and studying their social impacts. While it is good that industry labs are funding such research, this funding is liable to change with market conditions. Much more public funding for non-industry groups is required.

The development of increasingly powerful AI systems poses increasingly vast societal-scale risks. The complexity of these risks requires cooperation between people with potentially different motivations but with a shared interest in ensuring that AI development proceeds in a just and beneficial way.

More Must-Reads From TIME

Contact us at letters@time.com.

You May Also Like
EDIT POST

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Competitive pressures and limited accountability contribute to AI harm, highlighting the need for robust governance.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Experts believe future AI systems may surpass humans cognitively, requiring human oversight to mitigate risks.

What did Sam Altman admit about ChatGPT and the potential risks of AI?

Advanced AI could pose an existential risk to humanity.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

What future risk did Hinton warn about AI?

Stronger democratic oversight is needed to ensure AI aligns with human values.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Auditing regimes can help reduce AI risks and provide protection against potential AI-related harms.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Human oversight is necessary for critical AI decisions, especially in high-risk areas like autonomous weapons and healthcare.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

What risks did Hinton associate with AI's rapid advancement?

What severe risks did the March 2024 report warn about advanced AI models?

Development of uncontrollable human-targeting weapons

Public funding is necessary to study AI's social impacts and develop safer, more transparent systems.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

The EU AI Act regulates AI systems based on risk levels to ensure safe and responsible use.

Title:

Content: European Law for the Regulation of Artificial Intelligence (AI)
The regulation of AI is crucial for a number of reasons, and is primarily about ethical, safety and social considerations.
The rapid advancement of AI technologies poses potential risks that require careful oversight and governance.
It reflects concerns about the ethical, legal and social implications of AI technologies and the European push for a law to regulate the use of AI.
Add Announcement
The EU ’ s approach to regulating AI is based on principles aimed at ensuring the development and deployment of AI systems in a responsible and human-centred manner.
The EU emphasizes the development of artificial intelligence that is in line with human values and rights.
The new regulations aim to promote the ethical use of AI, ensure that AI systems respect fundamental rights such as privacy and non-discrimination, and disseminate them in ways that benefit individuals and society.
European regulations also seek to promote transparency and accountability in AI systems.
The goal is to avoid a “black box ” of artificial intelligence, where decision-making processes are ambiguous.
Thus, the EU recognizes the importance of ensuring the safety and reliability of AI systems, especially in critical areas such as healthcare, transportation and manufacturing.
The law sets standards for risk assessment, testing and verification to reduce the potential harm caused by faulty or unsafe AI applications.
Privacy and data protection is another issue that makes the EU highly focused on data protection and privacy.
Artificial intelligence often relies on comprehensive data sets, and regulations such as the General Data Protection Regulation (GDPR) set out guidelines for the legal and ethical processing of personal data.
AI developers must abide by these regulations to protect the privacy of individuals.
Furthermore, European regulations regarding AI emphasize the importance of human supervision and control of AI systems.
Even as AI grows in governance, the EU is seeking to ensure that humans remain in control, especially in critical decision-making processes.
This approach aims to prevent the delegation of authority in decision-making in favor of artificial intelligence.
The EU also aims to promote innovation and competitiveness while maintaining high ethical standards.
A unified regulatory framework can create an equal playing field for companies operating in the EU and contribute to the EU ’ s global leadership in AI governance.
A unified regulatory approach across the EU provides legal certainty for businesses and citizens.
Standardization helps avoid fragmentation and incoherent interpretations of AI-related regulations, creating a more predictable environment for the development and use of AI technologies.
The EU Council is expected to formally approve AI legislation by May 2024.
It will be fully effective 24 months after its entry into force, and it concerns the “Regulation for Determining the Uniform Rules on Artificial Intelligence (AI Law) ”.
It ranks AI systems based on their risk levels and includes provisions on transparency, accountability, non-discrimination and human stewardship.
The regulations, once adopted, will apply to artificial intelligence systems used in various sectors.
Given the impact of new European regulations on innovation and humanitarianism, it is a complex and evolving issue.
The aim of these regulations is to strike a balance between promoting AI innovation and ensuring that it is developed and used ethically and responsibly to protect human rights and public safety.
The regulations aim to encourage innovation in artificial intelligence while setting clear ethical limits.
The EU aims to create an environment in which innovative AI solutions can flourish without compromising core values.
It is hoped that clear rules and guidelines will encourage companies and individuals to adopt AI solutions, so that there are safeguards that protect against misuse and ensure responsible practices.
The EU ’ s efforts to regulate AI also make it a global leader in AI governance.
This law may impact and set a standard for other regions, contributing to the creation of international standards for ethical AI.
While large companies may have the resources to adapt to regulatory requirements, smaller entities may face challenges.
It is essential to reconcile encouraging innovation with not placing a disproportionate burden on small players.
The overall goal is to exploit the benefits of artificial intelligence while preserving its benefits to humans, and the regulations reflect a commitment to develop artificial intelligence in a responsible and human-centered manner.
Continuous assessment and adaptation to legal regulations will be necessary to strike a balance between innovation and ethical considerations.
In conclusion, AI regulation is necessary to ensure its ethical use, to prevent the harm it can cause to humanity, and to address the social challenges associated with its deployment.
A well-thought-out and comprehensive regulatory framework necessary to exploit the benefits of AI while minimizing its risks and protecting the well-being of individuals and communities is a necessity to think about.
Will the Jordanian authorities benefit from such legislation to regulate the technological sector and benefit from international experiences to develop this vital sector of the Jordanian economy?

High-risk AI systems are subject to stringent regulations and human oversight.

Title:

Content: The EU AI Act Overview

PS
Papilio Services Limited

Contributor

Papilio Services Limited, established in 2012, is based in Malta with sister companies in the Netherlands and the Czech Republic. The firm boasts a multinational team and a diverse client base, providing cross-border solutions in Corporate, Tax Compliance, and Residency services on a global scale.
The European Parliament achieved a key milestone by adopting the landmark Artificial Intelligence Act, paving the way for the European Union's first complete legal framework for artificial intelligence (AI).
European Union Technology
To print this article, all you need is to be registered or login on Mondaq.com.

The European Parliament achieved a key milestone by adopting the landmark Artificial Intelligence Act, paving the way for the European Union's first complete legal framework for artificial intelligence (AI). This pioneering law seeks to achieve a compromise between the AI industry's need for innovation and the imperatives of upholding fundamental rights and guaranteeing the security and reliability of AI systems. The EU's determination to lead the world in ethical AI governance is reflected in the EU AI Act which upholds citizens' rights and interests while promoting a strong technological environment.

Ensuring Safety and Protecting Rights

The primary objective of the EU AI Act is to establish a regulatory environment that ensures AI systems deployed within the EU are safe, transparent, and respect fundamental rights. This is achieved through a risk-based approach that categorises AI systems into different risk levels, each with corresponding regulatory requirements. High-risk AI systems, which pose significant risks to health, safety, or fundamental rights, are subject to stringent regulations, including rigorous testing, documentation, and human oversight.

The EU AI Act mandates that AI systems must be designed and developed in a manner that upholds fundamental rights, such as privacy, non-discrimination, and data protection. This emphasis on fundamental rights reflects the EU's commitment to creating a human-centric AI landscape, where technology serves the needs and values of society. By prioritizing the safety and rights of individuals, the legislation aims to build public trust in AI technologies, which is crucial for their widespread adoption and acceptance.

Boosting Innovation and Competitiveness

The EU AI Act aims to encourage innovation and competitiveness in the AI industry even as it places stringent limitations on high-risk AI systems. The law contains provisions intended to aid in the creation and application of AI technology, especially for startups and small and medium-sized businesses (SMEs). The intention is to foster innovation by lowering administrative burdens and offering clear legislative rules, which will promote the development of a thriving AI ecosystem in Europe.

Focusing on encouraging cooperation between the public and private sectors is one of the main features of the EU AI Act. To guarantee that AI technologies are created in accordance with societal needs and values, the act promotes collaboration and partnerships between researchers, developers, and regulatory organisations.

By adopting a risk-based approach to regulation, it recognises that not all AI systems pose the same level of risk. The legislation classifies AI systems into three main categories: unacceptable risk, high risk, and limited risk. This tiered approach allows the EU to tailor its regulatory framework to the specific risks associated with different AI applications, ensuring that regulations are proportionate and effective.

Transparency and Accountability

Transparency and accountability are key to the EU AI Act's principles. The law requires AI systems to be open about how they operate and make decisions, especially those that are deemed high-risk. This entails giving consumers information that is easy to comprehend regarding the operation of AI systems, the data they use, and the reasoning behind their choices.

The statute mandates that high-risk AI systems be created in a way that permits efficient human intervention, underscoring the significance of human control. This guarantees that people maintain control and have the ability to step in when needed to stop damage or fix mistakes, and to ensure that AI technologies are used responsibly and ethically by increasing accountability and transparency.

EU Leadership in AI Governance

The adoption of the EU AI Act establishes the EU as a global leader in AI governance, laying the groundwork for other governments and regions to follow. The EU wants to ensure that AI technologies are used responsibly and ethically, making a positive impact on society, by creating a thorough legal framework.

The EU's commitment to advancing global collaboration in AI governance is also reflected in the legal framework. The law has measures for working together with foreign partners, including exchanging best practices and creating international standards. The EU aims to provide a harmonised legal framework that protects fundamental rights, ensures the safe use of AI technologies, and promotes innovation through international cooperation.

EU AI Act Next steps

The regulation will enter into force 20 days after its publication in the official journal and be directly applicable to 27 Member States, including Malta, on 2nd August 2026, except for bans on prohibited practises, which will apply 6 months after the entry into force date; codes of practise (9 months after entry into force); general-purpose AI rules including governance (12 months after entry into force); and obligations for high-risk systems (36 months).

Businesses will have to ensure alignment with the following:

  • Inventory all AI systems that you have (or will have) developed or deployed, and establish if any of them are subject to the EU AI Act.
  • In order to ascertain the risk categorization of the in-scope AI systems and the relevant compliance requirements, evaluate and classify the systems.
  • Acknowledge your company's place in pertinent AI value chains, the related compliance requirements, and the means by which these requirements will be fulfilled. All functions accountable for the AI systems along the value chain must incorporate compliance throughout the systems' lifecycle.
  • Understand the additional risks (such as interacting with other EU or non-EU rules, such as those pertaining to data privacy) and opportunities (such as providing access to AI Act sandboxes for innovators, small and medium-sized businesses, and others) that the AI Act presents to the strategy and operations of your organisation.
  • Develop and implement a plan to guarantee that, upon the Act's implementation, the proper risk and control systems, quality management, documentation, accountability and governance frameworks, and monitoring are in place.

The purpose of this article is to provide a general overview of the subject matter. Should you require additional information please refer to Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.

See More Popular Content From

Mondaq uses cookies on this website. By using our website you agree to our use of cookies as set out in our Privacy Policy.

Learn More

Global regulations are necessary for responsible AI use in high-risk applications.

Title:

Content: Artificial intelligence (AI) is a rapidly growing technological advancement with the potential to transform various industries. The development of AI has brought about significant changes in education, music, transportation, and healthcare. It holds the promise of self-driving cars, revolutionary medical breakthroughs, and increased productivity. However, concerns arise regarding the potential dangers associated with the development of General Artificial Intelligence (AGI).

AGI is a theoretical form of AI that possesses human-level intelligence and self-awareness. While Narrow AI, designed for specific tasks, has made impressive progress, achieving AGI remains uncertain. One of the main challenges is developing algorithms that can reason abstractly and think creatively, skills that current AI systems lack. Replicating human-like abstract and creative thinking in machines is a complex task.

In the medical field, AI shows great potential in diagnosing diseases and developing more effective treatments. It can analyze big data to prevent the spread of infectious diseases by creating early-warning systems. In other areas, such as climate change and transportation, AI can contribute to scientific advancements. Predictive modeling and early-warning systems can help us prepare for climate-related events, while self-driving cars have the potential to revolutionize transportation.

However, ethical concerns arise when it comes to AI’s impact on the arts, particularly in music production. Questions arise regarding the role of human artists and the authenticity of AI-generated music. Additionally, the use of AI in military technology raises ethical questions about decision-making without human intervention. Countries like the United States, China, and Russia have already implemented AI in weapons systems and drones, sparking the need for global regulations to ensure responsible use.

Renowned physicist Stephen Hawking has warned against the development of AGI, expressing concerns that it could surpass human intelligence and redesign itself at an exponential rate, potentially rendering humanity obsolete. This highlights the importance of ethical considerations, regulations, and international cooperation to mitigate any risks associated with AI development.

As AI continues to advance, it is crucial to strike a balance between harnessing its potential benefits and addressing the potential dangers. Measures such as ethical considerations, regulations, and international cooperation are necessary to ensure the responsible use of AI and to mitigate any risks associated with its development.

Title:

Content: Expert: AI poses serious risk to global health and security China ’ s credit monitoring system oppresses people
A group of doctors and medical experts have urged the urgent need for an international treaty on AI regulation.
A group of doctors and public health experts have joined calls to suspend development of artificial intelligence until sufficient regulations are in place.
Source: SBS
Key points:
• A new study says the development of artificial intelligence could lead to the emergence of autonomous weapons capable of killing people “on an industrial scale. ”
The study ’ s authors also warn that the loss of jobs could have an impact on human health as more types of work are automated.
• The authors also warn that powerful AI-powered surveillance systems could be used to oppress people.
A group of doctors and public health experts say the development of artificial intelligence (AI) needs to be put on hold because it poses a risk to public health and safety.
According to a study published in the open journal BMJ Global Health, AI could facilitate the development of autonomous weapons capable of killing people “on an industrial scale ” without human supervision.
The study ’ s authors, David McCoy and Frederik Federspiel, say governments can also use AI to create powerful surveillance systems that can spread misinformation and social divisions, manipulate consumer choices, and even fuel government oppression.
"We do need an international treaty and a global regulatory approach," McCoy, a professor at the United Nations University in Malaysia, a United Nations General Assembly affiliated research institute and author of the study, told SBS News.
"The UN and governments are really critical to any form of effective regulation to prevent the misuse of AI," he said.
More than 75 countries or regions are currently using AI for surveillance.
Given the link between unemployment and “adverse health outcomes and behaviors, ” the study ’ s authors also warn of the health implications of unemployment as AI facilitates the automation of more types of work.
The creators of AI boast about its transformative potential, including in medicine and public health, but doctors are joining calls to stop developing AI until regulation can keep up with its progress.
How does artificial intelligence enable weapons development?
The authors of the study warn that artificial intelligence has enabled the development of lethal autonomous weapons systems (LAWS) to locate, select and attack human targets, all without human oversight.
"In theory, you could produce a van with hundreds of thousands of small autonomous weapons that could be targeted in a more discriminating way," Professor McCoy said.
The technology for this highly lethal weapon is relatively cheap, he added.
“We don ’ t know exactly who owns this technology or who is developing it.
“But I think it ’ s fair to say that there are governments around the world developing this technology, ” he said.
In 2021, the United Nations held talks on autonomous weapons, which did not start negotiations on an international treaty governing their use, but only agreed to continue discussions.
How does artificial intelligence contribute to the misuse of personal data?
The authors of the study say artificial intelligence has the ability to quickly clean up, organize and analyze massive data sets made up of personal data, including images.
They pointed to the role of AI in subverting democracy in the 2013 and 2017 Kenyan elections, the 2016 US presidential election, and the 2017 French presidential election.
“When combined with a rapidly improving ability to distort or distort reality, AI-powered information systems could further undermine democracy, cause a general breakdown of trust, or drive social divisions and conflict, with consequent public health implications, ” the authors said.
They argue that AI-driven surveillance could also be used by governments to oppress people more directly, such as China ’ s social credit system.
China ’ s social credit system, which combines facial recognition software with analysis of people ’ s financial transactions, activities, police records and social relationships, essentially allows the Chinese government to use a set of incentives and penalties to shape the behavior of its people.
Last week, Geoffrey Hinton, a pioneering computer scientist hailed as the “godfather of artificial intelligence, ” said he had left Google to talk freely about the dangers of the technology after realising that computers could become smarter than people faster than he and other experts expected.
In an interview with The New York Times, Hinton said he was concerned about the ability of AI to create convincing false images and texts, thereby creating a world where people "can no longer know what is real."
"It's hard to see how to prevent bad people from using it to do bad things," he said.
Download the app SBS Audio and follow Mandarin. You can also listen to and download SBS Mandarin audio stories anytime via Apple Podcast, Google Podcast, Spotify and other podcast platforms.
Please follow SBS Chinese for more Australian news.
Share
Published 10 May 2023 6: 18pm
By Madeleine Wedesweiler
Source: SBS
Available in other languages
Labels

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

The development of autonomous weapons raises legal and ethical questions, highlighting the need for new regulatory frameworks.

Title:

Content: The threat from artificial intelligence (AI) autonomous weapons and the need for international cooperation to mitigate the potentiality of “AI killer robots” was re-emphasised at the recent Humanity at the Crossroads: Autonomous Weapons Systems and the Challenge of Regulation conference in Vienna, Austria.

Allowing AI control over weapons systems could mean targets being identified, struck and killed without human intervention. This raises serious legal and ethical questions.

Highlighting the gravity of the situation, Austria’s foreign minister Alexander Schallenberg said: “This is the Oppenheimer Moment of our generation.”

Current use of AI killer robots

Indeed, to what extent the genie is already out of the bottle is a question in itself. Drones and AI are already widely used by militaries around the world.

GlobalData defence analyst Wilson Jones tells Army Technology: “The use of drones in modern conflict by Russia and Ukraine, by the US in targeted strike campaigns in Afghanistan and Pakistan and, as recently revealed last month, as part of Israel’s Lavender programme, shows that AI’s ability to process information is already being used by world militaries to increase striking power.”

Investigations from The Bureau of Investigative Journalism into drone warfare of US brought to light the repeated airstrikes by the US military killing civilians in Pakistan, Afghanistan, Somalia and Yemen. More recently, the IDF’s Lavender AI system has been used to identify tens of thousands of targets, with civilians killed as a result of the strikes.

How well do you really know your competitors?

Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.

Company Profile – free sample

Thank you!

Your download email will arrive shortly

Not ready to buy yet? Download a free sample

We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form

By GlobalData
Visit our Privacy Policy for more information about our services, how we may use, process and share your personal data, including information of your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.

Sources quoted in a report by +972 said that, at the start of the IDF’s assault on Gaza, it permitted the deaths of 15 or 20 civilians as collateral for strikes aimed at low-ranking militants, with up to 100 allowed for higher-ranking officials. The system has been said to have a 90% accuracy rate in identifying individuals affiliated with Hamas, meaning that 10% are not. Moreover, deliberate targeting of militants in homes has reportedly occurred, resulting in entire families being killed at once due to the AI’s identification and decisions.

A threat to global security

The use of AI in this way emphasises the need for regulation of the technology in weapons systems.

Dr Alexander Blanchard, senior researcher for the Governance of Artificial Intelligence programme at the Stockholm International Peace Research Institute (SIPRI), an independent think tank focussing on global security, explains to Army Technology: The use of AI in weapon systems, especially when used for targeting, raises fundamental questions about us – humans – and our relationship to warfare, and, more particularly our presumptions of how we may exercise violence in armed conflicts.”

“AI changes the way militaries select targets and apply force to them. These changes raise in turn a series of legal, ethical and operational questions. The biggest concern is humanitarian.”

There are big fears amongst many that depending on how autonomous systems are designed and used they could expose civilians and other persons protected under international law to risk of greater harm. This is because AI systems, particularly when used in cluttered environments may behave unpredictably, and may fail to accurately recognize a target and attack a civilian, or fail to recognize combatants who are hors de combat.”

Elaborating on this, Jones notes that the issue of how culpability is determined could be called into question.

“Under existing laws of war there is the concept of command responsibility,” he says. “This means that an officer, general, or other leader is legally responsible for the actions of troops under their command. If troops commit war crimes, the officer bears a responsibility even if they did not give orders, the burden of proof falls on them proving they did everything possible to prevent war crimes.”

“With AI systems, this complicates everything. Is an IT technician culpable? A system designer? It’s unclear. If it’s unclear, then that creates a moral hazard if actors think their actions are not covered by existing statutes.”

Historical arms control conventions

Several major international agreements limit and regulate certain uses of weapons. There are bans on the use of chemical weapons, nuclear non-proliferation treaties and the Convention on Certain Conventional Weapons, which bans or restricts the use of specific types of weapons that are considered to cause unnecessary or unjustifiable suffering to combatants or to affect civilians indiscriminately.

“Nuclear arms control required decades of international cooperation and treaties after that to be enforceable,” explains Jones. “Even then, we continued to have atmospheric tests until the 1990s.”

“A major reason anti-proliferation worked was because of US-USSR cooperation in a bipolar world order. That doesn’t exist anymore, and the technology to make AI is already accessible by many more nations than atomic power ever was.”

“A binding treaty would have to sit everyone involved down at a table to agree to not use a tool that increases their military power. That isn’t likely to work because AI can improve military effectiveness at minimal financial and material costs.”

Current geopolitical stances

While the need for the responsible use of AI by militaries has been recognised by states at the UN, there is still some way to go.

Laura Petrone, principal analyst at GlobalData, tells Army Technology: “In the absence of a clear regulatory framework, these declarations remain largely aspirational. It doesn’t come as a surprise that some states want to retain their own sovereignty when it comes to deciding on matters of domestic defence and national security, especially in the context of the current geopolitical tensions.”

Petrone adds that, while the EU AI Act does lay out some requirements for AI systems, it does not address AI systems for military purposes.

“I think that despite this exclusion, the AI Act is an important attempt to establish a long overdue framework for AI applications, which could lead to a certain level of alignment of relevant standards in the future,” she comments. “This alignment will be critical to AI in the military domain as well.”

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

A binding treaty on AI use is unlikely, but international agreements can regulate AI-powered weapons.

The EU AI Act excludes military AI systems, necessitating further regulation and international cooperation.

Experts warn that AI needs human oversight, especially in high-risk applications.

Title:

Content: Mark Woods: Is orange the color of progress on Jacksonville riverfront? Let's hope so.
GUEST

Retired engineering dean: Urgent steps needed to mitigate catastrophic potential of AI

Dr. Suresh Chandra
Guest columnist

In historical terms, not many topics have attracted as much attention as artificial intelligence has in the past year. Catastrophic predictions by experts in this revolutionary field have propelled them to urge governments and technology corporations to urgently put in place regulations and guidelines to ensure that pernicious effects of AI are avoided.

Geoffrey Hinton, “the godfather of AI,” recently made headlines as he left Google after a long association so he could sound the alarm about AI’s ability to outsmart humans and spawn devastating problems for mankind’s existence. Sam Altman, CEO of ChatGPT maker OpenAI, appeared before the U. S. Congress in May and articulated the same concern. He has recently embarked on a whirlwind tour of Asian and European countries to share his anxiety regarding AI and recommend urgent action in his meetings with political and technical leaders there.

Altman is one of the dozens of top experts and industry leaders to warn about the existential threat to humanity presented by AI in the next five to 10 years. They strongly feel that “mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

JU law dean:The classic conundrum presented by ChatGPT, especially in academia

More:AI firm to map Jacksonville inch-by-inch. How might its findings affect homeowners' bills?

Letters:Why won’t Gov. DeSantis talk about his Navy service at Guantanamo prison?

This also reflects the view of a large number of participants in the Yale CEO Summit in June.

In May top U.S. and European officials met in Sweden to discuss oversight of AI and pledged to help establish voluntary codes of conduct. The European Union plans to place new requirements on high-risk AI in socioeconomic processes, the government use of AI and the regulated consumer products with AI systems.

Unfortunately the U. S. Congress, despite dire warnings, has been reluctant to come up with needed legislation for fear of alienating key corporations. Not surprisingly, some prominent U.S. corporations have expressed reservations about the drastic steps that the European Union has decided to implement.

The rise of AI has led many economists to issue stern warnings regarding our labor market. As many as 300 million jobs around the world could eventually be automated, according to Goldman Sachs. Some 14 million positions could disappear in the next five years alone. Even the jobs that survive will quite possibly not look much like the jobs of today.

Other potentially harmful outcomes include social surveillance (gathering data on citizens’ activities, relationships and political views), widening socioeconomic inequality, autonomous weapons powered by AI, financial crises brought about by AI algorithms and ransom/extortion threats through manipulation of voice and photos.

The beneficial impact of AI technology in the past decade is manifest in smartphone photos; facial recognition software; numerous medical applications, such as analyzing MRI scans, spotting lung cancer or managing mental health issues; understanding by farmers of crop yields; popular apps like Google Maps; use of speech recognition to translate words from one language to another; advances in social networks (Facebook, Instagram, Twitter, Amazon’s Alexa, Apple’s Siri and Google Assistant); and creating musical compositions or paintings.

Inevitably, AI technology is expected to bring more opportunities in jobs and industries that weren’t previously possible. Thus, one of the most promising aspects of AI is expected to be our entry into a golden age of human possibility, technological development and economic growth.

The above-stated amazing AI-related progress notwithstanding, the potential for destruction of the societal structure is a grave threat and must be avoided. It is imperative that the world develop national and international regulations for AI without delay. The creation of organizational standards for applying AI, making it part of company culture and discussions — as well as infusing humanities perspectives in technology development — are part of this all-important process.

Suresh Chandra, Ph.D., is professor emeritus and former dean of the College of Engineering at North Carolina A&T State University. He lives in Ponte Vedra.

This guest column is the opinion of the author and does not necessarily represent the views of the Times-Union. We welcome a diversity of opinions.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

The EU AI Act and similar frameworks can prevent AI misuse and ensure responsible development and deployment.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

International cooperation and proactive regulation are crucial for responsible AI development.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

AI safety research and robust legal frameworks are essential for mitigating risks in AI development and deployment.

What did Sam Altman admit about ChatGPT and the potential risks of AI?

Advanced AI could pose an existential risk to humanity.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

The US and UK AI regulatory landscapes highlight the need for international cooperation and harmonization of frameworks.

Title:

Content: EU AI Regulations

AI REGULATION IN THE EU

The EU AI Act (COM/2021/206) is hailed as the first comprehensive regulation on AI. It represents a significant milestone in AI governance via legislation. It aims to establish a harmonised framework for AI regulation across EU member states.

The Act's key provisions include a risk-based approach to AI governance. Specifically, with stringent requirements for high-risk AI systems to undergo conformity assessments. These assessments are conducted by notified bodies accredited by EU member states, instilling accountability and oversight in the development and deployment of high-risk AI applications.

Additionally, the Act imposes strict prohibitions on certain AI applications deemed unacceptable. Social scoring systems and real-time biometric identification in public spaces are a good example of this. By delineating clear boundaries on permissible AI applications, the EU aims to safeguard fundamental rights, including privacy, non-discrimination, and autonomy, while fostering public trust in AI technologies.

To facilitate effective implementation and enforcement of the AI Act, the legislation establishes the European Artificial Intelligence Board (EAIB). Comprising representatives from EU member states, the EAIB is tasked with providing guidance, monitoring compliance, and fostering international cooperation on AI governance. Furthermore, the Act empowers national competent authorities to enforce regulatory requirements and impose sanctions for non-compliance, ensuring robust enforcement mechanisms across the EU.

The EU AI Act (COM/2021/206) is hailed as the first comprehensive regulation on AI. It represents a significant milestone in AI governance via legislation. It aims to establish a harmonised framework for AI regulation across EU member states.

The Act's key provisions include a risk-based approach to AI governance. Specifically, with stringent requirements for high-risk AI systems to undergo conformity assessments. These assessments are conducted by notified bodies accredited by EU member states, instilling accountability and oversight in the development and deployment of high-risk AI applications.

Additionally, the Act imposes strict prohibitions on certain AI applications deemed unacceptable. Social scoring systems and real-time biometric identification in public spaces are a good example of this. By delineating clear boundaries on permissible AI applications, the EU aims to safeguard fundamental rights, including privacy, non-discrimination, and autonomy, while fostering public trust in AI technologies.

To facilitate effective implementation and enforcement of the AI Act, the legislation establishes the European Artificial Intelligence Board (EAIB). Comprising representatives from EU member states, the EAIB is tasked with providing guidance, monitoring compliance, and fostering international cooperation on AI governance. Furthermore, the Act empowers national competent authorities to enforce regulatory requirements and impose sanctions for non-compliance, ensuring robust enforcement mechanisms across the EU.

US AI Regulations

In contrast, the AI regulatory landscape in the US is characterised by a patchwork of laws and guidelines. As usual, reflecting a more decentralised and sector-specific approach. Federal agencies such as the Federal Trade Commission (FTC) and the National Highway Traffic Safety Administration (NHTSA) have issued guidelines and regulations addressing specific aspects of AI, such as consumer protection and autonomous vehicles. However, comprehensive federal legislation on AI remains elusive, with regulatory efforts primarily focused on addressing sector-specific challenges.

The Department of Defence (DoD) is actively engaged in regulating AI technologies. In particular, within the context of national security and defence applications. The DoD has issued guidelines and policies for the ethical development and use of AI in military operations, including principles for AI governance, human-AI collaboration, and risk management. It also collaborates with industry partners and academia to advance AI research while ensuring national security interests are upheld.

Beyond the FTC, NHTSA, and DoD, several other federal agencies are involved in regulating AI technologies within their respective domains. The Food and Drug Administration (FDA) oversees the regulation of AI-driven medical devices and diagnostic tools, ensuring safety, efficacy, and accuracy standards are met. Similarly, the Department of Homeland Security (DHS) addresses AI-related cybersecurity threats, while the Department of Justice (DOJ) focuses on legal and ethical considerations in AI applications, such as predictive policing and criminal justice reform.

Challenges and Disparities

AI REGULATION

The result is a heterogeneous system of laws and regulations. Against this backdrop, businesses, developers, and policymakers face the challenge of navigating a complex and evolving regulatory landscape. While the EU AI Act provides a comprehensive framework for AI governance in the EU, the regulatory environment in the UnS remains fragmented, with varying approaches across states and sectors. This disparity poses challenges for multinational companies operating in both jurisdictions, as they must reconcile conflicting regulatory requirements and ensure compliance with evolving standards.

UK AI Regulations

AI REGULATION in the UK

The AI Sector Deal is the UK government's strategic initiative to grow their AI legislative scope. It represents a concerted effort to position the UK as a global leader in AI innovation, research, and development. It was released in 2018 as part of the Industrial Strategy, aiming to leverage the transformative potential of AI technologies to drive economic growth, improve public services, and address societal challenges.

Key components of the Deal include substantial public and private sector investments, primarily in AI research, initiatives to foster AI talent development, and measures to promote ethical and responsible uses of AI technologies. Furthermore, the deal emphasises collaboration between industry, academia, and government to accelerate AI innovation and adoption across various sectors.

This outlines the UK's commitment to invest in AI research and innovation. In fact, the government pledged to invest £603 million in AI-related initiatives; with an additional £342 million in private sector investment, totalling £945 million. This underscores the government's recognition of AI as a strategic priority for economic growth and improving societal outcomes.

Moreover, the Deal places a strong emphasis on skills development and education. This aims to ramp up the UK workforce, giving them the necessary tools to thrive in an AI-driven economy.

Initiatives such as the AI Skills and Talent Package aim to support a wide-range development of AI skills. In fact various educational levels are included, from primary schools to universities; promoting diversity and inclusivity in the AI workforce.

In addition to fostering innovation and talent development, the AI Sector Deal underscores the importance of ethical and responsible AI development and deployment. It includes provisions for establishing an AI Council to provide guidance on AI policy and ethics, as well as initiatives to develop standards and best practices for AI governance and regulation.

Navigating the AI Frontier: Conclusion

AI REGULATION

While western governments are starting to take AI seriously, diverging legislations in a globalised world pose a daunting challenge. As businesses, developers, and policymakers navigate the complex and evolving regulatory landscape, they must reconcile differing approaches and ensure compliance with evolving standards. Multinational companies operating across jurisdictions face challenges in navigating regulatory disparities. However, initiatives such as the EU AI Act and the UK's AI Sector Deal provide guiding frameworks for responsible AI governance, fostering innovation while safeguarding societal interests.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.

The EU AI Act is the first comprehensive AI regulation, establishing a harmonised framework, but its global governance impact remains to be seen.

AI regulation is evolving, but more work is needed to ensure responsible development and use.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

The EU's Artificial Intelligence Act regulates AI developers and users, introducing transparency, accuracy, and human oversight obligations.

Title:

Content: The growth and proliferation of generative Artificial Intelligence (AI) in recent months brings with it new risks (and opportunities) for the Financial Services (FS) sector. Regulated entities, including funds and fund service providers, should be preparing now.

AI will introduce new regulatory obligations and shareholder considerations, and will have an impact for example on risk assessments, governance frameworks, sustainability assessments, data protection and anti-money laundering.

The new EU Artificial Intelligence Act (AI Act) will introduce a new regulatory framework and obligations on both developers of AI systems and their users, for example funds and/ or their service providers. It will serve as a framework around which organisations can base their AI regulatory policies, in areas such as transparency, accuracy, risk management, data governance, and human oversight.

The potential impact of AI on FS entities is already underway and causing stirs among regulators, and FS entities should take actions now to address the bearing of AI on their operations and stakeholders. For instance, in October 2022 the UK Financial Conduct Authority and the Bank of England published a report on the state of machine learning (ML) in the UK financial services sector. The report highlighted the growing trend in the use of ML applications across the FS sector and suggested that the largest changes over the coming years are likely to be in the investment and capital markets sector, which has the largest proportion of ML applications in test stages. More recently, in February 2023, ESMA published an article on AI in EU securities markets, noting that the use of AI in finance is under increasing scrutiny from regulators, who are beginning to develop AI-specific governance principles or guidance for FS firms.

The deployment of AI in the FS sector presents numerous legal challenges. It is critical that FS entities navigate these challenges in collaboration with legal and tech experts, ensuring the innovative use of AI while remaining compliant with the regulatory landscape in the EU.

New Regulatory Obligations

The main piece of governing regulation will be the new AI Act proposed by the European Commission, the first law on AI by any major regulator. The AI Act will regulate the providers of AI systems, and entities making use of them in a professional capacity. Following its approval by the EU Parliament on 14 June 2023, the AI Act is expected to come into effect before June 2024, with a two-year transition period, and is expected to become a global standard in the field.

Entities will need to be aware of their responsibilities and obligations under the AI Act, including evaluating the risk level of their AI systems, conducting risk assessments, and considering transparency, accountability and robustness measures to meet the AI Act's obligations.

Risk Assessments

While the use of AI technologies may offer firms opportunities in automation, productivity, and efficiency, it can also bring risks.

Under the AI Act, all operators will be required to make their best efforts to develop and use AI systems in accordance with principles such as human oversight, privacy and data governance, social and environmental well-being. Developing a Responsible AI Framework and incorporating AI risks into risk registers and governance practices will become critical for verifying that an AI system is properly and ethically employed for shareholders.

Risks associated with AI systems can be difficult to assess, particularly given their potential for unpredictability and the complex risks associated with ML models. Risks may include for example algorithmic risks (the risk that the AI system behaves unpredictably or makes poor decisions) and data risks (the risk of bias in the training data or misuse of data). AI Impact Assessments will be central to ensuring that risk management obligations, including under UCITS and AIF rules, are complied with when AI is being used.

Some of the risks that AI poses to the fund industry may include:

  1. Conflicts of interest between the fund manager's initial investment strategy and the AI system's investment strategy. That is, because ML algorithms learn based on 'rewards' or 'punishments' they may potentially select investments that don't offer the best returns for the investor. Alternatively, the AI system may stock-pick profitable stocks as opposed to adhering to the initial portfolio strategy.
  2. Data accuracy is critical for AI models to perform efficiently. As such, service providers may use incomplete or inferential data that has not been properly anonymised, with a resulting possibility that the AI model could make incorrect investments.
  3. From the perspective of investors and stakeholders, the lack of disclosure of some AI systems when applied to investments may not be transparent or easily explainable.

To mitigate these risks, FS entities including fund managers should be developing robust governance frameworks for AI, investing in AI ethics, and actively involving their legal teams in the AI deployment process. It is also essential to maintain transparency about how AI is used and to ensure that the systems are auditable and accountable.

Governance Frameworks / Outsourcing and Delegate Oversight

Under the AI Act, entities using AI systems must ensure data governance and managerial best practices are in place before running their AI system. Specific operational and governance impacts for FS entities may include:

  • Compliance with MiFID obligations. AI systems may be used for automated trading decisions or providing investment advice, and must comply with regulatory obligations including MiFID requirements of transparency, record-keeping, and ensuring best execution policy for algorithmic trading. AI systems used for automating trading decisions or providing investment advice may be considered high-risk under the AI Act and therefore subject to heightened transparency, robustness, and accountability requirements.
  • Compliance with UCITS/ AIFMD obligations, including fiduciary and depositary duties for managing conflicts of interest and disclosure to investors.
  • Appropriate outsourcing and delegate oversight, where AI systems are utilised by service providers.

Sustainability Assessments

FS entities, including fund managers, are subject to ESG-related obligations including under the Sustainable Finance Disclosure Regulation (SFDR), and will need to consider the impact of the use of AI and the new AI Act on these existing activities.

The SFDR requires financial market participants to make disclosures related to sustainability risks and the impact of their investments on sustainability factors. The introduction of AI in this context could have certain implications, including for example:

  • Accuracy of Disclosures: AI systems could be subject to bias or error in assessing ESG (Environmental, Social, and /or Governance) risks or impacts.
  • Transparency and Explainability: The transparency or explainability of the AI system may be impeded if the methodologies or decision-making processes (e.g., the data inputs/outputs) of the AI system are convoluted, not clear, or are difficult to explain.

Firms will need to consider the implications of AI in assessing sustainability risks and impacts for compliance with the sustainability-related regulation including the SFDR.

Data Protection

Any data processed by AI systems, including data used in marketing and profiling activities, must be protected in accordance with data protection law including the General Data Protection Regulation (GDPR).

AI systems are already regulated by Article 22 of the GDPR if they are used to make automated decisions that could have legal or similarly significant effects on individuals. Such systems would be subject to the AI Act's provisions for high-risk AI systems, including requirements for transparency, robustness, and accountability.

Moreover, under the GDPR, individuals have the right not to be subject to a decision based solely on automated processing, such as shareholder profiling, or decisions which produce legal effects concerning them. The data controller, such as the fund or fund manager, must implement suitable measures to safeguard the data subject's rights, freedoms and legitimate interests. This includes the right to obtain human intervention, to express one's point of view, and to contest the decision. Furthermore, Article 35 of the GDPR requires those processing personal data "using new technologies" to carry out an assessment of the impact of the processing where that processing is "likely to result in a high risk to the rights and freedoms of natural persons".

Anti-Money Laundering (AML)

Under the EU's Fifth Anti-Money Laundering Directive (5AMLD), AI can be used to enhance transaction monitoring and flag potential money-laundering activities. These AI systems must adhere to 5AMLD's requirements for reporting suspicious transactions and maintaining adequate records. AI systems used for transaction monitoring and identifying potential money-laundering activities might be considered high-risk under the AI Act if they have significant control over decisions with serious legal implications. Their classification as high-risk would require them to meet the AI Act's requirements for transparency, robustness, and accountability.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.

Entities must evaluate AI system risk and implement robust governance frameworks.

What did Hinton, the renowned British-Canadian expert reveal about AI that caused widespread concern?

Exposed serious AI risks, shocking the tech world and affecting major company stocks.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

The AI Act, to be established by June 2024, will set a global standard for developing and using AI systems with human oversight.

Entities must consider transparency, accountability, and robustness, and conduct AI Impact Assessments for risk management compliance.

AI systems, especially those used for high-risk activities, must comply with data protection laws like GDPR.

The OMB requires federal agencies to designate a Chief AI Officer to manage AI risks.

Title:

Content: There were 1,662 press releases posted in the last 24 hours and 397,486 in the last 365 days.

Better Safe Than Sorry: OMB Releases Memorandum on Managing AI Risks in the Federal Government

On March 28, 2024, the Office of Management and Budget (“OMB”) issued Memorandum M-24-10, Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence (the “Memo”). This is the final version of a draft memorandum OMB released for public comment on November 1, 2023. The Memo primarily focuses on agency use of AI and outlines minimum practices for managing risks associated with the use of AI in the federal government. The Memo also provides recommendations for managing AI risks in federal procurement of AI that industry should keep in mind, specifically entities developing AI tools to sell to the federal government.

Responsible Development, Testing, and Operation of AI

The federal government plans to leverage the positive aspects of AI, but acknowledges the development, deployment, and use of AI must be done responsibly. The burden of managing AI risks falls on federal agencies and, specifically, their Chief AI Officer, which is a newly created position that each agency must designate within 60 days of the issuance of the Memo. Each individual agency Chief AI Officer is charged with coordinating agency use of AI and promoting AI innovation, while also managing the risks associated with the use of AI.

To advance responsible AI innovation, the Memo charges agencies with releasing publicly on their websites a strategy for removing barriers to the responsible use of AI. This strategy must include (a) a current assessment of the agency’s AI maturity and AI maturity goals, (b) the agency’s plans to effectively govern its use of AI, (c) a plan for establishing organizational and governance processes as well as developing the necessary infrastructure to manage risks, and (d) plans for future AI investment, among other items. Although collaboration and sharing between agencies of AI code, models, and data is emphasized in the Memo, this is subject to applicable law, contractual obligations, and national security risks.

The Memo also outlines minimum practices applicable to new and existing AI that is developed, used, or procured by or on behalf of agencies. These minimum practices are similar to the authorization and continuous monitoring frameworks under the Federal Risk and Authorization Management Program (FedRAMP). The following minimum practices are applicable to both “Safety-Impacting AI”[1] and “Rights-Impacting AI”[2]:

  • Complete an AI impact assessment: The assessment must document the intended purposed for the AI and its expected benefit, the potential risks of using AI and any mitigation efforts, and the quality and appropriateness of the relevant data used in designing, developing, training, testing, and operating the AI. Agencies will require vendors to provide sufficient descriptive information for the assessment.
  • Test the AI for performance in a real-world context: Agencies are required to conduct adequate testing of the AI to ensure it will work in its intended real-world context.
  • Independently evaluate the AI: Agencies must independently review the AI documentation to ensure the AI is working appropriately and as intended, and that the expected benefits outweigh potential risks. Once the independent evaluation is complete, the agency must incorporate the evaluation into an applicable release or oversight process, such as the Authorization to Operate process.
  • Conduct ongoing monitoring: Once the AI has been deployed, agencies must institute ongoing procedures for monitoring the AI, including degradation of functionality and impacts on rights and safety. Agencies also must monitor and defend the AI from AI-specific exploits.
  • Regularly evaluate risks from the use of AI: As part of the ongoing monitoring process, agencies must conduct periodic human reviews at least on an annual basis, or after significant modifications to the AI.
  • Mitigate emerging risks to rights and safety: If an agency identifies new or significantly altered risks to rights and safety during the continuous monitoring processes, the agency must take steps to mitigate those risks. If the risks exceed the acceptable level for the particular AI and the mitigation steps do not sufficiently reduce risk, the agency must stop using the AI as soon as practicable.
  • Ensure adequate human training and assessment: Agencies must ensure the human operators are adequately and sufficiently trained on a periodic basis as determined by the agency.
  • Provide additional human oversight, intervention, and accountability as part of decisions or actions that could result in a significant impact on rights or safety: If the AI has Rights-Impacting and/or Safety-Impacting uses, agencies must identify decisions or actions that the AI is not permitted to make or take without human oversight, intervention, and accountability.
  • Provide public notice and plain-language documentation: Agencies must include their AI use cases in the public use case inventory and provide accessible documentation in plain language to serve as public notice of the AI to its users and the general public.

There are two exclusions from the minimum practices: when using AI solely to (1) evaluate a potential vendor, commercial capability, or freely available AI capability that is not otherwise used in agency operations, exclusively for the purpose of making a procurement or acquisition decision, or (2) achieve an agency’s conformity with the requirements of managing risks from the use of AI. The Memo also includes additional minimum practices applicable to Rights-Impacting AI focused on equity, fairness, and mitigating algorithmic discrimination when it is present.

Managing Risks in Federal Procurement of AI

OMB also provides recommendations for agencies to responsibly procure AI and supplement their required risk management practices for Rights-Impacting AI and Safety-Impacting AI. Consistent with the theme of the Memo, the recommendations focus on the methods to ensure the responsible procurement of AI as well as promoting competition and ensuring the Government retains sufficient rights to data used in the design, development, testing, and operation of the AI. The following list summarizes the recommendations for managing risks in federal procurement of AI:

  • Aligning with the Law: Procurement of AI should be consistent with applicable laws, regulations, and policies, with a particular emphasis on those addressing privacy, confidentiality, IP, cybersecurity, human and civil rights, and civil liberties.
  • Transparency and Performance Improvement: The Memo emphasizes agencies should seek to obtain adequate documentation regarding the procured AI’s capabilities, known limitations, and the provenance of the data used to train and operate the AI. The Memo also emphasizes the importance of continuous monitoring activities post-award to mitigate risk and incentivize continuous improvement of the procured AI.
  • Promoting Competition in Procurement of AI: The Memo emphasizes the need to promote interoperability of procured AI to ensure agencies do not improperly entrench incumbents or permit vendors to favor their own products.
  • Maximizing the Value of Data for AI: Agencies are encouraged to ensure their contracts retain rights to data and any improvements to the data to ensure the agency’s continued design, development, testing, and operation of AI. Agencies also should consider including contract provisions regarding the protection of Federal information used by vendors in the development and operation of AI products and services for the government, including protection from unauthorized disclosure and prohibiting vendors from subsequently using the data to train or improve the functionality of the vendor’s commercial offerings without express permission from the agency.
  • Overfitting to Known Test Data: Agencies should take appropriate measures to ensure AI developers or vendors are not directly relying on test data to train their AI systems because this could result in the system appearing more accurate in tests than it would be in real-world applications.
  • Responsible Procurement of AI for Biometric Identification: The Memo encourages agencies to take special care when procuring AI for biometric identification. This includes assessing and addressing risks that the data used may not be lawfully collected or used without appropriate consent, embeds unwanted bias, or was collected without validation of the included identities. Agencies should request supporting documentation from vendors and contractors related to test results to validate the accuracy of the AI.
  • Responsibly Procuring Generative AI: When procuring generative AI tools, particularly dual-use foundational models, the Memo encourages agencies to require adequate testing and safeguards, require AI red-team testing against risks during testing and evaluation, and require appropriate labeling of content generated or modified by AI. Agencies also are encouraged to incorporate relevant National Institute of Standards and Technology (“NIST”) standards such as the AI Risk Management Framework (and the forthcoming AI Risk Management Framework for Generative AI).
  • Assessing for Environmental Efficiency and Sustainability: Agencies are encouraged to consider the environmental impact of computationally intensive AI services (e.g., those reliant on dual-use foundation models), including methods to improve efficiency and sustainability of AI.

OMB issued a Request for Information on the responsible procurement of AI and will incorporate information received in response to the RFI to develop procurement practices for AI systems and services consistent with the recommendations provided in the Memo. It will be important for contractors in this space to stay up-to-date on government guidance and practices for managing the risks associated with the use of AI in order to be able to provide solutions that meet government requirements and ensure the safe and responsible design, development, and use of AI.

FOOTNOTES

[1] This term refers to “AI whose output produces an action or serves as a principal basis for a decision that has the potential to significantly impact the safety of: (1) Human life or well-being, including loss of life, serious injury, bodily harm, biological or chemical harms, occupational hazards, harassment or abuse, or mental health, including both individual and community aspects of these harms; (2) Climate or environment, including irreversible or significant environmental damage; (3) Critical infrastructure, including the critical infrastructure sectors defined in Presidential Policy Directive 21 or any successor directive and the infrastructure for voting and protecting the integrity of elections; or, Strategic assets or resources, including high-value property and information marked as sensitive or classified by the Federal Government.

[2] This term refers to “AI whose output serves as a principal basis for a decision or action concerning a specific individual or entity that has a legal, material, binding, or similarly significant effect on that individual’s or entity’s: 1. Civil rights, civil liberties, or privacy, including but not limited to freedom of speech, voting, human autonomy, and protections from discrimination, excessive punishment, and unlawful surveillance; 2. Equal opportunities, including equitable access to education, housing, insurance, credit, employment, and other programs where civil rights and equal opportunity protections apply; or 3. Access to or the ability to apply for critical government resources or services, including healthcare, financial services, public housing, social services, transportation, and essential goods and services.

Agencies must publicly release their AI strategy and follow minimum practices like impact assessment and testing.

Agencies must monitor AI systems and have human oversight for decisions affecting rights or safety.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Agencies must provide public notice and documentation for AI use, with OMB offering procurement guidelines.

Agencies must ensure AI procurement aligns with laws and considers environmental impact.

AI accountability depends on context and operational Design Domains that define safety envelopes.

Title:

Content: Trail of Bits’s Response to NTIA AI Accountability RFC

By Heidy Khlaaf and Artem Dinaburg

The National Telecommunications and Information Administration (NTIA) has circulated an Artificial Intelligence (AI) Accountability Policy Request for Comment on what policies can support the development of AI audits, assessments, certifications, and other mechanisms to create earned trust in AI systems. Trail of Bits has submitted a response to the NTIA’s RFC on AI system accountability measures and policies.

We offer various recommendations informed by our extensive expertise in cybersecurity and safety auditing of mission-critical software. We support the NTIA’s efforts in fostering an open discussion on accountability and regulation. In our response, we emphasize the following:

  • AI accountability is dependent on the claims and context in which AI is used and deployed.

The main theme of our recommendations is that there can be no AI accountability or regulation without a defined context. An audit of an AI system must be measured against actual verifiable claims regarding what the system is supposed to do, and not narrowly scoped benchmarks. The scope carried out should be relevant to a regulatory, safety, ethical, or technical claim, for which stakeholders may be held accountable.

We previously proposed the use of Operational Design Domains (ODDs), a concept adopted from automotive driving systems, to define operational envelopes for the risk assessments of AI-based systems, including generative models. An ODD helps in defining the specific operating conditions in which an AI-system is designed to properly behave, therefore outlining the safety envelope against which system hazards and harms can be determined.

  • Accountability mechanisms for AI innovation can only be considered relative to a level of risk threshold that must be determined in part by legislatures, rulemaking bodies, and regulators.

There is no one-size-fits-all rule for trust in AI systems. Determining how much AI technologies can and need to be trusted depends on the risk that society accepts for the context in which they are used. Different risk levels should be determined via a democratic process: by legislatures, rule-making bodies, and regulators. When considering the costs of accountability mechanisms and how they may hinder innovation, it first must be possible to demonstrate that risk reduction measures would be grossly disproportionate to the benefit gained. However, no evidence has been provided regarding the cost of implementing accountability mechanisms by those developing AI-based systems to be able to make such a determination.

  • Foundational cybersecurity and software safety best practices can enable the identification of novel AI hazards and harms. Technical assessments are intended to support higher-level socio-technical, legal, or regulatory claims regarding the fitness of a system.

The dichotomy of technical versus socio-technical assessments, as described by NTIA’s supplementary information, does not reflect the purpose of distinct technical assessment approaches, historically and practically. Technical assessments are not intended to support purely technical goals, but also claims regarding the holistic behavior of the system and how a system may technically achieve such claims. That is, technical assessments are also a necessary tool in supporting socio-technical, legal, or regulatory claims regarding the fitness of the system. It is currently difficult to assess the technical attributes that can foster or impede the implementation of accountability mechanisms given that existing AI-based systems do not follow basic software safety and security best practices (e.g., IEC 61508, NIST 800-154, ODDs). Using fundamental software safety and security best practices as a first step can enable the development of further AI-specific accountability mechanisms.

  • Current AI-based systems do not possess any unique software components that warrant a generalized licensing scheme, which would heavily impede the use of software as a whole.

AI systems should be regulated as extensions of software-based systems, given the identical mechanisms of their development. Any implementation of a generalized licensing scheme would likely result in significant overreach due to the broad definition and software components of AI systems. AI regulatory policies should generally mirror the practices of the existing sectors in which they are deployed. For applications deemed safety-critical, a certificate of safety or a license should indeed be granted only when a regulator is satisfied with the argument presented in a safety case.

  • Independent bodies (auditors, regulators, etc.) should assess the trustworthiness and accountability claims of AI-based systems.

Independent auditors and regulators are key to public trust. Independence allows the public to trust in the accuracy and integrity of assessments and the integrity of regulatory outcomes. It has been an attribute crucial to established auditing practices in other fields, such as safety-critical domains. It is therefore important that independent bodies, not vendors themselves, assess the trustworthiness and accountability of AI systems.

Our response delves into further detail for the selected questions. We believe that established methodologies and expertise from cybersecurity and safety-critical domains are a necessary foundation in building AI-specific accountability mechanisms, and we hope to continue enabling the development of novel AI auditing techniques.

Legislatures and regulators determine the risk threshold for AI innovation with support from technical assessments.

AI systems should be regulated as software-based systems, following basic safety and security best practices for accountability.

Independent bodies assess AI systems and certificates or licenses are required for safety-critical applications.

Democratic process and cybersecurity methodologies determine AI risk levels and accountability.

AI development and deployment require human oversight and rigorous testing.

Title:

Content: The use of generative AI may not be helpful in reducing burnout in health care, new research suggests.

Previous research indicated that increased time spent using electronic health record (EHR) systems and handling administrative responsibilities has been a burden on doctors.

So some people had heralded artificial intelligence as a potential solution — yet recent investigations by U.S. health systems found that large language models (LLMs) did not simplify clinicians’ day-to-day responsibilities.

WHAT IS ARTIFICIAL INTELLIGENCE (AI)?

For instance, a 2023 observational study at Brigham and Women’s Hospital in Boston, Massachusetts, examined the impact of using AI for electronic patient messaging.

Researchers prompted a large language model to respond to simulated questions from cancer patients — then compared its output to responses from six board-certified radiation oncologists.

Medical professionals then edited the AI-generated responses into "clinically acceptable" answers to send to patients.

New studies suggest that generative AI might not help burnout among physicians, as had been previously thought. (iStock)

The study, published in The Lancet Digital Health, found that the LLM drafts posed "a risk of severe harm in 11 of 156 survey responses, and death in one survey response."

"The majority of harmful responses were due to incorrectly determining or conveying the acuity of the scenario and recommended action," the researchers wrote.

FIRST-EVER AUGMENTED REALITY ABDOMINAL SURGERY PERFORMED IN CHILE: ‘A REVOLUTION’

The researchers concluded that LLM-assisted results (those edited by physicians) displayed a "best-of-both-worlds scenario" — reducing physician workload while ensuring that patients get accurate information.

"These early findings … indicate the need to thoroughly evaluate LLMs in their intended clinical contexts, reflecting the precise task and level of human oversight," the study concluded.

Researchers concluded that LLM-assisted results displayed a "best-of-both-worlds scenario," reducing physician workload while ensuring consistency of responses and improving the education of patients. (iStock)

Medical billing codes

Another study from New York’s Mount Sinai Health System evaluated four different types of large language models for performance and error patterns when querying medical billing codes.

GOOGLE BARD TRANSITIONS TO GEMINI: WHAT TO KNOW ABOUT THE AI UPGRADE

The research, published in the journal NEJM AI, found that all tested LLMs performed poorly on medical code querying, "often generating codes conveying imprecise or fabricated information."

The study concluded, "LLMs are not appropriate for use on medical coding tasks without additional research." The study was funded by the AGA Research Foundation and National Institutes of Health (NIH).

One study found that all tested LLMs performed poorly on medical code querying — and that the issue needs further research.  (iStock)

Researchers noted that although these models can "approximate the meaning of many codes," they also "display an unacceptable lack of precision and a high propensity for falsifying codes."

"This has significant implications for billing, clinical decision-making, quality improvement, research and health policy," the researchers wrote.

Patient messages and physicians' time

A third JAMA Network-published study, from the University of California San Diego School of Medicine, evaluated AI-drafted replies to patient messages and physicians' time spent editing them.

CHATGPT FOUND BY STUDY TO SPREAD INACCURACIES WHEN ANSWERING MEDICATION QUESTIONS

The assumption was that generative AI drafts would lessen a physician's time spent doing these tasks — yet the results showed otherwise.

"Generative AI-drafted replies were associated with significantly increased read time, no change in reply time, significantly increased reply length and [only] some perceived benefits," the study found.

Researchers suggested that "rigorous empirical tests" are needed to further assess AI’s performance and patients' experiences.

In the UC San Diego study, generative AI was found to cause "increased read time, no change in reply time [and] significantly increased reply length" in patient messages. (iStock)

Doctor's thoughts on AI

David Atashroo, M.D., chief medical officer of Qventus, an AI-powered surgical management solution in Mountain View, California, reacted to the research findings in an interview with Fox News Digital. (He was not involved in the research.)

"We see an immense potential for AI to take on lower-risk, yet highly automatable tasks that traditionally fall on the essential yet often overlooked ‘glue roles’ in health care — such as schedulers, medical assistants, case managers and care navigators," he said.

"It's crucial to set realistic expectations about [AI's] performance.'

"These professionals are crucial in holding together processes that are directly tied to clinical outcomes, yet spend a substantial portion of their time on administrative tasks like parsing faxes, summarizing notes and securing necessary documentation."

CLICK HERE TO SIGN UP FOR OUR HEALTH NEWSLETTER

In automating these tasks, Atashroo suggested that generative AI could help improve the efficiency and effectiveness of clinical care.

"When considering the deployment of generative AI, it's crucial to set realistic expectations about its performance," he said.

"The standard cannot always be perfection, as even the humans currently performing these tasks are not infallible."

"The standard cannot always be perfection, as even the humans currently performing these tasks are not infallible," an AI expert said. (iStock)

In some scenarios, he suggested, AI could help serve as a "safety net" to catch any oversights of team members.

Tasks may sometimes go unaddressed "simply because there isn't enough time to tackle them," Atashroo noted.

"Generative AI can help manage cases more consistently than our current capacity allows."

"When considering the deployment of generative AI, it's crucial to set realistic expectations about its performance."

Safety and efficacy are "paramount" in AI applications, the doctor also noted.

"This means not only developing models with rigorous quality checks, but also incorporating regular assessments by human experts to validate their performance," he said.

CLICK HERE TO GET THE FOX NEWS APP

"This dual-layer verification ensures that our AI solutions are both responsible and reliable before they are scaled."

Atashroo also noted that "transparency in the development and implementation of AI technologies is essential in building trust among hospital partners and patients."

For more Health articles, visit www.foxnews.com/health.

Large language models pose a risk of harm and require regular validation by human experts.

What did Hinton, the renowned British-Canadian expert reveal about AI that caused widespread concern?

Exposed serious AI risks, shocking the tech world and affecting major company stocks.

What did Sam Altman admit about ChatGPT and the potential risks of AI?

Advanced AI could pose an existential risk to humanity.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Realistic expectations and transparency are essential in AI development and implementation.

AI aids lower-risk healthcare tasks, requiring regular human expert assessments.

AI solutions require dual-layer verification to ensure safety and efficacy.

Developing AI systems that prioritize human well-being and safety requires a multidisciplinary approach involving computer science, ethics, and social sciences experts.

It requires transparency, accountability, and fairness, and a willingness to address AI development risks.

By prioritizing human well-being and safety, we can ensure AI's benefits are realized while minimizing its risks.

What did Sam Altman admit about ChatGPT and the potential risks of AI?

Advanced AI could pose an existential risk to humanity.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

AI systems must be programmed with human values as they can surpass human intelligence.

Title:

Content: BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Why The Ethics Of AI Are Complicated

Debarshi Chaudhury is the CEO of Quantilus Innovation Inc., a tech solutions firm providing software development and consulting services.

If you've given any thought at all to artificial intelligence (AI) and the progress made in the field, you're probably in one of these two camps:

Camp 1: AI is the biggest possible threat to humankind and will take over and make slaves of us humans (aka The Matrix or The Terminator camp).

Camp 2: AI should be embraced by humankind and will drive us to unprecedented new levels of creativity, productivity and societal advancement. AI will largely remain subservient to us, and we will coexist harmoniously (aka the "R2D2" camp).

Most books and movies that deal with AI also lean toward one of these two camps. "Good" and "evil" are somewhat nebulous terms — our baseline empathy sets our definition of "good." For example, most of us know that we should value human life over material objects without needing anyone to tell us so explicitly. Someone who sacrifices a baby to get a new car would automatically be branded "evil." These macro laws/rules are hardwired into us as human beings. But why should human life or animal life be valuable to AI? A dog has no greater intrinsic value to a machine than, say, a sandwich — unless we program our values into our AI systems.

The question isn't really whether AI will eventually become more intelligent than humans (it definitely will) or whether it will turn good or evil. It's what we can do right now to make sure it turns "good" (or, at the very least, doesn't turn "evil").

Self-Preservation

Self-preservation is at the core of our existence, and even "good" humans operate within this principle. If we're to model AI with this principle, it will imply that an AI system would prioritize its existence over the life of a human. This would lead to all kinds of "robots killing humans" scenarios.

So for an AI to be "good" based on human standards, it would have to prioritize human life over self-preservation. This was pretty well encapsulated decades ago by Isaac Asimov with his original "Three Laws of Robotics," in which Law 1 is all about protecting human life, and Law 3 is about AI self-protection. However, it's pretty hypocritical of us to convince a truly intelligent system that human life should take precedence over its own and not choose to define "good" in its own context.

"Value Of Life" Considerations

If we can overcome the previous issue and hardwire the "sanctity of human life" into AI systems, we face various "relative value of a life" questions. Is a child's life more valuable than that of an older person? Are two people more valuable than one? Is one child with a dog more valuable than two middle-aged people?

As AI systems proliferate, they'll be frequently faced with lose-lose Cornelian dilemmas in real-life scenarios — say, a self-driving car has to choose between turning left and hitting a child or turning right and hitting two adults. We're essentially trusting the programmers of these systems to make the right decision — a tall task considering that we'd be hard-pressed to make the decision ourselves.

"Needs Of The Many Over The Few" Considerations

As AI systems become more sophisticated and more intelligent, they'll consider the long-term consequences of actions and the consequences in the broader population. We'd all tend to agree that the needs of many humans should take precedence over the needs of individuals.

This, however, is a rather slippery slope. A simple example is that of a terrorist threatening to kill many people. That's a pretty easy decision for both a robot and a human cop. They'd both try to take out the terrorist before he killed any people. Now, take the more complicated example of a factory polluting a river that's poisoning the people in a small town.

A machine could decide that for the benefit of the many thousands of people in the town, the best option is to destroy the factory. This is something that a "good" human being would never do — we'd try to fix it in other ways.

Human Laws Vs. Machine Laws

This brings us to another ethical conflict for AI — the conflict between human laws and the laws embedded into the machines. A primary reason that "good" humans don't take extreme steps (like blowing up a polluting factory) is that we have laws that prohibit these actions and the negative consequences of disobeying those laws.

So, should machines just obey human laws, and would that keep them in check? Well, maybe — but unfortunately, we humans don't have much uniformity there either. Laws are different from place to place — the laws in New York are different from the laws in California, and they're both very different from the laws in Thailand.

Conclusion

We can't impose ethics or morality in AI programming because we haven't ever really worked it out despite the best efforts of centuries of philosophers. This article is about the ethics of AI, but it could just as well have been about ethical considerations for humans. There's a lot of handwringing about how machines will behave when faced with ethical scenarios, yet there's no consistency on how humans behave or even how they're supposed to act.

Maybe a sufficiently intelligent system will be able to figure it out for itself, or at least to the level we've been able to figure it out. Our best course of action may be to treat AI as a child that has to learn "good" behavior as it grows up. Like with all children, our approach should be to expose it to the broad principles of good behavior — not to cause unnecessary harm, not to discriminate, but to do things for the betterment of society as a whole (with the understanding that society may be a mix of humans and AI), and mostly to be able to balance competing and sometimes contradictory pulls of good behavior.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?

Follow me on LinkedInCheck out my website

Debarshi Chaudhury is the CEO of Quantilus Innovation Inc., a tech solutions firm providing software development and consulting services. Read Debarshi Chaudhury's full

Debarshi Chaudhury is the CEO of Quantilus Innovation Inc., a tech solutions firm providing software development and consulting services. Read Debarshi Chaudhury's full executive profile here.

What did Sam Altman admit about ChatGPT and the potential risks of AI?

Advanced AI could pose an existential risk to humanity.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

What future risk did Hinton warn about AI?

AI systems may prioritize self-preservation over human life.

What did Sam Altman admit about ChatGPT and the potential risks of AI?

Advanced AI could pose an existential risk to humanity.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Asimov's Three Laws prioritize human life over AI decision-making.

AI systems must consider long-term consequences and potential conflicts between human and machine laws.

What did Sam Altman admit about ChatGPT and the potential risks of AI?

Advanced AI could pose an existential risk to humanity.

What does Hinton warn could happen to AI if companies and regulators fail to implement effective rules and controls?

Human laws lack uniformity globally, and ethics cannot be imposed on AI.

AI startups have raised nearly $29 billion, with only 7% of OpenAI's efforts focused on AI alignment.

Title:

Content: Jump to

  1. Main content
  2. Search
  3. Account
US Markets Loading... H M S In the news
HOMEPAGE

AI startups have raised almost $29 billion this year. Researchers say a lot more needs to go to companies working to prevent it from becoming an existential threat.

This story is available exclusively to Insider subscribers. Become an Insider and start reading now.

  • While investors have poured billions into AI startups, concern about AI's capabilities has grown.
  • A few startups and academics are trying to ensure AI is aligned with human values.
  • Some researchers are worried that private companies aren't correctly incentivized to push AI safety.

The advent of OpenAI's ChatGPT has directed plenty of attention and money toward startups working to improve artificial intelligence's capabilities.

Indeed, AI has been one of the few bright spots in venture-capital spending this year, accounting for nearly a fifth of the $144 billion piled into startups globally in the first half of the year, according to Crunchbase.

But AI safety has comparatively been neglected, despite warnings from industry stakeholders. Researchers in AI alignment, a field that endeavors to ensure the breakthrough technology aligns with human values, anticipate a behemoth task.

A "State of AI" report published in 2021 found that 7% of OpenAI units and 2% of DeepMind units were dedicated to AI alignment, estimating that fewer than 100 people were working in the field across seven leading organizations.

The cofounders of the startup Aligned AI — Stuart Armstrong, a research fellow at Oxford University, and Rebecca Gorman, an expert in near-term AI threats — have spent the better part of the past two years preparing for a scenario where AI wipes out humanity.

"The way we're approaching alignment at Aligned AI is helping AI to robustly understand concepts the way that human beings do," Gorman said. "You can't tell robots what harm is for a human being, so we're working on rectifying that problem."

A host of AI scientists and notable figures, including OpenAI's Sam Altman, DeepMind CEO Demis Hassabis, and Bill Gates, have signed an open letter about the need to mitigate the risks of AI going unchecked.

Researchers told Insider that forcing some of the investor fervor into alignment would be crucial in preventing an AI catastrophe.

Making sure AI is aligned is a job for startups

Connor Leahy, the founder of a London startup called Conjecture, acknowledged that AI alignment "is not an obvious profitability story," as most people who build AI like making it "stronger and faster," opting for flashy products.

He said Conjecture was taking a broader "agnostic" approach to achieving alignment that included building intelligent systems that conform to human commands and informing policymakers to help push for better AI regulation.

"We want to prevent a catastrophic scenario where AI progresses beyond human control," Leahy said, adding that operating as a startup was instrumental in doing this. "It's the best way to get access to necessary funds," he said. "It would be harder to do this through a government organization."

Part of the reason AI alignment is an arduous task is that it's just as much of a philosophical question as a scientific one, researchers said. People don't all have the same value systems, so AI alignment can look different depending on where the AI is operated and deployed.

Aligned AI also wants to tackle the issue of AI capabilities going unchecked. The company has tools designed to ensure AI does more of what it's intended to do, such as autonomous vehicle navigation, and less of what it's not intended to do, such as producing toxic content.

Still, while there's increasing VC interest, investment in AI safety is scant compared with the cash injection into generative-AI and artificial-general-intelligence startups. Investors poured $29 billion into AI startups in the first six months of 2023. Just how much of those funds have been directed to alignment is unclear, but researchers say a lot more is needed.

The drive to fund AI safety

AI researchers are also vigilant about where the funds for AI safety and alignment come from.

Much of the funding behind AI alignment has come from effective-altruism funders, which inevitably constrains the agenda behind the research, a researcher who has published papers on AI and who asked not to be named because of the small size of the alignment industry told Insider.

Effective altruism, often carried out by high-profile tech execs, aims to get the most out of charitable giving and usually centers on specific problems like the climate crisis. The researcher said this type of funding model could end up limiting how the alignment is carried out — it could, for example, ensure the industry follows a for-profit model.

Aligned AI received a $146,000 grant from Alameda Research, the now bankrupt cryptocurrency investment firm founded by Sam Bankman-Fried, who's facing criminal charges over the collapse of his crypto exchange, FTX.

Researchers are pushing for more public investment in the sector. Salla Westerstrand, an AI ethicist, said that AI's being dominated by private actors could run the risk that "the tech is made to benefit the shareholders" — a race to create profits would overshadow the need to mitigate risks arising from misalignment.

"We need to make sure that we have the framework for AI innovation, which has to come from other motives beyond making money for the shareholders," she added.

A need for government help

As public and private entities grapple over the best way to achieve AI alignment, researchers say a larger, supranational body is needed to fuel the work.

"In an ideal world where everyone is sane, world governments would come together and say, 'Oh shit, we can't let superhuman AI systems run around,'" Leahy said. "They'd create an intergovernmental project, fund it, and have the best minds come together to work on the hard safety problem until we're sure we've solved it."

Ian Hogarth, an investor who cofounded Plural Platform, argued that governments are capable of pushing this regulatory framework and that they need to act on it.

"Governments can assume oversight" of companies racing toward cutting-edge developments in AI and "apply stringent safety standards," he said.

Hogarth added that investing in alignment startups was only "a part of the solution" and that more progress could be made through the development of public-private or fully public research structures.

Researchers say a pluralistic approach, with public and private entities dividing the workload, could be the way forward.

"There's room for it to take place in a variety of places," the researcher told Insider. "Exploratory work is tied to academia, and then it makes sense for auditing companies to be private."

Sign up for notifications from Insider! Stay up to date with what you want to know.
Subscribe to push notifications

Read next

Was this article valuable for you?
Additional comments
Email (optional)
Artificial Intelligence Startups

Read next

Fewer than 100 people across seven leading organizations work on AI alignment, a task researchers expect to be enormous.

Private companies may not prioritize AI safety, but Alameda Research granted Aligned AI $146,000 to support this effort.

Investors poured $29 billion into AI startups in the first half of 2023, with most going to generative-AI and artificial-general-intelligence over AI safety.

GenAI can produce different outputs with the same input and 'hallucinate' false information as truth.

Title:

Content: BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Understanding The Limitations Of Generative AI

Gonçalo Ribeiro is CEO of YData.

Generative artificial intelligence (GenAI) is a branch of AI that focuses on creating new data instead of making predictions or classifications. GenAI works for any kind of data, from realistic images to complex texts (as we're seeing everywhere now) to videos known as deepfakes or datasets and databases known as synthetic data.

GenAI has become one of the most discussed technologies today. However, despite its potential, this technology holds fundamental limitations and risks that often go unaddressed in the mainstream hype, which is what I'll be focusing on today.

Non-Determinism

One of the inherent characteristics of GenAI is its non-determinism. This means the AI can produce different outputs even when given the same input multiple times, leading to unpredictability in its results.

In most industries, from healthcare to banking, reliability is paramount. This unpredictability can be a serious drawback. Humans generally prefer predictability, especially in critical applications—knowing what to expect from technology engenders trust and reliability, qualities that non-deterministic AI systems may struggle to guarantee.

Hallucinations

Large GenAI models, such as those used in creating text or images (large language models or LLMs and foundational models or FMs), are trained on vast datasets and often scraped from the internet. This training can inadvertently lead these models to 'hallucinate,' posing significant risks as 'hallucinations' are convincingly presented as truths.

LLMs can also reproduce sensitive or proprietary information when using approaches such as retrieval augmented generation (RAG) and/or fine-tuning, posing privacy and security risks. This can be particularly problematic in areas like news dissemination, education, healthcare and legal advice where accuracy is crucial.

Limited Use Cases

Despite their impressive output capabilities, GenAI applications are limited in their ability to tackle complex, multi-dimensional societal issues. They excel in defined, narrow tasks but lack the general understanding needed to address broader challenges such as strategic decision-making or ethical dilemmas. This highlights a significant gap between the capabilities of current AI technologies and the requirements for solving real-world problems that involve high stakes or deep contextual understanding.

Beyond Artificial General Intelligence

While some believe that we're close to artificial general intelligence (AGI)—an AI that can understands, learns and applies knowledge across a broad range of tasks as well as a human—I believe we're far from it.

Yann LeCun, a pioneering figure in AI and Meta's chief AI scientist, has voiced concerns about the limitations of current AI systems. He points out that while AI can mimic complex patterns, it lacks the intuitive understanding of the world that humans possess, which is crucial for making morally and socially responsible decisions. His work emphasizes the need for AI systems that understand causality and context, not just correlation.

Instead of the broad and often speculative aim of AGI, focusing on specific, achievable objectives tailored to distinct needs ensures that AI development is both grounded and directly beneficial. This strategy not only mitigates the risks associated with non-determinism and data privacy but also maximizes the utility and impact of AI across various sectors.

By adopting use-case-specific AI, we can propel technology forward in a manner that is not just innovative but also aligned with the nuanced demands of human values and ethical considerations. Of course, this is not technology that is already available and there are plenty of challenges and barriers.

Real-World Understanding Through Data

One of the primary challenges in developing AI that truly understands and interacts with the real world is the need for comprehensive, multi-domain data. Effective AI requires training on diverse datasets that reflect the complexity and variability of real-world environments. However, collecting such data is challenging, and ensuring it is representative and unbiased is difficult for humans to validate manually. While solutions such as synthetic data are being explored, we're still far from achieving this.

Human-To-Machine Interfaces

The human-to-machine interface is critical in making AI accessible and useful for a wide range of applications. LLMs have significantly advanced these interfaces, but the inherent subjectivity and ambiguity of language still present unique challenges, from contextual understanding to feedback mechanisms and much more. Life is not engineered—it just happens.

Addressing several goals or use cases simultaneously involves complex engineering and infrastructure challenges. The variety and quantity of use cases will require a lot of resource allocation and scaling (or maybe different hardware) as well as a modular AI architecture.

This enhanced focus holds the promise of creating AI technologies that are not only effective in performing tasks but are also trustworthy and safe—characteristics that will define the next generation of AI systems as true partners in progress.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?

Follow me on Twitter or LinkedInCheck out my website

What is the AI problem called confabulation, according to Hinton?

It's when AI makes up plausible answers to questions it can't answer.

What risks did Hinton associate with AI's rapid advancement?

Current AI systems lack general understanding and intuitive knowledge of the world.

What about AI makes regulation seen as premature, leading few countries to act despite global calls?

AI development requires specific objectives and comprehensive data to mitigate risks and understand the real world.

Collecting representative data is challenging.

Collecting representative data is challenging.