OpenAI boss says world 'right to be afraid' but should trust AI firms

Why is Donald Trump so opposed to regulating AI?

OpenAI boss Sam Altman thinks people should have more faith in his company and others like it to do the right thing when it comes to artificial intelligence (AI) development amid rising public concerns about all the risks.

"The world should trust that we are going to do the right thing because it's the right thing and we feel the magnitude of this," Altman said on Tuesday during a conference in San Francisco.

However, Altman noted that people were justified in their fears around AI, as the capability of the tools has progressed rapidly.

Speaking in Washington meanwhile, left-wing Senator Bernie Sanders and former Trump adviser Steve Bannon have both called for greater governance of AI.

Altman's comments came during an appearance at an annual conference hosted by software firm Salesforce.

"It doesn't take as much imagination as it used to for [us] to imagine how this could go wrong," he said. "I think the world is right to be afraid of this."

It was the first time he had spoken publicly since a post went viral last week by a researcher who quit the AI firm Anthropic. The researcher claimed AI could kill all humans by the end of the decade if left unchecked.

A handful of other AI executives and experts responded by saying they agreed with the assessment, though they didn't explain how they came to their conclusions or how exactly AI could accomplish such a thing.

The fervour over the claims has led to more scrutiny of AI development in recent days, and in response, Anthropic chief executive Dario Amodei called for the pace of all AI development to slow, external and urged governments to regulate the industry.

The post was applauded by Altman, as well as co-founder of Google's DeepMind Demis Hassabis, and Elon Musk, owner of social media site X and AI assistant Grok.

By Tuesday, more AI leaders were voicing support for self-regulation rather than government involvement.

Altman said that he felt AI companies like his own were capable of essentially regulating themselves.

"We will get it right, I'm very confident in our company's and industry's ability to do this safely," Altman said, adding that he was sure that they would "keep alignment and safety way ahead of capabilities" and if they couldn't, they would "slow down or stop".

Despite the assurances, political figures have cautioned against self-regulation.

Bernie Sanders, an independent senator from Vermont who votes with the Democrats, said decisions over AI's development have been made by "a handful of the richest people in the world", naming Elon Musk, Mark Zuckerberg and Jeff Bezos, among others.

Speaking at a conference hosted in Washington, he said: "The people of this country must make the decisions about AI, and not just a handful of oligarchs."

Bannon, who served as a White House strategist during Trump's first term, also cast doubt on the tech industry's ability to self regulate.

"We can never trust what an oligarch says because... they're disingenuous and they lie," Bannon said at the same conference.

After Altman's comments, Meta boss Mark Zuckerberg wrote on X that every AI lab has the ability and the incentive "to take its own actions" towards making AI tools and models that are designed for safety.

"Any lab that doesn't focus on alignment will fall behind," Zuckerberg wrote. "Labs face significant liability if their models cause harm, so they have a strong incentive to prevent this as well."

Jensen Huang, head of Nvidia, also said at the same Tuesday conference that the AI firms should decide if new versions of the technology should be released and not be managed by outside forces.

"We don't need new laws or regulations," Huang said, adding that there should not be a "false choice" between the speed of innovation and the safety of AI products.

Nvidia is the biggest company in the world by valuation, with its profits having boomed as a result of rampant demand for the AI computing chips it makes.

"Safety is paramount. However, safety is an engineering problem," Huang said.

He added that if at any time a leader of an AI company lacks confidence in their product they should choose to not release it.

"That's a very obvious thing to do," Huang said. "Run as fast as you can, but if at any time you feel the institution is not in control, take a pause."

Why doomsday warnings are not the only threat to the AI juggernaut

Trump says AI safety fears a 'hoax' as he rejects calls for greater safeguards

Some industry figures have argued resurgent AI fears were overblown and were being jumped on to generate hype for the industry.

During the conference, Altman addressed a crowd of hundreds of business people, telling them they should use AI tools to aid in their work, but also needed them in order to protect their businesses from potentially AI-enabled cyber attacks.

The idea of AI executives being left to entirely regulate themselves has struck some in the industry as a bad idea.

Jack Clark, an Anthropic executive and co-founder, told the BBC on Monday that leaving AI to be a "totally unregulated industry" was "rolling dice with immense risks".

Patrick Hillman, the chief operating officer of Logical Intelligence, which is chaired by Yann LeCun, a sage of the AI industry, noted Tuesday how little faith people have in tech companies to do anything truly in the public interest, external.

"The only institution that Americans might trust less than Washington these days is Silicon Valley. I have worked and lived in both and I assure you both have earned this scepticism," Hillman said.

"If you believe what you are building is dangerous, show us what you are prepared to stop doing," he added.

AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC

AI regulation faces political deadlock as calls grow for Congress to act

Leaders of OpenAI and Anthropic have said they have recently started working towards some sort of industry-wide agreement on safety.

Amodei, during a cameo at the conference on Tuesday, said Anthropic is now in "a dialogue with the rest of the industry" about committing to better safety standards and checks on AI tools and development.

He also said that one of the biggest surprises of the AI boom was not how the technology itself has advanced, but its broader impact.

"We didn't appreciate that it would lead to companies growing so fast, and how quickly they would become central to things," Amodei said.

OpenAI executive Chris Lehane said last week that the company was also working with other AI labs, external "to advance frontier AI standards, building a voluntary effort now, with or without government support." Those labs include Anthropic and Google Deepmind.

"With stakes this high, we cannot let the perfect become the enemy of the good," Lehane wrote.

Do you have any comments or questions about this story?