'Need the world to wake up': AI 'Godfather's' stark warning after Trump's speech

Professor Yoshua Bengio, the co-chair of the UN's Scientific Panel on AI, says big tech companies are taking enormous risks with their development of the technology and there needs to be an urgent change of direction.

"Its shouldn't be the decisions of just a few CEOs or a few governments to figure out what's good for humanity," Professor Bengio told 7.30 from New York, where he is set to address the UN Security Council overnight.

The winner of the prestigious Turing Award for his pioneering work on AI said international agreement on safeguards is essential to secure humanity's future.

"Dealing with those risks … should be an urgent mission for humanity, so that humans remain in charge and that we keep human dignity flourishing at the centre of our choices,"

Professor Bengio said.

His comments came after US President Donald Trump labelled concerns about AI "a hoax" in a speech to the UNGA and rejected a joint statement from 22 countries, including Australia, calling for humans to remain in control of AI.

The US "totally rejects any attempt to construct a globalist scheme to control the artificial intelligence being spoken of so much now," Mr Trump said.

However, Professor Bengio, one of the 'Godfathers' of AI, said the plea from the 22 countries was important.

"What happened with this group of countries is extraordinary and it is important," he said.

"And we have to continue in that direction. We need more countries. We need the world to wake up so that we can have influence on our future.

"The reason I am able to sleep at night and the reason why a country like Australia can make a difference is that we have agency right now.

"When we form coalitions, when we put our resources together, our minds, our energy, our talent, we can be in the game and we can bring the world into a place that's better aligned with our values, with democracy, and the idea that we should keep our sovereignty in a multilateral world."

'Like a recipe for suicide'

The comments come on the eve of a meeting in New York between Mr Trump and Chinese President Xi Jinping where AI is expected to feature prominently. 

Professor Bengio said American AI companies are about six months ahead of their Chinese rivals, but that competition should not get in the way of cooperation between Washington and Beijing.

"It is reason enough to sit at a table with the Chinese because, you know, they're humans too. They want to have a life. They want their children to have a life," he said.

"I think it is in the rational interest, self-interest of both parties to sit down and agree on principles like the ones that the heads of state put forward recently."

Professor Bengio says the greatest dangers posed by AI come from "recursive self-improvement" being developed by the world's leading tech companies — where artificial intelligence trains the next generation of AI.

"Imagine we have an AI that is smarter than us, so it is able to do that research and engineering all by itself," he said. 

"It could design an AI that will be even more powerful than itself and even less controllable and even more malicious. 

"That's like a recipe for suicide."

Professor Bengio will tell world leaders at the UN Security Council that they must act.

"The forces of competition between companies and countries is not healthy," he said.

"It is an international question because those powerful AIs, if we lose control of them, that's going to affect all of us."

Watch 7.30, Mondays to Thursdays 7:30pm on ABC iview and ABC TV