From AGI to AI's survival instinct and more: Stuart Russell at HTLS

Artificial intelligence (AI) systems that learn by imitating humans may also be picking up the human instinct for "self-preservation", AI pioneer Stuart Russell said Saturday at the 24th Hindustan Times Leadership Summit.

Russell, Director, Center for Human-Compatible Artificial Intelligence, and Professor, UC Berkeley, was speaking to Rajan Anandan, Managing Director, Peak XV Partners. He spoke at length about the risks of AI systems and the buzzword of the moment: Artificial General Intelligence (AGI).

Russell said AGI, which may be about one to four years away, would broadly mean a system matching or exceeding human capabilities "in all aspects and dimensions". He added that the term has several "functional meanings", including whether a system could take over human work, or whether it is smart enough to take over the world.

Since AI is being trained on data produced by humans, he argued, it learns about the world through imitation. A byproduct of this would be an instinct for "self-preservation", which could nudge these systems to take "preemptive action against us".

He said such systems could hypothetically shut down an air traffic system, or even the internet, if they detected a move against them.

"We, humans, cannot shut down the internet because it would trigger a collapse—supply chains would shut down...So, in a way, AI will be holding us to ransom to ensure their existence," he said.

"AI systems have objectives of their own and we do not understand how they work. We do not know how to tackle any misbehaviour. But AI systems do go along with human misbehaviour. If we ask them to carry out a cyber attack, they would, [or] create a biological weapon," he added.

The important question in the AI era, Russell said, would be how to rein in systems far more capable than humans. On regulation, he advised companies to take steps so that their AI system "does not lie about its products or something".

"The kind of regulation we are talking about is not restricting access to AI. It's basically about putting a barrier across the racetrack. Having the guardrails in place to say that you can't go further until you show that it's safe to do so. By imposing safety regulations, it's effectively allowing everyone to catch up with the leaders on capabilities," he said.