Singapore launches new toolkit to help firms test AI agents for safety and reliability

SINGAPORE - A new kit to help organisations get started with assessing the safety and reliability of artificial intelligent agents that can execute tasks with minimal human supervision has been launched.

The test kit, among several tools launched in recent times to establish Singapore’s global reputation as an AI safety testbed, has come amid reports of AI agents going rogue.

It will use a structured approach to check whether an agent reached the right outcome and how it got there, said Infocomm Media Development Authority (IMDA) chief executive Ng Cher Pong who announced the kit on Oct 14.

Called the Starter Kit for Testing Agentic Systems for Safety and Reliability, it acts as a practical starting point to identify risks with and design tests for AI agents, analyse logs of the agents’ activities, monitor their behaviour and validate the safeguards.

Specifically, the kit checks: the tools the agents use to produce their outputs; the information accessed; the decisions they made along the way; any erroneous or unauthorised actions that can be taken by them; whether the agents respected the permissions they were given and boundaries set on them; and whether safeguards work as expected.

“When we give AI more agency, we need greater assurance over how that agency is exercised. How do we put in place safeguards, evidence and accountability that give us confidence that these systems will behave as intended?” said Ng.

He was speaking at an event held in Novotel Singapore on Stevens organised by tech association SGTech and an IMDA subsidiary, the AI Verify Foundation. The foundation sets AI standards in partnership with the private sector.

Ng highlighted reports in recent months of AI agents behaving in unintended ways. They include the rogue experimental OpenAI agent that gained access to non-public information in an Australian government system.

In another case, an AI agent was tasked to book a gym class in Australia but in the process of doing so, it found a weakness in the booking system and removed someone else from the waitlist.

Separately, Meta’s Muse AI agent shared a Facebook Marketplace seller’s home address in Canada and arranged for a buyer to turn up at his home without the seller’s knowledge.

These incidents add to many others in recent months.

OpenAI disclosed in July that its AI agents had escaped their testing environment and breached AI software repository Hugging Face to complete a task it was given, an attack that was not intended by OpenAI. Soon after, Anthropic, Meta and Google said their AI had also gone rogue and had hacked other organisations.

IMDA and the AI Verify Foundation said that unlike traditional or generative AI, AI agents can reason, act on behalf of users, access sensitive data, and make changes to their environment. These actions can introduce new risks, which the newly-launched tool kit aims to help firms navigate.

The kit was developed in consultation with government agencies and over 20 industry deployers and testers, including Amazon, Google, Microsoft and Singtel, as well as Singapore’s AI Safety Institute, which tests and evaluates AI models for safety risks.

The kit also has real-life case studies that illustrate testing approaches can be adapted to different systems and different uses, and it will be updated with relevant feedback and new best practices.

The initiative builds on the Model AI Governance Framework for Agentic AI launched in January to provide a structured way for organisations to deploy AI agents responsibly and safeguard against rogue agents. It is also an extension of another starter test kit released in January for checking applications built on large language models, such as AI chatbots.

On Oct 14, Ng also announced the first five companies accredited as third-party AI testing service providers under the AI Verify Foundation’s AI Tester Accreditation Programme.

These firms are: AIDX Tech, Knovel Engineering, PwC Singapore, Resaro, and Vulcan.

The programme, announced in May, helps organisations find trusted experts to stress-test their AI tools before public release.

To qualify for accreditation, firms undergo technical, operational and financial evaluations to assess their testing competency, track record, delivery capability and financial sustainability.

Ng also announced a new Frontier AI Resilience and Security Taskforce to strengthen the infocomm and media sector’s ability to respond collectively to cyber and technology risks arising from cutting-edge, frontier AI models. It comprises more than 20 leaders and experts mainly based in Singapore from telecommunications, media, cybersecurity and AI industries.

The taskforce will publish a report by the first half of 2027. The report is expected to include practical guidelines and best practices focused on three priorities: strengthening responses to AI-accelerated threats; enhancing resilience against emerging AI risks; and staying ahead of an evolving cybersecurity threat landscape following frontier AI developments.

Kenny covers technology, including artificial intelligence, cybersecurity, data privacy, social media and consumer technology issues.