$16,000 prize for S’pore-developed AI assistant that carries out tasks for visually impaired users

SINGAPORE – Like JARVIS, the digital assistant from the Iron Man films, a locally developed AI-powered mobile phone assistant can carry out tasks for its users with a simple voice command.

Called Sunday, it was designed to help people with visual impairments navigate smartphone apps more easily, whether they want to book a ride, send a message or shop online.

But its creators believe it can benefit others too, including seniors who are less familiar with technology and busy parents.

“We are not just targeting people who are visually impaired. We are also targeting people who are less tech-savvy, people who have motor impairment like hand tremors,” said Devadharshini Kumar, 23, former computing science student at the Singapore Institute of Technology (SIT) and software engineer at a technology company.

Unlike Apple’s Siri and Amazon’s Alexa voice assistants, Sunday can execute multi-step processes on a smartphone, such as looking up a restaurant and sending its address to a friend, or finding a flight confirmation in an email and adding it to a calendar.

Kumar is part of the team behind Sunday that clinched the top prize of $16,000 at tech firm Huawei’s Tech4City competition, which invites youth and start-ups to use AI for the public good.

Held at The Meeting Point at LaunchPad @ One-North on Oct 27 where the results were announced, the fifth edition of the competition required teams to submit working prototypes for the first time.

More than 15 schools, including universities, polytechnics, the Institute of Technical Education and junior colleges, submitted 240 entries – the largest in the competition’s history.

Led by Rajaselvam Kalyanaraman, 23, a second-year computer science and design student at the Singapore University of Technology and Design (SUTD), the Sunday team also comprises Kumar; Kunalkumar Senthilkumar, 22, a first-year computer science and design student at SUTD; and Pravin R, 23, an engineering and product design student at SUTD.

To use Sunday, people have to download an app and give it permission to use an Android phone’s built-in accessibility functions, which allow it to interact with what is on the screen. They can then tell it what they want done.

For example, a user could ask Sunday to send a photo from their album to someone on WhatsApp, find a red shirt on Shopee or book a Grab ride. The assistant narrates its actions as it goes.

Behind the scenes, the user’s spoken request is converted into text and sent to Google’s Gemini AI system, which works out the steps needed. Sunday then uses the phone’s accessibility functions to carry out actions such as tapping, scrolling and typing, Kumar said.

The prototype currently works only on Android phones, as Apple’s iOS does not allow third-party apps the same access needed for Sunday to control other apps on an iPhone.

Sunday has built-in safeguards: It will stop when it encounters a Singpass login or banking function, and will not press the final button to authorise a payment.

Users have to complete these steps themselves. A person with visual impairment can use a screen reader, which reads out what is on the screen and lets them select the relevant button.

Kumar said the payment safeguard was requested by users who wanted to retain control over sensitive transactions.

The Sunday team said 73 people have trialled the app through the Singapore Association of the Visually Handicapped (SAVH) and other social service agencies.

For SAVH volunteer Gordon Ong, 69, downloading a document from an e-mail and sending it over WhatsApp is not an easy task. He is completely blind in his left eye and has very hazy vision in his right, making it difficult to press buttons accurately on his phone.

He said Google’s Gemini can describe the steps needed to complete a task, but he still has to carry them out himself. Sunday, in contrast, can perform the steps for him.

“This app is essential if there are complex, multi-step processes I have to do on my phone,” said Ong.

A beta version has been submitted to the Google Play Store for review and could be available in the next few weeks, subject to approval. The team plans to charge US$10 (S$13) a month and hopes to use the prize money to speed up the assistant’s responses.

AI-powered revision for O levels

Tech4City’s second prize went to EduNets, a team of six Singapore Institute of Management students and graduates who received $11,000 for an AI-powered revision website for O-level students.

The platform currently covers chemistry, mathematics and biology. Students can take quizzes and upload handwritten or typed notes, which the AI analyses to identify gaps or misconceptions in their understanding. It also provides flashcards for revision.

“Sometimes students will just dump their ideas and whatever they’re studying into notes. But actually, from your understanding, you’re missing some key important points that might actually help you score the extra mark,” said team leader Beatrice Faustine Irawan, 22, an accounting graduate.

About 125 students have tried the platform after hearing about it through the team’s personal networks. The team hopes to work with tuition centres on trials in 2027 and eventually expand the platform to cover the Primary School Leaving Examination and A-level curricula.

A finalist in the competition, Drill Mode, uses simulated scam scenarios to help users recognise how scammers use fear and pressure to persuade people to part with their money.

The project was inspired by the experience of Ng Wen Jing, a 21-year-old second-year computer engineering student at Nanyang Technological University. Her older sister lost nearly all her savings of $17,000 after a scammer posing as an Alibaba agent told her she needed to pay to cancel an insurance product.

“We have fire and emergency drills, but we don’t have scam drills. So we would like to raise awareness on scams,” said Ng, who added that the idea for Drill Mode came to her before the Government rolled out Scam Vaccine recently.

Users sign up with their phone number and e-mail address and can take part in simulated scam calls, messages and e-mails. After responding, they receive a debrief on what they did well and what they could improve.

Minister for Social and Family Development Masagos Zulkifli, who attended the finals of the competition, said that the true measure of innovation is not how sophisticated one’s technology is, but how much difference it can make to people’s lives.

“When we continue to nurture the spirit of innovation and create even more opportunities for our young people to turn promising technologies and ideas into solutions that benefit our society, AI becomes more advanced,” said Masagos.