As Festive Shoppers Turn To AI, Hindi Queries Face A Hidden Accuracy Gap

As Diwali shopping picks up, Indian consumers are leaning on AI chatbots more than ever. Google Trends data shows 51 per cent of festive shoppers credited AI with helping them decide what to buy, and consulted chatbots 7.5 times on average during a purchase journey. A separate InMobi report found 80 per cent of Indian shoppers are ready to use AI shopping assistants.

However, many of these shoppers ask questions in Hindi. And a Stanford study suggests that AI answers (in Hindi) may not be as reliable as those given in English. Stanford University's Institute for Human-Centered Artificial Intelligence (HAI) tested six commercial chatbots on questions about current news across six language regions. Hindi came last.

Hindi responses averaged 79.3 per cent accuracy, nearly 10 percentage points below the next-weakest region.

• Every chatbot tested performed worst in Hindi.
• Error volume was roughly twice that of any other region.

The Problem Is Retrieval, Not Comprehension

The chatbots did not struggle to understand Hindi. They struggled to find the right evidence. Two failures explained over 70 per cent of wrong answers:

• Retrieval failure (38.8 per cent): The chatbot could not find sufficiently relevant information.
• Source divergence (32.7 per cent): The chatbot found material on a similar topic, but the facts differed.

When the correct source was retrieved, the chatbots almost always extracted the right answer. The citations show where the gap begins. English Wikipedia was the most-cited domain for Hindi queries, ahead of every Hindi news outlet in the study's index. Nine of the ten most-cited domains across all responses were primarily English.

Kartik Sharma, CEO and founder of RankinLLM.ai, said the finding points to a structural flaw. "The models read Hindi perfectly well. They then went looking for the answer in English," he told NDTV. The pattern across all six chatbots, he said, means "this is not one company's blind spot."

What It Means For Shoppers And Brands

The study tested news questions, not purchases. It did not examine product or service queries. Still, Sharma said the same behaviour could shape commercial answers. A Hindi-speaking user asking which insurance plan suits their parents, or which university to pick for a child, could get an answer drawn from different sources than an English-speaking user would.

For brands, the risk is visibility. Sharma said a company can appear prominently in an English AI answer and be nearly absent from the Hindi one, because the two draw on different evidence. "A brand does not have one AI reputation," he said. "It does not even have one per language."

His advice is to stop treating this as a translation task. "Publishing Hindi pages on your own website will not fix it if no credible Hindi source anywhere else corroborates what you claim," he said. Brands should build credible Hindi coverage beyond their own sites and check whether AI tools actually retrieve it.

Limits Of The Findings

The researchers also found that accuracy dropped sharply when questions carried subtle factual errors or misleading assumptions. A separate free-response test showed a 16-17 per cent fall in absolute accuracy. All queries came from US-based servers, which may have pushed chatbots towards English sources.

None of this means every Hindi answer is wrong. It does show that a strong overall accuracy score can hide large gaps between languages.

With festive spending at its peak, verify important claims, such as prices, product specifications, financial products and admissions details, against credible sources before acting on them. This becomes particularly significant when the question is asked in Hindi.