IndQA

IndQA was OpenAI’s attempt to test something existing multilingual benchmarks did not: not whether a model can answer a general-knowledge question that has been translated into an Indian language, but whether it understands context that only makes sense within Indian culture in the first place. The set covers 2,278 questions across 12 languages and language varieties — including Hinglish, the code-switched mix common in everyday Indian speech — spanning ten domains from law and religion to food, history and media, drafted with 261 domain experts based across India.

OpenAI argued that benchmarks like MGSM and MMMLU mostly tested translated general knowledge or reasoning rather than reasoning that depended on local context, and built IndQA specifically to require the latter. Its announcement compared GPT-5 Thinking against other frontier systems and said performance had improved over successive model generations, but the company published no full numerical scores or public leaderboard alongside the release.

The benchmark arrived as OpenAI and its competitors were investing heavily in the Indian market through 2025, and it fits a broader pattern among frontier labs of building region-specific evaluations rather than relying solely on English-centric or translated tests as they compete for users outside English-speaking countries. As of this writing, IndQA has no independent public leaderboard or third-party dataset release to check OpenAI’s own reporting against.