BenHalluEval: A Multi-Task Hallucination Evaluation Framework for Large Language Models on Bengali
arXiv:2605.31483v1 Announce Type: new Abstract: Despite Bengali being the sixth most spoken language in the world, no prior work has systematically evaluated hallucination in large language models (LLMs) for Bengali. We introduce BenHalluEval, a fine-grained hallucination evaluation framework for Bengali covering four tasks: Generative Question Answering (GQA), Bangla-English Code-Mixed QA, Summarization, and Reasoning. We construct 12,000 hallucinated candidates using GPT-5.4 across twelve task-specific hallucination types, drawn from three existing Bengali datasets, and evaluate seven LLMs s
The proliferation of LLMs creates an immediate need for robust evaluation frameworks across diverse languages, and this work addresses a critical gap for Bengali.
Evaluating hallucination in highly-spoken non-English languages is crucial for responsible global AI deployment and building localized, trustworthy AI experiences for large populations.
The availability of a multi-task hallucination evaluation framework for Bengali will enable more targeted improvements in LLMs for this specific language, potentially increasing their utility and reliability.
- · Bengali-speaking users
- · Developers of Bengali LLMs
- · AI ethics researchers
- · Generic, unlocalized LLMs
- · Users relying solely on English-centric AI
BenHalluEval directly provides a much-needed benchmark for assessing the reliability of LLMs in Bengali.
This framework will likely accelerate the development of more accurate and less-hallucinatory LLMs for Bengali and potentially other under-represented languages.
Improved non-English LLMs could lead to greater digital inclusion and expanded economic opportunities for populations relying on these languages, reducing the digital divide.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL