Independent, non-commercial initiativeVarna, Bulgaria · Founded in 2026
MATREYA

Institute for Buddhism, Science & Artificial Intelligence

Wisdom · Inquiry · Compassion
For the benefit ofall living beings
English

Research

DharmaBench tests language models on Buddhist texts—without calling the result understanding

A 2025 ACL benchmark makes 13 defined tasks in Sanskrit and Classical Tibetan inspectable; it does not automate Buddhist authority.

Editorial illustration of stones and geometric forms arranged around an evaluation lens
Illustration. DharmaBench: a task map for evaluating models. Original AI-assisted editorial illustration, not a DharmaBench dataset scan or a claim of religious authority for a model.Matreya editorial team · AI-generated illustration · Original AI-assisted editorial illustration; rights reserved · Source & original

A December 2025 ACL paper introduces DharmaBench, a benchmark for language-model evaluation on Buddhist texts in Sanskrit and Classical Tibetan. The authors describe 13 classification and detection tasks: six for Sanskrit and seven for Tibetan, with four task types shared across the two languages. This makes a bounded set of philological and textual questions measurable rather than treating Buddhist traditions as a generic prompt domain. [1]

The practical benefit is auditability. The public repository exposes task data, annotation guidance and evaluation code, while the dataset release gives readers a second route to inspect what is included. A reported score can therefore be read beside the task definition, examples and split—not as an unexplained verdict delivered by a model. The authors state that the datasets are released under CC BY 4.0 and the code under Apache-2.0, so reuse still depends on checking the applicable terms. [1] [2] [3]

The boundary matters. A result on DharmaBench does not show that a system understands Buddhism, can settle a textual variant, or produces reliable translations in every context. It measures performance against labels and tasks defined by the project’s authors. Researchers and readers can use the benchmark to ask sharper questions about evidence; interpretation and responsibility remain human work. [1] [2]

Sources

  1. Hashiloni et al. — DharmaBench: Evaluating Language Models on Buddhist Texts in Sanskrit and Tibetan (IJCNLP-AACL 2025; DOI 10.18653/v1/2025.ijcnlp-long.114)
  2. Intellexus-DSI — DharmaBench repository: task data, annotation guidance and evaluation code
  3. Intellexus — DharmaBench dataset release