NVIDIA and IIT Madras's AI4Bharat Announce a 2.8-Trillion-Token Corpus for All 22 Scheduled Indian Languages -- Machine-Translated From English, Graded Against a 75-Point Bar, and Already Inside Nemotron 3.5 Super VL

India's sovereign-model programme has funded GPUs and model builders, as this digest covered on October 18. The scarcer input has always been text. On October 16, 2026, NVIDIA and AI4Bharat at IIT Madras published "Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline" on Hugging Face, announcing Nemotron-pretraining-Indic-v1. AI4Bharat's head, Mitesh Khapra, states the problem plainly: "we do not have enough data for Indian languages."

What is in the corpus

• Size. About 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, counted with the Gemma-4 tokenizer, across the 22 languages in India's Constitution. All 22 have native-script data; twelve also have romanized text, for 34 language-script configurations. NVIDIA notes the counts describe the retained corpus, not a training budget.
• Source. The documents are English, drawn from 15 subsets of NVIDIA's Nemotron-CC-v2 web corpus covering web text, question answering, math, code and specialised content.
• Translation. For twelve languages, Google's Gemma-4-31B-it translates whole documents and then judges its own output on a 100-point rubric inspired by the Multidimensional Quality Metrics framework; anything under 75 is repaired from the assessment feedback and rescored. That repair loop recovered 438.4 billion tokens. For the other ten languages, Indic Translate, a document-level model from Bodhan AI, an IIT Madras-incubated nonprofit, does the translation and Gemma-4-31B-it assesses it.
• Openness. The pipeline runs on NVIDIA's open-source NeMo Data Designer, every record keeps a source ID, and the recipe ships as a Nemotron Bharat translation plugin on GitHub. The corpus itself is "releasing soon"; the post does not name a licence.

Already used in a shipping model

NVIDIA released Nemotron 3.5 Super VL on October 15: a 120-billion-parameter mixture-of-experts model with 12 billion active parameters, text, image and video input, a context of up to 1 million tokens, and weights on Hugging Face under the OpenMDW 1.1 licence, "ready for commercial or non-commercial use". Its training recipe starts with 1 trillion tokens of distillation from Nemotron 3 Ultra Base. Later comes an Indic continued-pretraining stage the model card describes as "100B tokens: 10B selected Indic tokens and 90B tokens from the preceding blend." The Hugging Face post says those Indic tokens came from a twelve-language subset of the new corpus plus natural Indic text, and reports gains of 4.98 to 9.82 percentage points on five Indic benchmarks, with the caveat that the gains reflect the whole stage, not the synthetic corpus alone. On general benchmarks NVIDIA's blog reports 63.44% on Hermes Bench against 56.33% for Nemotron 3 Ultra, 87.05% on LangChain Evals, and 80.8% on the CharXiv chart-reasoning test; the model card lists 71.07 on SWE-bench Verified and 62.78 on Terminal-Bench 2.1.

What remains uncertain. The corpus is not yet downloadable, and its licence is unstated. Quality control is mostly automated, and for twelve languages the translator is also the judge; NVIDIA itself says the ten-language workflow "needs broader native-speaker validation", that duplicate removal is by source ID rather than meaning, and that translated text is no substitute for text originally written in Indian languages and does not fully capture local knowledge and culture. Ten billion Indic tokens in a 100-billion-token stage is a modest dose, and the Indic benchmark gains are NVIDIA's own.

Analysis: This is the first Indic pretraining set at a scale comparable to English web corpora, and it is free to anyone, including the IndiaAI Mission's funded builders. It is also, by construction, the English web rendered in 22 scripts, which carries English topics, sources and assumptions into every language. The right use is as a base layer: pretrain on it, then measure on evaluations written natively in each language and weight the native text heavily in the final stages. Builders who adopt it should publish which configurations they used, because a claim of training on 2.8 trillion Indic tokens will otherwise say little about Hindi, Santali or Manipuri quality.