OpenAI Open-Sources MultiLangBench -- Build a SQLite Engine in 20 Languages -- and Says GPT-6.1 Sol Migrates COBOL at 86.6% Where GPT-5.6 Sol Managed 5.4%

On October 19, 2026, OpenAI published "Advancing coding in low-resource programming languages", arguing that COBOL, Fortran, Visual Basic and OCaml have far less public code and documentation to learn from than Python or JavaScript, yet run banking, weather forecasting and reactor safety analysis. The post says the work was "guided by feedback from developers and enterprise customers who have told us that coding models often struggle with these languages," and introduces two evaluations: one for building new software, one for preserving what old software does.

MultiLangBench: one database engine, 20 languages

MultiLangBench asks a model to build a SQLite-compatible database engine in a specified language over an extended coding session. Database files are the shared interface: the candidate program receives a database and a SQL query, produces a modified database, and the reference SQLite implementation checks the contents, using tests adapted from SQLite's own suite. That design lets one test harness grade C, COBOL, Lean, Zig and PowerShell alike. The setup, test suite and per-language results are on GitHub as openai/MultiLangBench, created on October 19 under the Apache-2.0 licence.

• GPT-6 Astra at maximum reasoning effort averages an 87.4% test pass rate across the 20 languages, against 26.8% for GPT-5.6 Sol, and scores above 83% in 19 of 20. COBOL stays hardest: 10.9% to 72.1%. OCaml goes from 25.6% to 85.8%, Fortran from 21.4% to 83.5%, Visual Basic from 27.5% to 89.4%.
• GPT-6 Luna reaches 40.3% on average at an estimated $0.77 per sample, against $3.48 for GPT-5.6 Sol -- about 78% cheaper, but with COBOL only moving from 10.9% to 17.1%.

COBOL migration: parity or nothing

The second evaluation migrates COBOL programs to C++, Java, Kotlin, Python or Rust, graded by hidden tests that require byte-for-byte parity on observable outputs. "A migration is considered successful only if all tests pass." The post explains why that bar matters: COBOL's fixed-point arithmetic drops excess decimal digits unless rounding is requested, and "A rewrite that rounds those values instead can run without errors while producing different financial results." GPT-6.1 Sol scores 86.6% at low reasoning effort, against 5.4% for GPT-5.6 Sol at the same setting, at 61% of the API cost; at higher effort it reaches 97.9%, which OpenAI calls close to saturation. OpenAI adds that production use "still requires expert review and testing against the system's business rules and dependencies."

What remains uncertain. The two headlines use different models -- Astra on MultiLangBench, GPT-6.1 Sol on migration -- and the post gives no Astra migration score or 6.1 Sol MultiLangBench score, so the two results cannot be lined up. The migration test, its program set and its hidden tests are not released, so the 86.6% and 97.9% figures are OpenAI's own. Costs are "estimated" API costs, not measured bills. And a SQLite clone is a well-specified target with a reference oracle; real legacy estates have neither, which is exactly the gap the parity requirement is meant to expose.

Analysis: The useful part for engineering teams is the method, not the leaderboard. A shared file-format interface plus a reference implementation turns 'can the model write COBOL?' into an executable differential test, and the migration eval's all-tests-or-fail rule is the right default for financial code, where a plausible rewrite that rounds differently is worse than a failed one. Teams planning modernisation can copy both ideas this week: capture golden outputs from the legacy system, demand parity before anything else, and treat COBOL arithmetic semantics as a test category of its own. The open MultiLangBench suite also lets anyone check whether other vendors' models show the same gap between mainstream and low-resource languages -- the claim that matters more than OpenAI's own scores.