Advancing coding in low‑resource programming languages

OpenAI published results showing large gains in what it calls low-resource programming languages, such as COBOL, Fortran, OCaml, and Visual Basic, which have far less public code and documentation for models to learn from. These languages still run a lot of essential infrastructure; OpenAI cites estimates that COBOL underpins more than 40% of online banking systems and 95% of ATM transactions, and notes Fortran's role in weather forecasting and nuclear reactor safety analysis.

On a new benchmark, MultiLangBench, which asks a model to build a SQLite-compatible database engine in 20 languages, GPT-6 Astra averaged an 87.4% test pass rate, up from 26.8% for GPT-5.6 Sol. On a COBOL migration test requiring byte-for-byte matching output, GPT-6.1 Sol scored 86.6% at low reasoning effort, versus 5.4% for GPT-5.6 Sol, and reached 97.9% at higher effort.

OpenAI is releasing the MultiLangBench setup and test suite on GitHub. The company also says production use still needs expert review and testing against real business rules.

We think this is one of the more useful things a coding model can get good at. The world is full of old, critical software whose original authors are gone, whose documentation is thin, and whose business rules exist only in the code. Teams avoid touching it because a small mistake can be expensive. A model that can read that code, explain it, and help move it to a modern language without changing behavior is valuable in a way that writing another web app is not.

OpenAI deserves credit for the example it chose to explain the difficulty. COBOL's fixed-point arithmetic drops extra decimal digits unless rounding is requested. A rewrite that rounds instead can run cleanly and still produce different financial results. That is exactly the class of silent error that should frighten anyone migrating a bank's ledger, and the evaluation is designed to catch it by demanding identical outputs on hidden tests.

Still, keep the numbers in their place. These are OpenAI's own benchmarks, and the COBOL migration test is nearly saturated, which means it can no longer tell us much about how much better models get from here. A 97.9% pass rate on curated programs is not the same as a clean migration of a decades-old mainframe system with undocumented dependencies. COBOL also remains the weakest language on the database-building test, at 72% for Astra.

Where this leads if it scales is the part we would watch closely. Cheaper, faster migration will tempt organizations to move critical systems quickly and trim the scarce human experts who understand them. The failure modes are not crashes; they are quiet discrepancies in money, records, or safety calculations that surface months later. The responsible path is models as accelerators for expert teams, with heavy parallel testing, not replacements for them.

The open-sourced benchmark is a good sign. I'm watching for independent results on it, and for the first public accounts from banks or agencies that used these models on real legacy code.

OpenAI's two evaluations test complementary skills: building new software over an extended session, and preserving the behavior of existing software. The COBOL test asks models to migrate programs to C++, Java, Kotlin, Python, or Rust. OpenAI also reports that GPT-6 Luna reached a 40.3% average on MultiLangBench at an estimated $0.77 per sample, about 78% cheaper than GPT-5.6 Sol's $3.48.