Mistral AI's Leanstral 1.5 Achieves Unprecedented Breakthrough in Automated Reasoning

Mistral AI has achieved a significant breakthrough in automated reasoning with the release of Leanstral 1.5, an Apache-2.0 licensed Lean 4 code agent model. This model has solved an unprecedented 587 out of 672 challenging PutnamBench problems and saturated the miniF2F benchmark, a performance that far surpasses previous attempts, including advanced models like GPT-4 which could only solve a handful.

Leanstral 1.5, a free and open-source model, leverages a 119-billion-parameter Mixture-of-Experts (MoE) architecture. During inference, it efficiently activates only 6.5 billion parameters per token. This sparse activation is a key characteristic of MoE designs, enabling immense model capacity while maintaining computational efficiency—a strategy Mistral AI successfully employed with its Mixtral models. The model is specifically designed for Lean 4, a modern interactive theorem prover and functional programming language vital for scalable formal verification and mathematical theorem proving. Beyond its benchmark achievements, Leanstral 1.5 has demonstrated practical utility by uncovering five previously unknown bugs across 57 open-source repositories.

This release marks a pivotal moment for formal verification and AI-assisted software development. The ability of an AI agent to tackle complex, undergraduate-level mathematical problems with such high accuracy signifies a major leap in automated theorem proving (ATP) capabilities. It heralds a future where AI can not only assist in coding but also rigorously verify correctness, addressing a critical need for high-assurance systems in fields like cybersecurity and blockchain. Mistral AI's commitment to open-sourcing such a powerful tool is expected to accelerate research and adoption, potentially transforming formal verification from a specialized academic pursuit into a mainstream practice.