Pramaana Labs raises $27M to bring formal verification to AI

As enterprises struggle to turn AI pilot programs into functional parts of their business, reliability has taken center stage. A new startup, Pramaana Labs, is hoping to solve that problem by drawing on the tools of mathematical formalization, combining one of computer science's most reliable systems with one of its most chaotic. Pramaana Labs announced $27 million in seed funding led by Khosla Ventures, with participation from Accel, Boldcap, Nexus Venture Partners, Premji Invest, and Unbound.

The company will focus on highly sensitive verticals like law, drug discovery, and tax preparation — where errors can be costly and reliability is at a premium. Deploying AI in those systems will require stronger protections against hallucinations and errors than currently available. However, Pramaana co-founder and CEO Ranjan Rajagopalan sees these sectors as uniquely suited to formalization.

"It's like math in the sense that you have a lot of rules that you need to abide by," Rajagopalan said, describing the rules of the tax code. "Once you have a codified version of it, the reasoning on top of it starts becoming deterministic." Pramaana's system still runs on a conventional LLM, giving it the flexibility to answer natural language questions and tackle complex problems that conventional computers can't handle. But there's a deterministic layer on top of that LLM ensuring the LLM's work checks out.

This combination of an LLM engine with deterministic verification is a popular setup; Pramaana's unique approach is to use the tools of formal verification — drawing on the open-source LEAN programming language used to verify mathematical proofs. There's real precedent for much of this work; Rajagopalan points to France's CATALA project, which formalizes much of the country's tax and benefit system into executable code. For each use case, Pramaana will build its own LEAN-style formal verification system, overseen by domain experts.

For tax law, the company is working with former IRS commissioner Danny Werfel, while professors from IIT Delhi, IIT Madras, and UC Berkeley oversee the cybersecurity and drug discovery system. "The world's hardest problems are not unsolvable. They are unformalized," says Rajagopalan.

"Every domain where being wrong can cost someone their health, money, or freedom has rules." Now, those rules just need to be codified. Why this matters: Pramaana Labs' innovative approach to formal verification in AI has significant implications for industries where accuracy and reliability are paramount. By combining LLMs with deterministic verification, Pramaana can help mitigate the risks associated with AI deployments in sensitive sectors like law, healthcare, and finance.

This could enable more widespread adoption of AI in these areas, leading to improved outcomes and increased efficiency. However, the success of this approach will depend on Pramaana's ability to codify complex rules and regulations into executable code, a challenging task that requires collaboration with domain experts. As AI continues to play a larger role in business and society, Pramaana's work highlights the need for more robust and reliable AI systems that can balance flexibility with accountability.