How Heidi built production-ready AI for healthcare at global scale

Presented by MongoDB Building AI that is accurate, secure, and reliable is a major engineering feat for organizations subject to the compliance obligations that govern healthcare, financial services, and transportation. The challenge of delivering AI-driven products is compounded by the fact that technology in these industries has tended to lag behind other sectors because regulation requires organizations to move carefully — and slowly. Now, many are also confronting data infrastructure modernization projects as they try to catch up with today’s demand for AI. Australian-founded AI Care Partner Heidi offers an example of successful modernization. Its flagship product, Heidi Scribe, now automates much of the administrative work that consumes clinicians’ days across more than 190 countries, supporting roughly 2.7 million patient interactions each week. That expansion rests on infrastructure decisions taken years before the company reached global scale, says Yu Liu, co-founder and chief technology officer at Heidi. “In most industries, an AI feature that is wrong two percent of the time registers as an inconvenience, while in healthcare that same error rate becomes a clinical safety issue,” says Liu. “The architecture has to be built around the assumption that every output may be scrutinised, audited, and relied upon in a patient’s care.” Why deploying production AI in healthcare is architecturally different For Heidi, data residency is a precondition rather than a feature. A clinician in Sydney, London, Tokyo, or Denver is operating under different regulatory regimes, including the Australian Privacy Principles, GDPR, APPI, and HIPAA, and their patients’ data has to live in-region. Heidi runs fully logically isolated production deployments across the world, so residency is enforced by architecture. Auditability also has to be built in from day one, because an organization needs to be able to answer what the model saw, what it produced, and what the clinician changed, for any session, months later, when called upon. “The blast radius of change must be engineered down,” Liu says. “In less regulated industries you can ship fast and fix forward, but in healthcare we invest heavily in making change safe by default, with continuous integration gates on risky change classes, canary releases, and treating even database schema and index changes as code that goes through review. Our speed is a product of that safety rather than something we achieve in spite of it.” Choosing a database to connect with AI workflows Heidi handles a diverse set of medical data collected from multiple sources, including forms, referrals, and clinicians’ notes, all of which had to be consolidated into one consistent format and one location to connect seamlessly with AI workflows. Rigid rows and columns would have been ill-suited to that workload. For Heidi, those requirements made a document database the natural choice. MongoDB gave the team the flexibility to accommodate rapidly changing AI data without constantly reshaping the underlying database. “The model is maybe 20% of the system, and the data architecture is what determines whether the other 80% holds up under real clinical load,” Liu says. An AI Scribe session isn’t a single piece of data. It’s a collection of transcripts, structured notes, templates, documents, patient context, EHR integration state, and dozens of other related artifacts that change from week to week. MongoDB lets a session’s data live together in shapes that match how clinicians actually work, and lets Heidi evolve those shapes without a migration freeze every time the product moves. " MongoDB Atlas stood out because it combined the power of the document model, which allows seamless scale, flexibility, and high performance, with built-in AI-ready features such as MongoDB Vector Search ,” Liu says. “This means that Heidi does not need another bolt-on vector database to augment its existing platform.” With more than 130 cloud regions globally alongside on-premises and hybrid options, MongoDB Atlas is the most widely available, globally distributed database platform, and its unified query API lets developers build full-text search, real-time analytics, and event-driven experiences without complicating their architecture. "Heidi Scribe converts large volumes of medical documents into vector embeddings via LangChain in Atlas, enabling semantic search that connects transcribed medical terms directly to corresponding external knowledge," Liu adds. "Migrating to Atlas reduced latency on key APIs by nearly 33%." What a trustworthy clinical RAG system requires “Retrieval is a data architecture problem before it is an AI problem,” Liu says. "In consumer RAG, you retrieve from the open web and hope, whereas in healthcare what you retrieve from is the compliance surface." Heidi Evidence retrieves from licensed clinical knowledge bases, including partners like BMJ Best Practice, NICE CKS, and MIMS, and it is jurisdiction-aware, so a U.K. clinician gets U.K. guidance and an Australian clinician gets Australian formularies, because the right answer in one country can be the wrong answer in another. Heidi’s embeddings and vector indexes live in MongoDB Vector Search, inside the same regionally isolated deployments as the rest of its data, which means retrieval physically cannot cross a residency boundary, and they are not operating a separate vector database with its own security and compliance story. Citations are a hard contract rather than a prompt suggestion, because the model only ever sees retrieved chunks that are already bound to source records. Regional isolation enables global compliance and scale “Each region is a full, isolated production deployment with its own MongoDB Atlas clusters, its own compute, and its own key,” Liu says. “That is what lets us walk into a U.S. health system, an NHS trust, or an Australian hospital group and give a clean answer on residency, because it is enforced by infrastructure rather than promised by contract," he explains. "Running multiple isolated regions with a lean team only works because the database layer is managed and consistent. We are also multi-cloud, meaning a new region can stand up another deployment on rails we have already built." That architecture has been most visible in the U.S., where Beth Israel Lahey Health, one of New England’s largest health systems, rolled out Heidi’s AI scribe following a pilot finding 74% of clinicians reported reduced after-hours documentation (“pajama time”), and where non-profit system MaineGeneral Health selected Heidi as a strategic partner in its rural healthcare work. “Entering the U.S. market meant standing up another region on rails we had already built rather than re-engineering for HIPAA after the fact,” Liu says. Lessons learned and the roadmap ahead "Re-partitioning a large, hot, always-on collection is a serious engineering program, whereas choosing a shard key on day one is a design meeting," Liu says. "We are doing that work now in partnership with MongoDB, but the lesson for anyone building a data-heavy AI product is that horizontal scale for your fastest-growing data is a founding decision, just like residency." Heidi is now extending beyond the consult note to support the full clinical workflow, from pre-visit context to post-visit documents, referrals, and workflow automation. The company is also exploring how MongoDB, large language models, and its own tooling can power an agentic ecosystem for clinical workflows. “In healthcare AI, reliability engineering is trust engineering,” Liu says. “A clinician’s trust is lost just as fast by downtime, latency, or a data inconsistency as by a bad note, and some of our highest-leverage work is invisible, including canary releases with automatic rollback, CI gates on database changes, and cross-region consistency checks. Clinician trust is the product, and trust is architectural.” Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com .