Can local AI be good enough for voice-driven clinical documentation?
Clinical Voice
2026
Can a locally hosted open-source model turn spoken clinical-trial notes into structured documentation well enough to be useful — or does the workflow still need frontier AI?
A bilingual voice-to-CRF prototype benchmarking local open-source models against frontier systems, showing where local is good enough — and why system design matters as much as model choice.
Overview
Clinical Voice explores a practical decision: when sensitive clinical-trial data needs to become structured documentation, is a locally hosted model good enough—or does the workflow still need frontier AI?
We built and benchmarked a complete voice-to-CRF (Case Report Form) prototype in English and Polish: a native macOS collection app, a guarded backend, a purpose-built bilingual dataset, and an evaluation system comparing local open-source models with frontier alternatives.
The local stack nearly matched the frontier stack in Polish, while the frontier retained a clearer advantage in English. More importantly, local models became useful only when placed inside a constrained system with structured outputs, validation, and a human in the loop.
The problem space
Speaking can be faster and lower-friction than typing, especially when the documentation language is not the user’s first language.Automatic Speech Recognition (ASR) turns a spoken note into a transcript, and an LLM turns that transcript into a structured draft — but in a clinical workflow a plausible answer is not enough.
The system has to identify the right CDASH (Clinical Data Acquisition Standards Harmonization) fields, normalize values, respect closed taxonomies, detect missing information, and avoid inventing data. Every output therefore remains a draft for the clinician to review, correct, and complete.
Local deployment can reduce unnecessary data movement and give an organization more control over the stack. It does not create compliance on its own: documentation, access controls, auditability, retention rules, validation, and human oversight still have to be designed around the model.
Technology choices
What we evaluated, what we chose, and why.
We targeted selected CDISC CDASH domains and validated every envelope with Pydantic before sending it to a mock EDC (Electronic Data Collection) endpoint. LangGraph orchestrated extraction, validation, and a bounded gap loop; self-hosted Langfuse captured traces and evaluations.
We evaluated nine Automatic Speech Recognition (ASR) candidates and nine LLM candidates. The selected local stack paired Whisper large-v3 with Qwen3.6-35B-A3B; the reference stack paired GPT-4o Transcribe with Claude Sonnet 4.6. The goal was to find the tasks and languages where local trade-offs are acceptable—not to claim that local AI wins in general.
We built a React Native for macOS app with AVFoundation capture and native Liquid Glass controls. It records a note, submits it to the pipeline, and returns structured fields for human review instead of presenting the model output as final.
Public corpora were useful for exploration but created contamination and task-fit concerns. We therefore built a bilingual corpus from structured cardiology scenarios, using both human-recorded and synthetic speech while keeping the final model comparison focused on human recordings.
The POC in action
A user dictates a clinical note. The pipeline transcribes it, classifies whether the note contains medical information (i.e., whether it's on-topic), extracts a CRF envelope, removes empty records, validates the schema, and checks whether any required information is missing. If a gap is found, the system formulates a follow-up question and runs a bounded retry. If the envelope is complete, it is posted to the mock EDC backend.
The benchmark was divided into 3 layers: transcription, text-to-structure extraction, and the complete voice-to-CRF flow. This made it possible to see whether an error came from hearing the note, structuring it, or enforcing the final schema. The headline comparison below uses the full pipeline on human recordings.
Results and takeaways
End-to-end atom-level F1 metric on human recordings
In Polish, the two stacks produced almost identical point estimates. For this narrow, structured task, a locally hosted stack delivered practically competitive quality while keeping inference under the organization’s control.
The frontier stack scored 0.827 versus 0.719 for the local stack. Local deployment is therefore a context-specific product decision, not a blanket replacement for frontier models.
Schema-constrained output, validation gates, and targeted retries turned unstable model behavior into a usable workflow. The benchmark showed that changing the serving and output constraints could matter more than simply selecting a larger model.
Synthetic recordings overstated end-to-end performance on average by 6.8 percentage points. They helped scale testing, but they were not a reliable substitute for human speech with natural hesitations, accents, and recording conditions.
The POC was built to answer a feasibility question, not to support clinical use. It covered a narrow cardiology scope, used purpose-built scenarios and a mock EDC, and did not yet optimize latency, inference parameters, audio preprocessing, or production operations.
„Frontier models don't have to be the default choice. A local workflow costs more to build upfront, but from then on your monthly cost is predictable and you're no longer exposed to pricing or model changes from external providers. That independence — owning the whole setup — is what turns AI from a moving cost into a stable part of the product."
Next step
The next step is to turn the prototype into a production candidate: improve language-specific ASR and LLM tuning, add stronger audio preprocessing, reduce latency, validate with clinician-led recordings, measure correction effort and time saved, and replace the mock endpoint with a production-grade EDC integration.
Production work also needs access controls, audit trails, retention policies, monitoring, deployment controls, model-version governance, and documented human review. Only then should the workflow be assessed for clinical use. The benchmark could then be repeated in another non-native documentation language, with German as the next case.
