Evaluating Hallucinations and Clinical Safety in LLM‑Generated Summaries of Electronic Health Records
Updated on February 01, 2026 21 minutes read
Frequently Asked Questions
How much clinical expertise do I need before doing this kind of evaluation?
You can build the technical harness without being a clinician, but you should involve clinicians for evidence rules, risk weighting, and severity definitions. The goal is not to “become clinical,” but to encode clinical priorities into defensible evaluation decisions.
Can I do this with small datasets, or do I need thousands of encounters?
You can start with a few hundred encounters if they are deliberately sampled for high‑risk contexts. A small, targeted evaluation set that stresses negation, temporality, and high‑risk medications often reveals more than a large random sample.
Are automatic factuality metrics like QAGS or NLI-based checks enough on their own?
They are useful complementary signals, especially early on, but they are not sufficient for clinical safety by themselves. Clinical language has domain‑specific pitfalls, so you should calibrate these metrics against clinician‑annotated judgments on your own data.
How should I handle privacy and compliance when evaluating summaries?
Treat EHR text as sensitive and minimize storage of raw notes and outputs. HIPAA defines national standards for protecting PHI in the US, and GDPR treats health data as special category data in the EU, so your evaluation architecture should be designed with these constraints from the start.
Does an EHR summarizer count as clinical decision support?
It depends on intended use and how it influences decisions, but you should assume scrutiny increases as the system becomes more action-guiding. The FDA’s CDS guidance discusses how different software functions may be considered, including examples that distinguish Non‑Device CDS from device software functions.