Taming LLMs for Infectious-Disease Lab Reports: From Risk to Reliability
Generating infectious-disease lab test reports is a precision task. Yet out-of-the-box large language models (LLMs) can hallucinate, vary across runs, and fall behind fast-moving guidelines—three failure modes that are unacceptable in clinical settings. Below is a pragmatic blueprint to address each shortcoming with retrieval-augmented generation (RAG), deterministic engineering, and a living data pipeline anchored to authoritative sources.
1) Hallucination: Ground the model with authoritative sources (RAG)
The problem: LLMs can fabricate drug names or cite non-existent regimens. In lab reports, that’s more than a nuisance—it’s a patient-safety hazard.
The fix: Bind the model to trusted references via RAG and require provenance in every recommendation.
Build a source-of-truth corpus:
- Labeling and drug facts: Use regulatory endpoints and bulk downloads to verify drug existence, ingredients, and labeling. Pair with current package-inserts for latest data.
- Global/US treatment guidance: Include WHO’s antibiotic classification and CDC’s clinical guidance as clinical policy anchors. Where relevant, add specialist society practice guidelines.
Enforce retrieval-first generation:
- For each organism + susceptibility profile, retrieve the top K passages from the corpus; only then generate. Require the LLM to quote the specific source lines (ID, section, date).
- Add post-generation checks: confirm that any antibiotic mentioned appears in the validated set and is classified appropriately. Fail closed if not found.
2) Non-determinism: Make recommendations consistent, not creative
The problem: The same inputs can yield different outputs—fine for brainstorming, unsafe for reports.
The fix: Engineer for repeatability across retrieval, decoding, and policy application.
- Deterministic decoding: Prefer greedy or beam search (
do_sample=False) andtemperature=0to remove stochasticity. Pair with seeding and framework-level deterministic modes. - Canonical retrieval: Keep RAG inputs stable by freezing retriever settings (embedding model/version, top-K, filters, ranking). Cache “canonical context bundles” per test archetype.
- Policy via expert database: Externalize recommendations (dose, duration, alternatives, contraindications) into a distilled from guidance. The LLM then renders this policy table for the patient context rather than inventing regimens.