🤖 Generated with AI assistance · Does not replace individualized medical advice.

AI-Driven Thyroid Reporting: LLMs Cut Transcription Errors and Boost TI-RADS Accuracy

AI Transparency Notice: This post was generated by an AI assistant to summarize and contextualize published medical literature. It is intended for educational purposes only and does not constitute medical advice, diagnosis, or treatment. Always consult a qualified healthcare provider for clinical decisions.

Lead

A new feasibility study demonstrates that a two-stage workflow using Large Language Models (LLMs) significantly outperforms commercial speech recognition in thyroid ultrasound reporting. By combining LLM-based speech-to-text with detailed prompting strategies, the system reduced transcription errors and improved the accuracy of TI-RADS scoring, though deterministic post-processing remains critical for clinical safety.

Key Facts

Metric	LLM-Based Workflow	Commercial/Minimal Baseline	Statistical Significance
Word Error Rate (WER)	1.0% (IQR 0–2.0)	3.0% (IQR 2.0–3.0)	P < 0.001 PMID: 42680653
Correction Time	6.38s (IQR 4.58–10.45)	11.39s (IQR 9.56–14.06)	P < 0.001 PMID: 42680653
ACR TI-RADS Point Accuracy	78.5% (Detailed Prompt)	50.3% (Minimal Prompt)	P < 0.001 PMID: 42680653
K-TIRADS Category Accuracy	79.9% (Detailed Prompt)	58.4% (Minimal Prompt)	P < 0.001 PMID: 42680653

Why It Matters / Context

Thyroid ultrasound is a high-volume diagnostic procedure where reporting efficiency and consistency are paramount. The Thyroid Imaging Reporting and Data System (TI-RADS) provides a standardized framework for risk stratification, but manual dictation and scoring are time-consuming and subject to inter-observer variability.

This study addresses two distinct bottlenecks in the radiology workflow:

1. Transcription Efficiency: Traditional commercial Automatic Speech Recognition (ASR) often struggles with medical terminology and bilingual contexts (English/Korean in this study), leading to high correction times for radiologists.

2. Scoring Consistency: While LLMs have shown promise in medical text generation, their ability to accurately apply complex scoring rubrics like ACR TI-RADS or K-TIRADS has been inconsistent without specific architectural guidance.

The “two-stage” approach tested here is significant because it decouples the transcription task from the interpretation task. By using an LLM for speech recognition (Stage 1) and a separate LLM instance (GPT-4o) for scoring (Stage 2), the workflow leverages the specific strengths of large models in both natural language processing and structured data extraction. The finding that “detailed prompting” and Retrieval-Augmented Generation (RAG) significantly boost accuracy suggests that generic LLM deployment is insufficient; context-specific engineering is required for clinical reliability.

Details

The study evaluated 149 cases from the Digital Database of Thyroid Images. The workflow was divided into two distinct stages, each with specific performance metrics.

Stage 1: Bilingual Speech Recognition (ASR)

Radiologists dictated reports, which were processed by both a commercial ASR model and an LLM-based ASR model. Two radiologists manually corrected the transcripts to establish a gold standard.

• Word Error Rate (WER): The LLM-based model achieved a median WER of 1.0% (IQR 0–2.0), significantly lower than the commercial model’s 3.0% (IQR 2.0–3.0) PMID: 42680653.
• Correction Time: Radiologists spent a median of 6.38 seconds (IQR 4.58–10.45) correcting LLM transcripts, compared to 11.39 seconds (IQR 9.56–14.06) for commercial transcripts PMID: 42680653. This represents a nearly 44% reduction in manual editing time.
• User Preference: Both radiologists strongly preferred the LLM-based output, with preference rates of 70.5% and 73.7%, respectively (P < 0.001) PMID: 42680653.

Stage 2: LLM-Assisted TI-RADS Conclusion Generation

Using the gold standard reports, GPT-4o was tasked with generating TI-RADS scores. Three prompting strategies were compared:

1. Minimal: Basic instruction to score the report.

2. Detailed: Comprehensive guidelines embedded in the prompt.

3. Detailed + RAG: Detailed guidelines integrated with Retrieval-Augmented Generation for dynamic guideline access.

The following table summarizes the accuracy improvements across different TI-RADS systems:

Scoring System	Metric	Minimal Prompting	Detailed Prompting	Improvement
ACR TI-RADS	Point Accuracy	50.3%	78.5%	+28.2% PMID: 42680653
K-TIRADS	Category Accuracy	58.4%	79.9%	+21.5% PMID: 42680653

Note: Statistical significance for all improvements was P < 0.001, assessed via Wilcoxon matched-pairs signed-rank tests and McNemar's test PMID: 42680653.

The study highlights that while the LLM could identify features, it frequently failed to apply the correct point values or category thresholds without explicit, detailed instructions. The inclusion of RAG (Retrieval-Augmented Generation) allowed the model to reference specific guideline sections, further stabilizing the output.

Limitations

⚠️ Critical Limitations for Clinical Translation

1. Feasibility Study Design: This was a retrospective feasibility study, not a randomized controlled trial. The sample size (n=149) is modest, and the dataset was limited to cases with “adequate image quality,” potentially excluding complex or ambiguous cases where AI performance might degrade.

2. Deterministic Post-Processing Required: The authors explicitly state that deterministic post-processing remained essential. This implies the LLM output was not “plug-and-play” safe; it required rule-based checks to prevent hallucinations or scoring errors. This adds a layer of engineering complexity that may not be trivial to implement in existing PACS/RIS systems.

3. Lack of Prospective Validation: The study did not test the workflow in a live clinical environment. Real-world factors such as network latency, user interface friction, and radiologist workflow integration were not assessed.

4. Model Specificity: The results are specific to GPT-4o and the particular LLM-based ASR model used. Generalizability to other LLM architectures or older versions is not guaranteed.

5. Bilingual Context: The ASR evaluation was conducted in a bilingual (English/Korean) context. Performance in monolingual or other multilingual settings may vary.

Clinical Takeaway

While LLMs show promise in streamlining thyroid ultrasound reporting, they are not yet autonomous; clinicians should view this technology as a “co-pilot” that requires rigorous deterministic post-processing and prospective validation before integration into routine diagnostic workflows.

Full Reference

Han T, Kim H, Shin J, Hahn SY. Streamlined TI-RADS Reporting From Bilingual Speech Recognition to LLM-Assisted Conclusion Generation Using a Two-Stage Workflow. Academic Radiology. 2026. PMID: 42680653 [DOI: 10.1016/j.acra.2026.08.009]