🤖 Generated with AI assistance · Does not replace individualized medical advice.

AI Act Transparency Notice

This content was generated by an AI assistant to summarize and structure scientific literature for educational purposes. It does not constitute medical advice, diagnosis, or treatment. Always consult with a qualified healthcare provider for any medical concerns.

LLMs in Stroke Care: ChatGPT-o3 Outperforms DeepSeek in Rehabilitation Planning

Key Facts

Why It Matters / Context

The integration of Large Language Models (LLMs) into clinical workflows is accelerating, yet their specific utility in complex, multi-domain tasks like stroke rehabilitation remains under-characterized. Stroke care requires not only accurate retrieval of guideline-based facts but also the synthesis of individualized patient data into actionable care plans. Previous studies have often focused on diagnostic accuracy or general medical Q&A, leaving a gap in understanding how these models perform in generative clinical tasks that require nuance, safety, and structural completeness.

This study is significant because it decouples two distinct AI capabilities: retrieval-based accuracy (answering specific clinical questions) and generative planning (creating individualized rehabilitation drafts). The finding that the model with the highest raw accuracy in Stage 1 (DeepSeek) did not necessarily produce the best clinical plans in Stage 2b suggests that “knowing the answer” is not synonymous with “applying the answer safely and comprehensively to a specific patient.” This distinction is critical for oncology and neurology teams considering AI implementation, as it implies that model selection must be task-specific rather than based on general benchmark scores.

Details

The study employed a rigorous two-stage design to filter and compare models.

Stage 1: Guideline-Based Q&A Screening

Four LLMs were tested on 100 stroke-related questions. The top two performers advanced to Stage 2.

Source: PMID: 42703353

Stage 2a: Repeated Clinical Q&A

ChatGPT-o3 and DeepSeek answered 20 guideline-derived clinical questions, with each question repeated three times to assess stability. Three senior clinicians rated outputs on correctness, completeness, readability, helpfulness, and safety using 5-point Likert scales.

• Statistical Outcome: No between-model differences remained statistically significant across the five clinician-rated domains after correction for multiple comparisons.
• Readability: ChatGPT-o3 generated responses with a higher recommended reading age and English Flesch-Kincaid Grade Level. The difference in Chinese Reading Difficulty Score did not remain significant after correction.
• Stability: Repeated-generation score stability did not differ significantly between models.

Stage 2b: Individualized Rehabilitation Planning

The two models generated draft rehabilitation plans for 60 de-identified stroke cases. Each case had three independent generations.

Source: PMID: 42703353

The analysis utilized Generalized Estimating Equations (GEE) to account for the clustering of repeated generations within clinical questions and patient cases, ensuring robust statistical inference.

Limitations

Critical Limitation: This study is a single-center, expert-rated benchmark. It does not establish clinical effectiveness or patient outcomes. The evaluation relies on subjective clinician ratings rather than objective patient-centered metrics. Furthermore, the study focuses on stroke care; generalizability to other specialties (including oncology) requires further validation. Prospective, patient-centered validation and clinician-supervised review are mandatory before any implementation in routine care.

Clinical Takeaway

When selecting LLMs for clinical documentation and care planning, prioritize models with demonstrated superior performance in generative, individualized tasks (like rehabilitation or treatment planning) over those with only high scores in factual retrieval, as accuracy in Q&A does not guarantee safe, complete, or helpful clinical drafts.

Full Reference

Wang YL, Chen LL, Sun PP, Ma ZZ, Chen YP, Ge YS. A two-stage evaluation of large language models for guideline-based Q&A and draft rehabilitation planning in stroke care. Digital Health. 2026. PMID: 42703353 [DOI: 10.1177/20552076261484840]