🤖 Generated with AI assistance · Does not replace individualized medical advice.
Full-Text Inputs Boost LLM Accuracy in Oncology Trial Eligibility Classification
AI Transparency Notice: This post was generated by an AI assistant to summarize and contextualize published research. It is for informational purposes only and does not constitute medical advice, diagnosis, or treatment recommendations. Always consult a qualified healthcare provider for clinical decisions.
Lead
A study in JAMIA Open demonstrates that providing large language models (LLMs) with full-text articles significantly improves the accuracy of classifying oncology trial eligibility compared to using abstracts alone. By reducing information loss, full-text inputs allow models to better identify nuanced inclusion criteria often omitted in condensed summaries.
Key Facts
| Metric | Abstract Input | Full-Text Input |
|---|---|---|
| Accuracy | 86% (95% CI, 81%–91%) PMID: 42761820 | 92% (95% CI, 88%–95%) PMID: 42761820 |
| F1 Score | 0.90 PMID: 42761820 | 0.94 PMID: 42761820 |
| Dataset Size | 200 RCTs PMID: 42761820 | 200 RCTs PMID: 42761820 |
| Model Used | GPT-5 PMID: 42761820 | GPT-5 PMID: 42761820 |
Why It Matters / Context
Clinical trial matching is a persistent bottleneck in oncology care. Patients with eligible conditions are often not enrolled in appropriate trials due to the sheer volume of studies and the complexity of eligibility criteria. While LLMs have shown promise in automating this process, previous studies often relied on abstracts because they are shorter and cheaper to process. However, abstracts are inherently lossy; they summarize the study’s purpose and primary outcomes but frequently omit detailed inclusion/exclusion criteria, such as specific performance status requirements, prior therapy limitations, or organ function thresholds.
This study addresses the “signal vs. noise” hypothesis. The concern was that while full texts contain more signal (detailed criteria), they also introduce more noise (irrelevant background, methods, and discussion), which might confuse the model’s reasoning. The findings suggest that for current-generation LLMs like GPT-5, the benefit of the additional signal outweighs the cost of the noise. This is particularly relevant for oncology, where eligibility is often defined by granular clinical details (e.g., “no prior platinum-based therapy within 6 months”) that are rarely fully captured in a 250-word abstract.
Details
The study evaluated GPT-5’s ability to classify 200 randomized controlled trials (RCTs) in oncology. The specific task was binary classification: determining whether patients with localized disease and/or metastatic disease were eligible for the trial. Each trial was processed twice: once using only the abstract and once using the full text. The model’s outputs were compared against manually annotated ground-truth labels.
Performance was assessed using standard metrics: accuracy, precision, recall, and F1 score. Statistical significance between the two input methods was determined using the McNemar test, which is appropriate for paired categorical data.
| Input Type | Accuracy | 95% CI | F1 Score | Precision | Recall |
|---|---|---|---|---|---|
| Abstract | 86% | 81%–91% | 0.90 | Not specified in abstract | Not specified in abstract |
| Full Text | 92% | 88%–95% | 0.94 | Not specified in abstract | Not specified in abstract |
Note: While the abstract provides accuracy and F1 scores, it does not explicitly list precision and recall values for each arm. The F1 score, being the harmonic mean of precision and recall, indicates a balanced improvement in both metrics for the full-text condition.
The improvement from 86% to 92% accuracy may seem modest in isolation, but in the context of high-stakes clinical matching, a 6% absolute increase in accuracy translates to a significant reduction in false negatives (missing eligible trials) and false positives (suggesting ineligible trials). The McNemar test confirmed that this improvement was statistically significant, suggesting that the difference is not due to random variation in the dataset.
Limitations
Critical Limitation: The study was conducted on a relatively small dataset of 200 trials. While this is sufficient for a proof-of-concept, it may not capture the full diversity of oncology trial designs, including rare cancers or highly specialized immunotherapy protocols. Furthermore, the study focused on a specific binary classification task (localized vs. metastatic eligibility). Real-world trial matching requires multi-dimensional classification (e.g., specific biomarker status, line of therapy, comorbidities), which may present different challenges for LLMs. Additionally, the study used GPT-5, a proprietary model; results may vary with other LLM architectures or versions. The “noise” in full texts could become more problematic as model context windows expand or as models are fine-tuned for specific medical tasks, potentially altering the signal-to-noise trade-off.
Clinical Takeaway
Healthcare systems and trial-matching platforms should prioritize integrating full-text article ingestion into their LLM pipelines, as the marginal computational cost is outweighed by the significant gain in eligibility classification accuracy.
Full Reference
Weyrich J, Dennstädt F, Förster R, Schröder C, Aebersold DM, Zwahlen DR. More signal versus more noise: comparing full text and abstract as inputs for large language model-based classification of oncology trial eligibility criteria. JAMIA Open. 2026. PMID: 42761820 [DOI: 10.1093/jamiaopen/ooag179]