Adan Klotz, BMSc '26, MSc Candidate
Adam Klotzl
Large language model-assisted patient prescreening for nephrology clinical trials
-Project supervised by Dr. Pavel Roshanov, continuing work
Abstract
Introduction: To evaluate whether locally deployed LLMs accurately identify trial-eligible patients from clinic notes.
Background: Prescreening of patients for clinical trial eligibility is slow, costly, and prone to bias, and much of the information needed for eligibility decisions is stored as free-text in clinical notes. Large language models offer a potential solution.
Methods: We conducted a retrospective diagnostic accuracy study using nephrology clinic notes from 219 patients at a Canadian academic center. We evaluated 14 open-weight large language models to determine patients’ eligibility for 6 trials in kidney disease against a physician-determined gold standard. We compared providing all eligibility criteria in one prompt to providing each criterion in a separate prompt. Primary outcomes included sensitivity and specificity.
Results: Physician-determined eligibility ranged from 2.3% to 71.2% across trials. One-at-a-time prompting produced a more favorable balance between the primary outcomes of sensitivity and specificity than all-at-once prompting. Under one-at-a-time prompting, pooled sensitivity ranged from 0.57 to 0.94 and pooled specificity from 0.94 to 0.98; the highest pooled sensitivity was achieved by Ministral-14B (0.94, 95% CI 0.91-0.97). Under all-at-once prompting, pooled sensitivity ranged from 0.79 to 0.97, but specificity was lower and more variable, ranging from 0.44 to 0.90. Performance was similar for men and women. Screening took 31 seconds per patient with one-at-a-time prompting and 6 seconds with all-at-once prompting.
Conclusion: If provided one eligibility criterion at a time, several locally deployable open-weight large language models can prescreen patients for trials in nephrology with accuracy comparable to that of a physician reading the same free-text clinic notes. Privacy-preserving local deployment is a viable option for hospital and academic settings constrained by data governance and cost.
About Adam
Adam will begin his Master’s in Epidemiology and Biostatistics at Western University in September 2026 under the supervision of Pavel Roshanov. He currently works as a Research Assistant in the Department of Nephrology, where his experience with manual prescreening motivated the development of this research. Adam’s research interests include natural language processing, machine learning, biostatistics, clinical trial methodology, nephrology, and transplantation.
