Predictive Drift After Deployment: A Two-Year Audit of 31 Clinical Risk Models Across a Multi-Site Health System
Dr. Aisha Diallo (Riverstone Children's Hospital), Dr. Kwame Asante (Meridian Health System), Yuki Tanaka, MPH (Brookline Medical Center), Dr. Olufemi Bankole (Trent Valley University)
Clinical risk models are often validated once and then trusted indefinitely, but their accuracy can quietly erode as patient populations and care patterns change. We tracked 31 deployed models across eight hospitals for two years and found that nearly half lost meaningful accuracy within 18 months, often without anyone noticing. We describe a practical monitoring approach that flagged most of these failures early and required no specialized tooling to run.
In this issue
Original Research
Fairness Is Not a Setting: How Threshold Choices Reshape Equity in a Sepsis Alert Algorithm
Dr. Camille Dubois, Rosa Calderón, MPH (Valle Verde Community Health Network), Dr. Gregory Mbeki (Trent Valley University)
The same sepsis model can be fair or unfair depending on a single deployment decision about where to set its alert threshold. We show how that one choice shifted false-alarm burdens between patient groups and offer a framework for making the tradeoff deliberately rather than by accident.
Large Language Models for Drafting Clinical Notes: A Randomized Comparison of Accuracy and Time Saved
Dr. Raj Patel, Dr. Wei Chen (Lumen Health AI), Dr. Hannah Lindqvist (Brookline Medical Center)
We randomized 96 clinicians to draft progress notes with or without an LLM assistant and measured both documentation time and factual errors. The assistant saved time on average but introduced subtle factual errors in roughly one note in twelve, suggesting that review workflows matter more than the model itself.
The State of Clinical NLP: A Review of Methods, Benchmarks, and Open Problems
Dr. Benjamin Tran (Cascadia University), Dr. Mei-Ling Wong (Pinewood Regional Medical Group)
We survey a decade of clinical natural language processing, from rule-based extraction to transformer models, and trace which problems have genuinely been solved and which have merely been renamed. We argue the field's biggest remaining barrier is not modeling but the scarcity of shared, well-annotated clinical text.
Who Owns the Pipeline? Toward a Practical Data Governance Model for Learning Health Systems
Daniel Hofstetter, MS, Dr. Erik Johansson (Halcyon Health Plans)
Data governance is usually written as policy and then ignored in practice because no one owns the day-to-day decisions. We propose a lighter-weight governance model that assigns clear ownership to the people who actually build and maintain data pipelines.
MLOps in Health Systems: Lessons From Operationalizing Twelve Models in Production
Dr. Kwame Asante (Meridian Health System), Tobias Andersson, MS (Pinewood Regional Medical Group)
We report what it actually took to move twelve machine learning models from notebook to production in a hospital setting, including the failures that taught us the most. The recurring theme was that the modeling was the easy part and the monitoring, retraining, and handoffs were where projects lived or died.
Closing the Loop: A Brief Report on Clinician Trust After Transparent Model Explanations
Dr. Sofia Rinaldi (Riverstone Children's Hospital), Dr. Fatima Al-Rashid (Sable Digital Health)
We added plain-language explanations to a deterioration-risk alert and measured whether clinicians acted on it differently. Trust and appropriate use both rose modestly, but only when the explanation named the specific factors driving each individual alert.