Machine Learning
The Clinical Excellence Research Center studies applications of machine learning to electronic health record data and to administrative claims data for improving clinical care. These efforts use machine learning to provide forecasts, such as patients likely to incur high medical costs in the future, and patients at high risk of mortality as well as fundamental methods that enable the making of such forecasts via models that are stable of time, sites, and populations.
Clinical Excellence Research Center's faculty study the interplay of the AI model’s output, the decision-making protocol based on that output, and the capacity of the stakeholders involved to take the necessary subsequent action determines the clinical usefulness of model-guided care.
Publications and News
- – JAMA Network
Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review
CERC researchers examined how health care applications of large language models (LLMs) are currently evaluated and found that only 5% used real patient care data for LLM evaluation. Administrative tasks such as writing prescriptions and natural language processing and natural language understanding tasks such as summarization were understudied; accuracy was the predominant dimension of evaluation, while fairness, bias, and toxicity assessments were less studied.
- – Journal of Biomed Inform
Language models are an effective representation learning technique for electronic health record data
We demonstrated that using patient representation schemes inspired from techniques in natural language processing can increase the accuracy of clinical prediction models by transferring information learned from the entire patient population to the task of training a specific model, where only a subset of the population is relevant.
- – Nature Medicine
Estimate the hidden deployment cost of predictive models to improve patient care
Although examples of algorithms designed to improve healthcare delivery abound, for many, clinical integration will not be achieved. The deployment cost of machine learning models is an underappreciated barrier to success. Experts propose three criteria that, assessed early, could help estimate the deployment cost.
- – NEJM Catalyst
Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems
Based on research inspired by the framework of Machine Learning Models in a clinical setting, Stanford Health Care has developed a mechanism to identify fair, useful and reliable AI models (FURM) by conducting an ethical review to identify potential value mismatches, simulations to estimate usefulness, financial projections to assess sustainability, as well as analyses to determine IT feasibility, design a deployment strategy, and recommend a prospective monitoring and evaluation plan.
- – Nature Digital Medicine
Machine learning and atherosclerotic cardiovascular disease risk prediction in a multi-ethnic population
Machine learning (ML) algorithms present an opportunity to develop improved and more generalizable risk prediction models. By leveraging large-scale data—from electronic health records (EHRs), for instance—such algorithms can determine combinations of variables that reliably predict an outcome.
- – PubMed
Improving palliative care with deep learning
Automatic screening and notification saves the palliative care team the burden of time consuming chart reviews of all patients, and allows them to take a proactive approach in reaching out to such patients ...