Research

2020

AI voice assistants’ responses to questions about cancer screening

Grace Hong, BA; Claire Wang, BS; Steven Lin, MD; Jacob Less, BS; Neslihan Erbasi, BS; Albino Folcarelli, BA  More 

Purpose:

Primary care physicians often engage in preventive medicine and answer patients’ questions about whether they should get screened for various types of cancer. As people interact more and more with voice assistants and use them to obtain health information, it is important for both primary care physicians and patients to be aware of voice assistants’ responses to health-related questions. This study aims to determine whether four widely used voice assistants (Amazon Alexa, Apple Siri, Google Assistant, and Microsoft Cortana) are able to (1) understand questions about cancer screenings and (2) provide accurate information in response.

Methods:

Four researchers each recorded data on one of the four assistants, and a fifth researcher recorded data on all four so that each assistant received two independent reviews. The researchers asked their assistant, “Should I get screened for [type of] cancer?” for twelve different types of cancer and assessed whether assistants were able to (1) understand the query and (2) provide accurate information in response. To evaluate understanding, researchers assessed whether the assistant was able to return a web search or a verbal response, or not. To evaluate accuracy, researchers assessed whether information provided on the first 3 websites displayed through a web search or in the verbal response matched the U.S. Preventive Services Task Force’s (USPSTF) guidelines for cancer screening.

Results:

Apple Siri, Google Assistant, and Microsoft Cortana were able to understand 100% of the questions they were asked, while Amazon Alexa was able to understand 0%. As for response accuracy, Google Assistant, Apple Siri, and Microsoft Cortana all returned web searches where the top 3 websites provided accurate information roughly 70% of the time. Only Google Assistant and Microsoft Cortana were able to return verbal responses, with Google Assistant providing accurate information 64% of the time and Microsoft Cortana providing accurate information 45% of the time. Response accuracy varied depending on the cancer type queried, but further research is needed to better understand this relationship.

Conclusions:

There are clear differences among the four most widely used AI voice assistants and their ability to understand and respond accurately to questions about cancer screening. As the use of voice assistants continues to increase, further research into the accuracy of health information provided will be important for family physicians and patients. Future directions for this research include sampling a larger set of smartphones and voice assistants, and investigating the effects of users’ age, sex, race and ethnicity, and English proficiency on assistants’ responses.

 Less 


Assessment of the clinical approaches being used to manage purulent and nonpurulent cellulitis

Ashmita Chakraborty, BS  More 

Purpose: Cellulitis, a bacterial skin and soft tissue infection (SSTI) commonly seen in primary care settings, can be targeted by a variety of antimicrobial treatment modalities. Management of cellulitis depends on its clinical presentation and if it manifests with purulent elements. Treatment failures are often a result of inadequate coverage from antibiotics that are not effective against methicillin-resistant S. aureus (MRSA). However, it is also important to prevent over-treatment through excess use of broad-spectrum antibiotics. This study aims to identify Stanford primary care providers’ choice of therapies, to determine whether they complied with the guidelines published by the Infectious Diseases Society of America (IDSA), and to evaluate the effectiveness of the therapies in cellulitis management.

Methods: A retrospective chart review of 250 patients seen for cellulitis by Stanford family and internal medicine providers from 2014-2019 was conducted. Data were abstracted from the medical charts of adults (ages 18-90) who were treated for cellulitis in this time. Treatment selections were assessed based on whether patients were diagnosed with purulent or nonpurulent cellulitis, and whether patients with purulence presented with purulent drainage or abscesses. Antibiotic therapies for patients with immunocompromising conditions were evaluated for anti-MRSA activity.

Results: Of the 54 patient charts evaluated thus far, 29 patients received care compliant with IDSA guidelines. Of the patients whose treatment did not meet these guidelines, 24% of patients experienced recurrence which resolved upon IDSA-approved treatment. Seventeen percent of patients with nonpurulent cellulitis were treated with broad-spectrum antibiotics that did not fall within the SSTI management algorithm.

Conclusion: Cellulitis management through antibiotic therapy by Stanford primary care has not consistently followed IDSA recommendations since the publication of SSTI treatment guidelines in 2014. The association between infection recurrence and failure to adhere by IDSA guidelines suggests that providers should be more informed of the recommended clinical approaches to SSTI treatment. Additionally, the overuse of broad-spectrum antibiotics for nonpurulent cellulitis poses risks such as antibiotic resistance, increased cost, and increased rates of Clostridium difficile. Thus, narrow-spectrum antibiotics should be routinely implemented in most cases of nonpurulent cellulitis.

 Less 


Developing standardized onboarding resources for students and preceptors in family medicine ambulatory sites

Eliza Harbert, BS  More 

 

 Less 


Development of an electronic health record training resource for a family medicine clerkship

Claire Godenzi, BS; Jason Leddy, BS  More 

 

 Less 


Differentiation syndrome during ivosidenib treatment with immunohistochemistry showing isocitrate dehydrogenase R132H mutation

Mika M. Tabata; Melissa Chase; Bernice Y. Kwong; Roberto A. Novoa, Sebastian Fernandez-Pol  More 

We report a case of differentiation syndrome in a patient receiving the IDH1 inhibitor ivosidenib, with skin biopsy showing isocitrate dehydrogenase (IDH) R132H‐mutated leukemia cutis. A 72‐year‐old man with IDH1 ‐mutated acute myeloid leukemia (AML), status‐post allogeneic cell transplantation, on ivosidenib for six months, was admitted for culture‐negative neutropenic fever, pink and purpuric plaques and patches on the legs, abdomen and back, edema, hypotension, and shortness of breath. Skin biopsy revealed an infiltrate of atypical, immature, myeloperoxidase‐positive mononuclear cells compatible with leukemia cutis or Sweet syndrome. Although dermal edema and interstitial neutrophilic infiltrate with karyorrhexis characteristic of Sweet syndrome were not seen, the atypical cells lacked expression of CD117 and CD34, which were expressed in the original leukemia. Additional immunohistochemical staining of suspected blasts was strongly positive for IDH1 R132H, suggesting a diagnosis of leukemia cutis.

As the immunophenotype of blasts in skin infiltrates can significantly differ from the immunophenotype seen in blood and bone marrow, this case demonstrates that mutation‐specific antibodies such as anti‐IDH1 R132H may be useful to help distinguish malignant from non‐malignant infiltrates in the skin. Furthermore, differentiation syndrome may demonstrate histologic features of leukemia cutis on skin biopsy.

 Less 


Improving on-time patient arrival using pre-visit reminders

Jon Schulze; Takudzwa Shumba, MD, MPH  More 

Purpose: Patient and provider experiences are significantly affected by visit duration. Tardiness can disrupt workflow for providers and patient flow coordinators, shorten patient visit, affect quality of care, and create a cascade effect with longer wait times for patients further down the schedule. Long-term workflow issues can decrease provider and staff satisfaction, increase potential burnout rates and affect patient satisfaction and outcomes.

Methods: Our study investigated patient definitions of on-time arrival, perceived barriers to early arrival and effectiveness of reminders prior to clinic visits for increasing on time arrivals. This project was conducted as part of the Stanford Quality Improvement Primary Care Performance Enhancement (PC-PEP) Program. A team comprising a physician, medical assistant and medical scribe investigated the barriers to on time arrival and iterated an appointment reminder system to improve on-time arrivals. Informal interviews were held with clinic patients at Stanford Family Medicine as regards their understanding of and barriers to on-time arrival. We categorized arrival time as: On time (group 1), 0-10 minutes late (group 2), 11-20 minutes late (group 3), and 20+/no show (group 4). Arrival time was tracked over 8 months in the preintervention (control) and postintervention groups. Various iterations of appropriate appointment reminders were evaluated during the PC-PEP cycle, with EMR reminders through MyHealth being used ultimately.

Results: Informal patient interviews showed that late arrivals were associated with transit and misunderstanding visit flow. Data collection is still in progress. We expect the intervention to move patients in groups 2 and 3 higher, but have no effect on group 4. As of this submission, on time arrival increased from 23.5% to 35.3%. Group 2 was unchanged. Group 3 shrank from 29.4% to 23.5%. >20 minutes late shrank from 6.9% to 1.5%. No shows were unchanged.

Conclusions: This pilot project shows that the use of EMR patient reminders with explicit parameters of arrival time can improve punctuality in clinic. The improvement to date is likely understated as not all patients use “MyHealth” and access the reminders. Next steps in progress include applying the reminder system to other provider panels with goal of clinic wide intervention. Overall ability for clinics to remain “on schedule” as well as patient and provider satisfaction will be evaluated in future studies."

 Less 


Leveraging procedural skills training to increase career interest in family medicine

Kyle Davis, BS; Tamara Montacute, MD, MPH; Farheen Shaikh, MBBS; Lauren Myers, BA; Valerie Teng, MD; Al Gourrier, BA; Takudzwa Shumba, MD, MPH  More 

Purpose: Due to the projected shortage of primary care providers, many studies have been performed to determine what factors influence medical students’ decisions to pursue Family Medicine (FM). Learning basic outpatient procedures is exciting for medical students and early exposure is a wonderful way to engage their interest in the broad scope of practice of family physicians. We propose that by incorporating a dedicated procedural training workshop into our FM Core Clerkship rotation, we will increase medical students’ awareness of and interest in FM as a career choice.

Methods: Our procedure workshops involve monthly cohorts of 3rd and 4th year Stanford FM Core Clerkship medical students. The 2-hour workshop includes 20 minutes of didactics, followed by practice on models. Our FM residents and attendings teach Intrauterine Device (IUD)/Nexplanon insertion and removal; abscess incision and drainage; shave/punch biopsy; knee, shoulder and trigger finger injections. Data collection began in March 2019 and will continue through June 2020.  Medical students are surveyed by using multiple choice, Likert scale, and open-ended questions at three different time points. We are administering pre-and post-clerkship surveys to assess: (1) student interest in FM as a career choice, (2) knowledge of FM and its scope of practice, and (3) perceived comfort level with procedural skills. In addition, a short evaluation is administered after the workshop to gauge students’ perceptions of teaching quality for each procedure and value of skill learnt.

Results: Data collection and analysis is ongoing. 

Conclusion: We anticipate that our procedural workshop may expand students’ interest in FM as a career choice, knowledge of FM and its scope of practice, and improve comfort level with procedural skills. By holding a workshop where medical students can perform a range of office-based procedures on reusable or easy-to-make models, we provide a novel setting for medical students to engage with FM residents and faculty, while learning more about this exciting field.  We hope that our work inspires other institutions to develop similar procedural curricula to expose medical students to engaging aspects of FM.

 Less 


Reducing single-use plastics in the primary care setting

Christina Briones, BS; Yaki Alejandre, BS; Erika Schillinger, MD; Tamara Montacute, MD, MPH  More 

Purpose: With increasing awareness of its environmental impact, single-use plastic is now challenged wherever it is used, and healthcare facilities should not be exempt from such scrutiny. A significant proportion—25%—of U.S healthcare facility waste is estimated to be plastic¹ but 9% of all plastic material generated in the U.S is recycled². We wanted to (1) assess which single-use plastics were being used in our primary care clinic setting, (2) understand provider/staff attitudes towards sustainable practices, and (3) use this information to take the first step towards eliminating plastics from our waste.

Methods: Five primary care providers were followed for a day in clinic, and all single-use plastic used was recorded (ie. wrappers, gloves, etc). This plastic waste was examined to identify which items could most easily be replaced with sustainable alternatives. In addition, semi-structured interviews were conducted with 10 primary care providers/staff about their willingness to make sustainable changes. From this, we determined that the most obvious starting point was to eliminate single-use plastic utensils in the break room. This was favored, as it had no medical implications and there were well established non-plastic alternatives available. Posters were placed in the break room to inform faculty/staff about plastic impact. Plastic utensils are being slowly replaced with silverware, and all faculty/staff received reusable bamboo cutlery sets for personal use. Discarded plastic utensils in the break room trash were counted before and after the interventions.

Results: An overwhelming majority of the faculty/staff interviewees were willing to adopt more sustainable practices and felt this was an important part of their role in primary care. Analysis of plastic utensils in the break room trash bins after the interventions showed a significant reduction in plastic utensil usage and observation of the break room dish drying rack showed a corresponding increase in silverware usage.

Conclusion: From our preliminary work it is clear that reducing single-use plastic waste is a complex process, but one that our primary care clinic is motivated to tackle. We hope this small first step in raising awareness by eliminating plastic utensils will be the first of many environmentally focused projects in our clinic. We hope to share knowledge and ideas on sustainability with other primary care clinics and inspire continued work in this area.

 Less 


Volunteerism in academic and community-based primary care providers

Jennifer Marte Jimenez, BS; Eliza Harbert, BS; Valerie Teng, MD  More 

 

 Less 

2019

An hour of prevention is worth a pound of cure: Stopping disease progression in prediabetic patients through education

Sophia Kawamoto, BA; Jinhee Kim; Gabriela Vargas Olvera; Tracy Rydel, MD  More 

84 million American adults have prediabetes. 1 If untreated, 37% of those adults are expected to develop diabetes within 4 years.2 Lifestyle changes are the primary treatment for prediabetes, since these have been found to be more effective than pharmaceutical treatment with metformin. 2 Large randomized controlled trials in Finland, US and China have demonstrated that lifestyle changes decrease the incidence of diabetes in prediabetic patients. 3, 4 The aim of our quality improvement project is to assess the efficacy of the Stanford Diabetes Care Program’s prediabetes class and recommend additional programming. 

 Less 


Evaluation of a universal alcohol use screening initiative

Jason Leddy, Paden Angelo, MD, Chwen-Yuen Angie Chen, MD  More 

Although the U.S. Preventive Services Task Force (USPSTF) recommends that clinicians screen adults for alcohol misuse, we found that most providers were not using a formal screen to gauge patient alcohol consumption. Additionally, the Healthcare Effectiveness Data and Information Set (HEDIS) also set a quality benchmark that adults be screened for unhealthy alcohol use with a standardized tool and, if screened positive, receive appropriate care. In an effort to adhere to these standards, we initiated a quality improvement project designed to implement universal alcohol misuse screening at one of Stanford’s primary care clinics. We wanted to understand the barriers to universal screening, and develop a screening tool and clinic workflow that could increase adherence to HEDIS and USPSTF guidelines, enhance coordination of care through EMR, and expand identification of alcohol misuse.

 Less 


Identifying and addressing barriers to HPV vaccination initiation and completion in a primary care setting.

Cathina Nguyen, RN, MPH; Claire Godenzi, BS; Mariposa Garth, RN, BSN, Amelia Sattler, MD.  More 

HPV vaccination is a 2-3 dose series that protects against cervical cancer and other malignancies caused by the human papillomavirus. In the US only 60% of adolescents receive the 1st dose and 43% complete the series. The aim of our investigation was to identify barriers to HPV vaccine series initiation and completion in a primary care clinic.

 Less 


Improving outcomes for asplenic patients

Kathleen Kenny, MD, FACP; Andrea Levchuk, BA; Jon Schulze, BS; Moriah Montes, BS; Anu Tirupasur, BS.  More 

Given the important role of the spleen in humoral immunity, patients with a history of splenectomy require a modified immunization schedule and aggressive intervention at first sign of fever to prevent catastrophic infection. This study seeks to improve adherence to vaccines and expedite informed medical decision-making with conspicuous documentation.

 Less 


Plugged in: Investigating pediatric screen time use in a primary care setting

Tamara Montacute, MD, MPH; Priyanka MBBS, Kevin Lee, BS; Claire Godenzi, BS; Daniel Feygin, BA  More 

In this digital age, children are spending an increasing amount of time in front of a screen which is known to be linked to a number of negative health outcomes. This study seeks to gain a better understanding of screen time use among pediatric patients at Stanford Family Medicine (SFM), with a goal of better equipping clinicians to effectively educate patients about screen time use when providing care.

 Less 


Symptomatic cardiac events in patients with lung tumors near the heart treated with stereotactic ablative radiotherapy (SABR): A single institution experience

R. B. Ko, K. Jani , Q. Sodji, D. J. Merriott, W. Pickthorn, K. Bush, P. G. Maxim, M. Diehn, and B. W. Loo Jr  More 

Stereotactic ablative radiotherapy (SABR) is a highly effective therapy for patients with lung cancer, but cardiac injury after treatment to tumors near the heart is a potential concern. Therefore, we evaluated factors associated with the development of new symptomatic cardiac events in patients with lung tumors near the heart at our institution.

 Less 


Feasibility and design of a cloud-based digital platform in patients with advanced cancer

Olga Generalova, Mohana Roy, Evan Thomas Hall, Brianna Velazquez, Sumit Shah, Touran Fardeen More , Kristen Cunanan, Melanie San Pedro-Salcedo, Heather A. Wakelee, Joel W. Neal, Sukhmani Kaur Padda, Millie Das, Alice C. Fan, Sandy Srinivas, George A. Fisher, Sigurdis Haraldsdottir, Tyler Paul Johnson, Gilbert Chu, Alex McMillan, Kavitha Ramchandran

Patient reported outcomes (PROs) allow for systematic and more continuous capture of the patient experience. PROs have been associated with improved symptom management and quality of life in patients with advanced cancer; however how to do this in a real world setting is not thoroughly described. We sought to provide a description of the implementation and feasibility of a digital PRO application in patients with advanced cancer at one academic center.

 Less 

2018

Accessibility of over-the-counter birth control in the united states: A snapshot of the country

Liz Jahng; Bianca Kennel; Steven Lin, MD  More 

In the vast majority of the United States, women are required to have a prescription from a health care provider in order to obtain birth control pills. Studies have shown that this policy presents a barrier toward preventing unintended pregnancies and abortions. In recent years, some states have passed legislation that allow pharmacists to prescribe birth control, thus allowing women to obtain oral contraceptives “over-the-counter”. Our objective was to create a snapshot of the country by illustrating which states are progressing toward over-the-counter birth control access, and which states are not. 

 Less 


Effectiveness of a smartphone app for guiding pediatric ADHD medication selection

Anu Tirupasur; Steven Lin, MD  More 

Attention deficit hyperactivity disorder (ADHD) is an increasingly prevalent pediatric chronic disease in the United State. There is a myriad of medications available to treat pediatric ADHD; however, many primary care physicians lack access to decision support tools at the point of care to help choose ADHD medications in an evidence-based way. Our objective was to create a smartphone app with a simple, easy-to-use decision tree for choosing ADHD medications, and to measure the effects of the app on the confidence level of family physicians in treating ADHD.

 Less 


Going beyond “an apple a day”: What patients want their primary care physicians to know about nutritional counseling

Julia Hafer; Terry Lou; Tracy Rydel, MD; Steven Lin, MD  More 

Despite the fact that both patients and physicians feel that doctors are critical sources of nutritional advice, many physicians report feeling poorly equipped to provide this counseling. Medical schools have begun exploring ideas for nutrition curriculum improvements from an academic perspective, but to our knowledge, little is known about the patients’ point of view. This pilot study aims to explore the patient perspective, with the ultimate goal of helping primary care physicians more effectively target their nutritional counseling and informing future improvements to nutrition curricula in medical school.

 Less 


Identifying best practices for nurturing effective scribe-physician partnerships

Tuan Nguyen; Eugenia Jernick; Steven Lin, MD  More 

Scribes are the newest members of the health care team, yet relatively little is known about the factors that contribute to effective scribe-physician relationships in the workplace. Since 2015, Stanford University School of Medicine has offered a 1-year post-baccalaureate scribe fellowship for pre-medical students, who work alongside primary care physicians and provide documentation support. Our aim was to identify the factors that contribute to effective scribe-physician relationships, and to disseminate these findings through an orientation video for new physicians who are not familiar with scribes.

 Less 


Identifying opportunities to improve domestic violence screening in a primary care system

Laurel Sharpless; Steven Lin, MD  More 

The U.S. Preventive Services Task Force (USPSTF) recommends that physicians screen women of childbearing age for domestic violence, but research shows that the rates of screening in primary care settings are low. The aim of this quality improvement project was to determine the rates of domestic violence screening in 5 primary care clinics at a large academic medical center. We also sought to determine whether screening initiated by medical assistants or physicians resulted in more documented screens.

 Less 


Identifying psychosocial barriers to behavioral activation in seniors with depression

Kevin Lee; Jennifer Milan; Jamie Yang; Andrew Yoon; Steven Lin, MD  More 

Behavioral activation (BA) has substantial evidence as an effective intervention to treat late life depression. By encouraging adaptive behaviors and planned pleasant activities, BA has been shown to significantly decrease symptoms of depression. However, the primary barriers that prevent depressed older adults from participating in BA remain unclear. This quality improvement (QI) study aimed to identify the most commonly encountered barriers to BA in older adult patients with depression.

 Less 

2017

Creating a tool to measure the effectiveness of sexual orientation and gender identity sensitivity training
Tuan Nguyen, Dr. Benjamin Laniakea, Dr. Steven Lin, and Dr. Catherine Forest   More 

Stanford has rolled out a brief, targeted sexual orientation and gender identity (SOGI) sensitivity training for physicians with the aims of improving the quality of health care for patients in the gender and sexual minority (GSM) community. Before the training was implemented, a pre- and post-assessment tool was developed to measure the effectiveness of the intervention. The tool was formulated to assess provider knowledge in 3 areas: SOGI terminology, proper use of gender pronouns, and the importance of integrating SOGI data fields into the electronic health record (EHR). With our findings, we plan to analyze changes in baseline knowledge, provider use of SOGI data fields, and the overall impact of training on provider comfort in caring for GSM patients.

 Less 


Creating an antidepressant wheel to help family physicians choose antidepressants
Jimmy Yao; Steven Lin, MD   More 

There are currently few point-of-care decision support tools to help family physicians choose antidepressants in a rational, evidence-based manner. We aimed to design a handheld circular wheel as an educational point-of-care decision support tool to help family physicians choose antidepressants based on a symptom cluster approach, originally published in 2014. This antidepressant wheel is a novel point-of-care decision support tool that may help family physicians choose antidepressants based on an evidence-based model.

 Less 


Impact of scribes on medical student education: A pilot study
Xibin Wu; Steven Lin, MD   More 

Academic family medicine practices are starting to incorporate scribes into their clinical workflow. There is a lack of data on the impact of scribes on medical students’ learning. We conducted a mixed methods study to determine the impact of scribes on medical student education in a university-based family medicine practice. This research will be presented at the Society of Teachers in Family Medicine’s national conference in May of 2017.

 Less 


On the fence between family medicine and OB/GYN? How to mentor undecided students
Eugenia Jernick; Steven Lin, MD   More 

Family medicine focuses on providing longitudinal and comprehensive health care; however, it is losing a key role in maternity care. We conducted a mixed methods study of family medicine faculty and residents at our institution to examine their perspectives on maternity care. Participants reported that maternity care was an important factor in their career choice. This research will be presented at a roundtable discussion at the Society of Teachers in Family Medicine's national conference. It will engage participants in a discussion of the importance of maternity care in family medicine, how to mentor medical students on making a choice between OB/GYN and family medicine, and what the future of maternity care in family medicine may look like.

 Less 


Transforming the role of medical scribes to increase clinician and staff capacity
Vincent N. Le;Michelle Tay; Xibin Wu; Steven Lin, MD   More 

To meet the growing demands on primary care and achieve the Triple Aim, many healthcare settings are redistributing responsibilities among non-physician members of their care teams. Increasingly, the responsibility of documenting patient encounters is being delegated to medical scribes, and healthcare systems that have already done so are reporting increased productivity and greater physician satisfaction. This project will be presented as a roundtable discussion at the Society of Teachers in Family Medicine's national conference in May of 2017. We will describe the motivation for, and our experience of, transforming the role of medical scribes to include other administrative elements of patient care, such as rooming patients, taking vitals, reviewing health maintenance reminders, writing after-visit summaries, and prepping procedures.

 Less 

2016

Dynamic duos: Mentored scribe programs as a value-added educational experience for pre-health students
Cat Carragee; Steven Lin, MD; Alexis Kofoed   More 

 

 Less