5 Questions: Prober on new approach to admission interviews

Charles Prober

Instead of the traditional hour-long student interviews with a faculty member, Stanford and several other medical schools have adopted a new interview process known as the Multi-Mini Interview, or MMI. The process, developed in Canada, involves a series of short interviews over a two-hour period designed to measure character and critical-thinking skills rather than scientific knowledge.

Charles Prober, MD, senior associate dean for medical education, describes the new process in an Oct. 4 Medscape article co-authored by Charlene Hamada, assistant dean of student affairs, and Gabriel Garcia, MD, associate dean for medical school admissions. The article discusses why the change was made, and how it was implemented. Prober along with co-authors, implemented the new process at Stanford in 2010 and used it to help choose this year’s crop of incoming medical students. Prober discussed the points he made in his article with Inside Stanford Medicine writer Tracie White.

Q: What spurred your decision to change the traditional interview process?

Prober: Our traditional interview process consisted of a one-hour conversation between the applicant and one of our medical school faculty members and a second interview with one of our current medical students. These interviews did serve the purpose of addressing any questions that we may have had about the applicant’s file and the compatibility of the applicant with at least two members of our community. The long interview formats also allowed the candidates to probe questions that they may have had about Stanford School of Medicine.

We did recognize however that our interviewers may have developed some positive or negative biases about the applicant’s qualifications based upon an in depth review of their prior experiences. Also, as we described in the Medscape article, the “chemistry” between interviewers and applicants may have shaded the overall evaluation. Furthermore, it is difficult to standardize the format and content of these long interviews conducted by a large number of diverse faculty and students.

We became aware that there’s another way of doing the interviews. Experience with the Multi-Mini Interview was growing, especially in Canadian medical schools. And the long-term performance of medical students selected by MMI versus the more standard interview was positive. Our associate dean for MD admissions, Gabe Garcia, MD, and our director of MD admissions, Charlene Hamada, visited a number of medical schools that are using MMI, and were impressed. I joined them for the visit to McMaster Medical School in Ontario, Canada. Acknowledging my potential conflict of interest (I am Canadian), I too was impressed with the richness of information derived from the MMI process and the quality of the medical students that they had yielded since they adopted MMI approximately 10 years earlier.

Q: After the first year of use, can you describe what works well with this new method, and perhaps what the drawbacks are? Will you make any adjustments next year?

Prober: To the credit of our admissions office and innumerable volunteers who agreed to serve as raters, our “premier” year of MMI was a resounding success. It was a highly complex system to organize, and its successful introduction was facilitated by our new medical school building. Specifically, we were able to take advantage of the layout of our simulation floor that allowed us to smoothly move students from one interview room to the next. Also, because all rooms are under video surveillance from a central control suite, we could troubleshoot any technical difficulties in real time. Our only adjustment this year is to expand our case scenarios and our rich pool of raters. We certainly plan to make the change permanent and predict that it becomes the standard across the United States over the next five to 10 years. We already have had a number of medical schools consult with us as they anticipate their introduction of MMI.

Q: Empathy, dedication, ethical grounding — these are the qualities that you mentioned in the Medscape article, noting that the MMI process enables you to screen better for them. Does the vast increase in medical knowledge today make it more challenging for physicians to have the time to develop relationships with patients, further emphasizing the importance of strong interpersonal skills?

Prober: A physician’s interpersonal skills always have been critical to the most effective patient-centered interactions. A great deal of time in our curriculum is focused on optimizing communication with future patients. And as you point out, medical knowledge clearly is a critical characteristic of a successful physician, but this knowledge does not do patients much good if it cannot be communicated effectively. Thus we are simultaneously working on strategies to facilitate our students’ acquisition of medical knowledge, capacity to be effective lifelong learners and humanistic skills. Four years ago, we developed an entirely new program, Educators-4-CARE, specifically to help to support our students in the acquisition of the attributes of CARE: “Compassion, Advocacy, Responsibility and Empathy.” The 15 faculty recruited to constitute this program exemplify these characteristics and serve as role models for our students, beginning early in their medical education.

Q: In the Medscape article, you cite “fairness” as being one of the advantages of the MMI process. How do you make adjustments for the “tough scorers” in the MMI? And how are the interviewers chosen?

Prober: The structure of MMI mitigates any effect of “tough scorers” or, at the other extreme, “rosy-eyed” scorers. We have recruited about 250 raters to date. Raters include: faculty and staff from our School of Medicine, current medical students, residents, nurses, former patients and patient advocates, and faculty and staff from other schools on campus. After a brief training session, these volunteers begin serving as a rater at one of the stations in the MMI circuit. We typically have three circuits on each of our 15 interview days. Some raters participate on one day; some volunteer for multiple days. People interested in participating should contact: Char Hamada at the School of Medicine Office of Medical Student Admissions at (650) 723-4462.

We deeply value the breadth of our raters. They broaden the framing of our recruitment process and the interpersonal qualities felt to be important in physicians of tomorrow. Because raters see at least eight students on each recruitment day, they serve as their own controls. That is, those raters regarded as being “tough” are “tough” for all applicants they see in a given day and their scores are all corrected for their “toughness.” And because the overall score of each applicant is derived from the score of all eight raters, the effect of a single rater is diluted. Finally, because raters are not given the applicants’ files, they are blinded to all prior history and accomplishments and therefore not susceptible to the biases that may arise from a file review.

Q: You’ve met the first class of medical students to be chosen using MMI. Is there any notable difference in the group as a whole compared with previous years?

Prober: We are very pleased with our first class of medical students selected using MMI. I have heard many compliments about the “personality” of the class from near-peer medical students, staff and faculty. Nonetheless, it is too soon to tell much about the class until they fully settle in and enter the clinical phase of their medical education journey. We are optimistic!