Successes and Pitfalls in Running a Small Program MMI

Abstract

Purpose: Academic programs are faced with the important task of selecting health professional students who not only possess necessary cognitive abilities to perform their future job, but also have valuable personal/professional characteristics to draw upon in the provision of quality patient care. There is therefore "widespread agreement that it is desirable to broaden the scope of assessment beyond academic achievement" (Eva, Reiter, Trinh, Wasi, Rosenfeld, Norman, 2009, p.768) in candidate selection. The Child Life Studies Program at McMaster University used a 4-station multiple mini-interview (MMI) as part of the admission selection process. This study sought to determine the feasibility, degree of acceptability and reliability of this 4-station MMI design, and if there were any predictors of candidate's performance on the MMI. Methods: A group of 35 applicants in 2014, and 40 applicants in 2015 screened through admission procedures participated in a 4-station MMI. Each station was 15 minutes in length with 5 minutes for scoring each candidate. Anonymous stakeholder surveys were used in 2015 to assess participant and interviewer's perceptions and acceptability of the MMI in applicant selection. Generalizability coefficients were calculated to determine reliability. In addition, candidate's experience in healthcare, professional work experience, experience with children with disabilities, and previous child life course work (or lack thereof) were analyzed through independent t-tests to report any relationship with candidate's performance on the MMI. A one-way ANOVA was also completed to report any relationship between candidate's undergraduate degree type and his/her MMI performance. Results: The 4-station MMI was found to be feasible for a small program with a marked increase found in the number of candidates interviewed in a shorter period of time. It was also found to be highly acceptable among candidates and faculty. However, this studies' findings did not show statistically significant differences in MMI performance based on identified predictors, or undergraduate degree type. The reliability of the 4-station design in 2014 was G=0.718, however, was only G=0.089 in 2015, far lower than expected based on the prior year’s G Coefficient. The potential pitfalls in running a small program MMI are discussed with specific suggestions and modifications provided to enhance reliability of candidate selections across professions.