Improving MMI quality through assessor development and evidence-informed redesign: A systems approach

Heizer T., Vejražka M., Kolín J., Vlachová V.

Background

Multiple Mini-Interviews (MMIs) are widely used in medical school admissions, yet their measurement quality strongly depends on rubric design, assessor judgement and station alignment. At our faculty, repeated adjustments to station content alone did not lead to satisfactory reliability or discrimination. We therefore implemented a broader, system-level intervention targeting assessors, assessment design and candidate experience.

Summary of work

Across one admissions cycle, we implemented a multi-component intervention including:

(1) redesign of rating rubrics from an intuitive to an analytic structure;

(2) enhanced assessor training grounded in assessment literacy and basic psychometric principles, delivered through workshops and a faculty learning community;

(3) blueprinting of stations to explicit constructs;

(4) measures to reduce candidate anxiety through transparent communication and active interventions in the waiting room; and

(5) expansion of the circuit from four to five stations.

MMI performance was evaluated using standard psychometric indicators (station discrimination, internal consistency) and complemented by our newly developed visual diagnostic tool, highlighting rubric contributions and station difficulty.

Summary of results

Following the intervention, MMI stations demonstrated significantly improved psychometric characteristics, accompanied by increased internal consistency. Analytic rubrics showed reduced redundancy and greater construct specificity, while the deliberately retained global rubric consistently demonstrated low discriminative value. The newly developed visual diagnostic tool enabled rapid identification of underperforming items and misaligned stations, supporting timely quality-improvement decisions.

Discussion & conclusion

Improving MMI performance required coordinated changes across assessment design, assessor preparation and testing conditions, rather than isolated technical adjustments. Assessor training and the design of the analytic rubric were central to improved discrimination. Integrating standard psychometric monitoring with a newly developed visual diagnostic tool can support sustainable quality improvement in MMI.

Take‑home messages

MMI quality improves when assessment design and assessor development are addressed together, rather than through repeated station tweaks.
Our newly developed visual diagnostic tool supports easy identification of non-discriminating items in interview-based exams.