Actionable companion · Assessment and evidence of learning

From evidence to district action: When Assessment Theory Meets Generative AI: Reimagining SBA Design in Medical Education

AI-assisted editorial implementation guide. The district actions are proposed applications, not findings or recommendations attributed to the original authors. No additional peer review is asserted.

Read original paper Search paper agent CC BY 4.0

Evidence and context

Health-professions education context: test transfer to K–12 rather than assuming it.

Original authors’ abstract, reproduced under the linked source license:

Abstract Current evaluations of generative artificial intelligence (GenAI) in item-writing within medical education often concentrate on isolated dimensions such as validity or reliability, overlooking the broader theoretical foundations that underpin a trustworthy assessment design. This narrow emphasis risks oversimplifying GenAI’s role and obscuring how its adoption reshapes the relationship between quality, efficiency, and educational value. To address this complexity, this paper presents the Co-Created SBA Design (CCSD) framework, which reconceptualises assessment theory for the GenAI era through the lens of Van der Vleuten’s Utility Index. The framework offers a coherent structure for integrating GenAI into Single Best Answer development, maintaining equilibrium across the Utility Index dimensions while redefining collaboration among educators, higher education institutions, and GenAI, a technological partner that enriches the item-writing process. Within this triadic model, each contributor plays a distinct yet complementary role in sustaining assessment quality. Collectively, their interaction ensures that validity, reliability, educational impact, acceptability, and cost-efficiency remain balanced, supporting both educational integrity and sustainable innovation in medical education.

A 90-day district pilot

Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
Days 16–30 · Design. Redesign one low-stakes assessment: require a first attempt, an AI-assisted revision, a source-check log and a short oral explanation. Use the same rubric for the comparison task. Name an accountable owner and set the baseline, success threshold and budget before implementation.
Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
Days 61–90 · Decide. Blind-score reasoning, factual accuracy and unaided transfer on a fresh task. Record suspected misconduct separately from demonstrated evidence. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Nora Al-Shawee; Gerry McElvaney; Judith Strawbridge; Muirne Spooner. When Assessment Theory Meets Generative AI: Reimagining SBA Design in Medical Education. Perspectives on Medical Education (2026-03-12). https://doi.org/10.5334/pme.2033

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.