Agentic education · Learning science

Actionable companion · Assessment & reasoning

Learning with agency: STEM exam performance: Open‐ versus closed‐book methods in the large language model era

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Abstract Background The COVID‐19 pandemic accelerated the shift to remote learning, heightening scrutiny of open‐book examinations (OBEs) versus closed‐book examinations (CBEs) within science, technology, engineering, arts and mathematics (STEM) education. This study evaluates the efficacy of OBEs compared to CBEs on student performance and perceptions within STEM subjects, considering the emerging influence of sophisticated large language models (LLMs) such as GPT‐3. Methods Adhering to PRISMA guidelines, this systematic review analysed peer‐reviewed articles published from 2013, focusing on the impact of OBEs and CBEs on university STEM students. Standardised mean differences were assessed using a random effects model, with heterogeneity evaluated by I 2 statistics, Cochrane's Q test and Tau statistics. Results Analysis of eight studies revealed mixed outcomes. Meta‐analysis showed that OBEs generally resulted in better scores than CBEs, despite significant heterogeneity (I 2 = 97%). Observational studies displayed more pronounced effects, with noted concerns over technical difficulties and instances of cheating. Discussion Results suggest that OBEs assess competencies more aligned with current educational paradigms than CBEs. However, the emergence of LLMs poses new challenges to OBE validity by simplifying the generation of comprehensive answers, impacting academic integrity and examination fairness. Conclusions While OBEs are better suited to contemporary educational needs, the influence of LLMs on their effectiveness necessitates further study. Institutions should prudently consider the competencies assessed by OBEs, particularly in light of evolving technological landscapes. Future research should explore the integrity of OBEs in the presence of LLMs to ensure fair and effective student evaluations.

What kind of evidence is this?

Supporting AI learning science · Evidence synthesis

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

Aligned with PRISMA guidelines, our review focused on peer‐reviewed English articles from 2013 onwards examining OBE versus CBE impacts on university STEM students. This timeframe captures educational shifts pre‐ and during COVID‐19. Detailed course specifics are provided in Appendix S1.

Limitations and uncertainty · source passage

Our review stands as the most comprehensive synthesis of existing data comparing OBE and CBEs within STEM disciplines. By encompassing a range of subjects within this field, it enhances the robustness of the findings and provides a broad empirical basis for conclusions. This expansive approach across various STEM subjects allows for a more nuanced understanding of the impacts of examination type on student performance, offering valuable insights for educators and policymakers.

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Rasi Mizori; Muhayman Sadiq; Malik Takreem Ahmad; Anthony Siu; Reubeen Rashid Ahmad; Zijing Yang; Helen Oram; James Galloway. STEM exam performance: Open‐ versus closed‐book methods in the large language model era. The Clinical Teacher (2024-11-04). https://doi.org/10.1111/tct.13839

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.