Agentic education · Learning science

Actionable companion · Cognitive load & attention

From AI evidence to responsible practice: Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not findings or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Health-professions education findings do not establish effectiveness in K–12 classrooms.

Original authors’ abstract, reproduced under the linked source license:

Background ChatGPT is a large language model (LLM) based on artificial intelligence (AI) capable of responding in multiple languages and generating nuanced and highly complex responses. While ChatGPT holds promising applications in medical education, its limitations and potential risks cannot be ignored. Methods A scoping review was conducted for English articles discussing ChatGPT in the context of medical education published after 2022. A literature search was performed using PubMed/MEDLINE, Embase, and Web of Science databases, and information was extracted from the relevant studies that were ultimately included. Results ChatGPT exhibits various potential applications in medical education, such as providing personalized learning plans and materials, creating clinical practice simulation scenarios, and assisting in writing articles. However, challenges associated with academic integrity, data accuracy, and potential harm to learning were also highlighted in the literature. The paper emphasizes certain recommendations for using ChatGPT, including the establishment of guidelines. Based on the review, 3 key research areas were proposed: cultivating the ability of medical students to use ChatGPT correctly, integrating ChatGPT into teaching activities and processes, and proposing standards for the use of AI by medical students. Conclusion ChatGPT has the potential to transform medical education, but careful consideration is required for its full integration. To harness the full potential of ChatGPT in medical education, attention should not only be given to the capabilities of AI but also to its impact on students and teachers.

What kind of evidence is this?

Supporting AI learning science · Evidence synthesis

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

This study conducted a scoping review, described in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) guidelines [11].

Limitations and uncertainty · source passage

Numerous scholars expressed concerns about potential threats to academic integrity posed by ChatGPT and its potential misuse [22,28,38]. Many potential advantages of ChatGPT can also be potential avenues for unethical behavior. For example, ChatGPT may be used for cheating in exams to get higher scores [16]. Students might plagiarize content generated by ChatGPT in their papers, affecting their critical thinking abilities and academic integrity [5]. Additionally, ChatGPT may pose potential threats to ethical issues [22,39]. ChatGPT may trigger issues related to data privacy, patient privacy, student and teacher privacy, intellectual property, and so forth [13,22,39], and some scholars even proposed the possibility of bioweapon creation and reinforcement of authoritarian regimes [40]. Currently, there is a lack of specific regulations or guidelines to guide the use of ChatGPT [13] (Fig. 3, Supplement 3).

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Limit simultaneous instructions and unnecessary dialogue. Compare task performance and a suitable mental-effort measure; easier-feeling work is not automatically better learning.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Xiaojun Xu; Yixiao Chen; Jing Miao. Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review. Journal of Educational Evaluation for Health Professions (2024-03-15). https://doi.org/10.3352/jeehp.2024.21.6

Companion prepared 2026-11-01. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.