Actionable companion · Motivation & engagement

From AI evidence to responsible practice: ChatGPT as a Tool for Medical Education and Clinical Decision-Making on the Wards: Case Study

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not findings or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paper Search paper agent CC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Age / stage not established. Health-professions education findings do not establish effectiveness in K–12 classrooms.

Original authors’ abstract, reproduced under the linked source license:

Background Large language models (LLMs) are computational artificial intelligence systems with advanced natural language processing capabilities that have recently been popularized among health care students and educators due to their ability to provide real-time access to a vast amount of medical knowledge. The adoption of LLM technology into medical education and training has varied, and little empirical evidence exists to support its use in clinical teaching environments. Objective The aim of the study is to identify and qualitatively evaluate potential use cases and limitations of LLM technology for real-time ward-based educational contexts. Methods A brief, single-site exploratory evaluation of the publicly available ChatGPT-3.5 (OpenAI) was conducted by implementing the tool into the daily attending rounds of a general internal medicine inpatient service at a large urban academic medical center. ChatGPT was integrated into rounds via both structured and organic use, using the web-based “chatbot” style interface to interact with the LLM through conversational free-text and discrete queries. A qualitative approach using phenomenological inquiry was used to identify key insights related to the use of ChatGPT through analysis of ChatGPT conversation logs and associated shorthand notes from the clinical sessions. Results Identified use cases for ChatGPT integration included addressing medical knowledge gaps through discrete medical knowledge inquiries, building differential diagnoses and engaging dual-process thinking, challenging medical axioms, using cognitive aids to support acute care decision-making, and improving complex care management by facilitating conversations with subspecialties. Potential additional uses included engaging in difficult conversations with patients, exploring ethical challenges and general medical ethics teaching, personal continuing medical education resources, developing ward-based teaching tools, supporting and automating clinical documentation, and supporting productivity and task management. LLM biases, misinformation, ethics, and health equity were identified as areas of concern and potential limitations to clinical and training use. A code of conduct on ethical and appropriate use was also developed to guide team usage on the wards. Conclusions Overall, ChatGPT offers a novel tool to enhance ward-based learning through rapid information querying, second-order content exploration, and engaged team discussion regarding generated responses. More research is needed to fully understand contexts for educational use, particularly regarding the risks and limitations of the tool in clinical settings and its impacts on trainee development.

What kind of evidence is this?

Conversational & pedagogical agents · Qualitative / mixed methods

Stage: Age / stage not established. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

We conducted a brief, single-site pilot study of the publicly available ChatGPT-3.5 (OpenAI) by implementing the tool in the attending rounds workflow of an inpatient general internal medicine service in a large urban academic medical center in New York.

Limitations and uncertainty · source passage

While exploratory, this case study adds to the rapidly growing literature exploring the various uses and limitations of generative AI technologies, such as LLMs, in medical education. As previously stated, ChatGPT has been successfully tested in a growing number of medical training contexts, including writing medical notes and academic abstracts and completing the United States Medical Licensing Examination (USMLE); a surfeit of opportunities to explore the impact of generative AI tools in medical education has also been identified at the undergraduate, graduate, and professional levels [5,6,19-22]. Significant work remains, however, to better understand the roles of these technologies in medical education, including both the extent to which these technologies have already been integrated into educational programs (either formally or through casual use) as well as the true appetite for their future use [23,24]. At the same time, there is an ongoing need to better characterize and actively mitigate the risks of these tools’ use in care delivery, particularly among developing trainees. This study identified numerous limitations of the technology, many of which have been described elsewhere. These included difficulties validating information sources and specific references, inconsistent responses to similar prompts, and incomplete access to major databases and up-to-date material [19]. Although we were unable to confirm examples of misinformation or bias—such as the generated missing references—that may have occurred during our pilot study, our experience reflects larger grievances with current publicly available LLM tools, in particular around issues of output “trustworthiness” and response fidelity over time [25]. There is also the potential of ChatGPT and other tools to perpetuate cognitive and sociocultural biases [26,27], impacting both trainee development and overall care delivery; processes to better center equity in LLMs and mitigate bias-related AI harms are needed to address this [1,28,29].

Design the system before the pilot

Define whether the agent asks questions, gives hints, explains or assesses. Test hallucinated feedback, excessive praise, misleading confidence and responses when the learner is confused. A conversational interface alone does not establish autonomous agency.

Measure engagement separately from achievement. Check whether novelty-driven participation persists and whether it translates to independent performance.

A 90-day district pilot

Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
Days 16–30 · Design. Treat the paper as background evidence until its population is verified in the full text. Do not assign an age-specific intervention or extend adult results to children. Name an accountable owner and set the baseline, success threshold and budget before implementation.
Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
Days 61–90 · Decide. Extract the participant ages or education stage, recruitment, task and comparator before selecting a local pilot. If these cannot be established, retain the paper for design discussion rather than age-band efficacy claims. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Anthony Skryd; Katharine Lawrence. ChatGPT as a Tool for Medical Education and Clinical Decision-Making on the Wards: Case Study. JMIR Formative Research (2024-05-08). https://doi.org/10.2196/51346

Companion prepared 2026-11-01. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.