Actionable companion · Motivation & engagement
Learning with agency: A Comparative Analysis of ChatGPT and Medical Faculty Graduates in Medical Specialization Exams: Uncovering the Potential of Artificial Intelligence in Medical Education
AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.
Evidence and context
Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Health-professions learning must be distinguished from K–12 instruction.
Original authors’ abstract, reproduced under the linked source license:
Background This study aims to evaluate the performance of ChatGPT in the medical specialization exam (MSE) that medical graduates take when choosing their postgraduate specialization and to reveal how artificial intelligence-supported education can increase the quality and academic success of medical education. The research aims to explore the potential applications and advantages of artificial intelligence in medical education and examine ways in which this technology can contribute to student learning and exam preparation. Methodology A total of 240 MSE questions were posed to ChatGPT, 120 of which were basic medical sciences questions and 120 were clinical medical sciences questions. A total of 18,481 people participated in the exam. The performance of medical school graduates was compared with ChatGPT-3.5 in terms of answering these questions correctly. The average score for ChatGPT-3.5 was calculated by averaging the minimum and maximum scores. Calculations were done using the R.4.0.2 environment. Results The general average score of graduates was a minimum of 7.51 in basic sciences and a maximum of 81.46, while in clinical sciences, the average was a minimum of 12.51 and a maximum of 80.78. ChatGPT, on the other hand, had an average of at least 60.00 in basic sciences and a maximum of 72.00, with an average of at least 66.25 and a maximum of 77.00 in clinical sciences. The rate of correct answers in basic medical sciences for graduates was 43.03%, while for ChatGPT was 60.00%. In clinical medical sciences, the rate of correct answers for graduates was 53.29%, while for ChatGPT was 64.16%. ChatGPT performed best with a 91.66% correct answer rate in Obstetrics and Gynecology and an 86.36% correct answer rate in Medical Microbiology. The least successful area for ChatGPT was Anatomy, with a 28.00% correct answer rate, a subfield of basic medical sciences. Graduates outperformed ChatGPT in the Anatomy and Physiology subfields. Significant differences were found in all comparisons between ChatGPT and graduates. Conclusions This study shows that artificial intelligence models such as ChatGPT can provide significant advantages to graduates, as they score higher than medical school graduates. In terms of these benefits, recommended applications include interactive support, private lessons, learning material production, personalized learning plans, self-assessment, motivation boosting, and 24/7 access, among a variety of benefits. As a result, artificial intelligence-supported education can play an important role in improving the quality of medical education and increasing student success.
What kind of evidence is this?
Supporting AI learning science · Design requires full-text appraisal
Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.
Study methods · source passage
The medical specialty examination (MSE) in the field of medicine is an academically challenging assessment that medical school graduates can take at their discretion. As the name suggests, the MSE is a test that medical students take to gain specialization in various medical fields. This examination was introduced to address the increasing number of medical school graduates and the limited availability of specialist positions. The first examination in this field was administered in 1987. Candidates who achieve a satisfactory score in the examination are eligible to serve in training and research hospitals, medical faculties, or the Institute of Forensic Medicine, all under the jurisdiction of the Ministry of Health. Following this, they obtain the title of Specialist Doctor in the field they choose. The sole requirement to select a medical specialty and receive specialist training is to pass the MSE. The examination encompasses a wide range of subjects, both in clinical and basic sciences.
Limitations and uncertainty · source passage
This study was conducted using the ChatGPT-3.5 version. Results may vary when new versions or different language models are used. Therefore, it is important to consider that the findings obtained are limited to this specific version and results may vary with advanced models. The study used the number of correct answers as the main criterion in evaluating success. However, other important factors such as students’ clinical skills, practical applications, and knowledge of ethical issues should also be evaluated in medical specialty exams. There has been no evaluation of how ChatGPT performs in such areas of skills and knowledge. The study did not consider the impact of external factors (e.g., exam stress and student motivation) on the results. In real exam conditions, such factors can significantly affect students’ performance. The study evaluated success among ChatGPT and graduates based on only a single exam. A more comprehensive study can be conducted to examine how longer-term learning and performance are affected. The findings are based on a study on a specific exam and a specific artificial intelligence model. Therefore, it may be misleading to assume that the results are directly applicable to other exams or different artificial intelligence models. Additionally, more comprehensive research is needed to evaluate how artificial intelligence models will perform in real medical education settings and across a variety of learning conditions. These limitations should be taken into account when interpreting the results and implications of the study. To develop a more comprehensive understanding of the potential use of artificial intelligence technologies in education, such studies need to be considered in a broader context.
Design the system before the pilot
Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.
- Measure engagement separately from achievement. Check whether novelty-driven participation persists and whether it translates to independent performance.
- Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.
A 90-day district pilot
- Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
- Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
- Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
- Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.
The board decision brief
Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.
Attribution and reuse
Gülcan Gencer; Kerem Gencer. A Comparative Analysis of ChatGPT and Medical Faculty Graduates in Medical Specialization Exams: Uncovering the Potential of Artificial Intelligence in Medical Education. Cureus (2024-08-09). https://doi.org/10.7759/cureus.66517
Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.