Agentic education · Learning science

Actionable companion · Feedback & scaffolding

Learning with agency: Comparison of ChatGPT-4o and expert evaluation in endodontic education: a cross-sectional pilot study

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Abstract Background Artificial intelligence (AI) has the potential to enhance objectivity and scalability in educational assessment, yet its role in evaluating technical dental skills remains unclear. This study aimed to compare ChatGPT-4o based assessments with expert evaluations in undergraduate endodontic training and to explore student perceptions of AI-assisted feedback. Methods This cross-sectional pilot study was conducted during the 2024–2025 academic year with 32 dental students from a faculty of dentistry, who completed root canal treatments. Postoperative radiographs were evaluated by 10 years experienced endodontist and ChatGPT-4o was used to evaluate performance based on five standardized criteria: canal centering, homogeneity, procedural errors, apical shaping, and overall taper, each rated on a 5-point Likert scale. Inter-rater reliability was assessed via intraclass correlation coefficients (ICC), and Pearson correlation tested linear alignment. Students rated both feedback sources via Likert-scale questionnaires and open-ended comments; paired-sample t-tests compared the mean scores. Results Agreement between AI and expert evaluation was limited, with ICC ranging from 0.36 to 0.45, indicating poor to moderate reliability. Pearson r values were < 0.3 and not statistically significant, demonstrating weak linear correlation. While students rated AI-generated feedback as moderately useful, expert feedback scored higher across educational value (mean 4.29 vs. 3.90), clinical reasoning support (4.19 vs. 4.06), and reliability (4.00 vs. 3.91); differences were not statistically significant. Notably, 53.1% of students preferred a combination of AI and expert feedback for optimal learning. Conclusions AI-generated feedback was moderately useful to students, but expert feedback consistently scored higher. For complex psychomotor skills and radiographic interpretation, AI should serve as an auxiliary tool rather than an independent assessor. Further validation of advanced multimodal AI systems and development of hybrid frameworks combining algorithmic objectivity with expert judgment are recommended. Supplementary Information The online version contains supplementary material available at 10.1186/s12909-025-08358-2. Keywords: Artificial intelligence, Chatgpt-4, Endodontic education, Dental education

What kind of evidence is this?

Supporting AI learning science · Survey / observational study

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

This cross-sectional pilot study was conducted at the Department of Endodontics, Faculty of Dentistry, Istanbul Aydın University, during the 2024–2025 academic years. The study was embedded in a clinical training program for fourth- and fifth-year dental students with the aim of evaluating the feasibility of integrating AI-assisted assessment into endodontic education.

Limitations and uncertainty · source passage

In addition to the categorized responses, several open-ended comments offered further insight into students’ perceptions of the feedback systems. These statements reflect the perceived strengths and limitations of both modalities:

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Give feedback on the learner’s reasoning, then fade hints. Record whether the learner can complete the next task without that scaffold.
  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Suha Alpay; Yasemen Darafarin; Burcu Dagdelen; Sana Mahroos Mkhailef Al-Shammari; Isil Kaya Buyukbayram. Comparison of ChatGPT-4o and expert evaluation in endodontic education: a cross-sectional pilot study. BMC Medical Education (2025-12-01). https://doi.org/10.1186/s12909-025-08358-2

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.