Actionable companion · Learning design & outcomes
From AI evidence to responsible practice: Familiarity with ChatGPT Features Modifies Expectations and Learning Outcomes of Dental Students
AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not findings or recommendations attributed to the source authors. No independent peer review of this guide is asserted.
Evidence and context
Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Health-professions education findings do not establish effectiveness in K–12 classrooms.
Original authors’ abstract, reproduced under the linked source license:
Abstract Objectives The number of approvals for AI-based systems is increasing rapidly, although AI clinical trial designs lack consideration of the impact of human–AI interaction. Aim of this work was to investigate how reading of an AI system (ChatGPT) features/descriptions could influence the willingness and expectations for use of this technology as well as dental students’ learning performance. Methods Dental students (N = 104) were asked to learn about side effects of drugs used in dental practice via reading recommended literature or ChatGPT. Expectations towards ChatGPT were measured by survey, before and after reading of a system features description, whilst learning outcomes were evaluated via pharmacology quiz. Results Students who used ChatGPT (YG group) showed better results on the pharmacology quiz than students who neither read the description nor employed ChatGPT for learning (NN condition). Moreover, students who read the description of ChatGPT features yet did not use it (NG) showed better results on the pharmacology quiz compared with the NN condition, although none of them employed ChatGPT for learning. The NG students compared to the YG students had less trust in AI system assistance in learning, and after the AI system description reading, their expectations changed significantly, showing an association with quiz scores. Conclusions A majority of students in our cohort was reluctant to use ChatGPT. Furthermore, familarity (reading) with ChatGPT features appear to alter the expectations and enhance learning performance of students.suggesting an AI description–related cognitive bias. Hence the content description of ChatGPTshould be reviewed and verified prior to AI system use for educational purposes. Key words: Artificial intelligence, ChatGPT, Education dental, Ethics, Pharmacology
What kind of evidence is this?
Supporting AI learning science · Survey / observational study
Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.
Study methods · source passage
Total of 104 (22 male and 82 female) third-year dental students attending a pharmacology course were included in the study. Ethical approval was obtained from the Ethics Committee of the Faculty of Dentistry, number 36/50, 2023. Students were asked to learn and then write an essay on the side effects of drugs used in dental practice, by using usual literature (a pharmacology book)5 or an AI-based language processing tool that is freely available online—ChatGPT-3.5—after they voluntary decided which condition they would be in. Groups were as follows: YG, “yes” group (will use ChatGPT); NG, “no” group (will not use ChatGPT but will read its description); or NN, “none” group (will neither use ChatGPT nor read the description). Before reading the ChatGPT features description, only 13 participants were willing to be in the YG group, whilst 47 decided to be in the NG group and 44 in the NN condition. Interest in using ChatGPT as well as attitudes and expectations regarding this AI technology were measured in the YG and NG groups by survey, before and after reading a ChatGPT description (Figure 1). On the same day, all students completed an 11-item questionnaire with closed-ended questions in the Serbian language that was prepared based on a review of the literature assessing expectations and acceptance of AI technology,2,6 under the Unified Theory of Acceptance and Use of Technology (UTAUT) framework, a validated model used to analyse technology acceptance in a Serbian sample7 adapted to the objectives of the present study. Thus, the survey aimed to score, amongst others things, performance expectancy (“using ChatGPT would help me to write an essay better or faster, and it is useful”), effort expectancy (“I find ChatGPT easy to use”), or attitude (“Using ChatGPT is wise”). A 5-point Likert scale was used to measure attitudes and expectations for use of this AI technology, and mean sums of scores were obtained. The students rated statements from 1 (strongly disagree) to 5 (strongly agree); the higher the score, the higher degree of agreement.
Limitations and uncertainty · source passage
Present results suggest that a small number of students were interested in employing ChatGPT in learning, mainly because they doubted its performance abilities, safety, and reliability, and this finding is in the line with a previous investigation of AI acceptance by dental students.8 Despite a small number of participants, the YG group showed significantly better quiz results than the NN condition, whilst the difference in learning performance of the YG group compared with the NG group was not obtained, nor was the association of expectations with learning performance of YG participants, which has previously been suggested to represent placebo.2 Moreover, beside a small number of participants, regression analysis of YG group data revealed high multicollinearity. Thus, a high correlation between variables (expectations) represents difficulty in interpreting the effect of each expectation on task performance separately, because expectations' effects were confounded by their correlation. Due to these limitations, the exact contribution of ChatGPT to learning task accomplishment could not be established. Thus, focus in this research was on investigating how reading ChatGPT features descriptions affected students’ quiz performance, since post hoc analysis revealed that differences observed between NG and NN conditions in quiz scores could be considered reliable.
Design the system before the pilot
Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.
- Specify a measurable learning objective, a comparator and a task without tool assistance before adopting a system.
A 90-day district pilot
- Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
- Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
- Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
- Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.
The board decision brief
Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.
Attribution and reuse
Jelena Roganović. Familiarity with ChatGPT Features Modifies Expectations and Learning Outcomes of Dental Students. International Dental Journal (2024-04-26). https://doi.org/10.1016/j.identj.2024.04.012
Companion prepared 2026-11-01. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.