Agentic education · Learning science

Actionable companion · Cognitive load & attention

Learning with agency: Deception detection in educational AI: challenges for Japanese middle school students in interacting with generative AI robots

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Secondary school. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Educational materials that utilize generative AI (e.g., ChatGPT) have been developed, thus, allowing students to learn through conversations with robots or agents. However, if these artificial entities provide incorrect information (hallucinating), it could lead to confusion among students. To investigate whether students can detect lies from these artificial entities, we conducted an experiment using the social robot Furhat and we make it engage in various types of deceptive interactions. Twenty-two Japanese middle school students participated in ten teaching sessions with Furhat using a human and an anime facial appearances while employing different types of deception: Lying, Paltering, Pandering, and Bullshit. The results revealed that the majority of students were deceived by those lies. Additionally, the robot's facial appearance (i.e., social agency) affected both the learning effectiveness and the likelihood of being deceived. We conclude that an anime robot face is recommended to be used as it excelled in learning effectiveness as it attracts students attention. An anime face also provided protection against deceptive techniques due to its low social agency which leads to ineffectiveness in persuasion and deception. This study underscores the importance of preparing AI-based educational tools and scripts carefully to prevent the dissemination of false information produced through generative AI hallucinations to students.

What kind of evidence is this?

Embodied learning agents · Evidence synthesis

Stage: Secondary school. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

We conducted an educational HRI experiment at a Japanese public middle school in Hakodate. Twenty-two students participated in our experiment. All the students are of a Japanese ethnicity and their ages ranged from 14 to 15 years old. The number of students in the educational sessions with the robot ranged from 2 to 4 students per session.

Limitations and uncertainty · source passage

Certainly, the low number of participants is a limitation in our study. Moreover, we focused on only one type of students (i.e., middle school students), thus, varying the students to include other ages and grades which would consequently lead to needing a bigger sample would certainly lead to more significant results. However, there were many results in our study that highlight the dangers of relying on generative AI in educational scenarios. Thus, customized scripts are highly recommended especially for schools as students are not expecting to obtain faulty information from teachers. Thus, generative AI hallucinations dangers are serious and must be dealt with accordingly.

Design the system before the pilot

Compare the embodied interaction with an equivalent human-led or screen-based activity. Separate the effects of novelty and social presence from learning. Review physical access, sensory needs and adult supervision.

  • Limit simultaneous instructions and unnecessary dialogue. Compare task performance and a suitable mental-effort measure; easier-feeling work is not automatically better learning.
  • Measure engagement separately from achievement. Check whether novelty-driven participation persists and whether it translates to independent performance.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Ask learners to attempt a problem unaided, compare an agent hint, explain a revision and verify the final reasoning against a trusted source. Make the permitted use explicit for each assignment. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Measure unaided transfer and delayed retention, calibration of confidence and the ability to identify an incorrect AI explanation. Record whether help-seeking becomes answer copying. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Ahmed Salem; Kaoru Sumi. Deception detection in educational AI: challenges for Japanese middle school students in interacting with generative AI robots. Frontiers in Artificial Intelligence (2024-12-18). https://doi.org/10.3389/frai.2024.1493348

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.