Actionable companion · Feedback & scaffolding

Learning with agency: Measuring the effects of pedagogical agent cognitive and affective feedback on students’ academic performance

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paper Search paper agent CC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Age / stage not established. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

There is still a debate on the influence and effectiveness of pedagogical agents in a learning environment, especially on the means these agents employ for enhancing students’ academic performance. The current study aims at measuring the effectiveness of cognitive and affective feedback (CaAF) types that a human teacher and a virtual Affective Pedagogical Tutor (APT) used in their groups of students (control and experimental groups respectively) in an authentic long-term learning situation. Participants were a sample of 115 students carrying out collaborative activities in a “web design” course. Our findings showed that APT cognitive feedback (CF) significantly increased students’ learning outcomes compared to the human teacher’s feedback, whereas APT affective feedback (AF) only achieved partial success. Nevertheless, the study has some limitations: it is based on a single course and a specific academic context, limiting the generalizability of its findings. Additionally, while cognitive feedback demonstrated a clear impact, the analysis of affective feedback was less conclusive, and its design requires further refinement. Finally, the cross-sectional design of the study restricts the ability to assess whether improvements in learning outcomes persist over time. Future research directions include exploring the generalizability of results across diverse disciplines, deepening the analysis of affective feedback, and incorporating longitudinal studies to evaluate the durability of the observed effects.

What kind of evidence is this?

Conversational & pedagogical agents · Survey / observational study

Stage: Age / stage not established. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

Participants were a sample of 115 students attending the course “Web Design.” We randomly divided students in two big groups, a control and an experimental group, with 57 and 58 students, respectively. Then, each big group was further divided into smaller teams. We wanted the teams to be big enough so that to promote more interaction in the team forums. For this reason, we created teams between 4 and 7 members. More specifically, in the control group (supervised by the human teacher), 12 teams were formed randomly: nine teams of four members and three teams of seven members. In the experimental group (where the APT was acting), 16 teams were also formed randomly: three teams of seven members, one team of five members and eight teams of four members. In this sense, the groups have been randomly distributed; both samples have been independent and had a normal distribution.

Limitations and uncertainty · source passage

There is still a debate on the influence and effectiveness of pedagogical agents in a learning environment, especially on the means these agents employ for enhancing students’ academic performance. The current study aims at measuring the effectiveness of cognitive and affective feedback (CaAF) types that a human teacher and a virtual Affective Pedagogical Tutor (APT) used in their groups of students (control and experimental groups respectively) in an authentic long-term learning situation. Participants were a sample of 115 students carrying out collaborative activities in a “web design” course. Our findings showed that APT cognitive feedback (CF) significantly increased students’ learning outcomes compared to the human teacher’s feedback, whereas APT affective feedback (AF) only achieved partial success. Nevertheless, the study has some limitations: it is based on a single course and a specific academic context, limiting the generalizability of its findings. Additionally, while cognitive feedback demonstrated a clear impact, the analysis of affective feedback was less conclusive, and its design requires further refinement. Finally, the cross-sectional design of the study restricts the ability to assess whether improvements in learning outcomes persist over time. Future research directions include exploring the generalizability of results across diverse disciplines, deepening the analysis of affective feedback, and incorporating longitudinal studies to evaluate the durability of the observed effects.

Design the system before the pilot

Define whether the agent asks questions, gives hints, explains or assesses. Test hallucinated feedback, excessive praise, misleading confidence and responses when the learner is confused. A conversational interface alone does not establish autonomous agency.

Give feedback on the learner’s reasoning, then fade hints. Record whether the learner can complete the next task without that scaffold.
Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
Days 16–30 · Design. Treat the paper as background evidence until its population is verified in the full text. Do not assign an age-specific intervention or extend adult results to children. Name an accountable owner and set the baseline, success threshold and budget before implementation.
Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
Days 61–90 · Decide. Extract the participant ages or education stage, recruitment, task and comparator before selecting a local pilot. If these cannot be established, retain the paper for design discussion rather than age-band efficacy claims. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Marta Arguedas; Thanasis Daradoumis; Santi Caballé. Measuring the effects of pedagogical agent cognitive and affective feedback on students’ academic performance. Frontiers in Artificial Intelligence (2024-12-16). https://doi.org/10.3389/frai.2024.1495342

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.