Actionable companion · Feedback & scaffolding

Learning with agency: Intelligent tutoring systems for word problem solving in COVID-19 days: could they have been (part of) the solution?

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paper Search paper agent CC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Age / stage not established. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Abstract The COVID-19 pandemic led to the lockdown of schools in many countries, forcing teachers and students to carry out educational activities remotely. In the case of mathematics, developing remote instruction based on both synchronous and asynchronous technological solutions has proven to be an extremely complex challenge. Specifically, this was the case in topics such as word problem solving, as this domain requires intensive supervision and feedback from the teacher. In this piece of research, we present an evaluation of how technology is employed in the teaching of mathematics, with particular relevance to learning during the pandemic. For that purpose, we conducted a systematic review, revealing the almost complete absence of experiments in which the use of technology is not mediated by the teacher. These results reflect a pessimistic vision within the field of mathematics education about the possibilities of learning when the student uses technology autonomously. Bringing good outcomes out of a bad situation, the pandemic crisis may represent a turning point from which to start directing the research gaze towards technological environments such as those mediated by artificial intelligence. As an example, we provide a study illustrating to what extent intelligent tutoring systems can be cost-effective compared to one-to-one human tutoring and mathematic learning-oriented solutions for intensive supervision in the teaching of word problem solving, especially appropriate for remote settings. Despite the potential of these technologies, the experience also showed that student socioeconomic level was a determining factor in the participation rate with an intelligent tutoring system, regardless of whether or not the administration guaranteed students' access to technological resources during the COVID-19 situation. Keywords: Intelligent tutoring system, Problem-solving, Systematic review, Technology-based mathematics education, COVID-19, Socioeconomic level

What kind of evidence is this?

Intelligent tutoring & adaptive learning · Evidence synthesis

Stage: Age / stage not established. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

Before proceeding with the selection of studies from the initial search, we defined the inclusion and exclusion criteria as follows. (1) The articles should report empirical studies. Consequently, review studies, theoretical papers or classroom notes were excluded. (2) The studies had to evaluate at least one technological tool with which the study participants completed mathematical activities. An example of exclusion based on this criterion would be a study that used eye-tracking technology to assess externally what strategies students used while solving problems. (3) The studies had to be focused on the teaching and learning of mathematics. Studies framed in other subjects such as science, technology, etc., were discarded. (4) The studies had to offer enough information to enable us to determine the role that technology played in the learning process.

Limitations and uncertainty · source passage

This investigation has some limitations that allow us to make recommendations for further research. Regarding the systematic review, we focused our attention on journals with a specialization in mathematics education, but the review could be expanded to all journals, including those whose primary scope is technology. This would allow researchers to assess whether there is a gap between the research conducted in mathematics education and the technological opportunities available. With regard to the study carried out with HINTS as an example, for the measurements for poverty status, multi-national studies like PISA do not use administrative metrics such as free and reduced lunch-eligibility, since these metrics are generally not available and not comparable across countries (Avvisati, 2020). Although this issue would not arise in this study, the free and reduced lunch-eligibility measurement has been criticized for a number of reasons, including its sole reliance on income, the risk of misclassification, and the fact that it was not designed to be used as a measurement of low SES (Rutkowski et al., 2018). In addition, it would have been desirable to have a larger sample size and also a measurement of the students’ prior proficiency at solving word problems in a paper-and-pencil setting, before using HINTS. However, the lockdown situation offered limited possibilities and also made it impossible to gather such data.

Design the system before the pilot

Document the learner model, mastery rule, hint sequence and adaptation signal. Compare adaptive routing with the same content in a non-adaptive sequence. Check whether early errors lead to persistent low-expectation pathways.

Give feedback on the learner’s reasoning, then fade hints. Record whether the learner can complete the next task without that scaffold.
Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
Days 16–30 · Design. Treat the paper as background evidence until its population is verified in the full text. Do not assign an age-specific intervention or extend adult results to children. Name an accountable owner and set the baseline, success threshold and budget before implementation.
Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
Days 61–90 · Decide. Extract the participant ages or education stage, recruitment, task and comparator before selecting a local pilot. If these cannot be established, retain the paper for design discussion rather than age-band efficacy claims. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Javier del Olmo-Muñoz; José Antonio González-Calero; Pascual D Diago; David Arnau; Miguel Arevalillo-Herráez. Intelligent tutoring systems for word problem solving in COVID-19 days: could they have been (part of) the solution?. Zdm (2022-07-22). https://doi.org/10.1007/s11858-022-01396-w

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.