Agentic education · Learning science

Actionable companion · Feedback & scaffolding

Learning with agency: Development and application of a multi-modal task analysis to support intelligent tutoring of complex skills

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Age / stage not established. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Abstract Background Contemporary work in the design and development of intelligent training systems employs task analysis (TA) methods for gathering knowledge that is subsequently encoded into task models. These task models form the basis of intelligent interpretation of student performance within education and training systems. Also referred to as expert models, they represent the optimal way(s) of performing a training task. Within Intelligent Tutoring Systems (ITSs), real-time comparison of trainee task performance against the task model drives automated assessment and interactive support (such as immediate feedback) functionality. However, previous task analysis (TA) methods, including various forms of cognitive task analysis (CTA), may not be sufficient to support identification of the detailed design specifications required for the development of an ITS for a complex training task incorporating multiple underlying skill components, as well as multi-modal information presentation, assessment, and feedback modalities. Our current work seeks to develop an ITS for training Robotic Assisted Laparoscopic Surgery (RALS), a complex task domain that requires a coordinated utilization of integrated cognitive, psychomotor, and perceptual skills. Results In this paper, we describe a methodological extension to CTA, referred to as multi-modal task analysis (MMTA) that elicits and captures the nuances of integrated and isolated cognitive, psychomotor, and perceptual skill modalities as they apply to training and performing complex operational tasks. In the current case, we illustrate the application of the MMTA method described here to RALS training tasks. The products of the analysis are quantitatively summarized, and observations from a preliminary qualitative validation are reported. Conclusions We find that iterative use of the described MMTA method leads to sufficiently complete and robust task models to support encoding of cognitive, psychomotor, and perceptual skills requisite to training and performance of complex skills within ITS task models. Keywords: Task models, Task analysis, Intelligent tutoring system, Robot-assisted surgery, Psychomotor skills, Cognitive skills, Perceptual skills, Multi-modal task, Simulation-based training

What kind of evidence is this?

Intelligent tutoring & adaptive learning · Qualitative / mixed methods

Stage: Age / stage not established. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

The objective of this effort was to leverage best practices and existing TA methods to develop a novel approach to complex task decomposition and analysis that would enable highly granular modeling of both isolated and integrated task components, as well as identification of metrics necessary to assess performance and provide intelligent instruction in real-time. For example, advanced hand motion tracking metrics can be used to provide granular assessments of psychomotor skills. The MMTA was specifically designed to develop simulation-based training for complex tasks involving cognitive, perceptual, and psychomotor skills. While there has been increasing emphasis on both technical and non-technical skills within the surgical domain, and development of multiple evaluation techniques, these skills are largely trained and assessed in isolation (Dedy, Bonrath, Zevin, & Grantcharov, 2013). As demonstrated by Skinner (2014), psychomotor and cognitive/perceptual skill relevant to laparoscopic surgery are acquired and decay at differential rates when performed in isolation as compared to performance of the same skills in an integrated manner.

Limitations and uncertainty · source passage

This CTA approach provides an excellent model for a multi-step process targeting various aspects of simulation-based training design, which is broadly applicable to training systems design across a range of domains. The MMTA sought to expand on this method, including several key differentiating components. First, the MMTA adapts Step 3 of the above process to specifically target individual skill components (cognitive, perceptual, and psychomotor) and to emphasize the use of multi-modal information presentation via visual, auditory, and haptic stimuli to optimize intelligent instruction. Furthermore, the MMTA process and analysis was designed to not only capture the behaviors and observations of skilled practitioners and instructors, but also of novice1 and intermediate users to capture non-optimal and inaccurate deviations from the best path to performing a training task within the task model. Therefore, unlike the CTA method used by Cannon-Bowers et al. (2013), which included skilled practitioners and instructors, the MMTA design included surgical trainees with varying levels of experience within the RALS domain. Finally, the MMTA method, detailed in the following section, includes a step involving experts from a similar but distinct specialty domain in order to identify specialty-agnostic instructional design requirements and to assess generalizability of the resulting TA products.

Design the system before the pilot

Document the learner model, mastery rule, hint sequence and adaptation signal. Compare adaptive routing with the same content in a non-adaptive sequence. Check whether early errors lead to persistent low-expectation pathways.

  • Give feedback on the learner’s reasoning, then fade hints. Record whether the learner can complete the next task without that scaffold.
  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Treat the paper as background evidence until its population is verified in the full text. Do not assign an age-specific intervention or extend adult results to children. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Extract the participant ages or education stage, recruitment, task and comparator before selecting a local pilot. If these cannot be established, retain the paper for design discussion rather than age-band efficacy claims. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Anna Skinner; David Diller; Rohit Kumar; Jan Cannon-Bowers; Roger Smith; Alyssa Tanaka; Danielle Julian; Ray Perez. Development and application of a multi-modal task analysis to support intelligent tutoring of complex skills. International Journal of Stem Education (2018-04-15). https://doi.org/10.1186/s40594-018-0108-5

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.