Actionable companion · Learning design & outcomes

Learning with agency: Measuring Learner Satisfaction of an Adaptive Learning System

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paper Search paper agent CC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Tertiary educational institutions have continually implemented various educational technologies to support student learning. An example is adaptive learning systems, within which learners take ownership of their learning experience and accelerate future learning. Despite the importance of considering learner satisfaction in the development of such systems given how it has been widely acknowledged as an indication of the success of e-learning systems, research in the area of adaptive learning technologies for education has concentrated more on tailoring instruction to implement personalised learning. A review of instruments measuring learner satisfaction of virtual learning environments found that the learner satisfaction questionnaire (LSQ) that was developed to measure learner satisfaction of e-learning systems, and preliminarily validated by means of exploratory factor analysis, was most suitable for adaptation. This study sought to adapt and validate the LSQ for the purpose of measuring learner satisfaction of an adaptive learning system developed in-house. A total of 121 tertiary students recruited via availability sampling took part in this study. Hierarchical confirmatory factor analysis was performed as part of the validation. Results presented the adapted LSQ as a 14-item instrument that can be readily deployed on a broad scale basis. The adapted LSQ also yielded valid and reliable satisfaction scores both at the subscale as well as the overall scale level. Practical implications are discussed, noting that such scores could inform the further development and refinement of AdLeS or similar systems, with the view of benefiting students.

What kind of evidence is this?

Intelligent tutoring & adaptive learning · Survey / observational study

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

Participants comprised 81.2% of the part-time undergraduates enrolled in a level-one Calculus course across one semester in a tertiary educational institution (see Table 2) in Singapore. Their age ranged between 20 and 67 years (M = 27.68, SD = 7.02 years), and 22.2% identified themselves as female. There were 121 participants in total, and this was deemed adequate on the basis of the recommendations of [53,54], and considering the simplicity of the pre-specified measurement model proposed by [47] (see Appendix A).

Limitations and uncertainty · source passage

Results from stages 1 and 2 of this study suggested a three-factor second-order measurement model explaining learner satisfaction toward an in-house adaptive learning system (i.e., AdLeS). Items reviewed, retained, and added from stage 1 appeared to fit the three-factor second-order measurement model. The two additional items (i.e., CONT6—the system supports my learning, and PERS4—the system recommends topics that reflects my learning progress) were clustered within the CONT and PERS subscales, respectively. This is within expectation, as item CONT6 is related to how students perceive AdLeS as a support to their learning, and that content in AdLeS inevitably plays a role in this support; item PERS4 is related to how students perceive AdLeS to be adaptive, as reflected by how AdLeS recommends topics that reflect students’ learning.

Design the system before the pilot

Document the learner model, mastery rule, hint sequence and adaptation signal. Compare adaptive routing with the same content in a non-adaptive sequence. Check whether early errors lead to persistent low-expectation pathways.

Specify a measurable learning objective, a comparator and a task without tool assistance before adopting a system.

A 90-day district pilot

Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Lyndon Lim; Seo Hong Lim; Rebekah Wei Ying Lim. Measuring Learner Satisfaction of an Adaptive Learning System. Behavioral Sciences (2022-07-30). https://doi.org/10.3390/bs12080264

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.