Agentic education · Learning science

Actionable companion · Assessment & reasoning

Learning with agency: The Impact of Artificial Intelligence-Supported Instruction on Student Learning in STEM: A Systematic Review and Meta-Analysis

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Secondary school. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

The rapid integration of artificial intelligence (AI) technologies into educational contexts has introduced innovative instructional approaches, particularly within Science, Technology, Engineering, and Mathematics (STEM) education. Although an increasing number of empirical studies have examined AI-supported instruction, existing findings remain heterogeneous, making it difficult to draw firm conclusions about its overall effectiveness. This study aims to systematically synthesize experimental and quasi-experimental research on AI-supported instructional interventions in STEM education, quantify their overall effects on student learning outcomes, and examine potential moderating factors, including educational level, STEM discipline, and intervention duration. A comprehensive systematic literature search was conducted across Web of Science, Scopus, ERIC, ScienceDirect, and Google Scholar, covering studies published between 2005 and 2025. A total of 35 studies meeting predefined inclusion criteria were included in the meta-analysis. Effect sizes were calculated using Hedges’ g, and a Random Effects Model (REM) was employed to account for heterogeneity among studies. Moderator analyses were conducted for educational level, STEM discipline, and intervention duration. Publication bias was assessed using multiple diagnostic methods. The meta-analysis revealed a statistically significant overall positive effect of AI-supported instruction on student learning outcomes in STEM education (g = 0.67, 95% CI [0.49, 0.85], p < 0.001). Moderator analyses indicated that AI interventions were most effective at the high school level. Although Science and Mathematics disciplines showed slightly higher effect sizes, the between-group difference was not statistically significant (Q = 4.85, df = 2, p = 0.088). Regarding intervention duration, the highest effect size was observed in interventions lasting more than one month and up to two months, though no consistent pattern of increasing effectiveness with longer durations was found. Publication bias analyses suggested minimal influence on the overall findings. AI-supported instructional interventions demonstrate a moderately to highly positive impact on student learning outcomes in STEM education. The effectiveness of these interventions varies according to educational level, disciplinary context, and intervention duration. These findings provide robust empirical evidence supporting the pedagogical value of AI in STEM education and offer guidance for educators and policymakers regarding effective implementation.

What kind of evidence is this?

Supporting AI learning science · Evidence synthesis

Stage: Secondary school. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

This meta-analysis study was conducted in accordance with the PRISMA guidelines (Page et al., 2021) to obtain systematic and reliable answers to the research questions (Supplementary Material). The review protocol was registered on OSF (Registration DOI: https://doi.org/10.17605/OSF.IO/BQZ7W (accessed on 10 June 2026)). The main objective of the study is to systematically synthesize the findings of experimental research examining the effects of AI applications on learning achievement in STEM fields. Accordingly, a comprehensive literature review was carried out. The databases to be used in the search, as well as the inclusion and exclusion criteria for the studies, were determined in advance. The databases examined include Web of Science, Scopus, ERIC, ScienceDirect, Google Scholar, and the Council of Higher Education National Thesis Center.

Limitations and uncertainty · source passage

Despite the strengths of this meta-analysis, several limitations should be acknowledged. The high level of heterogeneity indicates that a substantial proportion of variance remains unexplained, suggesting the need for future research to examine additional moderating variables, including implementation fidelity, teacher involvement, and learner characteristics (Hwang et al., 2020a).

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.
  • Co-design accommodations with learners and specialists. Check access and error burden across groups without inferring subgroup effectiveness from small or unreported samples.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Ask learners to attempt a problem unaided, compare an agent hint, explain a revision and verify the final reasoning against a trusted source. Make the permitted use explicit for each assignment. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Measure unaided transfer and delayed retention, calibration of confidence and the ability to identify an incorrect AI explanation. Record whether help-seeking becomes answer copying. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Yunus Doğan; Zeynep Kılıç; Yusuf Kalınkara; Tarık Talan. The Impact of Artificial Intelligence-Supported Instruction on Student Learning in STEM: A Systematic Review and Meta-Analysis. Journal of Intelligence (2026-06-15). https://doi.org/10.3390/jintelligence14060109

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.