Actionable companion · Feedback & scaffolding

Learning with agency: What students ask matters: Static and longitudinal questioning patterns in student–LLM dialogue and self-reported cognitive engagement

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paper Search paper agent CC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Abstract This study examines students’ questioning as an interaction mechanism in student–LLM philosophical dialogue and how it relates to self-reported critical thinking (CT) and creative thinking (CrT) engagement. Participants were 106 first-year postgraduate students in a Chinese university general education course who completed an end-of-semester assignment using an institutional ChatGPT model embedded in the learning platform. Student questions in dialogue transcripts were coded into six categories aligned with the revised Bloom’s taxonomy. We then applied latent profile analysis (LPA) to identify static questioning profiles and group-based trajectory modeling (GBTM) to capture longitudinal development across conversation rounds, followed by subgroup comparisons on perceived CT and CrT. LPA yielded three profiles: Fact-focused, Explanation-focused, and Evaluation-focused Questioners. GBTM revealed two trajectories with a key divergence around Rounds 5–7: some students plateaued at application and analytical questioning, while others progressed toward evaluative and exploratory questioning. Fact-focused Questioners reported lower CT and CrT engagement scores; plateauing trajectories reported lower CT. Findings highlight how questioning patterns are associated with differential self-reported cognitive engagement in student–LLM dialogue and inform scaffold design to promote higher-order inquiry.

What kind of evidence is this?

Supporting AI learning science · Design requires full-text appraisal

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

A sequential explanatory mixed-methods design [37] was employed to comprehensively elucidate students’ questioning mechanisms during student–LLM philosophical dialogue. This design integrates qualitative and quantitative data to address complex educational phenomena [38], with four distinct phases capturing both static and longitudinal dimensions of questioning patterns: qualitative content analysis of chat logs; latent profile analysis (LPA) to identify static subgroups based on questioning type proportions; group-based trajectory modeling (GBTM) to reveal longitudinal trajectory patterns; and statistical comparison of subgroups and trajectory groups on cognitive outcome measures.

Limitations and uncertainty · source passage

Several limitations should be noted. First, the single-course and single-university design limits both generalizability and causal inference; the observed associations may reflect pre-existing individual differences rather than effects of questioning. Second, the primary outcomes were self-reported strategy engagement collected after a single dialogue activity, not objective pre–post change scores; accordingly, the findings reflect associations with task-specific engagement rather than evidence of cognitive growth, and the correlation with objective scores (CT: r = .642; CrT: r = .726) should be interpreted with caution due to shared source variance from the same dialogue transcripts. Third, although the sample of N = 106 was adequate for model estimation, relatively small subgroups (e.g., n = 23 for the Fact-focused profile) limit the precision of parameter estimates, and both LPA profiles and GBTM trajectories should be treated as exploratory pending replication with larger samples. Fourth, although inter-rater reliability was established on a randomly selected 20% subset (κ = 0.704, substantial agreement) and subsequent coding followed a calibrated procedure with ongoing consultation on uncertain cases, reliability was not independently verified for the full corpus.

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

Give feedback on the learner’s reasoning, then fade hints. Record whether the learner can complete the next task without that scaffold.
Measure engagement separately from achievement. Check whether novelty-driven participation persists and whether it translates to independent performance.
Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Jingru Qu; Ling Dai; Mengjie Yin. What students ask matters: Static and longitudinal questioning patterns in student–LLM dialogue and self-reported cognitive engagement. PLOS One (2026-09-08). https://doi.org/10.1371/journal.pone.0356183

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.