Agentic education · Learning science

Actionable companion · Feedback & scaffolding

Learning with agency: Assessing the Usability of ChatGPT for Formal English Language Learning

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Age / stage not established. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Recently, the emerging technologies have been constantly shaping the education domain, especially the use of artificial intelligence (AI) for language learning, which has attracted significant attention. Many of the AI tools are being used for learning foreign languages, in both formal and informal ways. There are many studies that have explored the potential of the recent technology “ChatGPT” for education and learning languages, but none of the existing studies have conducted any exploratory study for assessing the usability of ChatGPT. This paper conducts an assessment for usability of ChatGPT for formal English language learning. The study uses a standard questionnaire-based approach to ask participants about their feedback for usefulness and effectiveness of ChatGPT. The participants were asked for their feedback after performing series of tasks related to formal English language learning with ChatGPT. A variety of student participants were selected for this study with diverse English language proficiency levels, education levels, and nationalities. The quantitative analysis of the participant responses shed light on their experience with regards to the usability of ChatGPT for performing different English language learning tasks such as conversation, writing, grammar, and vocabulary. The findings from this study are quite promising and indicate that ChatGPT is an effective tool to be used for formal English language learning. Overall, this study contributes to the fast-growing research domain on using emerging technologies for formal English language learning by conducting in-depth assessment of usability for ChatGPT in formal English language learning.

What kind of evidence is this?

Supporting AI learning science · Survey / observational study

Stage: Age / stage not established. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

A total of 10 participants were recruited for this study, which corresponds to the well-established methodology, namely usability testing described in the Introductory part, and which follows the rule of only 5 users. The recruitment process consisted of disseminating regarding the research activity through the official communication channels of the “Norwegian University of Science and Technology (NTNU), Gjøvik”. The target participants for this activity were exclusively students enrolled in any department of the university. The dissemination post regarding the research activity is available at the link (https://shorturl.at/kuwS8) (accessed on 1 August 2023). Participant information, such as email, country, and current study program, was collected from each participant using an online registration form. The data were collected in a confidential manner to protect participant privacy and anonymity.

Limitations and uncertainty · source passage

Objective: To evaluate the potential of ChatGPT for English language learning in formal settings, by measuring its usability through exploring learners’ satisfaction levels with the responses generated by ChatGPT and learners’ perceptions and experiences about its ease of use, usefulness, potential benefits, and limitations while learners perform English language learning tasks namely writing, conversation, grammar, and vocabulary.

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Give feedback on the learner’s reasoning, then fade hints. Record whether the learner can complete the next task without that scaffold.
  • Limit simultaneous instructions and unnecessary dialogue. Compare task performance and a suitable mental-effort measure; easier-feeling work is not automatically better learning.
  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Treat the paper as background evidence until its population is verified in the full text. Do not assign an age-specific intervention or extend adult results to children. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Extract the participant ages or education stage, recruitment, task and comparator before selecting a local pilot. If these cannot be established, retain the paper for design discussion rather than age-band efficacy claims. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Sarang Shaikh; Sule Yildirim Yayilgan; Blanka Klimova; Marcel Pikhart. Assessing the Usability of ChatGPT for Formal English Language Learning. European Journal of Investigation in Health, Psychology and Education (2023-09-21). https://doi.org/10.3390/ejihpe13090140

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.