Agentic education · Learning science

Actionable companion · Feedback & scaffolding

Learning with agency: Exploring Artificial Intelligence-Generated Feedback for Pharmacy Students †

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Age / stage not established. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Background: The integration of artificial intelligence (AI) into educational environments is accelerating, with growing interest in its potential to enhance learning outcomes. This pilot study aimed to assess student perceptions regarding the quality and educational impact of AI-generated feedback. Methods: Second-professional-year pharmacy students (n = 148) enrolled in a drug literature evaluation course received AI-generated feedback for a draft assignment which required students to critically evaluate a recently published journal article. Feedback was generated by an AI assistant that offered iterative feedback based on the assignment rubric. After the activity, students completed an electronic survey instrument evaluating the AI feedback regarding its alignment with rubric criteria, accuracy, clarity, and tone. Student perceptions of the feedback’s impact on learning were also assessed. Results: A total of 148 students (100%) completed the activity and responded to at least one item of the survey tool. Most students agreed with the AI-generated feedback for each section of the assignment in regard to clarity (61.5% to 76.2%) and tone (66.9% to 74.3%), with lower agreement related to alignment with the rubric (33.8% to 60.4%) and accuracy (34.5% to 54.2%). Students also expressed concern about the reliability of AI feedback (71.2%) and risk for plagiarism (50%). Conclusions: AI-generated feedback was positively received in some areas. However, students expressed apprehension regarding its accuracy, reliability, and potential accusation of plagiarism. These findings suggest further refinement is necessary to enhance AI trustworthiness and educational value in academic settings.

What kind of evidence is this?

Supporting AI learning science · Survey / observational study

Stage: Age / stage not established. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

The instructional technology team at our institution developed an AI writing assistant, known as Charlie AI, designed to provide rubric-based feedback on student assignments. The AI assistant is powered by OpenAI’s GPT-4 model and offers iterative feedback based on the specific assignment rubric, allowing students to refine and resubmit their work while maintaining data privacy. A pilot active learning exercise using Charlie AI was incorporated into a written assignment for a second-professional-year, required drug information and literature evaluation course. The assignment required critical evaluation of a recently published journal article with respect to ten different sections: journal and authors, study subjects, study design, endpoints, statistical analysis, results, overall strengths, overall limitations, generalizability of results, and clinical impact (rubric provided in the Supplementary Materials). The assignment had a two-page limit. Students submitted their draft written assignment one week prior to the due date and Charlie AI provided feedback on each section of the assignment rubric. Students reviewed the AI-generated feedback during a designated class period and were permitted to discuss their feedback with their peers and the course instructors. After the activity students had the opportunity to revise their draft based on Charlie AI’s feedback before submission of the final assignment. The overall objective of this in-class activity was to foster critical thinking skills and help students understand the strengths and limitations of AI. This course includes three journal club assignments distributed through the semester. Historically, students enrolled in this course participate in three in-class peer-review sessions for the journal club assignments, during which time they receive electronic, anonymous feedback on their draft assignment from three of their classmates and provide feedback to three other classmates. Our experience with this peer-review activity has demonstrated enhanced learning behaviors, increased interest in peer-review, and improved self-competence regarding literature evaluation [17,18]. In this pilot, the traditional peer-review activity associated with the third and final journal club assignment was replaced with the Charlie AI feedback activity.

Limitations and uncertainty · source passage

One-hundred and forty-eight second-year pharmacy students reviewed the Charlie AI-generated feedback for their written draft assignment during class, with the opportunity to edit their draft before submitting their final assignment. All students enrolled in the course (100%, 148/148) completed at least one of the survey instrument items. Table 1 shows the results of student agreement with Charlie AI’s feedback in relation to its alignment with the assignment rubric, accuracy, clarity, and tone for each section of the assignment. Levels of agreement with Charlie AI-generated feedback varied across journal club rubric domains, with most students agreeing with the AI-generated feedback for each specific section of the assignment in regard to clarity (61.5% to 76.2%) and tone (66.9% to 74.3%), with lower agreement related to alignment with the rubric (33.8% to 60.4%) and accuracy (34.5% to 54.2%). Student agreement was lowest for the accuracy of feedback related to the more nuanced aspects of study evaluation such as the overall limitations (39.9%, 57/143) and generalizability of study results (34.5%, 49/142).

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Give feedback on the learner’s reasoning, then fade hints. Record whether the learner can complete the next task without that scaffold.
  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Treat the paper as background evidence until its population is verified in the full text. Do not assign an age-specific intervention or extend adult results to children. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Extract the participant ages or education stage, recruitment, task and comparator before selecting a local pilot. If these cannot be established, retain the paper for design discussion rather than age-band efficacy claims. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Diyaben Patel; Mariam Abouelsaad; Jason B. Reed; Amy Heck Sheehan. Exploring Artificial Intelligence-Generated Feedback for Pharmacy Students †. Pharmacy (2026-10-01). https://doi.org/10.3390/pharmacy14070147

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.