Agentic education · Learning science

Actionable companion · Assessment & reasoning

From AI evidence to responsible practice: Evaluating the Accuracy and Educational Potential of Generative AI Models in Pharmacy Education: A Comparative Analysis of ChatGPT and Gemini Across Bloom’s Taxonomy

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not findings or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paperSearch paper agentCC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Age / stage not established. Verify the participants, intervention and comparator before applying this study in a new setting.

Original authors’ abstract, reproduced under the linked source license:

This study evaluated the accuracy and educational potential of three generative AI models, ChatGPT 3.5, ChatGPT 4o, and Gemini 2.5, by addressing pharmacy-related content across three key areas: biostatistics, pharmaceutical calculations, and therapeutics. A total of 120 exam-style questions, categorized by Bloom’s Taxonomy levels (Remember, Understand, Apply, and Analyze), were administered to each model. Overall, the AI models achieved a combined accuracy rate of 77.5%, with ChatGPT 4o consistently outperforming ChatGPT 3.5 and Gemini 2.5. The highest accuracy was observed in therapeutics (83.3%), followed by biostatistics (81.7%) and calculations (67.5%). Performance was strongest at lower Bloom levels, reflecting proficiency in recall and conceptual understanding, but declined at higher levels requiring analytical reasoning. These findings suggest that generative AI tools can serve as effective supplementary aids for pharmacy education, particularly for conceptual learning and review. However, their limitations in quantitative and higher-order reasoning highlight the need for guided use and faculty oversight. Future research should expand to additional subject areas and assess longitudinal learning outcomes to better understand AI’s role in improving critical thinking and professional competence among pharmacy students.

What kind of evidence is this?

Supporting AI learning science · Design requires full-text appraisal

Stage: Age / stage not established. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

We administered 120 exam questions to ChatGPT 3.5, 4o, and Gemini 2.5. The questions were created by the domain expert faculty members based on widely used materials, including the RxPrep Course Book, APhA Complete Review for Pharmacy, Board Vitals, and UWorld for Pharmacy. The questions covered three main knowledge domains, including biostatistics, calculations, and therapeutics. Each topic area had 40 questions evenly distributed across 4 levels of Bloom’s Taxonomy (i.e., 10 questions per level), including Remember, Understand, Apply, and Analyze. The difficulty level of each question in the question pool was first determined by the domain expert faculty, then reviewed by two other faculty members. For conceptual questions, a multiple-choice or multiple-selection format was used with four answer choices. On the other hand, for calculation questions, the format of fill-in-the-blank was used. Initially, we administered these exam questions to ChatGPT 3.5 (February 2024) and evaluated its accuracy. After that, when the newer model ChatGPT 4o (May–June 2024) and Gemini 2.5 (July–August 2025) were available, we input the same questions and re-evaluated the accuracy. In our experiment, we selected the opt-out option in Data Controls in ChatGPT so that the previously entered data would not be used to improve the newer model; thus, the comparison across the models is valid and accurate. Generative AI was prompted to select the correct option listed in each question or to provide the correct answer for calculations. Questions were presented to each model one at a time during the study period, from February 2024 to August 2025. In our study, we used two different prompts for multiple-choice and calculation fill-in-the-blank questions, respectively.

Limitations and uncertainty · source passage

Despite its promising accuracy, AI models’ lower performance in quantitative and higher-order cognitive tasks highlights the need for structured integration and critical evaluation. Overreliance without verification could propagate misconceptions, particularly in dosage calculations or advanced pharmacotherapy. We plan to utilize inferential statistical testing to assess the results in our future research. Additionally, the current study is limited to three key content areas, including calculations, biostatistics, and therapeutics, which may not fully represent the breadth of a pharmacy curriculum. Future research will expand to include other areas such as pharmacology, medicinal chemistry, pharmaceutics, pharmacy law, and other areas such as those required by the Accreditation Council for Pharmacy Education in the Standards 2025 Appendix 1: Required Elements of the Didactic Doctor of Pharmacy Curriculum [21]. It should also assess AI-assisted learning outcomes longitudinally, focusing not just on response accuracy but also on improvements in students’ critical reasoning, problem-solving, and confidence in clinical decision-making. Educators and users of AI tools also may experience model drift and dataset shift over time, complicating implementation and evaluation [22]. Finally, our future study will also consider other popular AI models (e.g., DeepSeek) to have a more comprehensive evaluation.

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

  • Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

  1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
  2. Days 16–30 · Design. Treat the paper as background evidence until its population is verified in the full text. Do not assign an age-specific intervention or extend adult results to children. Name an accountable owner and set the baseline, success threshold and budget before implementation.
  3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
  4. Days 61–90 · Decide. Extract the participant ages or education stage, recruitment, task and comparator before selecting a local pilot. If these cannot be established, retain the paper for design discussion rather than age-band efficacy claims. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Tuan Tran; Uyen Le; Victor Phan. Evaluating the Accuracy and Educational Potential of Generative AI Models in Pharmacy Education: A Comparative Analysis of ChatGPT and Gemini Across Bloom’s Taxonomy. Pharmacy (2025-12-23). https://doi.org/10.3390/pharmacy14010001

Companion prepared 2026-11-01. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.