Actionable companion · Motivation & engagement

Learning with agency: Incorporating Generative AI Into a Health Informatics Curriculum to Build 21st Century Competencies: Multisite Pre-Post Study

AI-assisted editorial implementation guide, separate from the original research. Proposed actions are not claims or recommendations attributed to the source authors. No independent peer review of this guide is asserted.

Read original paper Search paper agent CC BY 4.0

Evidence and context

Title and abstract screening; stage labels describe reported settings, not verified numerical ages. Design labels require full-text appraisal. Reported stage(s): Higher education. Verify the participant population and task before transferring the findings.

Original authors’ abstract, reproduced under the linked source license:

Abstract Background We designed learning assignments for students to develop knowledge, skills, and professional attitudes about generative artificial intelligence (AI) in 2 different Master’s level courses in health informatics. Our innovative approach assumed that the students had no technical background or experience in using generative AI tools. Objective This study aims to offer generalizable methods and experiences on integration and assessment of generative AI content into the higher education’s health informatics curricula. The study’s central driver is the preparation of graduate students with generative AI tools, skills, ethical discernment, and critical thinking capacities aligned with the rapidly shifting job-market requirements, independent of graduate students’ backgrounds and technical expertise. Methods During the semester, students completed a pretest and posttest to assess knowledge about generative AI. Reflections explored their expectations and experiences using generative AI to complete their assignments and projects during the semester. Strong emphasis was placed on building skills and professional attitudes by using generative AI. Student engagement in behavioral, emotional, and cognitive domains was explored via detailed analysis of student reflections by faculty. Results Students at the University of Illinois Chicago increased their knowledge about generative AI from 81% to 93% through research of the basic generative AI concepts, as evidenced from outcomes of the open-book pre-and posttests given at the beginning and end of the capstone course. University of San Francisco students also improved from 77% to 80% by the end of the semester. Faculty analysis of student reflections upon completion of the course revealed primary interests in the essentials of generative AI, AI transformations to information and knowledge, and organizational changes influenced by AI adoption in the health care organizations, with ethics being a primary driver of students’ interests and engagement. Conclusions Data from student reflections provided insight into generative AI skills that students developed and that health informatics programs can consider incorporating into their curricula. Building competencies in generative AI will prepare students for the 21st century workforce and enable them to build skills employers are seeking in the new digital health environment.

What kind of evidence is this?

Supporting AI learning science · Comparative / pre-post study reported

Stage: Higher education. Screening labels describe reported context and study-design language. They are not a risk-of-bias rating or proof of effectiveness for an age band.

Study methods · source passage

This was a multisite study of assessment of assignments completed by Master of Health Informatics students at UIC and USF campuses in the fall of 2024. Across both sites, a total of 17 students participated in the research.

Limitations and uncertainty · source passage

This work was a collaboration between 2 distinctly different institutions of higher education in Illinois and California based on the common interest of swiftly bringing generative AI into the classroom. Our programs differ by student demographics, goals, and mode of education (hybrid vs online attendance). This allowed us to test various methods in 2 different academic environments. The cohorts of students were relatively small, and the courses were offered for one academic term, with plans for expanded offerings informed by these findings. We derive strengths by having two distinctly different academic environments and speed of innovation (summarizing results to quickly share with faculty colleagues around the world) and face limitations with our approach (small cohorts not allowing advanced statistical analysis, lack of identical course offerings, different student cohort characteristics). Demographic differences between our student cohorts introduce challenges to group comparisons and influence outcomes, making comparison methods potentially inaccurate. The knowledge test was foundational and may not equally reflect learning in both courses. Due to differences between the programs, we must be careful in making direct comparisons of outcomes between UIC and USF. However, these differences also represent opportunities to widen the scope of our work and generalize it to more schools and faculty looking to learn from our experience of introducing generative AI in various health informatics learning settings and to a wider audience of students. These factors could be excellent subjects of future scholarly inquiry.

Design the system before the pilot

Use this study to inform a learning mechanism or implementation question. It is not direct evidence that autonomous agents improve learning. Keep the proposed agent behavior separate from what the source actually tested.

Measure engagement separately from achievement. Check whether novelty-driven participation persists and whether it translates to independent performance.
Use common rubrics, blind scoring where feasible and a fresh unaided reasoning task. Document permissible assistance and avoid treating AI-detector scores as proof of misconduct.

A 90-day district pilot

Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
Days 16–30 · Design. Choose one disciplinary task. Require a first attempt, a documented AI consultation, a critique of the output and an independent explanation. Align the comparison group and scoring rubric with the same objective. Name an accountable owner and set the baseline, success threshold and budget before implementation.
Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
Days 61–90 · Decide. Blind-score reasoning and accuracy on a new task without AI. Distinguish model examination scores, student perceptions and actual learner gains; only the last directly tests educational benefit. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Freddie Seba; Miriam Isola; Laura Mills; Mohan Zalake; Jacob Krive. Incorporating Generative AI Into a Health Informatics Curriculum to Build 21st Century Competencies: Multisite Pre-Post Study. JMIR Medical Informatics (2025-12-16). https://doi.org/10.2196/76507

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.