Actionable companion · Assessment and evidence of learning

From evidence to district action: Accuracy, readability, and health information quality of AI chatbot responses for patient education on intermittent fasting in chronic kidney disease: a prospective structured evaluation

AI-assisted editorial implementation guide. The district actions are proposed applications, not findings or recommendations attributed to the original authors. No additional peer review is asserted.

Evidence and context

Health-professions education context: test transfer to K–12 rather than assuming it.

Original authors’ abstract, reproduced under the linked source license:

Background Intermittent fasting (IF) represents a dietary strategy with known metabolic benefits, however its safety and applicability in patients with chronic kidney disease (CKD) remains poorly understood. AI chatbots’ reliability for clinical and dietary guidance in clinically complex areas remains underexplored. No study has examined AI performance on IF in patients with CKD. This study evaluated the accuracy, consistency, readability, and quality of responses generated by ChatGPT-5 and Claude 4.5 Sonnet regarding IF in CKD. Methods Twenty-five patient-centered questions spanning seven clinical domains were developed by two board-certified nephrologists and submitted independently to ChatGPT-5 and Claude 4.5 Sonnet twice per model. Prompts were submitted under standard and unmodified user conditions without customized system instructions to represent a zero-shot evaluation that is illustrative of typical patient use. Responses were evaluated for accuracy, consistency (comparatively between responses), and appropriateness by the evaluators. Response readability and quality were assessed using the Flesch-Kincaid Grade Level (FKGL) formula and the DISCERN instrument respectively. Results Both ChatGPT-5 and Claude 4.5 Sonnet demonstrated identical accuracy scores during the first question prompts with notable improvement upon re-prompting after 24-hours. Both models demonstrated an identical internal consistency at 90%. ChatGPT-5 produced responses at the middle school level, better aligned with the US adult health literacy benchmark, while Claude 4.5 Sonnet generated responses at the high school level. DISCERN scoring was comparable between chatbots indicative of good response quality. Both chatbots had similar performance regarding individual subdomains. Conclusions Both chatbots illustrated clinically accurate and cautious responses to questions regarding IF in CKD. ChatGPT-5 responses were better suited to the average health literacy of the general United States adult population. Despite our findings, we do not recommend either chatbot be used as the primary source for clinical counseling regarding IF in CKD. It is important for clinicians to help to guide patients toward evidence-based clinical information.

A 90-day district pilot

1. Days 1–15 · Appraise. Have the curriculum lead and a teacher read the methods and limitations. Record the study design, sample, comparator, outcomes and evidence gaps. Define a local question that the study can help inform.
2. Days 16–30 · Design. Redesign one low-stakes assessment: require a first attempt, an AI-assisted revision, a source-check log and a short oral explanation. Use the same rubric for the comparison task. Name an accountable owner and set the baseline, success threshold and budget before implementation.
3. Days 31–60 · Test. Start with a small voluntary cohort, preserve an accessible non-AI route and require teacher review. Use synthetic or de-identified examples; approve data handling and tool terms through district processes.
4. Days 61–90 · Decide. Blind-score reasoning, factual accuracy and unaided transfer on a fresh task. Record suspected misconduct separately from demonstrated evidence. Compare with baseline, document uncertainty and stop or revise if learning worsens, access is unequal or critical errors persist.

The board decision brief

Report the learning problem, evidence quality, participating population, total cost including review time, measured benefit, errors, access differences and remaining uncertainties. Make a dated recommendation to stop, revise or expand. A positive perception survey alone does not establish learning improvement.

Attribution and reuse

Joel Shah; William Wells-Gatnik; Biff Palmer; Prakrati Acharya; Wisit Cheungpasitporn. Accuracy, readability, and health information quality of AI chatbot responses for patient education on intermittent fasting in chronic kidney disease: a prospective structured evaluation. Frontiers in Nephrology (2026-05-29). https://doi.org/10.3389/fneph.2026.1857793

Companion prepared 2026-10-30. Original authorship and DOI are preserved. The abstract is an attributed source extract; the implementation plan is new AI-assisted editorial material, offered under the same CC BY 4.0 license. This guide has not undergone independent peer review.