Journal DOI 10.68956/dyh5-kwte

Creative Commons source reading · Traumatic brain injury recovery

GPT-agents based on medical guidelines can improve the responsiveness and explainability of outcomes for traumatic brain injury rehabilitation.

Li Zhenzhu; Zhang Jingfeng; Zhou Wei; Zheng Jianjun; Xia Yinshui

Originally published in Scientific reports · 2024-04-01

From the article

Selected source text. JournalOS removes markup and shortens excerpts where indicated. The full article provides the complete context.

Abstract

This study explored the application of generative pre-trained transformer (GPT) agents based on medical guidelines using large language model (LLM) technology for traumatic brain injury (TBI) rehabilitation-related questions. To assess the effectiveness of multiple agents (GPT-agents) created using GPT-4, a comparison was conducted using direct GPT-4 as the control group (GPT-4). The GPT-agents comprised multiple agents with distinct functions, including "Medical Guideline Classification", "Question Retrieval", "Matching Evaluation", "Intelligent Question Answering (QA)", and "Results Evaluation and Source Citation". Brain rehabilitation questions were selected from the doctor-patient Q&A database for assessment. The primary endpoint was a better answer. The secondary endpoints were accuracy, completeness, explainability, and empathy. Thirty questions were answered; overall GPT-agents took substantially longer and more words to respond than GPT-4 (time: 54.05 vs. 9.66 s, words: 371 vs. 57). However, GPT-agents provided superior answers in more cases compared to GPT-4 (66.7 vs. 33.3%). GPT-Agents surpassed GPT-4 in accuracy evaluation (3.8 ± 1.02 vs. 3.2 ± 0.96, p = 0.0234).

Discussion

In this study, medical guidelines20 and agents based on GPT-4 were used to answer questions related to TBI rehabilitation. This system automatically evaluates the correctness of the answers, simultaneously providing relevant content from the medical guidelines to enhance explainability. The evaluation revealed that the responses generated by the guideline-based GPT-agents performed better in terms of accuracy, explainability, and empathy than those obtained by directly querying GPT-4.

Methods

This study employed a cross-sectional, non-human subject research design. A flowchart of the study design is shown in Fig. 1. As this study did not involve human or animal participants, and ChatGPT/OpenAI could freely access Kaggle.com via the API, Ethical Committee Approval was not required.

Editorial reading guide

  • How did the authors define recovery, and when did they measure it?
  • Does the outcome reflect a laboratory measure, a functional change or participation in daily life?