Between Knowledge and Care: A Mixed-Methods Evaluation of Generative AI for T2DM Self-Management from Patient and Physician Perspectives
Abstract
Generative AI is increasingly used for everyday health guidance, yet its clinical appropriateness in chronic disease contexts remains poorly understood. This paper presents a two-part mixed-methods study on Type 2 Diabetes Mellitus (T2DM), examining how patients and physicians assess AI-generated health information. Study~1 analyzes 784 participant reported patient queries to characterize seven informational need categories and develops a structured five dimensional physician rating rubric informed by patient query categories and clinician priorities (Accuracy, Safety, Clarity, Integrity, Action Orientation). Study~2 engages seven physicians scoring responses from four AI models and discussing evaluative reasoning through in-depth interviews. Models perform well on factual explanation and lifestyle guidance but consistently underperform on medication reasoning and emotional support. Two analytic concepts emerge from the data. The pre-visit primer frames AI as preparation for clinical encounters rather than as a replacement for physicians. The fluency illusion describes how polished language may convey epistemic authority that the clinical content does not support. Patients and physicians converged on three shared limitations (role boundaries, emotional inadequacy, personalization gaps) while diverging in evaluative emphasis, which informed four design directions, task-aware orchestration, risk-aware fallback, dynamic personalization, and emotionally attuned interaction.
Turn this paper into a full lesson
ArcXiv compiles a staged curriculum from this paper: 8-12 lessons across beginner → advanced, synthesised section guides, visuals, flashcards, a quiz, exercises, and on-demand deep dives per section. Grounded in the abstract, never invented.