Differential Diagnosis

googleJul 22, 20263 min readCurated summary

SymptomAI: Towards a conversational AI agent for everyday symptom assessment

SymptomAI explores whether conversational AI can conduct realistic symptom interviews and generate useful differential diagnoses outside curated medical vignettes. In a randomized national study of 13,917 participants, SymptomAI agents often performed as well as or better than clinician-generated differentials according to expert reviewers, particularly when they actively asked follow-up questions. The study also found that diagnoses associated with infectious illnesses corresponded with shifts in participants’ Fitbit biosignals, suggesting potential for large-scale health research. ## Moving Beyond Curated Medical Cases - Existing language-model evaluations often use detailed, synthetic, or highly structured patient vignettes. - Real patients may provide incomplete information, have varying medical literacy, or describe symptoms unpredictably during conversation. - SymptomAI was designed to test end-to-end symptom assessment in a more natural setting, while making clear that its outputs were research results rather than clinical diagnoses. ## National-Scale Study Design - 13,917 consenting participants were randomly assigned to one of five Gemini Flash 2.0 SymptomAI agents. - Participants described their symptoms, answered follow-up questions, received a differential diagnosis (DDx), and were given next-step recommendations. - Two weeks later, participants reported diagnoses received from healthcare providers. - Three board-certified clinicians reviewed the conversations, created their own differentials, and blindly ranked SymptomAI’s DDx against clinician-generated alternatives. ## SymptomAI Compared Favorably with Clinicians - Clinical reviewers preferred SymptomAI’s differential diagnosis over those from other clinicians in more than 50% of cases. - SymptomAI’s DDx was more likely to be ranked as the highest-quality option. - Using top-five accuracy—whether the eventual provider diagnosis appeared among five proposed diagnoses—reviewers found SymptomAI’s differentials accurate more often than the comparison clinician differentials. ## Follow-Up Questions Improved Accuracy - The study tested five interview strategies: - Dynamic Live and Dynamic Final agents could ask unrestricted follow-up questions. - Fixed Canonical and Flexible Canonical agents used standardized medical history questions. - The Base condition represented a user-led interaction with an unprompted language model. - Every agent-driven strategy significantly outperformed the Base condition. - The findings indicate that actively eliciting additional information is more effective than relying solely on what users initially choose to disclose. ## Strongest Results in Uncertain Cases - SymptomAI’s advantage over clinician baselines was greatest when clinicians expressed low confidence in their own differentials. - This suggests conversational AI may be especially useful as a second opinion or support tool in ambiguous cases, though the study does not establish that it can replace professional diagnosis. ## Connecting Diagnoses with Wearable Data - The researchers used SymptomAI’s diagnostic outputs as potential reference labels for analyzing population-scale physiological data. - Participants provided up to 30 days of Fitbit biometric data before their SymptomAI interaction. - Acute respiratory infection cases showed noticeable biosignal changes in the days leading up to symptom reporting. - These shifts appeared consistent with symptom onset and possible immune responses, although the provided text ends before presenting the full analysis. SymptomAI’s results support building conversational systems that ask structured follow-up questions and assist with differential diagnosis. Any practical deployment should retain clinician oversight, communicate uncertainty clearly, and treat AI-generated assessments as decision support rather than confirmed medical diagnoses.

Read original

Differential Diagnosis Gemini Conversational Ai Language Models+3

googleJun 12, 20263 min readCurated summary

Research into how AI can help users understand skin conditions

Google Research examined how AI tools can help non-experts understand skin conditions and decide what to do next. In a large study, AI substantially improved people’s ability to identify possible conditions, but it did not reliably improve the accuracy of recommended next steps. The research therefore argues that dermatology AI should be designed around human decision-making, safety, and clear guidance—not diagnosis alone. ## Why Dermatology AI Needs Human-Centered Research - More than half of adults use the Internet for health information, and about one-third use AI. - People often lack the medical vocabulary needed to search effectively—for example, searching for “red dots on legs” instead of “palpable purpura.” - Google Research has developed dermatology AI models, validated their generalization, and released datasets such as SCIN. - Earlier research found that online tools can improve condition recognition without necessarily helping people choose appropriate next steps. - The researchers emphasize studying how people interpret and act on AI-generated information. ## Large-Scale Evaluation of an AI Information Tool - A JAMA Dermatology study involved 2,345 participants reviewing de-identified skin-condition cases with images and structured medical histories. - Participants were assigned to one of three groups: - **Standard-search control:** Used familiar text-based search tools. - **AI group:** Used a prototype showing 3–7 AI-predicted conditions, textbook images, and information about symptoms and treatments. - **“Wizard of Oz” control:** Used the same interface, but with dermatologist-provided differential diagnoses presented as if generated by AI. - The AI interface increased participants’ willingness to name a condition: - More than 62% attempted a diagnosis with AI. - Only 41% did so using standard search. - Accuracy also improved: - AI users correctly identified a matching condition about 23% of the time. - Standard-search users achieved 8%. - The “perfect-prediction” interface reached 36%, showing that even accurate candidate lists did not make users nearly perfect. - AI users reported greater confidence, satisfaction, and satisfaction with the time spent searching. ## Identifying a Condition Does Not Guarantee Safe Action - The prototype intentionally avoided prescribing actions or making individualized diagnoses. - Treatment information was dermatologist-written and based on the condition name, rather than the severity or details of the specific case. - Choosing the right next step—such as home care, routine care, or urgent evaluation—remained difficult. - Next-step accuracy improved only slightly in the “Wizard of Oz” group, from 60% in the standard-search control to 63.5%. - The standard AI group showed no statistically significant improvement. - AI users were slightly more likely than control participants to recommend a less urgent action than dermatologists would: 30% versus 27%. - These findings show that identifying possible conditions is insufficient without stronger safety-oriented guidance. ## Studying Real Users and Diverse Communities - The researchers also conducted a qualitative study, published at ACM CHI, to examine how people use AI for their own active skin concerns. - The project partnered with Stanford’s Healthcare AI Applied Research Team and the Santa Clara Family Health Plan. - The community included many Medi-Cal users who rely on a healthcare safety net. - Researchers aimed to gather richer feedback than survey-based studies provide by observing real-world use. - Because participants spoke four primary languages, the application was translated into those languages, with multilingual volunteers or staff available to support communication. AI can make dermatology information easier to find and improve recognition of possible conditions, but it should not be treated as a substitute for professional judgment. Future tools should focus equally on urgency assessment, personalized context, uncertainty, and clear recommendations for when to seek medical care.

Read original

Differential Diagnosis AI Machine Learning Healthcare Ai+3

googleMar 11, 20263 min readCurated summary

Exploring the feasibility of conversational diagnostic AI in a real-world clinical study

The study evaluated Google’s conversational medical AI, AMIE, in a real-world primary care workflow rather than simulated cases. In a prospective, IRB-approved study at Beth Israel Deaconess Medical Center, AMIE conducted supervised pre-visit history-taking with 100 patients. Results suggested that supervised deployment was feasible and conversationally safe, while AMIE’s diagnostic and management-plan quality was broadly comparable to that of primary care physicians, with physicians performing better on practicality and cost effectiveness. ## Study Design and Clinical Workflow - Patients with new, non-emergency, episodic complaints used AMIE through a secure web link before an in-person or telehealth appointment. - A physician supervised each AI-patient interaction through live video and screen-sharing. - AMIE produced a transcript and summary for the patient’s primary care physician. - Independent clinical evaluators assessed: - The quality of the AMIE conversation - AMIE’s differential diagnoses - AMIE’s management plans - Comparable outputs from physicians - The study was prospective, single-center, single-arm, pre-registered, and IRB approved. ## Participants - 100 adults completed the AMIE interaction. - 98 attended their scheduled primary care appointments. - Participants represented varied ages, racial and ethnic groups, health literacy, technology literacy, and prior chatbot experience. - Compared with all 1,452 urgent care visits during the study period, participants tended to be younger, although the sample reflected the broader population’s female and white demographic skew. ## Safety Oversight - Human supervisors could stop an interaction if they observed: - Immediate risk of harm to the patient or others - Significant emotional distress related to the AI interaction - Potential clinical harm - A patient’s explicit request to end the session - No safety stops were required across the study. - The authors interpret this as evidence that supervised AMIE interactions were conversationally safe in this setting. ## Clinical Reasoning Performance - Three independent clinical evaluators reviewed each case using blinded, randomized assessments. - AMIE and physicians showed similar overall quality for: - Differential diagnoses - Management plans - Management-plan appropriateness and safety - Physicians performed better on the practicality and cost effectiveness of management plans. - AMIE’s differential-diagnosis accuracy was reported as high, including cases where the final diagnosis was confirmed through diagnostic testing. ## Patient and Clinician Experience - The study measured trust, perceptions, and acceptance among both patients and clinicians. - Patient trust in AI increased after interacting with AMIE. - Overall findings indicated that the system was well received within the supervised pre-visit workflow. The study supports cautious, supervised testing of conversational diagnostic AI in clinical environments. It does not establish that AMIE can independently replace clinicians; rather, it suggests that pre-visit information gathering may be a practical early use case, provided rigorous oversight, safety protocols, and further evaluation in larger and more diverse settings.

Read original

Differential Diagnosis Conversational Ai Chatbot Telehealth+4