Document Type

Presentation

Publication Date

4-10-2026

Faculty Mentor

Michelle Woodbury

Abstract

Objective

Latino populations have some of the highest rates of stroke (Simonetto et al., 2023) yet are utilizing in-person healthcare less frequently due to heightened fears and existing barriers to access (Vargas et al., 2019) (Silberberg, M., et al., 2018). Therefore, Latino stroke survivors may be turning to AI for stroke-related healthcare advice. However, current AI chatbots are primarily trained on English-language data (Park, 2024), which may introduce language bias when queried in other languages. The purpose of this project was to evaluate whether ChatGPT can serve as a reliable health information tool for Spanish-speaking individuals by comparing the accuracy of responses to stroke-related questions in English versus Spanish.

Methods:

A total of 32 English stroke-related questions were developed. An expert panel refined these to 14 questions based on relevance and clarity. Spanish translations were validated for readability and accuracy. Each question was entered into ChatGPT in English and Spanish, and responses were recorded in REDCap. Panelists evaluated each response using a standardized 1 to 5 accuracy scale, with a score of 2 or 3 indicating potentially harmful information. Median scores were used to assess accuracy.

Results 

The median accuracy was 5 for English responses and 4 for Spanish responses, which indicates an overall higher accuracy in English. Spanish responses had a greater variability in level of accuracy and were more frequently rated as potentially harmful. Approximately 10.7% of Spanish responses were marked as potentially harmful by at least one panelist, compared to 3.6% of English responses.

Conclusion

The results suggest that overall responses in Spanish are less accurate and may contain more harmful information than responses in English. While ChatGPT shows potential to be used as a health information tool in English, the accuracy and safety concerns limits its reliability to be used as such for Spanish speakers.

References

Park, S. (2024). AI Chatbots and Linguistic Injustice. Journal of Universal Language, 25(1), 99–119. https://doi.org/10.22425/jul.2024.25.1.99 

Simonetto, M., Sheth, K. N., Ziai, W. C., Iadecola, C., Zhang, C., & Murthy, S. B. (2023). Racial and Ethnic Differences in The Risk of Ischemic Stroke after Non-Traumatic Intracerebral Hemorrhage. Stroke, 54(9), 2401–2408. https://doi.org/10.1161/STROKEAHA.123.043160

Silberberg, M., Goldstein, L. B., Weaver, S., & Blue, C. (2018). Increasing Stroke Knowledge and Decreasing Stroke Risk in a Latino Immigrant Population. Journal of Immigrant and Minority Health, 20(6), 1490–1499. https://doi.org/10.1007/s10903-018-0690-0 

Vargas, E. D., Juárez, M., Sanchez, G. R., & Livaudais, M. (2019). Latinos’ Connections to Immigrants: How Knowing a Deportee Impacts Latino Health. Journal of Ethnic and Migration Studies, 45(15), 2971–2988. https://doi.org/10.1080/1369183x.2018.1447365

Download

Share

COinS