(1332) Comparison Between Artificial Intelligence Chatbots and Community Pharmacists Regarding Quality of Product Selection for Over-the-counter Cough and Cold Medications

Objectives: The objective of this study is to compare artificial intelligence (AI) chatbots to that of community pharmacists for the quality of response scores regarding product selection for over the counter (OTC) cough and cold medications. Community pharmacists play an important role in recommending self-care items for patients based upon their symptom presentation. However, the ease and accessibility of self-care items can allow for patients to completely by-pass community pharmacist involvement in self-care selections. Through technology, information is more readily available than ever. Through this access to technology, patients often turn to the internet to self-diagnose and choose the best course of action for their “diagnosis.” Sometimes the information that patients are using to make decisions for self-care may not be accurate and/or reliable. Finding ways to enhance the utility of technology as it expands is vital. There have been studies assessing the accuracy of AI chatbots in answering medical questions within various practice settings such as: oncology, ophthalmology, breast reconstruction, and bariatric surgery. No current data exists comparing AI chatbot responses to community pharmacists for OTC product selection regarding cough and cold medications.

Methods: This study is a prospective cohort, utilizing a test instrument composed of six open-ended questions related to OTC product selection for cough and cold items. An answer key will be created using the APhA Handbook of Nonprescription Drugs. Answers will be classified as “Preferred,” “Acceptable, but Not Preferred,” “Not Preferred,” and “May Cause Harm.” The instrument and answer key will be piloted on three community pharmacists to complete before the study is rolled out. The test will be administered to community pharmacists who: have at least one year of experience as a licensed community pharmacy practitioner in the state of North Carolina, practice site in Raleigh, NC at major community pharmacy chains (i.e. CVS, Walgreens, and Walmart) or independent community pharmacy, and willingness to answer verbally administered open-ended questions regarding product selection for adult OTC cough and cold items. The test will also be entered into three predetermined AI chatbots. Artificial intelligence chatbots were excluded from consideration if: they are not able to answer medical questions, they are specifically used for business development, they are specifically used for customer service-related tasks, and those with search engine-related web browser similarities such as images, videos, and hyperlinks. Response scores will be based on the following predetermined grading scale: 2 points for “Preferred”, 1 point for “Acceptable but not Preferred”, 0 points for “Not preferred”, and -1 point for “May Cause Harm”. The primary endpoint of this study is the overall test response scores regarding OTC product selection for cough and cold medications between AI chatbots and community pharmacists. Utilizing JMP 17 Pro, a two-sample t-test will be used to analyze the primary endpoint for this study.

Results: N/A

Conclusions: N/A