Almanac - Insights and Applications for the Healthcare CPD Community
Powered by
Alliance for Continuing Education in the Health Professions
  • Education
  • Outcomes
  • Leadership
  • Podcasts
  • Industry News
AI Bot ChatGPT Passes U.S. Medical Licensing Exams Without Cramming — Unlike Students
Tuesday, January 31, 2023

AI Bot ChatGPT Passes U.S. Medical Licensing Exams Without Cramming — Unlike Students

By: Medscape

Medscape (01/26/23) Ault, Alicia

Researchers at Ansible Health found that OpenAI's ChatGPT chatbot can pass parts of the U.S. Medical Licensing Examination (USMLE). The bot's algorithms are trained to forecast the probability of a given word sequence based  on the preceding words' context. The authors initially investigated its ability to aggregate all communications about housebound patients with chronic lung disease by asking it multiple choice questions. They presented a theoretical patient's signs and symptoms and asked ChatGPT to consider underlying causes or diagnoses, then refreshed and asked it to answer a sampling of USMLE questions. After another refresh the chatbot was asked to justify the correct and incorrect answers. ChatGPT's accuracy for the first step ranged from 43% — when "indeterminate" answers were added to the analysis — to 68%, when they were excluded. Its pass rate was 51%-58% in the second step and 56%-62% in the third step. Asking ChatGPT questions verbatim yielded accuracy of 36%-55% for Step 1, 57%-59% for Step 2,and 55%-61% for Step 3. Response justification produced 40%-62% accuracy for Step 1, 49%-51% for Step 2, and 60%-65% for Step 3. The results prompted the researchers to have ChatGPT co-author a preprint paper that has been posted on medRxiv. Ansible's Victor Tseng, MD, said ChatGPT composed the abstract and results sections "with minimal prompting and largely cosmetic adjustments from the human co-authors," in addition to significantly contributing to the introduction and methods sections. The findings are set to be published in a peer-reviewed journal soon.

Read More

 

Keywords:   

Related Articles

Bridging Competency Gaps for Newly Graduated Nurses Through Micro-Credentials: An Interpretative Descriptive Qualitative Study
industry news
Bridging Competency Gaps for Newly Graduated Nurses Through Micro-Credentials: An Interpretative Descriptive Qualitative Study

By: BMC Medical Education

Nursing Preceptor Development Using a Mobile Application
industry news
Nursing Preceptor Development Using a Mobile Application

By: Nursing Education Perspective

2023 Survey Shows Increased Investment in Medical Education and Rise of Omnichannel HCP Engagement Strategies
industry news
2023 Survey Shows Increased Investment in Medical Education and Rise of Omnichannel HCP Engagement Strategies

By: GeneOnline

Rethinking the Impact of Single Continuing Educational Activities: Navigating Complexity Through Insights from Real-World Evaluation and Contemporary Literature
industry news
Rethinking the Impact of Single Continuing Educational Activities: Navigating Complexity Through Insights from Real-World Evaluation and Contemporary Literature

By: Journal of CME

Alliance for Continuing Education in the Health Professions
2001 K Street NW, 3rd Floor North, Washington, DC 2006
P: (202) 367-1151 | F: (202) 367-2151 | E: acehp@acehp.org
Contact Us | Privacy Policy | Disclaimer | About
© Alliance for Continuing Education in the Health Profession
Login
Search