(1089) Impact of Mastery-based Skills Assessments in an Advanced Pharmacy Practice Experience (APPE) Readiness Course

Objectives: The primary objective of this study is to assess the change in PharmD student performance on four practice-based Objective Structured Clinical Exams (OSCEs) by comparing the average number of OSCEs passed with a satisfactory score before and after implementation of mastery-based practice assessments. Secondary objectives include comparing: (1) satisfactory rates on each of the four individual OSCE exams before and after implementation of the practice assessments (2) OSCE satisfactory rates among students who did and did not demonstrate mastery before the OSCEs and (3) OSCE satisfactory rates among students who did and did not participate in the optional practice assessments. Student perception of the intervention will also be measured.

Mastery-based learning is a pedagogical approach that allows students to demonstrate competency prior to summative exams through consistent performance on formative assessments. This method has been shown to improve student achievement and attitude toward learning. While previous literature is focused on lecture-based courses, some studies have found a mastery-based approach is also beneficial in skills-based assessments, specifically within healthcare education. This study will evaluate the impact of a mastery-based approach for skills development within a Doctor of Pharmacy curriculum.

Methods: At The Ohio State University College of Pharmacy, students complete four skills based OSCEs at the end of the third professional year as part of an APPE readiness course. The OSCEs include Outpatient Prescription Checking, Intravenous Product Checking, a Prescription Drug Monitoring Program Activity, and a Therapeutic Drug Monitoring Activity. To pass each OSCE station, students must avoid making a major error according to predetermined standards on each activity.

In Spring 2023, an educational intervention was implemented to allow students to demonstrate mastery prior to the OSCEs through four optional practice sessions. Each practice session included four assessments that were identical in format to the OSCEs but contained different scenarios. Students who performed satisfactorily on the same practice station on two separate occasions were deemed to have achieved mastery. To ensure appropriate designation of mastery, the practice assessment satisfactory threshold (two performances with no major or minor errors) was more stringent than the OSCE satisfactory threshold (one performance with no major errors). Students who achieved mastery on any practice assessment were granted a conditional satisfactory score on the corresponding OSCE. To compare outcomes for the study objectives, students who were granted a conditional satisfactory score were required to complete the corresponding OSCE and received a satisfactory if they made a genuine attempt regardless of errors.

For the primary objective, this study will utilize a retrospective approach to analyze the difference in average number of satisfactory OSCE scores between two cohorts (2022 and 2023) before and after implementation of the intervention. Descriptive and inferential statistics will be used to analyze the primary outcome. To assess the secondary objectives, satisfactory rates between cohorts for each individual OSCE will be compared. Additionally, data from the 2023 cohort will be analyzed to compare the differences in satisfactory OSCE performance among students who did and did not demonstrate mastery as well as the differences in satisfactory OSCE performance among students who did and did not participate in the practice sessions. Student perception of the intervention will be assessed through descriptive analysis of course evaluation data.

Results: In progress.

Conclusions: In progress.