To assess the effectiveness of adding a simulated anticoagulation clinic practical examination for formal assessment of PharmD students skills.
A practical examination requiring students to review a mock medical record and role-play a follow-up anticoagulation clinic visit with a standardized patient was designed. Students assessed the patients vital signs, laboratory values, and subjective complaints during an in-depth interview and documented clinical recommendations in a progress note.
Student feedback indicated that the simulated clinic was pertinent to preparation for experiential rotations. Ninety-five percent of the students recommended that it be continued as a required component of the course.
The simulated ambulatory care clinic exercise reinforces principles of anticoagulation management as well as the assessment of clinical data, performance of a patient interview, and written documentation of recommendations.
The broad impact of chronic disease has afforded pharmacists an opportunity to influence clinical and economic outcomes across many ambulatory care settings, including community pharmacies and pharmacist-managed clinics. Outcomes of pharmacist interventions in the areas of diabetes, asthma, hyperlipidemia, cardiac risk reduction, and anticoagulation are widely documented in the literature.1-3 Regardless of setting or targeted disease, successful disease management requires a core set of skills beyond therapeutic knowledge, including patient interviewing, physical assessment, counseling, and written documentation.3 The Disease Management course at Midwestern University College of Pharmacy-Glendale is designed to emphasize the knowledge and skills necessary for this practice focus through the development and monitoring of patient care plans for chronic cardiac, pulmonary, and endocrine conditions. It is offered during the final 2 didactic quarters of the curriculum and expands on a foundational course sequence entitled “Integrated Sequence,” which integrates medicinal chemistry, pharmacology, pathophysiology, and therapeutics.
The first quarter involves the instruction and formal assessment of vital signs technique, patient interviewing skills, and written documentation of recommendations through separate assignments and workshops. These skills are then integrated into a capstone practical examination administered during the second quarter through a simulated pharmacist-managed clinic focused on anticoagulation. Anticoagulation was chosen as the therapeutic context due to the well defined and documented role of pharmacists in this setting, and the ability to integrate additional cardiovascular medical conditions, such as hypertension and hyperlipidemia, into the scenario.
The simulated clinic was designed to expose students to information and decision making in a realistic clinical context, including a medical record, private examination room, and standardized patient. Standardized patients are integral to the simulation as they are trained to provide a consistent medical history and presentation of current symptoms.4 The use of standardized patients in various pharmacy communication courses as well as postgraduate continuing education and training programs has been documented in the literature.5-9
In order to more accurately assess PharmD students learning in disease management, in the fall of 2004, a practical examination involving interviewing simulated patients in a clinical setting was developed. Development of the practical examination required assembly of a mock medical record and patient scripts and training of standardized patients. A pool of standardized patients consisting of members of the lay public was recruited and participation coordinated for Midwestern University by a centralized program director for use in the physician assistant, osteopathic medicine, physical therapy, and pharmacy programs. Payment for the standardized patients was provided through the Department of Pharmacy Practice budget. The patients identified for this practical examination received foundational training by the centralized program director in the use of the evaluation tool and vital signs assessments. The course coordinator then provided a 2-hour training session for the patients specific to the role-playing scripts. The scripts included the medical, family, and social history of the mock patient, physical characteristics, and medication lists. Additionally, a script of preplanned answers to anticoagulation-related interview questions was provided to address signs or symptoms of bleeding or clotting and dietary intake of vitamin K. The training sessions incorporated role-playing with the course instructors to identify and correct areas of potential inconsistency during the interview. The completion of this process allowed for implementation of the capstone practical examination during the 10-week fall quarter offering of Disease Management II.
The practical examination was made up of a series of required sessions, including two case-based lectures, a preassignment and a workshop. The 2 case-based lectures focused on anticoagulation services for atrial fibrillation and deep vein thrombosis. Because the foundational therapeutics information was previously introduced in the Integrated Sequence, a focused review of drug-drug, drug-disease, and drug-food interactions with warfarin was provided, and the remainder of the lecture involved the completion of 2 complex patient cases. The case scenarios were designed to follow typical anticoagulation cases encountered in a clinic, and included discussion of new and follow-up patient interview formats, application of warfarin dosage adjustment algorithms, and documentation of findings and recommendations in a SOAP note. After the completion of the lectures, the preassignment involved review of a mock medical record of an anticoagulation patient, which contained progress notes, laboratory values, and consultation notes for the previous year. Each student had 1 week to review the medical record and make plans for his/her “visit” with the patient at the assigned appointment time.
The workshop was divided into 2 sections to accommodate all students. Seven “patient appointments” were conducted simultaneously over a 4-hour period for each section. Students participated in 2 phases, which together took approximately 90 minutes to complete. First, the students arrived for their scheduled appointment with the standardized patient and received an updated set of laboratory values including International Normalized Ratio (INR), complete blood count, electrolytes, and fasting lipid panel, and vital signs for their patient. Students were given approximately 10 minutes to review the updated objective data and make adjustments to their interview plans as appropriate. The students then proceeded to the patients room and were given 15 minutes to interview the patient and assess blood pressure and pulse. The interview was videotaped and was for information gathering only. Due to time restrictions for individual appointments, the students were instructed not to provide immediate recommendations to the patient verbally, as this would be done in writing in the SOAP note. However, they were instructed to answer any questions that the patient had. Prior to leaving the appointment, the standardized patient gave each student immediate feedback regarding the students communication style, organization of the interview, and physical comfort during the vital signs assessment. Content was not evaluated by the patient, and there was no discussion of information “missed” by the student due to an incomplete interview. The components of the evaluation are listed in Table . The standardized patients were encouraged to provide specific examples to reinforce the communication principle.
Once the verbal evaluation by the standardized patient was complete, the student then proceeded to a lecture hall to write a SOAP note documenting the findings of the appointment and provide recommendations for dosage adjustments, monitoring, and counseling. In their assessment of problems, the students utilized mock vital signs rather than the blood pressure and pulse of their particular patient for their determination of the medication plan. This standardization was necessary to simplify/streamline grading. Students were encouraged to utilize any references they brought with them; however, no collaboration was allowed during any point during the workshop process.
The students score was based entirely on the written documentation in the SOAP note, which reflected both the students preparation and the depth and breadth of the patient interview. The exam score targeted the completeness of the subjective and objective sections, identification of drug-related problems in the assessment, and complete and correct dosing recommendations, counseling, and monitoring parameters in the plan. The interviews were videotaped for the purpose of documentation allowing for the interview to be reviewed if a student challenged the assigned score. The students interview skills or techniques used in measuring vital signs were not reviewed during this examination since these skills had been evaluated multiple times during individual student assessments conducted in the first quarter of the course.
The therapeutic problems incorporated into each script varied for each section. The students were expected to assess the management of atrial fibrillation, hypertension, and hyperlipidemia using the updated objective data and subjective information identified during the workshop. Examples of therapeutic problems from past practical examinations are described in Table .
In 2005, a formal assessment of the students perceptions of the practical examination was performed. Students were asked to voluntarily complete a survey measuring their confidence in their patient assessment skills and satisfaction with the use of standardized patients.
From 2004 to 2005, approximately 250 students completed the simulated clinic model of the examination in the Disease Management course sequence. Students scored highly on the practical examination, with a mean of 90% for both years. The evaluations completed by the standardized patients indicated that over 90% of the students met each of the criteria measured in Table . The most common area of improvement identified by the patients was related to the flow and organization of the interview questions. Because most of the students used predeveloped interview forms, questions were sometimes asked in a robotic fashion that allowed for little flexibility or “natural” flow to the interview. A related concern was in regards to listening and responding to the patients answers. In some cases, students were not modifying follow-up questions based on the patients answers, which indicated difficulty in immediate processing of new information provided by the patient.
Anecdotally, students reported that the practical examination was an important stepping stone to their experiential rotations. Forty-six percent (56/123) of the students enrolled in the course participated in a voluntary survey, which was approved by the Midwestern University Institutional Review Board. The results of the survey, which consisted of statements evaluated on a 5-point Likert scale, are summarized in Table . Overall, the assignment was well-received. The use of standardized patients was overwhelmingly preferred over role-playing with faculty members or fellow students. However, only 55% of respondents indicated that the feedback from the standardized patients regarding their interview style was useful. The reason cited most often for this observation was the need for specific recommendations for improvement rather than the generalized comments more commonly received. Most students “agreed” or “strongly agreed” that the practical examination should be continued in future course offerings.
Several open response questions were also included on the survey instrument. When asked to identify what they liked most about the practical examination, many students expressed appreciation for the “realistic” setting and the ability to interview a patient rather than a peer or faculty member. One student identified that it was refreshing to be examined in a “non-multiple choice format” for those who do not excel at this type of test taking. When asked to identify the most important item learned during the experience, students listed the importance of asking the patient open-ended questions, value of preparing for the patient interview, the accuracy of the blood pressure technique used, and the importance of listening during the patient interview. When asked for recommendations for change, several students stated that after informally comparing experiences with peers following the workshop, they concluded that the feedback provided by the standardized patients needed further “standardization.” Their comments suggested that the depth of feedback was not consistent among the various standardized patients. Multiple students recommended that the examination be administered twice during the quarter to allow opportunity for improvement.
The practical examination using a standardized patient has been well received and is advantageous for several reasons. First, it allows faculty members to integrate several clinical skills into one assessment. Second, it provides an opportunity to reinforce clinical concepts of multiple chronic disease states taught across the curriculum. Finally, it provides students with a bridge between paper examinations and experiential rotations – one of the primary benefits from the students perspective as well.
The format is not without challenges, however, and improvements are made continuously in response to feedback from faculty members, standardized patients, and students. For example, during the first year, the scoring of the practical incorporated the standardized patients evaluation and comments. Due to the subjective nature of several questions, concern was raised over the ability to truly “standardize” the scoring. It was determined that the scores during the second year would be based entirely on the written documentation in the SOAP note. However, the verbal review of the evaluation by the standardized patient was to be continued. Additionally, it was concerning that only 55% of the survey respondents “agreed” or “strongly agreed” that the feedback from the patient was helpful for improvement. In response, training of the standardized patients during the second year incorporated increased role playing of the verbal feedback given to students and further discussion of the evaluation tool. This remains a high priority for training for future offerings, as it appears that this is the area of most difficulty in the standardization process. The evaluation tool is undergoing revision to utilize a more structured rubric that can be completed within the course of the interview and still allow for immediate feedback for the student.
The current scoring format does not allow for individual verbal feedback to students from a faculty member. Faculty reviews are performed for the individual skills assessments in the previous quarter, but time constraints limit this for the practical examination. Students are offered a voluntary opportunity to review and discuss the practical examination with a faculty member if desired. This involves a formal review of the videotaped interview and discussion of the SOAP note with recommendations for improvement. Approximately 30% of the class took advantage of this opportunity when it was offered most recently. Those who participated reported that it allowed them to identify specific areas for focus that were not necessarily identified by the patients during the interview process and they recommended that this be required of all student participants. Many students received feedback on how to make the interview flow more naturally by modifying questions from their preplanned script.
An additional limitation is related to the assessment of students perceptions of the activity prior to their experiential rotations. A post-experiential rotation survey would be the most valid measurement of the preparation provided by the activity and this will likely be incorporated into future course offerings.
This simulated clinic model could be modified for use at other institutions to teach and assess disease management skills, and could be broadened to other chronic disease states, such as diabetes or hyperlipidemia. Incorporation of this process in a pharmacy curriculum requires consideration of the resources required, including access to standardized patients, faculty time for training and development of the medical record and interview scripts, space allowing for individual consultations and videotaping, and grading assistance.
The use of a simulated clinic model provides an effective setting for a “capstone” assessment of skills necessary for delivery of care in pharmacist-managed clinics, such as the interpretation of data in a medical record, performance of a patient interview and vital signs assessment, and written documentation of recommendations. The use of an anticoagulation clinic setting provides further reinforcement of the principles of the management of common chronic cardiac conditions and exposes students to a viable practice environment for ambulatory care pharmacists.
The author would like to recognize the contributions of Dana Singla, PharmD, as the previous co-coordinator of the Disease Management course sequence.
In order to more accurately assess PharmD students learning in disease management, in the fall of 2004, a practical examination involving interviewing simulated patients in a clinical setting was developed. Development of the practical examination required assembly of a mock medical record and patient scripts and training of standardized patients. A pool of standardized patients consisting of members of the lay public was recruited and participation coordinated for Midwestern University by a centralized program director for use in the physician assistant, osteopathic medicine, physical therapy, and pharmacy programs. Payment for the standardized patients was provided through the Department of Pharmacy Practice budget. The patients identified for this practical examination received foundational training by the centralized program director in the use of the evaluation tool and vital signs assessments. The course coordinator then provided a 2-hour training session for the patients specific to the role-playing scripts. The scripts included the medical, family, and social history of the mock patient, physical characteristics, and medication lists. Additionally, a script of preplanned answers to anticoagulation-related interview questions was provided to address signs or symptoms of bleeding or clotting and dietary intake of vitamin K. The training sessions incorporated role-playing with the course instructors to identify and correct areas of potential inconsistency during the interview. The completion of this process allowed for implementation of the capstone practical examination during the 10-week fall quarter offering of Disease Management II.
The workshop was divided into 2 sections to accommodate all students. Seven “patient appointments” were conducted simultaneously over a 4-hour period for each section. Students participated in 2 phases, which together took approximately 90 minutes to complete. First, the students arrived for their scheduled appointment with the standardized patient and received an updated set of laboratory values including International Normalized Ratio (INR), complete blood count, electrolytes, and fasting lipid panel, and vital signs for their patient. Students were given approximately 10 minutes to review the updated objective data and make adjustments to their interview plans as appropriate. The students then proceeded to the patients room and were given 15 minutes to interview the patient and assess blood pressure and pulse. The interview was videotaped and was for information gathering only. Due to time restrictions for individual appointments, the students were instructed not to provide immediate recommendations to the patient verbally, as this would be done in writing in the SOAP note. However, they were instructed to answer any questions that the patient had. Prior to leaving the appointment, the standardized patient gave each student immediate feedback regarding the students communication style, organization of the interview, and physical comfort during the vital signs assessment. Content was not evaluated by the patient, and there was no discussion of information “missed” by the student due to an incomplete interview. The components of the evaluation are listed in Table . The standardized patients were encouraged to provide specific examples to reinforce the communication principle.
The use of a simulated clinic model provides an effective setting for a “capstone” assessment of skills necessary for delivery of care in pharmacist-managed clinics, such as the interpretation of data in a medical record, performance of a patient interview and vital signs assessment, and written documentation of recommendations. The use of an anticoagulation clinic setting provides further reinforcement of the principles of the management of common chronic cardiac conditions and exposes students to a viable practice environment for ambulatory care pharmacists.
In 2005, a formal assessment of the students perceptions of the practical examination was performed. Students were asked to voluntarily complete a survey measuring their confidence in their patient assessment skills and satisfaction with the use of standardized patients.
The practical examination using a standardized patient has been well received and is advantageous for several reasons. First, it allows faculty members to integrate several clinical skills into one assessment. Second, it provides an opportunity to reinforce clinical concepts of multiple chronic disease states taught across the curriculum. Finally, it provides students with a bridge between paper examinations and experiential rotations – one of the primary benefits from the students perspective as well.