Development and Pilot Testing of a Novel Tool for Evaluating Practical Skills in Hematopathology Residents in Pakistan

Bushra Moiz, Syeda Kauser Ali, Anila Rashid, Muhammad Shariq, Farheen Karim

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)

Abstract

Background: The mini-Clinical Evaluation Exercise (mini-CEX) and direct observation of procedural skills (DOPS) are reliable tools for work-based assessment of medical trainees. Tools of this type do not yet exist for evaluation of practical laboratory skills of pathology residents. Objective: We developed and piloted a 9-item instrument for direct observation of laboratory skills (DOLS). Methods: We used the DOLS tool with 10 hematopathology residents (PGY-1 to PGY-5) from Aga Khan University. Each resident was evaluated by 3 faculty members in the laboratory during 4 separate encounters using the DOLS instrument. We assessed construct validity, interrater reliability and G coefficient, feasibility of using DOLS, and learner satisfaction. Results: A total of 120 encounters were observed with a mean score (±1 SD) of 56.7% (±12.44). Assessment scores moderately correlated with the number of laboratory procedures previously performed by participants (r = 0.658 and 0.641; P = .0001) and with PGY level. Interrater reliability ranged between 0.47 and 0.96. Cohen's d was 1.64. Residents accounted for a large component of estimated variance (73%), suggesting DOLS can differentiate residents' laboratory skills; variance associated with assessors was small (0.01%). Residents reported being satisfied with the tool. Mean time (±1 SD) taken for observing and feedback was 17.89 ± 5.89 minutes. Conclusions: The new DOLS instrument could provide reliable scores for observing laboratory skills. Residents were satisfied with the tool, and rating times make the tool feasible for formative assessments.

Original languageEnglish
Pages (from-to)177-180
Number of pages4
JournalJournal of graduate medical education
Volume11
Issue number4
DOIs
Publication statusPublished - 1 Aug 2019

Fingerprint

Dive into the research topics of 'Development and Pilot Testing of a Novel Tool for Evaluating Practical Skills in Hematopathology Residents in Pakistan'. Together they form a unique fingerprint.

Cite this