Validation of a novel tool for assessing newborn resuscitation skills among birth attendants trained by the Helping Babies Breathe program

Objective To validate a simplified objective structured clinical examination (OSCE) tool for evaluating the competency of birth attendants in low-resource countries who have been trained in neonatal resuscitation by the Helping Babies Breathe (HBB) program. Methods A prospective cross-sectional study of the OSCE tool was conducted among trained birth attendants working at dispensaries, health centers, or hospitals in five regions of Tanzania between October 1, 2013, and May 1, 2014. A 13-item checklist was used to assess clinical competency in a simulated newborn resuscitation scenario. The OSCE tool was simultaneously administered by HBB trainers and experienced external evaluators. Paired results were compared using the Cohen κ value to measure inter-rater reliability. Participant performance was rated by health cadre, region, and facility type. Results Inter-rater reliability was moderate (κ = 0.41-0.60) or substantial (κ = 0.61-0.80) for eight of the OSCE items; agreement was fair (κ = 0.21-0.41) for the remaining five items. The best OSCE performances were recorded among nurses and providers from facilities with high annual birth volumes. Conclusion The simplified OSCE tool could facilitate efficient implementation of national-level HBB programs. Limitations in inter-rater reliability might be improved through additional training.