Applying a governance barometer to vaccine delivery systems: Lessons from a rural district of Pakistan

Introduction: Weak vaccine delivery systems in countries off-track for routine immunization targets, need in-depth evidence on system level barriers to be critically resourced and monitored. We applied a Balanced Score Card (BSC) approach in a rural underserved district of Pakistan to (i) identify critical areas needing support in the government vaccine delivery system; and (ii) for benchmarking improvements in the vaccine delivery system. Methodology: BSC was developed drawing on desk review, government consultations and field testing. 45 immunization indicators were finalized across 8 domains: human resource; vaccine supply; safe vaccination practice; cold chain maintenance; outreach preparedness; records & supervision; verifiable vaccination volume; and client communication. Data were collected through health facility assessments, client exit interviews and household vaccination assessment. A composite score was calculated for each domain and banded into unsatisfactory, borderline and satisfactory categories. 5 lowest ranking domains were targeted for 2 years of health systems strengthening (HSS) interventions. Post-intervention assessment tracked progress. Results: The district obtained a cumulative score of 51% (unsatisfactory) at pre-intervention and improved to 82% (satisfactory) at post-intervention. At pre-intervention, 4 domains scored satisfactory and 4 scored unsatisfactory. Unsatisfactory scores were received for: outreach preparedness; records & supervision; verifiable vaccination volume; and client communication. Post intervention 6 of 8 domains scored satisfactory and 2 moved from unsatisfactory to borderline. Highest percentage point (pp) improvements were seen in outreach preparedness (53 pp, p = 0.01), EPI supervision (52 pp, p = 0.01) and verified vaccination volume (46 pp, p = 0.02). 3 domains that were not intervened through HSS interventions had minimal change in scoring - cold chain maintenance (6 pp), safe vaccination practice (12 pp) and vaccine supply (11 pp). Conclusion: BSC served to prioritize interventions towards critical unmet needs for vaccine delivery in the district health system and particularly helped to improve outreach preparedness, EPI supervision and verified vaccination volume.