Development and Reliability of a User-Friendly Multicenter Phenotyping Application for Hemorrhagic and Ischemic Stroke

Background Annotation and Image Markup on ClearCanvas Enriched Stroke–phenotyping Software (ACCESS) is a novel stand-alone computer software application that allows the creation of simple standardized annotations for reporting brain images of all stroke types. We developed the ACCESS application and determined its inter-rater and intra-rater reliability in the Stroke Investigative Research and Educational Network (SIREN) study to assess its suitability for multicenter studies. Methods One hundred randomly selected stroke imaging reports from 5 SIREN sites were re-evaluated by 4 trained independent raters to determine the inter-rater reliability of the ACCESS (version 12.0) software for stroke phenotyping. To determine intra-rater reliability, 6 raters reviewed the same cases previously reported by them after a month of interval. Ischemic stroke was classified using the Oxfordshire Community Stroke Project (OCSP), Trial of Org 10172 in Acute Stroke Treatment (TOAST), and Atherosclerosis, Small-vessel disease, Cardiac source, Other cause (ASCO) protocols, while hemorrhagic stroke was classified using the Structural lesion, Medication, Amyloid angiopathy, Systemic disease, Hypertensive angiopathy and Undetermined (SMASH-U) protocol in ACCESS. Agreement among raters was measured with Cohen's kappa statistics. Results For primary stroke type, inter-rater agreement was.98 (95% confidence interval [CI],.94-1.00), while intra-rater agreement was 1.00 (95% CI, 1.00). For OCSP subtypes, inter-rater agreement was.97 (95% CI,.92-1.00) for the partial anterior circulation infarcts,.92 (95% CI,.76-1.00) for the total anterior circulation infarcts, and excellent for both lacunar infarcts and posterior circulation infarcts. Intra-rater agreement was.97 (.90-1.00), while inter-rater agreement was.93 (95% CI,.84-1.00) for TOAST subtypes. Inter-rater agreement ranged between.78 (cardioembolic) and.91 (large artery atherosclerotic) for ASCO subtypes and was.80 (95% CI,.56-1.00) for SMASH-U subtypes. Conclusion The ACCESS application facilitates a concordant and reproducible classification of stroke subtypes by multiple investigators, making it suitable for clinical use and multicenter research.