Skip to main content

Table 5 IRR between the two abstractors, and validity against the “gold standard” for select variables, (N=30)

From: The University of Texas Houston Stroke Registry (UTHSR): implementation of enhanced data quality assurance procedures improves data quality

Categorical variables Dichotomous variables IRR = Kappa (95% LCL)a Abstractor 1 vs. “gold standard” Abstractor 2 vs. “gold standard”
   N Abstractor 1 vs. Abstractor 2 Bias Index   N Agreement (%) Sensitivity (%) Specificity (%) Bias Index N Agreement (%) Sensitivity (%) Specificity (%) Bias Index
Stroke type Infarct 30 1.00 0   30 100 100 100 0 30 100 100 100 0
ICH 30 1.00 0   30 100 100 100 0 30 100 100 100 0
TIA 30 0.65 (0.02) −0.03   30 100 100 100 0 30 96.7 100 96.6 0.03
tPA therapy Yes 30 1.00 0   30 100 100 100 0 30 100 100 100 0
Symptomatic hemorrhage Yes 30 1.00 0   30 100 100 100 0 30 100 100 100 0
Disposition Death 30 1.00 0   30 100 100 100 0 30 100 100 100 0
Home 30 1.00 0   30 93.3 100 91.3 0.07 30 93.3 100 91.3 0.07
Inpatient rehabilitation 30 1.00 0   30 100 100 100 0 30 100 100 100 0
Skilled nursing facility 30 1.00 0   30 96.7 100 96.6 0.03 30 96.7 100 96.6 0.03
Initial presentation MHH 30 0.92 (0.76) 0.03   30 100 100 100 0 30 96.7 95.4 100 −0.03
Continuous variables   IRR = ICC d (95%LCL) a Abstractor 1 vs. “gold standard” Abstractor 2 vs. “gold standard”
  N Abstractor 1 vs. Abstractor 2 MD c (95%CI) b N Agreement (%) Correlation (95%CI) b MD c (95%CI) b N Agreement (%) Correlation (95%CI) b MD c (95%CI) b
Age (years) 30 1.00 0 (0, 0) 30 96.7 0.83 (0.65, 0.91) 1.4 (−1.5, 4.4) 30 96.7 0.83 (0.65, 0.91) 1.4 (−1.5, 4.4)
Onset time (minutes) 16 1.00 4.7 (−14.5, 5.3) 16 93.8 1.00 9.4 (−10.6, -29.4) 16 87.5 1.00 14.1 (−7.7, 35.8)
Arrival time (minutes) 29 1.00 0 (0, 0) 30 86.7 1.00 −0.2 (−1.9, 1.5) 29 89.7 1.00 −0.2 (−2.1, 1.6)
tPA time (minutes) 7 −0.48 (−0.89) NR 7 100 1.00 0 (0, 0) 7 57.1 −0.49 (−0.90, 0.45) NR
INR 25 0.98 (0.96) 0.02 (−0.02, 0.06) 25 80 0.98 (0.97, 0.99) −0.01 (−0.04, 0.02) 26 73.1 0.39 (−0.00, 0.67) 0.1 (−0.2, 0.4)
mRS on discharge 23 1.00 0 (0, 0) 23 95.6 0.98 (0.96, 0.99) −0.04 (−0.13, 0.05) 26 96.2 0.98 (0.96, 0.99) −0.04 (−0.12, 0.04)
  1. a 95% LCL = 95% Lower Confidence Limit; b 95% CI = 95% Confidence Interval; c MD = Mean difference for reliability study is calculated based on abstractor #1 minus abstractor #2; for validity study the difference is based on abstractor #1 or abstractor #2 minus the “gold standard”; d ICC = Intra-class correlation; NR = Not reported because calculations were misleading due to an error in re-abstraction by abstractor #2 who reported 3 wrong dates (i.e., a wrong day, month or year) out of 7 dates for patients who received tPA therapy, which resulted in a misleading mean difference.