Skip to main content
Download PDF

Use of deep artificial neural networks to identify stroke during triage via subtle changes in circulating cell counts

BMC Neurology volume 22, Article number: 206 (2022) Cite this article

Abstract

Background

The development of tools that could help emergency department clinicians recognize stroke during triage could reduce treatment delays and improve patient outcomes. Growing evidence suggests that stroke is associated with several changes in circulating cell counts. The aim of this study was to determine whether machine-learning can be used to identify stroke in the emergency department using data available from a routine complete blood count with differential.

Methods

Red blood cell, platelet, neutrophil, lymphocyte, monocyte, eosinophil, and basophil counts were assessed in admission blood samples collected from 160 stroke patients and 116 stroke mimics recruited from three geographically distinct clinical sites, and an ensemble artificial neural network model was developed and tested for its ability to discriminate between groups.

Results

Several modest but statistically significant differences were observed in cell counts between stroke patients and stroke mimics. The counts of no single cell population alone were adequate to discriminate between groups with high levels of accuracy; however, combined classification using the neural network model resulted in a dramatic and statistically significant improvement in diagnostic performance according to receiver-operating characteristic analysis. Furthermore, the neural network model displayed superior performance as a triage decision making tool compared to symptom-based tools such as the Cincinnati Prehospital Stroke Scale (CPSS) and the National Institutes of Health Stroke Scale (NIHSS) when assessed using decision curve analysis.

Conclusions

Our results suggest that algorithmic analysis of commonly collected hematology data using machine-learning could potentially be used to help emergency department clinicians make better-informed triage decisions in situations where advanced imaging techniques or neurological expertise are not immediately available, or even to electronically flag patients in which stroke should be considered as a diagnosis as part of an automated stroke alert system.

Peer Review reports

Introduction

Quick and confident recognition of stroke in the emergency department dramatically increases the odds of intervention and positive outcome by facilitating timely intra or inter-hospital referral to stroke-specific care [1]. Unfortunately, the tools available to emergency department clinicians for stroke recognition can be limited. In smaller rural hospitals in particular, there is often not on-demand access to the advanced neuroradiological imaging techniques and expertise needed to reliably detect early stroke pathology, especially in the case of ischemic events [2, 3]. In such situations, critical early triage decisions are made using symptom-based stroke recognition and severity scales, which can be limited in their accuracy [4, 5]. Furthermore, even in larger hospitals where advanced imaging techniques are more readily available, patients can be delayed in receiving such imaging due to a failure of the emergency medicine team to initially consider stroke as a diagnosis, especially in cases presenting with mild, ambiguous, or non-traditional symptoms [6]. As a result, anywhere from 10 to 25% of stroke patients are misdiagnosed at initial assessment in the emergency department, leading to life-threatening delays in care [7, 8]. Thus, if tools can be developed which can identify stroke using widely available and commonly used laboratory measures, they could be employed to help emergency department clinicians make better informed triage decisions when stroke is suspected, or even to automatically flag patients whom stroke should be considered as a possible diagnosis.

The complete blood count with differential (CBC + diff) is one of the most commonly ordered laboratory tests requested in the emergency department [9]; it can be performed in virtually any hospital lab, and increasingly at the point of care [10]. Stroke is pathophysiologically associated with a multitude of changes to the cellular composition of the peripheral blood. For example, stroke triggers robust activation of the innate arm of the immune system, and simultaneous suppression of the adaptive arm of the immune system [11, 12]; this phenomenon has been shown to result in a rise in the circulating counts of innate immune cells such as neutrophils and monocytes, and a decrease in the circulating counts of adaptive immune cells such as lymphocytes [13]. Furthermore, several studies have reported altered counts of red blood cells and platelets in stroke [14, 15], which is not surprising given their role in coagulation and thrombosis. Therefore, it is possible that stroke could be identified in the emergency department using stroke associated patterns of changes to the CBC + diff.

A recent study by Onder et al. [16] reported that the ratio between circulating neutrophil count and lymphocyte count can be used to identify stroke at hospital admission, but with limited accuracy. However, the neutrophil-to-lymphocyte ratio (NLR) is a simplistic metric, and fails to account for changes in counts of other cell populations measured as part of the CBC + diff, and the potential advantage of considering them all collectively. Thus, analysis considering the entirety of the CBC + diff using advanced pattern recognition techniques may be able to offer significantly greater levels of diagnostic performance. Artificial neural networks are biologically-inspired machine-learning algorithms that can be used to solve multivariate classification problems involving complex nonlinear or indirect relationships [17]. In this study, our aim was to determine whether a machine-learning strategy employing artificial neural networks can be implemented to identify stroke during triage using stroke-induced patterns of changes to circulating blood cell counts routinely measured as part of the CBC + diff.

Methods

Experimental design

A cohort of 160 patients with stroke and 116 stroke mimics were recruited from three geographically distinct tertiary care hospitals (Ruby Memorial Hospital, Morgantown, WV; University of Cincinnati Medical Center, Cincinnati, OH; Dell-Seton Medical Center, Austin, TX) between 2011 and 2019. Counts of red blood cells, platelets, neutrophils, lymphocytes, monocytes, eosinophils, and basophils were determined in peripheral blood samples collected at emergency department admission. An ensemble neural network model discriminating between stroke patients and stroke mimics using the collective counts of all seven aforementioned cell populations as input was developed in a randomly selected training set comprised of 75% of the total subject pool, and validated in a test set comprised of the remaining 25% of the total subject pool. The diagnostic performance of this ensemble neural network model was then compared to the diagnostic performance of the counts of each individual cell population, as well as NLR, using receiver operating characteristic (ROC) analysis. Finally, the potential clinical benefit of using the ensemble neural network model as a triage decision making tool as opposed to symptom-based assessments such as the Cincinnati Prehospital Stroke Scale (CPSS) and the National Institutes of Health Stroke Scale (NIHSS) was evaluated via decision curve analysis.

Patients

All patients admitted as suspected strokes were considered for enrolment, pending that study staff were available for blood collection. Patients who displayed definitive radiographic evidence of ischemic or hemorrhagic pathology on magnetic resonance imaging (MRI) or computed tomography (CT) were identified as strokes. Patients receiving a final definitive negative diagnosis for stroke upon neuroradiological imaging and clinical evaluation were identified as acute stroke mimics [18]. All diagnoses were adjudicated by an experienced stroke physician. Patients were excluded if they received a non-definitive diagnosis, were diagnosed with transient ischemic attack, reported a prior hospitalization within 30 days, were under 18 years of age, or were more than 24 h past symptom onset. Time from symptom onset was determined by the time the patient was last known to be free of neurological symptoms. Demographic information was collected from either the subject or legally authorized representative by a trained clinician.

Stroke scale scores

NIHSS scores were collected by a member of the clinical care team at the time of blood draw. CPSS scores were retrospectively calculated from individual NIHSS items similar to the methodology employed by Purrucker et al. [5] and Tarkanyi et. al. [19]. Specifically, values of greater than zero on item 4 of the NIHSS (facial palsy) were scored as abnormal for the facial droop item of the CPSS. Values of greater than zero on item 5 of the NIHSS (R/L arm motor drift) were scored as abnormal for the arm drift item of the CPSS. Finally, values of greater than 0 on either items 9 or 10 of the NIHSS (language and dysarthria, respectively) were scored as abnormal for the speech item of the CPSS.

Circulating cell counts

Venous blood was collected in K2EDTA vacutainers (Becton Dickenson, Franklin Lakes, NJ) and immediately submitted to the clinical pathology laboratory for CBC + diff testing as described previously [20]. Results were subsequently retrieved from the medical record. NLR was calculated as absolute neutrophil count divided by absolute lymphocyte count [16]. In situations where the counts of a cell population were too low for detection, zero values were used for downstream analyses.

Artificial neural network

Neural network analysis was performed in R version 3.6 [21] using the ‘neuralnet’ and ‘NeuralNetTools’ packages [22]. An ensemble neural network model was developed discriminating between stroke patients and stroke mimics in the training set using the counts of all seven cell populations as input. This ensemble model was comprised of five individual feedforward neural network sub-models, each constructed using identical architecture, which was selected based on a simple preliminary grid search of hyper-parameters. Each sub-model of the ensemble used the sigmoid activation function, and consisted of an input layer with 7 nodes accepting 0–1 scaled absolute cell counts, two hidden layers with 5 and 3 nodes respectively, and an output layer with a single node producing a predicted probability of stroke. These sub-models used were trained via backpropagation using random starting weights and an error threshold of 0.25 cross entropy to determine convergence. For each subject, the predicted probabilities of stroke generated across the sub-models were pooled by simple averaging to yield an ensemble predicted probability of stroke. The ensemble predicted probabilities of stroke were then subjected to ROC analysis to determine diagnostic performance. The diagnostic performance of the final ensemble model was subsequently evaluated in the test set, and statistically compared to that observed in the training set.

To gain insights into the associations between the counts of each cell population and stroke prediction probability in the final ensemble neural network model, Olden relative importance values [23] were generated for each sub-model and averaged.

Statistics

All statistics were performed in R version 3.6. Mann–Whitney U test was used to compare continuous variables, while Fisher’s exact test was used to compare categorical variables. Stratified U testing was performed using the ‘coin’ package [24]. The strength and significance of correlative relationships were assessed using Spearman’s rho.

The performance of continuous variables in binary classification was assessed via ROC analysis using the ‘pROC’ package [25]. Sensitivity and specificity values associated with the diagnostic cutoff yielding the highest combined value (Youden index) were reported. The bootstrapping method described by DeLong et al. was used to calculate 95% confidence intervals for area under curve (AUC) values, test the null hypothesis that AUC values = 0.5, compare AUC values from different ROC curves, and generate 95% confidence intervals for sensitivity and specificity values [26]. 1,000 bootstrap samples were used for all bootstrap calculations.

Decision curve analysis [27] was performed using the ‘dca’ package (https://github.com/ddsjoberg/dca). The predicted probabilities of stroke generated by the neural network ensemble were used to generate decision curves directly, while CPSS and NIHSS scores were first converted to prediction probabilities via binary logistic regression, which were subsequently used for decision curve generation. Decision curves were generated across threshold probabilities ranging from 0–1 and were smoothed via moving regression using a span value of 0.25.

In the case of all statistical testing, the null hypothesis was rejected when p < 0.05. The sample size was arbitrarily determined based on availability of resources. The parameters of all statistical tests performed are outlined in detail within the figure legends.

Results

Clinical and demographic characteristics

Across the total study sample, 84% of stroke cases were ischemic, and the remaining 16% were hemorrhagic. Stroke patients were significantly older than stroke mimics, but relatively well matched in terms of sex, race, and the prevalence of cardiovascular disease risk factors such as hypertension, diabetes, and dyslipidemia. Unsurprisingly, stroke patients displayed significantly higher scores on both the CPSS and the NIHSS than stroke mimics. Similar clinical demographic trends were observed within the subsets of patients comprising both the training and test sets used for neural network model development and evaluation (Table 1). The stroke mimic group displayed a relatively wide variety of final diagnoses, most common of which were seizure, complex migraine, hypertensive encephalopathy, psychogenic or psychiatric pathologies including conversion disorder, hypotension, and infection (Table 2).

Table 1 Clinical and demographic characteristics
Full size table
Table 2 Final diagnoses of stroke mimics
Full size table

Differences in circulating cell counts

Across the total study sample, neutrophil, monocyte, and red blood cell counts were significantly elevated in stroke patients compared to stroke mimics, independent of age. Basophil counts were also higher in the stroke group, but this difference was not statistically significant. Platelet and lymphocyte counts were both significantly lower in stroke patients compared to stroke mimics. Eosinophil counts were also lower in the stroke group, however this difference was not statistically significant. Unsurprisingly given the differences we observed in both neutrophil and lymphocyte counts, NLR was significantly higher in stroke patients relative to stroke mimics (Fig. 1A).

Fig. 1
figure 1

Comparison of circulating cell counts between stroke patients and stroke mimics. A Circulating red blood cell, platelet, neutrophil, lymphocyte, monocyte, eosinophil, and basophil counts, as well as neutrophil-to-lymphocyte ratios in stroke patients and stroke mimics at hospital admission. Bloxplots indicate median and interquartile range. Some axes are broken to display extreme values, as indicated by dashed line. Intergroup statistical comparisons were performed via two-sided Mann–Whitney U test, both stratified and unstratified by age quartile. B ROC curves depicting the individual abilities of circulating cell counts to discriminate between stroke patients and stroke mimics, compared to the CPSS and the NIHSS. Sensitivity and specificity values are indicated for the diagnostic cutoff which produced the highest Youden index. Bootstrapped 95% confidence intervals associated with all diagnostic statistics are indicated. P-values indicate the probability that area under ROC curve values differ from 0.5. *Statistically significant

Full size image

Despite many of these differences being statistically significant, none were adequate to discriminate between groups with high levels of accuracy when considered in isolation. The counts of the individual cell populations performed poorly in ROC analysis, producing AUC values ranging from 0.54–0.59, sensitivities ranging from 24.4–68.1%, and specificities ranging from 45.7–85.3%. NLR exhibited slightly better diagnostic performance than the counts of any individual cell population, producing an AUC of 0.61 (0.95 CI: 0.55–0.68), but could still only discriminate between groups with 55.6% (0.95 CI: 39.3–65.0%) sensitivity and 66.4% (0.95 CI: 50.8–74.1%) specificity. Comparatively, CPSS scores produced an AUC of 0.63 (0.95 CI: 0.57–0.670), and could differentiate between groups with 50.0% (0.95 CI: 36.8–58.1%) sensitivity and 68.0% (0.95 CI: 50.9–76.7%) specificity, while NIHSS scores produced an AUC of 0.70 (0.95 CI: 0.64–0.76) and could differentiate between groups with 53.1% (0.95 CI: 38.1–60.9%) sensitivity and 84.5% (0.95 CI: 65.8–91.2%) specificity (Fig. 1B).

Artificial neural network performance

The final ensemble neural network model developed in the training set is depicted in Fig. 2A. Our analysis of relative importance suggested that neutrophil, monocyte, and basophil counts were the strongest predictors of stroke across the entire ensemble model, all exhibiting positive associations with stroke in nearly every sub-model. This was followed by platelet and lymphocyte counts, which exhibited negative associations with stroke in nearly every sub-model. Red blood cell and eosinophil counts were the weakest predictors of stroke across the entire ensemble, but did still offer high predictive value in some sub-models (Fig. 2B).

Fig. 2
figure 2

Final ensemble neural network model. A Visual representation of the final ensemble neural network model developed in the training set, which was comprised of five individual sub-models. The synaptic weights and node bias terms associated with each of the five sub-models are indicated. All input values were scaled between 0 and 1 for training. Sigmoid activation functions were used at all nodes. Final ensemble stroke prediction probabilities were generated via simple averaging of the sub-model prediction probabilities. Pr, probability. B The relative importance of each cell type in each of the five individual sub-models which comprise the final ensemble neural network model, as calculated using the Olden method. Averaged importance values for each cell type across the five sub-models are also indicated. Values of 0 indicate little importance, while values approaching 1 and -1 indicate strong positive and negative associations with stroke prediction probability respectively. C ROC curves depicting the ability of the final ensemble neural network model to discriminate between stroke patients and controls in the training set, test set, and total study population. Sensitivity and specificity values are indicated for the diagnostic cutoff which produced the highest Youden index. Bootstrapped 95% confidence intervals associated with all diagnostic statistics are indicated. P-values indicate the probability area under ROC curve values differ from 0.5. Training set and test set AUC values were statistically compared using the DeLong method. *Statistically significant

Full size image

In terms of diagnostic performance, in the training set, the final ensemble model produced an AUC of 0.84 (0.95 CI: 0.78–0.89) in ROC analysis, and could discriminate between stroke patients and stroke mimics with 83.3% (0.95 CI: 69.2–91.7%) sensitivity and 70.1% (0.95 CI: 55.2–79.3%) specificity at optimal diagnostic cutoff. In the test set, the model produced an AUC of 0.78 (0.95 CI: 0.67–0.89), and could discriminate between groups with 57.5% (0.95 CI: 22.5–75.0%) sensitivity and 89.7% (0.95 CI: 58.6–96.6%) specificity. Finally, when considering the total subject pool, the final ensemble model produced an AUC of 0.82 (0.95 CI: 0.78–0.87), and could discriminate between groups with 68.1% (0.95 CI: 52.5–75.6%) sensitivity and 82.8% (0.95 CI: 68.1–89.7%) specificity. Statistical comparison of area under ROC curves revealed no statistical difference in model performance between the training and test sets (Fig. 2C).

When considering the total subject pool, correlational analysis revealed a weak but statistically significant positive relationship between time from symptom onset to blood collection and the prediction probability of stroke produced by the final ensemble neural network model in stroke patients, suggesting that the model may offer slightly lower levels of sensitivity at earlier time points in the progression of pathology (Fig. 3A). Areas under ROC curves generated in subsets of patients respectively evaluated within three hours, from three to six hours, and greater than six hours from symptom onset were highly similar; however, when considering the overall shape of the ROC curves, there did appear to be a slight shift from higher specificity at the earliest timeframe towards higher sensitivity at the latest timeframe (Fig. 3B).

Fig. 3
figure 3

Effect of time from symptom onset on model performance. A Relationship between time from symptom onset to blood collection and the prediction probability of stroke produced by the final ensemble neural network model, in both stroke patients and stroke mimics, across the total study population. Strength and significance of correlations were accessed using Spearman’s rho. B ROC curves comparing the ability of the final ensemble neural network model to discriminate between stroke and stroke mimicking conditions in patients evaluated less than 3 h, from 3 to 6 h, or greater than 6 h from symptom onset. Sensitivity and specificity values are indicated for the diagnostic cutoff which produced the highest Youden index. Bootstrapped 95% confidence intervals associated with all diagnostic statistics are indicated. P-values indicate the probability area under ROC curve values differ from 0.5. AUC values were statistically compared across timepoints using the DeLong method. *Statistically significant

Full size image

Comparison of the artificial neural network model to other decision-making tools

Statistical comparison of area under ROC curves revealed that the final ensemble neural network model provided a significantly greater overall level of diagnostic performance than the counts of any individual cell population, or NLR, in each of the training set, test set, and total subject pool (Fig. 4). Similarly, area under ROC curves associated with the final ensemble neural network model were significantly greater than those associated with CPSS scores across all subsets of subjects. Area under ROC curves associated with the final ensemble neural network model were also greater than those associated with NIHSS scores across all subsets of subjects, however this difference was only statistically significant in the training set and the total subject pool (Fig. 5A).

Fig. 4
figure 4

Comparison of the final neural network model to individual cell counts for stroke recognition. ROC curves depicting the ability of the final ensemble neural network model to discriminate between stroke patients and stroke mimics, in comparison to the abilities of each individual cell count, with respect to the training set, test set, and total study population. Bootstrapped 95% confidence intervals associated with area under ROC curve values are indicated. AUC values were statistically compared using the DeLong method. *Statistically significant

Full size image
Fig. 5
figure 5

Comparison of the final neural network model to symptom-based scales for stroke recognition. A ROC curves depicting the ability of the final ensemble neural network model to discriminate between stroke patients and stroke mimics, in comparison to CPSS scores and NIHSS scores, with respect to the training set, test set, and total study population. Bootstrapped 95% confidence intervals associated with area under ROC curve values are indicated. AUC values were statistically compared using the DeLong method. *Statistically significant. B Decision curves comparing the observed net benefit of using the final ensemble neural network model as a triage tool to those of the CPSS and the NIHSS, across a range of decision thresholds

Full size image

Our decision curve analysis suggested that the aforementioned differences in diagnostic performance would result in clinical benefit if the final ensemble neural network model were used as a triage decision-making tool as opposed to either the CPSS or the NIHSS, at least in our study population. It could be argued that the decision threshold to triage a patient as stroke should reasonably fall somewhere between a 10–50% probability that they are indeed having a stroke, depending on the specific triage decision (inter-hospital transfer, intra-hospital referral, etc.). In most situations, the use of a threshold over 50% would result in an unacceptable risk of triaging strokes as mimics, and the use of a threshold of less than 10% would incur unnecessary cost and resource strain associated with triaging too many mimics as strokes. Across this entire range of clinically relevant decision thresholds, the net benefit of using the final ensemble neural network model to triage stroke was greater than that of using either the CPSS or the NIHSS, in each of the training set, test set, and total subject pool (Fig. 5B).

Discussion

The primary aim of this investigation was to determine whether machine-learning can be used to identify stroke during triage by analyzing patterns of stroke-induced changes in routinely measured circulating blood cell counts. The final ensemble neural network model generated in our analysis was able to discriminate between stroke patients and stroke mimics with relatively high levels of accuracy at emergency department admission using only information extracted from the CBC + diff. Our results suggest that machine-learning based tools could be implemented in the future to help clinicians recognize stroke in the acute phase of care via this widely available and commonly performed laboratory test.

Many of the differences in circulating cell counts which we observed between stroke patients and stroke mimics were similar to those reported in prior investigations, providing further evidence that the cellular complexion of the peripheral blood is altered in stroke. However, to our knowledge, our analysis is one of the first to make direct case–control comparisons of these cell populations between stroke patients and true stroke mimics in the acute phase of care, as opposed to previous analyses which have predominantly employed neurologically normal controls [13,14,15, 28,29,30,31,32,33,34,35,36]. It has become increasingly evident that stroke triggers a general activation of the innate arm of the peripheral immune system, and a general suppression of the adaptive arm of the peripheral immune system [11, 12]. Accordingly, numerous prior studies of both ischemic and hemorrhagic pathology have reported that stroke induces an increase in circulating numbers of neutrophils and monocytes, and a simultaneous decrease in the number of circulating lymphocytes [13, 28,29,30,31, 35,36,37]. Our results are highly consistent with these prior reports, as some of the most statistically significant differences we observed between stroke patients and controls were in these three cell populations. A small number of prior studies of ischemic stroke have suggested a possible drop in circulating eosinophil counts early in pathology [32, 38]; while we did observe slightly lower eosinophil counts in stroke patients versus stroke mimics, this difference was not statistically significant. To our knowledge, there have been no prior case control studies of circulating basophil counts in stroke; we observed a slight increase in basophil counts in stroke patients versus controls, but this difference was again not statistically significant. However, given that there appeared to be a relatively strong positive association between basophil count and stroke in nearly every sub-model of our neural network ensemble, the effect of stroke on basophil count may warrant further investigation.

With respect to non-immune cell populations, several prior case control studies have reported lower circulating platelet counts in patients with both ischemic and hemorrhagic stroke at hospital admission [14]. It is theorized that this phenomena is at least in part a result of stroke-related increases in platelet activation and depletion in the absence of changes in platelet production [39]. Our results were strongly supportive of these prior studies, as we too observed significantly lower platelet counts in stroke patients versus stroke mimics. There is less definitive prior evidence regarding stroke-associated changes in red blood cell counts in stroke in the acute phase of care. There is evidence that conditions which result in both abnormally high or abnormally low red blood cell counts can increase the risk of stroke [40], however there have been surprisingly few case–control analyses of red blood cell counts in stroke patients during early pathology; those that have been performed have investigated ischemic stroke specifically, and reported either similar or decreased red blood cell counts in stroke patients relative to controls [15, 33, 34]. Here, we observed slightly higher red blood cell counts in the stroke group compared to the stroke mimic group, a difference that was only statistically significant after controlling for age. Interestingly, red blood cell count was strongly positively associated with stroke in two sub-models of our neural network ensemble, and strongly negatively associated with stroke in another two; because stroke can be secondary to both conditions that result in increases or decreases in red blood cell count, it is plausible that these different sub-models are modeling different stroke etiologies, which highlights a benefit of our ensemble learning strategy.

Despite many being statistically significant, none of the aforementioned differences we observed in cell counts between stroke patients and stroke mimics were able to discriminate between groups with high levels of accuracy in isolation, nor was the NLR. However, our final ensemble neural network model was able to identify stroke with levels of diagnostic performance that exceed those of the CPSS and the NIHSS, which are two of the most widely used symptom based tools used to evaluate stroke in the emergency department. Not only does this highlight the power of machine learning-based analysis and the benefit of diagnostically considering the collective pattern of changes across the entirety of cellular populations present in the blood, but it also suggests that the strategy which we employed here could have true clinical utility. Algorithmic evaluation of the CBC + diff using machine-learning tools could be used to provide a stroke probability which could be used to help emergency department clinicians make better-informed transfer, referral, and treatment decisions in situations when stroke is expected; this could be especially useful in smaller rural hospitals where stroke expertise or advanced imaging techniques may not be immediately available [2, 3]. Perhaps more intriguing, because a CBC is ordered for a majority of emergency department patients [9], it could be used to flag individuals who may be experiencing a stroke as part of an automated stroke alert system which is integrated with the electronic medical record. Such a system could expedite care in situations where stroke is not initially being considered as a diagnosis, such as cases with atypical presentations or mild symptoms [6], and help reduce treatment lags stemming from delayed recognition in frequently mistriaged populations such as the young and minorities [41, 42].

It is pertinent to note that this is not the first work to attempt to employ machine-learning methods to develop diagnostic tools for stroke. There have been several studies published by our group and others over the past decade that have used similar methodology to develop diagnostic models; however, they have largely focused on the analysis of neuroradiological images [43, 44] or non-standard-of-care molecular biomarkers [45,46,47,48], whereas to our knowledge, this study is the first to focus solely on the analysis of circulating cell counts which are routinely assessed as part of stat testing ordered in the emergency department. The aforementioned imaging-based tools certainly have demonstrated that they can serve an important role in the diagnostic pathway, however, they only have utility if a patient is first referred to said imaging based on positive results in initial symptom-based screening. Furthermore, the aforementioned strategies developed using non-standard-of-care biomarkers in conjunction with machine-learning methods have yet to be trialed in studies using true stroke mimics, and their diagnostic performance has never been directly compared to the symptom-based tools currently used for initial triage; even if they do show strong performance in future testing, they would be expensive to implement, and would require an initial suspicion of stroke to be ordered.

While our results are exciting, further refinements and validation are needed before such strategies could be clinically implemented. While the model we developed here only considered the absolute cell counts, there is other information available as part of the CBC + diff that could also be included to potentially increase accuracy. For example, platelet and red blood cell mean volume and distribution width have been shown to be altered in stroke [14, 49], and are now generated by most hematology analyzers. Furthermore, because of the relatively small number of hemorrhagic stroke patients in our study cohort, we chose to combine both ischemic and hemorrhagic stroke patients into a single pooled stroke group; while both types of stroke appear to be associated with a similar general pattern of cellular changes to the peripheral blood, there may be subtle differences that could further differentiate between the two. For example, while neutrophil, monocyte, and lymphocyte counts are all altered in both hemorrhagic and ischemic stroke [13, 28,29,30,31, 35,36,37], prior studies suggest that magnitude of these alterations may be different [35, 50]. Thus, in larger future investigations where it is more feasible, these stroke subtypes could be modeled as separate groups; given that they result in slightly different blood changes, doing so may actually improve the accuracy of overall stroke recognition. Also, given the size and preliminary nature of our analysis, we chose to exclude patients with transient ischemic attack; clearly, future studies will need to include and determine how to diagnostically account for this population.

Finally, in a future larger follow-up analysis, it is possible that a different neural network implementation or other type of machine-learning algorithm may offer better performance. Here, because of relatively small sample size, we decided to use an ensemble strategy and a relatively low error threshold for neural network convergence within the composite sub-models to increase generalizability. However, in a larger sample size, such hyper-parameters may be able to be adjusted to increase accuracy without as much risk of overfitting. Furthermore, different machine-learning classifiers can be better trained and offer better performance versus others in different use scenarios depending on factors such as the degree of linearity in the relationships between predictor variables and output variables or the ratio between the number of predictor variables and number of cases. While we used artificial neural networks successfully here, follow-up investigations employing more predictors or modeling different stroke subtypes as separate groups such should also assess and empirically compare additional algorithms such as support vector machine and random forest to identify the approach which will offer the best diagnostic performance.

Conclusions

Modeling patterns of stroke associated changes to the CBC + diff using machine-learning could yield tools with the potential to help emergency department clinicians make better informed triage decisions in situations where advanced imaging techniques or neurological expertise are not immediately available. Furthermore, the principles demonstrated in this work could provide the foundation for development of an automated stroke alert system which pulls hematology data from the electronic medical record and returns a flag to identify patients in which stroke is likely.

Availability of data and materials

The data that support the findings of this study are available from Valtari Bio Incorporated but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the corresponding author upon reasonable request and with permission of Valtari Bio Incorporated.

Abbreviations

CBC:

Complete blood count

CBC + diff:

Complete blood count with differential

AUC:

Area under curve

ROC:

Receiver operating characteristic

CPSS:

Cincinnati prehospital stroke scale

NIHSS:

National Institutes of Health stroke scale

NLR:

Neutrophil-to-lymphocyte ratio

References

  1. WJ Powers, AA Rabinstein, T Ackerson. et al. 2018 Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the american heart association/american stroke association. Stroke. 2018;49. https://doi.org/10.1161/STR.0000000000000158

  2. Shultis W, Graff R, Chamie C, et al. Striking rural-urban disparities observed in acute stroke care capacity and services in the pacific northwest: implications and recommendations. Stroke. 2010;41:2278–82. https://doi.org/10.1161/STROKEAHA.110.594374.

    Article  PubMed  Google Scholar 

  3. EC Leira, DC Hess, JC Torner, HP Adams. Rural-urban differences in acute stroke management practices: a modifiable disparity. Arch Neurol. 2008;65. https://doi.org/10.1001/archneur.65.7.887

  4. Rudd M, Buck D, Ford GA, Price CI. A systematic review of stroke recognition instruments in hospital and prehospital settings. Emerg Med J. 2016;33:818–22. https://doi.org/10.1136/emermed-2015-205197.

    Article  PubMed  Google Scholar 

  5. Purrucker JC, Hametner C, Engelbrecht A, et al. Comparison of stroke recognition and stroke severity scores for stroke detection in a single cohort. J Neurol Neurosurg Psychiatry. 2015;86:1021–8. https://doi.org/10.1136/jnnp-2014-309260.

    Article  PubMed  Google Scholar 

  6. Lever NM, Nyström KV, Schindler JL, et al. Missed opportunities for recognition of ischemic stroke in the emergency department. J Emerg Nurs. 2013;39:434–9. https://doi.org/10.1016/j.jen.2012.02.011.

    Article  PubMed  Google Scholar 

  7. Arch AE, Weisman DC, Coca S, et al. Missed ischemic stroke diagnosis in the emergency department by emergency medicine and neurology services. Stroke. 2016;47:668–73.

    Article  Google Scholar 

  8. Tarnutzer AA, Lee S-H, Robinson KA, et al. ED misdiagnosis of cerebrovascular events in the era of modern neuroimaging: a meta-analysis. Neurology. 2017;88:1468–77. https://doi.org/10.1212/WNL.0000000000003814.

    Article  PubMed  PubMed Central  Google Scholar 

  9. MM Johnson, KB Lewandrowski. Analysis of Emergency Department Test Ordering Patterns in an Urban Academic Medical Center: Can the Point-of-Care Option in a Satellite Laboratory Provide Sufficient Menu to Permit Full Service Testing. 2007;6:5

  10. Mooney C, Byrne M, Kapuya P, et al. Point of care testing in general haematology. Br J Haematol. 2019;187:296–306. https://doi.org/10.1111/bjh.16208.

    Article  PubMed  Google Scholar 

  11. Chamorro Á, Meisel A, Planas AM, et al. The immunology of acute stroke. Nat Rev Neurol. 2012;8:401–10. https://doi.org/10.1038/nrneurol.2012.98.

    Article  CAS  PubMed  Google Scholar 

  12. O’Connell GC, Tennant CS, Lucke-Wold N, et al. Monocyte-lymphocyte cross-communication via soluble CD163 directly links innate immune system activation and adaptive immune system suppression following ischemic stroke. Sci Rep. 2017;7:12940–12940. https://doi.org/10.1038/s41598-017-13291-6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Vogelgesang A, Grunwald U, Langner S, et al. Analysis of lymphocyte subsets in patients with stroke and their influence on infection after stroke. Stroke. 2008;39:237–41. https://doi.org/10.1161/STROKEAHA.107.493635.

    Article  PubMed  Google Scholar 

  14. Sadeghi F, Kovács S, Zsóri KS, et al. Platelet count and mean volume in acute stroke: a systematic review and meta-analysis. Platelets. 2020;31:731–9. https://doi.org/10.1080/09537104.2019.1680826.

    Article  CAS  PubMed  Google Scholar 

  15. Sharif S, Ghaffar S, Saqib M, Naz S. Analysis of hematological parameters in patients with ischemic stroke. Endocrinology & Metabolism International Journal. 2020;8:17–20. https://doi.org/10.15406/emij.2020.08.00271.

    Article  Google Scholar 

  16. Onder H, Cankurtaran M, Ozyurek O, Arslan G. Neutrophil-to-Lymphocyte ratio may have a role in the differential diagnosis of ischemic stroke. Neurol Asia. 2020;10:93–101.

    Google Scholar 

  17. Krogh A. What are artificial neural networks? Nat Biotechnol. 2008;26:195–7. https://doi.org/10.1038/nbt1386.

    Article  CAS  PubMed  Google Scholar 

  18. Kidwell CS, Warach S. Acute Ischemic Cerebrovascular Syndrome: Diagnostic Criteria. Stroke. 2003;34:2995–8. https://doi.org/10.1161/01.STR.0000098902.69855.A9.

    Article  PubMed  Google Scholar 

  19. Tarkanyi G, Csecsei P, Szegedi I, et al. Detailed severity assessment of Cincinnati Prehospital Stroke Scale to detect large vessel occlusion in acute ischemic stroke. BMC Emerg Med. 2020;20:64. https://doi.org/10.1186/s12873-020-00360-9.

    Article  PubMed  PubMed Central  Google Scholar 

  20. GC O’Connell, MB Treadway, AB Petrone. et al 2017 Leukocyte Dynamics Influence Reference Gene Stability in Whole Blood: Data-Driven qRT-PCR Normalization Is a Robust Alternative for Measurement of Transcriptional Biomarkers Laboratory Medicine 48 https://doi.org/10.1093/labmed/lmx035

  21. Ross I, Gentleman R, Ihaka R, Gentleman R. R: A Language for Data Analysis and Graphics. J Comput Graph Stat. 1996;5:299–314. https://doi.org/10.1080/10618600.1996.10474713.

    Article  Google Scholar 

  22. Beck MW. NeuralNetTools: Visualization and Analysis Tools for Neural Networks. J Statistical Software. 2018;85. https://doi.org/10.18637/jss.v085.i11

  23. Olden JD, Joy MK, Death RG. An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecol Model. 2004;178:389–97. https://doi.org/10.1016/j.ecolmodel.2004.03.013.

    Article  Google Scholar 

  24. Hothorn T, Hornik K, Wiel MA van de, Zeileis A. Implementing a Class of Permutation Tests: The coin Package. J Statistical Software. 2008;28. https://doi.org/10.18637/jss.v028.i08

  25. Robin X, Turck N, Hainard A, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:1–8. https://doi.org/10.1186/1471-2105-12-77.

    Article  Google Scholar 

  26. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45.

    Article  CAS  Google Scholar 

  27. Vickers AJ, Elkin EB. Decision Curve Analysis: A Novel Method for Evaluating Prediction Models. Med Decis Making. 2006;26:565–74. https://doi.org/10.1177/0272989X06295361.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Urra X, Cervera Á, Villamor N, et al. Harms and benefits of lymphocyte subpopulations in patients with acute stroke. Neuroscience. 2009;158:1174–83. https://doi.org/10.1016/j.neuroscience.2008.06.014.

    Article  CAS  PubMed  Google Scholar 

  29. Haeusler KG, Schmidt WUH, Föhring F, et al. Cellular Immunodepression Preceding Infectious Complications after Acute Ischemic Stroke in Humans. Cerebrovasc Dis. 2008;25:50–8. https://doi.org/10.1159/000111499.

    Article  CAS  PubMed  Google Scholar 

  30. Hug A, Dalpke A, Wieczorek N, et al. Infarct Volume is a Major Determiner of Post-Stroke Immune Cell Function and Susceptibility to Infection. Stroke. 2009;40:3226–32. https://doi.org/10.1161/STROKEAHA.109.557967.

    Article  PubMed  Google Scholar 

  31. Urra X, Cervera Á, Obach V, et al. Monocytes are major players in the prognosis and risk of infection after acute stroke. Stroke. 2009;40:1262–8. https://doi.org/10.1161/STROKEAHA.108.532085.

    Article  CAS  PubMed  Google Scholar 

  32. O’Connell GC, Chang JHC. Analysis of early stroke-induced changes in circulating leukocyte counts using transcriptomic deconvolution. Transl Neurosci. 2018;9:161–6. https://doi.org/10.1515/tnsci-2018-0024.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Chitsaz A, Tolou-Ghamari Z, Ashtari F. Preliminary evaluations related to the ranges of hematological and biochemical variables in hospitalized patients with stroke. Int J Prev Med. 2013;4:S347-352.

    PubMed  PubMed Central  Google Scholar 

  34. Mercuri M, Ciuffetti G, Robinson M, Toole J. Blood cell rheology in acute cerebral infarction. Stroke. 1989;20:959–62. https://doi.org/10.1161/01.STR.20.7.959.

    Article  CAS  PubMed  Google Scholar 

  35. Vural G, Gümüşyayla Ş, Akdeniz G. Neutrophil/Lymphocyte Ratio In Stroke Patients And Its Relation With Functional Recovery. Medeniyet Med J. 2018. https://doi.org/10.5222/MMJ.2018.83097.

    Article  Google Scholar 

  36. Jiang C, Wang Y, Hu Q, et al. Immune changes in peripheral blood and hematoma of patients with intracerebral hemorrhage. FASEB J. 2020;34:2774–91. https://doi.org/10.1096/fj.201902478R.

    Article  CAS  PubMed  Google Scholar 

  37. O’Connell GC, Treadway MB, Tennant CS, et al. Shifts in Leukocyte Counts Drive the Differential Expression of Transcriptional Stroke Biomarkers in Whole Blood. Transl Stroke Res. 2018. https://doi.org/10.1007/s12975-018-0623-1.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Wang J, Ma L, Lin T, et al. The significance of eosinophils in predicting the severity of acute ischemic stroke. Oncotarget. 2017;8. https://doi.org/10.18632/oncotarget.22199

  39. Tohgi H, Suzuki H, Tamura K, Kimura B. Platelet volume, aggregation, and adenosine triphosphate release in cerebral thrombosis. Stroke. 1991;22:17–21. https://doi.org/10.1161/01.STR.22.1.17.

    Article  CAS  PubMed  Google Scholar 

  40. Grotta JC, Manner C, Pettigrew LC, Yatsu FM. Red blood cell disorders and stroke. Stroke. 1986;17:811–7. https://doi.org/10.1161/01.STR.17.5.811.

    Article  CAS  PubMed  Google Scholar 

  41. Kuruvilla A, Bhattacharya P, Rajamani K, Chaturvedi S. Factors Associated With Misdiagnosis of Acute Stroke in Young Adults. J Stroke Cerebrovasc Dis. 2011;20:523–7. https://doi.org/10.1016/j.jstrokecerebrovasdis.2010.03.005.

    Article  PubMed  Google Scholar 

  42. Newman-Toker DE, Moy E, Valente E, et al. Missed diagnosis of stroke in the emergency department: a cross-sectional analysis of a large population-based sample. Diagnosis. 2014;1:155–66. https://doi.org/10.1515/dx-2013-0038.

    Article  PubMed  Google Scholar 

  43. Soun JE, Chow DS, Nagamine M, et al. Artificial Intelligence and Acute Stroke Imaging. AJNR Am J Neuroradiol. 2021;42:2–11. https://doi.org/10.3174/ajnr.A6883.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Mainali S, Darsie ME, Smetana KS. Machine Learning in Action: Stroke Diagnosis and Outcome Prediction. Front Neurol. 2021;12:734345. https://doi.org/10.3389/fneur.2021.734345.

    Article  PubMed  PubMed Central  Google Scholar 

  45. O’Connell GC, Petrone AB, Treadway MB, et al. Machine-learning approach identifies a pattern of gene expression in peripheral blood that can accurately detect ischaemic stroke. npj Genom Med. 2016;1:16038–16038. https://doi.org/10.1038/npjgenmed.2016.38.

    Article  PubMed  PubMed Central  Google Scholar 

  46. O’Connell GC, Chantler PD, Barr TL. Stroke-associated pattern of gene expression previously identified by machine-learning is diagnostically robust in an independent patient population. Genomics data. 2017;14:47–52. https://doi.org/10.1016/j.gdata.2017.08.006.

    Article  PubMed  PubMed Central  Google Scholar 

  47. O’Connell GC, Stafford P, Walsh KB, et al. High-Throughput Profiling of Circulating Antibody Signatures for Stroke Diagnosis Using Small Volumes of Whole Blood. Neurotherapeutics. 2019;16:868–77. https://doi.org/10.1007/s13311-019-00720-9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Tiedt S, Prestel M, Malik R, et al. RNA-Seq Identifies Circulating miR-125a-5p, miR-125b-5p, and miR-143-3p as Potential Biomarkers for Acute Ischemic Stroke. Circ Res. 2017;121:970–80. https://doi.org/10.1161/CIRCRESAHA.117.311572.

    Article  CAS  PubMed  Google Scholar 

  49. H Kara, S Degirmenci, A Bayir. et al. Red cell distribution width and neurological scoring systems in acute stroke patients. Neuropsychiatr Dis Treat. 2015;733. https://doi.org/10.2147/NDT.S81525

  50. Cortina MG, Campello AR, Conde JJ, et al. Monocyte count is an underlying marker of lacunar subtype of hypertensive small vessel disease: The role of monocyte count in lacunar stroke. Eur J Neurol. 2008;15:671–6. https://doi.org/10.1111/j.1468-1331.2008.02145.x.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would foremost like to thank the subjects and their families; this work would not have been possible without their selfless contributions. We would also like to thank Shanell Hill, PhD candidate, School of Nursing, Case Western Reserve University, for assistance with data management.

Funding

Work was funded by Valtari Bio Incorporated (Morgantown, WV), as well as new investigator start-up funds issued to GCO by the Francis Payne Bolton School of Nursing at Case Western Reserve University.

Author information

Authors and Affiliations

  1. School of Nursing, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH, 44106-4904, USA

    Grant C. O’Connell, Christine G. Smothers, Suebsarn Ruksakulpiwat, Bethany L. Armentrout & Chris Winkelman

  2. Molecular Biomarker Core, Case Western Reserve University, Cleveland, OH, USA

    Grant C. O’Connell

  3. Department of Emergency Medicine, College of Medicine, University of Cincinnati, Cincinnati, OH, USA

    Kyle B. Walsh

  4. Gardner Neuroscience Institute, University of Cincinnati, Cincinnati, OH, USA

    Kyle B. Walsh

  5. Dell Seton Medical Center, University of Texas, Austin, TX, USA

    Truman J. Milling & Steven J. Warach

  6. School of Nursing, Ohio State University, Columbus, OH, USA

    Taura L. Barr

  7. Valtari Bio Incorporated, Morgantown, WV, USA

    Taura L. Barr

Authors
  1. Grant C. O’Connell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kyle B. Walsh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christine G. Smothers
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Suebsarn Ruksakulpiwat
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bethany L. Armentrout
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chris Winkelman
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Truman J. Milling
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Steven J. Warach
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Taura L. Barr
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The study was conceived by GCO. KBW, TJM, SJW, and TLB supervised recruitment of patients and collection of clinical data. CW and SR aggregated and cleaned data collected across clinical sites. CGS and BLA scored stroke scales. GCO analyzed the data. The manuscript was drafted by GCO with contributions from KBW, TJM, CW, CGS, and BA. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Grant C. O’Connell.

Ethics declarations

Ethics approval and consent to participate

All methods were carried out in accordance with relevant guidelines and regulations, and were approved by the Institutional Review Boards of West Virginia University/Ruby Memorial Hospital (IRB protocol # 1410450461R001), University of Cincinnati Medical Center (IRB protocol # IRB16-00291), and Dell-Seton Medical Center (IRB protocol # STUDY20180221). Written informed consent was obtained from all subjects or their authorized representatives prior to any study procedures.

Consent for publication

Not applicable.

Competing interests

KBW, TJM, and SJW were financially compensated for their work by Valtari Bio Incorporated. TLB serves as Chief Scientific Officer of Valtari Bio Incorporated. GCO is a former employee of Valtari Bio Incorporated.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

O’Connell, G.C., Walsh, K.B., Smothers, C.G. et al. Use of deep artificial neural networks to identify stroke during triage via subtle changes in circulating cell counts. BMC Neurol 22, 206 (2022). https://doi.org/10.1186/s12883-022-02726-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12883-022-02726-x

Keywords

BMC Neurology

ISSN: 1471-2377

Contact us