Skip to main content
Download PDF

Personalized decision-making for aneurysm treatment of aneurysmal subarachnoid hemorrhage: development and validation of a clinical prediction tool

BMC Neurology volume 24, Article number: 65 (2024) Cite this article

Abstract

Background

In patients with aneurysmal subarachnoid hemorrhage suitable for endovascular coiling and neurosurgical clip-reconstruction, the aneurysm treatment decision-making process could be improved by considering heterogeneity of treatment effect and durability of treatment. We aimed to develop and validate a tool to predict individualized treatment benefit of endovascular coiling compared to neurosurgical clip-reconstruction.

Methods

We used randomized data (International Subarachnoid Aneurysm Trial, n = 2143) to develop models to predict 2-month functional outcome and to predict time-to-rebleed-or-retreatment. We modeled for heterogeneity of treatment effect by adding interaction terms of treatment with prespecified predictors and with baseline risk of the outcome. We predicted outcome with both treatments and calculated absolute treatment benefit. We described the patient characteristics of patients with ≥ 5% point difference in the predicted probability of favorable functional outcome (modified Rankin Score 0–2) and of no rebleed or retreatment within 10 years. Model performance was expressed with the c-statistic and calibration plots. We performed bootstrapping and leave-one-cluster-out cross-validation and pooled cluster-specific c-statistics with random effects meta-analysis.

Results

The pooled c-statistics were 0.72 (95% CI: 0.69–0.75) for the prediction of 2-month favorable functional outcome and 0.67 (95% CI: 0.63–0.71) for prediction of no rebleed or retreatment within 10 years. We found no significant interaction between predictors and treatment. The average predicted benefit in favorable functional outcome was 6% (95% CI: 3–10%) in favor of coiling, but 11% (95% CI: 9–13%) for no rebleed or retreatment in favor of clip-reconstruction. 134 patients (6%), young and in favorable clinical condition, had negligible functional outcome benefit of coiling but had a ≥ 5% point benefit of clip-reconstruction in terms of durability of treatment.

Conclusions

We show that young patients in favorable clinical condition and without extensive vasospasm have a negligible benefit in functional outcome of endovascular coiling – compared to neurosurgical clip-reconstruction – while at the same time having a substantially lower probability of retreatment or rebleeding from neurosurgical clip-reconstruction – compared to endovascular coiling. The SHARP prediction tool (https://sharpmodels.shinyapps.io/sharpmodels/) could support and incentivize a multidisciplinary discussion about aneurysm treatment decision-making by providing individualized treatment benefit estimates.

Peer Review reports

Background

As shown in multiple randomized controlled trials (RCTs), in aneurysmal subarachnoid hemorrhage (aSAH) patients that are equally eligible for endovascular coiling as for neurosurgical clip-reconstruction, endovascular coiling results in better functional outcome [1,2,3,4]. However, long-term follow-up revealed that patients who underwent neurosurgical clip-reconstruction had a lower rate of recurrent bleeding and retreatment of the target aneurysm and a higher degree of aneurysm obliteration [5, 6]. On average, this excess rebleeding and retreatment did not affect functional outcome [7].

Optimal aneurysm treatment is discussed in multidisciplinary teams where factors favoring neurosurgical clip-reconstruction or endovascular coiling are weighed. Examples of factors contributing to the choice for endovascular coiling are older age, worse clinical grade, vasospasm at presentation, and aneurysm location in the posterior circulation. On the other hand, neurosurgical clip-reconstruction is more likely to be performed in younger patients, patients with very small or very large (giant) aneurysms, wide-necked aneurysms, or middle cerebral artery aneurysms, or patients with concomitant intracerebral hemorrhage [8, 9]. When there is no preferred treatment, guidelines advise choosing endovascular coiling over neurosurgical clip-reconstruction [8, 9].

Average treatment effects found in RCTs apply to the study population as a whole and not to the individual patient [10]. Ideally, treatment decisions are based on the individualized treatment effects – or individualized treatment benefit. To estimate this, patient characteristics and their interaction with treatment have to be taken into account [10].

In the past, several studies developed models to predict outcome in aSAH patients, but none have focused on patient outcomes conditional on the choice of aneurysm treatment [11,12,13,14,15]. We hypothesize that the decision-making process could be improved by considering individualized treatment effects in terms of short-term functional outcome as well as the long-term durability of aneurysm treatment. Decision-making leading to a lower rate of rebleeding or retreatment while maintaining high probabilities of favorable functional outcome will benefit patients with aSAH. We aimed to develop and validate a prediction tool to predict benefit of endovascular coiling and neurosurgical clip-reconstruction in terms of short-term functional outcome and long-term durability of treatment in individual patients with aSAH.

Methods

We previously published the study protocol and statistical analysis plan [16]. We used data from the randomized International Subarachnoid Aneurysm Trial (ISAT, n = 2143) [3]. We adhered to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement reporting guidelines (Additional file 1; Supplemental Methods 1) [17].

Outcomes

We developed an ordinal regression model to predict short-term functional outcome (modified Rankin Scale; mRS) and a Cox regression model to predict time-to-rebleed-or-retreatment of the target aneurysm as an indicator of the durability of aneurysm treatment. We defined favorable functional outcome as an mRS score 0–2. In contrast with our initial statistical analysis plan, we used 2-month instead of 12-month mRS data. The reason for this change was to minimize the conditional dependence of the outcomes (the possibility that a rebleed or retreatment also affects the functional outcome or vice versa). Further, we used a Gaussian copula function to estimate the degree of residual correlation between the individual marginal probabilities of outcome (2-month favorable functional outcome and rebleed and retreatment within 10 years) in a joint model to determine if a separate interpretation of the probabilities is valid [18].

Candidate predictors

We pre-specified potential predictors based on a clinically driven approach by literature review and clinical expertise. We considered age, time-to-aneurysm-treatment, World Federation of Neurological Surgeons (WFNS) grade, CT Fisher grade, vasospasm at presentation, aneurysm location, aneurysm lumen size, aneurysm neck size, and aneurysm treatment as potential predictors of outcome.

The WFNS grade is a clinical severity score to predict outcome after SAH ranging from I to V, good to poor clinical condition respectively [19]. The CT Fisher grade is an imaging-based score classifying the extent of subarachnoid hemorrhage and is used to predict intracranial vasospasm. The score ranges from 1, no SAH, to 4, presence of intraventricular or intracerebral clots [20]. We collapsed aneurysm locations located in the posterior circulation into one category: “posterior aneurysm location”. Aneurysm neck size was collected categorically in ISAT (dichotomized as ≤ 4 mm and > 4 mm) and could therefore not be analyzed continuously. To enhance the potential clinical application, we dichotomized vasospasm at presentation into “present” or “absent”. Time-to-aneurysm-treatment was defined as the time difference between randomization and treatment in days. Treatment in the first 24 h was scored as “zero days”. Time-to-aneurysm-treatment was truncated at the 95th percentile (21 days, n = 35). Missing time-to-aneurysm-treatment values were imputed with the value of the 95th percentile. Any patient who had a target aneurysm that was not treated was excluded from analysis in the Cox regression model (n = 35, (2%)). If a patient both had a rebleed and was retreated we classified this as a “rebleed”.

Model development

The selection of predictors by univariable analysis for the multivariable model was based on a p-value of 0.20 to minimize overfitting and testimation bias [21]. We investigated non-linearity for continuous variables with restricted cubic splines. Based on clinical plausibility, we included (pre-specified) interactions of treatment assignment with age, aneurysm lumen size, aneurysm neck size, aneurysm location, and vasospasm at presentation. Further, we investigated the interaction between treatment assignment with the baseline risk of the outcome. The p-value for including a non-linear term was p < 0.05, and p < 0.01 for including an interaction term. Missing data were imputed with single imputation based on regression using the outcome, the candidate predictors with the addition of the patient’s sex. The proportion of missings was limited.

Model performance

We expressed model performance in terms of discrimination and calibration with the c-statistic and calibration plots. Because the c-statistic for ordinal models has a more complicated interpretation than for models with a binary outcome we also calculated a c-statistic for the prediction of favorable functional outcome.

Sensitivity analyses

We conducted two sensitivity analyses by re-running the model with separate outcomes (rebleed or retreatment) and with the 2-month and 12-month ordinal outcome (as originally intended) and evaluated the c-statistics.

Validation

We used bootstrapping for internal validation to estimate the degree of optimism in the c-statistics of the final models. Further, we conducted leave-one-cluster-out cross-validation. Centers were assigned to form 4 clusters of approximately equal sample size. We pooled the resulting cluster-specific c-statistics with random effects meta-analysis and report prediction intervals (PI) [22].

Benefit of treatment

For each patient, we predicted the probabilities of favorable functional outcome at 2 months and no rebleed or retreatment within 10 years follow-up with neurosurgical clip-reconstruction and with endovascular coiling. We derived treatment benefit for each patient – both in terms of favorable functional outcome at 2 months and no rebleed or retreatment within 10 years follow-up – by calculating the difference between outcome predictions of the two treatment modalities. For illustration, we described the patient characteristics of the population that had a large, ≥ 5% point, difference in treatment benefit of no rebleed or retreatment within 10 years with a negligible treatment benefit in terms of functional outcome. To assess the performance of the benefit predictions classical risk-prediction performance measures cannot be used. Comparing predicted benefit to observed benefit is not possible because the actual benefit of neurosurgical clip-reconstruction cannot be observed if a patient has been allocated to endovascular coiling (and vice versa). We used the c-for-benefit to assess the performance of the models in terms of benefit predictions [23, 24]. The c-for-benefit is defined as the proportion of all possible pairs of matched patient pairs with unequal observed benefit in which the patient pair receiving greater treatment benefit was also predicted to do so (Please see Additional file 1; Supplemental Table 1 for further explanation).

All statistical analyses were performed with R statistical software (version 4.1.1) using the rms (version 6.2.0), Hmisc (version 4.5.0), survival (version 3.3.1), mice (version 3.13.0), metafor (version 3.4.0), CalibrationCurves (version 0.1.2), HTEPredictionMetrics (version 0.1.1, https://github.com/CHMMaas/HTEPredictionMetrics) packages. To enhance the future application of the treatment benefit prediction models, we developed a web-based clinical prediction tool. This tool was developed with the shiny package (version 1.7.0).

Results

We used the full cohort of 2143 patients for the development of the model predicting short-term functional outcome and 2108 patients for the development model predicting long-term time-to-retreatment-or-rebleed (Additional file 1; Supplemental Table 2). In the full cohort, the mean age was 51.7 (SD 11.6) years and 63% were women (n = 1345). Sixty-nine percent (n = 1467) had a favorable outcome at 2 months. At the longest follow-up, 133 (6%) patients were retreated and 75 patients (4%) rebled.

The short-term functional outcome model included the predictors’ age, aneurysm treatment, time-to-aneurysm-treatment, aneurysm lumen size, aneurysm location, WFNS grade, CT Fisher grade, and vasospasm at presentation (Table 1). We found no evidence for non-linearity or interaction with treatment. The long-term durability of treatment model included the predictors’ age, aneurysm lumen size, aneurysm location, and aneurysm treatment (Table 1). We did not find significant interaction with treatment. Age was added to the model as a non-linear term.

Table 1 Main effects of the models predicting functional outcome and long-term durability of treatment
Full size table

The internally validated c-statistics were 0.64 (95% CI 0.63–0.67) for the model with the ordinal outcome, 0.75 (95% CI 0.71–0.77) for the prediction of favorable functional outcome, and 0.68 (95% CI 0.65–0.72) for the durability of treatment model. The pooled c-statistics after leave-one-out cross-validation were 0.72 (PI 0.67–0.77) for the prediction of favorable outcome at 2 months and 0.67 (PI 0.63–0.71) for the prediction of no retreatment or rebleed within 10 years follow-up. The models were calibrated well over the range with the majority of the observations (Additional file 1: Supplemental Figs. 1A-D and Supplemental Figs. 2A-D). We found a limited residual correlation of 0.21 (95% CI 0.13–0.29) between time-to-rebleed-or-retreatment and favorable functional outcome at 2 months indicating that no further modeling is necessary for dependency.

Sensitvity analyses

We did not observe any meaningful differences in the c-statistics in the sensitivity analysis by refitting the models with a separate outcome (rebleed or retreatment) or with a 2-month functional outcome versus a 12-month functional outcome.

Benefit of treatment

We found an average difference in the predicted probability of the 2-month favorable functional outcome of 6% (95% CI 5–10, range 3–10%, Fig. 1A), favoring endovascular coiling. The c-for-benefit of this model was 0.57 (95% CI 0.53–0.61).

Fig. 1
figure 1

A Predicted probabilities of favorable functional outcome. B And durability of treatment

Each dot represents an individual patient and his/her predicted probability of favorable functional outcome (mRS 0-2) or no rebleed or retreatment within 10 years with endovascular coiling and neurosurgical clip-reconstruction

Full size image

Further, we found an average difference in the predicted probability of no rebleed or retreatment within 10 years follow-up of 11% (95% CI 9–13, range 6–34, Fig. 1B), favoring neurosurgical clip-reconstruction. The c-for-benefit of this model was 0.57 (95% CI 0.50–0.63).

We identified 134 patients (6%) with a ≥ 5% point difference in the long-term durability of treatment favoring neurosurgical clip-reconstruction and with a negligible benefit of endovascular coiling in terms of short-term functional outcome. These patients had a mean age of 38 years (SD 9, Additional file 1: Supplemental Table 3) and had most often a WFNS grade I (97%), a CT Fisher grade 1–2 (83%), and no vasospasm at presentation (97%). Even though aneurysm lumen size, aneurysm location, and time-to-aneurysm-treatment were included in the models to predict outcome, they seem largely uninformative to identify patients that could benefit from choosing neurosurgical clip-reconstruction over endovascular coiling (Fig. 2A-H).

Fig. 2
figure 2

A-H Characteristics of population with large (≥ 5% point) benefit from neurosurgical clip-reconstruction

Abbreviations: ACA = anterior circulation artery, ACOM = anterior communicating artery, CT = computed tomography, ICA = internal carotid artery, MCA = middle cerebral artery, PCOM = posterior communicating artery, Post = aneurysms of the posterior circulation, WFNS = World Federation of Neurological Surgeons. Locations of aneurysms of the posterior circulation include: basilar artery, vertebral artery, superior cerebellar artery, anterior inferior cerebellar artery, posterior inferior cerebellar artery, and internal auditory artery. Each dot corresponds to a patient in the International Subarachnoid Aneurysm Trial. The dashed lines indicate a ≥ 5% point difference in predicted probability of favorable functional (modified Rankin Scale 0–2) at 2 months or the difference in predicted probability of no rebleed or retreatment within a 10-year follow-up. Because no interactions were included in the models, every patient in the ISAT cohort had benefit of endovascular coiling in terms of functional outcome and benefit of clip-reconstruction in terms of durability of treatment. The values on the axis are calculated by subtracting the predicted probabilities of outcome. X-axis: P(mRS < 3) with endovascular coiling – P(mRS < 3) with neurosurgical clip-reconstruction, and on the y-axis P(No event within 10 years) with neurosurgical clip-reconstruction – P(No event within 10 years) with endovascular coiling. This creates four quadrants: Upper left quadrant: Population which should be considered for neurosurgical clip-reconstruction instead of endovascular coiling because of negligible benefit in functional outcome and ≥ 5% point difference in durability of treatment favoring neurosurgical clip-reconstruction. Lower left quadrant: Treat according to current guideline recommendations. No ≥ 5% point difference in treatment benefit with both outcomes. Upper right quadrant: Treat according to current guideline recommendations. A ≥ 5% point difference in treatment benefit for both outcomes. Lower right quadrant: Treat according to current guideline recommendations. Only ≥ 5% point difference in treatment benefit from endovascular coiling in terms of functional outcome

Full size image

Consider the following aSAH patient: a 45-year-old woman, presenting with a WFNS grade I and CT Fisher grade 1 because of a rupture of a 4-mm anterior communicating artery aneurysm, and without vasospasm at angiography. She is treated on the third day. According to the models this patient has predicted probability of 95% with neurosurgical clip-reconstruction and 83% with endovascular coiling of no rebleed or retreatment within 10 years follow-up, meaning an absolute treatment benefit of 12% point in favor of neurosurgical clip-reconstruction (Fig. 3). In terms of a 2-month favorable functional outcome, this patient has a predicted probability of 90% with endovascular coiling and 87% with neurosurgical clip-reconstruction, meaning a 3% point absolute benefit in favor of endovascular coiling.

Fig. 3
figure 3

Visual representation of the web-based SHARP prediction tool

The SHARP prediction tool is accessible via https://sharpmodels.shinyapps.io/sharpmodels/. Please note that the SHARP prediction tool is purely illustrative

Full size image

Model presentation

We developed the web-based Subarachnoid Hemorrhage Benefit of Treatment Prediction (SHARP) tool. This tool can be used to calculate absolute estimates of short-term favorable functional outcome and long-term durability of aneurysm treatment with neurosurgical clip-reconstruction and endovascular coiling. The SHARP prediction tool is accessible via https://sharpmodels.shinyapps.io/sharpmodels/. The regression formulas are presented in the prediction tool. Presently, this tool is purely illustrative meant to inform an incentivize a discussion about optimal aneurysm treatment and has not undergone a medical device regulation registration procedure.

Discussion

We discovered that young patients who are in good clinical condition and have no extensive SAH or vasospasm at presentation benefit greatly from neurosurgical clip-reconstruction – compared to endovascular coiling – in terms of durability of treatment. At the same time, these patients did not benefit from endovascular coiling – compared to neurosurgical clip-reconstruction – because they had a high probability of favorable functional outcome regardless of the choice of treatment.

We did not find evidence for additive effects between treatment and the investigated predictors on functional outcome or durability of treatment. Because of this, the observed treatment benefit is purely risk-based. We applied a superior method to investigating heterogeneity of treatment effect than conventional subgroup analysis [2, 3, 25, 26]. First, because the tool’s risk-based benefit estimation is not hampered by the power issue of conventional subgroup analysis. Second, because a multivariable analysis is more reliable than a “one-variable-at-a-time” analysis.

Some decision problems require prediction tools to inform decision-making. For humans, it is impossible, to weigh a wide variety of patient characteristics and simultaneously consider non-linear effects. We developed models and the SHARP prediction tool that enables the prediction of individualized treatment benefit of endovascular coiling compared to neurosurgical clip-reconstruction accounting for short-term functional outcome and long-term durability of treatment for patients with aSAH. These estimates can be obtained relatively easily and could inform about the patient’s prognosis and support and incentivize a multidisciplinary discussion about optimal aneurysm treatment [27].

Limitations

Some limitations have to be taken into account. We developed our models on data that was published nearly two decades ago. It is suggested that, due to the increased experience of interventionalists and the introduction of new devices, current endovascular and neurosurgical practices have improved. Subsequently, the average functional outcome after aSAH may have improved and retreatment and post-interventional rebleed rates may have declined [28]. This would affect the generalizability of our study.

However, the guideline recommendations that shape current clinical practice are limited by the same generalizability issue. This was recently illustrated by a study that established a large degree of practice variability in the (aneurysm) treatment of aSAH [29]. Until a new trial investigating the safety and efficacy of endovascular versus neurosurgical aneurysm treatment, reflecting contemporary practice is conducted, generalizability will remain unclear. It is unlikely that such a trial will take place in the foreseeable future. The current study, based on state-of-the-art modeling techniques, aimed to improve decision-making with the best data available and provides some evidence-based nuance to one-size-fits-all policy advised by current guidelines.

The lack of contemporary data also complicates validation. To assess the external validity of the benefit predictions only randomized data with long-term follow-up suffices [23]. The next best alternative to fully independent external validation is internal–external cross-validation [30]. With internal–external cross-validation, the validity of the model is assessed over clusters within the development data (e.g., by study, center, study period, or country). Because of the temporal and geographical clustering of the ISAT data, we were able to review model performance over multiple dimensions. Because in ISAT, the patients had to be equally amenable to neurosurgical clip-reconstruction as to endovascular coiling patient inclusion depended on the interpretation of local stroke specialists. This created a different case-mix across the participating sites that strengthens our belief in the validity of the model across multiple settings. It must be noted that ISAT included European aSAH patients exclusively, and that generalizability towards another ethnic population must be investigated in the future.

Another limitation is that the performance of the risk prediction models was only modest. This can be explained by several factors. In the first place, Harrell’s c-statistic for an ordinal outcome is a conservative measure and cannot be interpreted equally as for a model with a binary outcome. Concerning the Cox model, censoring may have affected the rank ordering (i.e. patients with a short survival time can also have a low predicted risk because of censoring) [31]. In the second place, a trial population with stringent selection criteria leads to decreased heterogeneity in the study population which makes it more difficult two discriminate between patients [32].

To inform decision-making, we only considered predictors that are available at baseline. Especially in the durability of treatment model, in the future, performance may be improved by adding predictors that are associated with incomplete occlusion of the aneurysm. Examples of these predictors could be complex aneurysm morphology, arterial branches springing from the aneurysm dome, or aneurysm neck size as a continuous variable [7]. Presently, it is difficult to appreciate the performance of the models in terms of benefit predictions. As is often the case, the c-for-benefit was low, but this measure lacks adjusted benchmarks and cannot be interpreted similarly as the Harrell’s c-statistic [23]. In any case, it is reasonable to assume that reliable outcomes estimates will most likely lead to reliable benefit estimates.

When applying the model in practice it will be difficult to weigh short-term functional outcome against the long-term durability of aneurysm treatment. Moreover, we used a composite durability outcome even though retreatment and rebleeding obviously have different clinical consequences. We hypothesized that retreatment and rebleeding are part of the same continuum. Untreated revascularization or remnants can lead to future rebleeding. The sensitivity analysis showed that both outcomes were associated with equal predictors with a comparable effect size. To illustrate the clinical impact, we illustrated a specific population that had a large (≥ 5% point difference) benefit in durability from neurosurgical clip-reconstruction without a benefit in functional outcome from endovascular coiling.

Nevertheless, we stress that short-term functional outcome and long-term durability of aneurysm treatment should not be valued equally. Given that the results of this study are complicated to extrapolate to current practice, the risk and benefit predictions should support rather than determine decision-making. Durability of treatment could be taken into consideration in the absence of (clinically relevant) benefit of functional outcome. Additionally, the individualized prognostic estimates can serve as a benchmark when new complex endovascular techniques are considered over established techniques. As a next step, we need a decision model that integrates the effects on both outcomes to translate treatment benefit to quality-adjusted life expectancy. Such a model could further facilitate (shared) personalized decision-making.

Conclusions

In patients eligible for neurosurgical clip-reconstruction and endovascular coiling, on average, coiling leads to superior functional outcome. However, in some patients, only a negligible benefit of endovascular coiling over neurosurgical clip-reconstruction is expected, while at the same time, they do have much higher expected durability of treatment with neurosurgical clip-reconstruction compared to endovascular coiling. For these patients, best characterized as young and presenting in a favorable clinical condition, neurosurgical clip-reconstruction may be preferable. The SHARP prediction tool could support and incentivize a multidisciplinary discussion about personalized aneurysm treatment decision-making. Eventually, a new RCT is necessary to compare endovascular aneurysm treatment versus neurosurgical aneurysm treatment in a contemporary setting. Such a trial can be used to validate the proposed prediction tool.

Availability of data and materials

The data that support the findings of this study are available from the International Subarachnoid Aneurysm Trial (ISAT) investigators but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request. After publication, the R code of the models and the SHARP prediction tool will be made available and accessible via: https://github.com/WinkelJordi/SHARP.

Abbreviations

aSAH:

Aneurysmal subarachnoid hemorrhage

CT:

Computed tomography

ISAT:

International subarachnoid aneurysm trial

mRS:

modified Rankin Scale

PI:

Prediction interval

RCT:

Randomized controlled trial

SHARP:

Subarachnoid hemorrhage benefit of treatment prediction

WFNS:

World federation of neurological surgeons

References

  1. Koivisto T, Vanninen R, Hurskainen H, Saari T, Hernesniemi J, Vapalahti M. Outcomes of early endovascular versus surgical treatment of ruptured cerebral aneurysms. A prospective randomized study. Stroke. 2000;31(10):2369–77. https://doi.org/10.1161/01.str.31.10.2369.

  2. McDougall CG, Spetzler RF, Zabramski JM, et al. The barrow ruptured aneurysm Trial. J Neurosurg. 2012;116(1):135–44. https://doi.org/10.3171/2011.8.JNS101767.

    Article  PubMed  Google Scholar 

  3. Molyneux AJ, Kerr RS, Yu LM, et al. International subarachnoid aneurysm trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised comparison of effects on survival, dependency, seizures, rebleeding, subgroups, and aneurysm occlusion. Lancet. 2005;366(9488):809–17. https://doi.org/10.1016/S0140-6736(05)67214-5.

  4. Vanninen R, Koivisto T, Saari T, Hernesniemi J, Vapalahti M. Ruptured intracranial aneurysms: acute endovascular treatment with electrolytically detachable coils–a prospective randomized study. Radiology. 1999;211(2):325–36. https://doi.org/10.1148/radiology.211.2.r99ap06325.

    Article  CAS  PubMed  Google Scholar 

  5. Spetzler RF, McDougall CG, Zabramski JM, et al. Ten-year analysis of saccular aneurysms in the Barrow Ruptured Aneurysm Trial. J Neurosurg. 2019;132(3):771–6. https://doi.org/10.3171/2018.8.JNS181846.

    Article  PubMed  Google Scholar 

  6. Molyneux AJ, Birks J, Clarke A, Sneade M, Kerr RS. The durability of endovascular coiling versus neurosurgical clipping of ruptured cerebral aneurysms: 18 year follow-up of the UK cohort of the International Subarachnoid Aneurysm Trial (ISAT). Lancet. 2015;385(9969):691–7. https://doi.org/10.1016/S0140-6736(14)60975-2.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Campi A, Ramzi N, Molyneux AJ, et al. Retreatment of ruptured cerebral aneurysms in patients randomized by coiling or clipping in the International Subarachnoid Aneurysm Trial (ISAT). Stroke. 2007;38(5):1538–44. https://doi.org/10.1161/STROKEAHA.106.466987.

    Article  PubMed  Google Scholar 

  8. Connolly ES Jr, Rabinstein AA, Carhuapoma JR, et al. Guidelines for the management of aneurysmal subarachnoid hemorrhage: a guideline for healthcare professionals from the American Heart Association/american Stroke Association. Stroke. 2012;43(6):1711–37. https://doi.org/10.1161/STR.0b013e3182587839.

    Article  PubMed  Google Scholar 

  9. Steiner T, Juvela S, Unterberg A, et al. European Stroke Organization guidelines for the management of intracranial aneurysms and subarachnoid haemorrhage. Cerebrovasc Dis. 2013;35(2):93–112. https://doi.org/10.1159/000346087.

    Article  PubMed  Google Scholar 

  10. Kent DM, Paulus JK, van Klaveren D, et al. The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement. Ann Intern Med. 2020;172(1):35–45. https://doi.org/10.7326/M18-3667.

    Article  PubMed  Google Scholar 

  11. Witsch J, Kuohn L, Hebert R, et al. Early prognostication of 1-year outcome after subarachnoid hemorrhage: The fresh score validation. J Stroke Cerebrovasc Dis. 2019;28(10): 104280. https://doi.org/10.1016/j.jstrokecerebrovasdis.2019.06.038.

    Article  PubMed  Google Scholar 

  12. Witsch J, Frey HP, Patel S, et al. Prognostication of long-term outcomes after subarachnoid hemorrhage: The FRESH score. Ann Neurol. 2016;80(1):46–58. https://doi.org/10.1002/ana.24675.

    Article  PubMed  Google Scholar 

  13. Mascitelli JR, Cole T, Yoon S, et al. External Validation of the Subarachnoid Hemorrhage International Trialists (SAHIT) Predictive Model Using the Barrow Ruptured Aneurysm Trial (BRAT) Cohort. Neurosurgery. 2020;86(1):101–6. https://doi.org/10.1093/neuros/nyy600.

    Article  PubMed  Google Scholar 

  14. Jaja BNR, Saposnik G, Lingsma HF, et al. Development and validation of outcome prediction models for aneurysmal subarachnoid haemorrhage: the SAHIT multinational cohort study. BMJ. 2018;360:j5745. https://doi.org/10.1136/bmj.j5745.

  15. de Winkel J, Cras TY, Dammers R, et al. Early predictors of functional outcome in poor-grade aneurysmal subarachnoid hemorrhage: a systematic review and meta-analysis. BMC Neurol. 2022;22(1):239. https://doi.org/10.1186/s12883-022-02734-x.

  16. de Winkel J, Roozenbeek B, Dijkland SA, et al. Endovascular versus neurosurgical aneurysm treatment: study protocol for the development and validation of a clinical prediction tool for individualised decision making. BMJ Open. 2022;12(12):e065903. bmjopen-2022–065903. https://doi.org/10.1136/bmjopen-2022-065903.

  17. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. Br J Surg. 2015;102(3):148–58. https://doi.org/10.1002/bjs.9736.

    Article  CAS  PubMed  Google Scholar 

  18. Kotz S, Read CB, Banks DL. Encyclopedia of statistical sciences. Wiley; 1997.

  19. Report of World Federation of Neurological Surgeons Committee on a Universal Subarachnoid Hemorrhage Grading Scale. J Neurosurg. 1988;68(6):985–6. https://doi.org/10.3171/jns.1988.68.6.0985.

  20. Fisher CM, Kistler JP, Davis JM. Relation of cerebral vasospasm to subarachnoid hemorrhage visualized by computerized tomographic scanning. Neurosurgery. 1980;6(1):1–9. https://doi.org/10.1227/00006123-198001000-00001.

    Article  CAS  PubMed  Google Scholar 

  21. Steyerberg EW. Clinical prediction models : a practical approach to development, validation, and updating. Statistics for biology and health. Springer; 2009:xxviii, 497.

  22. van Klaveren D, Steyerberg EW, Perel P, Vergouwe Y. Assessing discriminative ability of risk models in clustered data. BMC Med Res Methodol. 2014;14:5. https://doi.org/10.1186/1471-2288-14-5.

  23. van Klaveren D, Steyerberg EW, Serruys PW, Kent DM. The proposed “concordance-statistic for benefit” provided a useful metric when modeling heterogeneous treatment effects. J Clin Epidemiol. 2018;94:59–68. https://doi.org/10.1016/j.jclinepi.2017.10.021.

    Article  PubMed  Google Scholar 

  24. Maas CCHM, Kent DM, Hughes MC, Dekker R, Lingsma HF, van Klaveren D. Performance metrics for models designed to predict treatment effect. medRxiv. 2022:2022.06.14.22276387. https://doi.org/10.1101/2022.06.14.22276387.

  25. Catapano JS, Louie CE, Lang MJ, et al. Outcomes in a case series of elderly patients with aneurysmal subarachnoid Hemorrhages in the Barrow Ruptured Aneurysm Trial (BRAT). World Neurosurg. 2020;139:e406–11. https://doi.org/10.1016/j.wneu.2020.04.007.

    Article  PubMed  Google Scholar 

  26. Catapano JS, Nguyen CL, Frisoli FA, et al. Small intracranial aneurysms in the Barrow Ruptured Aneurysm Trial (BRAT). Acta Neurochir (Wien). 2021;163(1):123–9. https://doi.org/10.1007/s00701-020-04602-4.

    Article  PubMed  Google Scholar 

  27. Kent DM, Rothwell PM, Ioannidis JP, Altman DG, Hayward RA. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials. 2010;11:85. https://doi.org/10.1186/1745-6215-11-85.

  28. Catapano JS, Labib MA, Srinivasan VM, et al. Saccular aneurysms in the post-Barrow Ruptured Aneurysm Trial era. J Neurosurg. 2021:1–8. https://doi.org/10.3171/2021.8.JNS211060.

  29. de Winkel J, van der Jagt M, Lingsma HF, et al. International Practice Variability in Treatment of Aneurysmal Subarachnoid Hemorrhage. J Clin Med. 2021;10(4). https://doi.org/10.3390/jcm10040762.

  30. Van Calster B, Steyerberg EW, Wynants L, van Smeden M. There is no such thing as a validated prediction model. BMC Med. 2023;21(1):70. https://doi.org/10.1186/s12916-023-02779-w.

  31. van Klaveren D, Gonen M, Steyerberg EW, Vergouwe Y. A new concordance measure for risk prediction models in external validation settings. Stat Med. 2016;35(23):4136–52. https://doi.org/10.1002/sim.6997.

    Article  MathSciNet  PubMed  PubMed Central  Google Scholar 

  32. Austin PC, Steyerberg EW. Interpreting the concordance statistic of a logistic regression model: relation to the variance and odds ratio of a continuous explanatory variable. BMC Med Res Methodol. 2012;12:82. https://doi.org/10.1186/1471-2288-12-82.

Download references

Acknowledgements

The authors wish to thank the authors of the International Subarachnoid Aneurysm Trial for supplying the data necessary to conduct this study and to thank Daan Nieboer from the Erasmus MC University Medical Center Rotterdam for his help with developing the Gaussian copula function.

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Department of Neurology, Erasmus MC University Medical Center Rotterdam, 40 Doctor Molewaterplein, P.O. Box 2405, 3015 GD, Rotterdam, Zuid-Holland, The Netherlands

    Jordi de Winkel, Bob Roozenbeek, Simone A. Dijkland & Diederik W. J. Dippel

  2. Department of Public Health, Erasmus MC University Medical Center Rotterdam, Rotterdam, Zuid-Holland, The Netherlands

    Jordi de Winkel, David van Klaveren & Hester F. Lingsma

  3. Department of Neurosurgery, Erasmus MC University Medical Center Rotterdam, Rotterdam, Zuid-Holland, The Netherlands

    Ruben Dammers

  4. Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Center Rotterdam, Rotterdam, Zuid-Holland, The Netherlands

    Pieter-Jan van Doormaal

  5. Department of Intensive Care Adults, Erasmus MC University Medical Center Rotterdam, Rotterdam, Zuid-Holland, The Netherlands

    Mathieu van der Jagt

Authors
  1. Jordi de Winkel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bob Roozenbeek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Simone A. Dijkland
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruben Dammers
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pieter-Jan van Doormaal
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mathieu van der Jagt
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. David van Klaveren
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Diederik W. J. Dippel
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Hester F. Lingsma
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Study Conceptualization (S.A.D., D.W.J.D, D.v.K., H.F.L, B.R., J.d.W.); Literature review (S.A.D., J.d.W.); Formal analysis (J.d.W.); Visualization (J.d.W.); Writing original draft (J.d.W.); Critical review of the manuscript (R.D., S.A.D., D.W.J.D., P-J.v.D., M.v.d.J, D.v.K, H.F.L., B.R., J.d.W.); supervision (H.F.L., B.R.). All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jordi de Winkel.

Ethics declarations

Ethics approval and consent to participate

The study was conducted according to the guidelines of the Declaration of Helsinki and informed consent was obtained from all subjects and their legal guardian(s) by the authors of the International Subarachnoid Aneurysm Trial that supplied the data necessary to conduct this study. The institutions’ medical ethical research committee (MERC) of the Erasmus MC University Medical Center Rotterdam approved the study protocol under the exemption category and waived the need for written informed consent (MEC-2020–0810).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

 Supplemental Methods 1. TRIPOD Checklist. Supplemental Table 1. Brief summary of the statistical concepts discussed in this paper. Supplemental Table 2. Baseline characteristics. Supplemental Figures 1A-D. Internal-external calibration plots of the model prediction short-term 2-month favorable functional outcome (modified Rankin Scale score 0-2). Supplemental Figures 2A-D. Internal-external calibration plots of the model predicting long-term within 10-year durability of treatment (no rebleed or retreatment). Supplemental Table 3. Baseline characteristics of the derivation cohort and the population that may benefit from neurosurgical clip-reconstruction.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Winkel, J., Roozenbeek, B., Dijkland, S.A. et al. Personalized decision-making for aneurysm treatment of aneurysmal subarachnoid hemorrhage: development and validation of a clinical prediction tool. BMC Neurol 24, 65 (2024). https://doi.org/10.1186/s12883-024-03546-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12883-024-03546-x

Keywords

BMC Neurology

ISSN: 1471-2377

Contact us