Diagnostic accuracy of the Eurotest for dementia: a naturalistic, multicenter phase II study

Participants

Between April and July, 2004, we enrolled patients who fulfilled the following criteria: 1) age older than 60 years; 2) followed continuously by the researchers who participated in this study for any clinical neurological reason, not limited to cognitive impairment; and 3) had an established clinical and cognitive diagnosis before being assessed using the Eurotest. Each researcher selected participants prospectively following a systematic, predetermined protocol: each participant was chosen on the basis of being the first patient seen by the researcher on any given day and fulfilling the inclusion criteria. Only one participant per day was included by each researcher, and each recruited at least 25 participants; this minimum of 25 was set in order to assure that each sub-sample was representative of the researcher's clinic. Being a phase II study, it was not considered necessary to recruit a prospective cohort of newly referred patients. Patients were excluded if they declined to participate or had previously participated in this study. The patient's assessment included a comprehensive clinical, neuropsychological and functional evaluation, as well as neuro-imaging and laboratory studies, according to recommendations by the Spanish Neurological Society [24]. The assessment instruments were selected by the clinical investigators on the basis of their clinical experience and practice. The patients' condition was classified into dementia (DEM) or no dementia (NoDEM) according to the DSM-IV criteria [23]. The diagnosis of dementia required: multiple cognitive deficits manifested by both memory impairment and other cognitive disturbances; significant functional decline from the previous level of functioning due to those cognitive deficits; and the manifestation of these deficits not exclusively during the course of a delirium.

Procedure

The Trans-Eurotest Study presents a cross-sectional multi-center naturalistic phase II study for diagnostic tests [21, 22] carried out in general neurology clinics. The Eurotest was administered at the end of a regular clinical interview, and its score did not influence the previous clinical or cognitive diagnosis. The patient's assessment included a comprehensive clinical and neuropsychological evaluation as well as neuro-imaging and laboratory studies.

The following variables were recorded for all participants: sex, age, literacy [as literate or illiterate (unable to read or write, or able to read or write only with difficulty)], completed primary education (no/yes), frequency with which they handled money (daily/not daily), subjective skills in handling money (poor/good), Global Deterioration Scale (GDS) stage (1: No cognitive decline (CD), normal; 2: very mild CD, subjective complaint of memory deficit, age associated memory impairment; 3: mild CD, mild cognitive impairment; 4: moderate CD, mild dementia; 5: moderately severe CD, moderate dementia; 6: severe CD, moderately severe dementia; and 7: very severe CD, severe dementia) [25], clinical diagnosis, the likelihood of a clinical diagnosis inducing cognitive impairment (no/yes), use of medication that could potentially affect cognitive performance (no/yes), results of the Eurotest and time needed to complete it, and results on a verbal (semantic) fluency test (sVFT; naming animals during one minute).

The instrument

The Eurotest [20] consists of three parts (see Additional File 1). The first part assesses knowledge of the different coins and bills available. One point is scored for each correct answer, and one point is subtracted for each wrong answer; the final score ranges from 0 to 15 points. The second part assesses the person's performance on five arithmetical tasks of increasing difficulty with 11 coins (counting, making change, adding, dividing by two, and dividing by three). Two points are scored for each correct answer on the first try, and one point is scored for each correct answer on the second try. The maximum time allowed for each task is 1 minute, and the final score ranges from 0 to 10 points. The third part assesses the person's recall of the number and type of coins used before, and the total amount of money involved. The final score ranges from 0 to 10 points. The total overall score on the Eurotest thus ranges from 0 to 35 points.

Between the second and the third part of the test, the sVFT was administered as a distraction task; the score on this test did not count for the final score on the Eurotest.

Statistical analysis

The results for the two groups were compared with Student's t test for continuous variables and the chi-squared (χ²) test for categorical variables. Diagnostic accuracy was assessed as area under the receiver operating characteristic (ROC) curve (aROC); the clinical diagnosis of dementia was the external gold standard. Sensitivity (Sn), specificity (Sp), and likelihood ratios were calculated for the total score on the Eurotest; the best cut-off score was identified as that which maximized Sn+Sp. The influence of socio-demographic, educational and clinical variables on the Eurotest score (the dependent variable) was determined by multiple linear regression analysis. The diagnostic accuracy of the Eurotest was analyzed by logistic regression, in which the presence of dementia (vs no dementia) was the dependent variable and the result on the Eurotest was the predictive variable. The effects of socio-demographic and educational variables were controlled for by comparing adjusted and non-adjusted models to determine associations with these variables and their possible confounding effects on the diagnostic accuracy. To assess convergent validity, we calculated partial correlations (controlling for socio-demographic and educational variables) between the Eurotest, scores on the sVFT, and GDS stage. The diagnostic accuracy of the Eurotest and the sVFT were contrasted by comparing their aROC from the same patients [26]. All parameters were calculated with their 95% confidence intervals (CI), and all the comparisons were two-tailed with an α error of 0.05. All statistical analyses were performed with the SPSS v.11.5 and MedCalc v.7.0 packages for Windows.

Ethical and methodological considerations

The study protocol was approved by the Torrecárdenas Hospital Institutional Review Board. Informed consent was obtained from all participants as well from the caregivers of the persons in the dementia group.

Both the Trans-Eurotest Study and this report have been conducted adhering to the STARD recommendations for diagnostic studies [27] (see Additional File 2).

Applicability and validity of the Eurotest

The results of the Trans-Eurotest Study show that the Eurotest is rapid and simple to administer, and is a useful, valid instrument that can be applied to patients who are illiterate. The diagnostic accuracy of the Eurotest is not influenced by socio-demographic or educational characteristics. The external validity of these conclusions is supported by the nature and design of the present study. The manner in which participants were recruited, the lenient inclusion criteria, the lack of exclusion criteria and the low rate of refusal to participate speak in favor of the study's pragmatic and naturalistic nature. In fact, more than half of the participants initially included had characteristics that would have made them ineligible for some related studies (e.g., conditions potentially able to induce cognitive impairment, use of medication potentially able to affect cognitive performance, or cognitive impairment without dementia). Moreover, the varied distribution of patients across different GDS stages is further assurance that the full spectrum of cognitive impairment was represented in our sample, from normality to severe dementia, including intermediate stages, particularly those representing cognitive impairment without dementia or subjective complaints of loss of memory. The participation of multiple centers and the number and varied backgrounds of the participating researchers further ensure that the spectrum of professional neurologists practicing in Spain was broadly represented. Thus the characteristics of the study, the researchers, and the patients guarantee the study's external validity, and thus the generalization of our results [28].

Scarcely 2% of the patients in this study were unable to complete the Eurotest in the time allowed. Because the test evaluated performance on day-to-day tasks that patients are usually familiar with, the test was readily accepted by most patients including those who were illiterate or whose level of education was very low. This fact contrasts with tests that involve paper-and-pencil tasks such as the Clock Test. Although these tasks can in theory be completed by illiterate persons, they are not well accepted by them [15]. Another feature that makes the Eurotest simple to administer is that, unlike most other instruments recommended by the AAN, it requires no cards, pictures or other objects, and no record sheets -only coins. With practice, it can be administered without a score sheet, making the Eurotest useful for patients who are hospitalized or who are otherwise unable to come to the neurology clinic. The Eurotest is short and requires less time than the 7 MS [14] or the MMSE [13]. Although the difference in time needed to complete the test differed significantly between persons with and without dementia, this difference was not relevant in practical terms.

The results of the Eurotest are not influenced by socio-demographic or educational variables such as level of literacy, or level of education. This is a major advantage over other available instruments, as the scores do not need to be adjustment for these variables. Moreover, neither the socio-demographic nor the educational factors improve the ability of the test to discriminate between patients with and without dementia.

Few instruments can document this independence from socio-demographic factors. The recently-described Prueba Cognitiva Leganés (PCL), a test that can be given to illiterate persons and whose results are not influenced by educational level, required much longer to complete (11.5 ± 3.2 minutes) despite the fact that 28.8% of the sample was excluded because of impairments potentially able to interfere with the test [29]. In contrast, the Eurotest took much less time to complete (8.2 ± 2.0 minutes) despite the fact that we excluded none of the patients because of cognitive impairment, and only 1.9% failed to complete the test because of cognitive limitations. The results of the MIS are not influenced by educational level [8], but this instrument cannot be used with illiterate persons, and has the further drawback of evaluating only memory. The diagnostic accuracy of the Eurotest was better that that of the sVFT in the same sample of patients and within the range of accuracy values found for other widely-used tests in other samples [20], although direct comparison of these findings with the present results would be inappropriate. It should be emphasized that the Eurotest achieved good diagnostic accuracy despite the fact that our sample of patients included persons who had cognitive impairment without dementia.

However, these results were less clear-cut than those recently reported for the 7 MS [30] and the PCL [29] in a sample of Spanish patients with a low level of education. The reason may lie in the facts that the refusal rate in these two studies, which involved the same sample, was high (27%), and that more than 20% of the participants were excluded because of sensory impairment. These figures contrast with the low rate of refusal to participate in the present study (2.1%), and with the fact that we did not exclude patients because of sensory impairment or for any reason other than refusal to participate. It should nevertheless be noted that these instruments require more than 10 minutes to administer, and are much more complex to administer and to score.

The structure of the Eurotest, which includes items intended to evaluate knowledge, calculation ability and recall, ensures appropriate content validity and face validity. The test also evaluates money handling ability, an important aspect of the patient's functional capacity. Although deterioration of money handling ability is a criterion in most universally accepted instruments used to diagnose dementia, this skill has not previously been included in any screening test. Ecological validity of the Eurotest is ensured by the every-day nature of the tasks and materials, which avoid making patients feel patronized, embarrassed or apprehensive. Adequate construct validity is ensured by the significant correlation between the Eurotest score and the GDS, a measure of the severity of deterioration that covers the full spectrum of cognitive impairment from normality to advanced dementia. Further evidence of construct validity is the correlation between the Eurotest scores and the sVFT, a widely used screening test.

Strength and limitations of the study

Among the strengths of this study are its sample size, external validity, consistency in the findings, and the fact that the diagnoses had been established previously and were not influenced by the test results. However, a few weaknesses of the study should be pointed out. The participation of many different researchers may have led to differences in how the diagnostic criteria were interpreted, and to possible misclassification bias. This source of bias was probably mitigated by the facts that all researchers were highly experienced practitioners, and that all based their diagnoses on a comprehensive clinical and neuropsychological evaluation, neuro-imaging and laboratory studies, and other widely used and generally accepted criteria. This limitation could in fact be considered a strength if we consider that the diagnoses used as the gold standard were those which were actually on record for the patients. Our patients, therefore, were exposed to all medical (treatment) and social consequences (restriction, protection, etc.) arising from their diagnosis, a fact that no doubt consequently lent a high degree of "consequential validity" [31]. Some of the patients with dementia had very recently been diagnosed whereas others had been treated over different periods of time, but the information on the duration of cognitive complaints or impairments was not recorded; we acknowledge the absence of this aspect of the representativeness of the cohort.

It may be argued as a limitation that the euro currency has been in use only since January 2001, 3 years before this study was carried out, and some non-demented patients had possibly not yet achieved a skillful handling of the new currency leading to some false positive results. According to our clinical experience this is very unusual; moreover, the fact of not acquiring such skills over that time period may reflect some cognitive impairment interfering with learning. Nevertheless, the false positive cases, if any, would count against the diagnostic accuracy of the Eurotest; over the course of time the number of any false positives would be expected to decrease, meaning that further improvements in specificity could be expected.

The brief period between the appearance of the euro as legal tender to the time of the assessment makes it difficult to determine whether the lower scores obtained by persons with dementia were the result of loss of skills possibly acquired shortly after introduction of the common currency, or whether the ability of some patients to learn the new currency was limited by impairments already present in 2001. This distinction, on which the present study sheds no light, may have important implications for the future usefulness of the Eurotest. Recent observations have suggested that financial abilities are impaired before other abilities are affected [32–34], so that testing money handling skills may be useful for the early detection of cognitive impairment or dementia.

The cross-sectional nature of a phase II study, such as the present study, did not allow us to evaluate the predictive power of the Eurotest. A further important limitation is the fact that the researchers who administered the test were not blinded to the patients' clinical diagnosis, and this may have biased how the test was scored. These limitations are characteristic of phase II studies of diagnostic instruments. Phase III studies should be planned so that the predictive power of the Eurotest can be accurately determined in prospective, independent studies in which experimenters are blind to the patients' diagnosis [21, 22, 35].