Validity and reliability of the Pain Assessment in Impaired Cognition 15 (PAIC15) observation scale in persons with aphasia

Abstract

Background

The use of self-report pain scales in persons with aphasia can be challenging due to communication and cognitive problems, while for assessing pain self-report pain is considered the gold standard (Harrison RA, Field TS. Post stroke pain: identification, assessment, and therapy. Cerebrovasc Dis. 2015;39(3–4):190–201.). An observational scale may be used as an alternative. This study examines the validity and reliability of the observational Pain Assessment in Impaired Cognition (PAIC15) scale in persons with aphasia.

Methods

Persons with aphasia were observed during rest and transfer by two observers using the PAIC15. The PAIC15 comprises 15 items covering the three domains of facial expressions, body movements, and vocalizations. When able, the participant completed four self-report pain scales after each observation. The observations were repeated within one week. For criterion validity, correlations between the PAIC15 and self-report pain scales were calculated and for construct validity, three hypotheses were tested. Reliability was determined by assessing internal consistency, and intra- and interobserver agreement.

Results

PAIC15 observations were obtained for 71 persons (mean age 75.5 years) with aphasia. Fair positive correlations (rest: 0.35–0.50; transfer: 0.38–0.43) were reported between PAIC15 and almost all self-report pain scales. Results show that significantly more pain was observed in persons with aphasia during transfer than during rest. No differences were found for observed pain between persons with aphasia who use pain medication and those without, or persons who have joint diseases compared to those without. Results showed acceptable internal consistency. Intra- and interobserver agreement was high for most PAIC15 items, particularly for the domains body movements and vocalizations during rest and transfer.

Conclusions

Recognition of pain in persons aphasia using the PAIC15 showed mixed yet promising results.

Introduction

Self-report pain scales are commonly used to assess pain in patients with aphasia. Examples are the the Numerical Rating Scale (NRS) [1], the Visual Analog Scale (VAS) [2] and Faces Pain Scale (FPS) [3]. Self-report pain scales require the person to be able to understand verbal and written instructions and to apply this information in his or her response, which limits the use in persons with aphasia. Persons with moderate to severe aphasia are also often excluded from pain research, which makes interpretation of applicability, usefulness and best practices in pain assessment in aphasia difficult, although very relevant [4, 5]. However, stroke patients with mild to moderately-severe aphasia have pain just as often as stroke patients without aphasia (e.g. due to shoulder pain and central pain) [6].

Smith and Bottemiller [7] found that 14% of stroke patients were not able to complete the FPS or NRS. Capacity to complete these scales was associated with the severity of stroke and severity of aphasia. Most studies focused on patients with mild to moderate aphasia [8]. Also, despite varied self-report pain scales, stroke patients are less likely than age-matched controls to be able to complete these pain scales [9, 10]. Evidently, an appropriate alternative method of assessment of the presence of pain in persons with aphasia who are unable to self-report is needed. An alternative to self-report could be the observation of a person’s behavior, as is common in patients with cognitive impairment [11, 12]. Observational pain scales have been used successfully as an alternative to self-report pain scales in people with advanced dementia [13].

In 2011, a European Cooperation in Science and Technology (EU-COST) initiative collaborated to improve pain assessment in persons with impaired cognition. This international multidisciplinary team of experts from 16 countries developed a universal meta-tool for the assessment of pain in persons with cognitive impairment. This meta-tool, the Pain Assessment in Impaired Cognition (PAIC15), is an observational instrument that includes the best items from existing pain scales to observe pain in persons with impaired cognition. The PAIC15 has shown satisfactory psychometric qualities in patients with impaired cognition, mostly with dementia [14, 15]. This pain observation instrument is available in 10 languages and comes with an internet-based E-learning module in three languages (German, Dutch, and English: https://paic15.com/en/e-training-en/). The PAIC15 is therefore a potentially suitable alternative for assessing pain in patients who are unable to self-report, such as those with aphasia [16]. This study aims to answer the following research question: ‘What is the validity and reliability of the Dutch version of PAIC15 in persons with aphasia?’

Methods

Study design

The current study was an observational cohort study to determine the validity and reliability of the Dutch version of PAIC15 in persons with aphasia. Persons with aphasia were observed using PAIC15 during rest and transfer. Rest situations could be lying in bed or sitting in a (wheel)chair. Transfer situations include physical moves from bed to (wheel)chair, repositioning in bed or a short walk. Observations were conducted by two observers and repeated within one week. The data were collected during the COVID19-pandemic between May 2019 and July 2021.

Participants

Speech and language therapists from 19 nursing home organizations in the Netherlands invited the persons with aphasia to participate in the study. The nursing home organizations participated in the University Network for the Care sector – South Holland (UNC-ZH). Further, we used personal networks to invite nursing homes to participate in the study. Inclusion criteria were: residing in a nursing home in a geriatric rehabilitation department or a unit for patients with chronic physical impairments, age 18 years or older, sufficient comprehension of the Dutch or English language before onset of aphasia, and diagnosed with aphasia regardless of cause or severity. A score of ≤ 68 on the ScreeLing [17] or ≥ 7 on the TokenTest [18] implies aphasia. If diagnostic examination was not possible, the speech and language therapist’s clinical judgement was decisive. Persons were excluded if they had a delirium, severe psychiatric disease, dementia, or a life expectancy ≤ 6 months according to the primary responsible physician.

Instruments

Questionnaires 1 and 2

Characteristics of persons with aphasia were assessed with two questionnaires. A informal caregiver or legal representative or the speech and language therapist, if possible together with the person with aphasia, completed the brief questionnaire 1 with questions about persons, hand dominance, and length of stay in the nursing home. Questionnaire 1 is showed in Additional file 1 [see Additional file 1]. The speech and language therapist collected demographic characteristics and reported other more medical characteristics of the aphasia and pain treatment using questionnaire 2. Questionnaire 2 is showed in Additional file 2.

Pain observation scale

Pain symptoms were observed for five to a maximum of ten minutes using the validated Dutch version of the PAIC15 [15, 16]. The PAIC15 includes fifteen items; five in each of the three domains of facial expressions, body movements and vocalizations. Scoring options are ‘not at all’ (0), ‘slight degree’ (1), ‘moderate degree’ (2), ‘great degree’ (3) and ‘not scoreable’ (X). For example, the first item ‘frowning’ is described as moving eyebrows downwards and contracting them. ‘Not at all’ is scored when frowning does not occur during the observation. If this item cannot be assessed (e.g., the person turns their head away), it is rated as: ‘not scoreable’. ‘Slight degree’ (1) is scored when frowning is observed but only briefly or with little intensity; ‘great degree’ (3) when frowning is observed frequently or continuously, ‘moderate degree’ (2) when this item is not constantly observed, but more frequent than briefly. The PAIC15 e-learning provides clear instructions on how to score each item.

Summed total scores range from 0 to 45; 0–15 for each domain. For the statistical analysis, all scores X (not scorable) were regarded as 0 [19, 20]. The observers were students of clinical neuropsychology or medicine, and nurses or speech and language therapists. They received 1.5 h of instruction and completed the PAIC15 e-learning (https://paic15.com/en/e-training-en/).

Self-report pain scales

The four self-report pain scales used were: the vertical NRS, VAS, FPS and a combined scale. The NRS ranges from 0 (no pain) to 10 (worst pain imaginable) (Hjermstad, 2011). The VAS consists of a 10-centimetre line with extremes labelled ‘no pain’ and ‘unbearable pain’. The person is asked to locate the pain intensity on the line. Half a centimetre is rounded up to whole numbers, e.g., 3.5 is counted as 4 centimetres [2]. The FPS comprises six coloured cartoon-faces with expressions no pain (dark green smiling face) to worst pain (dark red sad face) with the values 0, 2, 4, 6, 8, and 10 (Kim, 2006). An additional file shows the sel-report combined pain scale. This combined scale combines the self-report pain scales FPS and NRS [see Additional file 3]. This scale consists of the numbers zero to ten, coloured smiley faces, and written expressions of pain displayed along a vertical line. All self-report pain scales were offered in a vertical form for use in case of visual problems such as neglect or hemianopsia post stroke. The order of the first three scales was randomized, and the final self-report scale was always the combined scale.

Procedure

Questionnaires 1 and 2 were completed on paper before the observations and returned in a closed envelope. Persons with aphasia were observed during rest and transfer twice within 7 days by the same two observers. The observations were performed by trained research assistants who were not familiar with the person with aphasia.

Each observation was performed by two observers (A and B) independently (blinded). The observers were also blind with respect to the questionnaires and self-report pain scales. First, the participant’s language comprehension was checked using the FPS. The participant was asked: ‘Imagine you have no pain now, could you indicate which face on this scale fits this experience?’ and ‘Imagine you have a severe headache at this moment, which face on this scale fits this situation?’. If these questions were answered correctly, language comprehension to complete the self-report pain scales was assumed to be sufficient. If these questions could not be answered and self-report pain scales could not be completed, only PAIC15 was used during the observations. Next, the observers observed the participant for a minimum of 5 and a maximum of 10 min during rest and completed the PAIC15 form independently. Afterwards, if applicable, the participant completed the four self-report pain scales. This procedure was repeated during transfer. The participants were observed during transfer for a minimum of 5 min, even when the transfer sometimes took less time. The procedure was repeated within 7 days by the same observers. After both observations on measurement 1, the observers discussed the independent observations during rest and transfer and jointly completed a new observation form. This new observation form, with the consensus scores of PAIC15, was completed to minimize the risk of behaviour being overlooked and for quality purposes. If the observation during transfer was carried out first, the observation during rest took place after 30 min, to prevent the transfer influencing the observation during rest.

Statistical analysis

Descriptive statistics of the PAIC15

First, we perform general descriptive statistics of the PAIC15 consensus scores and the self-report pain scales if these were used. Because of non-normal distribution, data were expressed as medians with interquartile range (IQR). Second, the presence of responses of the individual PAIC15 items of the consensus scores were examined, and reported in percentages, during rest and transfer. Floor or ceiling effects are defined as ≥ 15% of PAIC15 total scores scored the lowest (0: not at all) or highest possible score (3: great degree) [21]. More than 5% missing scores of items per observation form were discussed, reported, and are not imputed. No PAIC15 observation form and no person was excluded.

Criterion validity

Regarding criterion validity, we expected moderate correlations between PAIC15 and the four self-report pain scales. Because the data was not normally distributed, Spearman’s correlation coefficient and a 95% confidence interval (CI) were used to calculate the correlations between the PAIC15 consensus scores and the four self-report pain scores of measurement 1 in order to determine criterion validity. To describe the strength of the correlation we used: less than 0.30 is poor, 0.3 to 0.5 is fair, 0.6 up to 0.8 moderately strong, and 0.80 and higher is very strong (Chan, 2003). See Table 1 for the definitions of types of validity adapted from COnsensus based Standards for the selection of health Measurement INstruments (COSMIN) as applied in this study [22].

Table 1 Definitions of types of validity

Construct validity

To determine construct validity, 3 hypotheses were tested:

• 1) More pain is expected during transfer compared to rest in persons with aphasia.

To assess the degree to which the PAIC15 is capable of measuring pain in persons with aphasia, we compared results of the PAIC15 between rest and transfer. Similar research in persons with dementia reported more observed pain during ADL compared to rest [23,24,25].

• 2) More pain is expected when persons with aphasia used pain medication compared to persons who did not use pain medication.

Also, research on pain in dementia reports more observed pain in persons who used pain medication compared to persons who do not used pain medication [26, 27] Persons still experience pain, even when they receive pain medication. Additionally, a study of hospitalized persons with dementia (n = 108) who experienced pain (assessed with Pain Assessment in Advanced Dementia (PAINAD)) found that 60% of those persons had received pain medication compared to 40% who did not receive pain medication (Boltz et al., 2021).

• 3) More pain is expected in persons with aphasia who have joint disease such as osteoarthritis or rheumatism versus those without joint disease.

Osteoarthritis was the most common joint disease, and joint pain was among the most frequent pain syndromes in Europe [28,29,30]. It is expected that persons with aphasia and joint disease will have more pain than persons without joint disease, due to the risk of increased pain from joint problems and the difficulty in communication due to aphasia.

First, a non-parametric Wilcoxon signed-rank (paired) test was used to examine whether more pain was observed during rest than during ADL. Subsequently, Mann-Whitney U tests were used to investigate if patients with aphasia who use pain medication experienced more pain than those without pain medication, and whether patients with aphasia and joint pain had more pain versus those without joint pain.

Reliability

The reliability of PAIC15 in persons with aphasia was determined by assessing internal consistency, intraobserver and interobserver agreement.

Internal consistency

The internal consistency of the PAIC15 of observers A and B together, during measurement 1, measurement 2, and measurements 1 and 2 together (consensus scores) was examined using Cronbach’s alpha. Cronbach’s α-values ranging from 0.70 to 0.95 are generally considered acceptable (Bland & Altman, 1997).

Intraobserver and interobserver agreement

The intraobserver and interobserver agreement of the individual items of PAIC15 were analysed using percentage agreement [21, 31, 32]. Percentage agreement is more suitable than for example Cohen’s kappa and interpretation by clinicians is more straightforward [32]. Cohen’s kappa is a relative measure of reliability, whereas percentage agreement is an absolute measurement. In clinical practice, the probability that another rater gives the same answers is of interest to healthcare professionals. Therefore, to assess intraobserver agreement, percentage agreement was calculated between the responses of each of the observers on measurements 1 and 2 during rest and transfer. Interobserver agreement was examined using percentage agreement between the PAIC15 4-point scores of observer A on measurements 1 and 2 compared to the scores of observer B on measurements 1 and 2 during rest and transfer. Percentage agreement was also calculated with dichotomized scores (0 = absent; 1,2,3 = present) of the PAIC15 scores of both observers on both measurements during rest and transfer. These percentage agreements of the dichotomized scores were compared with the percentage agreements of the PAIC15 scores using the 4-point scale. Percentage agreement below 70% was regarded as poor and percentage agreement of ≥ 70% was considered high [32]. The analyses were performed using IBM SPSS Statistics version 29 for Windows, 2022.

Results

The study flowchart is presented in Fig. 1 [see Fig. 1]. Data was collected during the COVID-19 pandemic and inclusion of persons with aphasia and collecting data took longer than expected due to the closure of a department of participated nursing homes or quarantine. Speech and language therapists of 14 nursing home organizations invited 95 persons with aphasia to participate; 82 persons with aphasia were included. Pain observations were performed by trained speech and language therapists (N = 4), nurses (N = 8), and trained master’s students (N = 7).

Fig. 1

Flowchart inclusion of nursing homes and persons with aphasia. a: SLP = Speech Language Pathologist often called speech and language therapist; b: nursing homes participated if one or two speech language therapist participated; c: Persons with aphasia without a diagnosis of aphasia, psychiatric disorder, or delirium; d: observations took place at a time when no transfer took place (for example because the person was in bed or wheelchair). NRS = Numeric Rating Scale; FPS = Faces Pain Scale; VAS = Visual Analoque Scale

The sample characteristics of the persons with aphasia are shown in Table 2. Almost two-thirds of the persons received pain medication (62%) and were able to complete at least one self-report pain scale (65%). Osteoarthritis or rheumatism were present in 9 (13%) patients (See Table 2).

Table 2 Characteristics of the 71 participating persons with aphasia

Descriptive statistics of the PAIC15

The descriptive statistics of PAIC15, based on the PAIC15 consensus scores of measurement 1, and the self-report pain scales in persons with aphasia are presented in Table 3. See Additional file 4 for the descriptive statistics of the individual observations using PAIC15 of observers A and B [see Additional file 4]. Table 4 shows the presence of responses on the individual PAIC15 consensus scores in percentages during rest and transfer. During rest, prevalence of all PAIC15 items was low except for the facial expression item ‘opening mouth’, which had a prevalence of ≥ 30%. More items with higher prevalence were found during transfer; three items in the domains facial expressions and one item in the domain vocalizations showed a prevalence of ≥ 30%. All other items had a lower prevalence than 30%, during rest and during transfer (See Table 4). In most cases, the item ‘resisting care’ was ‘not scoreable’ during rest because healthcare professionals were often not present or not providing care. Both during rest and transfer, there is a floor effect with frequencies higher of 15% on score ‘0-not at all’. No ceiling effect emerged. Less than 5% of responses were missing.

Table 3 Descriptive statistics of the PAIC15^a consensus scores^b and self-report pain scales^c in persons with aphasia during assessment 1

Table 4 Scores per item of PAIC15^a consensus scores (in percentages) during rest (N = 71) and during transfer (N = 70) in patients with aphasia

Validity

Criterion validity

Correlations between PAIC15 consensus scores and the self-report pain scales of measurement 1 during rest and transfer are shown in Table 5. The PAIC15 had fair positive correlations with NRS, VAS, FPS, and the combined scale during rest (ranging from 0.35 with NRS to 0.50 with VAS). During transfer, the correlations between PAIC15 and the NRS, VAS and the combined scale were fair positive, varying from 0.38 (combined scale) to 0.43 (VAS). The PAIC15 correlated poorly with FPS (0.26).

Table 5 Correlation of PAIC15^a consensus scores^b versus self-report pain scales total scores in patients with aphasia during rest and during transfer

Construct validity

For the construct validity, the results of the 3 hypotheses that were tested show that significant more pain was observed in persons with aphasia during transfer (median 3; IQR 2–6) than during rest (median 1; IQR 1–3); z = -4.15, p < .05.

Observations with the PAIC15 during rest showed more pain in persons with aphasia using pain medication (median 1.5; IQR 1-3.75) versus persons who use no pain medication (median 1; IQR 0–2). However, this difference was not significant, U (N_{using pain medication}= 44, N_{using no pain medication}= 26,) = 463, z = -1.36, p = .175. Similar results were found during transfer (with pain medication: median 3; IQR 2-6.75; without pain medication: median 3; IQR 2-5.5); U (N_{using pain medication}= 44, N_{using no pain medication}= 25,) = 487, z = − 0.80, p = .423. Our hypothesis was rejected.

During rest, less pain was observed in persons with joint diseases such as osteoarthritis or rheumatism (median 1; IQR 1-2.5) versus persons without these diseases (median 1; IQR 1–3). However, the difference was not significant; U (N_{osteoarthritis/rheumatism}= 9, N_{no osteoarthritis/rheumatism}= 55) = 238, z = − 0.19, p = .851. Similar results were found during transfer (with joint disease: median 2; IQR 1.5-6; without joint disease: median 3; IQR 2-6.25); U (N_{osteoarthritis/rheumatism}= 9, N_{no osteoarthritis/rheumatism}= 54) = 190, z = -1.05, p = .293. Our hypothesis was rejected.

Reliability

Internal consistency

The internal consistency of the PAIC15 was acceptable, varying between α = 0.73 and 0.93 during rest and between α = 0.82 and 0.85 during transfer. These values were assessed using the combined PAIC15 scores of observers A and B, during measurement 1, measurement 2 and measurement 1 and 2 together.

Intraobserver and interobserver agreement

Table 6 presents the intraobserver and interobserver agreement of the PAIC15 scores with the 4-point scale in persons with aphasia during rest and transfer. See Table 6 with percentages of ≥ 70% shaded- in green. Of the items in the domain facial expressions, all except ’opening mouth’ showed high intraobserver agreement (≥ 70%) during rest. During transfer, agreement was high only on the items ‘narrowing eyes’ and ‘raising upper lip’. Interobserver agreement was also high (≥ 70%) during rest. During transfer, only the items ‘narrowing eyes’ and ‘raising upper lip’ achieved high agreement (≥ 70%), as did intraobserver agreement. Of all items in the domains body movements and vocalizations, intra- and interobserver agreement was > 70% during rest and transfer. Percentage agreement was also assessed after dichotomization of the PAIC15 scores, indicating that pain related behaviours were either present (score 1–3) or absent (score 0). Intra- and interobserver agreement of the PAIC15 dichotomized scores are also presented in Table 6. This resulted in higher agreement percentages than when using the 4-point scale (Table 6). All dichotomized scores of the 15 items showed good reliability with percentages of 70 or higher for both intra- and interobserver agreement during rest and transfer.

Table 6 Intra- and interobserver agreement of the PAIC15^a scores (in percentages) during rest and transfer in 71 patients with aphasia, both with 4-point and dichtomized sore

Discussion

This study aimed to examine the validity and reliability of the pain observation instrument PAIC15 in persons with aphasia and is therefore of clinical value for professionals to optimize pain assessment in persons with aphasia. Descriptive statistics of PAIC15 show that self-reporting pain was not possible in one third of participants (24/71). The prevalence of individual items of the PAIC15 observed in persons with aphasia was low for most items. Higher prevalence was observed in the facial expressions domain. This is in accordance with findings of a PAIC15 study in a long-term care setting in patients with dementia [33]. The items of the domains body movements and vocalizations showed the lowest prevalence. This result was expected, because of the minimal movement of the musculoskeletal system during rest. Regarding results during transfer, the overall prevalence of the individual items of PAIC15 was higher compared to the results during rest, which was expected.

Validity

The results of the current study indicate fair criterion validity because of largely fair positive correlations between PAIC15 and the self-report pain scales that could be completed by persons with aphasia. This study utilized consensus scores of PAIC15 after discussing the scores recorded by observer A and B following independent observations during rest and transfer on measurement 1. These consensus scores were needed to assess the correlations between the PAIC15 and self-report pain scales. If we compare the consensus scores to the scores of the independent observations, a few of the consensus scores were higher. However, discussion of the combined independent observations by observers A and B still yielded a higher score. An implication of this study is that using two observers improves the PAIC15 scores, because two observers see more than one observer during rest and transfer.

Another important finding, in terms of construct validity and assessed with hypothesis 1: significantly more pain was observed with the PAIC15 during transfer compared to during rest. However, hypothesis 2 (more pain observed when treated with pain medication compare to no-pain medication) was rejected. Contrary to studies of pain and pain medication in persons with dementia [26, 27], we did not find more pain in persons with aphasia when pain medication was used compared to when not treated with pain medication. Many studies have stressed that pain after stroke was under-recognized and persons received inadequate pain management [9, 34]. When pain is under-recognized and undertreated, while treatment would be effective, this hypothesis may not be suitable. Hypothesis 3 was also rejected because there was no difference in observed pain in persons with aphasia with and without joint disease. Joint disease is one of the most frequent general causes of pain, yet indeed, joint disease is not specific to stroke patients. Stroke patients experience significant pain after stroke, especially headache, shoulder pain, pain from increased muscle stiffness, and central post-stroke pain which are not related to joint disease and are uncommon in this study sample [35, 36]. Therefore, this hypothesis may not work well in this population and further research on causes of pain in stroke patients is warranted.

Reliability

Acceptable internal consistency of PAIC15 in persons with aphasia was examined. We found that intra- and interobserver agreement for the items of the PAIC15 domains body movements and vocalizations are both good (≥ 70%). Results on the domain facial expressions show good intraobserver agreement for almost all items and good interobserver for all items during rest. This is contrary to the findings during transfer with a high percentage only on both intra- and interobserver agreement for the items ‘narrowing eyes’ and ‘raising upper lip’. These results resemble those of Van Dalen-Kok et al. [33] who also found that fewer items in the domain facial expressions had good intraobserver- and interobserver agreement during both rest and transfer. Lower intra- and interobserver agreement for the facial expression items suggest that these items are more difficult to observe in a clinical setting. Research of Oosterman et al. [37] reported that recognizing and observing facial expressions for pain assessment in dementia requires specific training and education [37]. Assessing pain based on the observation of facial expressions in persons with dementia can be compared to persons with aphasia, because of their impaired cognition and communications problems. Percentages of 70 or higher for both intra- and interobserver agreement indicate good reliability of PAIC15 with dichotomized scores. This implies that assessing the presence of a pain-related item using PAIC15 is more reliable than assessing the degree/intensity of the pain-related items of PAIC15 with the 4-point scale in persons with aphasia.

Strength and limitations

Our study is the first to explore alternative methods for the long-standing and distressing situation of poor assessment and management of pain in persons with aphasia. Other strengths include the use of clinical situations and providing elaborate training for the research assistants. Also, no other pain research in persons with aphasia has used several self-report pain scales and a combined self-report scale. A limitation is that we did not check the competency of the different raters after training. However, we used a standardized training and each first observation of an observer was carried out with the researcher for instructions and practice in using PAIC15 independently. The prevalence of individual items observed in persons with aphasia was low for most items. The scores 2 and 3 of the PAIC15 were rarely rated, due to the fact that the observed persons with aphasia showed few items and the observers struggled to differentiate between score 2 or 3. Deciding between a 2 or 3 could be difficult; when is someone frowning with a ‘moderate’ or ‘great degree’? The rating of these scores has recently been revised and adjusted in the online PAIC15 e-learning.

It is also possible that the low scores are due to failure to observe behavior described in the PAIC15 items in persons with aphasia after stroke. This could lead to the question whether reporting items ‘not at all’ means that these persons do not experience any pain? This raises questions about the applicability of the PAIC15 in this population. However, literature reported that persons with aphasia can also experience pain, especially if they have communication problems. If this is the case and a self-report pain scale cannot be completed, pain may not be detected. The PAIC15 can meet this need by observing possible behaviors that indicate possible pain.

Another limitation might be that the time between the observations of measurements 1 and 2 varied from 1 to 7 days, and the use of self-report pain scales was not checked again after 7 days. Depending of the recovery of the stroke, it is recommended to check if self-reporting of pain is possible a week later. Within rehabilitation, spontaneous recovery can certainly occur within 7 days or the situation changes, e.g., re-admission to hospital or discharge home. These changes could affect the intraobserver agreement more strongly if the interval is 7 instead of 2 days.

Results may have been influenced because of current study was conducted during the COVID-19 pandemic. Which means that participants were observed during a period with restrictive rules to reduce the spread of COVID-19. These circumstances may potentially have influenced the observed behaviors of the participants using PAIC15, Future studies are recommended to determine if the results are valid in the post-COVID era.

Conclusions

Results show fair criterion validity, and significantly more pain was observed during transfer compared to rest using PAIC15 regarding construct validity. Regarding reliability, we found an acceptable internal consistency of PAIC15 and good intra- and interobserver agreement for most PAIC15 items, particularly for the domains body movements and vocalizations in persons with aphasia. This study shows that PAIC15 can be used to assess pain in persons with aphasia. Further research in the daily practice setting should clarify whether combining PAIC15 with self-report and other clinical leads will deliver results that can be confidently used in practice.