HTML

ePub

PDF

INTRODUCTION

The COVID-19 pandemic disrupted health care systems in a number of low- and middle-income countries. National and international health organizations and public authorities recommended social isolation and lockdown as preventive measures to contain the spread of the Sars-CoV-2 virus,1 which encouraged the use of telehealth services for individuals seeking primary, secondary, and even tertiary care.2

One important risk group for COVID-19 infection consists of individuals with comorbidities and older adults.3 In fact, chronological age is a well-known predictor of numerous health outcomes. Age-related declines in multiple biological systems can lead to the deterioration of functional and cognitive performance,4 which could make individuals more susceptible to functional dependence, hospitalization, and increased morbidity and mortality.4 Recent data from the Canadian Longitudinal Study on Aging indicate that older adults with mild-to-moderate COVID-19, even those who are not hospitalized, experience significant functional decline.5 Therefore, monitoring physical and cognitive function indicators through telehealth programs is crucial for identifying those with clinically relevant changes and providing tailored prevention and treatment in this population.

However, there are some concerns about using telehealth programs to track health outcomes. For example, it is unclear whether:

1. 1. They are feasible in subgroups of older people and
2. 2. They can capture reliable indicators for clinical decision-making. Both of these questions remain unresolved, especially in the context of health public system in low- and middle-income countries.6 Previous studies have found poor reliability indicators for cognitive assessments in older adults,7,8 whereas functional mobility outcomes presented good reliability levels.6,9,10

In fact, performing functional and neuropsychological assessments through telehealth programs is a challenge due to issues related to standardization in uncontrolled (vs laboratory-based) settings, which could significantly impact reliability indexes and impede their translation into clinical practice.8,11 Although guidelines for telehealth assessment have been available for at least 20 years,12 further research on the subject is still necessary. Thus, we aimed to determine the feasibility and reliability of functional and cognitive measures through web videoconferencing with older adults in the context of the COVID-19 pandemic.

METHODS

We recruited 30 Brazilian community-dwelling older adults through social media (Instagram, Facebook, Twitter), TV, and radio advertising between July 7 and October 7, 2021.

The eligibility criteria were:

1. 1. Internet access and at least one electronic device with a front-facing camera (eg, smartphone, notebook, tablet) that allowed video calls;
2. 2. Age ≥ 65 years;
3. 3. No dementia or scores < 19 (out of 30 points) on the Montreal Cognitive Assessment (MoCA) on the first or second day of evaluation.13 We decided to use a lower cut-off score for cognitive impairment for better specificity in this sample;
4. 4. No severe impairment or untreated health conditions, such as angina pectoris or severe musculoskeletal problems, that could impede the assessments, in addition to no visual or auditory impairments.

The study was approved by the Federal Rural University of Pernambuco Research Ethics Committee (protocol 4.613.968) and all participants provided written informed consent prior to data acquisition. Insofar as possible, this article adhered to the Guidelines for Reporting Reliability and Agreement Studies.

This reliability study was conducted in the context of COVID-19 pandemic in Brazil. We scheduled 3 videoconferences with the participants on Google Meet, as detailed in Figure 1.

At the first meeting, a global screening (sociodemographic, clinical records, time availability, etc.) was performed to identify eligible participants. At the end of this meeting, a video tutorial detailed the procedures for the planned assessments. At the second and third meetings, a single trained evaluator (experienced with remote assessment through a pilot study) determined the reliability indexes of functional and cognitive performance measures on 2 non-consecutive days (48 h apart).

During the functional and cognitive measures, the participants were instructed to keep their device on (video and microphone enabled) until they completed the assessments. If a participant had technical problems (eg, a poor connection) or was not familiar with the device or application, a family member or caregiver could assist to ensure call quality.

The cognitive function assessments were performed first, followed by the functional tests. In the cognitive assessments, the participants were advised to remain in a quiet, private room without third parties to avoid assistance and interference. The tests were performed in the following order: the Montreal Cognitive Assessment (MoCA), parts A and B of the Trail Making Test (TMT), the Stroop test, and the verbal fluency test. The cognitive assessment lasted between 20 and 60 minutes. In the functional tests, we asked a family member or caregiver to help record all the assessments with an alternative device. These recordings were sent to the research staff for analysis by the same evaluator.

At the end of third meeting, to assess the feasibility of the functional and cognitive measures, the patients answered a standardized 14-item questionnaire about their overall experience (challenges, problems, and feasibility) and the videoconference format. Responses were given on a Likert scale ranging from 1 to 5.

Global cognition – Montreal Cognitive Assessment

Cognition was assessed using the videoconference version of the MoCA questionnaire, which is available on the organization’s official website.14 In this test, the maximum possible score is 30 points, with 1 point added for respondents with < 12 years of education.15

Processing speed – parts A and B of the Trail Making Test

Processing speed and executive function were assessed with parts A and B of the TMT, respectively.16 The test is an adaptation of the Oral Trail Making Test, similar to the remote version of MoCA, except that the score is shared during the video call and the test includes more points. A version in slide format was prepared, such that when the participant answers correctly, the evaluator moves on to the next slide. At this point, the score’s color is changed to red to highlight it for the participant. With a stopwatch, the examiner tracks the time required for the subject to correctly complete all the sequences, and records the number of errors. The evaluator immediately warns when a mistake has been made, so that it can be corrected. In this model, evaluation of the processing speed is prioritized.

Inhibitory control – stroop test

Inhibitory control was assessed with the Stroop Test, a neuropsychological test of attention to simultaneous tasks: a reading task and a color-naming task.17 The test was adapted for videoconferencing by displaying 3 specific images for each trial. For each trial, the examiner shared slides with 6 4-item lines. When the participant began answering, the examiner activated a stopwatch to record the time required to correctly identify all the items. The number of errors was also recorded. The Stroop effect was calculated as the time difference between the interference task and the color- naming task.

Verbal Fluency Test – animal naming

Participants were requested to name as many animals as possible in 1 m. A higher number of animals indicated better verbal fluency.18

Muscle endurance and power – chair rise test

The chair rise test (CRT) test involved a chair with a backrest and no armrests. When the evaluator gives the signal, participants sit down and stand up completely, with arms their crossed over their chest, as quickly as possible, for 30 s (timed on a stopwatch).19

To assess lower limb muscle power, an equation was used involving the number of repetitions in the first 20 s of the test and the participant’s body weight: mean power (watts) = -504 845 + 10 793 (body weight in kg) + 21 603 (repetitions in 20 s).20

Muscle strength – chair stand test

To assess strength, the time required to perform 5 complete cycles of sitting and standing from the chair was recorded.21

Functional fitness – sitting and rising test

The sitting and rising test was used to assess muscle fitness. This test counts the number of support points (hands and/ or knees or hands on knees or legs) the participant needs to and stand up from the floor and sit down again.22 Of a maximum score of 5 points each for sitting and standing, 1 point is deducted for each support point, and 0.5 point is deducted for any conspicuous imbalance. When the individual cannot sit down or stand up without help or requires more than 4 support points to get up, a score of 0 is recorded.

Statistical analysis

Data normality was determined with the Kolmogorov-Smirnov test. Assessment differences between days (meeting 2 vs. 3) were determined with a paired t-test or the Wilcoxon test. Reliability was determined through the intraclass correlation coefficient (ICC), the coefficient of variation, the standard error of measurement (SEM), the minimal detectable change (MMD), and Bland-Altman plots.

We computed the ICC using a 2-way mixed model and the absolute agreement method. ICC values were classified as follows: < 0.50 poor reliability, 0.50 – 0.75 moderate reliability, 0.75 – 0.90 good reliability, and > 0.90 excellent reliability.23 The coefficient of variation was computed as the ratio between the SD and the mean values of the differences. SEM, a measure of absolute error and precision, was computed as SEM = SD*√(1- ICC). Minimal detectable change, ie, the smallest change detectable by the measurement tool that can be interpreted as clinically significant, was estimated using absolute the SEM as follows: minimal detectable change = 1.96*SEM*√2. Finally, we performed a Bland-Altman analysis to examine the absolute agreement between the 2 assessment days and their limits of agreement, bias, and outliers.24 The closer the bias value is to 0, the greater the agreement between the measures. The further from 0 the confidence limits are, the lower the degree of agreement between the measures. Outliers are undesirable values that exceed the confidence limits.

The analytical procedures and graphic representation were performed using IBM SPSS Statistics version 25.0 and GraphPad Prism 9. The significance level of all analyses was set at p < 0.05, with a 95%CI.

RESULTS

According to the MoCA scores, 43 participants scored above and 13 below the eligibility cut-off. Table 1 shows the general characteristics of the 30 participants included in the analysis. Most participants were women (86.70%) with ≥ 12 years of education and a mean age of 69.77 (SD = 6.60) years.

They were relatively healthy (16.70% with hypertension and 3.30% with diabetes). Most had previous experience with videoconferencing (76.70%). Although they reported frequent Internet use, roughly 66.70% had technical problems with the protocol.

Regarding the cognitive function data (Table 2), our analysis showed mixed levels of reliability across measures: the MoCA, part B of the Trail Making Test, and the congruent and neutral trials in the Stroop test had excellent ICC (0.79 – 0.91), while part A of the Trail Making Test, the Incongruent trial in the Stroop test, the Stroop Effect, and the verbal fluency test had poor-to-moderate ICC (0.42 – 0.58). Bland-Altman analysis revealed that differences between assessment days were within the limit of agreement (LOA) for all cognitive function measures.

Absolute SEM values ranged from 1.12 (verbal fluency test) to 37.62 (part B of the Trail Making Test), while the minimal detectable change ranged from 3.10 to 104.27 for same variables.

Table 3 shows the intra-rater reliability for functional performance measures.

Excellent intra-rater reliability was found for all functional performance measures (ICC ranging from 0.90 [95%CI 0.78 – 0.95] for the sitting and rising test and 0.98 [95%CI 0.96 – 0.99] for the CRT). According to Bland-Altman analysis, differences between assessment days were within the LOA for CRT results: mean difference (MD) = -0.03 (SD = 1.67), LOA= -3.31 – 3.24; CRT in watts: MD = 10.08 (SD = 33.44), LOA = -55.45 – 75.62; CST: MD = 0.70 (SD = 2.26), LOA = -3.73 – 5.13; SRT = MD = 0.12 (SD = 1.63), LOA= -3.07 – 3.31.

Absolute SEM values ranged from 0.8 (Sitting and Rising Test score) to 18.5 (CRT in watts), and the minimal detectable change ranged from 2.3 to 51.4 for the same variables.

Table 4 shows the results of the standardized 14-item questionnaire on the videoconference assessment. Overall, most participants reported good to excellent levels of satisfaction regarding the technical features of the videoconference (eg, audio and video quality), as well as good levels of familiarization with the protocol.

DISCUSSION

The main findings of this study showed that:

1. 1. Videoconferencing a is feasible means of assessing functional and cognitive performance in the context of the COVID-19 pandemic;
2. 2. Functional performance assessment showed excellent reliability and may be easily used to track functional changes due to social isolation. On the other hand, reliability indexes of cognitive function varied significantly across measures, and they should be used with caution in telehealth programs.

The literature agrees that virtual physical performance measures have high reliability and are generalizable to healthy older adult populations.6,9,10 A recent study investigated the reliability of 3 functional tests (including the CRT) applied remotely to older adults, finding excellent reliability (ICC > 0.98),6 which corroborates our findings. The validity and high reliability of the CRT in videoconferencing has been demonstrated by other authors, which reinforces its use for remote assessment in a clinical environment.10 The Short Physical Performance Battery, which includes the CRT, has also been used in videoconference-based assessments and has shown good reliability indicators.6,9.

The reliability of cognitive tests applied via videoconference is still questionable. There was moderate reliability for overall MoCA scores in our sample. Similarly, a study comparing videoconference and in-person MoCA results among older adults with cognitive difficulties found that applying the test via videoconference did not affect cognitive performance results.7

In our study, the reliability indicators for part A of the Trail Making Test were poor, but those of part B of the Trail Making Test and the verbal fluency test were good.25 Because the scores on the second assessment were higher than the first, there was probably a learning effect. Although good reliability indicators were found for the 3 Stroop trials, the main measures of inhibitory control showed poor agreement. Thus, they must be carefully applied due to multiple attention demands during the videoconference for both the assessor and the participant.26 It is more challenging to adapt and apply cognitive tests than objective measures, such as functional tests, which could explain our findings.

Most neuropsychological tests that assess cognition involve a question-and-answer protocol and require little equipment. In this study, we applied the remote version of the MoCA via videoconference, which is available on the organization’s website. However, no validations of adapted versions have been published, especially for key groups (people with sensory impairments, such as older adults).

One difficulty in interpreting the results of videoconference evaluations is certainty of the participants’ vision and hearing. To minimize such interference, we included only participants without sensory impairment and/or who used their eyeglasses or hearing aids if necessary. Nevertheless, the device type used for the videoconference limited such assessments: not everyone had a headset with a microphone or a computer with high screen resolution. In addition, it is not recommended to use devices other than computers for telehealth assessments because they more prone to distractions, such as uncontrolled notifications or calls.8 Smartphones were the predominant device used in our study, and even with the potential problems, most participants reported the videocall quality as good or excellent. Using telehealth resources to monitor different domains of cognition is still a questionable practice, and the results must be interpreted with great caution in clinical decision making.

Approximately 76.70% of the participants reported experience with videoconferencing, which may have facilitated our study and the use of Google Meet. Furthermore, WhatsApp, the messaging app used to share videos of the functional tests, links, and photos from part A of the MoCA, was easily accessible to the participants due to its regional popularity.

In practical application, the virtual format can help overcome transportation and locomotion problems, one of the main reasons for low participation in research or heath care interventions.27 However, considering that our sample of older adults was generally healthy, our results cannot be generalized to individuals with physical or functional limitations. It has also been observed that technological products, such as videoconference apps, are acceptable to older adults, even those who are less familiar with them.28 Thus, telerehabilitation can be considered a viable alternative for monitoring physical performance in older adults when in-person measurements are not possible, such as during the COVID-19 pandemic. However, assistance for patients with special conditions should be prioritized in hospitals.

Videoconference assessments eliminated travel and waiting time, providing a comfortable environment while maintaining face-to-face contact. The questionnaire results indicated good acceptance of the method. In practical terms, this study showed that health indicators can be assessed safely at home through a method accessible to both professionals and the public, while maintaining most tests in their original format.

Study limitations included the 48-h interval between assessments, which may have produced a learning effect, while the lack of automation in complex tests, such as those that require simultaneous attention from the evaluator, may have compromised interpretation of the results. Another limitation is the fact that the results may not be generalizable, since the majority of participants were women, relatively healthy, highly educated, and had high digital literacy, with most reporting previous videoconferencing experience. Second, most of the participants performed well on functional tests, and functional capacity assessment was limited, ie, it did not include gait speed or balance tests. Third, this study did not have a matched control group, although its goal was not to compare cognitive functioning.

CONCLUSION

In healthy, educated adults, videoconferencing is a feasible alternative method of measuring functional and cognitive performance when in-person assessments are impossible. The functional performance measures showed excellent reliability, whereas the results of cognitive tests should be interpreted carefully, since their reliability indexes varied from poor to good. Overall, our results indicate that videoconferencing may be a useful way to assess the functional and cognitive status of older adults, as well as to track their clinical course in the context of the pandemic.

REFERENCES

Farias HS. O avanço da Covid-19 e o isolamento social como estratégia para redução da vulnerabilidade. Espaço e Econômica. 2020;(17). https://doi.org/10.4000/espacoeconomia.11357

Rezende GR, Mendonça LR, Santos DO, Reis ST. Efetividade da telemedicina na assistência í saúde em tempos de COVID-19: uma revisão sistemática. Saúde Colet (Barueri). 2021;11:7085-94. https://doi.org/10.36489/saudecoletiva.2021v11iCOVIDp7085-7094

Bonanad C, García-Blas S, Tarazona-Santabalbina F, Sanchis J, Bertomeu-González V, Fácila L, et al. The effect of age on mortality in patients with COVID-19: a meta-analysis with 611,583 subjects. J Am Med Dir Assoc. 2020;21(7):915-8. https://10.1016/j.jamda.2020.05.045

Falck RS, Davis JC, Best JR, Crockett RA, Liu-Ambrose T. Impact of exercise training on physical and cognitive function among older adults: a systematic review and meta-analysis. Neurobiol Aging. 2019;79:119-30. https://10.1016/j.neurobiolaging.2019.03.007

Beauchamp MK, Joshi D, McMillan J, Oz UE, Griffith LE, Basta NE, et al. Assessment of functional mobility after COVID-19 in adults aged 50 years or older in the canadian longitudinal study on aging. JAMA Netw Open. 2022;5(1):e2146168. https://doi.org/10.1001/jamanetworkopen.2021.46168

Ogawa EF, Harris R, Dufour AB, Morey MC, Bean J. Reliability of virtual physical performance assessments in veterans during the COVID-19 pandemic. Arch Rehabil Res Clin Transl. 2021;3(3):100146. https://doi.org/10.1016/j.arrct.2021.100146

Chapman JE, Cadilhac DA, Gardner B, Ponsford J, Bhalla R, Stolwyk RJ. Comparing face-to-face and videoconference completion of the Montreal Cognitive Assessment (MoCA) in community-based survivors of stroke. J Telemed Telecare. 2021;27(8):484-92. https://doi.org/10.1177/1357633X19890788

Marra DE, Hamlet KM, Bauer RM, Bowers D. Validity of teleneuropsychology for older adults in response to COVID-19: a systematic and critical review. Clin Neuropsychol. 2020;34(7–8):1411-52. https://doi.org/10.1080/13854046.2020.1769192

Jennings SC, Manning KM, Bettger JP, Hall KM, Pearson M, Mateas C, et al. Rapid transition to telehealth group exercise and functional assessments in response to COVID-19. Gerontol Geriatr Med. 2020;6:2333721420980313. https://doi.org/10.1177/2333721420980313

Peyrusqué E, Granet J, Pageaux B, Buckinx F, Aubertin-Leheudre M. Assessing physical performance in older adults during isolation or lockdown periods: web-based video conferencing as a solution. J Nutr Health Aging. 2022;26(1):52-6. https://doi.org/10.1007/s12603-021-1699-y

Hwang R, Bruning J, Morris NR, Mandrusiak A, Russell T. Home-based telerehabilitation is not inferior to a centre-based program in patients with chronic heart failure: a randomised trial. J Physiother. 2017;63(2):101-7. https://doi.org/10.1016/j.jphys.2017.02.017

Parks AC, Davis J, Spresser CD, Stroescu I, Ecklund-Johnson E. Validity of in-home teleneuropsychological testing in the wake of COVID-19. Arch Clin Neuropsychol. 2021;36(6):887-96. https://doi.org/10.1093/arclin/acab002

Apolinario D, Santos MF, Sassaki E, Pegoraro F, Pedrini AVA, Cestari B, et al. Normative data for the Montreal Cognitive Assessment (MoCA) and the Memory Index Score (MoCA-MIS) in Brazil: adjusting the nonlinear effects of education with fractional polynomials. Int J Geriatric Psychiatry. 2018;33(7):893-9. https://doi.org/10.1002/gps.4866

MoCA Cognition. Successful completion of 1-hour to administer paper-based MoCA. Remote MoCA testing. Available from: https://www.mocatest.org/remote-moca-testing/. Accessed in Feb. 09, 2020

Cecato JF, Montiel JM, Bartholomeu D, Martinelli JE. Poder preditivo do MoCa na avaliação neuropsicológica de pacientes com diagnóstico de demência. Rev Bras Geriatr Gerontol. 2014;17(4):707-19. https://doi.org/10.1590/1809-9823.2014.13123

Alves FO, Zaninotto ALC, Miotto EC, Lucia MCS, Scaff M. Avaliação da atenção sustentada e alternada em uma amostra de adultos saudáveis com alta escolaridade. Psicol Hosp (São Paulo). 2010;8(2):89-105.

Brandelero V, Toni PM. Estudo de validade do teste Stroop de cores e palavras para controle inibitório. Psicol Argum. 2015;33(80):282-97. https://doi.org/10.7213/psicol.argum.33.080.ao05

Rodrigues AB, Yamashita ÉT, Chiappetta ALML. Teste de fluência verbal no adulto e no idoso: verificação da aprendizagem verbal. Rev CEFAC. 2008;10(4):443-51. https://doi.org/10.1590/S1516-18462008000400004

Jones CJ, Rikli RE, Beam WC. A 30-s chair-stand test as a measure of lower body strength in community-residing older adults. Res Q Exerc Sport. 1999;70(2):113-9. https://doi.org/10.1080/02701367.1999.10608028

Smith WN, Rossi GD, Adams JB, Abderlarahman KZ, Asfour SA, Roos BA, et al. Simple equations to predict concentric lower-body muscle power in older adults using the 30-second chair-rise test: a pilot study. Clin Interv Aging. 2010;5:173-80. https://doi.org/10.2147/cia.s7978

Bohannon RW, Bubela DJ, Magasi SR, Wang YC, Gershon RC. Sit-to-stand test: performance and determinants across the age-span. Isokinet Exerc Sci. 2010;18(4):235-40. https://doi.org/10.3233/IES-2010-0389

Araújo CGS. Teste de sentar-levantar: apresentação de um procedimento para avaliação em Medicina do Exercício e do Esporte. Rev Bras Med Esporte. 1999;5(5):179-82. https://doi.org/10.1590/S1517-86921999000500004

Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-63. https://doi.org/10.1016/j.jcm.2016.02.012

Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135-60. https://doi.org/10.1177/096228029900800204

Wadsworth HE, Dhima K, Womack KB, Hart Jr J, Weiner MF, Hynan LS, et al. Validity of teleneuropsychological assessment in older patients with cognitive disorders. Arch Clin Neuropsychol. 2018;33(8):1040-5. https://doi.org/10.1093/arclin/acx140

Brearly TW, Shura RD, Martindale SL, Lazowski RA, Luxton DD, Shenal BV, et al. Neuropsychological test administration by videoconference: a systematic review and meta-analysis. Neuropsychol Rev. 2017;27(2):174-86. https://doi.org/10.1007/s11065-017-9349-1

Kumar KR, Pina IL. Cardiac rehabilitation in older adults: new options. Clin Cardiol. 2020;43(2):163-70. https://doi.org/10.1002/clc.23296

Crotty M, Killington M, van den Berg M, Morris C, Taylor A, Carati C. Telerehabilitation for older people using off-the-shelf applications: acceptability and feasibility. J Telemed Telecare. 2014;20(7):370-6. https://doi.org/10.1177/1357633X14552382

Notes

Author notes

Correspondence data Juliana Daniele de Araújo – Avenida Professor Moraes Rego, 1235 – Cidade Universitária – CEP: 50730-120 – Recife (PE), Brasil. E-mail: julianadanielearaujo@gmail.com

Conflict of interest declaration

IR