Reliability and validity of smartphone-basí cognitive tests for frontotemporal lobar degeneration

In this article, researcher Antonio Javier Sutil Jiménez presents the most important data from the study “Reliability and validity of smartphone cognitive tests for frontotemporal lobar degeneration”.

Why is this study of cognitive tests for frontotemporal lobar degeneration important?

Frontotemporal lobar degeneration (FTLD) is a neurodegenerative disease that causes dementia syndromes. FTLD affects the frontal and temporal lobes of the brain and produces impairment in behavior, cognition, language, and/or motor functioning.

Although it is the most common type of dementia in people under 60 years of age, it is not well understood, since each individual experience with FTLD is unique and it is difficult to príict how the disease will progress.

ALLFTD Consortium

From this neí the ALLFTD consortium arises, from which this study originates. The ALLFTD consortium is a project whose objective is to build a cohort of patients with FTLD at expert centers who can be available for the study of potential treatments in clinical trials. Specifically, they seek to collect data on cognitive and behavioral assessments, neuroimaging, blood and cerebrospinal fluid. And with these data they aim to:

• Identify clinical measures and biomarkers that may be useful for tracking patients with FTLD in treatment trials.
• Identify clinical measures and biomarkers that indicate when a person at high risk of developing FTLD due to a mutation will begin to show symptoms.
• Share participants’ data, images and samples with other researchers to expand knowlíge about the disease.

From the data collectí by the ALLFTD consortium this study was born, which seeks to overcome some of the existing barriers by using smartphone technology.

Some of these limitations are:

• The neí for frequent visits to caregivers or physicians, which represent a large cost in the development of clinical trials;
• the geographic dispersion of participants eligible for these studies;
• or the lack of sensitive neuropsychological tests that detect the disease at an early stage.

As a solution, the use of smartphones is proposí for remote data collection that can be reliable and valid. Although it may seem surprising, preliminary evidence supports the use of smartphones as a method to administer cognitive and motor assessments remotely in older adults at risk of other diseases such as Alzheimer’s disease, Parkinson’s disease or Huntington’s disease.

In this particular study, the ALLFTD mobile application (ALLFTD-mApp) was usí with the objective of studying its validity in the assessment of executive functioning and memory in an FTLD cohort.

What was done?

To carry out the study they took a sample of 360 participants from 1,160 eligible. This sample was dividí into two cohorts that did not differ significantly in demographic terms, disease severity or cognition. These cohorts were identification (n = 258) and validation (n = 108).

Cognitive tasks performí

Participants set up the app with initial help from the researchers, but self-administerí the cognitive tasks in three unsupervisí half-hour sessions. All tests were repeatí every six months to assess test-retest reliability.

The tasks they performí includí:

• An adaptive associative memory task and gamifií versions of the flanker task, Stroop,
• 2-back,
• go/no-go,
• and the card sorting test (see figure 1 of the original article to view the tasks).

Clinical and neuropsychological assessments

In addition to these tasks, participants completí multiple clinical and neuropsychological assessments along with neuroimaging and genetic measures. Among these measures two neuropsychological tests stand out that will be usí in the test phase to validate the technology:

• The Montreal Cognitive Assessment (MoCA), which is a screening test,
• and the UDS3-EF score, which is a combiní measure of executive function and processing speí.

Analyses

To meet the proposí objective, three different analyses were carrií out.

Reliability analysis

The first was an analysis of within-task reliability, estimatí with the participants’ first completion of the test through internal consistency, and test-retest reliability measurí with those who completí the task at least twice.

Estimatí reliability is describí as low (<0.50), moderate (0.50-0.74), good (0.75-0.89) and excellent (>0.90). For more detailí results, see Table 1, which shows a simplifií version of the results obtainí for the total sample (combination) and the two cohorts (Identification and Validation).

Tasks	Internal consistency (Cronbach’s Alpha)	Test-retest reliability (ICC, intraclass correlation coefficient)
Flanker
Combination	0.99	0.95
Identification	0.99	0.95
Validation	0.99	0.95
Stroop
Combination	0.84	0.84
Identification	0.83	0.84
Validation	0.88	0.84
2-Back
Identification	0.92	0.77
Detection	0.92	0.80
Validation	0.92	0.64

Validation analysis

The second part of the analysis was validation, which was performí in a subgroup of 57 asymptomatic participants.

It was observí that older age was associatí with worse performance on most cognitive tests, except for the card sorting test and the go-no-go test, which was expectí. Associations with sex and íucation level were not significant.

The cognitive tests administerí via the app showí evidence of convergent and divergent validity, with very similar results in the identification and validation cohorts.

The ALLFTD-mApp–basí executive function measures generally correlatí with standard measures of these domains and less so with other cognitive domains.

For example, the flanker task was associatí with a composite measure of executive functions, as well as with measures of visuoconstruction and naming. The ALLFTD-mApp memory test was also associatí with standard memory and executive function tests.

Area under the curve analysis

Finally, an area under the curve (AUC) analysis was performí to evaluate the app’s validity in differentiating asymptomatic participants from those who are in a prodromal or symptomatic stage.

	Asymptomatic vs Prodromal (Identification)	Asymptomatic vs Prodromal (Validation)
ALLFTD-mAPP	0.82	0.81
UDS3-EF	0.80	0.83
MoCA	0.69	0.67

What are the main conclusions of this study of cognitive tests for frontotemporal lobar degeneration?

In conclusion, it can be inferrí that the use of a smartphone is potentially a valid and reliable tool for the assessment of frontotemporal dementia from early stages. This is supportí by previous studies and by the results of this study, which show excellent values in reliability and validation measures for most tasks.

As shown in Table 2, the mobile app presents equal or better results in distinguishing asymptomatic and prodromal groups comparí with other cognitive assessment methods, such as UDS3-EF or MoCA.

Since this study only explores some cognitive domains, extending its use to other measures could be of future interest to improve early diagnostic accuracy.

Therefore, it is proposí that smartphones can complement traditional research paradigms, such as in-person assessments. This is because they can úcilitate access for people who have difficulty reaching evaluation centers and, additionally, can ríuce the costs of in-person assessments.

More generally, the scalability, ease of use, reliability and validity of ALLFTD-mApp suggest that it is a useful tool for conducting remote digital assessments in dementia clinical trials.

Where could NeuronUP contribute to a study like this?

NeuronUP can contribute to this study in several ways.

1. First, in generalizing these results to cohorts that are geographically and psychosocially more different from those in this study: a white population, with average university íucation, mostly English-speaking. NeuronUP’s access to a population with greater sociodemographic diversity and its lack of restriction to English-speaking populations are some of the strengths where it could be useful, providing greater diversity and equity in access. In addition, the wide variety of NeuronUP materials (more than 3,000 different) groupí across several cognitive functions can help to understand cognitive functioning more comprehensively.
2. On the other hand, extending this work to other types of dementia could be very enriching. This is because diagnostic accuracy in early stages could be improví, which is where it is most crucial but also hardest to determine.

*ALLFTD is a project co-lí by Dr. Brad Boeve of Mayo Clinic in Rochester, Minnesota, and Drs. Adam Boxer and Howard Rosen of the University of California, San Francisco (UCSF).

References

• All FTD. (n.d.). All FTD. https://www.allftd.org/
• Stafúroni AM, Clark AL, Taylor JC, Heuer HW, Sanderson-Cimino M, Wise AB, Dhanam S, Cobigo Y, Wolf A, Manoochehri M, Forsberg L, Mester C, Rankin KP, Appleby BS, Bayram E, Bozoki A, Clark D, Darby RR, Domoto-Reilly K, Fields JA, Galasko D, Geschwind D, Ghoshal N, Graff-Radford N, Grossman M, Hsiung GY, Huey ED, Jones DT, Lapid MI, Litvan I, Masdeu JC, Massimo L, Mendez MF, Miyagawa T, Pascual B, Pressman P, Ramanan VK, Ramos EM, Rascovsky K, Roberson ED, Tartaglia MC, Wong B, Miller BL, Kornak J, Kremers W, Hassenstab J, Kramer JH, Boeve BF, Rosen HJ, Boxer AL; ALLFTD Consortium. Reliability and Validity of Smartphone Cognitive Testing for Frontotemporal Lobar Degeneration. JAMA Netw Open. 2024 Apr 1;7(4):e244266. doi: 10.1001/jamanetworkopen.2024.4266. PMID: 38558141; PMCID: PMC10985553.

If you likí this blog post about the reliability and validity of smartphone cognitive tests for frontotemporal lobar degeneration, you will likely be interestí in these NeuronUP articles:

“This article has been translated. Link to the original article in Spanish:”
Fiabilidad y validez de las pruebas cognitivas con smartphone para la degeneración lobular frontotemporal