In this article, researcher Antonio Javier Sutil Jiménez presents the most important data from the study “Reliability and Validity of Smartphone Cognitive Tests for Frontotemporal Lobar Degeneration.”

Why is this study on cognitive tests for frontotemporal lobar degeneration important?

Frontotemporal lobar degeneration (FTLD) is a neurodegenerative disorder that causes dementia syndromes. FTLD affects the frontal and temporal lobes of the brain, leading to impairments in behavior, cognition, language, and/or motor functioning.

Although it is the most common form of dementia in people under 60 years old, it is not well understood because each individual’s experience with FTLD is unique, and it is difficult to predict how the disease will progress.

ALLFTD Consortium

In response to this need, the ALLFTD consortium was created, from which this study originates. The ALLFTD consortium is a project aimed at building a cohort of FTLD patients at expert centers that can be available for studying potential treatments in clinical trials. Specifically, they aim to collect data on cognitive and behavioral assessments, neuroimaging, blood, and cerebrospinal fluid. Their goals are to:

• Identify clinical measures and biomarkers that could be useful in monitoring FTLD patients in treatment trials.
• Identify clinical measures and biomarkers that indicate when a person at high risk for FTLD due to a mutation will begin showing symptoms.
• Share data, images, and samples from participants with other researchers to expand knowledge of the condition.

Using the data collected by the ALLFTD consortium, this study aims to overcome some existing barriers by utilizing smartphone technology.

Some of these limitations include:

• The need for frequent visits to caregivers or doctors, which significantly increases the cost of developing clinical trials;
• The geographic dispersion of eligible participants for these studies;
• Or the lack of sensitive neuropsychological tests to detect the condition in its early stages.

As a solution, the use of smartphones is proposed for remote data collection that can be both reliable and valid. Though it may seem surprising, preliminary evidence supports the use of smartphones as a method for remotely administering cognitive and motor evaluations in older adults at risk of other conditions such as Alzheimer’s, Parkinson’s, or Huntington’s.

In this particular study, the ALLFTD mobile app (ALLFTD-mApp) was used with the goal of assessing its validity in evaluating executive function and memory in a cohort with FTLD.

What has been done?

To carry out the study, a sample of 360 patients was selected from 1,160 eligible individuals. This sample was divided into two cohorts that did not differ significantly in terms of demographics, disease severity, or cognition. These cohorts were the identification cohort (n = 258) and the validation cohort (n = 108).

Cognitive tasks performed

Participants set up the app with initial help from researchers but self-administered the cognitive tasks in three half-hour sessions without supervision. All tests were repeated every six months to assess test-retest reliability.

The tasks performed included:

• An adaptive associative memory task and gamified versions of the Flanker task, Stroop task,
• 2-back,
• go/no-go,
• and the card sorting test (see figure 1 in the original article for details on the tasks).

Clinical and neuropsychological evaluations

In addition to these tasks, patients completed multiple clinical and neuropsychological evaluations, along with neuroimaging and genetic measures. Two neuropsychological tests stand out, which will be used in the test phase to validate the technology:

• The Montreal Cognitive Assessment (MoCA), which is a screening test,
• and the UDS3-EF score, which is a combined measure of executive function and processing speed tests.

Analysis

To meet the proposed objective, three different analyses were performed.

Reliability analysis

The first was a reliability analysis within the task, estimated with participants’ first attempt at the test through internal consistency and test-retest reliability measured with those who completed the task at least twice.

Reliability was classified as low (<0.50), moderate (0.50-0.74), good (0.75-0.89), and excellent (>0.90). For more detailed results, see Table 1, which simplifies the results obtained for the total sample (combined) and the two cohorts (identification and validation).

Tasks	Internal Consistency (Cronbach’s Alpha)	Test-retest Reliability (ICC, intraclass correlation coefficient)
Flanker
Combined	0.99	0.95
Identification	0.99	0.95
Validation	0.99	0.95
Stroop
Combined	0.84	0.84
Identification	0.83	0.84
Validation	0.88	0.84
2-Back
Identification	0.92	0.77
Detection	0.92	0.80
Validation	0.92	0.64

Validation analysis

The second part of the analysis was the validation, which was carried out in a subgroup of 57 asymptomatic participants.

It was observed that older age was associated with poorer performance on most cognitive tests, except for the card sorting test and the go/no-go test, which was expected. Associations with sex and education level were not significant.

Cognitive tests administered through the app showed evidence of convergent and divergent validity, with very similar results in both the identification and validation cohorts.

Executive function measures based on the ALLFTD-mApp generally correlated with standard measures of these domains and less with other cognitive domains.

For example, the Flanker task was associated with a composite measure of executive functions, as well as with visuoconstruction and naming measures. The ALLFTD-mApp memory test was also associated with standard memory and executive function tests.

Area analysis

Finally, an area under the curve (AUC) analysis was performed, which evaluated the validity of the app in differentiating asymptomatic participants from those in a prodromal or symptomatic stage.

	Asymptomatic vs Prodromal (Identification)	Asymptomatic vs Prodromal (Validation)
ALLFTD-mApp	0.82	0.81
UDS3-EF	0.80	0.83
MoCA	0.69	0.67

What are the main conclusions of this cognitive test study for frontotemporal lobar degeneration?

In conclusion, it can be inferred that using a smartphone is potentially a valid and reliable tool for evaluating frontotemporal dementia from its early stages. This is supported by previous studies and by the results of this study, which show excellent reliability and validation values for most tasks.

As shown in Table 2, the mobile app performs as well as or better than other cognitive assessment methods, such as the UDS3-EF or MoCA, in distinguishing between asymptomatic and prodromal groups.

Since this study only explores some cognitive domains, extending its use to other measures could be of future interest to improve early diagnostic accuracy.

Therefore, it is suggested that smartphones can complement traditional in-person research paradigms. They can facilitate access for individuals who have difficulty traveling to evaluation centers and can also reduce the costs of in-person assessments.

More generally, the scalability, ease of use, reliability, and validity of the ALLFTD-mApp suggest that it is a useful tool for conducting remote digital assessments in dementia clinical trials.

Where could NeuronUP contribute to a study like this?

NeuronUP can contribute to this study in several ways.

1. First, in generalizing these results to cohorts that are geographically and psychosocially more diverse than those in this study—mainly white, college-educated, English-speaking populations. NeuronUP’s access to a more diverse sociodemographic population and its non-restriction to English speakers are key strengths where it could be useful, contributing to greater diversity and equity in access. Additionally, the wide variety of NeuronUP materials (over 3,000) grouped into various cognitive functions could help to understand cognitive functioning more comprehensively.
2. Furthermore, extending this work to other types of dementias could be highly enriching. This could improve diagnostic accuracy in the early stages, where it is most critical but also hardest to pinpoint.

*ALLFTD is a project co-directed by Dr. Brad Boeve of Mayo Clinic in Rochester, Minnesota, and Drs. Adam Boxer and Howard Rosen of the University of California, San Francisco (UCSF).

References

• All FTD. (n.d.). All FTD. https://www.allftd.org/
• Staffaroni AM, Clark AL, Taylor JC, Heuer HW, Sanderson-Cimino M, Wise AB, Dhanam S, Cobigo Y, Wolf A, Manoochehri M, Forsberg L, Mester C, Rankin KP, Appleby BS, Bayram E, Bozoki A, Clark D, Darby RR, Domoto-Reilly K, Fields JA, Galasko D, Geschwind D, Ghoshal N, Graff-Radford N, Grossman M, Hsiung GY, Huey ED, Jones DT, Lapid MI, Litvan I, Masdeu JC, Massimo L, Mendez MF, Miyagawa T, Pascual B, Pressman P, Ramanan VK, Ramos EM, Rascovsky K, Roberson ED, Tartaglia MC, Wong B, Miller BL, Kornak J, Kremers W, Hassenstab J, Kramer JH, Boeve BF, Rosen HJ, Boxer AL; ALLFTD Consortium. Reliability and Validity of Smartphone Cognitive Testing for Frontotemporal Lobar Degeneration. JAMA Netw Open. 2024 Apr 1;7(4):e244266. doi: 10.1001/jamanetworkopen.2024.4266. PMID: 38558141; PMCID: PMC10985553.

If you enjoyed this blog post about the reliability and validity of smartphone cognitive testing for frontotemporal lobar degeneration, you may be interested in these NeuronUP articles: