What is the AI-based differential diagnosis study about?

A multimodal AI study using self-supervised transformer models to differentiate dementia etiologies across clinical, neuropsychological, and MRI data, aiming to improve early and accurate diagnosis and support personalized treatment decisions.

Why is AI important for dementia diagnosis?

AI can integrate heterogeneous, real-world data to detect subtle patterns, improve early differentiation between dementia types, reduce diagnostic errors, and enable personalized interventions.

What datasets and patient sample were used?

Nine independent datasets comprising more than 51,000 patients with diverse dementia etiologies were combined to provide representative multimodal training and validation data.

How does the model handle heterogeneous and missing multimodal data?

A transformer-based architecture converts varied features into fixed-length vectors and applies robust methods to mitigate missing data, preserving generalization and reducing training bias.

What performance did the AI model achieve?

The model showed high AUROC (micro 0.94–0.96; macro 0.90–0.93) for classification tasks; AUPR/AP metrics indicated good but variable precision across specific dementia types.

Did AI improve clinical diagnostic accuracy compared to experts?

AI assistance improved neurologist and radiologist diagnoses, e.g., neurologist gains included +12% for MCI, +15% for Alzheimer's, +26% for frontotemporal dementia, and up to +73% for prion disease.

AI-based dementia diagnosis

Researcher Antonio Javier Sutil Jiménez presents in this article the most relevant data on the study “AI-based differential diagnosis of dementia etiologies on multimodal data”.

Why is the study of dementia and artificial intelligence (AI) important?

The world’s population is aging and with it we face multiple age-related health risks. One of these risks is dementias, whose diagnosis increases by about 10 million new cases each year. The dementias are a group of diseases characterized by the deterioration of cognitive functions to the point of making daily life activities difficult or impossible to perform independently. Among them Alzheimer’s disease stands out, but there are many others such as vascular dementia, dementia with Lewy bodies or frontotemporal dementia.

How to intervene to reduce problems associated with dementias?

The answer would be related to the obtaining of early and accurate diagnoses that allow effective therapies to be performed and specifically directed at each disease. In 2017, the World Health Organization (WHO) already declared the need to improve diagnoses to respond to the increase in dementia cases worldwide. However, the different types of dementia are sometimes difficult to differentiate in early stages because the symptoms associated with each dementia are complex. Added to this is that different dementias can sometimes coexist which leads to a large number of diagnostic errors.

Gold Standard

These diagnoses have been mainly based on neuropsychological assessments. However, these diagnoses based on cognitive and behavioral tests have been attempted to be linked with tests that evaluate biological samples such as MRI scans, blood samples or cerebrospinal fluid, among others. Despite large efforts, a diagnostic “Gold Standard” test is still lacking. This is the name given to a diagnostic test that has high reliability when diagnosing a specific disease. This Gold Standard has been sought mainly through biological data without success so far.

Solution based on artificial intelligence (AI)

To address this situation, a group of researchers from Boston University together with researchers from across the United States proposed a solution based on artificial intelligence that would allow working with a large set of heterogeneous data. These data are heterogeneous because it is based on mimic the real world; that is, using the same tools that a clinical professional would use to provide a solution to the diagnostic and prevention problem that exists with dementias. The multiple types of data they addressed are: sociodemographic, neurological, physical, medical history and MRI scans.

What has been done?

To be able to apply an AI-based model a massive amount of data was needed so the researchers turned to nine independent datasets bringing together more than 51,000 patients with different types of dementia.

Patients with different types of dementia

The inclusion of a wide range of dementias with multiple causes was a fundamental step to obtain rich and varied characterization that could be representative of reality. Therefore patients were included who had dementia due to many different causes such as Alzheimer’s, strokes, frontotemporal degeneration, corticobasal degeneration, due to infections, drug abuse, etc.

Missing data problem

However, this approach based on a massive amount of data also makes it likely that there will be missing data. This is the opposite case to very controlled settings with a smaller number of patients, in which data loss is minimized. In this case, the approach requires very heterogeneous large datasets that allow the model to learn and be as realistic as possible. To solve the problem related to data loss, the researchers applied techniques that will provide maximum robustness during the early stages. The implementation of these robust methods and strategies prevents the AI training from being biased and therefore future predictions as well. In addition, standardized inclusion and exclusion procedures were followed that ensured the consistency and credibility of the results.

“Transformer”-type model

At this point, an important challenge is also raised which is the creation of a model that allows aggregating multiple types of data and parameters of different natures, which is called a “transformer” architecture. In this type of model all the different included features are taken and transformed into what is called a “fixed-length vector” following a specific strategy that allows creating a first layer of the model on which the rest will be built decoding this into a series of predictions.

To understand it better, we can think of this model like a recipe only that instead of different steps as we have in a recipe, the model is going to have different layers. But in the same way that to have a good dish after cooking we need good ingredients that will combine properly. The researchers wanted a good system for classifying patients and needed their ingredients; that is, the data they started from to be good and to be able to combine properly. In our analogy with this AI model it would be transform the different data we have into a common format (fixed-length vector). Because of this the created model will be very robust to data loss or incompleteness and will be able to make reliable predictions.

Self-supervised learning

Specifically they use a type of model called self-supervised learning. In this machine learning approach the model learns from data without the need for explicit labels. Unlike supervised learning, which requires a labeled dataset, self-supervised learning is based on finding structures and patterns in the data without direct human intervention to label them.

Training, validation and comparison with experts

What followed in the model development was the training, validation and comparison with experts phase. Training and validation processes are common in this type of study.

The training consisted of feeding the model using data from the different cohorts, integrating all modalities to learn patterns associated with different types of dementia. After training the model, validation was carried out using previously unseen data to ensure that the model can generalize its predictions to data it does not know.

Finally and in a novel way, the researchers tested the diagnoses made by the model with physicians specialized in dementia diagnosis. For this, they invited a group of 12 neurologists and 7 neuroradiologists to participate in diagnostic tasks for a subset of 100 cases with different types of dementia. They were given the data available for each of these 100 cases and asked to provide their diagnostic impressions, as well as a confidence score ranging from 0 to 100 for the diagnosis of each of the 13 possible labels. At this point they wanted to know if clinical judgments could be improved using the created model.

Subscribe
to our
Newsletter

What are the main conclusions of this dementia and artificial intelligence (AI) study?

The main results can be divided into three parts: healthy vs impairment classification, pathology classification and improvement of clinical judgment using the model.

The study evaluated the performance of an artificial intelligence (AI) model to classify individuals into three cognitive categories:

healthy or cognitively normal,
mild cognitive impairment
and dementia.

The metrics used to evaluate the model’s performance were ROC and PR curves. The results obtained can be seen in the following (table 1).

The metrics obtained for the ROC models reflect a high ability of the model to differentiate the three mentioned classes, since these metrics are evaluated from 0 to 1, with a value close to one indicating excellent classification.
For the AUAP metrics the values continue to indicate good model performance, requiring consideration of each class.

	AUROC	AUAP
Micro	0.94	0.90
Macro	0.93	0.84
Weight	0.94	0.87

Table 1. Model performance for healthy vs. impairment classification.

The second result of the study related to the model’s ability to diagnose ten different types of dementia, showing very significant results regarding its diagnostic accuracy.

The AUROC metric results again show great effectiveness of the model, being able to distinguish in most cases between the types of dementia.
On the other hand, the AUPR metrics continue to show that the model has good performance, although not excellent (see table 2), which would indicate that the model is more accurate for some diseases than others.

	AUROC	AP
Micro	0.96	0.70
Macro	0.90	0.36
Weight	0.94	0.73

Table 2. Average model performance for classifying dementias.

Finally, regarding AI-assisted clinical evaluation, the results indicate that it could increase diagnostic accuracy compared to what a clinical professional would achieve on their own. This is visible by the significant improvement in the classification of dementia types. For example:

The neurologists improved:

by 12% in detecting mild cognitive impairment,
by 15% for Alzheimer’s disease,
by 26% for frontotemporal dementia according to AUROC values.
particularly notable was the improvement for prion disease which improved by 73%.

The radiologists:

for Alzheimer’s-type dementia improved by 9%
by 6% for frontotemporal dementia,
notable also was the improvement for prion disease with 68%.

In conclusion, we can infer that future AI-based models could be of great help for diagnostic differentiation when it relies on the multifactorial nature of the different types of dementia. This could not only improve diagnosis but also facilitate the personalization of treatments and interventions in early stages of the disease.

Where could NeuronUP contribute to a study like this?

NeuronUP can contribute in many different ways to a study like this.

From NeuronUP one can contribute to the implementation of structures and technologies that would allow replicating or reproducing a study of these characteristics given that the datasets used are freely available.
The study could be replicated or reproduced using a different procedure that would be more precise or less costly.
In addition, techniques and models could be applied that allow being more precise and differentiate which types of data contribute more to the model or if good classification metrics can be obtained using only one type of data. An example of this would be using neuropsychological and sociodemographic data to try to obtain similar results.
Furthermore, NeuronUP has a large neurocognitive database so it could try to perform similar classifications that pinpoint which neurocognitive aspects could facilitate early detection and the prediction of decline. This is especially relevant given that dementias continue to have a very significant neuropsychological and behavioral impact. In recent years, great progress has been made in the search for biomarkers, but the patient’s quality of life remains paramount and the main concern for them and their loved ones. Therefore greater precision in this area or the development of a cognitive profile based on cognitive tests could be of great use.
Continuing with the neurocognitive section, since this is not widely addressed in the article the large databases NeuronUP has could be used to reproduce the study with a new cohort of patients. In addition, the type of neuropsychological data used in the study is not described; there may be different cognitive domains that are more relevant.

Principal investigator: Dr. Vijaya B. Kolachalama is an associate professor at Boston University and principal investigator of the Kolachalama Lab at the same university. His work is mainly directed to the application of artificial intelligence to medical problems. His mission is to create tools that help neurologists in real-world scenarios, mainly in neurodegenerative diseases.

Bibliography

Xue, C., Kowshik, S.S., Lteif, D. et al. “AI-based differential diagnosis of dementia etiologies on multimodal data”. Nat Med (2024). https://doi.org/10.1038/s41591-024-03118-z