The multifactoriality of neuropsychological tests: What do we value when we evaluate?

Neuropsychologist and researcher Ángel Martínez Nogueras addresses the problem of the multifactorial nature of neuropsychological tests. Specifically, the Stroop Test.

Let’s set the scene. Imagine an ordinary week in which you are evaluating several patients in the clinic. You are administering the Stroop Test, and on the Word-Color sheet you find the following:

• Patient 1 reaches item 23, makes 5 errors and corrects all 5 himself, without the evaluator’s intervention;
• patient 2 reaches item 19, makes 8 errors, but only detects and self-corrects 1; the evaluator gives immediate feedback after each error, the subject corrects himself, but continues making errors on subsequent items;
• subject 3 reaches item 21, does not make any errors, but hesitates on many items and almost makes mistakes before giving the correct response, finding it very effortful to control the response and not err;
• Subject 4 reaches item 9, committing 9 unself-corrected errors, does not benefit from the evaluator’s feedback, and abandons the test at item 9 due to the effort it requires, not reaching the full 45 seconds.

These are real cases from my practice and, respectively, are:

• A 54-year-old patient with stroke,
• a 32-year-old patient with traumatic brain injury,
• a 66-year-old patient with Cerebral Amyloid Angiopathy,
• a 69-year-old patient with Parkinson’s Disease.

In view of the above, can we conclude that in all four cases the low score they obtain is mediated by an inhibitory deficit? In other words, can we diagnose an inhibition deficit in all four patients?

I’ll let you anticipate the answer: indeed, no. So, where does this leave the almost routine assumption in neuropsychology that poor performance on the Word-Color part can be interpreted as a reflection of a deficit in inhibitory processes? I don’t know the exact place, but it’s certainly far from the goals of neuropsychological assessment.

Quoting Banich¹, “taking into consideration only the final score of the Stroop task is a limitation in understanding the Stroop effect, since that score reflects the sum of the cognitive processes involved in its resolution, but it does not inform us about the relative participation of those processes during its resolution”. I could not have expressed it better.

To clarify the above and specify the underlying problem addressed here, which is none other than the multifactorial nature of tests or assessment measures, I will focus on one in particular, the Stroop Test.

The multifactorial nature of neuropsychological tests: the Stroop Test

Below I present a brief summary of the complexity of the cognitive network required to successfully perform the Stroop Test. I will go beyond the explanatory capacity of theoretical models of inhibition and rely on the interesting concept of cognitive control.

To clarify, cognitive control would not be a cognitive function per se. Rather, it would emerge at a given moment through the specific interaction of cognitive processes such as working memory, cognitive flexibility, monitoring, inhibition and response selection. Its purpose would be to provide a dynamic and flexible adjustment of behavior according to the goals and demands of the task at hand, especially in non-routine situations that require non-habitual or non-prepotent responses, as is the case with the Stroop Test^2,3.

Cognitive processes included in the Stroop Test

When breaking down the cognitive control process as it is engaged to solve the Stroop Test, we would deploy a cascade of cognitive processes that includes:

Working memory

The relevance of working memory in tasks traditionally considered inhibitory is increasingly supported in the literature. In this respect, Tiego et al (2018) proposed a hierarchical model of inhibitory control in which working memory is situated as a higher-order process that modulates the performance of other cognitive processes, in this case, inhibition.

During the Word-Color part of the Stroop Test, a failure to maintain a sufficient activation level in working memory of both the task goals and the relevant information (naming the color of the word) could cause a failure in inhibition. Consequently, it would facilitate the emission of a preponderant response that is not appropriate to the context (reading the word).

Viewed another way, individuals with higher working memory performance would also be less susceptible to making inhibitory errors.

Inhibition

The process of inhibition, on the one hand, would prevent interference or the access to working memory of information that is not relevant for the correct performance of the task, facilitating the preferential processing of relevant information, that is, the color of the words.

And, on the other hand, it would allow “stopping” an automated response, such as reading words, enabling the selection and emission of an alternative response according to contextual demands, in this case, naming the ink color.

Therefore, one could say that for the resolution of the Stroop Test working memory and inhibition must act in a coordinated way. Indeed, there are studies that show that those subjects with greater inhibitory control and better working memory show a smaller Stroop effect than those who have poor inhibitory control and low working memory capacity^5,6.

Cognitive flexibility

The Word-Color part of the Stroop Test, besides posing a novel or unusual situation, also requires a rapid or agile change or updating both of the goal and of the response set compared to the previous parts of the test. That is, the Word and Color parts, since one goes from a situation in which one must respond to congruent stimuli, which require habitual or already learned responses, to another that requires producing a novel or infrequent response to incongruent stimuli that generate conflict.

Thus, a deficient performance of the cognitive flexibility process could lead to the emission of inadequate or perseverative responses. Either due to a lack of adjustment to new goals, by maintaining a previously successful but now inadequate response set according to the new task demands (response inertia), or because of difficulty changing or adjusting the response based on feedback received after committing an error^7,8.

Monitoring

The cognitive process of monitoring is usually characterized as an executive process in charge of timing the development of the activity, supervising that behavior adjusts to the requirements of the task or environment, and detecting errors or discrepancies between the response and the goal of the ongoing task, indicating that an update or change of response is required⁹.

Within the mechanism that constitutes cognitive control, for the monitoring process to act successfully it is necessary that working memory keeps the task goal active. Its purpose is to allow supervising that behavior remains aligned with it. And, furthermore, that cognitive flexibility acts appropriately, adjusting the response set in those cases where an error is detected or negative feedback is received about the outcome of the response given¹⁰.

An aspect related to monitoring studied using the Stroop task focuses on the analysis of both errors and self-corrections, analyzed separately, as well as the adjustment of the response based on feedback received after committing an error¹¹.

The previous viewpoint emphasizes that, alongside recording the number of items a patient is able to read, it would be truly interesting to collect qualitative aspects related to the analysis of errors, as a way to enrich the data and increase the precision of the conclusions drawn.

Processing speed

It is well known that slowing in information processing speed can negatively affect the performance of cognitive processes such as inhibition, working memory and flexibility during the execution of complex tasks.

On the other hand, we have publications and construct validity studies on the Stroop Test that indicate processing speed is an important factor for its proper resolution¹². Not for nothing, performance in the different parts of the test is measured based on the number of items a subject reaches in 45 seconds, that is, their execution speed.

Conclusion

Ultimately, and at heart, this text addresses the issue of the multifactorial nature of the tests we usually use for assessing cognition. Our goal in clinical practice of neuropsychology is not to tell a patient where their limit lies, that limit being established by psychometrics, but rather what causes that limit and how we can overcome it.

Imagine the magnitude of the error and the limitation in the development of neuropsychology into which we fall if we base our working hypotheses and clinical conclusions solely on the quantitative interpretation of a patient’s final test score.

In contrast, our goal can be nothing other than to approach the evaluation of a subject with the perspective or intention of unraveling the network of cognitive processes that underlies the performance of the tasks we propose, in order to determine the particular involvement of each of them in their attempt at resolution.

Some time ago I etched into my memory the following statement by Benedet91 “neuropsychology begins where psychometrics ends”, to which one could add that the path after psychometrics must be paved with theoretical models, and the better the theoretical model, the firmer the progress will be.

The text you have just read is a brief summary of an article published in the Revista Iberoamericana de Neuropsicología titled Stroop Test, more than inhibition. A review under the concept of cognitive control. At the following link you have access to the full article: https://neuropsychologylearning.com/portfolio-item/test-de-stroop-algo-mas-que-inhibicion-una-revision-bajo-el-concepto-de-control-cognitivo/

Bibliography

1. Banich MT. The Stroop Effect Occurs at Multiple Points Along a Cascade of Control: Evidence From Cognitive Neuroscience Approaches. Front Psychol. 2019 Oct 9;10:2164.
2. Lenartowicz A, Kalar DJ, Congdon E, Poldrack RA. Towards an ontology of cognitive control. Top Cogn Sci. 2010 Oct;2(4):678-92.
3. Soutschek A, Strobach T, Schubert T. Working memory demands modulate cognitive control in the Stroop paradigm. Psychol Res. 2013 May;77(3):333-47.
4. Tiego J, Testa R, Bellgrove MA, Pantelis C, Whittle S. A Hierarchical Model of Inhibitory Control. Front Psychol. 2018;9:1339.
5. Kane MJ, Engle RW. Working-memory capacity and the control of attention: the contributions of goal neglect, response competition, and task set to Stroop interference. J Exp Psychol Gen. 2003 Mar;132(1):47-70.
6. Duell N, Icenogle G, Silva K, Chein J, Steinberg L, Banich MT, et al. A cross-sectional examination of response inhibition and working memory on the Stroop task. Cognitive Development 2018; 47: 19-31.
7. Miyake A, Friedman NP. The Nature and Organization of Individual Differences in Executive Functions: Four General Conclusions. Curr Dir Psychol Sci. 2012; Feb;21(1):8-14.
8. Amieva H, Lafont S, Rouch-Leroyer I, Rainville C, Dartigues JF, Orgogozo JM, Fabrigoule C. Evidencing inhibitory deficits in Alzheimer’s disease through interference effects and shifting disabilities in the Stroop test. Arch Clin Neuropsychol. 2004 Sep;19(6):791-803.
9. Tirapu-Ustárroz J, García-Molina A, Luna-Lario P, Roig-Rovira T, Pelegrín-Valero C. Models of executive control and functions (I) [Modelos de funciones y control ejecutivo (I)]. Rev Neurol. 2008 Jun 1-15;46(11):684-92.
10. Grange JA, & Houghton G. Task switching and cognitive control: An introduction. In Grange JA, & Houghton G, eds. Task switching and cognitive control. Oxford University Press; 2014. p 1–26.
11. Miller, A., Bryant, K., Martincin, K., Livers, E., Martukovich, R., & Poreh, A. M. (2009, August). Examining the Self-Corrected and Non-Self-Corrected Errors on the Stroop Test. In Archives of Clinical Neuropsychology (Vol. 24, No. 5, pp. 441-441). Great Clarendon st, Oxford OX2 6DP, England: Oxford Univ Press.
12. Periáñez JA, Lubrini G, García-Gutiérrez A, Ríos-Lago M. Construct Validity of the Stroop Color-Word Test: Influence of Speed of Visual Search, Verbal Fluency, Working Memory, Cognitive Flexibility, and Conflict Monitoring. Arch Clin Neuropsychol. 2020 Jun 9:acaa034

If you liked this article about the multifactorial nature of neuropsychological tests, you might also be interested in:

“This article has been translated. Link to the original article in Spanish:”
La multifactorialidad de los test neuropsicológicos: ¿Qué valoramos cuando valoramos?