Vol. 3 nº 2 - Apr/May/Jun de 2009
Original Article Pages 81 to 87
 

The performance of the Mini-Cog in a sample of low educational level elderly
O desempenho do Mini-Cog em uma amostra de idosos com baixo nível educacional

Authors: Sergio Telles Ribeiro Filho1, Roberto Alves Lourenço2

PDF

Descriptors: neuropsychology, dementia, mass screening, aging, ambulatory care, validity.
Descritores:
neuropsicologia, demência, envelhecimento, programas de rastreamento, assistência ambulatorial, validade.

ABSTRACT:
Objectives: To study the criterion validity of the Mini-Cog in low educational level elderly. Design: Cross-sectional and validation design. Setting: Policlínica Piquet Carneiro, an outpatient unit of Rio de Janeiro State University Hospital, in Brazil. Participants: A convenient sample consisting of 306 individuals, 65 yrs or older, selected from April 8th to July 15th, 2002. Methods: All participants underwent comprehensive geriatric evaluations which included the Mini-Mental State Examination (MMSE) and the cognitive part of the Cambridge Examination for Mental Disorders of the Elderly - Revised (CAMCOG-R). They were classified as demented or non-demented (DSM-IV). A post-hoc analysis was performed on the data from the 3 word recall test of the MMSE, and the Clock Drawing Test from the CAMCOG-R, and respective scores were added and interpreted in accordance with the Mini-Cog protocol. Results: 293 individuals completed all the study steps; 211 had 4 or less years of schooling and were included in the data analysis. 32% had dementia. Mini-Cog sensitivity and specificity was consistently low independently of the different cut-off points considered. The best performance was found at the cut-off point of 2/3 which yielded sensitivity and specificity of 60% and 65%, respectively. Conclusion: The Mini-Cog is not a good cognitive screening tool for individuals with less than five years of formal education.

RESUMO:
Estudar a validade de critério do Mini-Cog em idosos com baixo nível educacional. Desenho: transversal e de validação. Cenário: Policlínica Piquet Carneiro, uma unidade ambulatorial do Hospital da Universidade do Estado do Rio de Janeiro, no Brasil. Participantes: Uma amostra de conveniência constituída de 306 indivíduos, 65 anos ou mais, selecionados entre 8 de abril e 15 de julho de 2002. Métodos: Todos foram submetidos a uma avaliação geriátrica abrangente, que incluiu o Mini-Exame do Estado Mental (MEEM) e a parte cognitiva do Cambridge Examination for Mental Disorders of the Elderly - Revised (CAMCOG-R), e foram classificados segundo sua condição de ser ou não portador de demência (DSM-IV). Foi feita uma análise post-hoc dos dados do teste de evocação de três palavras, incluído no MEEM, e do Teste do Desenho do Relógio, incluído no CAMCOG-R. Os resultados de cada teste foram agrupados e interpretados de acordo com o protocolo do Mini-Cog. Resultados: 293 indivíduos completaram todas as etapas do estudo, e 211 tinham quatro ou menos anos de escolaridade e tiveram seus dados analisados; 32% tinham demência. A sensibilidade e a especificidade do Mini-Cog foram consistentemente baixas, independente do ponto de corte considerado. O melhor desempenho, no ponto de corte 2/3, registrou sensibilidade e especificidade de 60% e 65%, respectivamente. Conclusão: O Mini-Cog não é uma boa ferramenta para triagem cognitiva de indivíduos com menos de cinco anos de educação formal.

Dementia is common in the geriatric population. It has been suggested that early detection leads to benefits for the patient and their family,1,2 and cuts costs.3 Early diagnosis thus constitutes an important factor where achieving this depends upon adequate screening performed at the primary care level. However, studies have shown that dementia is underdiagnosed by generalists, and this seems to be due to a relative reluctance of these professionals to apply cognitive screening tests, either because they are time consuming to apply, or because they are perceived as being uncomfortable for the patient or their family.4-6

The solution to this problem lies in the development of brief and easily applicable tests that are acceptable to patients, their caregivers, and health professionals. Additionally, these instruments must perform well in populations with heterogeneous characteristics with regard to age, cultural diversity, or diverse levels of education.

This latter point is of particular relevance in developing countries, which tend to have a large proportion of elderly with less than 5 years of formal schooling. More specifically for Brazil, the projected rapid expansion of the elderly population as a whole7 will result in a steep increase in the number of patients with dementia. Most of the instruments for detecting dementia currently in use however, such as the Mini-Mental State Examination (MMSE) 8 and the Clock Drawing Test (CDT),9-14 were developed in countries where mean educational levels are much higher. Both tests have been shown to perform poorly when applied to very low educational level elderly.15,16

The developers of the Mini-Cog sought to enhance CDT performance by adding a simple learning test, namely, the 3 word recall test.17-20 In this test, subjects are given a list of three words, just as in the MMSE, and the clock drawing is used as a distracter.

In the original paper, the Mini-Cog performed better than either the MMSE or the CDT alone in a sample with a relatively large ethnic and educational heterogeneity and which had a high prevalence of dementia. However, the number of subjects with less than 5 years' education was probably low, since the mean educational level of the demented patients was more than 8 years. No separate results for test performances in the subgroup with less than 5 years' schooling were cited.17 In another article, in which the Mini-Cog is applied to a large community-derived sample with a much lower prevalence of dementia, having less than 6 years of formal education was one of the exclusion criteria.18 Therefore, there is still little information on the psychometric characteristics of this test in low educational level elderly.

The objective of this study was to determine the accuracy of the Mini-Cog in an elderly population with predominantly low educational levels, treated in a primary care outpatient clinic.


Method

Sample selection


Between April 8th and July 15th, 2002, a convenient sample of 306 individuals was selected from elderly individuals who were seeking general medical treatment at the Internal Medicine Clinic of the Policlínica Piquet Carneiro, an outpatient unit of Rio de Janeiro State University Hospital. Individuals older than 65 years who attended our screening center seeking a general physician's office were invited to participate in a study to validate instruments to screen for dementia.21 The number of participants recruited daily depended on how many accepted the invitation, and was limited by the absorptive capacity of the research team at the time.

Inclusion criteria were having an age over 65 years and preserved hearing and comprehension, at least enough to fully participate in the study and sign an informed consent form. Exclusion criteria were reports - personal or through an informant - of a serious uncorrected visual or auditory deficiency; being at an advanced stage of cognitive disturbance, or having any mental illness that could compromise understanding and performance on the test procedures; having a native language other than Portuguese; difficulty in hand movement due to rheumatic or neurological diseases. After signing the informed consent form, the subjects were referred for a comprehensive geriatric evaluation, administered by a multi-professional team - a geriatrician, a registered nurse practitioner, a social worker, and a neuropsychologist. At the end of the evaluation, a meeting between the geriatrician and the neuropsychologist classified the patients into one of two dementia syndrome-based groups: demented and non-demented. Therefore, the confirmatory standard for the diagnosis of dementia was the consensual opinion of both professionals, which took into account both the clinical impression and the neuropsychological evaluation, and was based on the DSM-IV22 diagnostic criteria for dementia syndrome. Dementia severity was graded in accordance with the modified protocol used by the Brazilian Health Ministry to determine dispensing of medications for patients with Alzheimer's disease (http://dtr2001.saude.gov.br/sas/PORTARIAS/PORT2002/PT-843.htm). Thus, for the purposes of the present study, those whose scores were less than eight were classified as having severe dementia, those with scores between eight and seventeen points were assigned as having moderate dementia, and those who scored more than seventeen points were classified as having mild dementia. The patients were not further classified as to the cause of their dementia.

The Rio de Janeiro State University Hospital Ethics Committee approved the research protocol, including the Informed Consent Form. The study was supported entirely by the Rio de Janeiro State University.

Post-Hoc Mini-Cog and study procedures

In addition to the clinical assessment algorithms pertaining to each of the specialties involved, the individuals were submitted to a functional evaluation, which included the Activities of Daily Living Scale23 and the Instrumental Activities of Daily Living Scale;24 and were submitted to the Geriatric Depression Scale;25 and to the MMSE.8 The neuropsychometric tests included the MMSE and the cognitive part of the Cambridge Examination for Mental Disorders of the Elderly - Revised (CAMDEX-R),26 the CAMCOG-R. This includes a clock drawing task that uses the following instruction: "Draw the face of a large clock, place all the numbers inside and place the hands to show 11:10 (ten minutes past eleven)". The clock drawing of each patient - copied and identified with its respective register number - had previously been analyzed and scored retrospectively as part of another study,15 in accordance with Sunderland's method, by researchers who had no access to the patient's file and were blinded to the cognitive condition of the subjects assessed. Post-hoc analysis was performed as follows: the clock scores obtained following application of Sunderland's method greater than or equal to six were considered "normal", whereas scores less than six were considered "abnormal". In accordance with the original Mini-Cog methodology, as defined by Borson and Scanlan17,19 "The optimal algorithm had the following three rules: subjects recalling none of the words were classified as demented; those recalling all three words were classified as non-demented; and those with intermediate word recall (1-2) were classified based on the CDT ("abnormal=demented; normal=non-demented)".17

The data were entered and analyzed using the program SPSS v 9.0. ROC curves were plotted and the areas under the curves, their confidence intervals, and the best trade-offs between sensitivity and specificity were calculated. Comparisons between categorical variables were made using Pearson's chi-square test.


Results

A total of 306 subjects were recruited; 293 completed all the study procedures; 211 had 4 or less years of schooling and had their data analyzed. Of these, 153 (72.5%) were female, 59.7% stated they did not live with a partner, 82.5% were not working, 64.9% were retired, and 37% stated they had never attended school. In addition, 71.1% were under 75 years and 12.3% were older than 80 years of age (Table 1). According to the DSM-IV,22 32.2% fulfilled criteria for dementia syndrome. Age ranged from between 65 and 93 years, and had a mean (±SD), median and mode of 72.8 (±5.4), 72, and 73, respectively; mean ages were 74.0 yrs (±5.8) and 72.0 yrs (±5.0) for demented and non-demented groups, respectively. The mean (±SD), median and mode number for years of schooling were 1.8 (±1.7), 2, and 0 respectively. The means (±SD) and median for demented and non-demented groups were 1.28 (±1.49) and 0.5, and 2.06 (±1.66) and 2, respectively. As shown in Table 1, there was a statistically significant difference between demented and non-demented groups in the variables "Work" and "Schooling." The mean±SD of the MMSE scores for the demented and non-demented groups were 18.8±4.2 and 23.3±4.1, respectively (p=0.000), while medians were 19 and 24, respectively. Figure 1 summarizes the performances on the MMSE, stratified by years of schooling.





Figure 1. Box plot of Mini-Mental State Examination performance in a sample of low educational level elderly stratified by years of schooling.



Among the 68 individuals classified as having dementia, 44 (65%) were at a mild stage, and 24 (35%) were at a moderate stage whereas no patients had severe dementia.

The sensitivity and specificity of the Mini-Cog were 60% and 65%, respectively, using the original cut-off point of 2/3. Table 2 and Figure 2 demonstrate that using other cut-off points did not improve performance. The accuracy of the Mini-Cog also does not improve when its psychometric characteristics are calculated stratifying the sample by level of dementia severity, as shown in Figure 3.






Figure 2. ROC curve of Mini-Cog performance in sample of low educational level elderly.



Figure 3. ROC curve of Mini-Cog performance in sample of low educational level elderly, stratified by level of severity.



Discussion

The accuracy of the Mini-Cog as a screening tool for dementia in this sample of elderly with low educational levels was disappointingly poor. The psychometric characteristics calculated, e.g., sensitivity, specificity, and area under the ROC curve, approached random levels.

This contrasts to its performance in samples with higher levels of education, where it attains sensitivities and specificities greater than 90%17. This divergence was not surprising. The hypothesis that education is a crucial variable affecting the results of neuropsychological tests in general was previously demonstrated by Ostroski-Solis et al.27 This effect is much stronger than age, for example, and is most significant in those groups with the lowest levels of schooling, especially in those with less than five years formal education. Ardila et al.28 compared the performance of illiterates and educated professionals on a wide variety of neuropsychological tests and found that almost all of the abilities tested were strongly influenced by education. Abilities most highly dependent upon on schooling were visual and constructional abilities, abstract reasoning, and memory, including recall of word lists.

We have recently published results which indicate that even the CDT, which was once thought to be resistant to educational heterogeneity, does not in fact perform well in this population group.15 Other authors have also described the adverse effect of low schooling levels on CDT performance.16,29 The skills needed to draw a clock are apparently highly dependent on formal education, since it is not uncommon for low educational level non-demented subjects to be unable to draw a clock face, yet be able to tell the time indicated on a clock and to reason using concepts linked to the notion of the passage of time,30 abilities which are clearly necessary for functional independence. For comparative purposes, application of the CDT alone using Sunderland's scoring method in our group of patients yielded 59% sensitivity and 64% specificity.15

With regard to the 3-item recall test, which is also widely used as a brief test of verbal memory, and has been shown to have a high correlation with the results of the full MMSE,31,32 there is less information about how educational levels affect its performance. Even though it has some degree of correlation with other, more sophisticated psychometric tests, there is great variability in results when it is applied to normal individuals, with many non-demented patients recalling zero or only one word.33,34 Given Ostroski et al have demonstrated that a six-word recall test was highly influenced by education,27 it is likely that the 3 word recall is also influenced by this variable.

Therefore, although the Mini-Cog has definite advantages when applied to more highly educated patient groups it is not surprising that our data does not support its use in older individuals with low levels of education.35

Our study had some limitations. One such limitation was that complete information was not available from third parties (caregivers, relatives) for most of the patients evaluated, and this could have led to some misclassification errors.

Another point is that both the clock drawing and 3-word evocation tests were part of the neuropsychological test battery included in our diagnostic criteria algorithm. This information was included as part of that used to establish the diagnosis, and although these items represented only a small section of the full test they may have introduced some information bias.

The third aspect is related to limitations inherent to the post-hoc methodology itself. For example, we used a different scoring system for the clock drawings than that used in the original Mini-Cog. On the other hand, we demonstrated in a previous study that the psychometric properties of four different CDT scoring methods (one of which was Sunderland's, the others being Shulman's, Manos', and Wolf-Klein's methods) were very similar (they all performed very poorly) in a population of low educational level elderly,15 therefore the choice of which to use is probably irrelevant.

On the other hand, a positive aspect was that this sample was composed of individuals who were seeking primary health care, having signs and symptoms not necessarily related to cognitive disorders, and who accepted our invitation over consecutives days during the study period. This probably reduced the problem of selection bias, frequently seen in studies that select their samples based on diagnosis condition of the subjects, with the resultant risk of inflating the accuracy of the test.

In conclusion, the Mini-Cog is strongly influenced by educational level, and although there is some evidence in the international literature that it can be used to screen older subjects with higher educational levels, it appears unsuitable for dementia screening in individuals with less than 5 years of formal education.

Grant supports - The study was supported entirely by the Rio de Janeiro State University.


References

1. Lopez OL, Becker JT, Wisniewski S, et al. Cholinesterase inhibitor treatment alters the natural history of Alzheimer's disease. J Neurol Neurosurg Psychiatry 2002;72:310-314.

2. Hashimoto M. Does donepezil treatment slow the progression of hippocampal atrophy in patients with Alzheimer's disease? Am J Psychiatry 2005;62:676-682.

3. Lyseng-Williamson KA. Galantamine: a pharmacoeconomic review of its use in Alzheimer's disease. Pharmacoeconomics 2002;20:919-942.

4. Brodaty H. Screening for cognitive impairment in general practice: toward a consensus. Alzheimer Dis Assoc Disord 1998;12:1-13.

5. Ganguli M. Detection and management of cognitive impairment in primary care: The Steel Valley Seniors Survey. J Am Geriatr Soc 2004;52:1668-1675.

6. Valcour VG, Masaki KH, Curb JD, Blanchette PL. The detection of dementia in the primary care setting. Arch Intern Med 2000;160:2964-2968.

7. Silvestre J, Kalache A, Ramos LR, Veras RP. Population ageing in Brazil and the health care sector. Bold: Q J Int Inst Ageing 1998;7:4-12.

8. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189-98.

9. Shulman, KI. Clock-drawing: Is it the ideal cognitive screening test? Int J Geriatr Psychiatr 2000;15:548-561.

10. Schramm U, Berger G, Müller R, et al. Psychometric properties of Clock Drawing Test and MMSE or Short Performance Test (SKT) in dementia screening in a memory clinic population. Int J Geriatr Psychiatr 2002;17:254-260.

11. Manos PJ. The utility of the ten-point clock test as a screen for cognitive impairment in general hospital patients. Gen Hosp Psychiatry 1997;19:439-444.

12. Shulman KI, Gold DP, Cohen CA, Zucchero CA. Clock-drawing and dementia in the community: a longitudinal study. Int J Geriatr Psychiatr 1993;8:487-496.

13. Wolf-Klein GP, Silverstone FA, Levy AP, et al. Screening for Alzheimer's Disease by clock drawing. J Am Geriatr Soc 1989;37:730-734.

14. Sunderland T, Hill JL, Mellow AM, et al. Clock drawing in Alzheimer's Disease. A novel measure of dementia severity. J Am Geriat Soc 1989;37:725-729.

15. Lourenço RA, Ribeiro-Filho ST, Moreira IFH, Paradela EMP, Miranda AS. The Clock Drawing Test: performance in low educational level elderly. Rev Bras Psiquiatr 2008;30:309-315.

16. Storey JE, Rowland JTJ, Basic D, et al. Accuracy of the Clock Drawing Test for detecting dementia in a multicultural sample of elderly Australian patients. Int Psychogeriatr 2002;14:259-271

17. Borson S, Scanlan J, Brush M, et al. The Mini-Cog: a cognitive "vital signs" measure for dementia screening in multi-lingual elderly. Int J Geriatr Psychiatry 2000; 15:1021-1027

18. Borson S, Scanlan J, Chen P, Ganguli M. The Mini-Cog as a screen for dementia: validation in a population-based sample. J Am Geriatr Soc 2003;51:1451-1454

19. Scanlan J, Borson S. The Mini-Cog: receiver operating characteristics with expert and naïve raters. Int J Geriatr Psychiatry 2001;16:216-222.

20. Borson S, Scanlan JM, Watanabe J, Tu SP, Lessig M. Simplifying detection of cognitive impairment: comparison of the Mini-Cog and Mini-Mental State Examination in a multiethnic sample. J Am Geriatr Soc 2005;53:871-874.

21. Lourenço RA, Veras RP. Mini-Exame do Estado Mental: características psicométricas em idosos ambulatoriais. Rev Saude Publica 2006;40:712-719.

22. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th ed. Washington, DC; 1994.

23. Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged. The index of ADL: a standardized measure of biological and psychosocial function. JAMA 1963;185:94-99.

24. Lawton MP, Brody EM. Assessment of older people: self maintaining and instrumental activities of daily living. Gerontologist. 1969;9:179-186.

25. Yesavage JA, Brink TL, Rose TL, et al. Development and validation of a geriatric depression screening scale: a preliminary report. J Psychiatr Res 1982;17:37-49.

26. Roth M, Huppert FA, Montjoy CQ, Tym E. CAMDEX-R: the Cambridge examination for mental disorders of the elderly. Cambridge: Cambridge University Press; 1999.

27. Ostrosky-Solis F, Ardila A, Rosselli M et al. Neuropsychological test performance in illiterate subjects. Arch Clin Neuropsychol 1998;13:645-660

28. Ardila A, Rosselli M, Rosas P. Neuropsychological assessment in illiterates: visuospatial and memory abilities. Brain Cogn 1989;11:147-166

29. Marcopulos BA, Gripshover DL, Broshek DK, McLain RH. Neuropsychological assessment of psychogeriatric patients with limited education. Clin Neuropsychol 1999;13:147-56.

30. Tuokko H, Hadjistavropoulos T, Miller JA, Beattie BL. The Clock Test: a sensitive measure to differentiate normal elderly from those with Alzheimer disease. J Am Geriatr Soc 1992;40:579-584.

31. Fillenbaum GG, Wilkinson WE, Welsh KA, Mohs RC. Discrimination between stages of Alzheimer's disease with subsets of Mini-Mental State Examination items. An analysis of Consortium to Establish a Registry for Alzheimer's Disease data. Arch Neurol 1994;51:916-921.

32. Braekhus A, Laake K, Engedal K. The Mini-Mental State Examination: identifying the most efficient variables for detecting cognitive impairment in the elderly. J Am Geriatr Soc 1992;40:1139-1145.

33. Chandler MJ, Lacritz LH, Cicerello AR, et al. Three-word recall in normal aging. J Clin Exp Neuropsychol 2004;26:1128-1133.

34. Cullum CM, Thompson LL, Smernoff EN. Three-word recall as a measure of memory. J Clin Exp Neuropsychol 1993;15: 321-329.

35. Lourenço RA, Ribeiro-Filho ST. The accuracy of the Mini-Cog in screening low-educated elderly for dementia. J Am Geriatr Soc 2006;54:376-378.









1 MD, Laboratório de Pesquisa em Envelhecimento Humano, GeronLab, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, RJ, Brazil.
2 MD, MPH, PhD, Laboratório de Pesquisa em Envelhecimento Humano, GeronLab, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, RJ, Brazil. Internal Medicine Department, Faculdade de Ciências Médicas, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, RJ, Brazil.

Sergio Telles Ribeiro Filho- Rua Visconde de Pirajá, 414 / sala 518 - 2410-002 Rio de Janeiro RJ - Brazil. E-mail: sergiotell@globo.com

Disclosure: The authors report no conflicts of interest.

Received February 16, 2009. Accepted in final form May 21, 2009.

 

Home Contact