Program of neuropsychological stimulation of cognition in students: Emphasis on executive functions - development and evidence of content validity
Programa de estimulação neuropsicológica da cognição em escolares: Ênfase nas funções executivas - desenvolvimento e evidências de validade de conteúdo
Neuropsychology, a segment of neuroscience, is applicable in different professional settings, such as clinical and educational. Many studies and systematic reviews seeking to structure a body of knowledge in the field of neuropsychological rehabilitation and provide a higher level of evidence have been conducted.1-3,5
In this context, actions and interventions primarily target patients with some kind of brain damage,6
with the main goal of remedying cognitive losses caused by this damage. These interventions constitute a tertiary level of prevention.
Yet, despite the primary interest in rehabilitating cognitive deficits, another type of intervention aimed at preventing or improving levels of health and well-being has been gaining prominence among researchers, clinicians and educators. Such interventions, through preventive and promotional actions, are destined to strengthen and improve cognitive and emotional processes among children with typical development.7-9
These interventions represent the incorporation of neuropsychology into the educational context.
Executive functions (EF) are among the cognitive functions that have been targeted by early and preventive intervention programs. They are a group of abilities that manage and regulate cognitive, emotional, and behavioral functions.10-12
These abilities include three main components: inhibitory control, cognitive flexibility, and working memory.11
A large number of studies have associated these abilities with better school performance and with social and emotional competencies in children.13-17
The same studies suggest that executive deficits can increase the risk of developing learning and behavioral issues, as well as psychopathologies.18,19
The relevance of executive functions justifies the emphasis that researchers have been giving to interventions that improve these abilities during childhood, at home and at school.
Current early and preventive interventions use numerous methods, including computerized cognitive training, non-computerized games, physical activities, mindfulness, school curricula, and extracurricular programs.20
Some programs have already been tested, such as the computerized program Cogmed,21,22
the extracurricular school program Sarilhos do Amarelo,23
and the curricula Tools of Mind.24-26
Despite the growing international emphasis on this type of intervention, national studies are still incipient. In Brazil, the Intervention Program for Self-Regulation and Executive Functions (Programa de Intervenção em Autorregulação e Funções Executivas - PIAFEx8
) is noteworthy. It was developed, and has proven efficacy, to stimulate children at pre-school age27
and at first grade28
within the school setting.
Although studies provide information about the effectiveness of such programs, scant studies have systematically described the procedures used to develop intervention programs. Moreover, few studies report evidence on content validity. One possible source is to consult and follow the procedures and the theoretical, technical, and scientific rigor used in the development of standardized neuropsychological evaluation instruments.29-32
Fonseca et al.,33
for example, proposed a flowchart with each of the stages of the delicate adaptation process of neuropsychological instruments with verbal stimuli for use in Brazil. According to the authors, there are four essential stages:  translation;  analysis by non-expert judges;  analysis by expert judges; and  pilot study. Moreover, the authors of the instrument must analyze suggestions and adapt it based on suggestions given at each stage.
In this scenario of instrument development, it is essential to be cautious when dealing with content validity. Even though this investigation is closely tied to evaluation, it must also be applied to the development of intervention instruments. In this sense, if, in the context of evaluation, an appropriate battery must include items that explore a wide and representative range of the domain to be evaluated,34-36
then an intervention program must also be clear about the demands of each activity. Some strategies to verify evidence of content validity are the use of a consistent theoretical model to substantiate the development of activities, as well as subsequent analysis performed by expert judges.
Based on the previously mentioned stages for devising and adapting neuropsychological instruments, the objective of this study was to present the process of construction and evidence of content validity of the Program of Neuropsychological Stimulation of Cognition in Students: emphasis on Executive Functions, or PENcE (the acronym is from the original name in Portuguese, Programa de Estimulação Neuropsicológica da Cognição em Escolares: ênfase nas Funções Executivas
This program has an early and preventive characteristic, and is based on assumptions from neuropsychology and educational practices. The PENcE was planned to complement school curricula and targets school-aged children in 3rd
grades of Elementary School. Through many cognitive and playful activities, as well as the teaching of strategy in a systematic and explicit way, the program stimulates and leverages EFs and correlated processes for subsequent use in other contexts.
The process of creating the PENcE took place in four stages:  Internal stage of program organization;  Program construction;  Analysis by expert judges; and  Integration of judges' analysis and program finalization, as shown in the flowchart in Figure 1. The stage of analysis by expert judges was conducted with 15 professionals: 1 educational psychologist, 2 educators, 3 speech-language pathologists, and 9 psychologists with experience in neuropsychology. Each module of the program was evaluated by three expert judges. Three other judges assessed the program as a whole and analyzed which main executive component(s) is(are) involved in each activity. Table 1 shows area and level of education of each judge in each module.
Procedures and instruments.
Flowchart depicting each stage of the PENcE construction process.
The present study was approved by the Research Ethics Committee of the Pontifical Catholic University of Rio Grande do Sul. In each stage, authors held brainstorming sessions and made the changes necessary to the study, creating new versions. Below, in Figure 1, each stage of the PENcE is presented and explained.
Internal stage of program organization: this stage was organized in three sub-stages:  Literature review about EF intervention and stimulation programs among children;  Analysis of didactic content during the first years of Elementary School. For the period of two months, one of the authors observed a 3rd
grade class with the goal of understanding classroom dynamics, materials used, content, and curricular matrices from the school;  Meetings with two educators (teachers) with the objective of analyzing the possibility of including the program in the school curriculum, as well as obtaining more information about the school environment. Before creating the program, meetings were held with the team to think about the program's construction and to decide on next steps.
Stage 2. Program construction - The group established the frequency and length of meetings, as well as by whom the program would be mediated. General principles and guidelines to govern the program were created, and activities developed, adapted, and selected. The authors used the bibliographic review performed in Stage 138
, analysis of other programs823,25,39,
, analysis of games known to the public (for example, Finders Keepers, Uno, and Mastermind), and neuropsychology paradigms (go no-go, cancellation tests). Other tasks were created specifically for the intervention. This stage also had a team to operationalize tasks comprising an educational psychologist and two psychology students. At the end, the authors conducted a new brainstorming session and activities were subsequently organized and systematized so that each would have instructions and a description.
Analysis by expert judges - This procedure was carried out in three sub-stages:  Evaluation of the program as a whole: within and between sections, and organization of modules;  Evaluation of each activity of the program;  Evaluation of which function(s) or cognitive/neuropsychological component(s) is (are) stimulated in each activity. Also, using open-ended questions, judges were able to offer suggestions about changes or regarding new stimuli. For each module of the program, 3 expert judges gave their opinions and suggested adjustments. Judges answered an evaluation protocol (Figure 2 - example of the evaluation protocol used) in which each item of the activities was assessed using a Likert scale ranging from 1 to 4 points (1 - strongly disagree, item is not representative; 2 - disagree, item needs substantial reviews; 3 - agree, but item needs some reviews; and 4 - strongly agree, item is representative).
Example of evaluation form used in judges' analytical process.
Integration of judges' analysis and program finalization - After the analysis of the judges, each activity was thoroughly evaluated and discussed among the team. There were some reformulations, as well as a detailed review. The latest version of the PENcE was then generated.
A Content Validity Index (CVI) was calculated to determine the level of agreement between judges. According to Alexandre and Coluci40
, CVI allows measuring of the percentage of experts who agree with respect to a certain item, when Likert scales are involved. The index is calculated by adding the items rated 3 or 4 by the experts, and dividing the sum by the total number of responses in the item's evaluation. For this to be representative in groups of five or less subjects, everyone must agree. The following formula was used:
Results will be presented for each stage of the PENcE.
Results of Stage 1: Internal stage of program organization.
A systematic review38
was performed through bibliographical review and analysis of interventions already present in literature. In this review, 19 studies showed that EFs can be stimulated and developed in children with typical development. Furthermore, there was a predominance of studies using computerized programs, which involved, for the most part, stimulating working memory. Other studies used tasks involving pencil and paper, and some programs were incorporated into the school curriculum with the objective of improving self-regulation. Depending on the program format, effects and gains vary. Methods, modes, and techniques used in these studies were an inspiration to create the PENcE. Curricular standards from the Department of Education, curricular matrix analyses, and observations of the school environment during 2 months, allowed a better understanding of this setting and of the contents and materials used in Elementary School.
Results of Stage 2: Program Construction.
The program was organized in 4 modules, considering the main executive abilities: Module 1 - Organization and Planning; Module 2 - Inhibitory Control; Module 3 - Working Memory; and Module 4 - Cognitive Flexibility. Each module had the three following stages:
1 - Strategy acquisition:
Psychoeducation and Modeling - students are taught about what, where, how, and why use the strategies in each component of the EFs.
2 - Learning and Strategy Consolidation:
Students are stimulated to actively practice the strategies that were taught through playful and cognitive activities, as well as school-related tasks. For each module of the PENcE, this stage initially included from 6 to 8 activities, all developed based on information acquired in Stage 1.
3 - Reflection and Transfer to School and Everyday Activities:
Students reflect about how learning and strategies can be applied to various aspects of life and to school activities.
The program was outlined to be developed within the school environment, 3 times a week in 50-minute sessions, for 5 months. These sessions were mediated by the teacher, and assisted by a neuropsychologist. A story was created based on the movie "A Bug's Life." In this plot, each module was represented by an "Ant" that, facing its challenges, was helped by the "Active Mind Ants League." The "Ants," along with the "Active Mind Ants League," present the strategies to the students, encouraging them to learn and engage in the activities. For each module, activities were organized and included descriptions of tasks, instructions, and materials used. At the end of each module, a section called "Reviewing the Modules" was added, so that the abilities stimulated in the previous modules could be reviewed and integrated. At the end of this process, the first version of the program was reviewed by the authors and submitted to the evaluation by expert judges.
Results of Stage 3: Analysis by Judges.
Tables numbered from 2 to 9 show the evaluation of judges after the first version of the program. The tables were organized by modules, with their respective activities.
The overall evaluation of the program (Table 2), as well as the evaluation of each module (Table 3), showed a level of agreement (CVI) of 1. This suggests that, from the judges' perspective, each module stimulates what it intends to. Also, sequence and module organization are both adequate, as were activity length and order. With respect to specific modules (Tables 5 to 9 - available on the site), there was also agreement among judges on the vast majority of activities, showing that: there is coherence between the activity and its goal; activity descriptions are clear; and activities are adequate for the target age group and can be implemented in the school setting.
Table 4 shows which executive component(s) is (are) predominantly stimulated in each of the PENcE activities. In most activities, there was 100% agreement among judges. In the activities aimed at reviewing each module, the intent was to stimulate more than one component at the same time. Consequently, there was more than one percentage number. Activities that did not obtain a CVI equal to one on any of the evaluation criteria, or that did not have 100% agreement among judges with respect to the executive component that was stimulated, were reviewed and adapted. The revisions that were made are described in the section "Results of Stage 4: Integration of Judges' Analysis and Program Finalization."
Some activities have more than one level of complexity. For these activities, judges also evaluated whether the levels were equivalent, or level 2 was harder than level 1, or vice-versa. For all activities involving more than one level, the judges agreed that the level initially developed as the most complex one was, in fact, harder than the previous level. The judges suggested a few changes for some of the activities with the intent of making them clearer and easier to understand. Most of those changes were incorporated into the program. Some judges asked for more details or to add more information in the instructions, others proposed substituting the wording of some terms. After these considerations, the evaluation of judges was integrated into the latest version of the PENcE.
Results of Stage 4: Integration of Judges' Analysis and Program Finalization.
After the judges' analysis, adjustment and changes were incorporated into each module:
- Activity of Fitting Numbers: one of the judges understood that the activity stimulated, predominantly, inhibitory control. However, this judge also considered that the activity demanded planning and organization. To make the activity clearer, the following procedures were carried out: addition of instructions in writing, and directions for teacher to count the number of mistakes and pages that each student needed to complete the task. This helped check which students were able to plan the activity well before executing it.
Activity 5: Cooking - one of the judges understood that this activity was not suitable for the classroom setting due to time constraints. Therefore, the activity was broken down to be performed over two days. 1st
day: planning what is going to be cooked; 2nd
day: execution of activity and evaluation.
- Activity 1: Dancing - some examples were added, upon request by one of the judges.
Activity 6: Not everything is what it seems - because of the understanding that the activity was not coherent and its description not clear, it was excluded from the program.
- Activity 2: Numbering the sequences - rather than numbering a sequence using a material similar to a mat, it was decided that the stimuli would be projected onto a screen.
Activity 5: Differences Game - besides finding the differences between the images, students had to, at the same time, count from 50 to 20. Only after this countdown could they write down on a piece of paper the differences that they had observed.
- Although the CVI was appropriate for all criteria, in activity 4, "Matching Cards," one rule was added: students had to combine the cards with images of animals. In addition to the colors of the animals and the habitat where they live, the activity included the number of syllables of each word, with the objective of making the task more complex and challenging.
When there were disagreements with respect to changes and adaptations, they were submitted to one of the authors, who established a consensus. There was then a review of the whole program, finalizing it. The summarized final version is given in the appendix of this study. In the PENcE book40
, the whole program and its activities are described in detail so they can be replicated. After reading this material, other teachers can conduct the activities.
This study presented the development process and content validity evidence of the PENcE for school-aged children. The PENcE is a program for stimulating EFs that, through various cognitive and school-related activities, as well as through the teaching of strategies, aims to systematically leverage these abilities in the school setting. Creating a rehabilitation or a neuropsychological stimulation program requires rigorous and well defined stages and procedures. However, there is still no systematization of the process in the available literature. Existing programs do not specify how they were devised or constructed and do not report content validity evidence. In this scenario, the present study sought to follow the same methodological and technical rigor used to adapt or create neuropsychological evaluation instruments29,30,31,32,33
proposed a model with the steps necessary to develop a psychological instrument, which must follow a sequence. These steps comprise three main axes: theoretical procedures (involving construct definition, producing each item of the instrument, and content validity); empirical procedures (planning and application of the instrument, data collection); and analytical procedures (statistical analyses verifying that the instrument is valid, reliable, and regulated). In this study, the first axis was attained, providing a direction for the ensuing stages. This was the first stage of the validity work and of the search for effectiveness of the program. All stages contributed to identify positive aspects, possible flaws, verify understandability and level of complexity, adapt stimuli, and improve tasks of the program.
The first stage included a long process of research, reflections on the theme, and discussions among the team. This required the exploration of national and international bibliographies. The school environment and curricular matrices of Elementary School were closely monitored. This was a fundamental stage because it allowed the definition of aspects and theoretical models that would become the basis of the PENcE, as well as the constructs that would be stimulated. Based on the model created by Diamond11
, it was decided to include the central abilities (inhibitory control, working memory, and cognitive flexibility), which form the basis of the emergence of more complex or superior functions, and one complex executive function (planning and organization) because of its relevance to school performance. These definitions, along with a deeper understanding of other available studies, allowed the following steps to be developed underpinned by a solid theoretical basis.
In the construction stage, format, principles, and program guidelines were planned. Tasks that would compose each module, with their respective instructions and descriptions, were also planned. As is the case for the development of any rehabilitation program, some decisions had to be made, considering the objective of the program. Initially, it was decided that the program would be incorporated into the school curriculum. Programs of curricular adaptations adopt a wider and more naturalistic41
approach, and there is evidence that such programs tend to present better results in terms of acquiring abilities and transferring gains.7,20,24,26,28,42,43
It was also decided that the program would be based on the teaching of systematic and explicit strategies of the EFs. Meltzer39
holds that classroom interventions must include direct instructions of metacognitive strategies, and that they must be structured and systematic. Thus, with the goal of allowing more reflection and knowledge transfer to other contexts, numerous playful, cognitive, and school-related activities were proposed.
In the third stage, the judges' analysis revealed a high level of agreement among experts, both for the program as a whole and each activity. In this stage, the relevance of the tasks within the program was verified, allowing tasks to be adjusted and improved. The Expert Committee has the role of consolidating the presented version and helping produce the pre-final version of the instrument31,44
. Also, agreement among experts provides confirmation of content validity 34,36
. In the absence of other studies investigating evidence of content validity in cognitive stimulation interventions among healthy children, the index found here is in consistent with those of other studies determining this type of validity for instruments for neuropsychological evaluation43
. The existence of evidence of validity with respect to content makes it easier for its applicability to be tested and checked in typical samples and different clinical groups. These procedures allowed the derivation of content validity evidence for the PENcE. After checking all suggestions put forward by the judges, in the fourth stage of the study, changes and adjustments were made, resulting in the final version of the intervention program.
Given the lack of research about the development process of rehabilitation or neuropsychological stimulation programs, this study displays an innovative approach. It can help other researchers and clinicians to develop strategies and intervention programs, since it provides a systematization and outline of the process. This study also provides readers with a detailed understanding about each stage and offers insight on all the care and rigor applied in producing the PENcE. The PENcE can be a useful tool to education and healthcare professionals, since it can provide orientation for their practices and promote benefits for school-aged children in the promotion of EFs. Future studies should verify the effectiveness of the program for children in Elementary School I, and adapt the program for use in teenagers and clinical groups with potential executive dysfunctions, such as attention deficit hyperactivity disorder (ADHD), and learning related disorders.
Caroline de Oliveira Cardoso: literature search, study design; preparation and writing of the Dement Neuropsychol "Program of neuropsychological stimulation of cognition in students: emphasis on executive functions - development and evidence of content validity". Cardoso CO, et al. manuscript, statistical analysis. Natália Martins Dias: literature search; preparation and writing of the manuscript; scientific contribution; Alessandra Seabra: scientific contribution and critical content review; approval of the final version to be sent to the journal. Rochelle Peace Fonseca: study design; scientific contribution and critical content review; approval of the final version to be sent to the journal.
We thank CAPES for the doctoral fellowship awarded to Caroline Cardoso.
1. Cicerone KD, Dahlberg C, Kalmar K, Langenbahn DM, Malec JF, Bergquist TF, et al. Evidence-based cognitive rahabilitation: recommendations for clinical practice. Arch Phys Med Rehabil. 2000; 82(12):1596-615.
2. Gillespie DC, Bowen A, Chung CS, Cockburn J, Knapp P, & Pollock A. Rehabilitation for post-stroke cognitive impairment: an overview of recommendations arising from systematic reviews of current evidence. Clin Rehabil. 2015;29(2):120-8.
3. Reijnders J, van Heugten C, van Boxtel M. Cognitive interventions in healthy older adults and people with mild cognitive impairment: a systematic review. Ageing Res Rev. 2015;12(1):263-75.
4. Teasell RW, Foley NC, Bhogal SK, ChakraverttyR, Bluvol A. A rehabilitation program for patients recovering from severe stroke. Can J Neurol Sci. 2005;32(4):512-7.
5. Zehnder F, Martin M, Altgassen M, Clare L. Memory training effects in old age as markers of plasticity: a meta-analysis. Restor Neurol Neurosci. 2005;27(5):507-20.
6. Rajeswaran J. Neuropsychological rehabilitation: principles and applications. 1st ed. 528 London; Waltham, MA: Elsevier; 2013
7. Bierman KL, Nix RL, Greenberg MT, Blair C, Domitrovich CE. Executive functions and school readiness intervention: Impact, moderation, and mediation in the Head Start REDI program. Dev Psychopathol. 2008;20(3):821-43.
8. Dias NM, Seabra AG. Programa de Intervenção sobre a Autorregulação e Funções Executivas - PIAFEx. São Paulo: Memnon; 2013.
9. Jaeggi SM, Buschkuehl M, Jonides J, Shah P. Short- and long-term benefits of cognitive training. PNAS. 2011;108:10081-6.
10. Anderson P. Assessment and development of executive function during childhood. Child Neuropsychol. 2002;8(2):71-82.
11. Diamond A. Executive functions. Annu Rev Psychol. 2013; 64:135-168.
12. Hughes C, Ensor R. Executive function and theory of mind: Predictive relations from ages 2 to 4. Dev Psychol. 2007;43(6):1447-59.
13. Blair C, Diamond A. Biological processes in prevention and intervention: The promotion of self-regulation as a means of preventing school failure. Dev Psychopathol. 2008;20:899-91.
14. Bull R, Scerif G. Executive Functioning as a Predictor of Children's Mathematics Ability: Inhibition, Switching, and Working Memory. Dev Neuropsychol. 2001;19(3):273-93.
15. Carlson SM, Moses LJ, Claxton LJ. Individual differences in executive and theory of mind: An investigation of inhibitory control and planning ability. J Exp Child Psychol. 2004; 84(4):299-319.
16. Diamond A. Normal development of prefrontal cortex from birth to young adulthood: Cognitive functions, anatomy, and biochemistry. In Stuss D, Knight R (Eds.). Principles of frontal lobe function. New York: Oxford University Press; 2002:466-503.
17. Zorza JP, Mariano J, Acosta Meses A. Executive Functions as Predictors of School Performance and Social Relationships: Primary and Secondary School Students. Span J Psychol. 2016;19-23.
18. Blair C. As funções executivas na sala de aula. In: Tremblay RE, Boivin M, Peters RV(Eds). Enciclopédia sobre o Desenvolvimentona Primeira Infância [on-line]. Montreal: Centre of Excellence for Early Childhood Development e Strategic Knowledge Cluster on Early Child Development; 2013:1-8.
19. Snyder HR, Miyake A, Hankin BL. Advancing understanding of executive function impairments and psychopathology: bridging the gap between clinical and cognitive approaches. Front Psychol. 2015;6:1-24.
20. Diamond A, Lee K. Interventions shown to aid Executive Function development in children 4 to 12 years old. Science. 2011;333(6045):959-64.
21. Klingberg T, Fernell E, Olesen PJ, Johnson M, Gustafsson P, Dahlstrom, K, et al. Computerized training of working memory in children with ADHD - A randomized, controlled trial. J Am Acad Child Adolesc Psychiatry. 2005;44(2):177-86.
22. Thorell LB, Lindqvist S, Bergman NS, Bohlin G, Klingberg T. Training and transfer effects of executive functions in preschool children. Dev Sci. 2009;12(1):106-13.
23. Rosário P, Costa JC, Mourão R, Chaleta E, Grácio ML, Núñez JC, González-Pienda. De pequenino é que se auto-regula o destino. Educação: temas e problemas 2007;4:281-93.
24. Barnett WS, Jung K, Yarosz DJ, Thomas J, Hornbeck A, Stechuk R, Burns S. Educational effects of the tool of the mind curriculum: A randomized trial. Early Child Res Q. 2008;23(3):299-313.
25. Bodrova E, Leong DJ. Tools of the mind: the Vygotskian approach to early childhood education (2nd
ed). Columbus, OH: Merrill/Prentice Hall; 2007.
26. Diamond A, Barnett WS, Thomas J, Munro S. Preschool program improves cognitive control. Science. 2007;318(5855):1387-8.
27. Dias, NM, Seabra, AG. Is it possible to promote executive functions in preschoolers? A case study in Brazil. International Journal of Child Care and Education Policy 2015a;9:6.
28. Dias NM, Seabra AG. The Promotion of Executive Functioning in a Brazilian Public School: A Pilot Study with 1st Graders of Elementary. Span J Psychol. 2015b;5:18:E8.
29. Fonseca RP, Salles JF, Parente MAMP. Development and content validity of the Brazilian Brief Neuropsychological Assessment Battery NEUPSILIN. Psychol Neurosci. 2008;1:55-62.
30. Cardoso CO, Zimmermann N, Paraná CB, Gindri G, de Pereira AA, Fonseca RP. Brazilian adaptation of the Hotel Task A tool for the ecological assessment of executive functions. Dement Neuropsychol. 2015;9(2):156-64.
31. Salles JF, Fonseca RP, Parente MAMP, Miranda MC, Rodrigues CC, Mello CB, Barbosa T. Instrumento de Avaliação Neuropsicológica Breve Infantil NEUPSILIN-INF. São Paulo: Vetor; 2011.
32. Zimmermann N, de Pereira AP, Fonseca RP. Brazilian Portuguese version of the Patient Competency Rating Scale (PCRS-R-BR): semantic adaptation and validity. Trends Psychiatry Psychother. 2014;36(1):40-51.
33. Fonseca RP, Casarin FS, Oliveira CR, Gindri G, Soares, ECSS, Ortiz KZ, et al. Adaptação de instrumentos neuropsicológicos verbais: um fluxograma de procedimentos para além da tradução. Interação Psicol. 2011;15:59-69.
34. Pasquali L. Psicometria. Teoria dos testes na psicologia e na educação (2ª ed), Petrópolis, Rio de janeiro: Editora Vozes; 2003.
35. Primi R, Muniz M, Nunes CHSS. Definições contemporâneas de validade de testes psicológicos. In: Hutz CS (Org.), Avanços e polêmicas em avaliação psicológica. São Paulo: Casa do Psicólogo; 2009:243-266.
36. Urbina S. Fundamentos da testagem psicológica. Porto Alegre: Artmed; 2007
37. Cardoso CO, Fonseca RP. Programa de Estimulação Neuropsicológica da Cognição em Escolares: ênfase nas Funções Executivas. Ribeirão Preto: BookToy;2016.
38. Cardoso CO, Dias NM, Senger J, Colling APC, Seabra AG, Fonseca RP. Neuropsychological stimulation of executive functions in children with typical development: a systematic review. Appl Neuropsychol-Child. 2016.
39. Meltzer L. Promoting executive functions in the classroom. New York: The Guilford Press; 2010.
40. Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciênc Saúde Coletiva 2011;16(7):3061-8.
41. Hughes C. Changes and Challenges in 20 Years of Research into the Development of Executive Functions. Inf Child Dev. 2011;20:251-71.
42. Diamond A, Ling DS. Conclusions about Interventions, Programs, and Approaches for Improving Executive Functions that appear Justified and those that, despite much hype, do not. Dev Cogn Neurosci. 2015;18:34-48.
43. Dias NM, Seabra AG. Intervention for executive functions development in early elementary school children: Effects on learning and behavior and follow-up maintenance. Educational Psychology 2016;1-19.
44. Byrd D, Arentoft A, Scheiner D, Westerveld M, Baron IS. State of multicultural neuropsychological assessment in children: current research issues. Neuropsychol Rev. 2008;18:214-22.
1. Department of Psychology, Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre, and Universidade Feevale, Novo Hamburgo, RS - Brazil
2. Department of Psychology, Centro Universitário Fundação Instituto de Ensino para Osasco, São Paulo SP - Brazil
3. Department of Psychology, Universidade Prespiteriana Mackenzie, São Paulo SP - Brazil
4. Department of Psychology, Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre RS - Brazil
Caroline de Oliveira Cardoso
Graduate Department of Psychology / Pontifícia Universidade Católica do Rio Grande do Sul
Av. Ipiranga, 6690 / Building 11 / Rm 932
90610-900 Porto Alegre RS - Brazil
Received October 07, 2016.
Accepted in final form December 06, 2016.
Disclosure: The authors report no conflicts of interest.
This study was conducted at the Department of Psychology, Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre RS - Brazil.
Download Tables Site (Cardoso)