Psychology and Education
Received: 19 August 2021
Accepted: 17 April 2023
DOI: https://doi.org/10.5935/1980-6906/ePTPPE14759.en
Abstract: This study aimed to assess the effect of a Career Education intervention intended to promote social-emotional skills. A total of 62 students attending the 9th grade of a public school participated in this study. Data were obtained using the Inventory for the Assessment of Social and Emotional Skills (SENNA 2.0). The intervention effects were analyzed by comparing Intervention Group A (GA), Control Group (CG), and Intervention Group B (GB) and within groups. The results show statistically significant differences only between the groups and in two dimensions: Agreeableness in favor of Intervention Group A (GA) and Openness in favor of Intervention Group B (GB). These findings show the benefits an intervention program intended to promote the development of social-emotional skills can promote in Career Education. However, the small number of participants stands out in terms of limitations. Hence, studies with larger samples are needed to replicate the results. In addition, this study reveals methodological aspects to be considered in the design of Career Education programs, such as more sessions to reach more individuals and obtain more effective results over time.
Keywords: Socioemotional skills, socioemotional development, socioemotional learning, career education, intervention.
Resumo: Este estudo objetivou avaliar o efeito de uma estratégia de intervenção em Educação para Carreira visando o desenvolvimento das habilidades socioemocionais. Participaram 62 estudantes do 9º ano do Ensino Fundamental II de uma escola pública. Os dados foram obtidos por meio do Instrumento para Avaliação de Habilidades Socioemocionais (Senna 2.0). Os efeitos da intervenção foram analisados nas comparações entre o grupo de intervenção A (GA) e o grupo controle (GC), de Intervenção B (GB) e intragrupos. Os resultados mostram diferenças estatisticamente significativas apenas entre grupos e em duas dimensões: amabilidade, a favor do grupo de intervenção A (GA), e abertura ao novo no grupo de intervenção B (GB). Tais achados mostram benefícios que o programa de intervenção para o desenvolvimento das habilidades socioemocionais pode propiciar à Educação para a Carreira. Como limitação, destaca-se o pequeno número de participantes, tornando-se necessários estudos com amostras maiores, visando a replicação dos resultados. O estudo aponta aspectos metodológicos a serem considerados no delineamento de programas de Educação para a Carreira, com mais sessões, visando maior alcance e resultados mais efetivos ao longo do tempo.
Palavras-chave: Habilidades socioemocionais, desenvolvimento socioemocional, aprendizagem socioemocional, educação para a carreira, intervenção.
Resumen: Este estudio tuvo como objetivo evaluar el efecto de una estrategia de intervención en Educación para la Carrera dirigida al desarrollo de habilidades socioemocionales. Participaron 62 estudiantes del grado noveno de la Enseñanza Básica II de un colegio público. Los datos fueron obtenidos mediante el Instrumento de Evaluación de Habilidades Socioemocionales (SENNA 2.0). Los efectos de la intervención se analizaron en comparaciones entre el Grupo de Intervención (GA) y el Grupo de Control (GC), la Intervención B (GB) e intragrupos. Los resultados muestran diferencias estadísticamente significativas solo entre grupos y en dos dimensiones: Amabilidad a favor del Grupo de Intervención A (GA) y Apertura a lo nuevo en el Grupo de Intervención B (GB). Tales hallazgos muestran los beneficios que el programa de intervención para el desarrollo de habilidades socioemocionales puede brindar a la Educación para la Carrera. Como limitación, se destaca el pequeño número de participantes, lo que obligó a realizar estudios con una muestra mayor, con el objetivo de replicar los resultados. El estudio apunta aspectos metodológicos a ser considerados en el diseño de programas de Educación para la Carrera, con más sesiones, buscando mayor alcance y resultados más efectivos en el tiempo.
Palabras clave: Habilidades socioemocionales, desarrollo socioemocional, aprendizaje socioemocional, educación para la carrera, intervención.
Rapid transformations in life and work in this 21st century demand individuals to develop skills to deal with the challenges of a volatile, uncertain, complex, and ambiguous (VUCA) world. Such challenges were already taking shape at the end of the last century and raised concerns in the education, training, and professional guidance fields (Saleh & Watson, 2017). After the pandemic, a brittle, anxious, nonlinear, and incomprehensible (BANI) world emerged. In this context of metamorphosis, the earlier young individuals are qualified to deal with the numerous challenges at school and in life, the more capable they will be in promoting well-being and quality of life and performing as future professionals (De Fruyt et al., 2015). Therefore, this study aims to assess the effect of a Career Education intervention on the development of social-emotional skills in a group of 9th graders. Hence, to facilitate understanding, three thematic axes organize this section: (a) Career education, (b) Social-emotional skills, and (c) intervention programs intended to promote social-emotional skills.
The first axis, Career Education, emerged as a strategy to favor career development in the educational context. Since the 1970s, teaching institutions at all levels of education in different countries, but especially in the United States and Europe, have implemented programs with this objective. Career Education, an intervention modality in the field of Professional and Career Guidance, is considered a collaboration of Educational Psychology “to enable students to relate education and work and acquire general competencies for a positive career development, so to allow each individual to make paid or unpaid work a significant part of their lifestyle” (Hoyt, 2005, p. 24). Watts (2001) can provide a deeper understanding of Career Education issues in the international context. In Brazil, Munhoz and Melo-Silva, starting in 2010, addressed Career Education and put this topic on the Brazilian agenda; Munhoz and Melo-Silva (2011) stand out.
Sensitizing students about career competencies and skills is important within the scope of interventions in Career Education because these are essential for the students’ present and future lives. According to the National Common Curricular Base (BNCC) (Ministério da Educação, 2018, p. 8), competencies are defined as “the mobilization of knowledge (concepts and procedures), skills (practical, cognitive and social-emotional), attitudes, and values to solve everyday life complex demands, and fully exercise citizenship and being active in the world of work”.
The second axis in this introduction focuses on the concept of social-emotional skills. The term social-emotional is a construct of contemporary studies, with several conceptualizations linked to competencies and skills. Competence is a trait or personal characteristic related to superior performance in a given task or situation. Social-emotional competencies are considered an individual’s ability to make conscious decisions, regulate emotions, solve problems, and deal with challenges and unforeseen circumstances. Hence, competencies are related to healthy development throughout life and quality social relationships (Boyatzis, 2019). Many studies addressing the 21st century competencies adopt the noncognitive, social-affective, personal, and social-emotional learning terms, among others (Santos & Primi, 2014). Competencies formed by different skills and levels of personal resources, including intellectual ones. Skills result from acquired competencies and are associated with ‘knowing how to do.’ It is always possible to improve and connect them with others (Ministério da Educação, 2000). Social-emotional skills are expressed through thoughts, feelings, and behaviors that are malleable and can be developed throughout life through formal and informal learning (De Fruyt et al., 2015). Therefore, the term social-emotional skills1 is adopted here, as it is part of the denomination of the Instrument for the Assessment of Social and Emotional Skills (SENNA 2.0), adopted in this study. This instrument is organized into dimensions (also called macro-competences) and facets (also called competencies). Thus, the terms competencies and skills are used in the sense of human potential.
SENNA 2.0 is based on the five dimensions of the Big Five personality model to be used in comprehensive education (Primi et al., 2016). Personality traits develop through the interaction of personal and environmental factors (i.e., learning) and show considerable plasticity, especially during childhood and adolescence, when the first expectations of social role performance and professional aspirations arise (Primi et al., 2016). A key definition for this study is presented below.
Social-emotional skills can be defined as individual characteristics that: (a) originate in the reciprocal interaction between biological predispositions and environmental factors; (b) manifest themselves in consistent patterns of thoughts, feelings, and behaviors; (c) continue to develop through formal and informal learning experiences; and (d) influence important socioeconomic outcomes throughout an individual’s life (De Fruyt et al., 2015, p. 279).
Social-emotional skills are identified in the literature as essential for personal development, academic success, and adapting to work from the perspective of Primi et al. (2016). Moreover, social-emotional skills predict future performance. For this reason, as highlighted by Leal et al. (2020), they are relevant in a Career Education program as they help individuals cope with adult life’s challenges in the education and work spheres. Additionally, social-emotional skills encompass potentially malleable characteristics, the development of which is shaped by environmental factors, that is, formal and informal learning experiences, which are relevant in intervention programs.
Regarding this introduction’s third axis, intervention programs in learning and developing social-emotional competencies or skills, Barbosa and Melo-Silva (2023) conducted a systematic literature review covering ten years of analysis (2011-2020). The authors mentioned above concluded that the interventions vary considerably, and most originated in Europe and in the United States. In general, the studies reviewed do not explain the theoretical models supporting the programs. The studies reporting the theoretical framework mentioned various theories, though none predominate. The programs’ objectives include promoting social-emotional skills/ competencies and preventing development of problems due to failure to acquire these skills/ competencies. As for the target audience, most interventions were aimed at children or adolescents, or both. Regarding the structure and procedures, there is no standardization of the methods adopted in the interventions, as noted by Evans et al. (2015) and Freeman et al. (2014), neither in terms of the methods nor the duration of the interventions. On average, the interventions were implemented in 14 and 15 sessions, though they ranged from three (minimum) to 40 sessions (maximum). Each lasted one hour on average. Additive (extracurricular) or curricular strategies were found. Most studies reported on the instruments adopted and the main activities carried out. Regarding the evaluation of the intervention results, studies with a quantitative design, pre- and post-test, predominated. Regarding the interventions’ results, among the 28 programs reviewed by Barbosa & Melo-Silva (2023), 24 of them reported positive effects, not only regarding social-emotional development but also regarding other variables investigated, such as self-esteem, school readiness, decreased anxiety and depression symptoms, fewer problem behaviors, and decreased aggressiveness and career exploration.
According to Oliveira and Muzkat (2021) and as seen in longitudinal studies, social-emotional skills positively impact one’s health, learning, and affective and professional relationships, decreasing antisocial behaviors and increasing prosocial behaviors. However, the previous authors note that few studies evaluate the effectiveness of programs or intervention strategies aimed at the development of social-emotional skills. Thus, this study’s relevance in the context of middle schooling makes sense, especially addressing 9th graders in Brazil with the implementation of the BNCC (Ministério da Educação, 2018), which aims at comprehensive education, the development of various life competencies and skills, establishing ten key competencies for student development: (a) knowledge, (b) scientific, critical, and creative thinking, (c) cultural repertoire, (d) communication, (e) digital culture, (f) work and life project, (g) argumentation, (h) self-knowledge and self-care, (i) empathy and cooperation, and (j) responsibility and citizenship.
Some Brazilian and international studies addressing intervention programs that dialogue with this study’s objectives, i.e., to promote the development of social-emotional skills, were selected. The objective is to enable adolescents to deal with the complex demands of everyday life, fully exercise citizenship, and be active agents in the world of work in different contexts and with various participants and sessions. Programs focusing on learning social-emotional competencies or skills have reported positive effects on social-emotional development and the relationship with other important variables for an individual’s development, namely: social behaviors, conduct problems, stress, and academic performance (Durlak et al., 2011).
The studies by Chaux et al. (2017) in the Colombian context, and Romero et al. (2019) in the Spanish context show a decrease in problem behaviors and aggressiveness, which are demands of many schools. The study by Chaux et al. (2017) on the Classrooms in Peace program focuses on conflict resolution, bullying, and peer aggression. It addressed 1,154 children, aged between 7 and 10, attending from the 2nd to the 5th grades in 55 public schools. It took 40 sessions and included many participants. Romero et al. (2019) focused on training emotion recognition and regulation, conflict resolution, perspective, and social skills in 128 children between 8 to 10 years old with problem behaviors and comprised 19 sessions.
Garcia et al. (2019) performed a study in the Brazilian context to evaluate the effectiveness of the programa Amigos Divertidos [Fun Friends program] in reducing anxiety and depression symptoms and increasing social-emotional skills. It addressed 25 children between 5 and 7 and their respective caregivers. Pavoski et al. (2018) used the same method with ten children aged 6 and 7 and reported an increase in social-emotional skills and a decrease in social-related problems and anxiety and depression symptoms. Both studies report interesting results, but the results cannot be generalized due to small samples. From the perspective of thinking about the future by continuing to study and being creative at work, the study by Leal et al. (2020), also conducted in the Brazilian context, stands out. The Edu-Car Program was structured in two modules: social-emotional and career skills. The study addressed 116 students attending the 1st year of high school, aged 15 on average. The program comprised 12 sessions. The results show an improvement in career exploration. The Edu-Car program inspired this study’s design, where students attending the 9th grade were addressed to improve their knowledge in the field. Even though the age range is close, the participants in this study are in a transition period from one educational level to another.
The studies previously reported showed the benefits of programs in different countries and age groups, showing the relevance of studies assessing programs intended to promote social-emotional skills. However, the results cannot always be generalized or their effectiveness statistically observed. Methodological problems emerge in intervention assessments and therefore further research is increasingly needed. Thus, we highlight some studies that question the effectiveness of results.
Kiviruusu et al. (2016) addressed the Together at School program in the Finnish context. This program was designed to promote social-emotional skills among elementary school children, and teachers implemented it in all the classes. The investigation aimed to examine the program’s short-term effects on improving social-emotional skills and reducing psychological problems among boys and girls. The results from short-term interventions have shown no effects on children’s social-emotional skills or psychological problems. The previous authors report that such results are due to the relatively short follow-up period and note that, although the intervention was performed within six months, the time spent during classes with the program was short, in addition to the fact that the program was applied and assessed by the educators who developed the intervention, which possibly biased the results.
In turn, Zyga et al. (2018) assessed Kids Love Musicals! (KLM) in the American context to verify the feasibility of a musical theater program addressing students with intellectual disabilities in various school settings. The activities were developed in three forms of arts represented in musical theater: (a) music, (b) dance, and (c) theatrical representation and staging by students attending from the 1st to the 12th grade through eight sessions. The program was assessed through pre- and post-video recordings, coded into six social-emotional skill domains among all participants (n = 47). The results suggest that the program participants showed gains in eye contact, turn-taking, cooperative learning, engagement, social awareness, self-confidence, symbolic flexibility, and emotional understanding. However, although the program promoted social-emotional changes, the analyses indicate that the gains were related to school factors and the participants’ characteristics. The study’s authors note that a potential explanation for the results is the difficulty of standardizing interventions in the field, which consequently may cause problems in the construction of programs, resulting in negative effects in the evaluation of the results.
Another study reporting problems was Sidera Caballero et al. (2019), developed in Spain. The program aimed to improve coexistence and reduce aggressiveness among 64 high school teenagers by developing social-emotional skills. It lasted 11 sessions. The intervention was not effective in reducing aggressiveness or improving the level of empathy or moral disconnection. A worsening in school climate was also found in both the control and intervention groups. Thus, as in the two previous studies, the authors note that the frequency and intensity of the sessions were factors that may have influenced the study’s results, highlighting the relevance of these indicators for the success of the interventions. In summary, the authors note the need to identify the central elements of programs with the same focus to clarify how they should be developed, aiming to obtain effective changes.
As previously noted, interventions vary widely, and the methods and ways they are assessed are not standardized (Evans et al., 2015; Freeman et al., 2014). Additionally, the reviewed studies show that more significant gains are obtained in programs addressing young children. In contrast, the evaluation of programs addressing adolescents shows some limitations, as in the following study. The Collaborative for Academic, Social, and Emotional Learning (CASEL, 2005) indicates that social-emotional learning is a process that involves several strategies developed in the long term, as it is essential to have the initiatives of peers and family members, together with the school and the content developed there.
Considering the relevance of practices in Career Education and investments in social-emotional learning, it is relevant to investigate a Career education intervention implemented in a public school with Middle and High School students in the Brazilian context. Hence, this study hypothesizes that Career Education interventions favor the development of social-emotional skills and career skills. Therefore, this study aims to evaluate the effect of a Career Education intervention focused on developing social-emotional and career skills among 9th graders attending a Brazilian public school. Two groups (intervention and comparison) were analyzed at three points in time.
Method
Study design and participants
This study was based on a quasi-experimental design, with pre-test/post-test measures with a non-equivalent control group, whose characteristics will be presented later. A convenience sample was selected, considering this study was conducted in a school with only two 9th-grade classes. The sample initially consisted of 70 adolescents attending the 9th grade in 2019. Data were collected in a public state school in a city in the countryside of São Paulo. Two groups were organized: Intervention Group A (GA) and a Control Group (CG). Initially, the GA comprised 34 participants; most were 14 years old (94.1%) and male (61.8%). Considering only students with both characteristics, the group contains 31 participants. The CG comprised 38 participants at the beginning of data collection when most were male (57.5%) and were 14 years old (70%). Considering the students presenting both characteristics, the group comprised 31 participants who were part of the final sample of the CG. That is, the final sample considered in the analysis consisted of 62 participants. Intervention Group A (GA) received the intervention in the first semester of 2019, and the control group (CG) received the intervention in the semester of 2019 when its name was changed to Intervention Group B (GB). All groups were pre-tested simultaneously at the beginning of the study, and this testing was called Time 1. Post-test 1 (Time 2) was implemented at the end of the first semester. After Intervention Group B (GB) received the intervention, everyone underwent testing again at Time 3, called post-test 2.
As for the socioeconomic stratum of Intervention Group A (GA), according to the Brazil criterion (Brazilian Association of Research Companies [ABEP], 2016), the sample is mainly distributed among four strata, as follows: B2 (35.5%), A (29%), C1 (19.40%), and B1 (13%), with C2 (3.2%) underrepresented and D-E without representativeness. As for the socioeconomic stratum of the Control Group (CG), according to the Brazil criterion, the sample is distributed among three strata: B2 (45.2%), C1 (19.4%), and B1 (16.1%). Note that strata A (9.7%), C2 (6.5%), and D-E (3.2%) are poorly represented in this group.
Instruments
Inventory for the assessment of social-emotional skills – SENNA 2.0. In its first version, it was called the Social and Emotional (or Noncognitive) Nationwide Assessment (SENNA 1.0) (Primi et al., 2016). Then, it underwent changes, and a new version was developed and named Inventory for the Assessment of Social-Emotional Skills (SENNA 2.0) (Primi et al., 2021), which is the one adopted in this study. It is a measure that outlines the profile of social-emotional skills among children and adolescents aged 11 to 19 years old. It is based on the Big Five model’s dimensions, called macro-competencies in SENNA – in a broad and comprehensive model, which organizes five core dimensions with 17 more specific concepts called facets or competencies (Sette & Alves, 2021). In SENNA 2.0, the macro competencies are: (a) Agreeableness (A) [competencies: empathy, respect, trust]; (b) Self-management (C) [competencies: determination, organization, focus, persistence, responsibility], in the previous version it was called Conscientiousness; (c) Engaging with others (E) [competencies: social initiative, assertiveness, enthusiasm], in the previous version it was Extroversion; (d) Emotional Resilience (N) [competencies: stress tolerance, self-confidence, frustration tolerance], in the previous version it was called Emotional Stability or Neuroticism; and (e) Open-Mindedness (O) [skills: curiosity to learn, creative imagination, artistic interest].
As previously explained, the Big Five dimensions have the acronym of OCEAN, in which each letter represents one of these dimensions. Note that we call social-emotional skills macro-competencies, while the literature in the field calls them dimensions composed of facets. The dimensions explain the broader functioning, whereas the facets describe behavior more specifically (Sette & Alves, 2021).
Each facet or competency comprises items that address identity and self-efficacy issues. There are two versions of SENNA 2.0, one with 162 items (complete) and another one with 54 items. In this study, the 54-item version was adopted since it has the same psychometric values as the complete version but can be applied in situations that require a less time-demanding instrument. Such a factor was considered when we adopted this version, as it was applied three times to the same participants.
Note that the instrument’s internal consistency coefficients are higher than 0.70 in both versions (SENNA v1.0 and SENNA v2.0). Other psychometric properties of this instrument can be found in its technical manual (Primi et al., 2021). However, it should be noted that the congruence coefficients of the dimensions were 0.80 for Emotional Resilience, 0.79 for Agreeableness, 0.78 for Engaging with others, 0.92 for Conscientiousness, and 0.89 for Openness, which are considered satisfactory (Campos et al., 2020) and robust evidence of a psychometric instrument’s internal validity. Another validity evidence, this time of the test-criteria type, showed that all SENNA dimensions had a predictive value for performance in the scores of the Portuguese (R2=0.18) and mathematics (R2=0.12) subjects.
Socioeconomic classification in Critério Brasil (ABEP, 2016). This instrument is based on the IBGE’s Household Budget Survey (POF) and seeks to define large strata that meet the segmentation needs (by purchasing power) of consumers of Brazilian companies. The classification questionnaire comprises comfort items available in households, for example, the number of passenger cars exclusively for private use, the presence of piped water and paved streets, and the level of education of the family’s head. Based on the answers to the items, individuals are scored and classified. It was established that strata A corresponds to the range of 45-100 points, B1 to 38-44, B2 to 29-37, C1 to 23-28, C2 to 17-22, and D-E to 0-16 points. Changes were implemented to the classification methodology in June 2019, although the original structure proposed in 2015 is maintained. The 2015 Brazil Criteria was used in this study because the most recent version had not yet been released when this study project was designed, and data were collected.
Intervention program: procedures and structure
This intervention study is characterized as an action-research since it involves the researcher’s active participation in the problem under study to monitor and implement the intervention and research procedures step-by-step for data collection and subsequent analysis of the results (Bogdan & Biklen, 1994). This type of study aims to carry out changes in the social context by systematically collecting information. As for the methodology, this study implemented a quasi-experimental pre- and post-test design with a nonequivalent control group. In this type of design, two conditions are met: (a) there is a group that is “similar” to the treatment group, which can serve as a control group, and (b) there is the opportunity to obtain pre-test and post-test measurements from individuals in the treatment and comparison groups. With the control group, it is possible to control threats to internal validity due to history, maturation, testing, instrumentation, and regression (Shaughnessy et al., 2012). The use of the instruments before the intervention fulfills the objective of evaluating an individual’s skills (social-emotional) and diagnosing the group’s most pressing demands to delineate the actions and tasks better and, at the end of the intervention, assess whether the intended changes took place. In this methodological design, the control group’s role was to verify whether there were any changes between the group that received the intervention and the group that did not and whether changes were due to natural development or could be attributed to the intervention’s effect. This study is characterized as a quasi-experimental design since the “groups may not be equivalent, because the participants were not randomly distributed according to the conditions”; the two 9th-grade classes at the school participated in the study, and the groups were separated according to this criterion (9th-grade class A was chosen as intervention group A (GA) and 9th-grade class B was the control group (CG). The CG, now called intervention group B (GB), participated in the intervention in the following semester. The advantages of this methodology include a decrease in costs and the possibility of intervening in specific environments where a random distribution of participants is not possible, as is the case in this study, in which the best condition for data collection was the organization of groups per class (Shaughnessy et al., 2012). The school imposed this condition for the intervention to be implemented. The quasi-experimental design with pre-test and post-test with a nonequivalent control group is widely used in evaluation studies of programs and services.
A 14-session intervention program was designed, including pre- and post-test, with weekly meetings of approximately 90 minutes, which were implemented according to the school’s availability. The intervention focused on two central axes: (a) social-emotional skills, with five sessions, and (b) career skills, with five sessions that aimed at activating exploratory and self-knowledge behaviors, clarifying professional interests, and information about the world of work and fields of action. Another study was developed (submitted for publication) to evaluate the second axis of the intervention, whose career competencies were evaluated using the Maturity Scale for Professional Choice (EMEP) (Neiva, 1999, 2014). An evaluation questionnaire was also adopted, collecting the students’ feedback regarding their experience with the program and learning throughout the Career Education intervention. Other assessments are recorded to clarify the intervention strategy.
The content concerning the developing social-emotional skills axis started to be addressed in the program’s second session. The program’s first session included the presentation, rules establishment, and the exploration of the self-concept. The second session focused on the development of Agreeableness skills. The participants exchanged roles to practice some complex situations usually experienced at home and school, and “magical words” were presented and discussed with the support of videos. The third session was intended to promote Engaging with others. Hence, videos and role-playing (assertive, passive, and aggressive communication) were adopted. The fourth session promoted the Emotional Resilience skill, in which anchoring vignettes from SENNA 1.0 represented by characters were presented using slides. The task was to reflect on their characteristics, advantages, and disadvantages. At the end of the meeting, the participants discussed which character handled the situations better. Also, in this session, excerpts from the films “Despicable Me” and “Brave” were discussed, followed by relaxation techniques. The fifth session focused on the development of the Conscientiousness skill. The participants were asked to create a chart with their daily routine and another one listing things “I like and do; I like and do not do; I do not like and do; and I do not like, and I do not do.” This exercise encouraged them to reflect on time management and routine school activities. The sixth session addressed the last skill, Openness. The Journey through the Past, Present, and Future Technique was used. It is a mime game based on famous individuals’ professions. Another task addressing occupations was implemented in the seventh session using iPads when we transitioned to the second axis, which focused on career-building skills (from the 7th to the 11th sessions)— the 12th and last session of the intervention aimed to conclude the intervention. Hence, the group considered and discussed its history, and the participants provided individual feedback to assess the intervention.
Data collection and analysis
The school where the intervention would be implemented provided its consent before the project was submitted to and approved by the Institutional Review Committee at the hosting university (CAAE No. 03088818.3.0000.5407). Later, the school was contacted again to invite the students and obtain their parents’ consent. All 9th graders were invited to participate in the study, and those interested took home a free and informed consent form and a database authorization form for the parents to provide their formal authorization. Students whose parents consented and signed the authorizations were allowed to participate; otherwise, they were not included in the study.
The data obtained through SENNA 2.0 were first analyzed based on the Item Response Theory (IRT) using the RStudio 3.5.1 software (R Development Team, 2021). The objective was to create a standardized score for each of the domains analyzed in each group. Next, descriptive, and statistical inferences were performed based on the scores standardized for each of the SENNA 2.0 domains obtained through the TRI on a scale ranging from -3 to 3 to verify the effect of the intervention on the social-emotional skills at the three points in time. These analyses were performed using the Jamovi 1.6.3 program (https://www.jamovi.org). First, an assumption test was performed to verify normality (Shapiro-Wilk test). Then, statistical inferences were performed through Hypothesis Tests. In this specific case, we made comparisons between two means for independent samples for the analyses between groups (Student’s t-test and Mann-Whitney U test) and comparisons in dependent samples for the intragroup analyses (Student’s t-test and Wilcoxon x test). The effect size was calculated using Cohen’s d Coefficient and the Point-Biserial Correlation Coefficient (rpb). Additionally, the two groups were descriptively analyzed at three points in time, using means, medians, standard deviation, and standard error. The intervention effects are presented in the results in two large sections: the Between-group comparisons and Intra-group comparisons. In the Between Groups section, the results are organized according to the intervention’s different points in time; and in the Intra-groups section, according to the group assessed.
Results
In addition to the analyses presented in the results section, Table 1 presents the descriptive analysis, including means, medians, standard deviation, and standard error for each group at Times 1, 2, and 3 in the five domains of SENNA 2.0. The descriptive statistics are presented according to groups and intra-groups.

The descriptive analysis in Table 1 presents the means for both groups. The means in both groups were similar, except for the Agreeableness (A) and Openness (O) domains. The Intervention Group A (GA) obtained the highest mean in Agreeableness (A), while the Control Group (GC), later called the Intervention Group B (GB), obtained the highest mean in Open-Mindedness (O) in the following semester. A significant difference was found between the groups in this last domain.
Comparisons between groups
According to the results (ABEP, 2016), when the Shapiro-Wilk normality test was significant, that is, the p-value was equal to or less than 0.05, the results of the Mann-Whitney U test were presented. Otherwise, the result of the Student’s t-test was presented (Table 2).

Comparison between GA-CG at Time 1
A significant difference was found only in the Agreeableness domain (t(65)=2.29; p=0.02; d=0.56), indicating that Intervention Group A (GA) has an advantage over the Control Group (CG). Regarding the effect size (d =0.56), a moderate effect was found on the differences between groups at Time 1 in this domain. The remaining domains did not show statistically significant p-values. An interesting result is that, even though the Conscientiousness domain (C) was statistically significant, its p-value was low, with an effect size (rpb = 0.27) indicating that the CG median at Time 1 was lower than that obtained by GA.
Comparison between GA-CG at Time 2
The results concerning the analysis of normality at Time 2 (Shapiro-Wilk Test) indicate that only the Conscientiousness (C) domain (p=0.26) followed a normal distribution. Significance was obtained in the Student-t test, while the Mann-Whitney U test was considered for the other dimensions. No significant difference was found between the groups in the five domains of SENNA 2.0 at Time 2. Even though the effect sizes for the dimensions Agreeableness (rpb = 0.18) and Conscientiousness (d = 0.17) were considered small, Intervention Group A (GA) obtained higher median and mean in the Agreeableness and Conscientiousness domains, respectively, than the Control Group (CG).
Comparison between GA-GB at Time 3
At Time 3, the Control Group (GC)/ Intervention Group B (GB) had already received the intervention; hence, GA and GB were compared. Hence, the results concerning the normality analysis (Shapiro-Wilk Test) show that the Agreeableness (p=0.27), Conscientiousness (C) (p=0.18), and Emotional Resilience (N) (p=0.74) domains were normally distributed. Thus, the Student’s t-test was used for these dimensions, while the Mann-Whitney U test was performed for the Engaging with others (E) and Openness (O) dimensions.
Only Openness (O) showed a significant value (Mann-Whitney; p=0.04; rpb=0.31), indicating a difference between the groups. Hence, Intervention Group B (GB) obtained a median higher than Intervention Group A (GA). The effect size (rpb=0.31) is considered moderate but still indicates a positive correlation in favor of Intervention Group B (GB). Although the other dimensions did not present statistically significant differences between the groups, the effect size found for the Agreeableness dimension (d = 0.46) is considered moderate, indicating that Intervention Group A obtained a higher mean than Intervention Group B (GB).
Intra-group comparisons
Intervention Group A (GA)
First, data normality was tested using the Shapiro-Wilk test to identify whether Student’s t-test or Wilcoxon x test should be applied. Thus, some results were significant (p ≤ 0.05) for the referred test. Therefore, Table 3 presents the results obtained in the intra-group comparisons. The Student’s t-test is shown when the p-value is not significant, and the Wilcoxon x test when the p-value is significant.

Table 3 shows no statistically significant differences in the means and medians obtained by Intervention Group A (GA) at any of the three points in time. However, the Agreeableness (A) domain obtained the highest means in Time 2. The effect size was d=-0.03 when comparing Time 1 and Time 2 and d=0.04 when comparing Time 2 and Time 3. Conversely, the Openness (O) and Engaging with Others (E) domains obtained their lowest means in Time 2.
Control Group (CG)/Intervention Group B (GB)
Again, data normality was first tested using the Shapiro-Wilk test. Then, if the null hypothesis was that data were normally distributed, the Student’s t-test was used. Otherwise, the Wilcoxon W test was used. The results are shown in Table 4.

Similar to the results found for Intervention Group A (GA), the Control group/Intervention Group B did not present statistically significant results. However, there is a pattern in which the means and medians obtained in Time 2 were lower than in Times 1 and 3, except for the Conscientiousness (C) domain. When Times 1 and 2 were compared, this domain presented the highest median in Time 2; however, between Times 2 and 3, the highest mean was obtained in Time 3.
Discussion
This study evaluated the effect of a Career Education intervention focused on developing social-emotional and career skills among 9th-grade students attending a Brazilian public school. The students were assessed at three points in time: before the intervention, after the intervention, and Intervention Group B (GB) was assessed five months later. Note that the Control Group received the intervention after the GA post-test.
The results from the descriptive analysis presented in Table 1 show that the means obtained in the Engaging with Others (E) and Openness (O) domains were higher in Time 1 than in Time 2, while the remaining domains obtained higher means in Time 2. Note that the Control Group (CG) was expected to show no significant differences or very different means between these times. Regarding the means obtained in Times 2-3, the Agreeableness (A), Conscientiousness (C), and Emotional Resilience (N) domains continued to obtain the highest means in Time 2. On the other hand, the means obtained in the Engaging with Others (E) and Openness (O) domains were higher at Time 3; i.e., after Intervention Group B (GB), the former Control Group (GC), received the intervention.
In summary, the descriptive results concerning social-emotional skills presented in Table 1 indicate that the means obtained by Intervention Group A in Times 1 and 2 presented a low variation. However, greater variation was found in Time 3. Nonetheless, the results obtained by Intervention Group B (GB) did not present the same variation, possibly indicating individual differences (Primi et al., 2021).
Significant differences were found only in the comparisons between groups. At Time 1, Intervention Group A (AG) showed greater development in the Agreeableness domain. At time 3, Intervention Group B (GB), which initially was the control group, showed a greater development of the Openness (O) domain. Remember that this study adopted a quasi-experimental pre- and post-test design with a non-equivalent Control Group. Hence, even though the Control Group (CG) was “similar” to the treatment group, the groups were not exactly equal; thus, the non-equivalent term is emphasized (Shaughnessy et al., 2012). In this sense, the literature review by Barbosa and Melo-Silva (2023), Evans et al. (2015), and Freeman et al. (2014) note that the designs vary considerably, and there is no standardization of methods, structures, or the instruments adopted in interventions involving social-emotional skills. Therefore, such a lack of standardization leads to different results between interventions or even an absence of perceptible differences, as is the case here.
After the intervention, the differences disappeared; that is, the intervention implemented to the former Control Group (CG), now called Intervention Group B (GB), may have enabled greater equivalence between the groups in the Agreeableness domain. The difference showed by Intervention Group B (GB) in the Openness (O) domain after the intervention (Time 3) indicates that the intervention more strongly impacted this domain. Note that Intervention Group B (GB) had just completed the intervention; thus, the content was still fresh. Several factors may have influenced the last assessment. Among these, there is the effect of time, which may have negatively influenced the content recalled, and the fact that the test was applied three times, which potentially harmed the responses. Additionally, the participants reported in the feedback interview that they were focused on the end of the semester and school year and were concerned with the next school cycle.
The lack of statistically significant differences may be related to the sample size, which was small (n 62) and theoretically insufficient from a statistical point of view to observe valid differences. In this sense, studies addressing a more significant number of participants are needed. Another important aspect concerning the absence of statistically observable effects in the other domains when comparing the groups and intra-group is the relatively short duration of the intervention, which configures a limitation in this study. Even though the program was implemented in 12 sessions, only five focused specifically on developing social and emotional skills, which may have mitigated the effects of the intervention on this variable.
Furthermore, as noted in the literature review, interventions focused on developing social-emotional skills vary considerably, comprising from eight to 40 sessions, and not all interventions obtained the expected results. Zyga et al. (2018) implemented eight sessions and promoted changes in social-emotional levels, but the analyses indicated that the gains were related to school and individual factors. The study by Sidera Caballero (2019) comprised 11 sessions and also failed to reduce aggressiveness or improve the level of empathy or moral disconnection through the development of social-emotional skills. The intervention implemented by Kiviruusu et al. (2016) lasted six months. No differences were found, which the authors related to the relatively short class time devoted to the program. Leal et al. (2020) implemented 12 sessions and obtained positive social-emotional development and career exploration results. Garcia et al. (2019) and Pavoski et al. (2018) implemented 14 sessions each, which decreased anxiety and depression symptoms. The interventions implemented by Romero et al. (2019), with 19 sessions, and by Chaux et al. (2017), with 40 sessions, promoted decreased behavior and aggressiveness problems.
Thus, in addition to significant variability in the duration of interventions, all studies presented a higher number of sessions that specifically focused on social-emotional development than this study. The absence of results in this investigation corroborates the considerations of Kiviruusu et al. (2016), noting the relatively short follow-up period as a factor influencing the absence of intervention results.
Nevertheless, other factors must be considered when assessing the intervention effects on the development of social-emotional skills. Among them is the participants’ degree of participation and engagement, which may influence the absence of significant results when analyzing the effect of the intervention between groups and within groups. In this sense, one should consider that the program’s coordinator did not belong to the students’ social circle, i.e., she was an external agent from outside the school, which may have prevented the development of rapport, which could have encouraged better engagement.
Additionally, each student has a specific level of social-emotional skills development, as it becomes apparent in the comparisons between groups and within groups, in which a great dispersion is found in the means and medians obtained in the five domains, as shown in Table 1. Such heterogeneity indicates that the students’ development in the five domains was very distinct and broad, possibly reflecting the absence of statistically verifiable results. The content of each dimension or macro-competences developed in a session was also somewhat complex to be developed in a single meeting; psychological dimensions are not easily modified and require more extended programs. It may also be one of the limitations of both this study and intervention.
This study’s results support future studies and interventions. In addition, they indicate methodological aspects to be discussed and qualify practices and the development of public policies intended to universalize the availability of programs and services focused on Career Education and social-emotional development. As noted by Evans et al. (2015) and Freeman et al. (2014), Career Education practices are developed in different modalities and interventions; thus, studies on intervention models are recommended. As a result of this study, future studies are also suggested to address the instruments that evaluate interventions. Hence, future interventions are suggested to be implemented throughout the school year or even addressing all school stages, according to the initial objective of Career Education practices, to sensitize and promote career and life skills from childhood.
In this sense, an opportunity to assess the effects of long-term interventions in the future would be through the competencies proposed by the BNCC, which have been implemented differently by Brazilian schools. In addition to including the development of social-emotional skills at school, the ten key competencies include a Life and Work Project, similar to the Career Education practices developed mainly in the international context.
Taking into account the relevance of Career Education practices and the investment in social-emotional learning for personal development, academic success, adaptation at work, and other career variables, programs intended to promote social-emotional skills are increasingly needed (Durlak et al., 2011; Garcia et al., 2019; Leal et al., 2020; Pavoski et al., 2018; Primi et al., 2016; Romero et al., 2019). Programs successful in promoting social-emotional skills can encourage young people to engage with academic performance more and prepare students to deal with the new demands of life and work. Life and work demands are constantly changing and require workers to have various skills to better adapt to the world of work and deal with the innumerable and complex challenges the current context imposes on individuals and collectives.
References
Associação Brasileira de Empresas de Pesquisa [ABEP]. (2016). Critério Brasil 2015 e atualização da distribuição de classes para 2016. http://www.abep.org/criterio-brasil
Barbosa, D., & Melo-Silva, L. L. (2023). Programas de intervenção em habilidades e competências socioemocionais: Revisão sistemática da literatura. Revista Psicologia em Pesquisa, 17(2). https://doi.org/10.34019/1982-1247.2023.v17.34379
Bogdan, R. C., & Biklen, S. K. (1994). Investigação qualitativa em educação. Porto Editora.
Boyatzis, R. E. (2019). Emotional intelligence and its measurement. In Oxford Research Encyclopedia of Business and Management (pp. 1–20). Oxford University Press. https://doi.org/10.1093/acrefore/9780190224851.013.159
Campos, C. R., Oliveira, K. S., Lessa, J. P. A., & Primi, R. (2020). Adaptação e evidências de validade baseadas na estrutura interna da Escala de Domínios de Criatividade. Psico, 51(3), e34502. http://dx.doi.org/10.15448/1980-8623.2020.3.34502
Chaux, E., Barrera, M., Molano, A., Velásquez, A. M., Castellanos, M., Chaparro, M. P., & Bustamante, A. (2017). Classrooms in peace within violent contexts: Field evaluation of Aulas en Paz in Colombia. Prevention Science, 18(7), 828–838. http://dx.doi.org/10.1007/s11121-017-0754-8
Collaborative for Academic, Social, and Emotional Learning – CASEL. (2005). Safe and sound: An educational leader’s guide to evidence-based social and emotional learning programs. https://casel.org/safe-and-sound-guide-to-sel-programs/?download=true
De Fruyt, F., Wille, B., & John, O. P. (2015). Employability in the 21st century: Complex (interactive) problem solving and other essential skills. Industrial and Organizational Psychology, 8(2), 276–281. http://dx.doi.org/10.1017/iop.2015.33
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82(1), 405–432. http://dx.doi.org/10.1111/j.1467-8624.2010.01564
Evans, R., Murphy, S., & Scourfield, J. (2015). Implementation of a school-based social and emotional learning intervention: Understanding diffusion processes within complex systems. Prevention Science, 16(5), 754–764. http://dx.doi.org/10.1007/s11121-015-0552-0
Freeman, E., Wertheim, E. H., & Trinder, M. (2014). Teacher perspectives on factors facilitating implementation of whole school approaches for resolving conflict. British Educational Research Journal, 40(5), 847–868. http://dx.doi.org/10.1002/berj.3116
Garcia, L. M. R., Toni, C. G. S., Batista, A. P., & Zeggio, L. (2019). Evaluation of the effectiveness of the Fun FRIENDS Program. Trends in Psychology, 27(4), 925–941. http://dx.doi.org/10.9788/tp2019.4-08
Hoyt, K. B. (2005). Career education as a federal legislative effort. In K. B. Hoyt (Org.), Career education: History and future (pp. 3–74). National Career Development Association.
Kiviruusu, O., Björklund, K., Koskinen, H.-L., Liski, A., Lindblom, J., Kuoppamäki, H., Alasuvanto, P., Ojala, T., Samposalo, H., Harmes, N., Hemminki, E., Punamäki, R.-L., Sund, R., & Santalahti, P. (2016). Short-term effects of the “Together at School” intervention program on children’s socio-emotional skills: A cluster randomized controlled trial. BMC Psychology, 4(1), 27. http://dx.doi.org/10.1186/s40359-016-0133-4
Leal, M. S., Melo-Silva, L. L., & Taveira, M. C. (2020). Edu-Car for life and career: Evaluation of a program. Estudos de Psicologia, 37, e190016. http://dx.doi.org/10.1590/1982-0275202037e190016
Ministério da Educação. (2018). Base Nacional Comum Curricular (BNCC): Educação é a base. http://basenacionalcomum.mec.gov.br/images/BNCC_EI_EF_110518_versaofinal_site.pdf
Ministério da Educação. Instituto Nacional de Estudos e Pesquisas Anísio Teixeira. (2000). Avaliações e exames da educação básica: Relatório final 1999 – Enem. https://www.gov.br/inep/pt-br/centrais-de-conteudo/acervo-linha-editorial/publicacoes-institucionais/avaliacoes-e-exames-da-educacao-basica/ relatorio-final-1999-2013-enem
Munhoz, I. M. S., & Melo-Silva, L. L. (2011). Educação para a Carreira: Concepções, desenvolvimento e possibilidades no contexto brasileiro. Revista Brasileira de Orientação Profissional, 12(1), 37–48. http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1679-33902011000100006
Neiva, K. M. C. (1999). Manual: Escala de Maturidade para a Escolha Profissional (EMEP). Vetor.
Neiva, K. M. C. (2014). Manual: Escala de Maturidade para a Escolha Profissional (EMEP) (2a ed.). Vetor.
Oliveira, P. V., & Muzkat, M. (2021). Revisão integrativa sobre métodos e estratégias para promoção de habilidades socioemocionais. Revista Psicopedagogia, 38(115), 91–103. https://doi.org/10.51207/2179-4057.20210008
Pavoski, G. T. T., Toni, C. G. S., Batista, A. P., & Ignachewski, C. L. (2018). Prevenção universal e promoção de saúde em grupo de crianças a partir do Método Friends. Psico, 49(2), 148–158. http://dx.doi. org/10.15448/1980-8623.2018.2.26501
Primi, R., Santos, D., John, O. P., & De Fruyt, F. (2016). Development of an inventory assessing social and emotional skills in Brazilian youth. European Journal of Psychological Assessment, 32(1), 5–16. http://dx.doi.org/10.1027/1015-5759/a000343
Primi, R., Santos, D., John, O. P., & De Fruyt, F. (2021). SENNA Inventory for the Assessment of Social and Emotional Skills: Technical manual. Instituto Ayrton Senna http://dx.doi.org/10.31234/osf.io/byvpr
Romero, E., Gómez-Fraguela, X. A., Villar, P., & Rodríguez, C. (2019). Prevención indicada de los problemas de conducta: Entrenamiento de habilidades socioemocionales en el contexto. Revista de Psicología Clínica con Niños y Adolescentes, 6(3), 39–47. http://dx.doi.org/10.21134/rpcna.2019.06.2.1
Saleh, A., & Watson, R. (2017). Business excellence in a volatile, uncertain, complex and ambiguous environment (BEVUCA). The TQM Journal, 29(5), 705–724. http://dx.doi.org/10.1108/TQM-12-2016-0109
Santos, D., & Primi, R. (2014). Desenvolvimento socioemocional e aprendizado escolar: Uma proposta de mensuração para apoiar políticas públicas: Relatório sobre resultados preliminares do projeto de medição de competências socioemocionais no Rio de Janeiro. OCDE; Instituto Ayrton Senna; SEEDUC.
Sette, C. P., & Alves, G. (2021). Competências socioemocionais: A importância do desenvolvimento e monitoramento para a educação integral. Instituto Ayrton Senna. https://institutoayrtonsenna.org.br/app/uploads/2022/12/instituto-ayrton-senna-avaliacao-socioemocional-1.pdf
Shaughnessy, J. J., Zechmeister, E. B., & Zechmeister, J. S. (2012). Desenhos quase-experimentais e avaliação de programas. In J. J. Shaughnessy, E. B. Zechmeister, & J. S. Zechmeister, Metodologia de pesquisa em psicologia (R. C. Costa, Trad., 9a ed., pp. 316–346). AMGH.
Sidera Caballero, F., Rostan, C., Collell, J., & Agell, S. (2019). Aplicación de un programa de aprendizaje socioemocional y moral para mejorar la convivencia en educación secundaria. Universitas Psychologica, 18(4), 1–14. http://dx.doi.org/10.11144/Javeriana.upsy18-4.apas
Watts, A. G. (2001). Career education for young people: Rationale and provision in the UK and others European countries. International Journal for Educational and Vocational Guidance, 1(3), 209–222. http://dx.doi.org/10.1023/A:1012231803861
Zyga, O., Russ, S. W., Meeker, H., & Kirk, J. (2018). A preliminary investigation of a school-based musical theater intervention program for children with intelectual disabilities. Journal of Intellectual Disabilities, 22(3), 262–278. http://dx.doi.org/10.1177/1744629517699334
Notes
Author notes
* Correspondence concerning this article should be addressed to Lucy Leal Melo-Silva, Rua Faveiro, 244, Ribeirão Preto, SP, Brasil. CEP 14040130. Email: lucileal@ffclrp.usp.br