Resumen: La investigación sobre el compromiso escolar de los estudiantes ha crecido significativamente en la última década y su medición es de gran interés en la investigación educativa. Esto se podría explicar por su potencial para comprender problemas educativos importantes con respecto a las trayectorias académicas de los estudiantes, incluida la adaptación a la escuela, el rendimiento y los logros académicos, las tasas de finalización y la deserción escolar. Se ha evidenciado que los estudiantes que se sienten más comprometidos con su escuela experimentan una trayectoria escolar más positiva y tienen una vida más exitosa, por lo que se considera que el compromiso escolar es una variable protectora vinculada a tasas más bajas de delincuencia, abuso de sustancias y depresión. Para obtener una comprensión más profunda del cuerpo de investigación actual sobre el compromiso escolar de los estudiantes, esta revisión de la literatura tuvo por objetivo analizar las características metodológicas de la investigación empírica cuantitativa sobre ese compromiso de los estudiantes en la escuela secundaria. Como método se realizó una revisión sistemática de la literatura en las bases de datos Web of Science, Scopus y SciELO del período 2013 a 2020. Cuarenta y siete artículos cumplieron los criterios de inclusión establecidos. Los resultados evidenciaron que casi todos los estudios se realizaron en Estados Unidos, mientras que ninguno de los artículos analizados fue desarrollado en América Latina. Con respecto al tamaño de la muestra, se identificó que la mayoría de los estudios revisados tenían un tamaño de muestra entre 501 y 5000 participantes. En cuanto a los diseños de investigación utilizados en estos estudios, la mayoría utilizó un diseño correlacional, pero solo unos pocos implementaron diseños longitudinales o cuasi experimentales. Se identificó la coexistencia de diferentes marcos conceptuales sobre el compromiso escolar; sin embargo, la investigación concuerda en comprenderlo como un constructo multidimensional que implica el involucramiento del estudiante en actividades relacionadas con la escuela y su estudio. La diversidad conceptual del constructo de compromiso escolar se refleja en los numerosos instrumentos identificados en esta revisión, en los que las dimensiones más prevalentes del compromiso escolar fueron la dimensión conductual, cognitiva y emocional. Respecto de los tipos de variables incluidas en los estudios de compromiso escolar, se observó que se han incluido un gran número que cubren diferentes aspectos y temas relacionados con las experiencias académicas de los estudiantes, como las relacionadas con los propios estudiantes, así como las relacionadas con los padres, compañeros, profesores y escuelas. En conclusión, dado que el compromiso escolar de los estudiantes es una variable que se ha considerado crítica en contextos académicos, a medida que se realicen estudios futuros en este campo, será importante examinar la correlación de diferentes tipos de variables con el compromiso escolar de los estudiantes. Se podrían examinar las potenciales variables moderadoras que podrían surgir al realizar estudios en nuevos entornos educativos o culturalmente diversos, por ejemplo, con estudiantes con necesidades especiales. Esto es especialmente importante cuando se considera a la región Latinoamericana. Dado que la mayoría de los estudios se han realizado en Estados Unidos, se requiere considerar aspectos importantes antes de su implementación, como la validez de los instrumentos de medida, los que podrían estar sesgados si no se adaptan a la cultura Latinoamericana. Además, los estudios futuros deberían definir con precisión el constructo de compromiso escolar de los estudiantes y lograr un consenso en la investigación.
Palabras clave: revisión sistemática, compromiso escolar, educación secundaria, estudios cuantitativos.
Abstract: Student engagement research has grown significantly in the past decade and measuring it is of high interest in educational research. To gain a deeper understanding of the current body of research on student engagement, this review aims to analyze the design characteristics of empirical quantitative research on student engagement in secondary school. A systematic review was performed in the Web of Science, Scopus, and SciELO databases from the period 2013 to 2020. Forty-seven articles met the inclusion criteria. The results revealed that most studies were performed in North America and none in Latin America; the designs were mainly measuring variable associations. Teacher-related variables are the least examined variables. The instruments used to measure the student engagement and the constructs employed, shows there is substantial theoretical heterogeneity among studies. Future studies need to accurately define student engagement; give further attention to variables related to teachers, peers, families, and institutional conditions.
Keywords: Systematic review, student engagement, secondary school, school engagement, quantitative studies.
Revisiones
Revisión sistemática del compromiso escolar en educación secundaria
A systematic review of methodological aspects of student engagement research in secondary students
Centro Interamericano de Investigaciones Psicológicas y Ciencias Afines
Esta obra está bajo una Licencia Creative Commons Atribución-NoComercial-CompartirIgual 4.0 Internacional.
Recepción: 12 Abril 2021
Aprobación: 14 Diciembre 2022
In recent decades there has been increased research on student engagement, which could be explained by its potential in understanding important educational issues regarding students’ academic trajectories including adaptation to school, academic performance and achievement, completion rates, and dropping out of school (Clark, 2015; Fredricks et al., 2016; Fung et al., 2018). Besides, students’ engagement is a protective variable linked to lower rates of crime, substance abuse, and depression (Fredricks et al., 2016a). Further, students who feel more engaged in their school experience a more positive school trajectory and have a more successful life (Kızıldağ et al., 2017).
However, student engagement is a complex construct with many concepts coexisting (Appleton et al., 2008; Mirzaei-Alavijeh et al., 2018; Prendergast & Rickinson, 2019). Some of the concepts used to refer to student engagement are “engagement,” “academic engagement,” “school engagement,” “student engagement in academic work,” “engagement in schoolwork,” “student engagement in/with school”.
Furthermore, the operational definition and measurement of student engagement also vary (Appleton et al., 2008; Sinatra et al., 2015). The construct of student engagement is often described as having multiple dimensions. For instance, Fredericks et al. (2004) define student engagement as a meta construct that includes behavioral, emotional, and cognitive dimensions, while Appleton et al. (2008) describe four dimensions: academic, behavioral, cognitive, and psychological.
Previous systematic reviews on student engagement have focused on different elements of this construct. A significant number of them address how contextual factors influence student engagement, particularly the impact of the teacher-student relationship (Conner, 2016; Harbour et al., 2015; Quin et al., 2018), and the influence of peer relationships (Engels et al., 2017), while others analyze the impact of interventions on student engagement (Fredricks et al., 2019a). These systematic reviews have not described which variables of the student engagement construct are measured. Even more, these reviews do not describe what type of quantitative empirical research was employed, focusing particularly on secondary education. Moreover, there is no known systematization of the designs and instruments used to assess student engagement or the dimensions of the constructs considered in the studies.
Based on this, the objective of this systematic review is to describe the methodological aspects of quantitative empirical research focusing on student engagement in secondary school. Specifically, this research addresses the following questions: a) What methodologies have been used to study student engagement? b) What variables of student engagement have been studied? and c) What instruments have been used and what engagement dimensions are included in the instruments?
This systemic review aims to aid researchers by providing evidence on what has been done in the field of engagement and the current gaps in the knowledge in this field, which in turn could inform future directions in research, particularly relevant in less studied contexts such as Latin America.
To conduct a systematic review of recent literature on student engagement a methodology was designed following international guidelines (Moher et al., 2015). The method considered two different aspects: (a) the identification and selection of the studies; and (b) the extraction and the analysis of data from said studies.
This part included five stages (Figure 1). (1) Identification: a search of articles published between 2013 and 2020 (final inclusion date: June 20, 2020) was conducted in three databases (Web of Science, Scopus, and SciELO). Articles in English, Spanish and Portuguese were included. This timeframe was selected for two reasons. First, previous academic revisions on this topic were conducted prior 2013 (Lawson & Lawson, 2013; Wang & Degol, 2014; Wimpenny & Savin-Baden, 2013), which have provided relevant information on how studies have defined and studied the engagement of students, yet it is relevant to update this knowledge. Second, this timeframe has been particularly productive in research on student engagement. For instance, a search conducted in Scopus showed that from 2013 until 2020, more than 100 studies were conducted each year on student engagement. Therefore, this timeframe accounts for a productive period to examine the particularities of the research of student engagement. The keywords used were "engagement" and "secondary schools" OR "high school." (2) Removal of duplicates: the detection and elimination of articles that showed up more than once. (3) Eligibility: two independent researchers reviewed the papers using a protocol that outlined the objective of the study and keywords that should be included in the title and abstract of the papers, according to the aforementioned criteria. Articles included in the next stage had to be selected by both judges. In the cases of discrepancy, a third researcher was consulted. (4) Selection: the complete examination of the articles chosen in the previous stage, further eliminating the articles to only quantitative empirical studies focusing on secondary education. This process excluded: theoretical articles, qualitative empirical studies, studies conducted in non-academic settings, studies conducted in non-secondary levels, studies in which the main variable was not engagement, and studies with samples that did not include students. (5) Control for bias: To assess the rigor and objectivity of the selection process, two independent reviewers performed a full review of the process. The review encompassed the revision of all stages of the process, paying special attention to phases three and four.
Figure 1.
Flowchart of the search and screening process
To address the objective of the study a protocol was designed to extract information of the selected articles that includes the following: methodologies, variables of student engagement, and instruments. (Appendix 1).
The studies were grouped considering the classification of psychology designs proposed by Ato et al., (2013), widely used in quantitative research. This analysis reveals that most of the studies used associative designs (87.23, n = 41), and only 4.26 % (n = 2) used quasi-experimental designs (Appendix 1). As for the samples in the studies reviewed, samples ranging between 501 and 1000 students (29.79 %) were the most prevalent, followed by samples between 1001 and 5000 students (25.53 %). The analysis also identified the countries from which the samples were drawn of the studies, 36.17 % of them were in the United States, followed by Turkey and Australia both with 10.64 %. The representativeness of the rest of the countries is very low in comparison (Appendix 1). A closer analysis of the research sites selected highlights the predominance of studies conducted in North America (36.17 %) followed by studies done in Europe (27.66 %). Note that four studies (8.51 %) are cross-continental (Oceania, America, Europe, Asia).
While the studies used multiple variables to assess student engagement, these were grouped by constructs, which assisted in the development of 20 categories. A predominance of the variables related to students’ characteristics (61.03 %), followed by school variables (17.65 %). Besides, 9.56 % of variables focused on teachers, while 8.09 % of variables were related to family, and 3.68 % focused on peers. Table 1 shows that, among the most prevalent variables employed, 13.24 % were based on students’ characteristics or sociodemographic factors, 9.56 % on student’s academic behavior or skills, 8.82 % on the school environment, and 8.82 % on school characteristics.
Categories of variables included in studies of student engagement
A closer analysis of the most prevalent variables employed to study student engagement in our selection (N = 47) (Table 2) shows three main trends. First, more than half of the articles (53.2 % and 51.1 %) include student gender and age as control variables. Second, studies strongly focus on students’ academic performance and achievement with 23.4 % of them studying student academic achievement often measured by standardized tests in particular subject areas, while 19.1 % of studies examine students’ performance generally using grades. However, beyond that, there are very few variable trends, with most of the variables used in only one article.
Prevalent variables by category
The examination of the instruments used to assess student’s engagement leads to two main points. First, there is a vast number of instruments that have been used to measure engagement in secondary high school students with thirty-five instruments used in the studies being considered (Table 3). For instance, 59.57 % (N = 28) of the studies used a complete instrument to assess student engagement, while 40.43 % (N = 19) used a selection of items drawn from engagement instruments based on the dimensions of engagement assessed. Second, the Engagement and Disaffection Scale (Skinner et al., 2009) was the most widely used instrument, with three (3) studies using the entire scale and another three (3) studies using some subset. This scale measures behavioral and emotional engagement and behavioral and emotional disengagement. The Utrecht School Engagement Scale (Schafeuli et al., 2002) was the second most used instrument, with two (2) studies using the entire scale, and two (2) studies using a partial version of it. This scale assesses three dimensions of engagement: vigor, dedication, and absorption.
The final analysis incorporated the exploration of the dimensions of engagement considered by the instruments. The dimensions were grouped by the names provided in the articles and no inference was made about the conceptual framework from which these dimensions were drawn from. A total of 37 dimensions of engagement were identified, but only three were used regularly: behavioral engagement (N = 16), cognitive engagement (N = 15), and emotional engagement (N = 11). Falling far behind include measures like academic engagement, affective engagement, adaptive motivation, adaptive engagement, maladaptive motivation, and maladaptive engagement, included in two instruments each. The rest of the dimensions (N = 26) were only present in one instrument.
Instrument and dimensions of student engagement
As mentioned previously nearly all these studies were in the United States, while none of the papers analyzed were from Latin America. This finding is consistent with recent research on student engagement on primary and middle school levels conducted in Latin America (Hennig et al., 2019; Rigo & Donolo 2019) that highlights the lack of research on student engagement in this region. For one exception published after the collection of data of this study see the work of Ochoa-Angrino et al., (2020).
Concerning the sample size, we identified that the majority of the studies reviewed had a sample size between 501 and 5000 participants. According to Taherdoost (2017), this range of sample size is considered adequate to model the given population, these types of instruments, and the type of statistical analysis conducted. Further, this range of sample size is recommended (Marszalek et al., 2011) as they help avoid biased results.
As for the research designs used in these studies, the majority used correlation but only a few studies implemented longitudinal or quasi-experimental designs. This finding is similar to reviews of literature that have focused on intervention (Bond, 2020; Fredricks et al., 2019b) stressing that there are few experimental studies on student engagement interventions. This finding has implications for future research, as experimental research designs and longitudinal studies are necessary to understand developmental trends and interactions between student engagement and student, teacher, and family-related variables.
As previously stated, the complex nature of student engagement has resulted in a field of study with coexisting theoretical frameworks, which in turn promoted competing constructs of student engagement. This engagement is often defined as the student's participation in school-related activities and their achievement. Researchers (Fredricks et al., 2016b; Tomás et al., 2016) stress that student engagement is a multidimensional construct, which is reflected in the numerous instruments identified in this review. Our analysis highlights that the most prevalent dimensions of engagement are behavioral, cognitive, and emotional. This result suggests that despite the existence of different theoretical models (Hu et al., 2012; You & Sharkey, 2009) the model proposed by Fredericks et al. (2004) is the leading one, informing the design of many instruments as 25 % of the screened instruments include the three dimensions proposed by these authors. Further, the large amount of engagement dimensions (N = 37) is indicative of the theoretical heterogeneity in the research. This can be taken as supportive evidence of the large number of instruments that coexist for assessing student engagement, which has been studied previously (Fredricks & McColskey, 2012).
There is value in these coexisting conceptualizations of student engagement and the different instruments employed stress that student engagement is not a simple, easily identified and addressed phenomenon. These theoretical frameworks and conceptualizations can provide lenses through which to see how different audiences understand and think about this phenomenon, allowing for multifaceted solutions for a multifaceted problem. Further, given the different instruments to measure student engagement, researchers should check the alignment between their theoretical conceptualization of student engagement and the instrument they choose (Moreira & Dias, 2018). The variety allows for a much closer matching of the instrument chosen to research goals.
In examining the variables, an extensive number of variables have been included in studying student engagement, which cover different aspects and subjects related to students’ school experiences, including variables relating to students themselves, as well as those relating to parents, peers, teachers, and schools. This finding is not surprising given that engagement is a complex topic, and different types of engagement may have different outcomes. For example, cognitive engagement measured as test scores may show a correlation with future income, but when measured as time spent on homework, the finding is different. Since student engagement/participation is a variable that has been found critical in so many outcomes like student dropout rates, long-term health, future work life, it is exceptionally important to examine which variables have impacts on which outcomes. As future studies in this field are conducted, it is important to examine the correlation of different sets of variables to student engagement, as well as examining the interactions between potential moderators that could arise in conducting studies in new culturally diverse settings, or educational settings working with students with special needs. This is especially key when considering Latin America; since most studies are done in the US, current instruments and measures may be biased and will not appropriately measure what engagement looks like in understudied regions of the world or understudied populations.
The results of the aggregation of the student engagement variables into five groups (i.e., student, parent, teacher, peers, and institutional related variables) show that research has focused on student-related variables, particularly academic ones. The emphasis of these studies on individual variables of engagement however completely misses the contextual richness of student engagement explaining school achievement, focusing solely on the students’ responsibility of his/her academic success or failure.
When it comes to research, it is recommended that scholars should carefully consider their methods and theoretical frameworks in studying student engagement since there is such a wide field of different conceptualizations and instruments available. This is particularly relevant when studying engagement for students from different cultural and academic backgrounds, which are currently understudies. Including other kinds of methods for data collection such as online platforms (Henrie et al., 2015) might provide additional insights into this field of study.
Student engagement is a critical aspect of students’ school trajectories. The findings of this study suggest research should expand the scope of this field not only including students from different regions and cultural backgrounds but also including students whose school trajectories differ from regular students given specific educational needs. It is also important to highlight the relevance of including deeper contextual factors such as teacher, parent, peers, and institutional variables. It seems likely that such variables also have important impacts on engagement and including them in research will give policymakers more evidence about important factors to work with to promote student engagement and achievement in school. Further, our study provides relevant evidence on the complexities of measuring student engagement given the coexistence of different perspectives, providing information about theoretical and methodological issues that can guide future decision-making in research and policy to build learning environments that support the engagement of students.
Lastly, this article offers suggestions for future directions for research on student engagement in less studied contexts, such as Latin America. Researchers should consider collecting data on student engagement in this region to illuminate the educational experiences and trajectories of students.
While this study highlights relevant areas for future research, the analysis carries a major limitation, as this study focuses on reviewing only quantitative research. Future systematic reviews could take on the task of analyzing qualitative studies on student engagement, expanding the knowledge in this field.
Lastly, future studies should focus on conducting systematic reviews on the knowledge gathered during the 2020-2021 period, in which emergency remote education has been implemented throughout the world, affecting students’ trajectories.
Figure 1.
Flowchart of the search and screening process
Categories of variables included in studies of student engagement
Prevalent variables by category
Instrument and dimensions of student engagement
