INTRODUCTION

In recent years, with university studies now fully integrated into the European Higher Education Area (EHEA), a change in teaching methodologies is underway. As Prieto et al. (2021) and Tourón et al. (2021) explain, this change is driven by the need to achieve an engaging and motivating learning experience that connects with students’ interests, enabling them to achieve the skills they will eventually use in the workplace. (Lai et al., 2018). This implies rethinking the traditional classroom model in which information is only transmitted through lectures (Bok, 2017; McLaughlin et al., 2014). Moreover, the Covid-19 pandemic, which prompted a massive introduction of technology in teaching, has made this shift towards more active methodologies even more urgent. One of the methods that has attracted the most attention is the flipped classroom (hereafter, FC), also known as the inverted classroom, flipped learning, among other similar names. Prieto et al. (2021) demonstrate the growing volume of scientific publications focused on FC, which amounts to 52,000 citations in the last decade.

Bergmann and Sams (2012) argue that, broadly speaking, this methodological approach reverses the traditional way of teaching, in which teachers explain the theoretical content of their subject in class and students study and complete exercises or assignments at home. With FC, students are responsible for learning the theoretical concepts at home, before face-to-face sessions with teachers, from materials such as videos or explanatory texts prepared or proposed by their teachers. Classroom sessions with teachers are devoted to more meaningful tasks that require interaction between students and teachers, such as resolving doubts that may have arisen from individual lesson preparation, discussion, case studies, problem solving, etc. According to Prieto et al. (2021), FC provides the elements necessary to address the paradigm shift that higher education needs today: digitalisation, introduction of information technologies and a dynamic role of students in and out of the classroom that involves them in the learning process.

The basic principles of FC have been implemented through various teaching strategies, which can be considered precursors to FC. For example, Prieto et al. (2021), in a review of the evidence on FC, explain that these strategies were widely implemented prior to the popularisation of the term “flipped classroom”. The most common are peer instruction (PI), just-in-time teaching (JiTT) and team-based learning (TBL). According to Mazur (1997) and Medina et al. (2010), in the first case, students are responsible for raising and discussing questions that have arisen from studying the content previously at home. Novak et al. (1999) point out that in JiTT, teachers develop a questionnaire that they ask students to answer sometime before the face-to-face class (usually between 2 days and 1 hour). The teachers then use these answers to adapt the explanations and activities developed in class. This allows teachers to discover if any concepts have not been well understood, or to detect which aspects have generated the most interest in their students. By contrast, in TBL, according to Michaelsen et al. (2002), at the beginning of class, students individually answer a questionnaire (with content that they have previously prepared at home) and then meet in small groups to reach agreement on their answers. Finally, all the students (with the teacher) discuss the proposals and come up with the correct answers.

However applied, FC gives students a greater role in the classroom and motivates them more. This results in more meaningful learning and better understanding and retention of content (Prieto et al., 2014b; Romero-García et al., 2021).

Several systematic reviews and meta-analyses have assessed the efficacy of this methodology in university classrooms. Most of these studies (Bao-Zhu Li et al., 2020; Bredow et al., 2021; Chen & Hsu, 2022; Doğan et al., 2021; Evans et al., 2019; Galindo-Domínguez, 2021; Ge et al., 2020 ; Hew et al., 2021; Lin et al., 2021; Lo & Hew, 2019; Manoj et al., 2018; Martínez et al., 2019; Özdemir & Şentürk, 2021; Prieto et al., 2021; Shi et al., 2020; Talan & Batdi, 2020; Turan & Akdag-Cimen, 2019) agree that the methodology generally improves academic performance, albeit with only a moderate level of evidence. However, some systematic reviews (Sisi Li et al., 2020; Talan & Batdi, 2020; Turan & Akdag-Cimen, 2019) include papers showing neutral or, in some cases, favourable results for the control group.

The results of other research (Chen & Hsu, 2022; Ge et al., 2020; Martínez et al., 2019; Oudbier et al., 2022; Prieto et al., 2021; Talan & Batdi, 2020; Turan & Akdag-Cimen, 2019) point to benefits of FC such as improved student motivation. The results of Talan and Batdi’s (2020) study highlight students’ ability to overcome fear of failure, while the work of Talan and Batdi (2020) and Manoj et al. (2018) also indicate improved attendance and active participation in the classroom. Moreover, other authors (Sisi Li et al., 2020; Talan & Batdi, 2020; Turan & Akdag-Cimen, 2019) observe that students who follow this methodology seem to master self-learning skills better. Some researchers (Ge et al., 2020; Sisi Li et al., 2020; Xu et al., 2019) highlight problem solving and others (Chen & Hsu, 2022; Ge et al., 2020; Sisi Li et al., 2020) teamwork. In addition, some studies (Senali et al., 2022; Sisi Li et al., 2020; Talan & Batdi, 2020; Turan & Akdag-Cimen, 2019) point out that students working with FC improve communication skills, and several papers (Hew et al., 2021; Oudbier et al., 2022; Talan & Batdi, 2020) note the ability to self-manage time, while others (Oudbier et al., 2022; Senali et al., 2022; Talan & Batdi, 2020) highlight creativity and one (Turan & Akdag-Cimen, 2019) notes ICT proficiency.

With regard to the analysis of the areas of knowledge in which FC is applied in universities, the consulted works focus on the field of health (Bao-Zhu Li et al., 2020; Conte et al., 2021; Evans et al., 2019; Ge et al., 2020; Lin et al., 2021; Manoj et al., 2018; Oudbier et al., 2022; Sisi Li et al., 2020; Xu et al., 2019), foreign language teaching (Turan & Akdag-Cimen, 2019), engineering (Lo & Hew, 2019), the sciences (Doğan et al., 2021) and business and economic sciences (Senali et al., 2022). Other research includes different areas of knowledge. These include the work of Brewer and Movahedazarhouligh (2018), Hew et al. (2021), Martínez et al. (2019), Prieto et al. (2021), Shi et al. (2020) and Talan and Batdi (2020).

However, some recently published systematic reviews focusing on FC (Galindo- Domínguez, 2021; Hew et al., 2021; Oudbier et al., 2022; Prieto et al.,2021) note that a significant part of the studies published so far are not sufficiently rigorous in terms of their methods of analysis. In addition, they point out the need to increase the number of studies on the effectiveness of FC in the university context, to better specify more of the variables that influence the efficacy of FC or to compare the effectiveness of a specific intervention over time with specific groups. Therefore, our work consists of a systematic review of experimental or quasi-experimental studies focused on the university setting with the aim of gathering empirical evidence regarding the effectiveness of the application of FC, considering the following aspects: academic results, development of personal skills and competences, students’ assessment of FC and factors affecting its efficacy.

METHOD

The method used to achieve the proposed objectives was the systematic review of the published literature, following the indications of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement (Tricco et al., 2018) for this type of scientific review and its applications in the field of education (Sánchez-Serrano et al., 2022).

Search strategy, data sources and selection

The search for articles focused on three databases of scientific articles: Education Resources Information Center (ERIC), Web of Science (WoS) and Scopus. The systematic search included the following keywords: flipped classroom; effectiveness; higher education; undergrad*, experimental design. Each of the databases was scanned using the strategy described in Figure 1:

Figure 1
Flow diagram

The search procedure was first followed at the end of 2020 and then repeated with the same method in June 2022 to update the results. In this second search, six new studies met the parameters described above. Studies published between 2016 and June 2022 and dedicated to the evaluation of FC experiences with an experimental or quasi-experimental design were identified.

Inclusion of the studies

Articles were included in the review if they met the following criteria:

• Evaluation of FC experiences.

• Publication in 2016 or later.

• Experimental or quasi-experimental design.

• Focus on higher education.

• Publication in peer-reviewed journals.

As the focus of this study is on experimental or quasi-experimental studies and given that this was included in the search filters, qualitative studies, descriptions of experiences, literature reviews, articles on educational methodology and grey literature were excluded from the review.

Selection of the studies

The articles were reviewed independently by pairs of research team members, based on the aforementioned inclusion and exclusion criteria, and on an assessment rubric designed ad hoc by consensus of the research team and containing information on the following: identification and type of article selected, year and country of publication, type of study (review, meta-analysis, experimental or quasi- experimental, non-experimental), results (positive, negative, neutral), conclusions and other aspects (advantages, disadvantages, other lines of research). Studies that raised concerns were discussed at a meeting of the research team, at which a consensus was reached on whether to accept them for review.

The searches according to the strategies described resulted in 71 articles, of which 27 were included in the present study, as shown in Table 1. Based on the above criteria, most of the papers excluded did not focus on FC.

Table 1

Accepted and rejected studies

Data extraction and results

The initial data extraction for each article was carried out by pairs of research team members and subsequently agreed upon in meetings of the entire research team, which initially consisted of eight members, one of whom subsequently left the team. The 71 studies selected were divided between pairs of researchers, following the order in which they were searched in databases and, within these, alphabetically by surname of the first author. Eighteen articles were assigned to each two-member group for analysis. No substantive or content criteria were used for the distribution of the reading and analysis work. The variables listed in Table 2 were included.

Table 2

Variables included in the data extraction

Based on the definitions provided in the works studied, we understand the efficacy of FC to be a) improvements in academic performance and b) improvements in the acquisition of personal skills and competences. Data were also collected on c) student satisfaction with the method when compared to traditional systems and d) factors affecting the efficacy of FC.

Improvements in academic performance and in the acquisition of personal skills and competences are extracted from quiz results, mid-term and final exam grades, and drop-out rates. In all cases, student satisfaction with FC is drawn from surveys and questionnaires developed for this purpose.

The level of evidence of the studies included in the review has been classified according to the guidelines of the Scottish Intercollegiate Guidelines Network (SIGN, 2019). The level of evidence can be found in the annexes.

RESULTS

This review aims to pool the evidence regarding the effectiveness of FC at university. Twenty-seven experimental or quasi-experimental studies were selected. The main data of the articles included are presented in the annexes.

The works analysed are in two main geographical areas. Asia accounts for more than half of the articles, with Turkey (3), Taiwan (3) and China (2) standing out. The United States (7) is the country with the most publications on experimental or quasi-experimental FC studies.

Regarding the fields of knowledge of the selected articles, education and health- related degrees are clearly in the majority, and to a lesser extent those related to the fields of science and technology. Some disciplines are under-represented in these publications, and there is a complete absence of articles devoted to the humanities (except for English as a foreign language).

In terms of the level of the studies, in general, experimental or quasi-experimental studies have been conducted on undergraduate students in 24 articles, while in only a few cases (only 3 articles, 11% of the total) has work been done with students at higher levels.

Twenty-two studies with a control group and five studies without a control group or without a specified one have been identified and included in this review. In most of the studies, the control group followed a traditional lecture-based teaching method. Almost half (9 out of 22) have done so by randomly separating students, following the usual experimental method, while the rest have applied a criterion of convenience, using in previously formed class or subject groups (quasi- experimental).

Some studies have explored the influence of socio-demographic variables on the effectiveness of FC. For example, Fuentes et al. (2020) found that older students had higher levels of efficacy with this methodology and that having an occupation was associated with lower levels of FC efficacy. These authors (Fuentes et al., 2020) also point out that adequate values of family environment, motivation, self-esteem and autonomy were associated with higher levels of FC efficacy.

The results related to the aim of the review are presented below. These results are classified into a) assessment of efficacy (academic results and personal skills and competences), b) students’ assessment and c) conditioning factors affecting the efficacy of FC.

Assessment of efficacy

For the assessment of efficacy, we present the results from two perspectives: on one hand, the improvement of academic results and, on the other, the improvementof the acquisition of personal skills and competences. Like Fuentes et al. (2020), we present the results based on the improvement in the marks obtained in the assessment tests and the acquisition of personal skills and competences such as the ability to work collaboratively, student participation (in face-to-face and online sessions) or improved problem-solving skills. Table 3 summarises the results of the different studies in relation to the efficacy of FC from the two perspectives.

Table 3

Assessment of FC efficacy in the studies according to the effect on academic results and on the acquisition of personal skills and competences

Regarding the effect on the evaluation test scores, 20 studies (Aksoy & Pasli, 2022; Dong et al., 2021; El Sadik & Abdulmonem, 2021; Fanguy et al., 2017; Fuentes et al., 2020; Goh & Ong, 2019; Guo, 2019; Hava & Gelibolu, 2018; Huang et al., 2020; Hung, 2017; Khan et al., 2022; Leis & Brown, 2018; Loveys & Riggs, 2019; Maheshwari & Seth, 2019; Saglam & Arslan, 2018; Sezer & Abay, 2018; Sun & Wu, 2016; Webb & Doman, 2016; Wilton et al., 2019; Wozny et al., 2018) of the 27 analysed report an improvement in academic results in the experimental group. Five other studies (Afzal & Masroor, 2019; Cabi, 2018; Craft & Linask, 2020; Hung, 2017; Khan et al., 2022) report a neutral effect of FC compared to the traditional classroom. Campbell et al. (2022) find in their longitudinal research study in mathematics that the pass rate in the subject increases, but the average performance on the assessment test decreases with the introduction of FC. Goh and Ong (2019) find that FC is more beneficial in terms of academic performance for students with low performance in previous years. In contrast, the results of the study by Wozny et al. (2018) indicate that FC had a positive impact on mid-term assessments, and that this impact was greater for students with above-average academic achievements. Moreover, in the final (long-term) evaluations, the impact was only sustained for students with above-average marks. FC is also found to improve understanding of content in five studies (El Sadik & Abdulmonem, 2021; Khan et al., 2022; Maheshwari & Seth, 2019; Shaari et al., 2021; Webb & Doman, 2016). Regarding the sustainability of the results obtained, the authors of two studies (Craft & Linask, 2020; Wozny et al., 2018) note that FC influences the achievement of better results in the medium term, but not in the long term, while one study (Webb & Doman, 2016) shows that the results are only maintained in the long term in the control group. Table 4 summarises the results with respect to academic performance.

Table 4

Results of the studies with regard to FC and academic performance

In addition to academic performance, FC is associated with higher and better development of generic skills and competences (Guo, 2019; Khan et al., 2022). Table 5 presents the findings in relation to this aspect.

Table 5

Results of the studies in relation to the acquisition of personal skills and competences from FC

Regarding the increase in collaborative work, one study considers that FC represents a qualitative improvement (Sun & Wu, 2016) and another a quantitative improvement (Guo, 2019) in student interactions. Also, Maheshwari and Seth (2019) find that students in the experimental group are more engaged in their own learning. The same study considers that the interactive and more practical activities carried out in FC (participation in debates, classroom activities, application of practical concepts based on cases) lead to higher class attendance. Finally, three studies (Khan et al., 2022; Maheshwari & Seth, 2019; Saglam & Arslan, 2018) conclude that, from the perspective of students, they are more motivated to attend class and learn. From the teachers’ perspective, FC favours collaborative learning, in particular group work and problem solving (Maheshwari & Seth, 2019). In FC, the teaching is more dynamic and participative and doubts are more easily clarified, facilitating the understanding and learning of the subject (Maheshwari & Seth, 2019). According to two of the studies reviewed (Maheshwari & Seth, 2019; Sezer & Abay, 2018), this results in increased interaction with teachers.

Students’ assessment

Table 6 presents the results of students’ satisfaction with FC. Twelve studies report a high level of satisfaction. Notably, in Goh and Ong’s (2019) study, more than two-thirds of students in the experimental group responded that FC is very interactive and motivating; however, 40% still preferred the traditional method. Along these lines, in a quasi-experimental study (Afzal & Masroor, 2019) in which no significant improvement was found in the results of the students taught with FC, an increase in the acceptance and appreciation of FC was observed. Finally, another study (Cabi, 2018) concludes that negative evaluations towards FC are due to a lack of clarification of the topics to be prepared before class, a lack of resources and problems students have concentrating outside the classroom to work autonomously, leading to a preference for a traditional type of teaching. Regarding the assessment of student workload, in a quasi-experimental study, Canelas et al. (2017) report no significant differences between FC and the traditional method.

Table 6

Results of students’ satisfaction with FC

Conditions affecting the efficacy of FC

The reviewed studies agree that to be able to apply FC effectively, a series of conditions must be met by students, teachers and institutions.

Regarding the students, appropriate learning behaviour is necessary and dependent on the willingness and readiness of the students to accept this methodology (Shaari et al., 2021). Goh and Ong (2019) point out that it is necessary to implement FC gradually to facilitate students’ adaptation. On the other hand, Maheshwari and Seth’s (2019) study indicates that students need self-discipline, which is difficult at the beginning but is acquired over time. Time commitment is also higher: Craft and Linask (2020) compared FC with the traditional method, and found that its use has a statistically significant and positive effect on the number of hours students spend on the subject (3.1 hours per week more than their peers in traditional classes). Cabi’s (2018) study found that, in the experimental group, students previously spent between 1 and 2 hours outside the classroom studying the proposed topics. This effort and commitment of more time to prepare the background material in the classroom is also noted by Maheshwari and Seth (2019). As for teachers, Maheshwari and Seth (2019) consider that the success of FC lies in their ability to create a learning climate for students by managing possible shortcomings and dedicating time, effort and innovation to them. The study by Hava and Gelibolu (2018) highlights that it is essential for the teacher to explain the objectives and activities of FC at the beginning of the term, as students may show resistance to new methods or activities or, as Cabi (2018) explains, have difficulties in relating the content to the area of knowledge they are studying. Lack of motivation due to the effort involved in FC and being overwhelmed or bored are difficulties that need to be considered. For this reason, to prevent distraction and boredom, the reviewed studies highlight the importance of planning well and choosing activities that are entertaining and meaningful. Hava and Gelibolu (2018) recommend engaging students’ interest by using audio-visual material as a teaching resource instead of more traditional PowerPoint presentations. Sezer and Abay (2018) point to the desirability of using methods to encourage students to visualise game-based materials before coming to class. The success of FC, then, will depend not only on the technological tools used but also on how they are implemented. (Hung, 2017). Shaari et al. (2021) regard highly several advantages of materials that are prepared specifically for FC. They allow for increased student interest and motivation and the development of curiosity and critical thinking by helping students relate new learning to prior knowledge. Another study (Sezer & Abay, 2018) explains that effective planning is required before using this methodology. It is necessary to ensure the quality of the materials used, the technical problems that may arise, and the different capacities of teachers to implement the methodology and manage the process effectively. Students’ access to the materials must also be ensured (Craft & Linask, 2020) and, according to Cabi (2018), the appropriateness of the content to be studied (level of difficulty, amount of content, availability of resources, etc.) must be foreseen. It also requires content to be tailored to real-life problems and organised according to students’ learning needs (Cabi, 2018).

In relation to the teaching load, two articles (Dong et al., 2021; Fuentes et al., 2020) state that FC implies an increase in teacher dedication, as it requires a double effort: preparing before class and then conducting face-to-face classes. Finally, Craft and Linask (2020) stress that assessment formats must be considered. If students learn through problem solving in the classroom and are then tested in a different format, this may limit the results that are assessed.

Regarding institutions, two articles (Maheshwari & Seth, 2019; Sezer & Abay, 2018) explain that schools need to provide adequate resources and infrastructure to be able to implement FC and to support the teachers who implement it.

DISCUSSION

The purpose of this article is to provide evidence on the efficacy of FC at the university level. From a strictly descriptive point of view of the articles selected for this review, it is worth noting that there have been few experimental or quasi- experimental studies in European countries. In contrast, this methodology has been widely studied in Asian countries. These results are partly in line with another recent study by Al Mamun et al. (2022), which finds a high number of studies and scientific publications related to the use of FC in higher education in the USA, Taiwan and Australia.

As for the fields of knowledge in which the application of FC has been studied, education, health, science and technology predominate. One question is whether there are no published studies on the application of FC in the social sciences and humanities, or whether the methodology is not generally applied in these fields. It is also unclear whether studies have not been published because FC has not been implemented or whether unsuccessful attempts have been made and the results have not been published because they were unsatisfactory. This latter scenario has been noted as a possible research bias in some of the papers studied (Talan & Batdi, 2020; Xu et al., 2019).

With respect to the methodological rigour of the studies included in this review, a significant number show deficits in the information provided (e.g., more detailed data on students such as gender and age, methodological aspects, etc.). This aspect is also mentioned in other reviews and meta-analyses such as Chen et al. (2018) and Conte et al. (2021). Regarding the level of scientific evidence regarding the methodology of the studies, even though they are all experimental or quasi- experimental, they are classified as medium-low. This assessment is in line with the results obtained by Barranquero-Herbosa et al. (2022) in a recent systematic review on the application of FC in nursing studies. They conclude that the methodological rigour of the studies carried out is medium-low.

Regarding the academic results of students who learn using FC, it is widely acknowledged by the results of numerous studies on the subject that the academic results of these students are better than those of students who follow other learning methodologies (Barranquero-Herbosa et al., 2022; Chen et al., 2018; Conte et al., 2021; Evans et al., 2019; Ge et al., 2020; Lo & Hew, 2019; Martínez et al., 2019; Prieto et al., 2021; Shi et al., 2020; Sisi Li et al., 2020; Talan & Batdi, 2020; Xu et al., 2019). These studies reach similar conclusions regarding the improvement of academic performance. The experimental group experiences an overall positive effect compared to the control group. However, they also note that there are some studies with neutral results — in line with the studies in this systematic review by Afzal and Masroor (2019), Cabi (2018), and Craft and Linask (2020) — and, in a minority of cases, the results favour the control group. Along these lines, Chen et al. (2018) conducted a systematic review on the efficacy of FC in which they found favourable results for experimental groups only in quasi-experimental cohort studies; in studies with randomised groups, no improvements in academic performance were observed. Another longitudinal study (Maya, et al. 2021) following the same subject with different groups of students finds that after the application of FC for several years, students’ academic results improve in relation to the pass rate, from 88% to 100%.

Numerous studies included in this review highlight the development of students’ personal skills and competences as a result of the application of FC. In this regard, Brewer and Movahedazarhouligh (2018) credit the application of FC with enhancing the acquisition of 21st century skills¹, which include lifelong learning. Sousa et al. (2021) also attribute to FC good results in soft skills (personal, social and communication skills), which are in demand by employers, according to Robles (2012). Other published studies provide evidence on other specific personal skills, such as active participation in the classroom (Bao-Zhu Li et al., 2020; Ge et al., 2020; Talan & Batdi, 2020; Xu et al., 2019); self-learning skills (Barranquero-Herbosa et al., 2022; Sisi Li et al., 2020; Talan & Batdi, 2020; Turan & Akdag-Cimen, 2019); problem- solving and creative-thinking skills (Bao-Zhu Li et al., 2020; Ge et al., 2020; Sisi Li et al., 2020; Talan & Batdi, 2020; Xu et al., 2019); teamwork (Bao-Zhu Li et al., 2020; Ge et al., 2020; Sisi Li et al., 2020; Xu et al., 2019); communication skills² (Bao-Zhu Li et al., 2020; Ge et al., 2020; Sisi Li et al., 2020; Talan & Batdi, 2020; Turan & Akdag-Cimen, 2019; Xu et al., 2019); the ability to self-manage time (Talan & Batdi, 2020); and increased confidence (Barranquero-Herbosa et al., 2022). Other skills not been found in this review are found in the literature, such as the ability to overcome fear (Talan & Batdi, 2020), class attendance (Kumar et al., 2017; Talan & Batdi, 2020), resilience (Bao-Zhu Li et al., 2020; Ge et al., 2020; Xu et al., 2019) and ICT proficiency (Turan & Akdag-Cimen, 2019).

Along the same lines, and from the student's perspective, there is greater motivation, supported by studies such as those of Romero-García et al. (2021), Talan and Batdi (2020) and Turan and Akdag-Cimen (2019). In his review of the literature, Prieto (2021) calculated the size of the effect in the meta-analysis by Zheng et al. (2020): a 24% percentile increase in the level of students’ motivation to learn in FC environments. Studies included in this review by Maheshwari and Seth (2019) and Sezer and Abay (2018) highlight the greater interaction with teachers using FC than with other methodologies. In the same vein, other authors (Brewer & Movahedazarhouligh, 2018; Sisi Li et al., 2020; Turan & Akdag-Cimen, 2019) conclude that students feel more engaged³ with the subject, the teaching staff⁴ and their peers. In summary, the results of our review agree, as mentioned by Asksoy and Pasli (2022), that FC is a promising, learner-centred didactic approach.

On the other hand, research studies report varied and sometimes conflicting experiences from the perspective of students who have been taught using FC. In general, despite the drawbacks of FC, most of the reviewed literature (Barranquero- Herbosa et al., 2022; Kumar et al., 2017; Prieto et al., 2021; Sisi Li et al., 2020; Talan & Batdi, 2020; Turan & Akdag-Cimen, 2019) concludes that students prefer FC to other more lecture-based methodologies, although some research (Brewer & Movahedazarhouligh, 2018) indicates that results vary in this regard. In line with the results found in this research, the following negative aspects (from the students’ point of view) were also highlighted: firstly, according to Barranquero-Herbosa et al. (2022) and Sisi Li et al. (2020), learners have to carry a heavier workload than in traditional classrooms; secondly, according to Brewer and Movahedazarhouligh (2018), some learners are uncomfortable with FC because they are used to traditional classes; and finally, according to the results obtained by Barranquero-Herbosa et al. (2022), more technology is needed to apply FC and to motivate students through the use of narrative and gamification strategies to change their study habits with FC (Prieto et al., 2014a).

Cabi (2018) and Hew and Lo (2018) note that content planning is critical to academic improvement and conclude that the best results are obtained when the teacher provides a short review of students’ previous homework at the beginning of the face-to-face lesson. However, Conte et al. (2021) consider that if supplementary online materials were made available, results like FC could be obtained in traditional classes. In the meta-analysis by Shi et al. (2020), some variables concerning the implementation of FC in relation to its effectiveness are also controlled and it is observed that FC is more effective when teachers integrate active and collaborative individualised pedagogical approaches, namely team learning. On the other hand, the results of a longitudinal study (Campbell et al., 2022) conducted with a sample of university students conclude that the effectiveness of FC may not be fully observed, especially in the early stages of implementation, and a certain amount of implementation time is required before a positive change in student performance is observed.

The limitations of this review include the heterogeneity of the educational, geographical, cultural and linguistic environments in which experimental or quasi- experimental FC studies are conducted. This fact makes it difficult to establish uniformity in establishing common criteria to ensure the efficacy of the application of this methodology. And many studies do not provide data on the type of FC carried out. Another limitation is the lack of methodological rigour in quasi-experimental studies, as they do not use randomised groups and therefore do not provide a high level of evidence in relation to the type of methodology according to the SIGN classification (2019).

FUTURE LINES OF RESEARCH

Although FC has been proven to be effective, certain aspects have not yet been validated. Future lines of research could explore how the acquired skills are maintained over time and applied outside the classroom (Canelas et al., 2017; Wilton et al., 2019). Secondly, it is necessary to explore which active methodologies have a greater impact on the effectiveness of FC. Finally, it is necessary to analyse how to further study the impact of FC in terms of socio-demographic and academic variables such as gender (Craft & Linask, 2020; Saglam & Arslan, 2018), academic achievements, previous knowledge and timetable planning, among others (Craft & Linask, 2020). The effects of FC on other university disciplines that have been studied less (especially in the humanities) and non-university disciplines (primary and secondary education) could also be investigated (Saglam & Arslan, 2018).

CONCLUSIONS

After the analysis of the evidence on the effectiveness of the application of FC in the university setting, it can be concluded that the objectives of this systematic review, which studies the effectiveness of FC, have been achieved in relation to the following aspects:

• Academic results. FC is shown to improve academic outcomes (specifically in comparison with lecture-based teaching methodologies).

• The development of personal skills and competences related to transversal competences. FC contributes to the acquisition of personal skills related to the competences recognised as transversal in the university environment and, therefore, contributes to the comprehensive training of students.

• The students’ assessment. Most students give a positive assessment of FC.

Finally, it should be noted that both the characteristics of the students (motivation, self-discipline and dedication) and of the teachers (time, motivation for innovation and the use of quality audio-visual material), as well as those of the academic institutions (adequate infrastructure and sufficient resources) condition the success of the application of this methodology.

These conclusions should be contextualised by considering that the level of evidence found is assessed as medium-low. On the other hand, experimental studies with larger, randomised, control group samples are needed to increase the evidence currently available for the application of FC at university level. Despite this, the application of FC is considered to be a good option in university education, as it offers good results for students and teachers, achieving more meaningful learning accompanied by the development of transversal skills and competences needed in today’s labour market.

REFERENCES

Afzal, S., & Masroor, I. (2019). Flipped classroom model for teaching undergraduate students in radiology. Journal of the College of Physicians and Surgeons-Pakistan, 29(11), 1083-1086. https://doi.org/10.29271/JCPSP.2019.11.1083

Aksoy, B., & Pasli, E. (2022). Examining effects of the flipped classroom approach on motivation, learning strategies, urinary system knowledge, and urinary catheterization skills of first-year nursing students. Japan Journal of Nursing Science, 19, e12469. https://doi.org/10.1111/jjns.12469

Al Mamun, M. A., Azad, M. A. K., Al Mamun, M. A., & Boyle, M. (2022). Review of flipped learning in engineering education: scientific mapping and research horizon. Education and Information Technologies, 27, 1261-1286. https://doi.org/10.1007/s10639-021-10630-z

Barranquero-Herbosa, M., Abajas-Rustillo, R., & Ortego-Mate, C. (2022). Effectiveness of flipped classroom in nursing education. A systematic review of systematic and integrative reviews. International Journal of Nursing Studies, 21(135), 104327. https://doi.org/10.1016/j.ijnurstu.2022.104327

Bego, C. R., Ralston, P. A. S., & Knight, A. (2022). Improving performance in a arge flipped barrier mathematics course: a longitudinal case study. International Journal of Mathematical Education in Science and Technology, 53(7), 1916-1933. https://doi.org/10.1080/0020739X.2020.1850899

Bergmann, J., & Sams, A. (2012). Flip your classroom: reach every student in every class every day. International Society for Technology in Education.

Bok D. (2017). The struggle to reform our colleges. Princeton New Jersey Princeton University press.

Bredow, C. A., Roehling, P. V., Knorp, A. J., & Sweet, A. M. (2021). To flip or not to flip? A meta-analysis of the efficacy of flipped learning in higher education. Review of Educational Research, 91(6), 878-918. https://doi.org/10.3102/00346543211019122

Brewer, R., & Movahedazarhouligh, S. (2018). Successful stories and conflicts: a literature review on the effectiveness of flipped learning in higher education. Journal of Computer Assisted Learning, 34(4), 409-416. https://doi.org/10.1111/jcal.12250

Cabi, E. (2018). The impact of the flipped classroom model on students’ academic achievement. International Review of Research in Open and Distributed Learning, 19(3), 202-221

Canelas, D. A., Hill, J. L., & Novicki, A. (2017). Cooperative learning in organic chemistry increases student assessment of learning gains in key transferable skills. Chemistry Education Research and Practice, 18(3), 441-456. https://doi.org/10.1039/C7RP00014F

Chen, H. R., & Hsu, W. C. (2022). Do flipped learning and adaptive instruction improve student learning outcome? A Case study of a computer programming course in Taiwan. Frontiers in Psychology, 14(12), 768183. https://doi.org/10.3389/fpsyg.2021.768183

Chen, K. S., Monrouxe, L., Lu, Y. H., Jenq, C. C., Chang, Y. J., Chang, Y. C., & Chai, P. C. C. (2018). Academic outcomes of flipped classroom learning: a meta-analysis. Medical Education, 52(9), 910-924. https://doi.org/10.1111/MEDU.13616

Conte, D. B., Zancanaro, M., Guollo, A., Schneider, L. R., Lund, R. G., & Rodrigues-Junior, S. A. (2021). Educational interventions to improve dental anatomy carving ability of dental students: a systematic review. Anatomical Sciences Education, 14(1), 99-109. https://doi.org/10.1002/ase.2004

Craft, E., & Linask, M. (2020). Learning effects of the flipped classroom in a principles of microeconomics course. Journal of Economic Education, 51(1), 1-18. https://doi.org/10.1080/00220485.2019.1687372

Doğan, Y., Batdı, V., & Yaşar, M. D. (2021). Effectiveness of flipped classroom practices in teaching of science: a mixed research synthesis. Research in Science & Technological Education, 41(1) 393-421. https://doi.org/10.1080/02635143.2021.1909553

Dong, Y., Yin, H., Du, S., & Wang, A. (2021). The effects of flipped classroom characterized by situational and collaborative learning in a community nursing course: a quasi-experimental design. Nurse Education Today, 10, 105037. https://doi.org/10.1016/j.nedt.2021.105037

El Sadik, A., & Abdulmonem, W. (2021). Improvement in student performance and perceptions through a flipped anatomy classroom: shifting from passive traditional to active blended learning. Anatomical Sciences Education, 14(4), 482-490. https://doi.org/10.1002/ase.2015

Evans, L., Bosch, M. L. V., Harrington, S., Schoofs, N., & Coviak, C. (2019). Flipping the classroom in health care higher education: a systematic review. Nurse Educator, 44(2), 74-78.https://doi.org/10.1097/nne.0000000000000554

Fanguy, M., Costley, J., & Baldwin, M. (2017). Pinch hitter: the effectiveness of content summaries delivered by a guest lecturer in online course videos. International Review of Research in Open and Distance Learning, 18(7), 242-266. https://doi.org/10.19173/irrodl.v18i7.3208

Fuentes, A., López, J., Parra, M. E., & Morales, M. B. (2020). Diseño, validación y aplicación de un cuestionario para medir la influencia de factores exógenos sobre la eficacia del aprendizaje invertido. Psychology, Society, & Education, 12(1), 1-16. Educational Technology & Society, 24(3), 44-60.

Ge, L., Chen, Y., Yan, C., Chen, Z., & Liu, J. (2020). Effectiveness of flipped classroom vs traditional lectures in radiology education: a meta-analysis. Medicine, 99(40), e22430. https://doi.org/10.1097%2FMD.0000000000022430

Goh, C. F., & Ong, E. T. (2019). Flipped classroom as an effective approach in enhancing student learning of a pharmacy course with a historically low student pass rate. Currents in Pharmacy Teaching and Learning, 11(6), 621-629. https://doi.org/10.1016/J.CPTL.2019.02.025

Guo, J. (2019). The use of an extended flipped classroom model in improving students’ learning in an undergraduate course. Journal of Computing in Higher Education, 31(2), 362-390. https://doi.org/10.1007/s12528-019-09224-z

Hava, K., & Gelibolu, M. F. (2018). The impact of digital citizenship instruction through flipped classroom model on various variables. Contemporary Educational Technology, 9(4), 390-404. https://doi.org/10.30935/CET.471013

Hew, K. F., Bai, S., Huang, W., Dawson, P., Du, J., Huang, G., Jia, C., & Thankrit, K. (2021). On the use of flipped classroom across various disciplines: insights from a second-order meta-analysis. Australasian Journal of Educational Technology, 37(2), 132-151. https://doi.org/10.14742/ajet.6475

Hew, K. F., & Lo, C. K. (2018). Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Medical Education, 18(38), 1-12. https://doi.org/10.1186/s12909-018-1144-z

Huang, H. L., Chou, C. P., Leu, S., You, H. L., Tiao, M. M., & Chen, C. H. (2020). Effects of a quasi-experimental study of using flipped classroom approach to teach evidence-based medicine to medical technology students. BMC Medical Education, 20(1), 31. https://doi.org/10.1186/S12909-020-1946-7

Hung, H. T. (2017). Language teaching and technology forum: the integration of a student response system in flipped classrooms. Language Learning & Technology, 21(1), 16-27.

Khan, U., Al Naymat, G., Ayoubi, R., Mustafa, M., & Hussain, H. (2022). Gamified flippend classroom learning: Which approach is more efficient in business educacion? The International Journal of Management Education, 20, 100595.

Kumar, S., McLean, L., Nash, L., & Trigwell, K. (2017). Incorporating active learning in psychiatry education. Australasian Psychiatry: Bulletin of Royal Australian and New Zealand College of Psychiatrists, 25(3), 304-309. https://doi.org/10.1177/1039856217689912

Lai, H., Hsiao, Y. L., & Hsieh P. J. (2018). The role of motivation ability, and opportunity in university teacher’ continuance use intention for flipped teaching. Computers & Education, 124, 37-50.

Leis, A., & Brown, K. (2018). Flipped learning in an EFL environment: does the teacher’s experience affect learning outcomes? The EuroCALL Review, 26(1), 3-13. https://doi.org/10.4995/eurocall.2018.8597

Li, B. Z., Cao, N. W., Ren, C. X., Chu, X. J., Zhou, H. Y., & Guo, B. (2020). Flipped classroom improves nursing students’ theoretical learning in China: a metaanalysis. Plos One, 15(8), Artículo e0237926. https://doi.org/10.1371/journal.pone.0237926

Li, S., Liao, X., Burdick, W., & Tong, K. (2020). The effectiveness of flipped classroom in health professions education in China: a systematic review. Journal of Medical Education and Curricular Development, 238212052096283. https://doi.org/10.1177/2382120520962838

Lin, H. C., Hwang, G. J., Chang, S. C., & Hsu, Y. D. (2021). Facilitating critical thinking in decision making-based professional training: An online interactive peer-review approach in a flipped learning context. Computers & Education, 173, 104266. https://doi.org/10.1016/j.compedu.2021.104266

Lo, C. K., & Hew, K. F. (2019). The impact of flipped classrooms on student achievement in engineering education: a meta-analysis of 10 years of research. Journal of Engineering Education, 108(4), 523-546. https://doi.org/10.1002/JEE.20293

Loveys, B. R., & Riggs, K. M. (2019). Flipping the laboratory: improving student engagement and learning outcomes in second year science courses. International Journal of Science Education, 41(1), 64-79. https://doi.org/10.1080/09500693.2018.1533663

Maheshwari, P., & Seth, N. (2019). Effectiveness of flipped classrooms: A case of management education in central India. International Journal of Educational Management, 33(5), 860-885. https://doi.org/10.1108/IJEM-10-2017-0282

Manoj, K. P., Renumol V. G., & Sahana, M. (19-22 de april de 2018). Flipped classroom strategy to help underachievers in Java programming [Ponencia de Congreso]. 2018 International Conference on Learning and Teaching in Computing and Engineering, New Zealand. https://doi.org/10.1109/LaTICE.2018.000-7

Martínez, T. S., Díaz, I. A., Rodríguez, J. M. R., & Rodríguez-García, A. M. (2019). Efficacy of the flipped classroom method at the university: meta-analysis of impact scientific production. REICE. Revista Iberoamericana Sobre Calidad, Eficacia y Cambio En Educación, 17(1), 25-38. https://doi.org/10.15366/REICE2019.17.1.002

Maya, C., Iglesias, J., & Giménez, X. (2021). Clase investida síncrona en asignaturas STEM. Revista de Educación, 391, 15-39

Mazur, E. (1997). Peer instruction: a user’s manual. Prentice Hall.

McLaughlin, J. E., Roth, M. T., Glatt, D. M., Gharkholonarehe, N., Davidson, C. A., Griffin, L. M., Esserman, D. A., & Mumper, R. J. (2014). The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Academic Medicine, 89(2), 236-243. https://doi.org/10.1097/acm.0000000000000086

Medina, J. L., Jarauta, B., & Imbernon, F. (2010). La enseñanza reflexiva en la educación superior. Octaedro.

Michaelsen, L., Bauman, A., & Dee, L. (2002). Team-based learning: a transformative use of small groups. Praeger Publishers.

Novak, G., Gavrin, A., Wolfgang, C., & Patterson, E. (1999). Just-in-time teaching: blending active learning with web technology. Pearson.

Oudbier, J., Spaai, G., Timmermans, K., & Boerboom, T. (2022). Enhancing the effectiveness of flipped classroom in health science education: a state-of- the-art review. BMC Medical Education, 22(1), 1-15. https://doi.org/10.1186/s12909021-03052-5

Özdemir, A., & Şentürk, M. L. (2021). The effect of flipped classroom model on students’ academic achievement in science and mathematics education: a meta-analysis study. Journal of Educational Technology, 18(3), 22-41. https://doi.org/10.26634/jet.18.3.18071

Prieto, A., Díaz, D., Monserrat, J., & Reyes, E. (2014). Experiencias de aplicación de estrategias de gamificación a entornos de aprendizaje universitario. ReVisión. Revista de Investigación en Docencia Universitaria de la Informática, 7 (2), 76-92.

Prieto, A., Diaz, D., & Santiago, R. (2014). Metodologías Inductivas: el desafío de enseñar mediante el cuestionamiento y los retos. Digital Text.

Prieto, A., Barbarroja, J., Corell, A., & Álvarez, S. (2021). Eficacia del modelo de aula invertida (flipped classroom) en la enseñanza universitaria: una síntesis de las mejores evidencias. Revista de Educación, 391, 149-177. https://doi.org/10.4438/1988-592X-RE-2021-391-476

Robles, M. (2012). Executive perceptions of the top 10 soft skills needed in today’s workplace. Business Communication Quarterly, 75, 453-465. https://doi.org/10.1177/1080569912460400

Romero-García, C., Paz-Lugo, P., Buzón-García, O., & Navarro-Asencio, E. (2021). Evaluación de una formación online basada en Flippend Classroom. Revista de Educación, 391, 65-88.

Saglam, D., & Arslan, A. (2018). The effect of flipped classroom on the academic achievement and attitude of higher education students. World Journal of Education, 8(4), 170. https://doi.org/10.5430/WJE.V8N4P170

Sánchez-Serrano, S., Pedraza-Navarro, I., & Donoso-González, M. (2022). ¿Cómo hacer una revisión sistemática siguiendo el protocolo PRISMA? Usos y estrategias fundamentales para su aplicación en el ámbito educativo a través de un caso práctico. Bordón: Revista De Pedagogía, 74(3), 51-66. https://doi.org/10.13042/Bordon.2022.95090

Scottish Intercollegiate Guidelines Network. (2019). A guideline developer’s handbook. SIGN.

Senali, M. G, Iranmanesh, M., Ghobakhloo, M., Gengatharen,D., Tseng, M. L., & Nilsashi, M. (2022). Flipped classroom in business and entrepreneurship education: a systematic review and future research agenda. The International Journal of Management Education, 20(1), 100614. https://doi.org/10.1016/j.ijme.2022.100614

Sezer, B., & Abay, E. (2018). Looking at the impact of the flipped classroom model. Medical Education, 63(6), 853-868. https://doi.org/10.1080/00313831.2018.1452292

Shaari, N. D., Shaari, A. H., & Abdullah, M. R. (2021). Investigating the impact of flipped classroom on dual language learners’ perceptions and grammatical performance. Studies in English Language and Education, 8(2), 69-70. https://doi.org/10.24815/siele.v8i2.18872

Shi, Y., Ma, Y., MacLeod, J., & Yang, H. H. (2020). College students’ cognitive learning outcomes in flipped classroom instruction: a meta-analysis of the empirical literature. Journal of Computers in Education, 7(1), 79-103. https://doi.org/10.1007/S40692-019-00142-8

Sousa, S., Peset, M. J., & Muñoz-Sepúlveda, J. A. (2021). La enseñanza híbrida mediante flipped classroom en la educación superior. Revista de Educación, 391, 123-142

Sun, J. C. Y., & Wu, Y. T. (2016). Analysis of learning achievement and teacher- student interactions in flipped and conventional classrooms. International Review of Research in Open and Distance Learning, 17(1), 79-99. https://doi.org/10.19173/irrodl.v17i1.2116

Talan, D. T., & Batdi, D. V. (2020). Evaluating the flipped classroom model through the multi-complementary approach. Turkish Online Journal of Distance Education, 21(4), 31-67. https://doi.org/10.17718/TOJDE.803351

Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., … & Straus, S. E. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Annals of Internal Medicine, 169(7), 467-473. https://doi.org/10.7326/m18-0850

Tourón, J. (2021). El modelo flipped classroom: un reto para una enseñanza centrada en el alumno. Revista de Educación, 391,11-13.

Turan, Z., & Akdag-Cimen, B. (2019). Flipped classroom in English language teaching: a systematic review. Computer Assisted Language Learning, 33(5-6), 590-606. https://doi.org/10.1080/09588221.2019.1584117

Webb, M., & Doman, E. (2016). Does the flipped classroom lead to increased gains on learning outcomes in esl/efl contexts? The Catesol Journal, 28(1), 39-67.

Wilton, M., Gonzalez-Niño, E., McPartlan, P., Terner, Z., Christoffersen, R. E., & Rothman, J. H. (2019). Improving academic performance, belonging, and retention through increasing structure of an introductory biology course. CBE Life Sciences Education, 18(4), Artículo ar53. https://doi.org/10.1187/CBE.18-08-0155

Wozny, N., Balser, C., & Ives, D. (2018). Evaluating the flipped classroom: a randomized controlled trial. Research in Economic Education, 49(2), 115-129. https://doi.org/10.1080/00220485.2018.1438860

Xu, P., Chen, Y., Nie, W., Wang, Y., Song, T., Li, H., Li, J., Yi, J., & Zhao, L. (2019). The effectiveness of a flipped classroom on the development of Chinese nursing students’ skill competence: a systematic review and meta-analysis. Nurse Education Today, 80, 67-77. https://doi.org/10.1016/j.nedt.2019.06.005

Zhamanov, A., Yoo, S. M., Sakhiyeva, Z., & Zhaparov, M. (2018). Implementation and evaluation of flipped classroom as IoT element into learning process of computer network education. International Journal of Information and Communication Technology Education, 14(2), 30-47. https://doi.org/10.4018/IJICTE.2018040103

Zheng, L., Bhagat, K. K., Zhen, Y., & Zhang, X. (2020). The effectiveness of the flipped classroom on students’ learning achievement and learning motivation: a meta- analysis. Journal of Educational Technology & Society, 23(1), 1-15.

ANNEX 1

Summary of the main characteristics and level of evidence from experimental studies

ANNEX 2

Summary of the main characteristics and level of evidence from quasi-experimental studies

Notes

Información adicional

How to reference this article: Bosch-Farré, C., Cicres, J., Patiño-Masó, J., Morera Basuldo, P., Toran-Monserrat, P., Lladó Martínez, A. & Malagón- Aguilera, M. C. (2024). Effectiveness of the flipped classroom methodology in higher education: A systematic review. Educación XX1, 27(1), 19-56. https://doi.org/10.5944/educxx1.35773