HTML

ePub

PDF

Introduction

In a highly mediated society, videogames have played an increasingly relevant role in the daily lives of individuals. This provides an explanation for the growing research interest on this medium. However, studies on this topic have systematically adopted a perspective focused on effects theories and, in particular, on the negative potential of videogames, in the behavioral and emotional fields. The scientific value of the body of research regarding the influence of videogames on aggressive behavior cannot be ignored, but the multiple questions arising from the systematic and/or continuous use of videogames should also be analyzed, to consider possible positive outcomes (Granic, Lobel, & Engels, 2014).

Videogames and Learning

Even though, the specific learning outcomes of videogames are not studied so extensively, the pedagogical value of Play has been broadly documented and studied since the foundations of Psychology. Evolutionary Psychology, for instance, considers Play an adaptive strategy, that allows humans to easily generate a repertoire of innovative behaviors, increasing exploration and cooperation (Pellegrini, Dupuis, & Smith, 2007). Also, Developmental Psychology, in its founding, recognizes Play as an important strategy to reproduce real-life conflicts, to achieve ideal solutions and to amend negative feelings (Piaget, 1962). The development of Social Cognition in children is also a process considered to be founded in Play experiences and its elements (Vygotsky, 1978).

A game can be broadly defined as “a system in which players engage in an artificial conflict, defined by rules, that results in a quantifiable outcome”. Its precise definition has raised a complex discussion between authors, in the field of game studies. Although there are multiple perspectives, it is possible to identify some common aspects between them, such as, the existence of: rules that orient and limit players; conflict or contest; a clear goal-orientation or outcome-orientation; activities, processes, or events that involve decision-making (Salen & Zimmerman, 2010).

In our framework, we adopt this concept, where the great majority of videogames, commercial or not, can be included. Therefore, it is considered that videogames are designed for players to actively engage with their systems and for these systems to provide feedback to players’ interactions. Videogames can require competition and/or cooperation and can be played in a wide range of platforms, from consoles to computers or smartphones, among others (Granic, Lobel, & Engels, 2014).

Regarding videogames and learning, an important distinction between videogames also arises. The dichotomy between the videogames specifically developed for learning and/or research purposes, and the learning outcomes of playing the so called commercial games. Considering a review of 52 studies, conducted between 2010 and 2016, only five studies approached the learning and motivational gains of commercial games (Costa, Tyner, Henriques, & Sousa, 2016).

The Pedagogical value of Videogames

The relevance of videogames in the learning and cognitive enrichment processes has been increasingly studied, although is a relatively new field. One of the most referred outcomes of playing videogames are the motivational aspects of gaming, based on the ability of videogames to provide immediate and concrete feedback of performance, balance optimal levels of challenge and frustration, and use of failure as a motivational tool for large-scale success (Granic, Lobel, & Engels, 2014). Research also reinforces the affective potential of videogames, considering their ability to induce more positive and powerful emotions than real life contexts (McGonigal, 2011), that increases exponentially if immersion factors are taking in account.

Videogames are considered an engaging media, which in itself could be a facilitating factor in the learning process. But in addition, its complexity requires the coordination of a number of variables, such as understanding complex processes, managing diverse social networks (such as teams or clans), and creative expression with digital tools (Squire, 2008). A strong body of research supports long-lasting positive effects of videogames on basic cognitive elements, involved in the learning process, such as perception, attention, memory, and decision-making (Eichenbaum, Bavelier, & Green, 2014). Moreover, research also supports the potential of videogames to develop other cognitive aspects crucial to the learning process, particularly in formal educational settings, such as executive functioning or selective attention (Ashinoff, 2014). A growing body of evidence supports the integration of game analysis and production across the curriculum (de Freitas & Ott, 2013), however the adoption of videogames as a pedagogical resource in educational environments highly depends on the attitudes of all stakeholders towards their educational potential. Regarding this, there is a body of research supporting the existence of positive attitudes towards the pedagogical value of videogames from main stakeholders in Education and particularly in formal school (Saéz-López, Vázquez-Cano, & Domínguez-Garrido, 2015; Sousa, Henriques, & Costa, 2017).

Meta-analysis in Education and Learning

A meta-analysis is a secondary study, which sets up a process of statistical data summarization for a Systematic Review (SR). It aims to bridge the gap between scientific research and policy through systematic synthesis of research findings (Sanchéz-Meca & Marín-Martínez, 2010). Consequently, a meta-analysis study provides knowledge base for policy-makers and practitioners, helps to identify knowledge gaps and prevailing degrees of uncertainty, and promotes the cumulative development of science (Sutton, 2000). The reliability of each study is based on the level of evidence attributed to each research design, with systematic reviews and meta-analysis studies at the top level, when considering the hierarchy of Scientific Evidence (Ackley, Ladwig, Swan & Tucker, 2007), represented in Figure 1.

Typically, meta-analysis studies have gained prominence because of their relevance in decision-making, in the context of clinic and health practice. However, currently this type of research design has gained relevance in the social and educational areas, as it allows the sustained reflection on intervention strategies. Between 1976 and 2011, a review of databases found a total of 5206 publications in the field of education, using meta-analysis as a research design (Ahn, Ames, & Myers, 2012).

The present study aims to summarize the learning improvements obtained with GBL approaches in RCTs, adopting a meta-analysis procedure. This objective materializes to different research goals, it intends to provide data on the efficacy of GBL interventions and it intends, in a certain extent, to legitimate the role videogames can assume in the learning process, and their pedagogical value.

Method Sample and Selection Process

The process of gathering data begun with a schematic database search, including Ebsco, PubMed, ACM and ResearchGate, by using a specific advanced search formula: [game-based learning AND (randomized controlled trial OR RCT) AND (videogames OR digital games)]. ResearchGate was also used as a data collection tool, to access Grey Literature. Grey Literature is usually defined as literature “which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers” (Farace, 1998). This process constitutes the Identification phase.

The following phases were based in the selection criteria, developed to match, as much as possible, the research objective. First, papers should have an experimental research design, using Randomized Controlled Trials. Second, they should use digital GBL strategies with clear learning objectives. It was also mandatory for outcome measures to evaluate concrete learning gains and to compare them with a pedagogical approach without videogames. Analogical GBL approaches weren’t considered. Approaches based on gamification of the learning process were also excluded.

Using the above defined criteria, 68 scientific papers were obtained. The selection process throughout the final sample is represented in Figure 2. In the Screening phase, the criteria were applied at a superficial level, meaning that only titles, abstracts and general information were considered. In the Eligibility phase the criteria was followed by thoroughly analyzing each study, before select the final sample.

Procedure

The selected papers were analyzed considering two different matrices. First, some general aspects of the research were categorized, namely year of publication, sample size (N), context of the study, population, learning goals, outcome measures, type of videogame and platform. The gathered data was analyzed using the Statistical Package for the Social Sciences (IBM SPSS), version 22. Next, studies were analyzed considering the reported quantitative results, which were summarized using the Comprehensive Meta-Analysis Software (CMA), version 2.

Results

Fourteen studies matched the inclusion criteria. The present systematic review as a total sample of 1685 subjects, with an average of 120,35 individuals per included study. Samples in the studies ranged between 29 and 458 subjects.

Regarding the year of publication, it is possible to observe that all studies are from the last 10 years. Thus, it can be ensured that the reported data relate to the most recent scientific evidence and SR quality. The distribution by years was as follows: 2017 – 1 study (7,1 %), 2016 – 2 studies (14,3%), 2015 – 1 study (7,1 %), 2014 – 3 studies (21,4%), 2013 – 2 studies (14,3%), 2010 – 1 study (7,1 %),2009 – 2 studies (14,3%), 2008 – 1 study (7,1%) and 2007 – 1 study (7,1 %).

Probably because of the inclusion criteria above defined (RCT was mandatory), the most studied population were medical students (35.7%), an area where this research design is widely used. Despite this, there is a multiplicity of studied populations, namely children (21.4%), teenagers (21.4%), clinical populations (7.1%), tweens (7.1%) and engineering students (7, 1%). Consequently, the studies took place in many different contexts: Higher Education (42,9%), Basic Education (28,6%), Secondary Education (21,4%) and Clinical Rehabilitation (7,1%).

The game-based learning strategies implemented aimed the development of very different areas of competence, such as medical practices, health literacy, physics, computer science, mathematics, science, history, pedestrian safety and engineering principles. To measure the effective learning gains, the research designs included baseline and endline measures, namely, knowledge tests (71,4%), learning scales (21,4%) or observation protocols (7,1%).

The videogames were, mainly, classified by the researchers as educational games (28,6%) or serious games (28,6%), but there were also others classified as computer-based board games (14,3%), role-playing games (7,1%), simulation games (7,1%), quizzes (7,1%), virtual reality games (7,1%) or GPS/location based games (7,1%). The categories of analysis were those referred by the authors and, as such, are not mutually exclusive.

The used platform is usually the PC (57,1%), but are also used web-based games (28,6%), and games for smartphone (7,1%) or Nintendo Wii® (7,1%). Web-based games were considered separately, since they can have a multi-platform use (PC, smartphone, tablet, etc.).

In Study 1, results showed that medical students can improve their pediatric knowledge, using a web-based game (Sward, Richardson, Kendrick, & Maloney, 2008) (g = 0,283: CI = 95%; -0,109 - 0,676). Study 2 supported children’s knowledge acquisition in the field of health literacy, namely nutrition, by using an educational game (Rosi et al., 2016) (g = 0,138; CI = 95%; -0,168 - 0,443). In study 3, a e-learning system, with an integrated role-playing game, was effective to improve medical students’ knowledge about urology (Boeker, Andel, Vach, Frankenshmidt, 2013) (g = 0,692; CI = 95%; 0,355 - 1,029). Study 4 makes usage of a serious game to effectively increase medical students’ learning outcomes about patient safety (Dankbaar, Richters, & Kalkman, 2017) (g = 1,072; CI = 95%; 0,571 - 1,573). In Study 5, game-based learning was used to effectively improve learning gains and knowledge retention about anatomy and physiology (Rondon, Sassi, & Furquim de Andrade, 2013) (g = 0,374; CI = 95%; -0,394 - 1,141). Study 6 reports higher learning gains about physics in secondary students, with the use of a physlet-based game, compared with expository instruction (Ülen, Čagran, Slavinec, & Gerlič, 2014) (g = 0,884; CI = 95%; 0,429 - 1,339). In Study 7, a serious game was used to teach computer science in high schools (Papastergiou, 2009) (g = 0,782; CI = 95%; 0,351 - 1,213). Study 8 supports visual and verbal learning gains in schizophrenia patients using a virtual reality-based videogame (López-Martín, Segura-Fragoso, Rodríguez-Hernández, Terrer, & Polonio-López, 2016) (g = 0,291; CI = 95%; -0,320 - 0,902). In Study 9, digital game-based learning was used as a strategy to enhance mathematics learning outcomes in a basic education context (Hung, Huang, & Hwang, 2014) (g = 0,751; CI =95%; 0,162 - 1,339). Study 10 reports a more effective process in learning best medical practices, when using game-based learning, in the professional development of physicians (Telner, Bujas-Bobanovic, & Chan, 2015) (g = 0,751; CI = 95%; 0,162 - 1,339). Study 11 shows effective learning outcomes related to science and physical activity in elementary school students (Sun & Gao, 2016) (g = 0,241; CI=95%; -0,291 - 0,774). In Study 12, a GPS/Location-based game is used to enhance secondary students’ historical knowledge (Huizenga, Admiraal, Akkerman, & Dam, 2009) (g = 0,326; CI = 95%; 0,132 - 0,521). Study 13 uses an observational protocol to measure concrete behavior change in children, after an intervention with a videogame about pedestrian safety (Arbogast, Burke, Muller, Ruiz, Knudson, & Upperman, 2014) (g = 0,199; CI = 95%; -0,006 - 0,404). In Study 14, engineering students learning related to complex theoretical knowledge was improved with a game-based learning approach (Ebner & Holzinger, 2007) (g = 0,191; CI = 95%; -0,328 - 0,711).

Using a fixed-effect analysis, the effect ratio for the 14 studies is 0,387 with a confidence interval of 0,291 to 0,484, which indicates that a Game-Based Learning approach can increase the learning process outcomes by at least 28%, and perchance by as much as 47% in comparison with traditional approaches (expository or self-study). Z = 7,870, SE = 0,049, p <.001. Hedges’g was used because it provides a superior estimate of the standardized mean difference in small samples (Higgins, 2003).

The studies have demonstrated an absence of reporting bias since the funnel plot was symmetrical. The heterogeneity between studies in the analysis is significant and moderate (I2 = 51,803; p < .05).

Conclusion

In this study, we analyzed fourteen studies, from an initial group of 68 studies, summarizing the learning gains in GBL approaches, using videogames as a pedagogical tool. In the summarized results of our sample, the GBL approaches seems to show consistent learning gains in a broad range of areas of implementation. The meta-analysis results points toward the existence of statistically significant differences between groups, being GBL interventions more effective than traditional approaches, usually expository. Considering these data, using videogames can be considered a valid approach in the learning process, supported by strong scientific evidence, which can frame its wider adoption as an intervention methodology.

In addition, the collected sample shows that significant differences are found in a multiplicity of contexts, from basic school to vocational training, and with different types of population, from children to clinical populations. This data reinforces the idea of GBL as transversal approaches in educational settings, able to promote several subjects within one videogame. This data also reinforces the prominent future of videogames as valuable learning tools, framing future interventions in fields like education or training.

Some limitations of this study may include some heterogeneity of the research designs and the requirement for RCT as a factor of exclusion of a significant part of the sample studies. Also, research designs comparing pre and post intervention measures lack the evaluation of the learning progress, as a more qualitative construct, and this is a very relevant aspect in education.

Future research must include the studies that are being carried out, separating the analysis of effectiveness by each specific context, for example the school context from the context of adult professional development. Moreover, aspects related with specific game designs and gameplay environments should be analyzed, regarding its effectiveness in learning, and discriminating the more suitable for the promotion of each type of subject or skill.

References

Ackley, B., Ladwig, G., Swan, B., & Tucker, S. (2007). Evidence-Based Nursing Care Guidelines: Medical-Surgical Interventions (1st ed.). St. Louis, Missouri: Mosby.

Ahn, S., Ames, A., & Myers, N. (2012). A Review of Meta-Analyses in Education: Methodological Strengths and Weaknesses. Review Of Educational Research, 82(4), 436-476.http://dx.doi.org/10.3102/0034654312458162

Ashinoff, B. (2014). The potential of videogames as a pedagogical tool. Frontiers In Psychology, 5.http://dx.doi.org/10.3389/fpsyg.2014.01109

Boeker, M., Andel, P., Vach, W., & Frankenschmidt, A. (2013). Game-Based E-Learning Is More Effective than a Conventional Instructional Method: A Randomized Controlled Trial with Third-Year Medical Students. Plos ONE, 8(12), e82328.http://dx.doi.org/10.1371/journal.pone.0082328

Costa, C., Tyner, K., Henriques. S., & Sousa, C. (2016). A Review of Research Questions, Theories and Methodologies for Game-Based Learning. Journal of Content, Community & Communication, 4(2), 2-16.

Eichenbaum, A., Bavelier, D., & Green, C. (2014). Videogames - Play that can do serious good. American Journal Of Play, 7(1), 50-72.

Farace, D. (1998). Perspectives on the Design and Transfer of Scientific and Technical Information. GL’97 Conference Proceedings, 354-359.

Granic, I., Lobel, A., & Engels, R. (2014). The benefits of playing videogames. American Psychologist, 69(1), 66-78. http://dx.doi.org/10.1037/a0034857

McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. New York: Penguin Press.

Pellegrini, A., Dupuis, D., & Smith, P. (2007). Play in evolution and development. Developmental Review, 27(2), 261-276.http://dx.doi.org/10.1016/j.dr.2006.09.001

Piaget, J. (1962). Play, dreams, and imitation in childhood. New York: Norton.

Rosi, A., Dall’Asta, M., Brighenti, F., Del Rio, D., Volta, E., & Baroni, I. et al. (2016). The use of new technologies for nutritional education in primary schools: a pilot study. Public Health, 140, 50-55. http://dx.doi.org/10.1016/j.puhe.2016.08.021

Sáez-López, J. M., Miller, J., Vázquez-Cano, E., & Domínguez-Garrido, M. C. (2015). Exploring Application, Attitudes and Integration of Video Games: MinecraftEdu in Middle School. Educational Technology & Society, 18 (3), 114–128.

Salen, K., & Zimmerman, E. (2010). Rules of play. Cambridge, Massachusetts: The MIT Press.

Sánchez-Meca, J., & Marín-Martínez, F. (2010). Meta-analysis in psychological research. International Journal Of Psychological Research, 3(1), 150. http://dx.doi.org/10.21500/20112084.860

Sousa, C., Henriques, S.,& Costa, C. (2017). Are Videogames a waste of time? - A multi-stakeholder approach. EDULEARN 17 Proceedings, 2336-2341. http://dx.doi.org/10.21125/edulearn.2017.1484

Squire, K. (2008). Video Games and Education: Designing Learning Systems for an Interactive Age. Educational Technology, 17-21.

Sutton, A. (2000). Methods for Meta-Analysis in Medical Research. New York: John Wiley & Sons.

Sward, K., Richardson, S., Kendrick, J., & Maloney, C. (2008). Use of a Web-Based Game to Teach Pediatric Content to Medical Students. Ambulatory Pediatrics, 8(6), 354-359. http://dx.doi.org/10.1016/j.ambp.2008.07.007

Vygotsky, L. (1978). Mind in Society: The Development of Higher Psychological Functions. Cambridge, MA: Harvard University Press.

IR