HTML

ePub

PDF

I. INTRODUCTION

Constructing a taxonomy of metacognitive activities is important, since it allows the assessment, in a suitable way, of the metacognitive skills (MS) of students, characterizing their metacognitive behaviors during the problem solving (PS) learning.

In this order to construct a taxonomy of metacognitive activities, it is important to understand that PS learning requires that the student integrate his/her mathematical knowledge and the way how to use it; nevertheless, this integrated approach is a challenging task for students [1]. For instance, [2] propose that “a student who knows the area calculation formula of a parallelogram can easily solve a problem that is aimed at directly calculating the area of a parallelogram. However, when the student needs to calculate the area of a parallelogram within a novel type of question, she/he may fail to transfer prior knowledge to the task at hand and may not be able to solve the problem”.

Therefore, MS play an important role in PS, since they “pertain to the acquired repertoire of procedural knowledge for monitoring, guiding, and controlling one’s learning and problem-solving behavior.” [3]. In this order, MS allow students to interiorize knowledge and their learning activities, with the purpose of adapting those activities to the situational demands, thus optimizing their PS results [4], [5].

Nevertheless, students do not acquire MS naturally, either because they lack opportunities or because they do not see the importance of investing their efforts in the construction of such skills [6]-[7]. Therefore, teachers can employ assessment methods to adapt their teaching strategies according to the students’ characteristics, in this way, instructors may foster the MS use [8]-[9].

MS occur through cognitive activities [3], since “one cannot engage in planning without carrying out cognitive activities, such as generating problem-solving steps and sequencing those steps” [10].

The simple fulfillment of cognitive activities does not lead to having MS; on the contrary, MS occur when metacognitive activities regulate cognitive activities [3]. In this context, [3] states that the metacognitive activity resembles a General, and cognitive activities resemble an army, where the General cannot win the war without soldiers (cognitive) and neither can a disorganized army.

The metacognitive activity is essential in novel situations or when the automatic responses are not adaptive [11], [12]. Consequently, metacognitive activities follow the guidelines of the metacognitive strategies that allow taking decisions in compliance with a given objective, selecting pertinent information, and organizing activities in a logical way [13]. Hence, metacognitive strategies are sequential processes devoted to monitoring and controlling cognitive activities, with the purpose of assuring the fulfillment of an objective [14]. In this order, the metacognitive activity is an executive function which comprises a set of essential cognitive processes for the metacognitive regulation of learning [11], [12].

Several authors [15], [16]-[17] claim that MS are: i) planning, which is the selection of appropriate strategies, and the localization of factors affecting performance, ii) monitoring, which is the possibility to carry out, understand, and modify the achievement of the task, and iii) evaluation, which is the verification of the nature of the actions and decisions taken by the student to identify their efficiency.

The accomplishment of MS involves cognitive and metacognitive activities, as well as metacognitive strategies. For this reason, a method which aims to characterize MS should keep this relationship into account. For example, online assessment facilitates the evaluation of MS, considering the mentioned relationship.

The findings of online methods are strong predictors of learning outcomes [18], since they assess students during PS, as online assessment start from the actual student’s performance during PS [6], [19]-[20]. In addition, they look for information, considering the specific domain where the students solve problems [21].

Likewise, online methods are thinking-aloud protocols (TAP) and Logfiles. On the one hand, the student verbalizes his/her thoughts while solving the task during TAP. On the other hand, Logfiles provides detailed information of the cognitive activities expressed by the student during the execution of a cognitive challenge that implies the use of a computer [22].

The TAP and Logfiles provide information that a group of judges interprets and codifies through a system of categories established in a taxonomy of metacognitive activities [19]. This taxonomy should describe the MS, regarding the metacognitive activities that take part in the task resolution of a specific domain [23].

Diverse taxonomies of metacognitive activities have been proposed; for instance, in [24] the authors report a taxonomy of metacognitive activities to analyze the learning process of psychology university students in hypermedia environments. In a similar way, in [23] a general taxonomy is proposed to examine high school students during the reading of history texts and solving physics problems. In addition, [19] expose a taxonomy to describe activities used by kids while they solve mathematics problems.

The taxonomies described above are an invaluable contribution to the assessment of MS; nonetheless, they have several deficiencies. First, these tools are designed only in the English language, which means that their implementation in Spanish language studies results in a process that requires an adaptation and validation in this language.

A second deficiency is that, out of the authors mentioned before, only [24] present descriptions of their metacognitive activities; nevertheless, this information is not sufficient, since it is not possible to identify which type of expressions (verbal or nonverbal) performs the student when he/she utilizes a particular activity. In consequence, it is difficult that other investigations employ this taxonomy.

For the above reasons, those works that wish to apply the taxonomy proposed by [24] may fall into two scenarios. In the first, they could make research efforts to extend the taxonomy descriptions, in this order to understand the meaning of each metacognitive activity. In the second scenario, the research studies would assume the ambiguity of the description, and with only this information, they could analyze their students, nonetheless, their results would be debatable since they depend exclusively on the coder’s judgement.

It is worth highlighting that in [25] the authors of this paper, reported a first version of the taxonomy with the use of technological tools.

The aim of this paper is to document the method used by the authors to construct the taxonomy of metacognitive activities which describe the MS of engineering university students during the PS learning.

The taxonomy presented here is within the limited field of taxonomies designed in the Spanish language; this taxonomy permits the creation of wide descriptions of metacognitive behaviors. Hence, the disclosed tool may be used by other research studies.

II. METHODOLOGY

This section provides a scenario which explains how to construct a taxonomy of metacognitive activities. First, the authors expose the requirements that the taxonomy should be expected to meet. Second, it presents the method used, and finally, the steps created to build the tool. These steps are explained in section III.

A. Requisites

This research draws from the three requisites proposed by [23] to construct a taxonomy of metacognitive activities:

1) The taxonomy should expect the metacognitive activities to be suitable to describe the students’ behavior in the interest domain.

2) The taxonomy should be complete in terms of its components, in order to cover declarations that go beyond literal texts, that is, actions performed by the student which are not possible to be detected in the recording.

3) The taxonomy should be related with other taxonomies specialized in metacognitive activities and divulged in the contemporaneous literature, with the purpose of allowing the proposed taxonomy to have slight divergences with other already existing taxonomies.

B. Method

The authors of the present work used the constant comparison method to construct the taxonomy of metacognitive activities that meet the three requisites outlined before. This method has been utilized in other studies as in [23] and in [26] to generate a system of categories.

The method is based on the comparison and systematic analysis of information, to find verbal and nonverbal patterns, and identify events through the saturation of data, and not the test or verification of previously established hypothesis [27]. In this regard, this method highlights the importance of analyzing and comparing information systematically with the purpose of verifying common behaviors.

C. Steps created.

The meeting of the requisites and the utilization of the constant comparison method allowed for the creation of steps with which the taxonomy of metacognitive activities was construct (see fig. 1).

The seven steps permit the construction of the taxonomy, ending up with three main categories and 28 metacognitive activities distributed in the following way: 6 metacognitive activities in the planning category, 17 in monitoring, and 5 in evaluation. Each activity relies on its respective description.

III. RESULTS

This section contains a detailed description of the way each step of fig.1 was carried out; besides, the authors document how to meet the specified requirements in [23].

Step 1: Gather metacognitive activities which occur in contexts like those experienced during the PS learning.

The authors recovered specialized literature through a snowball sampling technique; in this way, the following documents were retrieved: [6], [19], [20],[23], [24], [26], [28]. Later, the authors gathered metacognitive activities from these texts.

Step 2: Selection of metacognitive activities that belong to the planning, monitoring, and evaluation categories.

The authors filtered out the metacognitive activities gathered in step 1, to keep those activities that suit the categories of project.

This filtering considered the reports published in the specialized literature; for example, the authors retrieved the activity “explain and justify strategies” from [23] and assigned this activity to the evaluation category, since in [23] they report that this activity describes assessment MS.

Step 3: Gathering information in the field.

The authors applied six pilot tests of PS to students, regarding that each student takes a PS test. The individuals belong to the program of Industrial Engineering of the Faculty of Business Sciences of the Universidad Tecnológica de Pereira.

In some cases, the exams required to use ProModelTM software (version 8.6.2.1037), which is a computer tool that helps the engineering students to solve complex problems, then, it was necessary to employ the Logfiles to evidence the PS process. TAP occurred in all the exams to video capture the verbalization of the students’ thoughts.

Step 4: Transcription and codifying of gathered information.

The researchers transcribed the TAP information of each pilot test, considering the observations evidenced with the Logfiles. After, auditors individually coded one transcription at a time. The codifying regarded assigning metacognitive activities to fragments of the transcription. The fig. 2 exposes an example of the encoding result where the codes are in red, green, and yellow colors, and the Spanish transcription is surrounded by a blue box.

Step 5: Identification of convergences.

The researchers collected the codified fragments with the same metacognitive activity, then identified the relevant terms in the fragments or in the label of the metacognitive activity, following their own perspective. The authors looked up these words in the dictionary of the Spanish Royal Academy (Real Academia Española - RAE). The Table I summarizes the relevant words of the fragment and the metacognitive activity (underlined terms), as well as the definitions extracted from the RAE.

Step 6: Write a memo of semantic analysis.

In this step, the authors drew from the postulate of [29], who state that the taxonomy is a key structure to categorize the knowledge centered in semantics. Therefore, the researchers analyzed semantically the fragment and the metacognitive activity that had been previously enriched with the dictionary information. Thus, the authors recognized behavior patterns of the student during PS. The Table I exposes the memo of the activity “Detecting errors”.

Step 7: Elaboration of the description.

The authors built the description of each metacognitive activity from an iterative study of the patterns found in the previous step. The patterns have a very important role because describe how the students had learned to solve problems. Consequently, the researchers repeatedly analyzed the semantics of the fragments in order to find data which allowed the characterization of the student behaviors during PS. For this reason, the set of patterns found in pilot tests permits to consolidate the description of a metacognitive activity.

The authors finished the construction of the metacognitive activities when they achieved the saturation of information, that is, when there was not new evidence in the pilot tests. It is important to highlight that the descriptions do not imply necessarily that a metacognitive activity precedes or succeeds another; therefore, the descriptions do not suggest a temporary order for auditors when they analyze the TAP and the Logfiles reports.

The following paragraphs show an example of the description associated with the metacognitive activity “Detecting errors”, which is present within the constructed metacognitive activities.

Example:

The researchers built the description of the activity “Detecting errors” starting with the patterns of the metacognitive activities. In this order, the auditors analyzed: i) the terms that characterized the fragments assigned to that particular activity; ii) the information from the Logfiles as an evidence that the student had detected an error; iii) the fragments that encoders had labeled simultaneously with diverse metacognitive activities. The incidences associated to that specific activity were identified because:

Pattern 1

The terms “be wrong”, “screw up” and “but (followed by an activity)” represent the moment when the student detects an error in his/her process or in the task execution, fragments that the students recognized as wrong, or when they forgot to place certain element and so screwed it up.

There are occasions when the student expresses that he/she is wrong, without specifying which the error is, and thanks to the nonverbal information of the TAP transcriptions, the auditors identified the particularities of the flaw.

Pattern 2

The student does not verbalize associated terms once they detect an error; instead, he/she employs words which express an apology for the mistake made (and not verbalized); for example, an event where the student used the term “sorry” without having expressed his error.

Pattern 3

Situations where it was possible to evidence that the student detected an error thanks to the confirmatory information of the Logfiles. In occasions, the student does not verbalize terms associated with the error detection (“be wrong,” “mess it up,”, and “but (preceded by an activity), nor does he express words representing apologies for a mistake made; instead, those fragments have data from the Logfiles, which allow to verify that the student has detected an error.

For example, “I forgot to put the 20 meters to this band (the student opens a window and adds the missing information)”. In this situation, the student verbalizes ambiguous terms and proceeds to correct his/her error (action detected by the Logfiles).

Likewise, the auditors identify the fragments where the Logfiles information indicates that the student is wrong and shows the mistake. In this order, the Logfiles facilitate to identify the moment when the student detects an error and proceed to correct it.

Pattern 4

The auditors consider when fragments had been labeled in two or more metacognitive activities simultaneously. In specific, the encoders regard that the student usually corrects his/her mistakes when he/she identifies it. Therefore, most of the fragments assigned to “Detecting errors” are labeled with the activity “Take a corrective approach”, which belongs to the monitoring category.

It is convenient to point out that the simultaneous assignment of these activities does not depend on the presence of Logfile information; on the contrary, it is given transversally to the patterns discussed above.

In this way, the auditors created a description made up of patterns for each metacognitive activity of the taxonomy, thus finishing its construction.

IV. Conclusions And Further Research

This paper describes a method to construct a taxonomy of metacognitive activities. It is a tool that contains an organized set of activities leading to the analysis of the metacognitive behavior of university students during their PS learning. The exposed taxonomy becomes a framework to characterize the MS that take part in the PS learning.

Likewise, the taxonomy is an essential supporting tool for the suitable analysis of the TAP declarations and the Logfiles. The document presented here attempts to be a guide for future projects to build taxonomies of metacognitive activities.

Future studies could use this taxonomy with a different population, leading to the tool strengthening.

References

[1] M. V. J. Veenman and D. van Cleef, “Measuring metacognitive skills for mathematics: students’ self-reports versus on-line assessment methods,” ZDM - Math. Educ., vol. 51, no. 4, pp. 691–701, 2019, doi: 10.1007/s11858-018-1006-5.

[2] Z. Ç. Özcan, “The relationship between mathematical problem-solving skills and self-regulated learning through homework behaviours, motivation, and metacognition,” Int. J. Math. Educ. Sci. Technol., vol. 47, no. 3, pp. 408–420, 2016, doi: 10.1080/0020739X.2015.1080313.

[3] M. V. J. Veenman, “Learning to self-monitor and self-regulate,” in Handbook of research on learning and instruction, 2011, pp. 197–219.

[4] R. Azevedo, D. C. Moos, A. M. Johnson, and A. D. Chauncey, “Measuring cognitive and metacognitive regulatory processes during hypermedia learning: Issues and challenges,” Educ. Psychol., vol. 45, no. 4, pp. 210–223, 2010, doi: 10.1080/00461520.2010.515934.

[5] E. Baten, M. Praet, and A. Desoete, “The relevance and efficacy of metacognition for instructional design in the domain of mathematics,” ZDM - Math. Educ., vol. 49, no. 4, pp. 613–623, 2017, doi: 10.1007/s11858-017-0851-y.

[6] M. Van Der Stel and M. V. J. Veenman, “Metacognitive skills and intellectual ability of young adolescents: A longitudinal study from a developmental perspective,” Eur. J. Psychol. Educ., vol. 29, no. 1, pp. 117–137, 2014, doi: 10.1007/s10212-013-0190-5.

[7] A. Zohar and S. Barzilai, “A review of research on metacognition in science education: current and future directions,” Stud. Sci. Educ., vol. 49, no. 2, pp. 121–169, 2013, doi: 10.1080/03057267.2013.847261.

[8] J. C. Campione, A. L. Brown, and M. L. Connell, “The Teaching and Assessing of Mathematical Problem Solving,” in Metacognition : On the importance of understanding what you are doing., 3rd ed., no. January 1988, 1988, pp. 93–114.

[9] M. E. Bernal, M. Gómez, and R. Iodice, “Interacción Conceptual Entre Interacción Conceptual,” Rev. Latinoam. Estud. Educ., vol. 15, no. 1, 2019, doi: 10.17151/rlee.2019.15.1.11.

[10] M. V. J. Veenman, B. H. A. M. Van Hout-Wolters, and P. Afflerbach, “Metacognition and learning: Conceptual and methodological considerations,” Metacognition Learn., vol. 1, no. 1, pp. 3–14, 2006, doi: 10.1007/s11409-006-6893-0.

[11] V. Anderson, R. Jacobs, P. J. Anderson, and M. Spencer Smith, “Development and assessment of executive function: From preschool to adolescence,” in Executive Functions and the Frontal Lobes, vol. 7, no. 9, 2008, pp. 123–154.

[12] A. Diamond, “Executive functions,” Annu. Rev. Psychol., vol. 64, pp. 135–168, 2013, doi: 10.1146/annurev-psych-113011-143750.

[13] M. Quiroga, “La metacognición como función ejecutiva : su rol en la compresión de textos,” Rev. del Dep. Let., vol. 5, pp. 516–528, 2016.

[14] A. Peña-ayala and L. Cárdenas, “A Conceptual Model of the Metacognition Activity,” in Metacognition: Fundaments, Applications, and Trends A Profile of the Current State-Of-The-Art, 2015, pp. 39–74.

[15] G. Schraw, “Promoting General Metacognitive Awareness,” Instr. Sci., vol. 26, no. 1, pp. 113–125, 1998, doi: 10.1007/978-94-017-2243-8_1.

[16] P. Georghiades, “From the general to the situated: Three decades of metacognition,” Int. J. Sci. Educ., vol. 26, no. 3, pp. 365–383, 2004, doi: 10.1080/0950069032000119401.

[17] F. Orrego, M., Tamayo, O., & Ruíz, Unidades didácticas para la enseñanza de las ciencias. 2016.

[18] M. V. J. Veenman, “Assessing Metacognitive Skills in Computerized Learning Environments,” in International Handbook of Metacognition and Learning Technologies, 2013, pp. 157–169.

[19] A. Desoete, “Multi-method assessment of metacognitive skills in elementary school children: How you test is what you get,” Metacognition Learn., vol. 3, no. 3, pp. 189–206, 2008, doi: 10.1007/s11409-008-9026-0.

[20] C. Y. Wang, “Exploring general versus task-specific assessments of metacognition in university chemistry students: A multitrait–multimethod analysis,” Res. Sci. Educ., vol. 45, no. 4, pp. 555–579, 2015, doi: 10.1007/s11165-014-9436-8.

[21] G. Schellings, “Applying learning strategy questionnaires: Problems and possibilities,” Metacognition Learn., vol. 6, no. 2, pp. 91–109, 2011, doi: 10.1007/s11409-011-9069-5.

[22] M. V. J. Veenman, R. D. Hesselink, S. Sleeuwaegen, S. I. E. Liem, and M. G. P. Van Haaren, “Assessing developmental differences in metacognitive skills with computer logfiles: Gender by age interactions,” Psihol. Teme, vol. 23, no. 1, pp. 99–113, 2014.

[23] J. Meijer, M. V. J. Veenman, and B. H. A. M. Van Hout-Wolters, “Metacognitive activities in text-studying and problem-solving: Development of a taxonomy,” Educ. Res. Eval., vol. 12, no. 3, pp. 209–237, 2006, doi: 10.1080/13803610500479991.

[24] R. Azevedo and J. G. Cromley, “Does training on self-regulated learning facilitate students’ learning with hypermedia?,” J. Educ. Psychol., vol. 96, no. 3, pp. 523–535, 2004, doi: 10.1037/0022-0663.96.3.523.

[25] M. E. Bernal Loaiza, M. Gómez Suta, L. M. Ochoa Salinas, M. Castaño Ramírez, and R. Iodice, “Herramientas para evaluar las habilidades metacognitivas en estudiantes de ingeniería al resolver problemas en simulación de eventos discretos,” in EIEI, ago. 2019.

[26] M. Pressley, “Development of Grounded Theories of Complex Cognitive Processing : Exhaustive Withinand Between-Study Analyses of Think-Aloud Data,” Issues Meas. Metacognition, pp. 261–296, 2000.

[27] R. J. Cuñat Giménez, “Aplicación De La Teoría Fundamentada ( Grounded Theory ) Al Estudio Del Proceso De Creación De Empresas,” Decis. Glob., pp. 1–13, 2007.

[28] L. De Backer, H. Van Keer, and M. Valcke, “Exploring the potential impact of reciprocal peer tutoring on higher education students’ metacognitive knowledge and regulation,” Instr. Sci., vol. 40, no. 3, pp. 559–588, 2012, doi: 10.1007/s11251-011-9190-5.

[29] B. Carrion, T. Onorati, and P. Diaz, “Designing a semi-Automatic taxonomy generation tool,” Inf. Vis. - Biomed. Vis. Vis. Built Rural Environ. Geom. Model. Imaging, IV 2018, pp. 266–271, 2018, doi: 10.1109/iV.2018.00053.

Notas de autor

IR