HTML

ePub

PDF

Introduction

This article presents the results of a study with an experimental design, which aimed to test the effectiveness of a course for teaching self-regulation writing skills (SWC). The main objective of the course was to teach self-regulating skills in writing academic texts. This proposal was based on three premises regarding academic writing: 1) Recognizing writing skills as a tool, not only for communication, but also as an instrument for “objectifying, organizing, reviewing, modifying, increasing, clarifying, and building thinking and reflection. 2) Writing, seen as a skill in permanent development rather than an acquired ability. 3) Recognizing the practice of academic writing as an exercise specific to each discipline, which is why its development is linked to a concrete and well-defined learning situation.

Studies on academic writing agree in defining it as a truly complex and difficult cognitive task that must be learned and developed to achieve expertise (Harris, Graham, MacArthur, Reid & Mason, 2011; Cislaru & Olive, 2018). Kellogg (2008) explains that it takes at least two decades to use writing as a tool to build knowledge, during which the subject moves through three stages. The first one is known as “knowledgetelling” (Kellogg, 2008, p. 6); the second one is the intermediate stage, in which the subject seeks “knowledge-transforming” (Kellogg, 2008, p. 6); the third one is the stage that mature writers would reach when they seek to benefit their readers, which is why they pursue “knowledge crafting” (Kellogg, 2008, p. 7). According to Bereiter & Scardamalia (1986), generating the content, creating and organizing a textual structure, formulating a high-level plan and goals, quickly and efficiently implementing the mechanical aspects of language, and reviewing are the five areas of writing competency that prove to be the most difficult in learning and developing this complex task. These difficulties in earning are evidenced at all levels of schooling, from basic to postgraduate levels, and translate into a deterioration in students’ attitudes toward the task of writing.

Based on the identification of this generalized issue, a wide number of teaching strategies have been formulated. For this study, it is relevant to understand the developed interventions based on the approach to the process. One of the lines that have made the most progress in this regard is the one on teaching strategies (Harris et al., 2011; Malpique, Veiga, & Frison, 2017), focused on integrating self-regulation and writing in intervention programs; scientific evidence has demonstrated that teaching strategies have achieved a significant impact on the writing students’ performance at different schooling levels. Four meta-analyses aimed at identifying and clarifying the evidence from studies on teaching writing indicate that interventions associated with teaching strategies are characterized by containing resources and/or procedures, such as teaching writing planning, reviewing, and editing strategies; setting clear and specific goals about what is being sought; engaging students in pre-writing activities; presenting writing models; providing tools for students to monitor their own writing and writing behavior; and finally, providing enough time (Graham & Perin, 2007; Koster, Tribushinina, De Jong & van den Bergh, 2015; Rogers & Graham, 2008).

Within the framework of interventions on teaching strategies, self-regulation emerges as an essential component. Self-regulation of writing is understood as a set of processes, at the cognitive, emotional-motivational, contextual, and behavioral levels, that the writer uses to achieve the set objectives and improve their writing skills (Zimmerman & Risemberg, 1997). Specifically, self-regulation of writing is evidenced in three phases: a) forethought, in which the subject sets clear goals, prepares a plan consistent with the goals, and anticipates and chooses the strategies needed to carry out the task; b) implementation, in which they carry out an exercise of constant monitoring and systematically observes the development of the text as a function of the goal; and c) evaluation, in which they judge and verify how close, or how far, their text is from the initial goals, and finally, takes concrete actions regarding the process, if applicable. Implicit in this interest in training in self-regulation is the understanding that it plays an essential role in writing for several reasons. First, it has been proven that the most skilled writers are more selfregulated than those who are inexperienced. Second, there is a directly proportional relationship between the development of a writer’s self-regulating skills and their level of writing expertise. Third, the development of self-regulating skills increases with age and training, which is reflected in increased writing performance (Budde, Glaser & Brunstein, 2012 in Torrance et al., 2012; Graham, 2006; Palermo & Thomson, 2018).

The tool and its theoretical basis

The scaffolding, in its broadest sense, refers to the guide or aid a trained individual can provide to another, so they reach a potential level of learning (Wood, Bruner & Ross, 1976). A direct metaphor is established with the scaffolding, given the needed but temporary nature of the aid or support; this concept has taken on a particularly important meaning in virtual educational environments, thus consolidating a whole line of research focused on the design and validation of this type of tool (Hederich, Camargo & López, 2015; Hederich, Camargo & López, 2018). From a virtual education approach, the term scaffolding refers to technological tools designed so the student reaches a learning goal that they would not achieve on their own. When designing a virtual scaffolding, students are expected to successfully reach learning achievements and be prepared to work autonomously in these education environments (Hederich, Carmargo & López, 2015; Lachner, Burkhart & Nückles, 2017).

In the course proposed in this study, the scaffolding serves the function of modeling and making explicit the complete writing self-regulation process, for which modules of questions with a reflective role are developed, distributed thus: 1) the forethought phase, in which the student has the opportunity of structuring and organizing the task’s environment, planning the required time, choosing the cognitive strategies and formulating expectations regarding scientific article writing; 2) the performance phase, in which the text writing is written and where cognitive processes involved in this task are deployed (planning, text generation, and editing), whose training has proven to have considerable effects on text quality (Limpo, Alves & Fidalgo, 2013) and 3) the reflection phase, which allows the student to make a general balance of the writing task, make judgments about their performance and the process, and take actions related to such judgments.

Regarding metacognitive awareness, it is known that it constitutes a key component in the execution of writing tasks and that, in addition, it operates as a facilitator of self-regulation processes since it allows the subject to transfer knowledge, skills, and strategies from one learning situation to another (Azevedo & Witherspoon, 2009; Schraw, 1998). It is known that low levels of metacognitive awareness are related to the absence of adequate strategies to develop the writing task (Kellogg, 1994; McCormick, 2003; Negretti, 2012). In the particular case of the course presented, the increase in metacognitive awareness is developed through two actions: 1) providing information to the student about the conditions in which the course begins and 2) of the so-called metacognitive activators , corresponding to messages that are displayed to the student, in the form of recommendations or reflective questions, through pop-up windows on the platform.

The course design presented here is based on the idea that writing should be taught as a process, not as a product, which requires training students in the use of cognitive and metacognitive strategies. In this regard, scientific evidence has demonstrated that strategy teaching has had a significant impact on the writing performance of students at different levels of education (Harris et al., 2011).

Within the framework of strategy teaching interventions, writing self-regulation appears as an essential component. Writing self-regulation is understood as a set of processes, at the cognitive, affective-motivational, contextual, and behavioral levels, employed and managed by the writer to achieve stated objectives and improve their writing skills (Zimmerman & Risemberg, 1997). Training self-regulation as a strategy is justified by the idea that it plays an essential role in writing insofar as there is a directly proportional relationship between the development of the writer’s self-regulated skills and their level of expertise in writing (Burnham, 1994; Lamb, 1997; Zimmerman & Risemberg, 1997).

Specifically, writing self-regulation is evidenced in three phases: 1) forethought, in which the subject sets clear goals, elaborates a coherent plan with the goals and anticipates and chooses strategies necessary to carry out the task; 2) performance, in which they perform an exercise of constant monitoring and systematically observe the development of the text as a function of the goal; and 3) evaluation, in which they judge and verify how close, or how far, their text is to the initial goals and finally, take concrete actions regarding the process, if required (Zimmerman & Risemberg, 1997).

Summarizing, studies have concluded that interventions reporting an important effect on the development of writing skills are associated with training in strategies and are characterized by favoring aspects associated with writing planning, reviewing and editing process teaching, training in the formulation of well-defined goals and purposes, involving students in reflection exercises and, in general, by trying to increase metacognitive awareness. Hence, the purpose of the course, as a teaching tool, is to model the process of self-regulation and, at the same time, to favor an increase in the writer’s metacognitive awareness during the writing task.

Detailed presentation of the tool

The course presented is managed through a virtual learning environment programmed into the Moodle learning management system. The essential purpose of the environment is to increase the academic-text writing self-regulating ability in university students, which is why it seeks to promote the interaction of the processes’ key agents: a) the student, as an active participant, autonomous and committed to his own education process as a writer and b) the teacher, who has the role of assisting and guiding the student in the process the virtual environment proposes and of guiding the writing through activities of planning, writing and editing the text; in this sense, the teacher acts as a tutor, insofar as he guides the processes, and as a model, insofar as he presents himself as a writer.

In general terms, the course proposes an exercise in which the student will write a scientific article in approximately 12 weeks. We hypothesize that students will train themselves in the selfregulating writing process and increase their metacognitive awareness through the resources available within their reach during this time.

In that regard, although the main purpose ofthe course is not to teach how to write scientifictexts, self-regulation training should produce apositive effect on the quality of the texts.

To achieve its objectives, the virtual environment has two fundamental elements whose effect is examined separately and combined; the first one is the Self-regulated Writing Scaffolding (SWS), and the second corresponds to the strategies to increase metacognitive awareness (IMA) (figure 1). The SWS is organized in units that lead the student, step by step, through the three phases of the self-regulation process: forethought, performance, and reflection, acting as a modeler of the process. In each phase, a series of resources are provided, allowing the student to think about the writing task, organize a plan, execute it, exercise control over it, and evaluate himself.

On the other hand, strategies for IMA are implemented through two resources: 1) the report, to the students themselves, of their initial conditions as writers and 2) the metacognitive activators , corresponding to messages in the form of recommendations or reflective questions that are presented through pop-up windows. In general, these strategies have the objective to offer information of a metacognitive nature, intended to encourage the student to reflect on the task progress as a function of the goal they have set. It is different from the scaffolding since it does not model any process; it only intends to make suggestions, recommendations and promote reflection.

The structure of the course is explained in detail below. The SWS is presented first, describing how the phases of forethought, performance, and reflection work. Then, the IMA strategies are explained and how the two strategies are connected.

Strategy 1: Self-regulating writing scaffolding

In general terms, the SWS works as a writing self-regulation modeler, and it aims to guide the student in the construction of academic texts based on training in virtual environments. It is important to note that the SWS’s objective is not to teach writing, even though we expect that by modeling the self-regulation process and teaching the writer to self-regulate, they will significantly improve their writing skills. The following diagram summarizes the proposed interaction between behavior self-regulation, and writing cognitive processes.

Just as outlined in the scheme, the SWS is organized in three main phases: forethought, performance, and reflection, based on the proposal of the cyclical model of self-regulation processes and sources of motivation by Zimmerman and Campillo (2003) and on the proposal developed by Hederich, Camargo and López (2015), which consisted of the design of AMADIS (Distance Self-Regulating Metacognitive Scaffolding) 1 , whose principles share the cyclical explanation of self-regulation based on three phases, which the authors named preparation, performance, and reflection.

In the SWS, each phase has tools for the student to reach the objectives of the self-regulation process. As it can be observed, the three phases are related and establish a cyclical process. It begins with forethought, which leads to performance once completed. In this second phase are the cognitive writing processes (planning, text generation, and editing) proposed by Hayes and Flower (1981, 1983) and Hayes (1996); in other words, there is where text production occurs. Finally, the reflection phase allows the student to evaluate their behavior and the results of their work. It may lead them to readjust either specific elements of the text, for which they will return to the performance phase, or its structural elements, in which case they will return to the forethought phase. The operation of each phase is described below.

Forethought phase

In this phase, the writer analyses the task, and based on the formulation of a goal, chooses strategies that will allow him to reach it. The writer can manage the resources related to motivation: their self-efficacy, their expectations about the results, the value they assign to the task, and their orientation toward the goal.

Forethought develops in three dimensions: the environment, the individual, and the expectations. The essential purpose is preparing for selfregulating behavior in the writing process, and it is materialized through questionnaires that propose a series of questions aimed at creating an opportunity to reflect on and analyze the task explicitly. In the environment dimension, the writer is instructed to structure the environment of the task in terms of location, technological equipment, and resources, company, noise level, lighting, and support resources, which help them organize themselves to begin the task; these strategic choices are permanently adjusted according to the impact they have on the development of the writing task. The objective of guiding the student to structure the environment is to make this process explicit to them.

In the personal dimension, the writer is instructed to plan the time they will invest in the task, in terms of weekly hours and total text production time, and to adjust the goals. Leading the student to think about the use of cognitive strategies such as those proposed in this dimension (planning and time management, goal setting, self-evaluation standards, and organization of the text to be written) aims to develop self-regulation of the effectiveness as writers.

The objective of the expectations dimension is so that the writer reflects on the perceived difficulty of the task. In that same direction, they are led to examine their perception of efficacy, their degree of motivation, and their interests and needs. We expect that by promoting a reflection on these aspects in the students, they will be able to identify to what extent they will have to adjust their expectations to obtain a better achievement and reach the set goal.

Performance phase

The performance phase is designed so that the writer completes the task of producing the text. This phase directly corresponds to cognitive writing processes (Hayes & Flower, 1986; Hayes, 1996), which is why it is proposed that the student executes the planning, writing, and editing of their text. To this end, the phase is divided into three stages corresponding to each of these processes.

Planning stage

The objective of this stage is for the student to search, gather, and present ideas about what they will write and consider the audience and purpose of their text. This stage is organized into three parts. The first consists of general planning of the article, the second is planning of each section of the article, and the third is monitoring the process.

Three basic inputs are provided in the general planning of the article: a) recommendations about different ways to prepare a writing plan; b) a model of a published scientific article outline; and c) a model writing plan. In addition, at this stage, the student is encouraged to propose a general objective of the text, formulate the main thesis, and determine what type of audience they will target.

Once the student has completed this stage, they can move on to plan each section of the scientific article (introduction, methodology, results, and discussion). To develop this task, they are reminded of the importance of revisiting the ideas proposed in general planning and to move forward toward the details of each of those sections.

The planning of each one of the units has five elements:

1. An introduction to the subject of scientific articles.

2. Text models: a text written by the teacher during the course.

3. Text editor for writing the plan.

4. Forum section.

5. Resources. They correspond to support materials that constitute an important tool to grade the writing process, specifically of academic texts such as scientific articles.

Lastly, monitoring serves to control task progress and compliance with what was established in the forethought phase. To this end, three simple questionnaires that revisit essential aspects from the forethought phase are presented. The first one has the objective of evaluating if it is necessary to adjust the planning of time and initial goal; the second one, of examining if the perception of the difficulty of the task, the efficacy, of the motivation or the interests has changed; the third one seeks to evaluate the student’s perception on the article’s progress. According to the answers given in this section, the student will be able to move forward to the writing stage or return to the forethought phase to adjust their goals.

Writing stage

At this stage, the student shapes the written document based on the ideas generated during planning to present a full version of the text. To this end, resources related to the aspects that must be considered when writing a scientific article are made available. In addition, the section has resources such as the tutor’s writing model, in which the teacher exemplifies the writing process exercise; the text editor, in which the draft of the document is available and on which they will continue to write; and finally, a discussion forum, in which each student’s progress is presented and discussed by the participants.

When there is a complete version of the article, scaffolding leads the student to monitor again, which has the same function as in the previous section: to control the progress of the task and the fulfillment of what was established in the immediately preceding phase. When answering all the questions, the option of moving forward to the editing stage or returning to the writing stage is offered as required.

This stage has a writing workshop, a tool offered by Moodle for the collection, review, and peer review of student work. In this workshop, students upload their writings to the platform, which are then randomly assigned by the system for peer evaluation. Once the student participates in the workshop, presents their article, and peer-evaluate others, they move forward to the editing stage.

Editing stage

Editing is fundamental to the writing process since it allows evaluating both the process and the final product. This stage allows detecting and correcting errors and making changes and adjustments to the text. To do so, the student assesses the written document’s problems and seeks solutions to improve them.

First, the evaluations the peers made on the scientific article in the writing workshop are analyzed. At the same time, the writer carries out a self-evaluation process presented through a survey organized in four dimensions: 1) knowledge depth and breadth, 2) quality of thought, 3) quality of communication, and 4) references and format; this review, based on peer feedback and self-evaluation, leads the student to make the necessary adjustments to their text and present a new version of it.

Reflection phase

Once the student completes their participation in the second writing workshop, they move onto the reflection phase. In this phase, the results obtained in the task are evaluated by comparing them to the goal that had been formulated at the beginning of the course. At this point in the course, the student is expected to have a broad view of the process and to be able to make judgments regarding their performance, the achievements reached, and the deficiencies or weaknesses to overcome (Zimmerman, 2011).

The reflection phase completes by answering two questionnaires. The first one is oriented toward the dimension of the environment. It seeks to assess the relevance of the environment in which writers worked, the company, the use of the tutor aid, and any additional resources. The second corresponds to the personal dimension, which evaluates the efficacy of the forethought of time and goals, specifically, the fulfillment of the initial plan, the adjustment of objectives during the process, the plan usefulness, and the review processes.

Strategy 2: Increasing metacognitive awareness

The IMA corresponds to the second strategy used in the virtual learning environment to teach selfregulating during writing. This strategy is executed through two specific actions: 1) the student is informed about the use of their own resources and how they perform cognitive processes during the writing, and 2) recommendations on how to improve or grade the development of writing through short messages are offered, expressed in reflexive questions or suggestions. The way each of these actions works is explained in detail below.

Inform the student about the use of their own resources

The first task that the student develops when logging into the platform is the diagnostic assessment, whose objective is to characterize the participant in two aspects: competencies in academic writing and levels of writing self-regulation. To that end, two instruments are available: 1) Writing Test and 2) Writing Self-regulation Questionnaire (WSQ).

vWriting test

This test is designed to examine proficiency in academic text writing. This test is applied online and consists of writing an academic text based on a previous reading. The task is designed in the Moodle platform, which allows controlling the time limit (60 minutes).

Writing Self-regulation Questionnaire (WSQ)

The second instrument is the Writing Self-Regulation Questionnaire (WSQ). It seeks to establish the students’ perception of their self-regulation in the development of writing tasks in the planning, writing, and reviewing phases, as well as their levels of motivation and self-efficacy. This test is applied online and assessed on a 6-point Likert scale. At the end of the questionnaire, the system prepares and delivers the result for each dimension to the student through a report that is displayed on the screen. With this report, we expect to increase the participant’s metacognitive awareness on how they perceive their planning, writing, and reviewing processes and on their levels of motivation and self-efficacy.

Recommendations to grade the writing

The second action of the IMA strategy is materialized with the presentation of periodic messages in the platform, named metacognitive activators. The course has a total of 42 messages, which appear throughout all the units, according to the type of task or information presented. The 42 messages are categorized based on two general components of metacognitive awareness: cognition knowledge and cognition regulation; each of them comprises a group of sub-processes and skills (Schraw & Dennison, 1994). Table 1 presents some examples of activators located in their corresponding component and sub-process.

Metacognitive activators appear as tooltips, also called pop-up descriptions. The main purpose of this tool is to make the student aware of how they are carrying out the written production task and give them recommendations to grade the process. In general, these messages are expected to promote the development of a higher level of awareness and focus their attention more on completing the task.

Methodology

Study Design

To validate the tool, a study with an experimental design was proposed. Its objective was to compare the effects of the virtual learning environment according to four possible conditions. The course has two specific strategies: self-regulating writing scaffolding (SWS) and different tools to increase metacognitive awareness (IMA). The effects of these strategies were observed separately and combined.

In formal terms, it was a study with a random sample that followed a 2x2 factorial design, in which the condition of having been exposed to the SWS (yes/no) and that of having been exposed to the IMA strategy (yes/no) is varied. This combination generates four groups. The following diagram explains the design better:

For all groups, self-regulating writing and performance in writing pre-test and post-test were applied

Participants

Forty-six (46) students participated in the study; 25 (54.3%) were enrolled in the Master of Technology program, 13 (28.2%) in the Master of Education program, and 8 (17.3%) in a Ph.D. in Education program, all from a public university in Bogotá, Colombia. The age of the participants ranged between 24 and 60 years old (mean = 39.8, S. D= 11.77).

The 46 participants were randomly distributed into four groups. Each group had 11 students, except for the scaffolding group, which had 13 participants. All the participants were informed of the purpose of the study and gave their consent to be part of the sample. Table 2 describes the distribution of the sample in the four groups formed for the experiment.

Instruments

Writing Pre-test

The writing pre-test measure used was the test designed as part of the assessment tools, whose data, in this case, were analyzed as a covariable of the writing performance dependent variable.

The test groups the results into five broad categories:

• Pragmatic structure

• Preparation and support

• Textual structure

• Grammatical structure and

• Formal aspects

All the texts the participants produced were evaluated by two judges who are academic writing experts. To evaluate the agreement between judges, the correlations of the evaluations were examined, and, in general, they reported very high relationships. In the category of pragmatic structure, the correlation was .985 (p<.001), the categories preparation and support, textual structure, and grammatical structure obtained a value r=.984 (p<.001), and formal aspects had a value r=.977 (p<.001). The correlation between the evaluations of the two judges for the total test was r=.991 (p<.001).

For experimental purposes, the groups that received IMA strategies (activator group and combined group) were informed of their results in the writing test and self-regulation questionnaire before beginning the course. Although the test was applied in the other two groups, students were not told their results.

Writing Post-test

The final writing test consisted of a literature review scientific article on a subject. The rubric designed for the writing pre-test was adjusted to evaluate the articles. The adjustments were made as a function of the type of text that, in this case, is explanatory. In that order of ideas, the five ca- tegories of the first rubric (pragmatic structure, preparation and support, textual structure, grammatical structure, and formal aspects) were kept, but the indicators (15) were adjusted, so they fulfilled the function of describing an explanatory text. Each indicator was scored on a scale from 1 to 10.

The adjusted rubric was examined by two experts in language sciences who made recommendations that were received to make the necessary corrections. On this occasion, all final writing tests were evaluated by two judges who had also evaluated the pre-test. The correlations of the evaluations of both judges were very high and significant in all cases. In pragmatic structure, the correlation was .973 (p<.001); in preparation and support, it was .959 (p<.001); in textual structure, it was .980 (p<.001); and in grammatical structure and formal aspects, the r value was .957 (p<.001). The correlation for the total test was .971 (p<.001).

Writing Self-regulation Questionnaire-WSQ

WSQ was used as a measure of perception about self-regulating writing skills. This instrument is comprised of 56 statements evaluated through a 6-point Likert scale, where (1) corresponds to completely disagree and (6) to completely agree.

The statements corresponded to actions related to writing planning, writing, review processes, and emotional factors, such as the perception of motivation and the sense of self-efficacy regarding this task.

In terms of the instrument’s reliability, the results of the questionnaire as a whole indicate that it is reliable at a very high level (α=.918). Table 3 presents the value of Cronbach’s alpha for each category in the application of the instrument.

Procedure

The participants were randomly assigned into the four groups of the experiment. In every case, the WSQ pre-test and the writing test were applied. Once the initial test section was passed, access to the course contents was given in the corresponding condition.

All the groups received information on the platform about the characteristics of the scientific articles. After that, the three groups where an experimental condition operated accessed specific versions of the course, according to their condition. The course lasted 12 weeks, during which each participant accessed the corresponding course to develop the proposed units and tasks.

At the end of the intervention, each participant answered a self-evaluation survey and received feedback on the work carried out, both from their peers and from the tutor. As post-test measures, the review article was used as a measure of writing performance and the WSQ as a measure of perception of writing self-regulation.

Statistical analysis of the data was carried out using the Statistical Package for Social Science (SPSS) software version 21.0. Statistical tests included the Kruskal-Wallis test, Wilcoxon Test and parametric correlations (Pearson’s r).

Ethical considerations

For this study, we followed ethical protocols related to the voluntary participation of those who were part of the experiment. They were all informed of the objectives and scope of the research, and they read and signed informed consents for the use of the information obtained, clarifying that they could withdraw from the study at any time. Once the experimental phase was completed, participants had access to all versions of the course.

Results

Differences in Course Dedication

As part of the analysis of behaviors the proved to be indicators of students’ self-regulation, the differences between the groups were examined concerning the total hours in the course, the total number of logins into the platform, and global retention rates. Table 5 shows the descriptive statistics of these variables for each one of the four groups and the results of the tests of differences between the groups (H Kruskal-Wallis Test).

Just as it is shown, regarding total hours on the platform, it was found that the groups with the longest time on the platform and with the highest number of logins were the groups with activators + scaffolding and the group of only activators. According to the Kruskal-Wallis test, the differences between the four groups are not significant concerning the time online c2(3)=6.38 p=.094, but they are significant in the number of logins c2(3)=7.85 p=.049.

Regarding the course retention variable, defined as the percentage of students who actually completed the course versus the total of those who started it, the highest percentage of retention is in the group with self-regulating scaffolding, in which, of 13 enrolled, 12 remained until the end (92%). At the other end, the control group obtained the lowest retention rates (27%). The groups of only activators and scaffolding+activators show an intermediate situation. The results indicate significant differences in favor of the group that worked only with the scaffolding c2(3)==10.92 p=.012.

Writing test

The differences in the ranges between the writing pre-test and post-test compared to each condition were examined through the Wilcoxon test (see Table 6). The data show significant differences in all dimensions for the group with only activators and the group with only self-regulating scaffolding. However, it should be noted that in all the groups, there was an increase in participants’ average performance in all writing dimensions.

In the pragmatic dimension, the highest mean of the post-test was reached by the group with the scaffolding (M=16.3; SD=2.42), followed by the course with activators (M=14.9, SD=3.60). In all the courses, it is possible to observe an increase in the mean of the dimension in the post-test; however, the Wilcoxon test analyses only show appreciable levels of significance for the group with activators (p=.028) and the group with the scaffolding (p=.042).

Similar results were found in the preparation dimension, in which the highest average in the post-test was again obtained by the group with the scaffolding, followed by the group with activators. It is possible to establish that the increase in average student performance is statistically significant for the course with the scaffolding (p=.018) and to a lesser extent, but significant, for the course with activators (p=.028).

In the textual dimension, the groups with the highest average are, again, the group with the scaffolding and the group with activators. The Wilcoxon test yielded results of significant differences in both groups (p=.028).

The highest averages for the grammatical dimension were reached, again, in the course with activators. The groups with the scaffolding and with scaffolding+activators reached similar averages, while the control group obtained the lowest average. This dimension shows significant increases in the average performance in the group with activators (p=.028). While the test results for the scaffolding group are not significant, presenting them is worthwhile because of their proximity to the accepted value (p=.058 > 0.05).

For the formal dimension, it was found that the course with the scaffolding reached the highest average performance; on the other hand, the groups with activators and with activators + scaffolding obtained exactly the same average. Again, the control group obtained the lowest results in comparison with the others. The difference between the most significant ranges is observed in the scaffolding group (p=.018); in the activator group, the difference significance value is .028.

In general, when examining the total result of the test, it is observed that the group with the greatest increase in writing performance corresponds to the course with the scaffolding. In addition, the Wilcoxon test analyses again indicate significant levels of differences between the pretest and post-test ranges in the scaffolding course (p=.018) and the course with activators (p=.028). As it can be observed, one of the first conclusions that can be reached with these data is that the conditions that most definitely favored the increase in students’ writing performance were those in which the two strategies for teaching writing self-regulation were tested separately and, specifically, it is the course with self-regulating scaffolding (SWS) that is the most effective. In this regard, it is striking that the course that integrated both strategies did not end up favoring as much as expected.

Perception of writing self-regulation

The results of means and standard errors in and pre-test and post-test are presented in table 6, together with the results of tests of significance of the differences between pre and post-test for each group. The comparative analyses between the WSQ pre-test and post-test means for each of the experimental conditions show some differences that should be considered.

In the motivation scale, there was an increase in the post-test mean, versus the pre-test in the control group, in the group with activators, and in the group with the scaffolding; the differences in the pre and post-test of the groups of activators + scaffolding and control were not significant.

There are no significant differences in the mean of the pre-test and the post-test in the self-efficacy scale. Similar behavior is observed in the data of the planning scale, where only a non-significant slight increase in the post-test mean is observed in the control group and in the review scale in which the pre-test and post-test means are almost the same in every group.

The writing scale presents statistically significant differences between pre-test and post-test for the group with activators (p=.027). There are no significant differences in the control group, the scaffolding group, and the combined group.

In general, it is possible to observe that there is barely any effect of the different experimental conditions on the results of self-regulation perception. Only in two dimensions, motivation and writing, the perception of self-regulation increases, and it only occurs in the groups that experienced the two strategies separately: the group of only activators and the group with only scaffolding. In the other two groups, in scaffolding + activators and in control, there are no significant differences between the pre and post-test in any of the self-regulation dimensions considered.

Analysis of relationships between variables

Finally, the correlations between the total hours in the course, total logins, and the results of the writing post-test were examined. Table 8 presents the associations between the variables. Firstly, it was not possible to establish any significant association between the total hours in the course and the dimensions of the writing posttest. On the other hand, the total logins indicator showed significantly high relationships with the pragmatic dimension (r=.552; p=.004), with the preparation dimension (r=.491; p=.013), and with the total result of the post-test (r=.424; p=.034). In conclusion, the more logins an individual made into the platform and interacted with it, the more the development of writing skills was positively affected.

Two aspects should be highlighted concerning the results of the relationships between the variables of the course dedication and the results of writing performances. Firstly, that all relationships are positive, which means that the higher the number of logins and the longer the writer spent on the course, there is an increase in their performance. Secondly, it is possible to observe that the significant correlations are only present in the total logins into the course, which leads us to think that to improve writing performance, it is more favorable to log in more for shorter periods than fewer logins with very long dedication periods.

Discussion

Research into writing skills teaching, implemented through technological aids or computational tools, has shown that these types of technologies represent an important aid for teachers by providing valid and reliable grades and the possibility of giving feedback to students’ works (Allen, Jacovina, & McNamara, 2016; Lachner, Burkhart & Nückles, 2017). However, one of the main difficulties virtual education programs face is the high dropout rate, a factor associated with limitations in student’s levels of autonomy and motivation, which undoubtedly leads to academic failure.

Some of the main reasons that encourage students to leave these virtual training programs are associated with socio-emotional factors such as the feeling of isolation they experience during the time of interaction with the resource or platform, which has a negative impact if the student does not have enough independence and autonomy to overcome those feelings and complete the process (Chakor & Faddouli, 2016). Additionally, the tutor experiences a task overload by having to assist a large group of students, which impedes the response time versus all possible requests (Dussarps, 2015).

In view of the difficulties described above, numerous studies have demonstrated the effectiveness of the use of scaffolding to favor the development of self-regulating skills in virtual environments (Azevedo, 2005; Azevedo & Cromley, 2004; Graesser et al., 2007; Hederich, López, & Camargo, 2016). In the same line, the results of this study provide important evidence regarding the benefits that the use of self-regulating scaffolding entails on several aspects of the education process.

In the first place, concerning dropout rates, it was found that the conditions in which the two strategies of self-regulation (scaffolding group and activator group) were tested separately significantly decreased the dropout percentage. Specifically, the condition that most favored retention was the self-regulating scaffolding course (92% remained); this indicates that the step-by-step modeling of the self-regulation process largely serves the purpose of decreasing the student’s feeling of isolation during their interaction with the platform by providing a well-structured guide and explicitly teaching text planning, writing, and editing strategies. This result, added to the fact that this same group showed the highest number of hours logged into the course and the highest number of student logins, indicates that this strategy presents important advantages in the development of self-regulation.

A second aspect to consider concerning the effects of the self-regulating scaffolding group on factors associated with self-regulation is the increase in motivation (self-reported) at the end of the intervention. According to previous studies, the development of high levels of motivation toward writing is a very important result of the implementation of this type of teaching program (MacArthur & Graham, 2016). In this sense, the present study confirms the data from previous studies indicating that teaching programs on writing incorporating strategies to increase students’ motivation, such as the explicit formulation of goals, self-evaluation, and the explanation of the usefulness and functionality of the task, produce better results both in writing and in motivation itself (Graham, Harris, & Chambers, 2016).

Regarding achievements reached in writing improvement, the scaffolding group (SWS) also proves to be the most effective for this purpose. According to Harris et al. (2011), modeling is critical to developing effective self-regulating education interventions since it allows demonstrating how and when to use self-regulating strategies throughout the writing process. The results of evidence-based practices have demonstrated that modeling specific actions, such as choosing specific objectives for the writing task, the formulation of a defined and explicit plan, monitoring behaviors, self-evaluation, and selfreinforcement, among others, are determinants to reach higher levels of writing (Graham, Harris, & Chambers, 2016).

On the other hand, the group with metacognitive activators does not seem to have the same potential in guaranteeing students remaining in the course. In that sense, the presentation of guiding messages, recommendations, and advice do not replace the explicit guidance in the process and, therefore, does not diminish the feeling of isolation experienced by the student. However, this condition seems to have achieved an increase in two self-regulating factors: the motivation and perception that students who remained obtained on their writing skills. In this sense, the activator strategy fulfilled the objective of increasing metacognitive awareness in these aspects of writing self-regulation. Among students, the activators course generated a high- er ability to judge their skills and the results of their writing. According to the data, it was evidenced in the motivation increase and a better self-assessment in the writing task.

Regarding the increase in the level of writing performance, the group of metacognitive activators also showed favorable results in all the dimensions examined. The presentation of information associated with knowledge and self-regulation cognition, components of metacognitive awareness, influenced the ability to manage writing processes and to address all the elements required to write a quality text (control over the content, relevance, organization, integrity, grammar, spelling and punctuation, and reference standards). This result is consistent with previous studies demonstrating that providing metacognitive and strategic information through messages and pop-up windows leads to improved writing skills (Berthold, Nückles & Renkl, 2007; Nückles, Hübner, & Renkl, 2009).

In contrast to the good results in writing performance exhibited in the courses that experienced the self-regulating scaffolding (SWS) and metacognitive activators separately, the course that integrated both self-regulating strategies did not make significant progress. As it was possible to observe in the results section, the combined group obtained an important dropout rate (55%), evidenced poor progress in all of the self-regulation scales, showed lower averages in the writing test versus the former two groups, and the difference between their initial state and their final achievement was not significant. Faced with this low achievement panorama, it seems that having too many stimuli present in the platform, expressed on activators that appear permanently on the screen plus the modeling offered by the self-regulating scaffolding, is not favorable for the development of self-regulating skills.

In general, as previously indicated, students who experimented with this course likely faced an excess of information (activators) and demands related to the increase in self-regulating behaviors (scaffolding), causing a decrease in motivation or interest to remain in the course and move forward with the writing task. Therefore, the over-stimulation to which they were subject acted as a hindrance to successful completion in the platform.

An in-depth analysis of the poor results obtained in the combined group, both in self-regulation and in writing performance, leads to the conclusion that the effect of the combination of the two strategies is detrimental to the learning processes. The reason for the negative effect of the combination is the cognitive overload experienced by students exposed to three tasks simultaneously. On the one hand, they have a task with high cognitive demand, which requires the availability of several mental resources for its execution: writing (Kellogg, 1994; Kellogg, 2008; Olive & Piolat, 2005). On the other hand, scaffolding serves as metacognitive support compelling them, to a certain extent, to be aware of the monitoring, control, and reflection processes necessary to be successful. The scaffolding operation is not distracting; it is rather a process modeler. Finally, the metacognitive activators fulfill a warning function; they completely change the attention focus, generating interference in the development of the writing task itself.

From this overview of cognitive complexity, subjecting an inexperienced writer to answer an academic writing task and exposing them to two stimuli with different purposes at the same time (to increase metacognitive awareness and to increase levels of self-regulation) causes an overload in the attentional executive control available in the working memory, reducing the resources available for the successful fulfillment of the goal; it is a result that is reflected in low performance. Hence, it is possible to conclude that the best path is to use the strategies separately, as was the case of the activators group and the scaffolding group (SWS).

This analysis points to the possibility of dividing the components to which the students of the metacognitive activators group were subject. As it is possible to recall, this intervention had two distinct parts. In the first part, some questionnaires are answered, and the system provides a report with initial recommendations. In the second, metacognitive activators appear with some regularity, interrupting and changing the attention focus. It is possible that by eliminating this last action, which could have more of a distracting potential, the application of the questionnaires and the reading of the resulting report will achieve an increase in awareness in addition to that achieved by the use of the scaffolding. This question will remain for our future research.

Concerning the control group, it was possible to observe that it was always lagging behind the results of the other groups in terms of writing performance, and the difference between its initial condition and its final achievement did not prove to be statistically significant.

Indications for future developments

In general terms, the findings of this study have both theoretical and educational implications concerning writing self-regulation and its teaching and the possibilities for future developments. In the first place, a clearer and more elaborate approach to a definition of writing self-regulation that attempts to bring together the differentiating and distinctive factors of the processes involved was reached. In this order of ideas, we propose, as a definition of writing selfregulation, all the controlling and monitoring actions autonomously, voluntarily, and deliberately carried out by the writer to manage and coordinate the interaction between the plan, text generation, and reviewing stages to achieve the stated writing goals. Such actions involve not only cognition but also an individual’s emotions and behavior.

Secondly, concerning the implications of the teaching of self-regulation and following the same line of other research, the analyses of the present study allow evidencing that modeling the self-regulation process through the scaffolding produces three main effects. On the one hand, it offers a greater guarantee of remaining in the course, reducing the generalized problem of online courses of dropouts. On the other hand, the levels of motivation regarding writing are increased, a sign that the value students assign to this task is high, despite the high cognitive demands it represents (MacArthur & Graham, 2016; MacArthur, Philippakos, & Graham, 2016). Finally, there is an evident increase in writing performance in all of the discursive dimensions, which ultimately validates the success of the program.

In addition, it was possible to corroborate that the two conditions of the program separately favor the development of writing skills. Similar to the results reported by Berthold et al. (2007) and Nückles et al. (2009), this study evidenced that the use of metacognitive activators is useful when informing the student about their conditions, recommending actions to grade their process and providing information about different strategies to manage the cognitive processes involved in writing.

Finally, as indications for future developments, the possibility of making the most sophisticated version of the virtual learning environment, including factors such as a more flexible self-regu- lating scaffolding, is posited. In other words, it is suggested that the student can navigate with greater freedom in the course, without all the restrictions that this first version of the tool imposed by presenting a fixed sequential structure.

On the other hand, based on acknowledging the usefulness of metacognitive activators for writing performance, it seems important to recommend that, in their interaction with the self-regulating scaffolding, they be able to be deactivated under the student’s choice, thus reducing the “overstimulation” experienced by the participants in this study.

Lastly, despite the advances made through this study in the field of writing self-regulation, several questions remain unsolved and deserve to be addressed in future research. For example, what measurement strategies or techniques, beyond questionnaires, enable tracking writers’ selfregulating actions operating in computational environments? To what extent would technological developments allow providing feedback on the writing process by intelligent tutors? Are the self-regulating actions or behaviors learned by the writers stable over time, and can they be transferred to writing tasks with different textual typologies? These questions and many others remain to be explored in the future, and their answers will determine the progress of developing more sophisticated tools and resources for writers on training to successfully learn selfregulating skills.

Referencias

Allen, L. K., Jacovina, M. E., & McNamara, D. S. (2016). Computer-Based Writing Instruction. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of Writing Research (pp. 316-329). New York: The Guilford Press. https://files.eric.ed.gov/fulltext/ED586512.pdf

Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning ? The role of self-regulated learning. Educational Psychologist, 40(4), 199-209. https://doi.org/10.1207/s15326985ep4004

Azevedo, R., & Cromley, J. G. (2004). Does Training on Self-Regulated Learning Facilitate Students’ Learning With Hypermedia? Journal of Educational Psychology, 96(3), 523-535. https://doi.org/10.1037/0022-0663.96.3.523

Azevedo, R., & Witherspoon, A. (2009). Self-regulated learning with hypermedia. In D. Hacker, J. Dunlosky, & A. Graesser (Eds.), Handbook of metacognition in education (pp. 319-339). Nueva Jersey: Routledge.

Berthold, K., Nückles, M., & Renkl, A. (2007). Do learning protocols support learning strategies and outcomes? The role of cognitive and metacognitive prompts. Learning and Instruction, 17(5), 564-577. https://doi.org/http://dx.doi.org/10.1016/j.learninstruc.2007.09.007

Budde, S., Glaser, C., & Brunstein, J. (2012). Implementation of self-regulated writing strategies in elementary classes. In M. Torrance, D. Alamargot, M. Castelló, F. Ganier, O. Kruse, A. Mangen, L. Tolchinsky & L. Van Waes (Eds), Learning to write effectively: current trends in European research, Studies in writing series (pp. 71-74). Bingley: Emerald.

Burnham, S. (1994). For writers only. New York: Ballantine.

Chakor, Y., & Faddouli, N.-e. (2016). Abandonment of Learners MOOC Problematic Analysis and proposed solutions. International Journal of Computer Applications, 153(2) 35- 37. https://doi.org/10.5120/ijca2016911977

Cislaru, G., & Thierry, O. (2018). Le processus de textualisation. Analyse des unités linguistiques de performance écrite. Louvain-La-Neuve: De Boeck Supérieur.

Dussarps, C. (2015). L’abandon en formation à distance [Abandonment in distance education]. Distances et Médiations Des Savoirs, 10. https://doi.org/10.4000/dms.1039

Flower, L., & Hayes, J. (1981). A Cognitive Process Theory of Writing. College Composition and Communication, 32(4), 365-387. https://doi.org/10.2307/356600

Graesser, A. C., Wiley, J., Goldman, S. R., O’Reilly, T., Jeon, M., & McDaniel, B. (2007). SEEK Web tutor: Fostering a critical stance while exploring the causes of volcanic eruption. Metacognition and Learning, 2, 89-105. https://doi.org/10.1007/s11409-007-9013-x

Graham, S. (2006). Strategy instruction and the teaching of writing: A meta-analysis. In C. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 187-207). New York: Guilford.

Graham, S., & Perin, D. (2007). Writing next: Effective strategies to improve writing of adolescents in middle and high schools. A Report to Carnegie Corporation of New York. Washington, D.C.: Alliance for Excellent Education.

Graham, S., Harris, K., & Chambers, A. (2016). Evidence-based practice and writing instruction. En C. MacArthur, S. Graham, & J. Fitzgerald, Handbook of writing research (pp. 211-226). New York: The Guilford Press. https://www.guilford.com/excerpts/graham_ch1.pdf?t

Harris, K. R., Graham, S., MacArthur, C. A., Reid, R., & Mason, L. H. (2011). Self-Regulated Learning Processes and Children’s Writing. En B. Zimmerman & D. Schunk, D. (Eds.) Handbook of Self-Regulation of Learning and Performance, (pp. 187-202). New York: Routledge/Taylor & Francis Group.

Hayes, J. R. & Flower, L. S. (1983). Uncovering cognitive processes in writing: An introduction to protocol analysis. In P. Mosenthal, L. Tamor, and S. Walmsley (Eds.), Research on writing: Principles and Methods (pp. 207-220). New York: Longman.

Hayes, J., & Flower, L. (1986). Writing Research and the Writer. American Psychologist, 41, 1106-1113. https://doi.org/10.1037/0003-066X.41.10.1106

Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing. In C. M. Levy & S. Ransdell (Eds.), The science of writing: Theories, methods, individual differences, and applications (pp. 1-27). Lawrence Erlbaum Associates, Inc.

Hederich, C., Camargo, Á., & López, O. (2015). Amadís: un andamiaje para el desarrollo de la autorregulación en la educación virtual [Amadis: a scaffolding for the development of self-regulation in virtual education]. Bogotá: Universidad Pedagógica Nacional - Colciencias.

Hederich, C., Camargo, Á., & López, O. (2018). Andamiajes computacionales para favorecer la autorregulacióndel aprendizaje: una revisión de investigaciones [Computational scaffolding to favor self-regulation of learning: a review]. En N. Arias (Ed.), Aportes a la investigación sobre educación virtual desde América Latina. Comunicación, redes, aprendizajes y desarrollo institucional y social (pp. 117-140). Bogotá: Politecnico Grancolombiano.

Hederich, C., López, O., & Camargo, A. (2016). Effects of the use of a flexible metacognitive scaffolding on self-regulated learning during virtual education. International Journal of Technology Enhanced Learning, 8(3/4) 199-216. https://doi.org/10.1504/IJ-TEL.2016.082321

Kellogg, R. (1994). The psychology of writing. Nueva York: Oxford University Press.

Kellogg, R. T. (2008). Training writing skills: A cognitive developmental perspective. Journal of Writing Research, 1(1), 1-26. https://doi.org/10.17239/jowr-2008.01.01.1

Koster, M., Tribushinina, E., de Jong, P. F., & van den Bergh, H. (2015). Teaching children to write: A meta-analysis of writing intervention research. Journal of Writing Research, 7(2), 249-274.

Lachner, A., Burkhart, Ch., & Nückles, M. (2017). Formative computer-based feedback in the university classroom: Specific concept maps scaffold students’ writing. Computers in Human Behavior, (72), 459 - 469. https://doi.org/10.1016/j.chb.2017.03.008

Lamb, B. (1997). Booknotes: America’s finest authors on reading, writing,and the power of ideas. New York: Random House.

Limpo, T., Alves, R., & Fidalgo, R. (2013). Children’s high-level writing skills: Development of planning and revising and their contribution to writing quality. British Journal of Educational Psychology, 84, 177-193. https://doi.org/10.1111/bjep.12020

MacArthur, C. A., & Graham, S. (2016). Writing research from a cognitive perspective. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 24-40). New York: The Guilford Press. https://www.guilford.com/excerpts/macarthur.pdf?t

MacArthur, C. A., Philippakos, Z. A., & Graham, S. (2016). A multicomponent measure of writing motivation with basic college writers. Learning Disability Quarterly, 39(1), 31-43. https://doi.org/10.1177/0731948715583115

Malpique, A., Veiga, A., & Frison, L. (2017). Self-regulated strategies for school writing tasks: a cross-cultural report. Psychology of Language and Communication, (21)1, 244-265. https://doi.org/10.1515/plc-2017-0012

McCormick, C. B. (2003). Metacognition and Learning. In I. B. Weiner, D. K. Freedheim, W. M. Reynolds, J. A. Schinka, & G. E. Miller (Eds.), Handbook of Psychology: Educational Psychology (pp. 79-102). New Jersey: John Wiley & Sons.

Negretti, R. (2012). Metacognition in student academic writing: a longitudinal study of metacognitive awareness and its relation to task perception, self-regulation, and evaluation of performance. Written Communication, 29(2), 142-179. https://doi.org/10.1177/0741088312438529

Nückles, M., Hübner, S., & Renkl, A. (2009). Enhancing self-regulated learning by writing learning protocols. Learning and Instruction, 19(3), 259-271. https://doi.org/10.1016/j.learn-instruc.2008.05.002

Olive, T., & Piolat, A. (2005). Le rôle de la mémoire de travail dans la production écrite de textes [The role of working memory in written production]. Psychologie Française, 50(3), 373-390. https://doi.org/http://dx.doi.org/10.1016/j.psfr.2005.05.002

Palermo, C,. & Thomson, M. (2018). Teacher implementation of Self-Regulated Strategy Development with an automated writing evaluation system: Effects on the argumentative writing performance of middle school student. Contemporary Educational Psychology, (54), 255-270. https://doi.org/10.1016/j.cedpsych.2018.07.002

Rogers, L., & Graham, S. (2008). A meta-analysis of single subject design writing intervention research. Journal of Educational Psychology, 100(4), 879-906. https://doi.org/10.1037/0022-0663.100.4.879

Scardamalia, M., & Bereiter, C. (1987). The psychology of written composition. Hillsdale. NJ: Lawrence Erlbaum.

Scardamalia, M., & Bereiter, C. (1986). Written composition. In M. Wittrock (Ed.), Handbook on research on teaching. New York: McMillan.

Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26(1-2), 113-125.

Schraw, G., & Dennison, R. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 460-475. https://doi.org/10.1006/ceps.1994.1033

Torrance, M., Alamargot, D., Castelló, M., Ganier, F., Kruse, O., Mangen, A., Tolchinsky, L., & Van Waes, L. (2012). Learning to write effectively: current trends in european research. Iwa: Emerald.

Wood, D., Bruner, J., & Ross, G. (1976). The role of tutoring in problem solving. Journal of child Psychology and Psychiatry, (17) 2, 89-100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x

Zimmerman, B. (2011). Motivational sources and outcomes of self-regulated learning and performance. En B. Zimmerman, & D. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 49-64). New York: Routledge.

Zimmerman, B., & Campillo, M. (2003). Motivating self-regulated problem solvers. In J. E. Davidson & R. J. Sternberg (Eds.), The psychology of problem solving (pp. 233-262). New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511615771

Zimmerman, B. J., & Risemberg, R. (1997). Becoming a Self-Regulated Writer: A Social Cognitive Perspective. Contemporary Educational Psychology, 22(1), 73-101. http://dx.doi.org/10.1006/ceps.1997.0919

IR