All studies on error correction were retrieved with a combination of error, correction, written, writing, feedback, grammatical, grammar, and accuracy. An online database search was also conducted (e.g., ProQuest, ERIC, JSTOR, Google Scholar, Scopus) as well as a journal search (e.g., Journal of Second Language Writing, System, Language Learning, International Journal of Applied Linguistics). A manual search of referenced relevant articles in previous meta-analyses was also conducted (e.g., Kang & Han, 2015).

The online and manual search resulted in 146 studies. Studies considered for this analysis were based on empirical efforts that targeted grammatical accuracy alone or in combination with other surface or textual-level issues to assess the short and long-term impact of feedback. For critical discussion purposes, it is important to note that the inclusion criteria also covered studies that did not include reports or a control group.

The analysis did not include: (1) unpublished MA theses or doctoral dissertations, (2) descriptive studies on learner views about error corrections, (3) studies investigating only oral CF, (4) studies that addressed written errors of learners exclusively with oral CF strategies, comments, or praise and criticism, (5) studies analyzing teacher response or learner response correctness, (6) studies regarding the differential effect of peer and teacher feedback, (7) studies in which feedback was provided strictly by peers or (8) studies endorsing grading methods. After appraising the articles based on the inclusion/exclusion criteria, a total of 76 articles were selected to answer the research question that guided this systematic analysis.

Feedback purpose refers to what the action of giving feedback actually promotes. Although written CF has been researched extensively for decades, the rationale behind examining improved accuracy in written CF has varied over the years (Polio, 2012). A useful distinction exists between feedback for accuracy and feedback for acquisition (see Manchón, 2011). Based on this classification, written CF promotes: I) immediate accuracy (i.e., feedback for accuracy); 2) the development of accuracy or study of second language acquisition (SLA) processes (i.e., feedback for acquisition) and 3) both short and long-term accuracy (i.e., feedback for accuracy and acquisition). Furthermore, research on written CF has mostly focused on feedback for acquisition studies (cf. Figure 1).

3.1.1. Feedback for Accuracy

Many studies focus solely on improving accuracy and improvement shown on the final rewritten version after feedback was given on the original draft (e.g., Fathman & Whalley, 1990). Two major characteristics are helpful in identifying studies about feedback for accuracy. One is their research agenda, which refers to the learning-to-write dimension (cf. Manchón, 2011), meaning that feedback for accuracy studies are directed at L2 writing with the goal of analyzing how learners can improve their self-editing and revision skills (Ferris, 2010). The other is design. Studies on feedback for accuracy refer to metrics for improved accuracy of revised texts (Bitchener, 2009) after comparing two versions of the same text. Although these studies are worth highlighting, since they are L2 writing oriented, some were conducted in classroom-based settings at the expense of methodological rigor. Lack of a control group proved to be the main shortcoming (e.g., Ferris & Roberts, 2001), raising concerns about the need for research designs that are both methodologically acceptable and pedagogically feasible (cf. Bitchener & Ferris, 2012).

Figure 1
Number of Written CF Studies Based on Their Purpose
Source: Elaborated by author

Brown (2012) defines feedback scope as “the number and type of errors that are addressed-either with a comprehensive approach or a focus on a limited range of error categories” (p. 863). Specifically, different error types have been targeted in written CF studies, such as grammatical and non-grammatical errors (e.g., van Beuningen et al. 2012). Nevertheless, despite the type of error, written CF may include focused (i.e., selective) CF or unfocused (i.e., comprehensive) CF. Literature has generally referred to focused CF feedback as one that addresses “a single or a limited number” (Stefanou & Révész, 2015, p. 264) of linguistic categories, usually one or two. On the other hand, unfocused CF has been traditionally understood as feedback that corrects “all (or at least a range of) errors” (Ellis et al., 2008, p. 356). In a recent meta-analysis of written CF studies, Liu and Brown, (2015) proposed a classification that distinguished the scope by the exact number of targeted errors: highly focused (i.e., highly selective) (1 error type), mid-focused (i.e., mid selective) (2-5 error types), and unfocused CF (i.e., comprehensive) (> 5 error types). Table 1 shows the number of studies addressing any one scope type either in single form or by comparing them.

Table 1
Number of Written CF Studies per Scope

Elaborated by author

Feedback type deals with the strategies used for providing written CF. The reviewed literature refers to eight different strategies which have been implemented as grouping feedback variables (i.e., in combination with others) or as single feedback variables (i.e., unsupplemented). Adapted from Ellis (2009), Table 2 lists the strategies available to correct learners’ written errors and briefly describes them.

Table 2
Overview of Written CF Strategies

Source: Adapted from Ellis (2009)

With respect to feedback strategies, the status of the field reveals that some have received more empirical attention than others. In Figure 2, direct corrections comprise the most examined strategy whereas concordances together with check marks and models have been largely overlooked.

Figure 2
Percentage of Strategy Use in Written CF Studies
Source: Elaborated by author

Feedback content refers to the issues that are the emphasis of the CF treatment. Written CF has been mostly conceived as feedback on linguistic issues. For this reason, studies on learner response to grammar correction are more prevalent in literature. Storch (2010) argues that “it is feedback on language use, termed written corrective feedback (WCF), which seems to have attracted the most research attention recently” (p. 30). Nevertheless, to a lesser degree the bulk of written CF studies has also included research that examines grammatical accuracy improvement by including groups that receive additional grammar instruction (e.g., Frantzen, 1995) or feedback on content (e.g., Fathman & Whalley, 1990; Fazio, 2001; Kepner, 1991; Sheppard, 1992). Other studies have treated grammar issues together with organization and content (e.g., Stiff, 1967) or lexical issues (e.g., Lalande, 1982). The content in feedback studies thus far can be seen in Figure 3.

Figure 3.
Number of Written CF Studies by Content
Source: Elaborated by author

Feedback frequency refers to the number of written CF sessions in a given study. Although there is no agreed-upon definition, Storch (2010) is the only researcher that makes a clear distinction regarding duration: a one-shot treatment vis-à-vis a sustained treatment. The author refers to one-shot design CF studies as those that give CF on one occasion only whereas sustained ones are those that give CF on more than one writing text.

Based on this definition, sustained treatment studies are more common (see Figure 4), especially in early CF studies. They provided CF over the course of a term and (coincidentally) also targeted multiple errors (e.g., Chandler, 2003; Ferris, 2006). More recent sustained CF studies target either a few linguistic categories (e.g., Ferris, Liu, Sinha, Senna, 2013) or a broader range of categories (Lavolette, Polio, Kahng, 2015). Although the one-shot treatment was less prevalent in early studies (Fathman & Whalley, 1990; Ferris & Roberts, 2001), it has become the most popular design for SLA-oriented feedback studies and focuses either on a narrow number of preselected error categories (e.g., Stefanou & Révész, 2015) or on a large array of categories (Santos et al., 2010).

Figure 4.
Number of Written CF Studies by Frequency
Source: Elaborated by author

Another variable that causes wide disparity in CF studies is the type of writing assignment that is assessed which ranges from dicto-gloss tasks (e.g., Shintani et al., 2014) and journal entries (e.g., Fazio, 2001) to picture-based stories (Frear & Chiu, 2015) and fables (e.g., Farrokhi & Sattarpour, 2012). Figure 5 displays writing tasks in written CF research out of which three elicitation tasks stand out: picture-based narratives, picture-based description and argumentative texts.

Upon interpreting the results, the broad gamut of elicitation tasks is a significant variable to consider. Murphy and De Larios (2010) argue that readers should remember that tasks can vary in terms of the restrictions imposed on learners and the levels of cognitive complexity of the tasks. For example, although both picture-based stories and essays are time-compressed tasks, they differ in “that the former [is] less cognitively complex in terms of conceptualization and linguistic encoding demands than the latter” (p. vii). This is also the case for picture description tasks which may vary with regards to the complexity of the pictures or the degree of expected precision in the description (Hanaoka & Izumi, 2012).

Figure 5
Number of Written CF Studies by Elicitation Task
Source: Elaborated by author

The seventh source of variation pertains to the individual responsible for addressing the written errors of the student. Although studies on researcher-initiated feedback are predominant in research literature, the teacher is not the only source of written CF available to learners. Classmates, friends, tutors, researchers, the actual writers, and software programs (Ferris, 1997) may also be considered as additional sources of feedback. In our research, there are studies where the individual who provides feedback is a teacher (e.g., Abedi, Latifi, & Moinzadeh, 2010), a teacher-researcher (e.g., Ahmadi, Maftoon, & Mehrdad, 2012), or a software program (e.g., Lavolette et al., 2015), among others. Figure 6 summarizes current feedback sources in feedback literature.

The role of the corrector or the person responsible for correcting learner errors has been a matter of interest since early discussions (e.g., Hendrickson, 1978) and is still an underexplored and underreported variable. The feedback source matters since it could play a role in interpretation of the results. For example, Fazio (2001) considers that an affective factor concerning the feedback source plays a role in feedback outcome. As the author acknowledges, there might not have been a feedback effect because corrections were provided by a teacher other than the familiar one, “which may have had an effect on the manner in which students reacted to their feedback” (p. 245).

Figure 6
Number of Written CF Studies by Source
Source: Elaborated by author

Feedback medium refers to the different response modes to a learner’s written output. Overall, the method for delivering written CF is pen-and-paper compositions, face-to-face interaction (e.g., conferencing) or computers. Most empirical evidence on the effects of written CF on learners’ revised and/or new texts originate from studies that researched the differential effect of various CF techniques and were delivered in the same manner, namely on paper (e.g., Bonilla et al., 2017) (see Figure 7). Also, learners’ written inaccuracies have been addressed with combined pen-and-paper compositions as well as conferencing (e.g., Bitchener & Knoch, 2009a, 2009b). Computers, which according to Hyland and Hyland (2006), allow synchronous (i.e., in real time) or asynchronous (i.e., delayed) communication, have also been used to observe learners’ written grammatical accuracy, though to a lesser degree. Computers have been used to generate or mediate CF such as automated text evaluation (e.g., El-Ebyary & Windeatt, 2010; Nagata, 1997), for example, via concordance files (e.g., Gaskell & Cobb, 2004) or Google Docs (e.g., Shintani & Aubrey, 2016).

Figure 7
Number of Written CF Studies by Medium
Source: Elaborated by author

Number of Written CF Studies by Medium

Source: Elaborated by author

Advances in technology and, particularly, computer-assisted CF, have made it possible for research to be conducted in other feedback medium not available before (e.g., effect of feedback timing). Extensive research by authors (Lavolette et al. (2015), Shintani (2015), Shintani and Aubrey (2016)) now provides evidence of the effect of synchronous CF (i.e., feedback while learners compose a text) as well as asynchronous CF (i.e., feedback after text completion). Novel studies by the aforementioned authors compare differing feedback conditions provided via the same medium (i.e., via computer). Yet, less empirical data can be obtained, regarding the differential effect of various feedback modalities (but see Sheen, 2010).

Additional variation is observed from student response to written CF. Feedback processing refers to how students use the feedback that is provided. As Ellis (2009) explains, students can be asked to deal with written CF in three different ways: (1) respond to the feedback and revise the text, (2) address the feedback without making corrections or (3) refuse to address the feedback (i.e., receiving CF with no further action required). Figure 8 shows the extent that feedback processing form has been implemented in feedback literature.

Figure 8.

Figure 8
Number of Studies per Feedback Processing
Source: Elaborated by author

Differences in learner variables provide more sources of variation in written CF studies. Evans, Hartshorn, McCollum and Wolfersberger (2010) define learner variables as those “that the learner brings to the learning experience” (p. 448). Learner factors include “age, language aptitude, memory, learning style, preferences, personality, motivation, language anxiety, learner beliefs” (Ellis, 2010, p. 339), “nationality, cultural identity, learning style, values, attitudes, beliefs, socioeconomic background” (Evan el at., 2010, p. 448), “personality … goals and expectations, motivations, past learning experiences, preferred learning styles and strategies, content knowledge and interest, time constraints, attitudes towards the teacher, the class, the content, the writing assignment” (Goldstein, 2004, p. 67), and “reactions to teacher feedback and their investment in the course” (Hyland & Hyland, 2006, p. 88). Unfortunately, few learner factors have been addressed empirically and tend to be underreported. An example is learner’s (language) proficiency level, acknowledged as a crucial learner variable that could determine the extent of feedback effectiveness (e.g., Bitchener & Knoch, 2010; Ferris, 1999). Although it has been suggested that written CF should be provided based on learner proficiency level (e.g., Guénette, 2007), it is not always possible to identify a participant’s entry level proficiency (n = 34) and, if stated, (n = 42), it is not always reported correctly. The problem then lies in the L2 teachers’ inability to find practical applications in their classes and L2 writing or SLA researchers to replicate studies as emphasized by Guénette (2007).

The aspect that has received the most empirical consideration in literature regarding written CF is learner’s attitude towards written CF (e.g., Bonilla et al., 2017) (see Figure 9). Other aspects include educational level (e.g., van Beuningen et al., 2012), language analytic ability (e.g., Benson & DeKeyser, 2018), grammatical sensitivity and knowledge of metalanguage (e.g., Stefanou & Révész, 2015) where only the last three were examined as mediating variables.

Figure 9
Number of written CF Studies Addressing Learner Factors
Source: Elaborated by author

The last source of variation is the setting for instructional written CF. According to Ellis (2010), “contextual variables include … macro factors related to the setting in which learning takes place (e.g., immersion, foreign language [FL], and L2 settings)” (p. 341). The most common macro distinction for written CF studies are SL and FL contexts (Ellis, 2010; Murphy & de Larios, 2010). Figure 10 depicts various written CF settings. Most research includes studies conducted with ESL learners in the United States (USA) (e.g., Hartshorn et al., 2010) or in other English-dominant settings such as New Zealand (e.g., Bitchener, Young, & Cameron, 2005), Australia (e.g., Storch & Wigglesworth, 2010), Hong Kong (e.g., Lee, 1997), or Canada (e.g., Qi & Lapkin, 2001). There are also FL studies in the United States conducted with students learning Spanish (e.g., Frantzen, 1995;Kepner, 1991), German (e.g., Semke, 1984) and Japanese (e.g., Nagata, 1997).

Figure 10.
Number of Written CF Studies by Macro-Context
Source: Elaborated by author

Problem: A few feedback for acquisition studies require revision.

This may be understandable when the ultimate goal is the acquisition of L2 forms (cf. Ferris, 2004). Nevertheless, two reasons can be discerned regarding the importance of including revision in a design: they are the theoretical and pedagogical relevance for L2 writing practitioners. According to Ferris (2010), L2 composition teachers have not lost interest in determining strategies that help learners effectively edit and revise their work. Revision may also be useful for learners to acquire an L2 (Ferris, 2004) and may provide written CF researchers with more insight to further substantiate results. Therefore, literature may benefit from studies with blended designs to fill the void between theory and practice and yield results that could be pedagogically and theoretically relevant for two lines of inquiry: L2 Writing and SLA.

Problem: Until now, a large bulk of written CF studies have taken place in English-dominant environments.

This is particularly problematic because “feedback is an area of work that affects all writing teachers and their students, [and as such] it is important that literature be augmented by research studies conducted in different parts of the world” (Lee, 2014, p. 2). It is paramount to stop assuming that FL learners outside and in the United States as well as SL learners in the United States and other English-dominant contexts are comparable (cf. Ferris, 2010). Clearly, L2 knowledge, motivation, and learning experiences (among others) that these types of learners bring to L2 writing may be quite different, potentially influencing how they engage and benefit from feedback. Not surprisingly, Evans et al. (2011) add that the effectiveness of feedback may vary to such an extent that what works well in one context may not necessarily work well in another one. That is why further studies in underrepresented geographical areas are necessary to move this field forward and gain knowledge about feedback practices for particular settings.

Problem: A considerable number of studies do not consider the communicative purpose of an elicitation task (as a potential influential factor of feedback outcome) as well as its ecological validity (as a factor to determine the applicability of research findings).

An example related to the communicative purpose of the elicitation task is journal writing, which (contrary to its very intention) is thought to be discouraging if corrected. As Guénette (2007) explains, its purpose is “to encourage students to write and foster a comfortable and positive writing environment” (p. 47). Hence, targeting learners’ written errors with such tasks may be counterproductive. Second, findings that emerge from a given study may also be analyzed from the perspective of their ecological validity and the extent to which the task itself is applicable. If tasks are not part of the curriculum, since they are meant for research purposes only (e.g., Bitchener & Knoch, 2010; Sheen, 2007; van Beuningen et al., 2012), the ecological validity of the elicitation task could be listed as another variable that hinders transferring findings from a laboratory-like setting to the actual L2 writing class context. Consequently, it is recommended to assign tasks that reflect those usually assigned in an L2 class and that allow freedom of expression (cf. Bruton, 2009a).

Problem: Key design variables are vaguely reported or not mentioned at all.

A basic requirement for seeking useful answers to other instructional contexts is rigorous reporting of all variables. Nevertheless, research does not always reveal who provides feedback, who the learners are, how they write and receive the feedback, what they will do with it, and what they gain from engaging with it. These information gaps prevent firm conclusions to be drawn about the effectiveness of written CF. Along these lines, Guénette (2007) explains that,

rather than interpret the conflicting results as a demonstration of the effectiveness or ineffectiveness of corrective feedback on form, I suggest that findings be attributed to the research design and methodology, as well as the presence of external variables beyond the control and vigilance of the researchers. (p. 40)

The importance of reporting all relevant individual and contextual factors in detail must, therefore, be addressed to obtain realistic and accurate conclusions.

Problem: A large bulk of the research base focuses only on accuracy outcomes.

Responding to L2 learners’ written errors is far more complex than drawing conclusions exclusively from product data. Ferris et al. (2013) argue that “[viewing] student texts in isolation do not provide researchers or teachers enough information about if/how/why WCF helps student writers improve (or does not). Rather, there must be careful consideration of the larger, classroom characteristics, the teacher and the learners themselves” (p. 324), which implies that further research is required to gain more insight into what learners bring to the feedback experience (cf. Evans et al., 2010) and how that could impact feedback outcome. Therefore, empirical feedback studies that add a qualitative component to its design and address learner factors that have been neglected in empirical feedback studies such as learner proficiency level (Bitchener, 2012) would be a valuable addition to the research base.

Problem: The measures for accuracy and design characteristics bear large discrepancies.

Part of the research on written CF includes studies that lack a control group (e.g., Asassfeh, 2013), used differing genres for pretest and (immediate and delayed) posttest comparisons (e.g, Diab, 2015), omitted tracing errors (e.g., van Beuningen et al., 2008) and had learners receive feedback without requiring that it be processed it in any way (e.g., Fazio, 2001). Nevertheless, researchers have described what the ideal design must have if the effectiveness of written CF is to be measured. Bitchener (2008) explains that posttest measurements are only valid if tasks are comparable with the pretest; the author then warns that varied genres must not be used. Other requirements include a control group and delayed posttest to be able to measure retention. Along similar lines, Bruton (2009b) adds that learners must be able to use the feedback received with targeted features tracked in subsequent writing, Thus, future studies have important issues to resolve in their attempt to measure accuracy without methodological criticism.