Abstract: This paper explores the possibilities offered by new Artificial Intelligence tools when applied with Error Analysis to English as a Foreign Language (EFL) teachers to design assessments that improve their students’ written proficiency in the context of a university course at the B2 level of the Common European Framework of Reference (CEFR). A comparative corpus study was designed to evaluate the impact of integrating both disciplines, in which half of the productions were evaluated with artificial intelligence tools. Relevant conclusions were obtained, as new intelligent technologies bring new perspectives that effectively help EFL learners produce more appropriate written texts. Among other findings, error elimination was observed in more than 70% of the cases and a better application of grammatical rules in more than 75%.
Keywords: education, language learning, linguistics, second languages, curriculum.
Resumen: Este trabajo explora las posibilidades que ofrecen las nuevas herramientas de Inteligencia Artificial al ser aplicadas con el Análisis de Errores a los docentes de Inglés como Lengua Extranjera (EFL) para diseñar evaluaciones que mejoren la competencia escrita de sus estudiantes en el contexto de un curso universitario de nivel B2 del Marco Común Europeo de Referencia (MCER). Para evaluar el impacto de la integración de ambas disciplinas se diseñó un estudio comparado de corpus en el que la mitad de las producciones se evaluaron con herramientas de inteligencia artificial. Se obtuvieron conclusiones significativas, ya que las nuevas tecnologías inteligentes aportan nuevas perspectivas que ayudan eficazmente a los estudiantes de EFL a producir textos escritos más apropiados. Entre otros hallazgos, se observa la eliminación del error en más de un 70 % de los casos y una mejor aplicación de reglas gramaticales en más de un 75 %.
Palabras clave: aprendizaje de lenguas, educación, evaluación de la educación, lengua extranjera, lingüística.
Error Analysis and Artificial Intelligence: Preliminary exploration of English as a Foreign Language written productions[1]
Análisis de Errores e Inteligencia Artificial: Evaluación preliminar de la producción escrita en Inglés como Lengua Extranjera
Recepción: 03 Agosto 2023
Aprobación: 13 Octubre 2023
Natural Intelligence (NI) is understood as the type of intelligent behaviour humans display, whereas Artificial Intelligence (AI) is usually defined as the capabilities and abilities machines exhibit to replicate complex behaviours and communicate with humans (Grewal, 2014). Recently, we have witnessed numerous scientific advances in AI and its applicability to human learning (Markauskaite et al., 2022; Shin, 2018; Shin et al., 2021; Viberg, 2022). The direction of these advances in the discipline seems to indicate the direction desired by theorists and practitioners: the full integration of the capabilities shown by machines in the different teaching-learning processes that humans experience in their day-to-day lives, with a strong interest in the development of machine-human communication systems (Silva, 2018).
Currently, science considers that devices that perform complex tasks successfully have some degree of AI, even more so when the device adapts to different possibilities and response patterns without much margin for error (de Lemos & Grzes, 2019). In other words, newer artificial intelligences initiate functions similar to cognitive functions usually attributed to human thinking (Silva, 2018), such as communication, problem-solving capabilities, and, more recently, learning and response (Magid et al., 2022).
Different study areas in AI focus on aspects of the discipline. A pre-eminent area within this new discipline is Natural Language Processing (NLP) (Zhang et al., 2020). This broad area of research is particularly interested in machine translation and machine-human-machine communication. Another relevant research area is Object Recognition Studies (ORS) (Daniel, 2018; Kumar, 2021), which today focuses on the recognition of physical objects (such as parked cars or moving objects) and is often linked to human-machine-human communication on a basis: beeps and warning sounds (Viberg, 2022). Although both areas show an interest in human-machine-human communication, as can be seen, different areas of AI address aspects of human daily life.
A novel research area uses AI theories and concepts applied to human education. Within educational studies, one of the most prominent approaches to applied AI involves understanding this discipline as a tool to simplify the process of managing, directing, planning, and conducting a class (Göçen, 2020), as well as a way aimed at increasing teaching effectiveness and curriculum success (Yunus & Rajendran, 2021). Such a line of research connects to Error Analysis (EA), a discipline framed within Applied Linguistics, L2 Studies, and Language Acquisition and Development Studies (Livingstone, 2011; Kumar, 2021). The link between AI Educational studies (AIEd) and EA lies in the existing relationship between error identification tools and devices designed to recognize and deal with learners’ failures (such as Trinka.ai or Grammarly.com) (Macías, 2023; Torben & Strasser, 2022).
Consequently, the present study explores the possibility that these new AI tools can improve English learners’ written proficiency as a second language (L2) in a context where Spanish as L1 predominates. Samples are collected from students of a university English language course at CEFR level B.1.3 with 70 students enrolled. In a scenario such as the one described, one of the most notable reasons for failure and dropout of the subject is the inability to substantially improve written proficiency (Cabrera, 2014; López Urdaneta, 2011).
Consequently, the text addresses whether the union of the theoretical background underlying the linguistic discipline of EA, one of the most prominent tools for improving writing proficiency in EFL, and new intelligent technologies can provide fresh perspectives and strategies that contribute to the effectiveness in the production of written texts, enabling the selected learners (study population) to produce more proper and natural written outputs.
In contemporary linguistic theory, error no longer equates to equivocation (Corder, 1967; Corder, 1993; Selinker, 1972). Nowadays, errors are considered linguistic features that indicate the efficiency and effectiveness of the language teaching-learning process (Johansson & Hofland, 1994). The EA approach explores and explains why a given mistake is made at a precise moment in the teaching-learning process (Richards & Renandya, 2002). The possibilities show EA linked to the practice of foreign and second language teaching, turning this discipline into paramount for curriculum design success.
Thus, the ability to understand and deal with errors can benefit the teaching-learning process on several levels: assessing student performance, allowing educators to observe success and failure in the teaching-learning process, and exploring ways to implement more efficient curricula that may permit students to avoid making more errors and, on the contrary, produce better linguistic outcomes (Clerk & Rutherford, 2010; Macías, 2023).
One of the most notable contributions of EA is the setting up and theorization of the interlanguage hypothesis. Selinker presented it in 1969 when he spoke of "the provisional grammatical structure constructed by second language learners on their way to the target language" (as cited in McLaughlin, 1987, p. 69). However, William Nemser (1971), an expert in Comparative and EA, offered the most widely accepted definition of the concept of interlanguage conveying that "the learner's speech at a given time is the patterned product of a linguistic system distinct from the mother tongue and the target language and internally structured" (1971, p. 116).
Following Corder (1967) and Selinker (1972), McLaughlin (1987) describes the central cognitive processes of second and interlanguage learning on which, to some extent, the practical application of EA rests (Brown, 1994; Gass & Selinker, 2008): (a) language transfer and interference from the learners’ native language (L1); (b) overgeneralizations of rules in the target language, and (c) misconceptions of rules and usages in the target language.
CALL (Computer Assisted Language Learning) was introduced in linguistics and engineering studies in the 1950s; however, it took some time to become an effective teaching-learning tool (Chapelle, 2003). The basis of the CALL approach to second language teaching and learning is simple but compelling: Computers and computing devices can aid in second/third language learning if designed correctly (Levy, 1997).
The CALL approach remained a research field for some time before teachers, instructors, and learners used it in teaching and learning (Levy, 1997). Experts in the field claim that during the 1990s, the potential of CALL began to be applied (Chapelle, 2003). The underlying reason for the late use of CALL perspectives in teaching and learning can be found in the perceived lack of dissemination of the necessary technology to homes and educational institutions; schools, universities, students, and families had no access to computer resources in the early stages of the discipline’s development in the 1960s and 1970s (Schulze, 2011). It is often argued that the 1990s brought about the present technological revolution and facilitated the rise of personal computer-based learning systems (Chauhan et al., 2022).
CALL, as an approach and methodology, and AI applied to L2 studies have benefited from the theoretical background and academic debates, not only around computer science (Popovic & Ney, 2022) but also EA, error handling, and the interlanguage hypothesis (Jodaei, 2012). Recently, the effective integration of computer-assisted feedback in EA has been a fertile topic of discussion in the research field (Macías, 2023).
In this regard, recent research (Cabrera et al., 2014; Magid et al., 2022; Steadman & Kraut, 2018; Wobst & Lueg, 2022) claims that modern technology allows learners to practice and obtain feedback on their written and oral skills. It demonstrates that modern devices have incorporated effective technology required for testing processes. Therefore, CALL is a helpful tool for analyzing errors, as it contributes to identifying trends that ultimately lead to making mistakes (or avoiding them).
In the 1950s, one of the most notable personalities in applied AI, Alan Turing, defined what can be unequivocally understood as an intelligent system or device (French, 2000). Turing posed what is known today as Turing's imitation game: If a human listener cannot distinguish whether their speaker is a machine or a human, then we can speak of an artificially intelligent device/system/tool (Popenici & Kerr, 2017).
Concerning the contemporary situation for education studies, AI is a familiar concept and tool. One of the most striking recent contributions to integrating AI and education studies (linked to EA) is found in Luckin’s studies (2010). The expert claims that AI can support the development of technologies that enhance learning by minimizing process errors and maximizing the chances of positive outcomes. According to Luckin, AI-assisted models allow learners to work at their own rhythm and enable teachers and instructors to witness their students' processes from an external and objective perspective. It is also reported that the model helps teachers and instructors intervene whenever necessary and in varied modalities, not only face-to-face but also through distance tutoring (Underwood & Luckin, 2011).
Recent research (Kessler, 2021) affirms that modern technologies (especially those capable of processing and analyzing texts and oral productions) and experiences with the latest technological and communicative improvements (social media and virtual reality) allow teachers and instructors to effectively grasp the new dimensions of reality and the challenges of their students. Hence, Kessler (2021) argues that all the knowledge gained through these experiences helps teachers to produce a more individualized teaching-learning process by considering the specific cognitions and concrete needs of a particular learner. This has an overt impact on the feedback process: the more information the teacher can gather, the better knowledge they will have of the specific situation; it should result in better feedback that can have a dual, computer-assisted, and teacher-based nature (Chaudhry & Kazim, 2021). Thus, AI-based tools and data collection should help to develop better and more personalized teacher-student support and to achieve success in the learning process.
Today, it is undeniable that digital tools positively affect various educational processes and contribute to improving and boosting the system (Al-Fequi, 2012). AI applications, tools, and devices favor changing the roles played by educational actors and agents (education centers, teachers, managers, administrative staff, students, etc.). According to Dickson (2017), these new technologies will entirely and drastically change the interactional patterns between teachers/instructors and learners/students, as machines will offer an interactive educational solution to traditional problems. Dickson also states that these new technologies and AI tools can solve the problem of large-group interaction, which is especially important when providing feedback, improving student performance, and enhancing attitudes toward the teaching-learning process (2017).
Nowadays, teaching English as a second and foreign language benefits enormously from the introduction of AI, in particular, and Information and Communication Technologies (ICTs), among others. Language classrooms are very artificial spaces for learning, and communication and interaction often need to be improved. However, when these communicative and interactive situations are granted, learners can practice real-life skills (Haupin, 2016). As such, introducing technologies capable of simulating reality and forcing learners to communicate—imposing real-life simulated difficulties—can benefit the process as a whole (Dickson, 2017). These new apps, programs, and devices implement and enhance communication skills; through communication programs, conversational skills can be stimulated by introducing accurate, realistic, virtually interactive, and hands-on language training (Rabah, 2020).
Radwan (2020) has identified and described the various uses of AI to overcome difficult classroom situations and improve the teaching-learning process. According to him, the combination of educational tools and AI can:
Be used to develop and enhance the ability to comprehend reading passages.
Develop and enhance learners' translation skills using machine-assisted translation.
Help learners improve pronunciation via automatic speech recognition tools.
Help break down barriers for visually and hearing-impaired learners using text-to-speech tools.
Improve writing proficiency through writing assessment techniques, registers, and automated handwriting correction tools.
On the one hand, the present study aims to evaluate and determine, preliminarily, the usefulness of introducing AI tools in teaching English to know the impact of these on written proficiency in a university course of English as a foreign language whose target level is B2 (upper intermediate). On the other hand, in a more practical approach to the discipline, we intend to explore the possibilities that AI offers for the feedback and assessment of learners of English as a foreign or second language. Our objective is to understand if AI helps the monitored learners to produce better writing (more appropriate to the pursued objective) and, therefore, if it improves the group’s teaching-learning competencies.
The identified problem to be researched constitutes the fundamental motivation of this study. Written competence and performance are often disconnected from daily practice in English as an L2, and this situation follows a recurrent and considerable delay in the feedback given to learners.
The starting point of this study implies that AI-based tools offer almost immediate and interactive feedback, and some of them enable and enhance the automation of feedback logs. Thus, we intend to demonstrate that combining EA with new intelligent technologies gives rise to possibilities for substantially improving the written production of language learners, particularly those learning English as a foreign and second language. It will be achieved by examining if there is a strong connection between these new tools and their feedback possibilities and improvements in students' written proficiency when assessed with AI-based tools.
This study attempts to answer the following questions addressing the described problems to fulfill the objectives:
What are the most appropriate strategies and methods for employing AI in teaching/learning English in a university course?
Is AI effective in developing different English teaching/learning processes and measuring their results?
Is it feasible to use AI tools to improve written proficiency?
Do learners perceive AI tools as a viable and enriching alternative to traditional feedback?
A group of 70 B.1.3 level English learners constitute the population of this research (performance and proficiency) in a single class at a public university in Madrid.
Students were asked to produce a text similar in number of words and topic to those they usually develop in the classroom; specifically, to write an opinion essay on a free issue of approximately 250 words, collected and anonymized by the teacher. Samples for this research were obtained by applying the spontaneous sample collection procedure, including collecting data from students when they focus more on content than grammatical style. Academic assignments and papers, tests, and essays can be taken as sources for data collection. Topics through the spontaneous procedure may include "personal information, future plans, religious, social and political issues, and the like" (Keshavarz, 2022, p. 80). The population participated 100%.
To meet the objectives, we designed a methodology based on a practical five-step method, which draws on the methodology of corpus and corpora linguistic studies. The following steps are essential to understand how this study was conducted and, if desired, to replicate it:
Step 1: creation of the first study corpus or Corpus 1. Seventy writing samples were collected from university English learners with an approximate proficiency in the target language B.1.3. The group members possess varying degrees of EFL proficiency, and approximately 65/70 have produced a language that meets the descriptors and criteria of the target level.
Step 2: division of Corpus 1 into study units. Once the samples are collected, they are first anonymized using labels to avoid possible interference (Kaiser et al., 2021; Zaki & Nosofsky, 2007), and second, they are randomly divided into two groups (35 writing samples in each group). These constitute the principal units of study.
Step 3: analysis of the study units. For each of these, one type of analysis is applied. One group (unit 1) is scanned via AI technology, capable of processing and evaluating texts at the grammatical and lexical level (Trinka.ai was used in this study). The second group was handled more traditionally: correction by the teacher with pen and paper but with the same level of analysis and detail.
Step 4: creation of the confirmation corpus or Corpus 2. This group is a mirror image of Corpus 1: seventy samples of written output by the same target population. Here, the main objective assesses (in a preliminary way) the possibilities offered by AI-based proofreading and text processing compared to a more traditional teacher-led proofreading system: It is essential to create a confirmation corpus that allows us to understand if and when errors are repeated. The analysis focuses on the repetition of and the nature of mistakes (Gass & Selinker, 2008; Rutherford, 2022).
Step 5: exploratory contrastive analysis of Corpus 1 and Corpus 2. Once Corpus 2 has been examined in detail, it is necessary to contrast it with the results of Corpus 1. This contrast allows us to inspect whether the AI minimizes the errors produced, as Kessler (2021) suggested.
The results of this preliminary research are obtained after the contrastive analysis of Corpus 1 and 2 described in the methodology and research design.
Built on the corpus study methodology, it has shed light on how AI coupled with EA can form an effective tool within AIEd. It has also answered some questions about the effectiveness of AI-based tools in the process of assessment and feedback to learners of English as a foreign or second language.
On the one hand, AI has proven to be an effective tool in correcting EFL learners' writing. If we look closely at the results of both units of study, we will observe that the percentage of repeated errors (present in the same learner in both units of study) is lower in the AI-based correction system. Only 25% of the students who received AI-based feedback repeated in their second writing the errors found in the first writing performed, compared to 40% of those who received teacher-based feedback.
The relationship between the participation system and the distribution of error repetition and error correctness can be seen in Figure 1:

Secondly, this work has contributed to understanding the nature of the most frequent errors in the group and their distribution. The different nature of the errors committed by the students indicates the deviations produced in the teaching-learning process. This circumstance has essential curricular implications since knowing the nature and distribution of learners' errors helps teachers/instructors to (a) become aware of them; (b) find ways to help learners overcome them, and (c) evaluate the curriculum to ensure elimination of the errors and avoid their repetition and fossilization (Murad & Mahmood, 2018; Rao, 2018).
From another perspective but with the same intention, contemporary linguistic theory asserts that one of the many natures of the most frequent errors in the language learning process could be attributed to Negative Language Transfer (NLT) or interference of the learners' and trainees' native language in their EFL production/output (Gass & Selinker, 2008; Goker, 2021).
Regarding this study, it is essential to consider both perspectives of error: As an indicator of the outcome of the teaching-learning process and as a sample of the internal process that every learner experiences when learning a language effectively, are that both are included in the design of the programs and syllabi of the subjects to minimize the impact of negative transfer.
The results indicate that negative transfer between languages is a common obstacle in all group members and that the distribution of this type of error is widespread and frequent in both groups: 72% of errors in the AI-assisted modality and 77% in the AI-based system. Besides, 77% in the teacher-based system can be attributed to negative transfer; these results are consistent with other recent research suggesting that NLT remains the leading source of error fossilization for EFL and ESL learners (Arabski, 2006; Yunus & Rajendran, 2021).
The results related to the impact of negative transfer can be observed in Figure 2:

Another significant finding of this study is the high level of satisfaction of the participants and the high degree of inferred and perceived efficacy. In this regard, all participants were asked (via an anonymous survey) about three key questions: (a) to what extent they were satisfied with the scoring process and its results; (b) to what extent they considered effective to have been evaluated in the way they were (AI-assisted or teacher-assisted), and (c) whether they would have preferred to have been selected for the opposite group. Sixty-eight responses were obtained for a total student population of 70 students; therefore, with a response rate of 97%, the results for this group-class can be inferred. Figure 3 illustrates these results:

With these results, it can be inferred that the combination of both disciplines, AI applied to Education (AIEd) and EA, brings positive and significant results regarding error elimination (prevention of fossilization) and learner satisfaction. Applying AIEd techniques and using AIEd tools in the EFL classroom also offer the possibility of personalizing the teaching-learning process through the individual and immediate collection of each learner's errors.
Through the introduction of so-called AIEd models, capable of processing written and oral texts, teachers and learners can easily create diaries of development and approximation to the target language and level, a feature already incorporated to a greater or lesser extent in the different AI tools currently available and of automatic production through memory (storage of written texts) and recognition of patterns and trends (EA).
Although including these models (AIEd) is undoubtedly a great advantage, we must not forget that the existing tools are still under development and have limitations and shortcomings when understanding human language in its complete dimension. Certainly, these aids allow understanding and correcting written productions at the grammatical, lexical, and structural levels, with little margin of error; however, their capacity for semantic and pragmatic comprehension is still constrained, and both areas will require the intervention of a teacher (Hoppin et al., 2023; da Silva, 2022).
Therefore, it can be concluded that although the AIEd and CALL models represent an attractive and helpful approach for including EA and the prevention of error fossilization in the EFL and L2 classroom, in their current release, they cannot fully take care of neither the teaching-learning process nor the analysis and correction of texts. However, they would serve as indicators of success or failure of curricula and teaching actions by presenting errors distributed by patterns and trends, and by showing themselves as elements of improvement in learner motivation and satisfaction.
This study aims to explore, preliminarily, the opportunities that AI tools and their integration with EA can provide in the teaching-learning process of English as a second and foreign language to Spanish L1 university students. With the results and findings, we can infer that the combination of both disciplines, AI and EA, favors the development of a personalized teaching-learning method that students and learners perceive as satisfactory and, likewise, introduces the theoretical background of both disciplines in the daily teaching practice.
By incorporating AIEd models capable of processing, teachers and learners can maintain developmental and performance records to analyze and observe patterns of error repetition, error fossilization, and error elimination. Thus, it can be argued that AIEd approaches constitute active, engaging, and successful models and lessons that overcome linguistic difficulties such as negative transfer (the principal source of error in the addressed population). The elimination of errors and, consequently, the prevention of fossilization foster better adaptation of production to the rules of the target language, i.e., progress and success (Alderson, 2005).
However, it is also important to remember that new technologies suffer from limitations and are currently more capable of evaluating and correcting linguistic production from a grammatical and lexical point of view, whereas other essential elements in the language learning process, such as semantics and pragmatics, still require human-based correction (Allerton et al., 2003; Silva, 2018). Therefore, the higher the target level in the target language, the higher the degree of teacher-based feedback, even if technology allows mapping part of the teaching-learning process onto AIEd systems and devices, an aspect linked to Mann and Welsh's reflection on the so-called Fourth Industrial Revolution:
The rapid expansion of technology and digital applications that characterizes the “Fourth Industrial Revolution” is changing the way we live, work, and learn. It is a revolution driven by the fusion and amplification of emerging advances in artificial intelligence, automation, and robotics, and multiplied by far-reaching connectivity among billions of people with mobile devices that offer unprecedented access to data and knowledge. (Mann & Welsh, 2017, p. 4)
AIEd and the CALL approach benefit from advances in new technologies and theories of applied linguistics, such as EA, but currently cannot fully take over the teaching-learning process, as AI is unable to develop semantics or pragmatics and, consequently, cannot correctly interpret written and spoken productions of humans at complex levels (Kasirzadeh & Gabriel, 2021; Mahmood, 2021; Rapaport, 2005; Steels, 2022).
Despite their limitations, digital tools and AI help develop learning methods and strategies and should be considered when designing curricula for English as a foreign language (but also for other languages) that translate the reality of learners and students into the classroom, increasing their motivation and perceived satisfaction with their learning process.
Conflict of interest: The author claims do not have a conflict of interests.


