INTRODUCTION

The growing offer of non-linguistic subjects taught in English as a Medium of Instruction (henceforth EMI) in Higher Education (henceforth HE) has become a reality in the last three decades for two main reasons (Wilkinson & Zegers, 2006). First, the internalization of HE as a strategy of economic development and social recognition (Dearden, 2015) has brought the implementation of EMI courses. Second, the increasing number of academic publications focused on research exploring EMI teaching in HE has dramatically increased: 63 out of 83 research studies since 1995 were published between 2011-2015 (Macaro, Curle, Pun, An & Dearden, 2018). However, despite the popularity of EMI, current practices may still leave significant questions behind, as is the case of assessment.

This research aims at presenting and discussing the most relevant challenges regarding EMI assessment in a middle-sized (i.e., more than 12,000 students) private University in Madrid, Spain. So, this study acknowledges

• The growing trend towards EMI in HE and explains the main features of the institutional context;

• The results of the questionnaire administered to teachers and the two focus groups carried out;

• The main challenges of EMI assessment will be outlined as compared to Content Language Integrated Learning (henceforth CLIL).

LITERATURE REVIEW

EMI teaching in HE differs from other approaches, which also use the target language as the medium of instruction; for example, CLIL. The main differences between EMI and CLIL point to the following:

• There are no clear language policies or guidelines in HE settings with respect to EMI teaching (Dearden, 2015);

• There might be a lack of students´ knowledge of English as a Foreign Language (henceforth EFL), mostly at entry levels (Hellekjær, 2005); and

• CLIL does take into account the integration of content and language, while EMI apparently does not (Dafouz, 2011; Kirkpatrick, 2014).

For that reason, it is more usual to find CLIL practices in Elementary and Secondary Education rather than in HE. This is so because linguistic competence is a basic skill to be accomplished along with content knowledge.

If the CLIL approach consists of a dual-system of content and language integration (Coyle, Hood & Marsh, 2010), the assessment of academic subjects should also consider these two aspects: content and language. In the case of CLIL –and apparently EMI programs, in which language proficiency may not be considered a prerequisite for students but it is rather developed at the same time that content knowledge is learnt, formative assessment is especially relevant when used in combination with other student-centered methodologies (Ball, Kelly & Clegg, 2015): first, because of its task-based nature and the wider variety of classroom interaction that it promotes; second, it can also help to motivate students to develop a positive attitude towards content along with a simultaneous improvement in the vehicular language performance.

In contrast, EMI programs usually take language proficiency for granted, and thus, there is no agreement on how to integrate content and language aspects, i.e., accurate and transparent assessment criteria for EMI subjects is likely to be less feasible (Macaro, Curle, Pun, An & Dearden, 2018). It is then worth considering that assessment is never neutral but it does have an influence on the teaching and learning process. According to Alderson and Wall (1993), tests having important consequences usually influence teaching and learning regarding the following: what and how teachers teach; what and how students learn; the rate and sequence of teaching and learning; the degree and depth of teaching and learning; and attitudes referring to content, methodologies, etc. of teaching and learning. In fact, there is extensive research about the effects of standardized testing on students’ outcomes and motivation Chapelle & Douglas, 1993; Shohamy, 1993; Alderson & Wall, 1996; Bailey, 1996; Cheng, 1997; Cheng, Watanabe & Curtis, 2004) among other factors. Conversely, assessment, which does not have important consequences, will have no washback effect (Weir, 1990; Alderson & Wall 1993; Alderson 2004) at all.

Therefore, the EMI teaching routines examined in this article can be only met if “assumptions underlying assessment practices are empirically tested.” (Wilkinson & Zegers, 2006: 61). Therefore, this study reflects on how to improve the quality of EMI teaching throughout the display of the shortcomings in assessment in a specific HE context in Spain.

METHODOLOGY

This research considers EMI teachers´ assessment practices in a medium-sized private University in Madrid, Spain, during 2017/18. For this purpose –and taking into account regulations about assessment in HE and recommendations about EMI assessment–, the study aims to answer the following research questions:

1. 1. How is EMI assessment conducted?
2. 2. What assessment tools do EMI teachers use?
3. 3. What is the role of language in EMI subjects?

This is a mixed-methods research dealing with both quantitative and qualitative data in order to provide a full picture and a complete understanding of the information collected (Johnson, Onwuegbuzie & Turner, 2007) by two means: on the one hand, quantitative data in the form of a validated questionnaire (Otto, 2018) were collected and analyzed; on the other hand, qualitative data through focus groups were collected to follow up or refine results from quantitative data. Thus, the teachers' questionnaire was aimed to gather general information about teachers' training on EMI, the primary assessment tools they use and the influence that language aspects have on the final grade (if any). In relation to the focus groups, they were conducted with the primary goal of clarifying some of the questions in the questionnaires, and to give teachers the opportunity to comment on any relevant issues about assessment.

To assure the reliability of the study, the questionnaires and the focus groups were designed around pointed and relevant questions, making the study apt to be replicated in the future. Although the study is considered representative of the actual conditions in which assessment takes place, it does not attempt to generalize results to other EMI contexts, but rather to describe the context in detail for recommendations regarding assessment practices.

The participants in this study (N=22) are all teachers working at the University examined. They are specialists in several areas such as Business and Management, Social Science and Education, to name just a few. Most of them (95%) are Spanish native speakers, so they need to certify a B2 level of EFL. However, the proficiency language levels may also vary significantly once they have access to the teaching of EMI subjects as is common in other contexts (Klaasen & Räsänen, 2006).

RESULTS

First, the results of the questionnaire show that the majority of the EMI teachers (68%) have received no training on how to teach content subjects through EFL. Those who have received EMI-related training (32%) have done so through seminars or courses. The training mostly happened while pursuing university studies. In no case did the EMI teachers specify that the University has offered them the opportunity to get trained through internal seminars or courses.

With reference to the frequency of assessment, 72% of participants prefer to assess students “regularly.” In contrast, only 14% or the respondents consider assessing students “every day” and through “mid-semester and final examinations only”, as seen in Figure 1. Nevertheless, the frequency of assessment was not clarified in the questionnaire as EMI teachers express, they assess students in a “continuous way.” However, the fact that the teachers assess the students´ work in a continuous way does not imply that they follow a formative assessment approach:

It is also noticeable that the majority of EMI teachers (77%) would need more guidance about assessment since the University provides no specific instructions. Concerning those teachers receiving advice on EMI assessment, one asserts that she received instructions to measure content knowledge over language issues (Teacher 1, personal communication, February 13, 2018). She is also unsure of whether sometimes the students fail the tasks because of content knowledge or language impeding them to express accurately. Another teacher states that she gives one out of ten marks to language issues (Teacher 2, personal communication, February 13, 2018). In opposition, one teacher reveals that she never grades language errors because she will also make some mistakes occasionally (Teacher 3, personal communication, February 13, 2018). In general, EMI teachers claim that they do not feel responsible for the (correct use of) language so they do not assess it.

In terms of rubrics, most teachers (59%) do not use rubrics in order to assess EMI tasks. Nonetheless, there is also a significant number of teachers (41%) who do use rubrics. Among the answers about what kind of rubrics the teachers use, one teacher explicitly points to analytic rubrics where two components are assessed: language and content knowledge. However, although one of the participants refers to the fact that the English language is weighted up to 10% out of the final grade, the application of analytic rubrics is not made clear by the remaining respondents in the focus group. This comment might indicate that the EMI teacher in question uses some rubric that integrates both content and language.

Among the assessment tools, the most usual ones among EMI teachers are written exams (100%), oral presentations (91%), written works (86%), debates (86%) and classroom observation (77%). As for the latter, which is highly rated, most of the teachers clarify that this is considered as “participation” according to the University particular assessment standards. In contrast, few teachers use self-assessment (18%) and peer-assessment (9%) techniques. Since the assessment criteria are pre-determined by the University itself, teachers have no “voice” about the weight of the examinations on the final grade:

In relation to the weight of EFL in EMI assessment, there is a clear preference among teachers (48%) to consider language up to 25% out of the students´ final grade. It is also worth highlighting the number of teachers (29%) who decide not to consider the use of EFL in order to assess the students. Less than a fifth of the teachers (19%) consider language aspects between 25-50% (19%) and very few teachers (9%) claim that they range it over 50%:

Additionally, only very few teachers (18%) think that English obstructs the achievement of the academic objectives. In other words, almost all teachers (82%) think that English cannot be considered as an obstacle to the achievement of EMI subjects, it might imply that the teachers do not consider any other sort of teaching apart from lecturing in which students do not participate actively.

The strategies teachers use to compensate for the fact that the subjects are taught in EFL have been classified into several categories, being scaffolding and language scaffolding the most prominent ones as seen in Figure 4. Language scaffolding deserves closer attention due to the relevance of language for the expression of content knowledge. Some of these strategies include offering extra-vocabulary, examples and synonyms, repetitions and what is termed as “language support.” Furthermore, one teacher also stresses that, in some cases, she has found that code-switching is beneficial when used in combination with translation techniques so that students can notice the typical EFL patterns that will allow them to construct academic discourse (Teacher 3, personal communication, February 13, 2018). This strategy can also be beneficial for students in order to grasp academic writing conventions, which are usually challenging for them. Finally, translation is also paramount for the same respondent as she indicates that she generally translates technical words into Spanish (Teacher 3, personal communication, February 13, 2018), although the students are not allowed to use Spanish in examinations: Scaffolding (N=5); language scaffolding (N=4); content reinforcement (N=2); translation (N=2); additional material (N=1); CLIL (N=1); external aids (N=1); extra aids (N=1); self-reflection tools (N=1); subject option: L1 (Spanish) (N=1).

In conclusion, teachers have received no internal training regarding EMI in general and EMI assessment in particular. Moreover, the assessment tools are pre-determined by the University itself, written examinations being the most regular ones. Even though teachers recognize that EFL cannot be considered as an obstacle for the subjects' goals, EMI teachers acknowledge the importance English has to get access to content knowledge and are capable of expressing it accurately.

DISCUSSION

The discussion serves to give answers to the research questions: How is EMI assessment conducted? What assessment tools do EMI teachers use? What is the role of language in EMI subjects? Moreover, in the course of the research, it was also necessary to analyze the type of EMI training and the strategies used by teachers in order to compensate for limited language proficiency.

One of the most significant conclusions of this study is that most teachers (68%) are not adequately trained on EMI-related methodologies. We can distinguish two main teacher profiles regarding EMI subjects:

1. 1. Teachers with a language background or who have received some training on integrating language in content subjects; and
2. 2. Teachers with non-language related background who have not received any specific training on EMI teaching.

Given the results, it may be necessary to properly train EMI teachers on how to best understand the challenges that the vehicular language can pose for EMI students, and how to make this vehicular language feasible so that they can effectively deal with academic literacies (Wilkinson, 2013). In particular, those teachers whose university background is non-language related point out the massive effort that teaching EMI subjects implies for them: “teaching English for a non-native speaker is an additional effort, mainly in a Spanish context […] Mentally, it is an extra effort.” (Teacher 3, personal communication, February 13, 2018)

Furthermore, EMI practices show the lack of integration of content and language aspects and the lack of responsibility on the part of the teachers for student's proficiency in EFL. However, although teachers do not seem to be responsible for language-related aspects, they do apply different teaching strategies in an apparent attempt to facilitate students' access to content:

• Those regarding general scaffolding aimed at developing content knowledge, which does not explicitly have to do with attention to the foreign language;

• As teachers are aware of the challenges when tackling subjects in EFL, language scaffolding tries to compensate for the students' limited language proficiency (Pavón, 2018).

However, some teachers are not entirely aware of the full potential of scaffolding when approaching EMI teaching practices. For example, one of the EMI teachers refers to “the CLIL approach” (Teacher 1, personal communication, February 13, 2018) as part of the language scaffolding necessary for mental content processing. Therefore, this EMI teacher distinguishes between CLIL and EMI teaching, maybe considering that CLIL relies not only on content delivery but also on language acquisition. The following sections give an answer to the three research questions: first, how is EMI assessment conducted?; second, what assessment tools do EMI teachers use?; and finally, what is the role of language in EMI subjects?

Summative assessment takes priority over formative assessment

The assessment practices for EMI subjects in the context of analysis mostly considers summative assessment, despite that formative assessment aims at motivating students to bridge the learning gaps (Black & William, 1998), which is especially crucial for subjects taught through EFL. This fact is explained as follows:

• The University does not have (any) specific assessment criteria or have adapted itself to EMI teaching since they tend to use the criteria for the same subjects taught in Spanish (students´ L1), which is mainly summative;

• EMI teachers are restricted to pre-established assessment standards that take into consideration the final examination, which leaves little room for improvement and adaptation of the general assessment criteria to different subjects. The Degree Plan emphasizes the mid-term and final examinations over any other tasks to measure students´ learning outcomes. It seems that in the attempt to follow formative assessment teachers opt for continuous assessment, as both terms are often treated interchangeably;

• EMI teachers do not consider assessment rubrics. The majority of them (59%) do not apply specific rubrics to class activities, and some teachers seem to be unsure what a rubric is. Those teachers who use rubrics for EMI subjects prefer not to assess the students´ level of EFL since the undergraduate students´ language proficiency is usually not appropriate enough.

As for the latter, some teachers complained about the students EFL level as is also common in other European contexts (Hellekjær, 2005). The disparity of levels within the same group of learners affects those who have a low-middle level as well as the students with a higher level because they may be even left behind. Therefore, one teacher states that she is quite reluctant to use rubrics in order to avoid students from “losing marks” (Teacher 4, personal communication, February 13, 2018).

Same assessment tools used for EMI and non-EMI subjects

Another significant outcome of this investigation is that EMI teachers do not adapt assessment tools to EMI subjects. The lack of training on EMI has a direct effect on the assessment practices, which are typical for monolingual and bilingual itineraries. Thus, no methodological changes are implied such as curricular adaptations and new methodological approaches but rather the foreign language is merely accommodated into the subject. Furthermore, the lack of EMI teaching training and guidelines might also have an impact on how the content is delivered and how students perceive the subjects.

The lack of training could lead to failure as regards the general and specific competencies and the learning objectives required for EMI subjects. Even though the majority of the EMI teachers (77%) stated that the University does not impose upon them any specific assessment criteria for EMI subjects, the lecturers would instead use no specific guidelines (73%) or rubrics (59%).

Furthermore, there are also no definite criteria to assess EMI subjects, neither from a general perspective as to the relationship between the percentages of content and language, nor even for EMI activities, which mostly consist of written examinations (100%) and oral presentations (91%) according to the questionnaire.

The role of language in EMI settings

In relation to the role that language plays in EMI settings, although teachers recognize the fact that studying in another language is very challenging for students, it seems that language does not play a significant role for teachers, and it does not have an adverse effect on subject goals. In general, students' difficulties in EFL are due to their limited language proficiency when they access HE (Pavón, 2018) given the fact that, the University does not have a clear policy for entry levels, which should be very beneficial (Halbach, Lázaro & Guerra, 2017).

Moreover, teachers do not feel compelled to deal with language-related aspects for two main reasons: first, teachers lack the expected language awareness (Andrews, 2007) they need to address language issues effectively; and second, teachers tend to feel insecure about their foreign language proficiency as they consider they are not “language experts”.

In short, EMI teachers are aware of the shortcomings that they may have concerning their knowledge of English as a vehicular language as well as the students´ lack of language proficiency in the context of analysis. In order to overcome those difficulties, EMI teachers employ the strategies they consider useful to learn content, but many of them also lack the training to do so.

CONCLUSION

The study has assisted to observe first-hand the current practices taking place in a real higher education context as regards EMI assessment, as well as teachers' perceptions on the most common challenges they face daily. However, no possible changes could be considered unless the University formally recognizes what teaching in a foreign language implies:

• The students’ entry level once they access a bilingual Degree. Considering the students’ prior language knowledge requires the creation of standard entry tests, which measure language ability following valid criteria for students to be able to express content knowledge, i.e., students being capable of receiving and producing language in academic contexts;

• The teachers’ training and the design of clear EMI guidelines for assessment. Separate guidelines for subjects are needed considering that teaching EMI subjects involves more than merely teaching in a foreign language: knowledge of specific academic discourse, language teaching-learning strategies and collaboration with the Language Department; and

• Assessment tools should be aimed at developing not only content knowledge but also language skills as EMI teaching is expected to cover subject knowledge and language acquisition.

Due to the impact assessment has on teaching and learning, more attention should be paid to the way EMI assessment is currently conducted in the context of the study. Besides, more efforts need to be addressed to raise awareness among practitioners on how to integrate content, language, and skills.

Acknowledgments

The authors wish to acknowledge the participation of EMI team members for their commitment and invaluable contributions to the development of this study.

REFERENCES

Alderson, J. C. (2004). Foreword. In L. Cheng, Y. Watanabe & A. Curtis (Eds.), Washback in language testing: Research contexts and methods. London: Lawrence Erlbaum.

Alderson, J. C., & Wall, D. (1996). Editorial in language testing, 13(3), 239-240.

Alderson, I. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115-129.

Andrews, S. (2007). Teacher language awareness. Cambridge, UK: Cambridge University Press.

Bailey. K. M. (1996). Working for washback: A review of the washback concept in language testing. In Language Testing, 13(3), 257-279.

Ball, P. Kelly, K., & Clegg, J. (2015). Putting CLIL into practice. Oxford: Oxford University Press.

Black, P. & William, D. (1998). Assessment and classroom learning. In Assessment in education: Principles, policy & practice, 5(1), 7-74.

Chapelle, C. A. & Douglas, D. (1993). Foundations and directions for a new decade of language testing. In D. Douglas & C. Chapelle (Eds.), A new decade of language testing research (pp. 1-22). Arlington, VA: TESOL Publications.

Cheng, L. (1997). How does washback influence teaching? Implications for Hong Kong. In Language and Education, 11(1), 8-54.

Cheng, L., Watanabe, Y. & Curtis, A. (Eds.). (2004). Washback in language testing: Research contexts and methods. Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers.

Coyle, D., Hood, P., & Marsh, D. (2010). Content and language integrated learning. Cambridge, UK: Cambridge University Press.

Dafouz, E. (2011). English as the medium of instruction in Spanish contexts. In Y. Ruiz de Zarobe, J. M. Sierra, & F. Gallardo del Puerto (Eds.), Content and language integrated learning. Contributions to multilingualism in European contexts. Berlin: Peter Lang, 189-209.

Halbach, A., Lázaro, A., & Guerra, J. (2013). La lengua inglesa en la nueva universidad española del EEES. In Revista de Educación, 362, 105-132.

Hellekjær, G. (2005). The acid test: Does upper Secondary EFL instruction effectively prepare Norwegian students for the reading of English textbooks at colleges and universities. University of Oslo, Oslo.

Johnson, R., Onwuegbuzie, A., & Turner, L. (2007). Toward a definition of mixed method research. In Journal of Mixed Method Research, 1(2), 112-133.

Kirkpatrick, A. (2014). The language(s) of HE: EMI and/or ELF and/or multilingualism?’ The Asian Journal of Applied Linguistics, 1(1), 4-15.

Klaasen, R. & Räsänen, A. (2006). Assessment and staff development in higher education for English-medium instruction: A question-raising article. In R. Wilkinson & V. Zegers (Eds.), Bridging the assessment gap in English medium higher education (pp. 235-255). Fremdsprachen in Lehre und Forschung 40. Bochum, Germany: AKS Verlag.

Macaro, E., Curle, S., Pun, J., An, J., & Dearden, J. (2018). A systematic review of English medium instruction in higher education’. In Language Teaching, 51(1), 36-76.

Otto. A. (2018). Assessment in CLIL: The balance between the content and the language. Madrid bilingual secondary schools as a case study. Unpublished PHD thesis, Alcalá University, Spain.

Pavón, V. (2018). La controversia de la educación bilingüe en España. In Tribuna Norteamericana, 26, 21-27.

Shohamy, E. (1993). The power of tests: The impact of language tests on teaching and learning. Washington, D.C.: The National Foreign Language Center at John Hopkins University.

Weir, C. (1990). Communicative language testing. New York: Prentice Hall.

Wilkinson, R. & Zegers, V. (Eds.). (2006). Bridging the assessment gap in English medium higher education’. Fremdsprachen in Lehre und Forschung 40. Bochum, Germany: AKS Verlag.

Wilkinson, R. (2013). English-medium instruction at a Dutch university: Challenges and pitfalls. In A. Doiz, D. Lasagabaster, & J. M. Sierra (Eds.), English-medium instruction at universities: Global challenges (pp. 3-24). Bristol: Multilingual Matters.

Notas de autor

joseluis.estrada@uca.es

Información adicional

How to cite: Otto, A., & Estrada Chichón, J. L. (2021). Analysing EMI Assessment in Higher Education. Revista Tempos e Espaços em Educação, 14(33), e15475. http://dx.doi.org/10.20952/revtee.v14i33.15475

Authors' Contributions: Otto, A.: conception and design, analysis and interpretation of data, drafting the article, critical review of important intellectual content; Estrada Chichón, J.L.: conception and design, acquisition of data, critical review of important intellectual content. All authors have read and approved the final version of the manuscript.