Abstract: Digital technology, as one of the elements of the Knowledge Society, assumes a determining role in educational systems, where the role of teachers is fundamental. Along these lines, different institutions support the importance of teaching digital skills, proposing models and conceptual frameworks. In them, the knowledge and skills that teachers must develop are classified, combining technological, professional, organizational and pedagogical capacities, with different dimensions and descriptors. The objective of this article is to compareand evaluate the feasibility of the DigCompEdu European Digital Competence Framework for Teachers (JRC) and the Common Framework for Teaching Digital Competence (INTEF). To do this, it is decided to carryout a theoretical reflection on said frameworks and a subsequent Delphi study with the cofficient of expert competence (CEC). 335 people participated, of which 275 (82.1%) were considered experts (CEC≥0.8). The results indicate that, although they are very similar, the European DigCompEdu Teaching Digital Competence Framework is the most endorsed by experts. Finally, the idea that both are viable proposals to develop teaching digital skills through personalized training itineraries is presented.
Keywords:Teaching digital competenceTeaching digital competence,DigCompEduDigCompEdu,EvaluationEvaluation,DelphiDelphi.
Resumo: A tecnologia digital, como um dos elementos da Sociedade do Conhecimento, assume um papel determinante nos sistemas educativos, onde o papel do professor é fundamental. Nesse sentido, diferentes instituições defendem a importância do ensino de competências digitais, propondo modelos e marcos conceituais. Neles,classificam-se os conhecimentos e as competências que os professores devem desenvolver, combinando capacidades tecnológicas, profissionais, organizacionais e pedagógicas, com diferentes dimensões e descritores.O objetivo deste artigo é comparar e avaliar a viabilidade do DigCompEdu European Digital Competence Framework for Teachers (JRC) e do Common Framework for Teaching Digital Competence (INTEF). Para tal, opta-se por realizar uma reflexão teórica sobre os referidos enquadramentos e posterior estudo Delphi com o coeficiente de competência do perito (CCP). Participaram 335 pessoas, das quais 275 (82,1%) foram consideradas especialistas (CCP≥0,8). Os resultados indicam que, embora sejam muito semelhantes, o European DigCompEdu Teaching Digital Competence Framework é o mais endossado por especialistas. Por fim,apresenta-se a ideia de que ambas são propostas viáveis para desenvolver o ensino de competências digitais por meio de roteiros de treinamento personalizados.
Palavras-chave: Ensino de competência digital, DigCompEdu, Avaliação, Delphi.
Educação e Tecnologia
Comparative European DigCompEdu Framework (JRC) and Common Framework for Teaching Digital Competence (INTEF) through expert judgment
Quadro Comparativo Europeu DigCompEdu (JRC) e Quadro Comum para o Ensino de Competência Digital (INTEF) por meio de opinião de especialistas
Julio Cabero-Almenara cabero@us.es
Juan Jesús Gutiérrez-Castillo jjesusgc@us.es
Antonio Palacios-Rodríguez aprodriguez@us.es
Julio Barroso-Osuna jbarroso@us.es
Recepción: 09 Octubre 2020
Aprobación: 04 Enero 2021
Publicación: 08 Marzo 2021
One of the radical transformations that has taken place in education has come as a consequence of the strong presence that Information and Communication Technologies are having in these institutions. But, unfortunately, this presence has not always been accompanied, regardless of educational level, by training plans (OTERO; MORANTE; LÓPEZ, 2018; SUÁREZ-RODRÍGUEZ et al., 2018; SILVA et al., 2019). As some authors have pointed out, many times they have even been given little importance (BERNAL-BRAVO et al., 2019). Other studies affirm that even this phenomenon occurs for those who in the future would dedicate themselves to teaching (BERNAL-BRAVO et al., 2019).
The reasons pointed out by teachers for their low training do not refer to interest in ICT, but rather as a consequence of lack of institutional support, time, lack of resources and the non-existence of training plans (MÁRQUEZ; LEIVA-OLIVENCIA; LÓPEZ-MENESES, 2018). On the other hand, when the training has been carried out, it has been carried out under models focused more on instrumental and technological aspects than on pedagogical and didactic dimensions (SEMERCI; AYDIN, 2018; CABERO ALMENARA; MARTÍNEZ GIMENO, 2019).
This aspect leads to coining terms such as Digital Teaching Competence (DTC). This concept does not refer exclusively to training processes focused on the mere instrumental management of technologies. The European Union (2018) points out that this is one of the key competences that citizens must master in general and teachers specifically in the Knowledge Society.
Such is the significance that the DTC theme is acquiring, which is becoming a consolidated line of research in the Spanish scientific context (RODRÍGUEZ-GARCÍA; SÁNCHEZ; RUIZ-PALMERO, 2019). This ranges from its description to the search for diagnostic instruments (ESTEVE-MON; GISBERT-CERVERA; LÁZARO-CANTABRANA, 2016; DURÁN CUARTERO; PRENDES ESPINOSA; GUTIÉRREZ PORLÁN, 2019; INTEF, 2017; PADILLA-HERNÁNDEZ; GÁMIZ-SÁNCHEZ; ROMERO-LÓPEZ, 2019), and their certification (DURÁN CUARTERO; PRENDES ESPINOSA; GUTIÉRREZ PORLÁN, 2019; AMAYA AMAYA et al., 2018; GUDMUNDSDOTTIR; HATLEVIK, 2017).
In general, the DTC can be understood as that set of knowledge, skills or strategies of the teaching profession that allow solving the educational problems and challenges posed by the so-called Knowledge Society (PRENDES ESPINOSA; GUTIÉRREZ PORLÁN, 2011; CABERO-ALMENARA; BARROSO-OSUNA, et al., 2020). In short, as Durán Cuartero, Prendes Espinosa, and Gutiérrez Porlán (2019) point out, they are a set of knowledge, skills and attitudes necessary for a teacher to make effective use of ICT from its different aspects, assuming pedagogical-didactic criteria for an effective integration of the ICT in your teaching experience.
Throughout more than a decade, there have been many attempts to define indicators, standards or models that are considered essential in the Teaching Digital Competence (CABERO-ALMENARA; BARROSO-OSUNA, et al., 2020; CABERO-ALMENARA; ROMERO-TENA; PALACIOS-RODRÍGUEZ, 2020). In this article, those frameworks that can be considered the most used at the Spanish level have been selected (CABERO ALMENARA; MARTÍNEZ GIMENO, 2019; LÁZARO-CANTABRANA; USART-RODRÍGUEZ; GISBERT-CERVERA, 2019; PADILLA-HERNÁNDEZ; GÁMIZ-SÁNCHEZ; ROMERO-LÓPEZ, 2019; RODRÍGUEZ-GARCÍA; SÁNCHEZ; RUIZ-PALMERO, 2019; SILVA et al., 2019; CABERO-ALMENARA; PALACIOS-RODRÍGUEZ, 2019):
European Framework for Teaching Digital Competence DigCompEdu (REDECKER; PUNIE, 2017).
Common Framework for Teaching Digital Competence (INTEF, 2017).
The European Framework for Digital Teaching Competence DigCompEdu is designed by the Joint Research Center of the European Union or JRC (REDECKER; PUNIE, 2017) at the end of 2017. Its purpose is to unite European educational policies having said framework of reference. DigCompEdu is the product of a series of scientific studies carried out at the local, national, European and international levels (CABERO-ALMENARA; GUTIÉRREZ-CASTILLO, et al., 2020; GHOMI; REDECKER, 2019; REDECKER; PUNIE, 2017; REISOĞLU; ÇEBI, 2020). Furthermore, this framework is presented as a Digital Competence model with 6 differentiated competence areas (Figure 1).
Each area is associated with a series of competencies that teachers must possess in order to promote effective, inclusive and innovative learning strategies, using digital tools (REDECKER; PUNIE, 2017). Specifically, its areas are:
1. Professional engagement: focuses on teachers’ work environment.
2. Digital resources: related to the sources, creation and distribution of digital resources.
3. Teaching and learning: the fundamental competence of the whole “DigCompEdu” framework is knowing how to design, plan and implement the use of digital technologies in the different stages of the teaching and learning process.
4. Assesment: linked to the use of digital tools and strategies in the evaluation and improvement of teaching-learning processes.
5. Empowering learners: use of digital tools for the empowerment of students.
6. Facilitating learners´ digital competence: on how to develop and facilitate students’ Digital Competence.
In turn, DigCompEdu proposes six levels depending on the competence qualification (Figure 2). The most basic level is called Newcomer (A1), which would correspond to teachers with very little experience and contact with educational technology, and the highest, Pioneer (C2), where teachers who lead innovation with ICT would be found.
The Ministry of Education, Culture and Sport of Spain launches, through the National Institute of Educational Technologies and Teacher Training (INTEF), a project to define the Common Framework for Digital Teaching Competence. For this, it is based on the DigComp digital competence model, Digital Competence for Citizenship, defined by JRC (FERRARI, 2013; VUORIKARI et al., 2016; INTEF, 2017). Like DigCompEdu, it is a generic digital competence model for trainers, whose areas can be seen reflected in Figure 3.
These areas can be summarized as:
1. Information and data literacy: identify, locate, retrieve, store, organize and analyze digital information, evaluating its purpose and relevance.
2. Communication and collaboration: communicate in digital environments, share resources through online tools, connect and collaborate with others through digital tools, interact and participate in communities and networks; intercultural awareness.
3. Digital content creation: create and edit new content (texts, images, videos …), integrate and rework previous knowledge and content, make artistic productions, multimedia content and computer programming, know how to apply intellectual property rights and licenses use.
4. Safety: personal protection, data protection, protection of digital identity, use of security, safe and sustainable use.
5. Problem solving: identifying needs and digital resources, making decisions when choosing the appropriate digital tool, according to the purpose or need, solving conceptual problems through digital means, solving technical problems, creative use of technology, update your own competence and that of others.
In addition, each area is associated with a series of competencies that are developed in Figure 4.
Finally, six progressive levels of management skills are established. This structure is designed to identify a teacher’s level of digital competence. A progressive level of development and autonomy is established that starts from level A1 and continues up to the maximum level, C2.
Based on the comments, this article aims to compare and evaluate the DigCompEdu European Digital Competence Framework for Teachers (JRC) and the Common Framework for Teaching Digital Competence (INTEF).
This research aims to compare and evaluate the DigCompEdu European Digital Competence Framework for Teachers (JRC) and the Common Framework for Teaching Digital Competence (INTEF). That is, to know their differences in content and evaluations regarding their suitability. To do this, two analysis techniques are combined:
1. Comparison of the content of the competence frameworks through crossed matrix.
2. Evaluation of the frames using Delphi design with coefficient of expert competence (CEC) and contrast study with effect size.
The expert judgment basically consists of requesting a series of people to demand a judgment about an object, an instrument, a teaching material, or their opinion regarding a specific aspect (CABERO ALMENARA; CARMEN LLORENTE CEJUDO, 2013).
This strategy is increasingly widespread in educational research-evaluation (ROBLES GARROTE; CARMEN ROJAS, 2015; GALICIA ALARCÓN; BALDERRAMA TRÁPAGA; EDEL NAVARRO, 2017), and specifically in the type of studies that concern us (CABERO ALMENARA; CARMEN LLORENTE CEJUDO, 2013). In addition, it is closely associated with Delphi studies (LÓPEZ-GÓMEZ, 2017). The recurring problem with this method is that the concept of expert is quite polysemic. Therefore, there is no unambiguous conceptualization of it that helps to specify its defining characteristics. Therefore, the results obtained depend directly on the quality of the experts selected for the evaluation process. For this, there are different procedures that range from contemplating the profile of the selected expert to other more complex ones such as the CEC (CABERO ALMENARA; BARROSO OSUNA, 2013; ALMENARA; MORO, 2014; LÓPEZ-GÓMEZ, 2017).
In the present study, two mechanisms were established for their selection; First, we selected them taking into account that they met the following criteria:
Teaching at Universities in the subjects of ”Educational Technology”, ”New Technologies applied to Education”, or ”Information and Communication Technologies Applied to Education”.
To have experience in the field of teacher training in ICT.
To have published an article on literacy in educational technology, digital skills, audiovisual literacy, in Spanish and Latin American magazines, in the last five years.
One of the problems associated with expert judgment concerns the number of experts required for the application. Most of the proposals range between 15 - 20 experts (MALLA; ZABALA, 1978) and 15 -35 (LANDETA RODRÍGUEZ, 2002). In this case, since there were problems working with a large database and only one round of evaluations was made, the decision is made to work with as many as possible.
The interest in refining the selection process of the final experts is carried out by applying the CEC (CABERO ALMENARA; BARROSO OSUNA, 2013; ALMENARA; MORO, 2014; LÓPEZ-GÓMEZ, 2017). This is obtained from the self-perception that the expert has about his level of knowledge regarding the subject analyzed, as well as from the sources that allow him to argue the decision adopted.
To obtain it, the formula is used: K = 1/2 (Kc + Ka). Where Kc is the "knowledge coefficient" and Ka is the argumentation coefficient.
The values used to determine the position of the expert are:
● 0.8 < K < 1.0 = high competition coefficient
● 0.5 < K < 0.8 = mean competition coefficient
● K < 0.5 = low coefficient of competitio
The number of emails sent according to the criteria initially taken into account was 747.35 responses were received. Of the total responses and, after making the appropriate calculations, 275 experts have a K > 0 . 8, making up the sample of this study.
Regarding the areas, unlike the Common Framework for Teaching Digital Competence published by INTEF which is divided into five areas, DigCompEdu is divided into six. It is not about a differentiation only in nomenclature, but in concepts and contents. The difference between the two frameworks is purely conceptual, since the Spanish framework strictly focuses on the digital competence of teaching staff, while the one published by JRC establishes other variables more focused on the digital competence of students or educational organizations.
Likewise, another fundamental difference between DigCompEdu and the Spanish Teaching Digital Competence Framework is that the latter includes 21 competencies while DigCompEdu raises them to 22. However, as can be seen in the following distribution comparison (Table 1); Although with different nomenclature, most of the competences published by the INTEF have been considered in DigCompEdu.
The only competencies in the Digital Teaching Competency Framework published by INTEF that are not mentioned in DigCompEdu are competencies 2.6., 3.4. and 4.4.
Finally, DigCompEdu establishes the same competency levels that are used in the INTEF model, being level A1 the most basic level and C2 the most advanced. Together, DigCompEdu assigns a role nomenclature. In the case of INTEF model, it is not applicable since its objective is to serve as the basis for an official certification of said competence; it is not a descriptive objective as in the case of DigCompEdu.
It begins by presenting the means and standard deviations achieved for each frame, globally and in each of the dimensions (Tables 2 and 3). For a correct interpretation of the scores, it must be taken into account that the response scale used ranges from 1 = VN = Very negative / strongly disagree to 6 = VP = Very positive / Strongly agree.
The analysis of the scores achieved indicates three fundamental aspects:
1. In both frames the mean scores are very high. This denotes the high perceptions shown by the judges, regarding their usefulness.
2. The DigCompEdu model scores slightly higher than the INTEF model.
3. The low standard deviations indicate a strong agreement between the diversity of the answers offered by the judges.
Next, in order to know if there are statistically significant differences in the assessments made by the experts on the adequacy of the frameworks, the following hypotheses are formulated:
H0 (null hypothesis): There are no significant differences between the assessments made by the experts, with an alpha risk of 0.05.
H1 (alternative hypothesis): There are significant differences between the evaluations made by the experts, with an alpha risk of 0.05.
For this, the Wilcoxon signed rank test for related samples (SIEGEL, 1976) is used, as well as Cohen (1988) to analyze the effect size. These tests are applied to the total assessment of both models as it is not possible to make a comparison between areas. The results are presented in Table 4.
Source: Own elaboration.
As can be seen, the results allow rejecting H0 ( p ≤ . 05 ). This indicates that there are significant differences between the assessments made by the experts. Considering the values of Cohen (1988), this difference is considered moderate.
The work allows to obtain a series of conclusions. First, the similarities and differences of the two proposed models are exposed: DigCompEdu and INTEF. Although the general structure is very similar, DigCompEdu establishes other variables focused on the digital competence of students or educational organizations. Second, the different frameworks and competencies that are incorporated within them have been positively scored by judges. This leads us to point out that they are well-consolidated proposals and that they serve to indicate to teachers the digital skills that they must develop to carry out their professional activity (FALLOON, 2020). At the same time, the assessments that different authors have made regarding the fact that the frameworks presented are those that have the greatest significance from an international perspective are confirmed (PRENDES ESPINOSA; GUTIÉRREZ PORLÁN, 2011; DURÁN CUARTERO; PRENDES ESPINOSA; GUTIÉRREZ PORLÁN, 2019; CABERO ALMENARA; MARTÍNEZ GIMENO, 2019; LÁZARO-CANTABRANA; USART-RODRÍGUEZ; GISBERT-CERVERA, 2019; RODRÍGUEZ-GARCÍA; SÁNCHEZ; RUIZ-PALMERO, 2019; SILVA et al., 2019; CABERO-ALMENARA; PALACIOS-RODRÍGUEZ, 2019; CABEZAS et al., 2020).
Another of the contributions of the work also allows to point out that there has been discrimination between the frames by the judges. In other words, the judges value the DigCompEdu model more significantly. Such finding will serve in the project that finances this study to approach the teacher training plan from the perspective selected by the judges, although they must also indicate to our institutions the guidelines on where to establish the training plans for teachers in DTC. In any case, it is important not to confuse the results with the fact that the INTEF model is not significant for acquiring DTC (PADILLA-HERNÁNDEZ; GÁMIZ-SÁNCHEZ; ROMERO-LÓPEZ, 2019). It is recalled that the results depend on expert judges and that the INTEF model has also been scored very positively.
The study presents a series of limitations that open future lines of research that must be considered. On the one hand, it should be noted that, although a very conscientious process has been used for the selection of experts, there is always doubt about the significance of their choice. Therefore, it is proposed to carry out research of a similar nature on the specific features of teaching in each discipline / subject: both university and non-university. The study can also be replicated in two or three rounds, this would require less use of experts and would require a prior commitment from them to participate in the research in a longer time.
Finally, as a future prospect, the results of this research can be used to guide teachers in the recognition and effective development of digital skills for the use of ICT in training processes (BARRAGÁN-SÁNCHEZ et al., 2020). In addition, a possible strategy for the development of digital skills for teachers could be consolidated according to the assessment of the experts.
This article is part of the project Design, production and evaluation of t-Mooc for the acquisition by teachers of teaching digital skills (RTI2018-097214-B-C31) funded by the Ministry of Science, Innovation and Universities.
