Assessing Digital Competence Among Health Science Undergraduates: a Critical Analysis

Evaluación de las competencias digitales en estudiantes de ciencias de la salud: un análisis crítico

Antonieta Herrera-Lillo antonieta.herrera@uvm.cl

Universidad Viña del Mar, Chile

Gabriela Urrejola-Contreras

Universidad Viña del Mar, Chile

In the educational context of science and in the training of professionals in the health field, it is evident how information and communication technologies (ICTs) have played an active role in this process, driven by the rapid technological advances, characteristic of this era [1]. This scenario, marked by the digital revolution, has seen the rapid incorporation of digital resources into educational environments, such as the use of screens, interaction with virtual platforms, simulation, and online assessments, which are examples of a process that has enabled technology-mediated and assisted learning [2,3].

Digital competence among students is a topic of growing interest in the educational field, given the impact of technology on daily and academic life. Digital competence refers to the ability of students to use digital tools effectively, critically, and safely. According to the European Commission (2018) [4], this competence encompasses five key areas: information and data, communication and collaboration, digital content creation, safety, and problem-solving. Recent studies have shown that while many students are active technology users, there are significant variations in their levels of digital competence [5]. Digital competence is essential for success in higher education and the labor market [6].

In this regard, this strategic management between the educational world and technology is possible as long as teachers and students have adequate and timely development of digital competences [7,8]. In fact, this digital literacy is strengthened when actions are promoted with teachers and students, such as acquiring digital skills, adopting learning methods linked to technology, and connecting development with open and available digital resources [9,10].

Although new generations of students have been receptive and inclined to use technology for various purposes, including education, the recent COVID-19 pandemic and the global health crisis tested their level of digital competence [11,12]. For the most part, attendance at classes, workshops, and educational activities for students in the health field was suspended, with virtually all educational instances moving to virtual environments using synchronous and asynchronous resources, even raising questions about their usefulness among students [13]. Initially, these platforms served as repositories for educational content, but they quickly provided resources aimed at fostering constant interaction with students, tools that support learning progression and tracking [14]. However, students faced difficulties with tasks that required critical thinking, problem-solving, searching for specific information, and creating or modifying resources to communicate an idea or narrative, which even led to levels of technostress [15,16].

Investigating digital competencies among students in various health science disciplines is crucial to addressing the challenges of the 21^st century. The increasing digitalization of the healthcare sector requires future professionals not only to master their specific fields but also to be proficient in using digital technologies. Self-perception of these competencies is fundamental, as it directly influences students' confidence and willingness to incorporate technologies into their professional practice [17].

Self-perception of digital competence can reveal discrepancies between actual and perceived skills, identifying areas where students need more support. The study by Van Laar et al. (2020) [18] highlights a significant gap between self-assessment of digital competencies and actual skills, underscoring the need for accurate and contextualized evaluation in the educational field. A positive self-perception of these competencies is associated with greater adoption and effective use of technologies [19,20].

Moreover, research on self-perception allows for the adaptation of educational curricula to the specific needs of each specialty, helping to design more effective training programs [21].

In this regard, several authors have worked on both the evaluation and validation of tools that estimate levels of digital competence in higher education [22]. Although there are various studies assessing teachers [23], recent reviews recognize that there are few focused on evaluating students [24].

Although there are approaches in studies to assess digital literacy in undergraduate and graduate students in health sciences [25], their results have revealed different scenarios. While some studies conclude that students consider themselves competent in few actions and demonstrate problematic use of technology in tasks that involve accessing, using, managing, and generating information [26], other research suggests that although students develop informal digital learning practices and experiment with technology, they were unable to link it to their study programs and the disciplinary training of their health-related careers [27].

A reflection arises from the study of the intentions and behavior of students who use mobile devices as learning tools. This model focuses on the Unified Theory of Acceptance and Use of Technology (UTAUT-2). Recent research revealed that, based on the data collected, digital natives' intention to use digital devices was influenced by habit, hedonic motivation, and effort expectancy. However, age, gender, and experience acted as moderators of behavioral intention to use [28].

In this context, a practical application of the theoretical foundations of digital competence in health education becomes essential when applied to areas related to research. For example, a recent study identified key predictive skills associated with gender, such as creativity and attitude towards new changes, as well as the percentage of time dedicated to research [29]. Another experience addresses how the transition to greater digital competence hinges on the macroevolution of teaching. This observation enabled recent Spanish research to characterize teacher digital profiles, and based on this criterion, design new pedagogical approaches capable of incorporating technological variables [30].

Therefore, researching digital competencies in health sciences students through self-perception is an urgent need [28]. It not only helps to identify and close skill gaps but also ensures that future professionals are better prepared to face the challenges of a digitalized medical practice. This approach, supported by key research and references, highlights the importance of integrating digital competence into higher education in health [20]. To evaluate digital competencies in health science students through self-perception.

This study employs a quantitative, non-experimental, cross-sectional, and descriptive design aimed at evaluating the digital competencies of Health Sciences students through self-perception. The quantitative approach allows for the collection of objective and measurable data, facilitating statistical analysis and the generation of generalizable results [31]. Since the study does not intend to manipulate variables, the non-experimental design is appropriate, allowing the phenomenon to be observed in its natural context [32].

The choice of a cross-sectional design is useful as it conducts the study at a single point in time, gathering specific data without requiring longitudinal follow-up. This provides an instantaneous view of the digital competencies of first-year students, making it easier to identify strengths and areas for improvement [33]. Finally, the descriptive scope offers a general and detailed view of the digital competencies in this group, which can serve as a foundation for future research [31]. Thus, this methodological combination is considered ideal for achieving the study’s objective and generating an initial understanding of the digital competencies of students at Universidad Viña del Mar.

The target population was first-year students from the School of Health Sciences at the Universidad Viña del Mar in the Valparaíso region, Chile. The inclusion criteria established that students must be enrolled at the time of completing the questionnaire and must be taking courses in the first semester of 2024. Therefore, students who did not meet the afore mentioned inclusion criteria did not participate in the study. Regarding the sample, a non-probability convenience sampling approach was used due to the technical feasibility of access to the unit of analysis. The target population of this study was 360 students, and the final sample consisted of 186 people from 8 programs at the School of Health Sciences.

To collect the information, the Digital Competence Assessment Questionnaire for Higher Education Students (CDAES) was used as the instrument. This questionnaire, developed by Gutiérrez-Castillo et al. in 2017, was initially validated with university students in Early Childhood or Primary Education in the Autonomous Community of Andalusia, Spain. The psychometric properties of the CDAES demonstrate its reliability and validity as an instrument for assessing university students' technological competencies. The overall internal consistency of the questionnaire is very high, with a Cronbach's alpha of 0.966, indicating a strong correlation among items. The reliability of each dimension is also adequate, with alpha values of 0.891 for Technological Literacy, 0.850 for Information Search and Processing, 0.867 for Critical Thinking, Problem Solving, and Decision Making, 0.878 for Communication and Collaboration, 0.888 for Digital Citizenship, and 0.925 for Creativity and Innovation.

The six-factor structure of the CDAES was supported by the analyses conducted, providing evidence of its validity. For content validity, 17 experts in "Educational Technology" and "ICT applied to education" from universities in Spain and Latin America were consulted. Construct validity was also assessed through exploratory factor analysis, using Varimax rotation and Kaiser normalization [34].

The instrument evaluates university students' perception of their digital skills. It consists of six dimensions subdivided into 22 indicators, covering a total of 43 questions. The questionnaire uses an item format with a Likert scale. On this scale, participants indicated their level of digital competence by assigning a value from 1 to 10. A value of 1 reflected the perception of complete inability, while a value of 10 indicated absolute mastery of the described skill.

The questionnaire was administered between March and May 2024, with a response rate of 52%. Data collection was carried out through the Google Forms platform. For analysis, descriptive statistics (central tendency measures, frequencies, and percentages) were calculated. Responses were classified as negative, neutral, or positive for the six study dimensions. Values one, two, and three were considered negative responses; values four, five, six, and seven were considered neutral responses; and values eight, nine, and ten were considered positive responses.

Figure 1 summarizes the characteristics of the participants. Graph A shows the distribution by gender; Graph B, the distribution by age ranges; Graph C, the distribution by academic programs; and Graph D, the time spent on screens or digital technology (Figure 1).

Figure 1
Characterization of the sample by gender, age, academic program, and hours of exposure to digital technology.

For the digital literacy dimension, the highest scores were recorded in items 1 (use of different types of operating systems installed on a computer), with 53% of the responses, and 7 (communication with others using synchronous tools), with 51% representation. In contrast, regarding emerging technologies, such as web page design (Item 8) and the use of collaborative work software (Item 9), low scores were observed, representing 19% and 13%, respectively.

Regarding Dimension 2, focused on Research and Information Management, the statistical data revealed an average level in this area of evaluation. The dispersion of scores relative to the mean suggests considerable variability in research and information management skills among the participants (Table 1).

Table 1
Percentages According to Dimension 1: Digital Literacy - ICT Functioning and Concepts - Dimension 2: Research and Information Management for the Questionnaire on the Assessment of Digital Competence of Higher Education Students (CDAES).

Dimension 3, focused on Critical Thinking, Problem Solving, and Decision Making, showed that the scores for each item were neutral. This could indicate a symmetrical distribution of scores in this dimension. In Dimension 4, Communication and Collaboration, participants scored within a wide range, from a minimum of 10 to a maximum of 90 points, suggesting significant diversity in the competencies evaluated in this dimension (SD 21.65). The mode, set at 70 points, indicated that this specific score was the most frequent among the participants evaluated (Table 2).

Table 2
Percentages According to (Dimension 3: Critical Thinking, Problem Solving, and Decision-Making - Dimension 4: Communication and Collaboration) for the Questionnaire on the Assessment of Digital Competence of Higher Education Students (CDAES).

In Dimension 5, Digital Citizenship Competence, scores were concentrated around the mode (35 points). The low variability reinforced the consistency in the scores, indicating a homogeneous performance in this dimension. In Creativity and Innovation, participants scored an average of 37.3, indicating a medium level. The maximum score was 60, showing that some participants achieved the highest score. The coefficient of variation reflected a relatively moderate variability (Table 3).

Table 3
Percentages According to Dimension 5: Digital Citizenship - Dimension 6: Creativity and Innovation for the Questionnaire on the Assessment of Digital Competence of Higher Education Students (CDAES).

Regarding the analysis of negative, neutral, and positive responses, it was identified that neutral responses predominated in all dimensions, especially in Dimension 6, with 58%. Positive and negative responses varied, but, in general, positive responses outweighed negative ones in each dimension, except in Dimension 3, where negative responses reached 15%. Dimension 4 was particularly notable for its very low percentage of negative responses (1%) (Figure 2).

Figure 2
Distribution of negative, neutral, and positive responses regarding the six dimensions of the Questionnaire on the Assessment of Digital Competence of Higher Education Students (CDAES).

The findings of this study indicate that health sciences students self-perceive a moderate level of digital competence; however, they report that the efficient use of technology in activities related to critical thinking, research, and innovation is challenging for them, and they lack all the necessary tools to carry it out. Despite this, there are elements that emerge from this study, as well as from other experiences, where students possess skills derived from interaction with technology, such as the use of mobile devices and the coordination of activities through social media communication. [35,36]. Indeed, health students perceive themselves as comfortable with these tasks, suggesting that students have digital literacy, communication, and collaboration skills through technological tools. This criterion may influence the greater preference of students for online courses [37,38]. From this point, it is possible to infer that social networks can act as a mediating platform for interaction with academic purposes, with YouTube standing out as an example. This tool promotes greater user participation and democratizes knowledge [39,40].

Regarding technology for tasks related to research and information management, this study found that students recognize difficulties in creating graphs, concept maps, and diagrams, which are useful for systematizing, organizing, classifying, and interpreting data. This information is similar to that obtained in a study involving three universities in the European community [41]. In this line, a recent experience in France concluded that the digital divide still exists when students are required to utilize ICTs for academic tasks such as creation and collaborative management [42].

Another relevant point arises from the analysis of the creativity and innovation dimension, which reported that students have a high adaptability to new scenarios and technological environments, providing a suitable foundation for continuous improvement programs in digital competencies. However, students noted that the greatest challenge relates to difficulties in using ICTs to build knowledge, simulate, explore, and solve more complex processes. Therefore, this element highlights the need to address the role of integrating technology as support in seeking solutions to academic problems within an e-health context [43-45].

In this sense, it is possible to think that the difficulty students have in transferring technological skills to academic contexts may partly be due to the acquisition of digital competencies arising from purely intuitive, commercial, and social mechanisms [46,47]. On the other hand, although students had early access to digital resources, the lack of integration of digital tools in the curriculum and their limited use by instructors in academic areas [48] may be related to the limitations currently observed by the educational community, including the lack of digital competencies, excessive online distraction, lack of motivation and/or awareness, and technical problems that discourage the intention to use [49] and the effective and continuous use of digital tools in their studies [50].

Finally, the challenge is the integration of technology into learning and the systematic development of digital competencies in health education programs. This facilitates adaptation to an ever-changing environment and promotes digital transformation in the healthcare field [51], playing a crucial role in professional training and the future quality of healthcare [52].

Regarding the limitations of this work, we first face the sample size (n =186 students). The small number of participants and the lack of homogeneity in the sample composition, such as the uneven distribution of students from different health-related programs, limit the generalization of the findings to the entire student population. Additionally, considering the instruments used for data collection, it is possible to account for the reliance on self-reported questionnaires, which may result in potential biases of overestimation or underestimation of the competencies reported by the students.

Another aspect arises regarding the heterogeneity in the age of the participants, as most of the study subjects are under 22 years old, posing a challenge to extrapolate the results to older populations. Additionally, the experiences and perspectives of younger participants could differ substantially from those of older individuals, which compromises the applicability of the results to broader and more diverse contexts [53].

It is recommended to complement this study with a longitudinal design that would enable a comprehensive follow-up of students in higher-level courses. This approach would allow for a more accurate capture of the progress in digital competence acquisition over time [54,55]. By observing the development of these skills in more advanced stages of academic training, it will be possible to identify growth patterns and assess the evolution of digital competencies in more specialized contexts [56].

Therefore, to address the limitation associated with evaluation based solely on students' perceptions, it is suggested to incorporate an instrument that combines both quantitative and qualitative approaches [57]. The implementation of assessment tools that include both students' perceptions and the observation of their competencies will provide a more comprehensive and accurate view [58]. Finally, a promising perspective for future research would involve exploring the connection between the advancement of digital skills and the concept of e-health [59]. This perspective would allow for an understanding of how students' digital skills correlate with their ability to leverage information technologies in the health context [60]. Evaluating whether a strong development of digital competence translates into greater adoption and understanding of the e-health concept could provide valuable insights for the comprehensive education of students in fields related to health and technology [61].

Although most students possess skills consistent with adequate digital competence, they acknowledge having difficulties using technology in activities that require information management, research, creativity, and innovation in an academic context. The continuous improvement of digital competence among students and guiding the use of technology to solve problems can provide added value in the e-health landscape.

We would like to express our sincere gratitude to the team for their invaluable contributions to the study design and for their insightful discussions on the results obtained.