The advancement of emerging technologies, particularly AI, is profoundly reshaping the contemporary educational landscape. Recent research highlights the urgent need to redefine teaching skills in light of this transformation (Celik, 2023; Holmes et al., 2019; UNESCO, 2023). Far from being limited to technical mastery, these competencies include a critical understanding of the impact of technology on teaching and learning processes, the ethical use of data, ensuring inclusion, and designing personalized and meaningful experiences.

The digital transformation has brought about significant structural changes in education, affecting not only the tools available and pedagogical dynamics but also regulatory frameworks and the professional roles of teachers. In this context, technologies such as augmented reality, big data, blockchain, and AI itself not only open up new teaching possibilities but also impose the need for a profound reconfiguration of teachers' professional competencies (Bo, 2024; UNESCO, 2024b).

This new paradigm requires teachers to perform more complex functions: designing learning experiences, acting as critical mediators between technology and the curriculum, and managing digital knowledge ethically. Consequently, technological integration in the classroom cannot be reduced to instrumental or technical use but requires a contextualized pedagogical approach capable of assessing its social, ethical, and educational implications and adapting them to the needs and trajectories of students (Cabero-Almenara et al., 2020).

Various international initiatives, such as the DigCompEdu framework (Ghomi & Redecker, 2019) and UNESCO's guidelines for teacher competence in ICT, as well as those currently specific to the use of AI, have established benchmarks for the development of digital competences in teachers. However, these frameworks tend to take a generalist perspective, focusing on cross-cutting skills and established technologies, without addressing in sufficient depth the specific challenges posed by emerging technologies, particularly AI. This limitation underscores the need to develop models that integrate specific dimensions of critical appropriation of these technologies, aiming to promote new digital literacies that position teachers as active agents of transformation within contemporary educational ecosystems.

Although DigCompEdu (Ghomi & Redecker, 2019) is a widely recognized framework for assessing teachers' digital competence, its generalist orientation limits its ability to address the specificities of Artificial Intelligence in depth. This instrument describes cross-cutting areas of competence applicable to various digital technologies, without differentiating between established and emerging tools, or explicitly integrating ethical and curricular dimensions specific to AI. In contrast, AI-ED-SAT focuses exclusively on the educational use of Artificial Intelligence, incorporating indicators linked to the conceptual understanding of algorithms, contextualized pedagogical application, ethical reflection, and strategic curricular integration. Thus, the AI-ED-SAT does not seek to replace frameworks such as DigCompEdu, but rather to complement them, offering a level of detail adapted to the unique challenges and opportunities presented by AI in educational settings.

The inclusion of Artificial Intelligence into education represents one of the most significant transformations in recent years, with direct implications for teaching practices, pedagogical models, and teaching-learning processes. Its presence manifests itself in multiple forms: intelligent tutoring systems, adaptive platforms, learning analytics, chatbots, recommendation engines, and automated feedback tools (Holmes et al., 2019; Zawacki-Richter et al., 2019).

This technological expansion offers significant opportunities to personalize learning, optimize assessment processes, and alleviate teachers' administrative tasks. However, it also poses substantial challenges that cannot be ignored. These include algorithmic biases, the opacity of automated systems, privacy violations, and the risk of delegating fundamental pedagogical decisions to systems that operate with logic alien to human judgment (Chesterman, 2021; Ferrante, 2021; Huang, 2023; Selwyn, 2019).

In this sense, AI cannot be conceived solely as a functional tool, but rather as a complex socio-technical phenomenon that involves political, pedagogical, and ethical decisions (Flores-Vivar & García-Peñalvo, 2023). Its implementation in education requires teachers not only to acquire operational skills, but also to develop a critical literacy that enables them to understand how algorithms work, identify their potential biases, and assess their social and educational implications (García Peñalvo et al., 2024; Lane et al., 2023; Luckin, 2017).

A recent and relevant ethical reference is the Safe AI Education Manifesto (Alier Forment et al., 2024), which sets out clear recommendations for the responsible use of AI in educational contexts. Its fundamental principles include transparency in systems, accountability, protection of student privacy, promotion of equity, and fostering critical literacy in the face of algorithmic technologies. These principles reinforce the ethical dimension of the AI-ED-SAT, particularly in items related to bias identification, automated decision-making, and educational impacts, thereby consolidating the instrument as a tool that integrates updated ethical principles relevant to contemporary teaching practice.

Despite this outlook, teacher training in AI remains in its incipient stages. Existing proposals tend to focus on instrumental aspects, without incorporating a comprehensive view that considers both its pedagogical and ethical dimensions (Delgado et al., 2024; Holmes & Tuomi, 2022). Given this gap, it is essential to design valid and reliable instruments that allow for the evaluation not only of teachers' familiarity with AI, but also of their ability to integrate it in a critical, contextualized, and pedagogically meaningful way. To this end, it is necessary to articulate rigorous methodological processes that combine expert judgment with robust statistical analysis, as proposed by Sampieri (2018) and Muñiz and Fonseca-Pedrero (2019).

The emergence of Artificial Intelligence (AI) in educational settings has consolidated its position as a disruptive technology, with applications ranging from personalized learning to the automation of assessment and administrative tasks (Holmes et al., 2019; Owan et al., 2023). Its implementation poses significant challenges that require teachers to have a deep understanding of its technical foundations and to critically reflect on the associated risks, such as algorithmic biases, opacity in decision-making, technological dependence, and inequalities in access (Celik, 2023; Sharma et al., 2024).

In this context, developing specific AI skills is a strategic priority for education systems. Unlike generic digital skills, which focus on managing digital resources or virtual environments, teaching skills in AI requires a multidimensional understanding that integrates technical, ethical, pedagogical, and curricular aspects. Various conceptual documents have begun to develop this new competency profile, including UNESCO's (2024a, 2024b) AI competency frameworks for teachers and students, UNESCO's AI and the Future of Teaching document (UNESCO 2021), the update of the DigCompEdu framework to the AI context (Caena & Redecker, 2019), and the studies by Chen et al. (2020), Chiu et al. (2024), and Ng et al. (2023).

Khreisat et al. (2024) propose a structure based on four interrelated areas:

Understanding the principles and functioning of AI.

This proposal reflects a broader conception of the role of teachers, which transcends the instrumental use of technology. Rather than merely operating tools, teachers must position themselves as reflective and critical agents, capable of evaluating the pedagogical meaning, social consequences, and curricular relevance of AI in their educational practices. As Holmes et al. (2019), Zawacki-Richter et al. (2019), and Ayuso del Puerto and Gutiérrez Esteban (2022) warn, many existing frameworks remain descriptive or eminently technical, which limits their ability to guide truly transformative professional development.

From this perspective, it is necessary to create assessment tools that respond to this complexity. Such tools must extend beyond measuring declarative knowledge to encompass dimensions such as pedagogical competence, critical thinking, and curricular integration. The AI-ED-SAT instrument, developed within the framework of this study, follows this logic, as it is structured around four key dimensions: conceptual understanding, pedagogical application, ethical dimension, and curricular integration. This approach aims to capture the complexity of teacher professional development in the era of AI, acknowledging the need for critical literacy that enables educators to navigate the challenges posed by emerging technologies proactively.

In a context of educational transformation driven by emerging technologies, it is crucial to have valid and reliable instruments for diagnosing the level of teaching competence in relation to artificial intelligence. In this sense, self-assessment is a formative and reflective strategy that enables the identification of strengths and areas for improvement from a situated perspective (Panadero et al., 2019; Rachbauer et al., 2025).

Although there are numerous instruments for measuring teacher digital competence (Cabero-Almenara et al., 2020; Ghomi & Redecker, 2019), most focus on generic technical skills, without specifically addressing the pedagogical, ethical, and curricular complexities posed by AI. This limitation has created an urgent need for assessment models that capture not only teachers' technological familiarity but also their ability to integrate AI in a critical, contextualized, and pedagogically meaningful way (Celik, 2023; Delgado et al., 2024).

The construction of instruments with these characteristics requires a rigorous methodological approach that combines theoretical, empirical, and practical criteria. In recent years, a mixed methodology has been established that includes systematic literature review for item development, validation through expert judgment (such as the Delphi method), and psychometric analysis (content and construct validity, internal reliability) (Diefes-Dux et al., 2010; Muñiz & Fonseca-Pedrero, 2019; Sampieri, 2018). In this process, participatory and interactive approaches, such as Design-Based Research (DBR), have proven particularly useful in ensuring the semantic and pedagogical relevance of the instrument.

Self-assessment has also established itself as an effective tool for encouraging professional reflection, providing data for the design of personalized training plans, and promoting teacher agency (Borge et al., 2005). Unlike external assessment models, it places teachers at the center of the diagnosis and improvement process.

The AI-ED-SAT instrument aligns with this framework, as it is designed to assess teaching competencies related to AI in educational contexts comprehensively. Its development is based on a critical review of the specialized literature and recent reference frameworks that address the pedagogical, ethical, and curricular implications of AI (Celik, 2023; Holmes et al., 2019; Zawacki-Richter et al., 2019).

Based on this analysis, four key dimensions were identified that make up a competency profile geared toward a critical and formative appropriation of AI:

Conceptual understanding of AI

This includes basic knowledge of algorithms, machine learning, and adaptive systems, with a focus on their educational applications. This dimension seeks to position teachers as informed actors in the face of dominant technological discourses (Holmes et al., 2019).

It should be noted that, although some competencies may appear to overlap—particularly between the pedagogical use of AI tools and their curricular integration—the two are distinguished by their level of application: while the second dimension refers to educational use in the classroom (microplanning), the fourth focuses on the strategic incorporation of AI into general curriculum planning (macroplanning), including interdisciplinary approaches and institutional alignment. This differentiation avoids ambiguities and captures different levels of teacher appropriation.

These dimensions are articulated in a complementary manner, enabling a holistic assessment of the teaching role within the digital ecosystem. Unlike fragmented or overly technical models, the AI-ED-SAT proposes an integrated framework where pedagogical action, conceptual knowledge, curricular vision, and ethical commitment are intertwined in the same assessment device.

Figure 1 Figure 1 Diagram of the instrument Figure 1 Diagram of the instrument

Figure 1 presents a schematic representation of the AI-ED-SAT model, highlighting the interrelationship between its dimensions and its theoretical foundation. The graphic order of the dimensions —Pedagogical Use, Conceptual Understanding, Curriculum Integration, and Ethics and Considerations — does not imply a hierarchy, but rather reflects a circular, non-linear arrangement that emphasizes their interdependence. The dimensions of the model do not follow a predetermined order of acquisition but rather form a multidimensional framework that can be developed in parallel, complementary, or combined ways, depending on the teacher's profile, previous experience, and educational context.

In short, the incorporation of AI in the educational field requires a comprehensive understanding on the part of teachers, combining conceptual mastery with critical, ethical, and pedagogical appropriation. Despite advances in the theoretical definition of these competencies, a lack of specific, valid, and reliable instruments remains, hindering the rigorous and contextualized diagnosis of these competencies.

The AI-ED-SAT is conceived as a response to this need. Its methodological design, based on a combination of qualitative and quantitative approaches, including Design-Based Research, expert judgment, and factor analysis, seeks to ensure the theoretical and psychometric soundness of the instrument, as well as its educational usefulness in improving teaching practice.

This study is part of a quantitative, descriptive, and correlational approach aimed at designing, validating, and conducting psychometric analysis of a teacher self-assessment instrument on the use of Artificial Intelligence (AI) in education. Methodologically, the principles of Design-Based Research (DBR) (Brown, 1992) were integrated with current guidelines for the validation of educational instruments (Muñiz & Fonseca-Pedrero, 2019; Sampieri, 2018).

The process was structured in three main phases:

Construction of the questionnaire, based on a systematic review of recent literature.

Figure 2 Figure 2 Stages of the development and validation process Figure 2 Stages of the development and validation process

A pilot test was carried out as an intervention, in which the instrument was applied to individuals with characteristics similar to those of the target sample. The purpose of this test was to adjust those items that might require modifications in terms of operability, wording, or other aspects, as well as to optimize the appearance of the instrument (Hernández-Sampieri et al., 2006).

The literature review on teaching competencies in the use of Artificial Intelligence (AI) in education led to the configuration of the first structure of the self-assessment questionnaire around two fundamental dimensions that cover the spectrum of analysis of teaching practice: (1) the pedagogical integration of AI in the classroom and (2) teaching competency in AI and its impact on teaching.

The first draft was constructed using a process based on the design-based research (DBR) methodology, as described by Brown (1992). The process was structured in three phases: in the first, a review of the literature on AI in education and teaching competencies in digital environments was carried out; in the second, the dimensions were defined and the items that would make up the questionnaire were developed; and in the third, the instrument was validated using the Delphi method with a panel of experts and learning (Chen et al., 2020; Holmes et al., 2019; Luckin, 2017; UNESCO, 2024b).

The instrument, called AI-ED-SAT (Artificial Intelligence in Education – Self-Assessment Tool), was designed to assess the level of teacher competence in four key dimensions:

General knowledge of AI and its educational application

Initially, 40 items were developed, written as statements and distributed evenly across the four dimensions (10 items per dimension). Responses were structured on a 5-point Likert scale:

1 = Strongly disagree | 2 = Disagree | 3 = Neither agree nor disagree | 4 = Agree | 5 = Strongly agree

The decision to include a balanced number of 10 items per dimension was based on the criteria of structural symmetry and comparability between dimensions, seeking to ensure a balanced psychometric analysis. Although the nature of the dimensions could have allowed for an extension or reduction of items, an initial equitable distribution was chosen as a starting point for validation, in line with common practices in similar instruments (Muñiz & Fonseca-Pedrero, 2019). Subsequently, the validity analysis enabled the instrument to be refined, resulting in a final version comprising 38 items.

This scale allows for the capture of differentiated levels of self-assessment without overloading decision-making and is suitable for factor analysis. The validation of the instrument sought to ensure that the indicators adequately reflected the integration of AI in education and the teaching skills necessary for its effective implementation.

The final version of the questionnaire, comprising 38 items distributed across four dimensions, is available in full in Appendix 1. The inclusion of the complete instrument provides a detailed overview of the indicators used to assess teaching competence in AI, facilitating its possible replication in subsequent studies or training programs.

The second phase of the study aimed to validate the content of the preliminary draft of the questionnaire that had been previously developed. To this end, the Delphi method was used, widely recognized in the field of Education Sciences for its ability to clarify complex issues through a structured process of consultation among experts organized in a panel (Reguant-Álvarez & Torrado-Fonseca, 2016). This iterative technique fosters informed consensus through successive rounds of evaluation, enabling a progressive convergence of informed opinions (Linstone & Turoff, 2002; Reguant-Álvarez & Torrado-Fonseca, 2016).

The instruments used for data collection consisted of questionnaires designed to assess specific dimensions and items, including sections for qualitative observations. The collection process was organized with a flexible yet limited timeframe to facilitate the effective participation of a panel of experts, who were distributed geographically and had varying availability.

At this stage, the panel of experts was formed, prioritizing the representativeness and professional diversity of the participants over the absolute number of participants. To this end, selection criteria were defined to ensure a balance between up-to-date knowledge on the integration of Artificial Intelligence (AI) in educational contexts and the teaching skills required for its effective application.

Although the panel included a higher proportion of university educators, this choice was made in response to the need for experts with a systematic view of teacher development and experience in continuing education, which is crucial for assessing cross-cutting skills, such as those related to AI. However, the presence of primary and secondary school professionals with direct experience in applying AI technologies in the classroom was ensured, as well as accredited specialists in artificial intelligence with scientific output and participation in educational innovation projects.

Based on the preliminary structure of the instrument, the central criterion was to include professionals from three key profiles: (1) teachers from different educational levels with experience in the pedagogical use of emerging technologies; (2) specialists in AI applied to education; and (3) teacher trainers in the field of educational technology. This strategy enabled the integration of a multidisciplinary perspective, consistent with the study's objectives.

Although there is no normative consensus on the ideal size of Delphi panels, various authors suggest practical guidelines. Generally, it is considered that a minimum of ten participants ensures the stability of the consensus, as samples of fewer than seven may compromise representativeness (Reguant-Álvarez & Torrado-Fonseca, 2016). In line with this premise, Skulmoski et al. (2007) argue that, for relatively homogeneous panels, a sample of between ten and fifteen experts can be methodologically sound.

Twelve experts who met the established criteria were invited to participate in the study. The invitation included a detailed description of their professional profile, information about their teaching and research roles, and a self-assessment of their suitability for the study's purpose.

All the experts accepted the invitation, forming a panel with a high level of specialization. Seventy-five percent of the group held doctoral degrees, comprising seven university trainers in teacher training programs, three basic education teachers (from primary and secondary education) with direct experience in using AI in the classroom, and two specialists with combined experience in academic research and teacher training in educational AI. Both experts have published indexed works on AI in education, served as advisors on technology integration projects in educational institutions, and collaborate in international networks focused on academic innovation and digital ethics.

In addition, a balanced gender distribution was sought in the composition of the panel, which included seven women (58%) and five men (42%), in line with the representativeness and diversity criteria established in the methodological design.

Common inclusion criteria included:

A minimum of five years' experience in educational technology, teacher training, or applied research on AI.

Table 1 Table 1 Composition of the Delphi expert panel

Category	Number
Total number of experts	12
With doctoral degrees	9
University trainers	7
Primary and secondary school teachers	3
Specialists in AI and teacher training	2

The researchers in charge supervised data collection during the validation phase, which was conducted through an iterative process of consulting experts, utilizing email as the primary channel of communication. The procedure was structured in two successive rounds, which was considered adequate to facilitate the convergence of opinions and reach a methodologically sound consensus (Linstone & Turoff, 2002). From the outset, participants were informed about this structure and the degree of involvement required, with the aim of ensuring transparency and commitment on the part of the panel (Reguant-Álvarez & Torrado-Fonseca, 2016).

The primary purpose of the first round was to evaluate the quantitative and qualitative aspects of the items in the instrument, with a focus on two fundamental dimensions: conceptual relevance and linguistic clarity. At the quantitative level, the experts were asked to rate each item on a five-point Likert scale, where one represented "not relevant" or "unclear," and five indicated "very relevant" or "very clear." At the same time, space was provided for open comments, which allowed for the collection of qualitative suggestions aimed at improving the wording, accuracy, or appropriateness of the items. This mixed approach, based on previous work (Gallant & Luthy, 2020; Grand-Guillaume-Perrenoud et al., 2023; Paulin et al., 2024; Ramírez, 2019; Ramírez-Montoya & Lugo-Ocando, 2020), enriches the instrument from an interpretive perspective.

Based on the analysis of this first phase, six items were reformulated based on qualitative observations, and two were eliminated because they were considered redundant or conceptually weak. These adjustments led to the design of the form for the second round, which focused on re-evaluating the modified items. Again, a five-point Likert scale was used to assess their relevance and clarity after the modifications. The objective of this second iteration was to confirm the validity of the changes introduced and consolidate a final version that reflected the panel's consensus. The results obtained enabled the process to be closed with a definitive structure comprising 38 items, distributed across four theoretical dimensions.

The results of both rounds were analysed using a combination of quantitative techniques:

Descriptive statistics: calculation of mean, standard deviation, and percentiles for each item.

This analytical approach enabled the accurate identification of items that achieved high levels of consensus, as well as those that required adjustments or elimination.

Once the expert judgment validation process was complete, the revised version of the instrument underwent a pilot study with a sample of 128 teachers from various educational contexts in Spain and Latin America. The selection was intentional, based on criteria of accessibility and institutional diversity.

Sample profile:

Primary education: 34%

The application was carried out online. The sample consisted of teachers from seven Spanish-speaking countries: Spain (n = 49), Mexico (n = 27), Argentina (n = 18), Colombia (n = 12), Chile (n = 9), Peru (n = 7), and Uruguay (n = 6). This distribution was based on criteria of institutional accessibility and geographical diversity, ensuring a balanced representation of different educational contexts. In terms of academic level, the proportion of participants remained similar across all countries, with representation from all three levels: primary, secondary, and tertiary education.

All participants signed a digital informed consent form before accessing the questionnaire. The data collected were processed using IBM SPSS v28 and AMOS v24 software and underwent three types of statistical analysis:

a) Internal consistency

The internal reliability of the questionnaire was estimated using Cronbach's alpha coefficient, calculated individually for each dimension and for the instrument as a whole. This analysis made it possible to evaluate the internal consistency of the items grouped around the different theoretical constructs:

General knowledge about AI: α = 0.88

These values demonstrate high internal consistency across all dimensions, exceeding the threshold of 0.70 recommended by specialized literature.

In addition to Cronbach's alpha coefficient, McDonald's omega coefficients were calculated, which exceeded the threshold of 0.85 in all cases (total ω = 0.94). This metric, considered a more robust estimate of reliability in scales with multidimensional structures (McDonald, 1999), reinforces the evidence of the instrument's high internal consistency. Although no temporal reliability analyses (test–retest) or factor invariance tests by subgroups (educational level, gender, or country) were applied in this phase, due to the cross-sectional nature of the design and the sample size, the relevance of these techniques for future validation research with longitudinal designs or larger samples is recognized.

Although the number of participants can be considered adequate in relation to the total number of items in the instrument, various methodological studies (Costello & Osborne, 2005; Lloret-Segura et al., 2014; MacCallum et al., 1999) agree that sample adequacy does not depend solely on absolute size, but on other factors such as item communality, the number of expected factors, and the robustness of statistical indicators. Along the same lines, Hogarty et al. (2005) point out that the quality of factorial solutions is more influenced by the average communality and overdetermination of factors than by the sample size itself. In this case, the high internal consistency of the instrument (α = 0.93), the KMO sample adequacy index = 0.91, the significance of Bartlett's sphericity test, and factor loadings greater than 0.60 in all items allow us to consider the sample of 128 teachers as methodologically sound for the factor analysis performed.

b) Exploratory factor analysis (EFA)

In order to identify the underlying structure of the questionnaire, an EFA was performed using the maximum likelihood method and Varimax rotation. The adequacy of the sample was confirmed by:

KMO index = 0.91

Both indicators showed excellent conditions for the application of factor analysis. Factor loadings were greater than 0.60 for all retained items, and the total variance explained reached 67.8%, distributed evenly among the four proposed dimensions.

In accordance with methodological recommendations and to provide greater transparency in the results, Annex 2 includes the complete matrix of factor loadings, communalities, and specific variances corresponding to the 38 items retained in the exploratory factor analysis. This information enables a detailed examination of the factor structure of the instrument and supports the robustness of the model obtained.

c) Confirmatory factor analysis (CFA)

To verify the empirical adequacy of the theoretical model derived from the EFA, a CFA was performed using a two-factor model, evaluating the quality of the fit using the following indicators:

χ²/df = 2.34

All the values obtained indicate an excellent fit of the factorial model, confirming the structural validity of the AI-ED-SAT instrument as a tool for teacher self-assessment on the use of Artificial Intelligence in educational settings.

Although it is methodologically recommended to apply EFA and CFA on independent samples, several studies (Brown, 2015; Worthington & Whittaker, 2006; Lloret-Segura et al., 2014) recognize that in applied research contexts with limited samples, it is acceptable to use both procedures on the same sample, provided that the statistical results are robust and the limitation is made explicit. In this study, this strategy was chosen for exploratory and initial validation purposes, recognizing the need for future replication in broader contexts.

During the first round of the Delphi method, the 40 items that made up the questionnaire were evaluated by the panel of experts in relation to two fundamental aspects: conceptual relevance and linguistic clarity. Although most items received high ratings, six were reformulated and two were eliminated because they did not meet the minimum threshold of the 80th percentile in the relevance dimension.

In the second round, the experts reevaluated the adjusted items. The results were analyzed using Kendall's coefficient of agreement (W), whose values showed a statistically significant level of agreement in both dimensions evaluated. This finding supports the existence of consensus among experts, despite the diversity of their profiles and approaches.

Table 2 Table 2 Expert agreement – Kendall's coefficient

Aspect evaluated	Kendall's W	Chi-square	df	Sig. (p)
Relevance	0.27	84.44	26	< .001
Clarity	0.24	74.53	26	< .001

To ensure content validity, the approach of Mauri et al. (2007) was applied, adapting the cutoff point to the 80th percentile instead of the arithmetic mean. Additionally, a conditional relationship between relevance and clarity was established: the clarity of an item was only evaluated if it had previously been considered relevant. In this way, conceptually weak items were avoided from a formal perspective. The concordance coefficients obtained reflect an adequate consensus, which justifies the incorporation of the reformulated items and the exclusion of those that did not meet the established criteria.

Although Kendall's coefficients are moderate (W = 0.27 for relevance and W = 0.24 for clarity), these values are consistent with the diverse nature of the expert panel and the complementary qualitative approach of the Delphi method. Previous studies have highlighted that intermediate levels of agreement may be methodologically acceptable in heterogeneous panels, particularly when combined with qualitative analyses that enrich interpretation and facilitate informed adjustments to the items (Grand-Guillaume-Perrenoud et al., 2023; Reguant-Álvarez & Torrado-Fonseca, 2016). Therefore, the magnitude of W reflects a reasonable balance between diversity of perspectives and sufficient consensus to support the content validity of the instrument.

Internal reliability analysis was performed on the pilot sample (N = 128 teachers) using Cronbach's alpha coefficient, both for the individual dimensions and for the instrument as a whole. All coefficients greatly exceeded the reference value of 0.85, indicating high internal consistency between the items in each dimension and in the questionnaire as a whole:

Table 3 Table 3 Internal consistency (Cronbach's alpha)

Dimension	Cronbach's Alpha
General knowledge about AI	0.88
Pedagogical use of AI	0.9
Ethics and considerations	0.87
Curricular integration of AI	0.91
Total AI-ED-SAT	0.93

In addition to the internal consistency analysis, descriptive statistics were calculated for the scores obtained in the pilot test. The means per dimension ranged from 3.2 to 4.1 (on a 1 -5 Likert scale), indicating a generally positive perception among teachers. The dimension with the highest score was “Pedagogical use of AI” (M = 4.1, SD = 0.6), followed by “General knowledge about AI” (M = 3.8, SD = 0.7). The dimensions “Curricular integration” and “Ethics and considerations” had means of 3.5 (SD = 0.8) and 3.2 (SD = 0.9), respectively, suggesting areas with greater scope for professional development among teachers.

This result supports the internal consistency of the instrument and confirms that the items are grouped logically and consistently within their respective constructs.

To explore the underlying structure of the questionnaire, exploratory factor analysis (EFA) was performed using the maximum likelihood method with Varimax rotation. The sample adequacy was excellent (KMO = 0.91) and Bartlett's sphericity test was significant (χ² (df = 703) = 3982.23, p < .001), which justified the application of the factorial model.

Considering the geographical diversity of the sample, special attention was paid to possible lexical and usage differences between Spain and Latin America. During the design of the items, idiomatic expressions and regional technical terms were avoided, prioritizing clear, neutral, and accessible language. Both the Delphi phase and the pilot test included spaces for linguistic observations, which were analyzed qualitatively and, where necessary, led to the reformulation of potentially ambiguous terms. This care contributed to improving the semantic validity of the instrument and its applicability in diverse Spanish-speaking educational contexts.

The EFA revealed four main factors, in line with the proposed theoretical structure, which together explained 67.8% of the total variance. Factor loadings remained above 0.60 in all dimensions, confirming an adequate level of item saturation in their respective factors.

During this process, the 40 original items were used. For the presentation in Table 4, a reduced set of 10 representative items (2-3 per dimension) was selected to illustrate the most prominent factor loadings concisely and facilitate readability. The complete matrix, which includes the 38 final items after wording adjustments and statistical refinement, along with their specific communalities and variances, is presented in Appendix 2.

Table 4 Table 4 Factor loadings of selected items (EFA)

Item	Factor loading (EFA)
Item 1	0.72
Item 2	0.81
Item 3	0.75
Item 4	0.77
Item 5	0.8
Item 6	0.69
Item 7	0.73
Item 8	0.78
Item 9	0.74
Item 10	0.82

These results show that the selected items maintain a clear and consistent structure in each dimension and are representative of the factorial pattern obtained for the instrument as a whole.

To empirically confirm the structural validity of the questionnaire, a confirmatory factor analysis (CFA) was conducted on the theoretical model derived from the EFA. The model evaluated was a two-factor model, which allowed for the verification of both the general common variance and the specific variance attributable to each dimension.

The fit indices obtained were:

χ²/df = 2.34

These values fall within the ranges established as indicators of excellent fit (Hu & Bentler, 1999), confirming that the structure of the AI-ED-SAT instrument adequately reflects the proposed theoretical model. Specifically, the CFI (Comparative Fit Index) and TLI (Tucker-Lewis Index) exceed the threshold of 0.95, while the RMSEA remains below 0.06, indicating the model's validity from a multidimensional perspective.

This analysis provides solid evidence that the questionnaire has a robust factorial structure, capable of accurately capturing the four theoretical components proposed: knowledge about AI, pedagogical use, ethics and considerations, and curricular integration.

In line with the Delphi method-based methodological process, and with the aim of ensuring the clarity, relevance, and theoretical consistency of the items, the instrument was refined after qualitative and quantitative analysis of the validation rounds. In this process, two items were eliminated because they did not meet the established relevance threshold, and six more were reformulated due to issues of clarity, ambiguity, or suitability for the teaching profile. Table 5 presents a summary of the actions taken on the items during this expert review phase.

Table 5 Table 5 Items eliminated and reformulated after the Delphi process

Original item	Action taken	Justification	New version (if applicable)
Item 5 – “I understand the mathematical and logical foundations of AI.”	Removed	Low level of relevance for the general teaching profile; excessively technical content; did not exceed the 80th percentile.	—
Item 12 – “I have designed learning activities that incorporate AI.”	Removed	Uncommon level of implementation in the average teaching profile; low relevance as perceived by experts.	—
Item 4 – “I am familiar with terms such as machine learning and neural networks.”	Reformulated	Technical language lacking sufficient clarity; more accessible wording suggested.	I am familiar with general terms related to AI, such as “machine learning” or “neural networks.”
Item 7 – “I can explain to other teachers how basic AI works.”	Reformulated	Ambiguity in the term “basic AI”; need for greater precision.	I can explain the fundamental concepts of AI to other teachers in a simple way.
Item 16 – “I use chatbots or virtual assistants to answer students' questions.”	Reformulated	Technical ambiguity and diversity of interpretation; imprecise wording.	I use tools such as chatbots or virtual assistants to help answer students' questions.
Item 27 – “I identify the social impacts of AI in education.”	Reformulated	Item too broad; specifying the focus is recommended.	I identify how AI can affect access, equity, and social interaction in educational contexts.
Item 33 – “I am familiar with curriculum frameworks that incorporate AI in education.”	Reformulated	Ambiguity about the availability of frameworks; context not easily generalizable.	I am familiar with examples of curriculum proposals or policies that promote the inclusion of AI in education.
Item 36 – “I use AI-based simulations to reinforce concepts in the classroom.”	Reformulated	Unclear terminology; more practical and concrete wording is suggested.	I use AI-supported simulations or interactive tools to reinforce student learning.

The growing incorporation of digital and hybrid modalities in higher education requires university teacher training to integrate specific skills for teaching mediated by emerging technologies, including Artificial Intelligence. AI-ED-SAT, with its four-dimensional structure and self-reflective nature, offers an operational framework for diagnosing and developing these skills in diverse academic contexts. In digital environments, its results can guide the selection of AI-based resources, strategies, and pedagogical approaches, promoting the adaptation of content and methodologies to virtuality. In hybrid modalities, the instrument facilitates the identification of practices that coherently integrate the potential of AI in both physical classrooms and virtual environments, promoting pedagogical coherence and digital inclusion. Its application in university professional development programs also enables the design of personalized, evidence-based training itineraries, enhancing adaptive learning and the continuous updating of teachers in response to technological advances.

One of the distinctive strengths of the tool lies in its self-assessment format, which encourages personal reflection and positions teachers as active protagonists in their continuing education. Unlike assessment models focused on generic digital competencies, the AI-ED-SAT focuses on a specific technology—AI—and its contextualized pedagogical use, which represents an innovative and necessary contribution in the face of the rapid integration of AI systems into educational platforms, virtual learning environments, and automated assessment devices (Baltazar, 2023; Owan et al., 2023).

From a methodological perspective, the combination of the Design-Based Research (DBR) approach with the Delphi technique provided rigor, flexibility, and contextual sensitivity in the development of the instrument. This approach enabled iterative and consensual development, articulating the theoretical foundation through practical validation by a panel of experts. Previous research has supported the effectiveness of this methodological combination in developing complex educational instruments (Diefes-Dux et al., 2010; Rachbauer et al., 2025), especially when a balance between technical soundness and pedagogical appropriateness is required.

These findings are consistent with previous studies that have identified greater teacher proficiency in the technical use of digital tools compared to more complex ethical or curricular dimensions (Celik, 2023; Chen et al., 2020). Compared to similar instruments focused on general digital competencies, the AI-ED-SAT presents slightly higher means on items related to the specific pedagogical application of AI. This difference can be attributed to the instrument's contextualized approach, which is designed specifically to capture teachers' perceptions of the use of artificial intelligence in real educational contexts.

Despite the promising results, it is essential to acknowledge certain methodological limitations that affect the interpretation of the findings. First, the pilot sample was selected for convenience, which introduces potential bias and limits the generalization of the results to other educational contexts. Second, the instrument was validated exclusively in Spanish-speaking countries, which restricts its cross-cultural applicability. These limitations should be taken into account when evaluating the validity and utility of the questionnaire in other educational settings.

In this sense, the results should be understood as a preliminary validation of the instrument. It is recommended that the study be replicated in larger and more diverse samples, particularly in higher education contexts and virtual learning environments, to strengthen the generalizability of the findings and refine the instrument's sensitivity to different educational realities.