Introduction

Thomas, French and Humphries (1986) defined sport domain as a complex system of tactical knowledge production about the current situation and past events (declarative tactical knowledge), combined with player’s learning ability during technique acquisition process (procedural knowledge). Declarative tactical knowledge in sport refers to the knowledge on rules, players’ positions/functions, basic offensive and defensive strategies, and understanding of game tactical-technical logic (to know what to do) (Thomas et al., 1986). Procedural tactical knowledge (PTK) refers to performance and creation of movements (techniques), to know what and when to do, selecting the most adequate actions according to different competition situations (French and Thomas, 1987).

The assessment of PTK in team sports and in basketball, specifically, contributes to indirectly verify whether planning and structuring of contents and activities during the learning/training process actually favor the learning of tactics and techniques of that sport (Arias and Castejon, 2012; Memmert and Harvey, 2008).

Literature reports the development and validation of several instruments to assess PTK in basketball. French and Thomas (1987) proposed an observation tool for the analysis of game tactical performance, based on ball control, decision-making, and technical skill execution, through the assessment of throwing and passing techniques. Chen, Hendricks and Zhu (2013) developed and validated the Basketball Offensive Game Performance Instrument (BOGPI). It evaluates player’s offensive tactical performance in 3vs.3 situations, based on technical skill execution, decision-making, and support to the player with the ball during the execution of dribbling, passing, and throwing. Folle et al. (2014) developed the Instrument of Assessment of Individual Technical-tactical Performance in formative categories in basketball. This tool allows the analysis of individual game actions in offense and defense, as well as the components of technical-tactical performance related to adaptation, decision-making, and efficacy in competitive matches. Martínez and Ibáñez (2016) proposed the Instrument of Assessment of Learning and Performance in Basketball (IALPB), which evaluates offensive actions performed by players with or without the ball and actions performed by players in defense of the players with or without the ball. Beside the evaluation of each individual action based on decision-making and technical execution, the IALPB is innovative in the assessment of actions final efficacy.

Based on the presented literature, the current instruments proposed for evaluating PTK in basketball assess only individual actions of players in offense or defense, during competitive matches or 3vs.3 games in half basketball court (only one hoop). This means that actions involving two or three players, for example, on ball and out of ball screens, are not evaluated or only the player who sets the screen is assessed. These collective actions are also important during basketball learning-training process and, therefore, should be assessed.

Therefore, the aim of this study was to develop and perform a preliminary validation of an instrument for assessing PTK in basketball in 12-19 years old players in a 3vs.3 situation in half (one hoop) or full basketball court (two hoops). The assessment was performed via the observation of individual offensive (player with or without the ball) and defensive actions (on ball defender or out of ball defender), as well as collective actions involving two or three players in offense and defense.

Methods

Type of Study

The development, preliminary validation, and analysis of psychometric properties of the Procedural Tactical Knowledge Test for Basketball (PTKT: Bb) configures an instrumental investigation design (Ato, López and Benavante, 2013). Therefore, this study followed the model proposed by Pasquali (2010) for elaborating psychometric instruments, which is based on theoretical, empirical (experimental), and analytical (statistical) procedures.

Participants

This study was approved by the local research ethics committee. Content analysis of the first version of PTKT: Bb was performed by five judges: three basketball coaches with sport sciences graduation and at least a specialist degree on team sports area and two researchers on team sports teaching. All judges had a mean experience of 15.4 years as basketball coaches in formative categories or as a researcher on sports pedagogy, respectively.

The PTKT: Bb was performed by 161 male basketball players aged between 12 and 19 years (see Table 1).

Within and between-observer reliability of PTKT: Bb was verified based on the scores observed by two Physical Education teachers with a minimum experience of 10 years as basketball coaches in elite formative categories. These observers were chosen in order to favor the utilization of PTKT: Bb by the target audience to whom the instrument was developed, that is, Physical Education teachers and basketball coaches.

Variables

The development and preliminary validation of the PTKT: Bb identified as variables the content validity for theoretical procedures and reliability and construct validity for experimental and analytical procedures.

Content Validity

Content validity of the first version of the PTKT: Bb was calculated using the Content Validity Coefficient (CVC) proposed by Hernández-Nieto (2002). The three categories that constitute content validity, namely, language clarity (LC), practical relevance (PR), and theoretical relevance (TR), were calculated based on the answers given by judges, using a 5-points Likert scale.

Reliability

Considering the subjective component during the evaluation process in PTKT: Bb, betweenobserver reliability was determined as the stability of results provided by the two observers for the assessment of the 161 players. Within-observer reliability was based on stability of results from the first to the second evaluations performed by the same observer at different moments (Atkinson and Nevill, 1998).

Construct Validity

Representability of the 34 items/criteria initially proposed for the PTKT: Bb to assess PTK was investigated. The procedure used to verify the existence of construct validity of PTKT: Bb was an exploratory factorial analysis (EFA), as recommended by the literature on psychometrics Pasquali (2010).

Procedures

The PTKT: Bb items elaboration process was based on various instruments of assessment of PTK in team sports (Gréhaigne, Godbout and Bouthier, 1997; Gréhaigne, Richard and Griffin, 2005; Memmert, 2006; Memmert and Roth, 2003) and in basketball (French and Thomas, 1987; Lamas et al., 2011). Content validity was, then, verified calculating the CVC categories LC, PR, and TR of the 34 items initially proposed for the instrument.

Elaboration and application of protocols A and B were performed to determine whether the tactical individual and collective actions in offense and defense were actually performed by players.

After protocols were performed, two observers analyzed players’ actions according to the items validated for content. Both observers were previously trained for the use of PTKT: Bb and the assessment of players’ PTK. Each observer received a DVD with videos for training, a document with instructions on the observation and registration processes, a DVD with the videos of protocols A and B, and a registration form. After observers training process and clarification of all doubts, the evaluation of players’ actions started and the within and betweenobserver reliability and construct validity of the PTKT: Bb were verified.

Procedures to verify PTKT: Bb content validity followed the recommendations of literature on psychometry (Cassepp-Borges, Balbinotti and Teodoro, 2010; Pasquali, 2010), which comprised the evaluation of items by five judges using a Likert scale and calculation of CVC. Literature reports the use of these procedures for the elaboration and validation of instruments used for analysis of formation of expert players in team sports (García-Martín, Antúnez and Ibáñez, 2016), assessment of referees’ performance (García-Santos and Ibáñez, 2016), control of socioemotional factors in young soccer players (Gomez-Carmona, Cervera and Benito, 2014), and assessment of basketball teaching-learning process in scholar players (González, Ibáñez, Feu and Galatti, 2017). However, these studies differ in the number of judges (11 to 14) who evaluated instruments’ content and, therefore, report the use of V of Aiken (Aiken, 1985) or inferential statistics to obtain an objective measure of content validity.

Instrument

The PTKT: Bb aims to assess PTK in basketball, through the observation and registration of individual and collective actions in offense and defense during a 3vs.3 situation in half court with only one hoop (protocol A) or full court with two hoops (protocol B), according to a verification list that served as an absolute classification grade (Thomas, Nelson and Silverman, 2007).

Initially, there were 34 items, comprising 16 individual (eight in offense and eight in defense) and 18 collective actions. The latter were subdivided in collective actions between two (four in offense and four in defense) or three (five in offense and five in defense) players. These actions aimed to identify players’ behaviors and their frequencies. Only successful actions both in offense and defense were registered. Player’s final assessment is given by the total number of action performed, which allows the analysis of relative frequency or percentile.

The instrument establishes a 4-minute duration for both protocols in half court and full court. In 3vs.3 in half court rules were the same as in the FIBA (International Basketball Federation) 3X3, while in 3vs.3 in full court rules were the same as the official FIBA rules for 5vs.5.

For the application of PTKT: Bb two teams of three players each must be formed and athletes differentiated by numerated vests from 1 to 6. Team formation criterion is based on the subjective perception of teacher/coach of the level of tactical-technical skills and physical conditioning of participants, in order to compose balanced teams. The protocol is filmed for later observation and analysis of actions according to the PTKT: Bb items. Camera used to film protocol A was positioned between the 3-point line and the mid court line of the basketball court. For protocol B, the camera was positioned out of the basketball court, at a distance of 2 meters at the extension of the mid court line.

Protocols were filmed with two cameras (JVC® HD-520 and SONY® DCR-SR45). Data analysis was performed in software SPSS® (Statistical Package for Social Science) version 19.0 for Windows®. Microsoft Excel 2010 for Windows® was used to calculate CVC.

Data Analysis

CVC was calculated in order to determine content validity of LC, PR, and TR for each item of the instrument (CVCi) and for the instrument as a whole (CVCt). Only the items that reached a CVC value above .80 for PR and TR were included in the instrument. The items that did not reach this cut off value for LC were reformulated and reevaluated by the judges (HernándezNieto, 2002).

The Kappa of Cohen coefficient was used to verify within and between-observer reliability. The within-observer reliability was based on 10% (2,866 actions) of observed actions (Tabachnick and Fidell, 2001), which were reevaluated after 15 days from the first evaluation (Hill and Hill, 2008). For between-observer reliability all 12,558 actions performed by the 161 players in both protocols were considered. The level of agreement was classified as suggested by Landis and Koch (1977).

Evidence of construct validity was investigated through a separate EFA for each protocol (A and B), as well as for the individual actions in offense and defense, and for collective actions between two or three players in offense and defense (Pasquali, 2010). Kaiser-Meyer-Olkim measure of sampling adequacy (KMO) and Bartlett’s Test of sphericity were calculated. The principal component analysis and initial Eigenvalue criterion were used in order to extract and determine the number of factors. The Varimax orthogonal rotation with Kaiser normalization was also used (Field, 2009).

Results

Theoretical procedures: evidence of PTKT: Bb content validity

The items 1 (player in offense moves in order to occupy a free space) and 5 (player in offense dribbles the ball in order to create space and break defense) were excluded from the instrument (CVC < .80) (see Table 2).

Analytical procedures: PTKT: Bb reliability

The PTKT: Bb presented substantial within-observer agreement for protocol A and almost perfect agreement for protocol B. Results also showed between-observer substantial agreement for both protocols (see Table 3).

Analytical procedures: PTKT: Bb construct validity

Models 1 and 2 refer to the individual and collective actions, respectively, for protocol A (see Tables 4 and 5). For protocol B, the EFA identified one satisfactory final model only for individual actions. The model for collective actions did not reach the minimum number of three items for the dimension defense and, therefore, protocol B was excluded from the preliminary version of the instrument. KMO values for protocol A were above .50, but were interpreted as medium values, as recommended in the literature (from .70 to .79) (Field, 2009).

Discussion

Procedures to verify PTKT: Bb content validity followed the recommendations of literature on psychometry (Cassepp-Borges, Balbinotti and Teodoro, 2010; Pasquali, 2010), which comprised the evaluation of items by five judges using a Likert scale and calculation of CVC. Literature reports the use of these procedures for the elaboration and validation of instruments used for analysis of formation of expert players in team sports (García-Martín, Antúnez and Ibáñez, 2016), assessment of referees’ performance (García-Santos and Ibáñez, 2016), control of socioemotional factors in young soccer players (Gomez-Carmona, Cervera and Benito, 2014), and assessment of basketball teaching-learning process in scholar players (González, Ibáñez, Feu and Galatti, 2017). However, these studies differ in the number of judges (11 to 14) who evaluated instruments’ content and, therefore, report the use of V of Aiken (Aiken, 1985) or inferential statistics to obtain an objective measure of content validity.

Reliability and construct validity evidence of PTKT: Bb

Preliminary validation procedure of PTKT: Bb presented substantial within-observer agreement for protocol A, almost perfect agreement for protocol B, and substantial betweenobserver agreement for both protocols. These results are lower than those reported by Martínez and Ibáñez (2016) for the IALPB, which presented between-observer values of reliability above .81, classified as almost perfect agreement (Landis and Koch, 1977). Considering that this is a preliminary procedure, the next phases of validation process aim to reach satisfactory reliability with between-observer reliability values above .80 (Matos, 2014).

Several studies on the development and validation of instruments of assessment of PTK in team sports (Collet, Nascimento, Ramos and Stefanello, 2011; Costa, Garganta, Greco, Mesquita and Maia, 2011; Memmert, 2006; Memmert and Roth, 2003) and in basketball (Chen et al., 2013; Folle et al., 2014; Martínez and Ibáñez, 2016) used the within and between-observer agreement as measures of construct validity. Literature states that EFA is a statistical method often used to verify construct validity of questionnaires, as described in soccer literature (Gomez-Carmona et al., 2014) and sports psychology (Guedes and Netto, 2013). In the case of PTKT: Bb, EFA showed evidence of acceptable construct validity for protocol A, which allows the utilization of this procedure as an alternative way of measuring construct validity in future instruments of assessment of PTK in team sports.

With respect to factorial models extracted from EFA, results showed that items were grouped into two factors/dimensions for individual actions in protocol A. Items grouped in both factors represented basic principles in offense (ball possession maintenance, ball transportation, and scoring points) and defense (ball possession regain and defense of goal post/hoop) (Bayer, 1986; Gréhaigne et al., 1997), as proposed in literature.

Literature also reports the assessment of the individual actions in offense and defense of PTKT: Bb in competitive matches. Several studies report differences in the frequency of utilization of actions involving space creation without the ball, space creation with ball dribbled and space creation with ball not dribbled in male players of different ages, from mini basketball to professional categories (Lamas et al., 2011), the influence of type of pass (inside pass) in offense efficacy (Courel, Suárez, Ortega, Piñar and Cárdenas, 2013), the influence of type of defense, switch defense and level of opposition as criteria for differentiating between winnining and losing teams (Álvarez, Ortega, Gómez and Salado, 2009), and players’ preference of defensive actions of stealing the ball and intercepting passes during matches (Montero, Ezquerro and Saavedra, 2009). Regarding formative categories, U’18 teams differ from senior teams in the number of offensive and defensive rebounds (García, Ibáñez, Parejo, Cañadas and Feu, 2010) and in the total number of 2-point shots (García, Ibáñez and Feu, 2010). Besides, U’14 and U’16 categories of both genders are characterized by a higher number of ball possessions compared to professional categories, and the number of 3-point shots is the main aspect that differentiates U’16 players from U’14 players (García, Ibáñez, Parejo, Feu and Cañadas, 2011).

As discussed above, the assessment proposed by PTKT: Bb included the frequency of defensive rebound, frequency of shots after creating space with dribbling, and frequency of passes to a free teammate, which favors the teaching-learning process assessment in basketball, as well as the identification of players’ and teams’ level of performance in formative categories.

With respect to the factorial model for collective actions in offense and defense for protocol A, several studies investigated these actions in competitive matches. Lamas et al. (2011) reported that the on ball screen was the most frequent action against man-to-man defense in 18-19 years old players. On the other hand, the out of ball screen was the less frequent action among all players from young to senior. Álvarez et al. (2009) found that both winning and losing teams performed equal frequencies of the action called “support to the player with the ball” and both presented high percentages of efficacy in this action. Finally, the actions involving three players in defense are contents recommended by literature on sports pedagogy and on basketball, which aim to develop players’ skills at switch defense, help defense, defense rotations, and strong or weak side defenses (Remmert, 2003).

Conclusions

Results suggest that the preliminary version of PTKT: Bb presents satisfactory psychometric properties for content validity. With respect to reliability, this instrument presented low between-observer agreement levels. It also presented acceptable psychometric properties for construct validity in protocol A. The continuation of the validation process of PTKT: Bb is recommended in order to obtain satisfactory psychometric properties. Literature on psychometry suggests a new data collection with a greater number of participants, a new verification of within and between-observer reliability, and a new analysis of construct validity (Pasquali, 2010), such as confirmatory factorial analysis or discriminant analysis. After these procedures and the confirmation of construct validity, the test should be normalized, reaching satisfactory psychometric properties.

References

Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1), 131–142. https://doi.org/10.1177/0013164485451012

Álvarez, A.; Ortega, E.; Gómez, M. A., & Salado, J. (2009). Study of the defensive performance indicators in peak performance basketball. Revista de Psicologia Del Deporte, 18(3), 379–384.

Arias, J., & Castejon, F. J. (2012). Review of the Instruments Most Frequently Employed to Assess Tactics in Physical Education and Youth Sports. Journal of Teaching in Physical Education, 31, 381–391. https://doi.org/10.1123/jtpe.31.4.381

Atkinson, G., & Nevill, A. M. (1998). Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Medicine, 26(4), 217–238. https://doi.org/10.2165/00007256-199826040-00002

Ato, M.; López, J. J., & Benavente, A. (2013). Un sistema de clasificación de los diseños de investigación en psicología. Anales de Psicología, 29(3), 1038–1059. https://doi.org/10.6018/analesps.29.3.178511

Bayer, C. (1986). La enseñanza de los juegos deportivos colectivos. Barcelona: HispanoEuropea.

Cassepp-Borges, V.; Balbinotti, M. A. A.; & Teodoro, M. L. M. (2010). Tradução e validação de conteúdo: uma proposta para adaptação de instrumentos. In L. Pasquali (Ed.), Instrumentação Psicológica: fundamentos e práticas (pp. 506–520). Porto Alegre: Artmed.

Chen, W.; Hendricks, K., & Zhu, W. (2013). Development and validation of the basketball offensive game performance instrument. Journal of Teaching in Physical Education, 32, 100–109. https://doi.org/10.1123/jtpe.32.1.100

Collet, C.; Nascimento, J. V. D.; Ramos, V., & Stefanello, J. M. F. (2011). Construction and validation of a technical-tactical performance evaluation instrument in volleyball. Revista Brasileira de Cineantropometria & Desempenho Humano, 13(1), 43–51.

Costa, I. T.; Garganta, J.; Greco, P. J.; Mesquita, I.; & Maia, J. (2011). Sistema de avaliação táctica no Futebol (FUT-SAT): Desenvolvimento e validação preliminar preliminary validation, 7, 69–84.

Courel, J.; Suárez, E.; Ortega, E.; Piñar, M., & Cárdenas, D. (2013). Is the inside pass a performance indicator? Observational Analysis of elite basketball teams. Revista de Psicologia Del Deporte, 22(1), 191–194.

Field, A. (2009). Descobrindo a estatística usando o SPSS. Porto Alegre: Artmed.

Folle, A.; Quinaud, R. T.; Barroso, M. L. C.; Rocha, J. C. S.; Ramos, V., & Nascimento, J. V. (2014). Construção e validação preliminar de instrumento de avaliação do desempenho técnico-tático individual no basquetebol. Construção E Validação Preliminar de Instrumento de Avaliação Do Desempenho Técnico-Tático Individual No Basquetebol, 22(3), 405–418. https://doi.org/10.4025/reveducfis.v25i3.23085

French, K., & Thomas, J. (1987). The relation of knowledge development to children’s basketball performance. Journal of Sport Psychology, 9, 15–32. https://doi.org/10.1123/jsp.9.1.15

García-Martín, A.; Antúnez, A., & Ibáñez, S. (2016). Análisis del proceso formativo en jugadores expertos: validación de instrumento. Revista Internacional de Medicina Y Ciencias de La Actividad Física Y Del Deporte, 16(61), 157–182. https://doi.org/10.15366/rimcafd2016.61.012

García-Santos, D., & Ibáñez, S. (2016). Diseño y validación de un instrumento de observación para la valoración de un árbitro de baloncesto (IOVAB). SPORT TK-Revista EuroAmericana de Ciencias Del Deporte, 5(2), 15–26.

García, J.; Ibáñez, S., & Feu, S. (2010). Estadísticas de juego que discriminan las selecciones participantes en los Campeonatos del Mundo de 2006 de Baloncesto, en función del nivel y género de los equipos. Kronos, 17, 57–63.

García, J.; Ibáñez, S.; Parejo, I.; Cañadas, M., & Feu, S. (2010). Análisis de los campeonatos del mundo de baloncesto masculino (2002 y 2006). Diferencias entre jugadores con diferentes niveles de experiencia (Senior y Junior). Motricidad: European Journal of Human Movement, 24, 133–145.

García, J.; Ibáñez, S.; Parejo, I.; Feu, S., & Cañadas, M. (2011). Diferencias entre nivel de juego y categoría de los jugadores en etapas de formación. Revista Española de Educación Física y Deportes, 395, 13–28.

Gomez-Carmona; P. M., Cervera, V., & Benito, P. J. (2014). Diseño y validación de un cuestionario socio-emocional para jóvenes futbolistas de élite. Revista Internacional de Medicina y Ciencias de La Actividad Fisica Y Del Deporte, 14(55), 545–559.

González, S. E.; Ibáñez. S.; Feu, S. M. & Galatti, L. R. (2017). Programas de intervención para la enseñanza deportiva en el contexto escolar, PETB y PEAB: Estudio preliminar. Retos, 31, 107–113.

Gréhaigne, J. F.; Godbout, P., & Bouthier, D. (1997). Performance assessment in team sports. Journal of Teaching in Physical Education, 16(4), 500–516. https://doi.org/10.1123/jtpe.16.4.500

Greháigne, J. F.; Richard, J. F., & Griffin, L. L. (2005). Teaching and Learning Team Sports and Games. New York and London: Routledge.

Guedes, D. P., & Netto, J. E. S. (2013). Participation Motivation Questionnaire: tradução e validação para uso em atletas-jovens brasileiros. Revista Brasileira de Educação Física e Esporte, 27(1), 137–148. https://doi.org/10.1590/S1807-55092013005000003

Hernández-Nieto, R. A. (2002). Contributions to Statistical Analysis. Mérida: Universidad de Los Andes.

Hill, M., & Hill, A. (2008). Investigação por questionário. Lisboa: Sílabo.

Lamas, L.; Rose-Junior, D.; Santana, F.; Rostaiser, E.; Negretti, L., & Ugrinowitsch, C. (2011). Space creation dynamics in basketball offence: validation and evaluation of elite teams. International Journal of Performance Analysis in Sport, 11, 71–84. https://doi.org/10.1080/24748668.2011.11868530

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159–174. https://doi.org/10.2307/2529310

Martínez, S., & Ibáñez. S. (2016). Diseño y validación de un instrumento para la medida del aprendizaje y el rendimiento en baloncesto (IMARB) (Unpublished Master’s thesis). Facultad de Ciencias del Deporte. Universidad de Extremadura, Cáceres.

Matos, D. A. S. (2014). Confiabilidade e concordância entre juízes: Aplicações na área educacional. Estudos Em Avaliação Educacional, 25(59), 298–324. https://doi.org/10.18222/eae255920142750

Memmert, D. (2006). Developing creative thinking in a gifted sport enrichment program and the crucial role of attention processes. High Ability Studies, 17(1), 101–115. https://doi.org/10.1080/13598130600947176

Memmert, D., & Harvey, S. (2008). The Game Performance Assessment Instrument (GPAI): Some Concerns and Solutions for Further Development. Journal of Teaching in Physical Education, 27, 220–240. https://doi.org/10.1123/jtpe.27.2.220

Memmert, D., & Roth, K. (2003). Diagnostics of individual tactical performance in sports games. Spektrum Der Sportwissenschaft, 15, 44–70.

Montero, A.; Ezquerro, M., & Saavedra, M. (2009). Factorial Analysis of the Skills Preferred by Basketball Players: A Study in Young Players. Revista de Psicología Del Deporte, 18(3), 429–432.

Pasquali, L. (2010). Testes referentes a construto: teoria e modelo de construção. In L. Pasquali (Ed.), Instrumentação Psicológica: fundamentos e práticas (pp. 165–198). Porto Alegre: Artmed.

Remmert, H. (2003). Analysis of Group-Tactical Offensive Behavior in Elite Basketball on the Basis of a Process Orientated Model. European Journal of Sport Science, 3(3), 1– 12. https://doi.org/10.1080/17461390300073311

Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). Needham Heights, MA: Allyn & Bacon.

Thomas, J. R.; French, K. E., & Humphries, C. A. (1986). Knowledge development and sport performance: Directions for motor behavior research. Journal of Sport Psychology, 8, 259–272. https://doi.org/10.1123/jsp.8.4.259

Thomas, J. R.; Nelson, J. K., & Silverman, S. J. (2007). Métodos de pesquisa em atividade física (5th ed.). Porto Alegre: Artmed.