In recent decades, numerous scientific studies have addressed the question of the conceptualisation and measurement of the psychological skills or strategies deployed by athletes during competition due to their close connection with sporting performance during training and competition (Gimeno, Buceta, & Pérez-Llantada, 2007; Hardy, Roberts, Thomas, & Murphy, 2010; Kremer & Morán, 2008; Smith, Schutz, Smoll, & Ptacek, 1995). The psychological skills measurement models that have acquired the greatest degree of scientific notoriety include the Psychological Skills Inventory for Sport (PSIS; Mahoney, Gabriel, & Perkins, 1987); the Athletic Coping Skills Inventory (ACSI-28; Smith et al., 1995); the Cuestionario de Características Psicológicas Relacionadas con el Rendimiento Deportivo (CPRD, Gimeno, Buceta, & Pérez-Llantada, 2001); the Ottawa Mental Skills Assessment Tool (OMAT-3, Durand-Busch, Salmela, & Green- Demers, 2001) and the Test of Performance Strategies (TOPS, Thomas, Murphy, & Hardy, 1999).

The original TOPS (Thomas et al., 1999) was based on the conclusions drawn from the comparative studies of various measurement instruments conducted by Vealey (1988), Thomas and Over (1994) and Hardy, Jones and Gould (1996). These authors posited a measurement model using 7 factors or psychological skills deployed by athletes during training and competition, namely Self-talk (instructions and messages of encouragement athletes address to themselves), Emotional control (the capacity to regulate negative emotions), Automaticity (the ability to perform movements and actions whilst competing without awareness), Goal setting (the setting of specific goals related to the competitive effort), Imagery (the mental visualization of the circumstances and sensations surrounding competition), Activation and relaxation (the regulation of arousal to optimum levels and the use of cognitive and behavioural techniques to reduce tension), and Attentional control (the ability to control intrusive thoughts and refocusing attention). A series of factor analyses led the authors to create two further versions of TOPS, known as TOPS-2 and TOPS-3 (Hardy et al., 2010; Thomas, Hardy, & Murphy, 2007) in which they extended the original measurement model by increasing the number of factors from 7 to 9. This was achieved by splitting Activation and Relaxation into two separate factors and adding a further factor, which they termed Negative thinking (the appearance of failure-related thoughts that may appear whilst competing). TOPS-3 therefore contains two subscales: one that can be applied to training sessions and another for use in competition, each containing the same 9 factors. Numerous research studies have obtained favourable evidence for the psychometric characteristics of the two subscales in a range of contexts and cultures (Debois, Quillet, Sylvestre, & Calmels, 2004; Fletcher & Hanton, 2001; Jackson, Thomas, Marsh, & Snethurst, 2001; Katiskas, Donti, & Psychountaki, 2011; Saadatifard, Keshtidar, & Khoshbakhti, 2014).

TOPS is currently considered to be one of the most widely used tests in the field of sports psychology (Weinberg & Gould, 2010), and is commonly used to assess the efficiency of programmes designed to improve athletes’ psychological skills (Woodcock, Duda, Cumming, Sharp, & Holland, 2012). Furthermore, it is the only specialised instrument for measuring psychological skills within the context of training (Weinberg & Forlenza, 2012). The aim of this study was to psychometrically adapt the TOPS-3 competition subscale to Spanish culture, thereby providing a useful tool for assessing psychologically skills that are closely linked to optimising sport performance.

Method

Participants

A total of 1003 Spanish athletes of both sexes participated in the study (75.7% men, 24.3% women) from 43 different sports, including football (22.8%), athletics (7.8%), Gaelic football (6.6%), futsal (5.4%), triathlon (4.2%), basketball (4.1%), orienteering (4%) and volleyball (3.3%). The remaining 41.8% practised the following sports: motorcycling, swimming, padel, tennis, long distance running (more than 5,000 m), canoeing, tennis, handball, rugby, chess, weightlifting, billiards, mountain sports, archery, judo, table tennis, cycling, gymnastics, fencing, American football, underwater sports, competitive dance sport, water polo, taekwondo, boxing, sailing, wrestling, equestrianism, skating, pétanque, shooting, karate, skiing, air sports, surfing and car racing. All participants had to be aged 18 or over and holders of a federation license at the time of the study. Failure to comply with these criteria meant that athletes under 18 and that did not hold a federation license were excluded from the study, even though they were training and competing regularly. The participants were aged between 18 and 62 (M = 29.25; SD = 10.95). 22.9% were competing at a local or regional level; 49.5% at autonomous community level, 22.4% nationally and 5.2% internationally. The number of weekly training sessions ranged between 1 and 12 (M = 3.48; SD = 1.71), lasting between 15 and 500 minutes (M = 101.36; SD = 39.80). The number of years’ experience in sport stood at between 1 and 49 (M = 11.08; SD = 7.74).

Instruments

For the adaptation to Spanish, the competition subscale of the latest version of the Test of Performance Strategies (TOPS-3; Thomas et al., 2007) was chosen, consisting of 36 items listed under 9 factors, each comprising four items: Self-talk, Emotional control, Automaticity, Goal setting, Imagery, Activation, Relaxation, Attentional control and Negative thinking. Hardly et al. (2010) reported that Cronbach’s alpha values for this version were above the .70 cutoff for all factors except Automaticity, which obtained a score of .63. The overall goodness of fit indices for the measurement model were also satisfactory: χ2 = 1089.62 (p<.001); Non-Normed Fit Index (NNFI) = .98; Comparative Fit Index (CFI) = .98; Root Mean Square Error of Approximation (RMSEA) = .04; Standardized Root Mean Square Residual (SRMR) = .05.

Athletes were asked to state the frequency with which they deploy each of the psychological strategies included in the test when competing. Likert-scale type response options were provided, consisting of 5 options where 1 = Never; 2 = Rarely; 3 = Sometimes; 4 = Often; 5 = Always. Seven items were formulated in the opposite direction (the four items of Emotional control, items 1 and 4 of Attentional control and item 3 of Negative thinking). These scores were inverted to calculate the total score of each subscale. In addition to the TOPS-3 items, athletes completed a set of sociodemographic questions and their answers were used to describe the characteristics of the sample.

Procedure

The general recommendations proposed by Balluerka, Gorostiaga, Alonso-Arbiol and Haranburu (2007) and Muñiz, Elosua and Hambleton (2013) were used for the adaptation, which consisted of breaking down the adaptation process into several phases. Firstly, permission was obtained from the authors of the original test, who indicated that the TOPS-3 version should be used for the purpose of the adaptation. This version was then translated into Spanish by a professional translator. The translation of the items was reviewed by two experts and slight adjustments were made to the wording of several items to make them easier for the athletes to understand fully. The final phase of the process consisted of the translation back into English by another professional translator. In those cases, in which the translation failed to match the original version exactly, an agreement was reached between the translator and committee of experts whereby the wording included the original contents of the item but was adapted in such a way as to ensure that the athletes could fully understand the concept under assessment.

During a second phase, cognitive interviews were held with a pilot group comprising 12 athletes (10 men and 2 women) to obtain information on those items that were ambiguous or difficult to understand. Based on the information collected, a new version of the test was drawn up which was then given to a second pilot group of 9 athletes (5 men and 4 women), who reported no difficulties in understanding and completing the test.

Once the translation and cultural adaptation of the test items were finalised, the next stage consisted of contacting a large number of federated athletes and clubs to inform them of the aims of the research project and request their cooperation. The data were collected during November 2016 and January 2017, either at the clubs’ facilities or via an online form. A standard protocol was followed, whereby the athletes completed the test anonymously and their details were processed in accordance with Spanish data protection legislation.

Data analysis

The IBM SPSS Statistics, version 20.0 and IBM SPSS Amos Graphics, version 19, software packages were used to analyse the data. The first stage of the analysis involved calculating the descriptive statistics of the items (mean, standard deviation, skewness and kurtosis). This was followed by confirmatory factor analyses in order to obtain evidence of the measurement model’s validity. Maximum Likelihood was used to estimate the parameters and bootstrap to estimate the standard errors. The final stage consisted of estimating the reliability of the test from two perspectives (internal consistency and composite reliability) and the invariance of the measurement model.

Results

Initial description of responses to items

Table 1 shows the mean, standard deviation, skewness and kurtosis for each item included in the Spanish version of the TOPS-3. No missing or out-of-range values were detected on any of the test items. The results show mean values of between 1.95 for item 1 in Negative thinking, and 3.98 for item 1 in Attentional control. In all cases, standard deviation is equal to or less than 1.36. Negative skewness was recorded in the vast majority of items, apart from those included in the Relaxation and Negative thinking factors and item 1 of Self-talk. Kurtosis is also negative in most items, with the exception of item 2 of the Activation factor, items 1, 3 and 4 of Attentional control, and item 1 of Negative thinking, which register positive kurtosis values.

Confirmatory factor analysis

The model shown in Figure 1 was specified in accordance with the theory posited by Thomas et al. (2007). The model was over-identified with 666 elements in the variance-covariance matrix, 108 parameters for estimation (36 factor loadings, 36 error variances and 36 factor correlations) and 558 degrees of freedom.

Factor and error variance was constrained to 1, leaving 36 factor loadings and the 36 correlations free in order to prevent model under-identification on AMOS. Finally, and in order to maximise the model’s overall goodness of fit to the data, the original model was re-specified to include correlation between items 1 and 2 of the Automaticity factor, 3 and 4 of the Imagery factor, as well as 1 and 4 of Attentional control, and 3 and 4 of Negative thinking.

Table 2 shows the factor loadings (λ) and error variances (δ) with their corresponding p-values. Statistical significance is attained for all parameters (p<.003).

Pearson’s correlation coefficients between 9 factors ranged from .065 to .606, the majority of which were statistically significant (p<.01) and positive, with the exception of the correlation between Negative thinking and the other factors, which were negative. In contrast, the only correlations that failed to attain statistical significance were Automaticity, Imagery and Relaxation factors with Emotional control, and also the correlation between Relaxation and Attentional control.

The model’s overall goodness of fit indices were as follows: χ2 (554) = 1584.291 (p<.001), χ2/gl = 2.860; Goodness of Fit Index (GFI) = .918; Tucker-Lewis Index (TLI) = .925; Comparative Fit Index (CFI) = .934; Root Mean Square Error of Approximation (RMSEA) = .043 (90% CI; .041 – .046); Standardized Root Mean Square Residual (SRMR) = .055.

Reliability analysis

Table 3 shows Cronbach’s Alpha values and composite reliability for each of the TOPS-3 factors and the test total. In all cases, the values were higher than .70. Eliminating any of the test items reduced the model’s reliability. The corrected item- total correlations were above .50, with the exception of items 1 and 2 of Imagery (.456 and .492, respectively), items 1 and 4 of Activation (.464 and .482, respectively) and item 4 of Negative thinking (.476).

Invariance of the measurement model

In order to verify the measurement model’s invariance, the athletes were divided into two groups by sex: male (n = 759) and female (n = 244). The first stage of this process was to specify the original measurement model for both groups, incorporating the re-specifications of the correlations between the items listed above and with unconstrained parameter estimation for both groups (unconstrained model). A further 3 sequential models were specified with constrained parameter estimation. In the case of the first (the measurement weights model), it was constrained by equalling the factor loadings for both groups, whilst in the second (the structural covariance model), in addition to the first constraint, the condition of equality was added to the correlations between factors and the factor variances. The third and final model (the measurement residuals model) includes a third constraint, in addition to the previous two, whereby the error variances and error correlations are equal in both groups. Table 4 shows the χ2 and CFI values for each of the four models referred to previously.

Evidence of the measurement model invariance was obtained by comparing the final three models with the original unconstrained model. The difference in χ2 between the unconstrained model and the measurement weights model did not attain statistical significance (χ2 = 22.392; p>.05), which can be considered favourable evidence indicating the invariance of the factor loadings in both groups of athletes. No statistically significant difference were observed between the unconstrained and the structural covariance models (χ2 = 82,534; p>.05), an indicator of factor correlation invariance and of factor variances invariance. However, statistically significant differences were observed between the unconstrained and measurement residuals models (χ2 = 151,144; p<.05), indicating a lack of variance equivalence in errors for both groups of athletes.

Nevertheless, by replacing the classic criterion of χ2dif with that proposed by Cheung and Rensvold (2002), which measures the difference in CFI (differentiating between two models when the difference between their CFI is greater than .01), we could conclude that homogeneity or invariance of the measurement model does exist for both male and female athletes in all the parameters compared, including error variances and error correlations.

Normative data for the TOPS-3 competition subscale

The athletes’ overall scores were used to draw up the normative data (mean and standard deviation) for each of the factors included in the Spanish adaptation of the TOPS-3 competition subscale, as shown in Table 5.

Conclusions

The aim of this study was to develop an adaptation for the Spanish context of the Test of Performance Strategies 3 competition subscale (TOPS-3, Thomas et al., 2007). Work was divided into two stages: an initial phase consisting of a theoretical review of the research background in measuring athletes’ psychological skills, focusing particularly on the description of the TOPS-3; and a second, empirical phase, which included the translation of the test into Spanish, and its application on a wide sample of athletes characterised by a high degree of variability in terms of their sex, age and the sports practised, and finally an in-depth statistical analysis of the psychometric properties of the adaptation.

The resulting version includes the key characteristics of the original English version, which classifies the psychological skills deployed by athletes in competition into nine broad factors or areas, each containing four indicators (items). All the goodness of fit indicators for the measurement model (i.e. test validity) were satisfactory in accordance with the criteria proposed by Schermelleh-Engel, Moosbrugger and Müller (2003), both in terms of the overall goodness of fit and the individual fit of each of the parameters included in the model, exceeding those of other cultural adaptations such as the Greek (Donti & Katsikas, 2014; Katsikas et al., 2011) or the Iranian versions (Saadatifard et al., 2014).

Test reliability was analysed from two perspectives, both of which produced equivalent and highly positive results. Both Cronbach’s Alpha and the composite reliability values were above the .70 cutoff for each of the factors and for the overall test (Carmines & Zeller, 1979; Nunnally, 1978). Likewise, favourable evidence was found for the measurement model invariance based on the athletes’ sex. The analyses indicate equivalence in factor loadings for male and female athletes, factor correlation and factor variance. As for error variance, the comparison between models produced unfavourable results when χ2 was used, although the results were favourable when CFI was applied, as posited by Cheung and Rensvold (2002). Regarding the divergent results corresponding error variance equivalence, it should be noted that authors such as Byrne (2009) consider that they are of little relevance in invariance studies. In contrast, loading factor equivalence is of vital importance, followed by equivalence in correlations between factors and factor variances.

There are several measurement tools in Spanish that share similar objectives to the TOPS-3, although each has its own distinctive and unique features (Gimeno et al., 2001; Hernández- Mendo, Morales-Sánchez, & Peñalver, 2014; Molinero, Salguero, & Márquez, 2010; Mora, García, Toro, & Zarco, 2001). The adaptation of the TOPS-3 to the Spanish context will enable coaches, athletes and researchers to assess competition-specific psychological skills related to sporting performance (Taylor, Gould, & Rolo, 2008), based on an alternative measurement model that includes new skills such as Self-talk and Activation.

The study provides evidence of the reliability and internal validity of the factors included in the Spanish adaptation of the TOPS-3 competition subscale, although it does not provide evidence of its external validity, convergent validity (with other similar instruments), or discriminant validity (with regards to other instruments measuring related variables). Another limitation of the study may be the classical approximation followed to adapt the test, which may be complemented by incorporating new approaches such as the Item Response Theory. Additionally, a future analysis could examine the effects of the directionality of items (formulated positively or negatively) on the individual reliability of each item and on the total reliability of the measurement instrument, following Solís-Salazar (2015). Indeed, all these could be possible future lines of research, together with similar studies into the TOPS-3 adapted to the field of sports such as those conducted by Bastos, Corredeira, Probst, and Fonseca (2012) in Portugal or Goudas, Kontou and Theodorakis (2006) in Greece, or adaptations of the training subscale such as that undertaken by Saadatifard et al. (2014) in Iran.

Acknowledgements

This research has been carried out with financial support from Consellería de Cultura, Educación e Ordenación Universitaria, Xunta de Galicia (ED431B 2016/017).

References

Balluerka, N., Gorostiaga, A., Alonso-Arbiol, I., & Haranburu, M. (2007). La adaptación de instrumentos de medida de unas culturas a otras: una perspectiva práctica [Test adaptation to other cultures: A practical approach]. Psicothema, 19(1), 124-133.

Bastos, T.L., Corredeira, R., Probst, M., & Fonseca, A.M. (2012). Preliminary analysis of the psychometric properties of the Portuguese version of the test of performance strategies (TOPSp) in athletes with disabilities. European Journal of Adapted Physical Activity, 5(2), 7-22.

Byrne, B.M. (2009). Structural equation modeling with AMOS. New York: Psychology Press. https://doi.org/10.4324/9780203805534

Carmines, E.G., & Zeller, R.A. (1979). Reliability and validity assessment. Thousand Oaks, CA: Sage Publications. https://doi.org/10.4135/9781412985642

Cheung, G.W., & Rensvold, R.B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233-255. https://doi.org/10.1207/s15328007sem0902_5

Debois, N., Quillet, J., Sylvestre, A., & Calmels, C. (2004). Contribution à la validation en langue française d’un outil de mesure des stratégies mentales: Le «test des stratégies de performance» [Contribution to the validation of a measuring tool for mental strategies in the French language: the «test of performance strategies»]. France: Institut National du Sport, de l’Expertise et de la Performance (INSEP).

Durand-Bush, N., Salmela, J.H., & Green-Demers, I. (2001). The Ottawa Mental Skills Assessment Tool (OMSAT-3). The Sport Psychologist, 15, 1-19. https://doi.org/10.1123/tsp.15.1.1

Donti, O., & Katsikas, C. (2014). Factorial validity of the Test of Performance Strategies 2-competition scale (TOPS 2-cs) in greek athletic population. Biology of Exercise, 10(1), 13-22. https://doi.org/10.4127/jbe.2014.0070

Fletcher, D., & Hanton, S. (2001). The relationship between psychological skills usage and competitive anxiety responses. Psychologyof Sportand Exercise, 2, 89-101. https://doi.org/10.1016/s1469-0292(00)00014-5

Gimeno, F., Buceta, J.M., & Pérez-Llantada, M.C. (2001). El cuestionario «Características Psicológicas Relacionadas con el Rendimiento Deportivo» (CPRD): características psicométricas [The «Psychological Characteristics Related to Sport Performace questionnaire» (CPRD): psychometric properties]. Análise Psicológica, 19(1), 93-113. https://doi.org/10.14417/ap.346

Gimeno, F., Buceta, J.M., & Pérez-Llantada, M.C. (2007). Influencia de las variables psicológicas en el deporte de competición: evaluación mediante el cuestionario Características Psicológicas Relacionadas con el Rendimiento Deportivo [The influence of psychological variables on sports performance: Assessment with the Questionnaire of Sports Performance- related Psychological Characteristic]. Psicothema, 19(4), 667-672.

Goudas, M., Kontou, M., & Theodorakis, Y. (2006). Validity and reliability of the Greek version of the Test of Performance Strategies (TOPS) for athletes with disabilities. Japanese Journal of Adapted Sport Science, 4(1), 29-36.

Hardy, L., Jones, G., & Gould, D. (1996). Understanding Psychological Preparation for Sport: Theory and Practice of Elite Performers. Chichester: Wiley.

Hardy, L., Roberts, R., Thomas, P.R., & Murphy, S.M. (2010). Test of Performance Strategies (TOPS): Instrument refinement using confirmatory factor analysis. Psychology of Sport and Exercise, 11(1), 27-35. https://doi.org/10.1016/j.psychsport.2009.04.007

Hernández-Mendo, A., Morales-Sánchez, V., & Peñalver, I. (2014). Replicación de las propiedades psicométricas del Inventario Psicológico de Ejecución Deportiva [A replication of the psychometric properties of the sports performance psychological inventory]. Revista de Psicología del Deporte, 23(2), 311-324.

Jackson, S.A., Thomas, P.R., Marsh, H.W., & Smethurst, C.J. (2001). Relationships between flow, self-concept, psychological skills, and performance. Journal of Applied Sport Psychology, 13, 129-153. https://doi.org/10.1080/104132001753149865

Katsikas, C., Donti, O., & Psychountaki, M. (2011). Psychometric Properties of The Greek Version of the Test of Performance Strategies-Competition Scale (TOPS-CS). Journal of Human Kinetics, 29, 133- 139. https://doi.org/10.2478/v10078-011-0047-4

Kremer, J., & Moran, A.P. (2008). Pure sport: Practical Sport Psychology. London: Routledge.

Mahoney, M.J., Gabriel, T.J., & Perkins, T.S. (1987). Psychological skills and exceptional athletic performance. The Sport Psychologist, 1, 181- 199. https://doi.org/10.1123/tsp.1.3.181

Molinero, O., Salguero, A., & Márquez, S. (2010). Propiedades psicométricas y estructura dimensional de la adaptación española del Cuestionario de Estrategias de Afrontamiento en Competición Deportiva [Psychometric properties and dimensional structure of the Spanish adaptation of the Coping Inventory for Competitive Sport]. Psicothema, 22(4), 975-982.

Mora, J.A., García, J., Toro., S. & Zarco, J.A. (2001). Cuestionario de estrategias cognitivas en deportistas. Manual [Athlete cognitive strategies questionnaire. Manual]. Madrid: TEA.

Muñiz, J., Elosua, P., & Hambleton, R.K. (2013). Directrices para la traducción y adaptación de los tests: segunda edición [International Test Commission Guidelines for test translation and adaptation: Second edition]. Psicothema, 25(2), 151-157. https://doi.org/10.7334/psicothema2013.24

Nunnally, J.C. (1978). Psychometric Theory. New York: McGraw-Hill.

Saadatifard, E., Keshtidar, M., & Khoshbakhti, J. (2014). Is Test of Performance Strategies (TOPS) a Precise Tool for Iranian Adult Athletes? Middle-East Journal of Scientific Research, 22(8), 1219- 1227.

Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8, 23-74.

Smith, R.E., Schutz, R.W., Smoll, F.L., & Ptacek, J.T. (1995). Development and validation of a multidimensional measure of sport-specific psychological skills: The Athletic Coping Skills Inventory-28. Journal of Sport & Exercise Psychology, 17, 379-398. https://doi.org/10.1123/jsep.17.4.379

Solís-Salazar, M. (2015). The dilemma of combining positive and negative items in scales. Psicothema, 27(2), 192-199. https://doi.org/10.7334/psicothema2014.266

Taylor, M.K., Gould, D., & Rolo, C. (2008). Performance strategies of US Olympians in practice and competition. High Ability Studies, 19, 19- 36. https://doi.org/10.1080/13598130801980281

Thomas, P.R., Hardy, L., & Murphy, S.M. (2007). Test of Performance Strategies. Unpublished manuscript. Retrieved from http://www.topsfirst.com/

Thomas, P.R., Murphy, S.M., & Hardy, L. (1999). Test of Performance Strategies: Development and preliminary validation of a comprehensive measure of athletes’ psychological skills. Journal of Sports Sciences, 17(9), 697-711. https://doi.org/10.1080/026404199365560

Thomas, P.R., & Over, R. (1994). Psychological and psychomotor skills associated with performance in golf. The Sport Psychologist, 8, 73-86. https://doi.org/10.1123/tsp.8.1.73

Vealey, R.S. (1988). Future directions in psychological skills training. The Sport Psychologist, 2, 318-336. https://doi.org/10.1123/tsp.2.4.318

Weinberg, R., & Forlenza, S. (2012). Psychological skills. In G. Tenenbaum, R.C. Eklund & A. Kamata (Eds.), Measurement in sport and exercise psychology (pp. 381-392). Champaign, IL: Human Kinetics.

Weinberg, R., & Gould, D. (2010). Fundamentos de psicología del deporte y el ejercicio físico [Psychological foundations in sport and exercise]. Madrid: Médica Panamericana.

Woodcock, C., Duda, J.L., Cumming, J., Sharp, L-A., & Holland, M.J.G. (2012). Assessing mental skill and technique use in applied interventions: Recognizing and minimizing threats to the psychometric properties of the TOPS. The Sport Psychologist, 26, 1-15. https://doi.org/10.1123/tsp.26.1.1