Introduction

Field referees (FRs) in cooperation with assistant referees (ARs) have full authority to control and regulate the behaviour of players and coaches as well as to warranty the application of the seventeen rules of the game of the Federation International Football Associated (FIFA) (Castillo, Yanci, Casajús, & Cámara, 2016). Although soccer refereeing seems to be a simple task, there are many factors, such as the high number of decisions, the subjectivity for interpretation and the contextual variables (e.g. period of time, level of teams and referees, experience), which could have an influence on official matches (Canovas, Reynes, Ferrand, Pantaléon, & Long, 2008; Castillo, Castagna, Cámara, Iturricastillo, & Yanci, 2017; Pietraszewski et al., 2014). Given that incorrect decision-making could influence on the final score of the match and considering the influence of some contextual variables, to succeed on officiating official matches is the main task of match officials. Thus, with the aim of ensuring a proper course of the game, FRs and ARs have a specific role to make decisions (Mallo, Frutos, Juárez, & Navarro, 2012). While FRs use a wide diagonal system and are placed, whenever possible, to the left of the ball and close enough to the play to observe the game, but without interfering it, ARs are usually placed in line with the penultimate player (including the goalkeeper) of the team that defends the goal of his midfield (Coleclough, 2013). In this way, the percentage of error of match officials during soccer matches is minimized.

It has been observed that one hundred and thirty-seven decisions have been taken by officials during official matches (Helsen & Bultynck, 2004), where direct free kicks and throw-ins have been observed to be most repetitive ones. Some authors have observed soccer referees success in the 80-85% of the occasions (Gilis, Helsen, Catteeuw, Van Roie, & Wagemans, 2009; Van Meerbeek, Van Gool, & Bollens, 1987). Besides, the percentage of error during decision making in the off-sides and direct free kicks in high-level referees was about 6.0% and 7.6%, respectively (Catteeuw, Gilis, Wagemans, & Helsen, 2010; Oudejans et al., 2005). Due to the importance of the decisions taken by match officials’ during refereeing, it would be interesting to address an exhaustive analysis of the effectiveness of FRs and AR’s decisions taking. Moreover, the officials’ success could be influenced by both, contextual factors and the zone of the soccer field. Nevertheless, no previous study has analyzed whether these factors can affect to the success and error on decision-making during official matches.

Therefore, the aims of this study were, on the one hand, to determine the reliability of an observational analysis tool (ARBI.e) designed ad hoc to observe, code and analyse match officials decisions in soccer, and on the other hand, to analyse the degree of success in referee decisions during an international tournament according to contextual factors (e.g., period of the match, tournament ´s phase and score) and the zone of the soccer field.

Methods

Observational design

Within the observational methodology, the design in which this study was proposed was located in quadrant III, considering the basic features of nomothetic, punctual and multidimensional (Anguera, Blanco-Villaseñor, Hernández-Mendo, & López-Losada, 2011). This was due to the fact that several units of observation (e.g. several referees) were codified, in a specific time period (e.g., the international championship) and from an observation tool that agglutinated several criteria or macro categories (e.g., period of the match, field zones, referees’ decisions, etc.).

Participants

Participants were experienced FIFA top-class FR (n = 20, age = 40.4 ± 2.3 yr, height = 182.2 ± 0.1 cm, body mass = 76.3 ± 8.5 kg) and AR (n = 40, age = 40.6 ± 3.1 yr, height = 178.2 ± 0.1 cm, body mass = 73.9 ± 4.2 kg) selected worldwide to participate in the tournament. Thirty-two national teams participated in the 63 matches (preliminary phase = 48 matches and final trials = 15 matches) tournament. Fifteen matches of each phase (a total of 30 matches) were selected for analysis. The same number of matches in each tournament phase was analysed considering also the extra time in final trials. The first two matches of each group were chosen during the preliminary phase to analyse the referees’ decisions. A total of 3,605 official decisions (FRs = 917 observations, playing in advantage (PA) = 30 (0.89%), penalty (P) = 5 (0.13 %), simulations (SIM) = 28 (0.78%), direct free kick (DFK) = 843 (24.44%), indirect free kick (IFK) = 11 (0.30%); ARs = 2,589 observations, corner (C) = 326 (8.43 %), off side (OS) = 137 (3.72 %), goal (G) = 73 (1.91 %), throw in (TI) = 1439 (40.53 %), goal kick (GK) = 614 (16.03 %); Other = 99 observations, dropped ball (DB) = 15 (0.43 %), yellow card (YC) = 78 (2.23 %), red card (RC) = 6 (0.17 %)) were analysed. The study was conducted according to the Declaration of Helsinki and met the ethical standards in Sport and Exercise Science Research (Harriss & Atkinson, 2015).

Instruments

An observation tool (ARBI.e) was designed, composed of contextual variables (period of the match, tournament´s phase, score), field zones (Table 1) and two criteria (referees’ decisions and success). The categorical system developed for the matches observation is shown in Tables 1 and 2, as reported elsewhere (Helsen & Bultynck, 2004), which assigns to FRs, ARs and other referee’s decisions.

Contextual variables

The success percentage in referee decisions by the field referees (FR), assistant referees (AR) and other decisions according to the following contextual variables was analysed: period of the match (each 15 min period as 0 -15 min, 15 – 30, 30 – break time, 45 – 60, 60 – 75, 75 – full time), tournament phase (preliminary phase and final trials) and score (when the match is draw, or there is a difference of 1, 2, 3 and more than 3 goals).

Table 1

Codification of the zones of the field observation tool´s categorical system

Table 2

Codification of the match officials’ decision and success in an observation tool´s categorical system (ARBI.e).

Procedures

The matches were recorded during live broadcast in .avi format for registration through the Measuring and Observation Tool in Sports (MOTS) observation software (Castellano, Perea, Alday, & Hernández-Mendo, 2008). Prior to the analysis and the matches, two observers (the principal researcher and an elite referee officiating in the Spanish First Division) carried out the codification of a group match of this same tournament to assess the quality of the data. The two observers received a 10 h training course in order to learn the use of the MOTS and to familiarize with the observational tool. The principal researcher registered the game in two 10 days apart occasions with the MOTS. The expert referee registers also the analysis of the same match. Therefore, a total of three registers were carried out in order to calculate the intra-observer and inter-observer concordance index. Subsequently, only the principal researcher made the complete analysis of the 30 matches.

Data analysis

The results are presented in absolute values (frequencies) and percentages (%).The data recorded by the MOTS software were exported to an Excel 2011 Microsoft spread-sheet in .xlsformat. Then, they were exported and stored in the Statistical Package for Social Sciences package (SPSS Inc, Version 20.0 Chicago, IL, USA) in .sav format, for further analysis. To determine the intra-observer and inter-observer concordance index, Cohen´s Kappa index was calculated, through which it was expected to obtain coefficients higher than 0.70. According to Catteeuw et al. (2010), chi square goodness-of-fit tests was used to examine overall success rate in each field zone. The existence of significant differences between the successes/errors of the FRs and the ARs according to several contextual factors (e.g. period of the match, tournament phase) was also analysed. The level of significance was set at p < 0.05.

Results

The reliability of different observations were optimal for the criteria decisions assigned to the FRs, ARs, other decisions, success, field zone, tournament phase, match period and score. Cohen's Kappa values were from 0.914 to 1.00 and from 0.827 to 1.00 for intra-observer and inter-observer analysis, respectively, as shown in Table 3.

Table 3

Intra- and inter-observer reliability.

FR: field referee, AR: assistant referee, OBS1: first observation of the principal researcher, OBS2: second observation of the principal researcher, OBS3: observation of the elite referee

Figure 1 shows the percentage of success in the official decisions assigned to FRs (Figure 1A), to ARs (Figure 1B) and other decisions (Figure 1C). These results show that FRs succeed in their decisions in 100% of the cases of PA and P. They also have a percentage of success of 92% and 87% in the signalling of IFK and DFK, respectively. In contrast, they fail in 36% of decisions regarding SIM. On the other hand, the ARs success was over 98% in C, TI, GK and G, and 92% in OS. In addition, a percentage of accuracy of 100% has been observed when indicating DB and showing RC, and of 90% when showing YC. Significant differences (p < 0.05) were found in the decisions´ success in the FRs and the ARs. However, no significant differences were found (p > 0.05) in the success of other decisions.

Figure 1
Success percentage in referee decisions by field referees (1A), assistant referees (1B) and other decisions (1C) during the international tournament.

PA = playing an advantage, P = penalty, SIM = simulation, DFK = direct free kick, IFK = indirect free kick, C = corner, OS = off side, G = goal, TI = throw in, GK = goal kick, DB = dropped ball, YC = yellow card, RC = red card.

The results obtained in relation to the official decisions according to the zone of the field (zones) show that the percentage of success in any zone is above 95% (93.4% - 96.5%). No significant differences were found (p > 0.05) in the accuracy between each field zone.

With respect to the period of the match, the results obtained reveal a percentage of success over 95% (94.7% - 96.8%) in each 15 min period. Also, the percentage of success obtained by the match officials was 95% in both tournament phases (preliminary phase and final trials). No significant differences were found (p > 0.05) in success percentage in relation to the previous contextual factors.

Success percentage in official decision attending to the score in every moment of the match is shown in Figure 2. The results show a success rate of 96%, 93%, 96%, 100% and 98%, when the match is draw, or there is a difference of 1, 2, 3 and more than 3 goals, respectively. Significant differences (p < 0.05) were found in the official decisions success depending on the score in every moment of the match.

Figure 2
Success percentage in referee decisions by the field referees (FR), assistant referees (AR) and other decisions according to the score during the international tournament. Discussion

Discussion

The aims of this study were, on the one hand, to check the reliability of the ARBI.e tool as a notational system to observe, code and analyse match officials decisions in soccer and, on the other hand, to analyse the degree of success in official decisions according to contextual factors (e.g., period of the match, tournament phase and score) and the zone of the soccer field, during an international tournament. To the best knowledge of the authors, this was the first study to analyse the percentage of success in official decisions according to specific contextual factors of soccer in an international tournament. The main findings were: a) despite the high levels of success, the FRs showed a greater number of errors in relation to the ARs; higher percentage of error in the SIM category, b) the field zone and contextual factors do not influence the success of the decision making outcome made by international referees, with the exception of the score at the moment of decision-making.

The research in high-performance sports requires the development and use of reliable assessment tools to enable the development of an objective assessment of really takes place during the tournament, in order to optimize the training processes and to maximize the performance in competition (Martínez-Gallego et al., 2013; Vučković et al., 2013). The high inter and intra observer concordance rates obtained in the reliability tests applied to the ARBI.e tool (0.827 – 1.000), suggest that his tool is reliable for the observation of referee decisions in professional soccer. Based on these results, the tool can be used to improve strategies in decision-making by soccer referees.

The interpretation of the refereeing laws seems easy to apply, however, the high number of decisions that have to be made, along with the influence of contextual factors, make refereeing a difficult activity to carry out (Weston et al., 2012). The results obtained in the present study show an average error rate of 5% in the referees’ decisions, likely due to their high refereeing experience (Canovas et al., 2008) and level (Pietraszewski et al., 2014) of the participants. Nevertheless, previous studies in top-class referees have reported an error rate higher than ours (< 6% vs. 15-20%) in international tournaments (Van Meerbeek et al., 1987; Gilis, Helsen, Catteeuw, Van Roie, & Wagemans, 2009; Mallo, Frutos, Juárez, & Navarro, 2012). The lower error rate presented in our investigation in comparison with the other reported studies could be due to the optimal selection of match officials to officiate the international championship. During the last period of the competitive season, match officials receive a specific training focused on physical conditioning, positioning and decision-making strategies in order to improve their competence in refereeing activity. These differences show the importance to apply specific training programs to soccer referees with the aim of optimizing its performance in competition (Castagna, Abt, & D’Ottavio, 2007; Weston et al., 2012).

Even though the performance of match officials has improved in recent years, there are some categories in which the possibility of improvement is still high. In this line, in our study, an error of 36% was observed in the designation of the SIM by FRs, while the ARs presented an error of 8% in the signage of OS. These results coincide with previous studies, developed in both national (Oudejans et al., 2005) and international (Catteeuw, Gilis, Wagemans, et al., 2010) AR, where error rates of 6.0% and 7.5% in OS were observed, respectively. These results highlight the need to establish not only training strategies, but also to improve the decision-making skills (Catteeuw, Helsen, Gilis, & Wagemans, 2009). Finally, although the percentage of success was very high in FRs and ARs, significant differences (p < 0.05) in the official decisions´ success between FR and AR were found. These differences seem to be due to a better executive attention on the part of the ARs in relation to the FRs (Pietraszewski et al., 2014). In addition, some categories assigned to FRs in which they must take subjective decisions (e.g., PA or SIM), could facilitate the increase of errors in this group, as shown by the obtained results.

Attending to our knowledge, this is the first study that analyses the decisions of the referees based on the field zone where they occur. Possibly, because the present study was developed with top-class international referees, no significant differences were found in the degree of success/error between each zone. The high experience and performance level of the referees help them achieve better spatial positioning in the field, which is reflected in the acquisition of advantageous visual angles (Hüttermann, Noël, & Memmert, 2017) facilitating the success in different game decisions. Even though contextual factors (e.g., home team or crowd´s size) seem to influence the decision making outcomes in national tournaments (Buraimo, Forrest, & Simmons, 2010; Buraimo, Simmons, & Maciaszczyk, 2012), the results obtained show that international referees were not affected by external factors (e.g., period of the match or tournament´s phase) during an international tournament matches, with the exception of the score at the time of the decision. Although it has been shown that in matches where the score is adjust (draw, +1 or +2 difference) referees make different decisions (e.g., extra time or fouls number) than in matches where the score difference is broad (+3 or > +3) (Lago-Peñas & Gómez-López, 2016), the percentage of success in these matches is significantly greater due to the low sample analysed (only 2 matches) in our study.

This study is not exempt of limitations. The main one is not to be analysed whether the physical and physiological demands affect to the score and whether the match officials’ activity during previous minutes can affect to the success rate. For further research it would be advisable to associate the referees’ decisions and the physical and/or physiological responses during the minutes previous to the incorrect decisions across the official matches. This idea it would be interesting in order to assess whether the fatigue influence on the referees decision making.

The practical applications of the present study will be explained. At first, the ARBI.e tool can be a valid and reliable tool to improve strategies in decision-making by soccer referees, so it would be advisable to use it in order to analyse the decision-making of soccer referees during official matches. In addition, specific training strategies focused on ameliorating the subjective decisions (e.g., PA or SIM) assigned to FRs could be implement in referees’ training programs. Finally, given that these results have demonstrated that the score (i.e., draw, +1, +2, +3; >3 goals) at the time of the referees’ decision influence the success on decision-making, it would convenient to analyse whether an intervention program focused on decision-making and affected by this contextual variable (i.e., score) could improve the referees’ success in official matches.

Conclusion

Top-class international field and assistant referees showed a high success rate in decisions made during an international tournament. However, the FRs showed a greater number of errors in relation to the ARs, and both they must improve their performance in any of the categories analysed (SIM and OS respectively). On the other hand, the international referees´ decisions success is not influenced by the field zone or contextual factors. These findings suggest that, despite the high performance level of international referees, it would be advisable to apply specific training programs based on decision-making skills and mainly in simulations to optimize international match officials´ decision making.

Acknowledgments

The authors thank the professional referee Eduardo Prieto Iglesias for his involvement and collaboration in this study.

References

Anguera, M. T.; Blanco-Villaseñor, A.; Hernández-Mendo, A., & López-Losada, J. L. (2011). Diseños observacionales: ajuste y aplicación en psicología del deporte. Cuadernos de Psicología del Deporte, 11(2), 63–76.

Buraimo, B.; Forrest, D., & Simmons, R. (2010). The 12th man?: refereeing bias in English and German soccer. Journal of the Royal Statistical Society: Series A (Statistics in Society), 173(2), 431–449. https://doi.org/10.1111/j.1467-985X.2009.00604.x

Buraimo, B.; Simmons, R., & Maciaszczyk, M. (2012). Favoritism and referee bias in european soccer: evidence from the spanish league and the uefa champions league. Contemporary Economic Policy, 30(3), 329–343. https://doi.org/10.1111/j.1465-7287.2011.00295.x

Canovas, S.; Reynes, E.; Ferrand, C.; Pantaléon, N., & Long, T. (2008). Types of errors by referees and perception of injustice by soccer players: a preliminary study. Psychological Reports, 102(1), 99–110. https://doi.org/10.2466/pr0.102.1.99-110

Castagna, C.; Abt, G., & D’Ottavio, S. (2007). Physiological aspects of soccer refereeing performance and training. Sports Medicine, 37(7), 625–46.

Castellano, J.; Perea, A.; Alday, L., & Hernández-Mendo, A. (2008). The measuring and observation tool in sports. Behavior Research Methods, 40(3), 898–905.

Castillo, D.; Castagna, C.; Cámara, J.; Iturricastillo, A., & Yanci, J. (2018). Influence of team’s rank on soccer referees’ external and internal match loads during official matches. Journal of Strength and Conditioning Research, 32(6), 1715–1722. https://doi.org/10.1519/JSC.0000000000002040

Castillo, D.; Yanci, J.; Casajús, J. A., & Cámara, J. (2016). Physical fitness and physiological characteristics of soccer referees. Science & Sports, 31(1), 27–35. https://doi.org/10.1016/J.SCISPO.2015.11.003

Catteeuw, P.; Gilis, B.; García-Aranda, J.-M.; Tresaco, F.; Wagemans, J., & Helsen, W. (2010). Offside decision making in the 2002 and 2006 FIFA World Cups. Journal of Sports Sciences, 28(10), 1027–1032. https://doi.org/10.1080/02640414.2010.491084

Catteeuw, P.; Gilis, B.; Wagemans, J., & Helsen, W. (2010). Offside decision making of assistant referees in the English Premier League: impact of physical and perceptual- cognitive factors on match performance. Journal of Sports Sciences, 28(5), 471–481. https://doi.org/10.1080/02640410903518184

Catteeuw, P.; Helsen, W.; Gilis, B., & Wagemans, J. (2009). Decision-making skills, role specificity, and deliberate practice in association football refereeing. Journal of Sports Sciences, 27(11), 1125–1136. https://doi.org/10.1080/02640410903079179

Coleclough, J. (2013). Soccer coaches’ and referees’ perceptions of tackle incidents with respect to the laws of the game. International Journal of Performance Analysis in Sport, 13(2), 553–566. https://doi.org/10.1080/24748668.2013.11868669

FIFA. (2000). Regulations on the Organisations of Refereeing in FIFA Member Associations. Retrieved from: https://resources.fifa.com/mm/document/tournament/competition/01/28/10/42/defs_regulationsorganisationrefereeinginfifama_inhalt.pdf

Gilis, B.; Helsen, W.; Catteeuw, P.; Van Roie, E., & Wagemans, J. (2009). Interpretation and application of the offside law by expert assistant referees: perception of spatial positions in complex dynamic events on and off the field. Journal of Sports Sciences, 27(6), 551–63. https://doi.org/10.1080/02640410802702178

Harriss, D., & Atkinson, G. (2015). Ethical standards in sport and exercise science research: 2016 Update. International Journal of Sports Medicine, 36(14), 1121–1124. https://doi.org/10.1055/s-0035-1565186

Helsen, W., & Bultynck, J.-B. (2004). Physical and perceptual-cognitive demands of top- class refereeing in association football. Journal of Sports Sciences, 22(2), 179–89. https://doi.org/10.1080/02640410310001641502

Hüttermann, S.; Noël, B., & Memmert, D. (2017). Evaluating erroneous offside calls in soccer. PloS One, 12(3), e0174358. https://doi.org/10.1371/journal.pone.0174358

Lago-Peñas, C., & Gómez-López, M. (2016). The Influence of referee bias on extra time in elite soccer matches. Perceptual and Motor Skills, 122(2), 666–77. https://doi.org/10.1177/0031512516633342

Mallo, J.; Frutos, P. G.; Juárez, D., & Navarro, E. (2012). Effect of positioning on the accuracy of decision making of association football top-class referees and assistant referees during competitive matches. Journal of Sports Sciences, 30(13), 1437–45. https://doi.org/10.1080/02640414.2012.711485

Martínez-Gallego, R.; F Guzmán, J.; James, N.; Pers, J.; Ramón-Llin, J., & Vuckovic, G. (2013). Movement characteristics of elite tennis players on hard courts with respect to the direction of ground strokes. Journal of Sports Science & Medicine, 12(2), 275– 81.

Oudejans, R.; Bakker, F.; Verheijen, R.; Gerrits, J.; Steinbrückner, M., & Beek, P. (2005). How position and motion of expert assistant referees in soccer relate to the quality or their offside judgements during actual match play. International Journal of Sport Psychology, 36(1), 3–21.

Pietraszewski, P;, Roczniok, R.; Maszczyk, A.; Grycmann, P.; Roleder, T.; Stanula, A.; … Ponczek, M. (2014). The elements of executive attention in top soccer referees and assistant referees. Journal of Human Kinetics, 40(1), 235–43. https://doi.org/10.2478/hukin-2014-0025

Van Meerbeek, R.; Van Gool, D., & Bollens, J. (1987). Analysis of the refereeing decisions during the world soccer championship in 1986 in Mexico. In T. Reilly, A. Lees, K. Davids, & W. J. Murphy (Eds.), Science and Football(pp. 377-382). London: E & FN Spon.

Vučković, G.; James, N.; Hughes, M.; Murray, S.; Sporiš, G., & Perš, J. (2013). The effect of court location and available time on the tactical shot selection of elite squash players. Journal of Sports Science & Medicine, 12(1), 66–73.

Weston, M.; Castagna, C.; Impellizzeri, F. M.; Bizzini, M.; Williams, A. M., & Gregson, W. (2012). Science and medicine applied to soccer refereeing: an update. Sports Medicine, 42(7), 615–31. https://doi.org/10.2165/11632360-000000000-00000

Author notes