Citation counts as an indicator for research impact has its roots in the Anglo Saxon legal system. A court judging a case has to follow precedents laid down by higher courts; thus in citing authorities to back up new arguments, a lawyer has to check if they are still valid and have not been overturned by later sentences (DE BELLIS, 2009). Following this procedure, the Californian citation index of sentences was developed in 1860.

In 1955, a chemist Eugene Gardfield (GARFIELD, 1955) revolutionized research with his concept of citation index and citation searching, which led to the actual Science Citation Index in 1961. In the 1970s, citation counts articles received became an indicator of great relevance in the evaluation of research performance in the field of science. In the literature, there are several metrics to measure an individual’s research impact. Many of these indicators are ranking measures that provide quantitative estimates of the relative importance of a scientist within his/her field.

In the 1970s, the use of citation analysis to produce indicators of scientific performance based on citation counts began to attract exceptional attention from policy decision makers because these indicators produced numerical indexes that were easily applied, understood and calculated (DE BELLIS, 2009).

The introduction of citation-based measures to assess the performance of researchers, groups or even countries, fostered a race for the development of unbiased indicators. This long and winding road has witnessed the development and growth of many indexes. The most cited indexes are the H-index, (HIRSCH, 2005) with 5,832 citations in Google Scholar at the moment this paper was being written, and G index (EGGHE, 2013) with 1,332 citations in Google Scholar.

According to De Bellis (2009) the H-index is a better predictor of future individual achievements than traditional indicators that compute/calculate total citation counts, mean citations per paper, and number of papers. The H-index combines both quantity and impact in one measure (MINGERS, 2008). Hirsch (2005) defined the index as: “a scientist has index h if h of his/her N papers have at least h citations each and the other (N − h) papers have no more than h citations each”. Thus, a researcher with an H-index of 10 means that he/she has 10 publications with at least 10 citations from each paper. The H-index was designed to be applied at a micro level (GLÄNZEL, 2006).

The H-index was welcomed by the scientific community as a corner stone in the advancement of measuring the impact of a given author, subject matter, a journal, institution or even a country. Mingers (2008) makes an interesting review of the advantages of this index. Among many others, this author highlights the following: (a) it measures both productivity and influence or impact of papers in a single measure (VÍLCHEZ-ROMÁN, 2014), (b) it is simple to calculate and easily understood (VÍLCHEZ-ROMÁN, 2014), (c) it can be applied at several levels of aggregation: individual, research group, journal or department; (d) any type of research output can be included and it is not affected by outputs with zero citations and it correlates with other standard bibliometric measures.

Although the H-index has been widely accepted it has also been criticized because of its drawbacks. For example, Adams (2014) states that the H-index is innately flawed as it is not responsive to field or career stage, it does not control for the age of documents or citations, and the same group of articles will yield a different H-Index in different databases. About the limitations of H-index Gajendra and Singh (2009) stated that “though H-index is a powerful technique for measuring performance, it has a number of limitations: (a) it depends on number of publications; (b) it is affected by the length of the scientist’s career. As Vílchez-Román (2014) pointed out ‘the H-index penalize young researchers or those who are at the beginning of their scientific careers’; (c) ‘all highly cited papers get equal weight’. Finally, Waltman and Van Eck (2012) concluded the H-index cannot be considered an appropriate indicator of a scientist’s overall scientific impact. Another drawback of H-index is that all authors receive full credit each for a multi-authored paper.

To overcome the abovementioned limitations, different authors have suggested a series of variations based on the procedure of the H-index. For example, the H2 index was developed by Kosmulski (2013), the Hw-index by Egghe and Rousseau (2013), the tapered H-index by Anderson et al. (2008) the Hi-index by Batista et al. (2006), the Hm by Schreiber (2008), the Hp by Wan et al. (2007), the generalized H-index by Radicchi et al. (2008), Burrell (2007), the rational H-

index by Ruane and Tol (2008), the dynamic H-index by Egghe (2007) and the correction factor for H-index by Iglesias and Pecharromán (2007). More recently, Ferrara and Romero (2013) developed the Discounted H-index.

The G-index (1,332 citations in Google Scholar) has been the most widely accepted and developed of the H-index variations. It was proposed by Egghe (2013). The G-index was introduced based on the average of highly cited papers (G-index is more or equal to H-index). Two scientists having equal H-index may have a different G-index. The main advantage of this index is that it overcomes the limitation of the H-index of calculating highly cited papers the same as lower-cited ones (GAJENDRA; SINGH, 2009). The G-index favors the highly cited articles.

The research group of the Centre for Science and Technology Studies. based at Leiden University Moed et al. (1995) developed the Crown indicator (458 citations in Google Scholar). Leydesdorff and Opthof (2011) criticized the remaining problems in the Crown indicator because of its drawbacks based on the statistical procedure used, mainly the use of the mean as measure of central tendency when the median was the appropriate one. Later, the Crown indicator was modified and further developed by (LEYDESDORFF; OPTHOF, 2010; MOED, 2010; WALTMAN, et al., 2011a; WALTMAN, et al., 2011b). Van Raan (2006) presented evidence to back the Crown indicator as an appropriate indicator to measure research performance. The main limitation of this measure is the complexity and the number of operations required to calculate the indicator.

Other authors such as Järvelin and Persson (2008) developed the “Discounted Cumulated Impact for enhancing the sensitivity” measure to the age of the publication and quality of the cited articles. Antonakis and Lalive (2008) created the Index of Quality and Productivity to correct the straight citation counts for scholarly productivity, the age of academic career of authors and the specific citation trends of a field in relation to an expected citation rate. Lundberg (2007) developed the citation-z score to attain the normalization of citation impact at the level of individual publication.

When analyzing all these indexes taking into consideration the four criteria proposed by Finch (2010) for the selection of a single good indicator to measure an author’s performance, they fail in one, two or all of these criteria. Furthermore, these indexes are more effective when measuring the citation-based impact of high impact authors or authors that mainly publish their papers in journals with high impact factors.

Pudovkin and Garfield (2009) suggested the Percentile Rank Index and Author Superiority Index. The main idea of Pudovkin and Garfield (2009) is that in the fields of science with a low citation intensity (e.g. mathematics, taxonomy, or national publications) some other measures might be more effective. This indicator shows the performance of a targeted author on the background of his/her peers by comparing papers of the target author with the papers of other authors within the same publication in a journal or book. When analyzing the Author Superiority Index (ASI) measure to assess a researcher’s performance, Finch (2010) posed that the main limitation of ASI is its dependence on the number of papers. As a solution for this limitation, the Relative ASI (RASI) was introduced.

To measure the citation-based performance of authors from a country in a specific research area we used the Relative Author Superiority Index. The reasons for using this index were as follows: (1) index overcomes the self-citation effect; (2) Relative ASI is an effective measure when authors have a low scientific production and low number of citations.

At present, the most widely used databases for citation analyses are WoS, Scopus and, more recently, Google Scholar. For the present study, the data was retrieved from WoS database. The decision to use the WoS database as the source for the study was based on the reasons posed by Ronda-Pupo et al. (2014). It is the world’s leading database in publications and reports of citations Adams and King (2009). Over 20 million researchers from over 3,800 institutions in 98 countries use WOS. This database provides access to information from approximately 8,500 of the most prestigious, influential research journals in the world. WoS produces annual indicators that allow measuring the performance of the countries analyzed that are generally accepted by the scientific community. Finally, WoS includes the necessary fields for obtaining information and creating matrixes of data for quantitative analyses.

The data retrieval process included four steps. First, a general search using the ‘advanced search’ feature, as follows: CU (Country) = Cuba and WC (WoS Category) = Biotechnology; Applied Microbiology, time spam = 1988 to 2013, inclusive. We set the upper limit of the time frame threshold into 2013 to avoid including recent articles with few citations in the study.

The document type was ‘articles’ and the citation indexes used was Science Citation Index Expanded, retrieving 685 documents. The records were ranked by the field ‘authors’ within the citation report window and we selected the top 25 authors with the highest scientific production. The results evidenced the presence of Lotka’s law (LOTKA, 1926) because 25 researchers account for 72% of the scientific production. Thus, a sample of these 25 authors was selected to measure their citation-based impact factor. In this selection, the author selected was not necessarily from a Cuban research institution, but plays an important role in the scientific output of Cuban Biotechnology research.

Third, a citation report for each of the 25 authors was created ranking the records from the highest to lowest according to their number of citations. Finally, we calculated the values for the Relative ASI. In the following section, the procedure to calculate the Relative ASI is explained.

The Relative Author Superiority Index is based on the methodology developed by Pudovkin and Garfield (2009). For the calculation of this index, a procedure involving a three-step process is required. First, a Percentile Rank Index (PRI) is obtained for each of the individual papers an author has published then, the ASI, which is based on PRI values, is assessed for all the papers published by an author. Finally, the Relative Author Superiority Index that is based on the values of ASI was calculated.

The PRI for each paper is based on the citation rank of the paper being evaluated among all the papers published in the same journal in the year the paper appeared. This way the drawback of age in previous indexes is solved because the papers are of the same age as the other papers under comparison. This suggestion is also in line with the approach of Pudovkin and Garfield (2004) for characterizing journal impact factors and overcoming problems with differences in citation frequency in different fields of science.

To calculate the PRI, all the papers published by the targeted author were downloaded from the selected database. The average citation rate was then calculated for each author’s paper in the journal it had been published by retrieving all the papers published by that journal in the same year. The Percentile Rank Index was calculated using the formula:

PRI=(NP-R+1)/N*100

Where NP stands for the number of papers in the year set of the journal; R for the descending citation rank of the paper (among all the papers published in the journal the year the target paper appeared). In case of ties (several papers having the same citation frequency), each of the tied values is assigned the average of the ranks for the tied set. For example, if a target paper is the most cited paper in a journal in a year, it is PRI = 100.

The expected PRI value draws the line to establish if the article is above or under the expected value for the PRI. The value of the expected PRI was calculated using the formula suggested by PUDOVKIN, et al. (2012).

PRIExp=(50+1/NP*100/2)

Where NP stands for the number of papers in the year set of the journal.

Once the PRI is calculated, the authors’ overall citation performance is obtained through the ASI index. Pudovkin and Garfield (2009) propose three thresholds for ASI; the first threshold is the number of articles with a PRI ≥99, the second for the articles with a PRI ≥95, and the final threshold is the articles with a PRI ≥75. For the present study we added a PRI ≥50 percentile.

When analyzing the ASI measure to assess a researcher’s performance, Finch (2010) posed that the main limitation of ASI is that it depends on the number of papers. To overcome this drawback, we created the RASI, which ensures the establishment of the overall citation-based performance of each individual.

RASI≥50=(ASI≥50/NP)

Where ASI ≥50 is the number of papers of the author under evaluation above ASI 50 and NP stands for the number of papers.

Table 2 shows the values of the indicators of research performance of scholars from the Cuban Biotechnology research group. The expected median value of PRI for the 496 papers is 50. Three hundred and fourteen papers (63%) out of 496 were above the expected median PRI. This shows that these papers are cited above the median of the papers published in the journals they appeared. Of the evaluated authors (23), 92% have an average PRI above the median of expected PRI for their papers.

Table 2.
Summary of Scientific Production and citation-based performance indexes for the 25 top Cuban researchers in Biotechnology 1988-2013.

The median PRI value for the 496 papers is 66, which is significantly higher than the expected PRI global median value of 50. Because the variable is not normally distributed, we ran a Mann-Whitney U-Statistic=90921,500, T=278350.500 n(small)=496 n(big)=496, p<0.001. The difference in the median values between the two groups is greater than expected and a statistically significant difference was found (p<0.001).

The use of the average PRI adds insight to the analysis. For example, authors with few published papers, such as Martin C., show a higher performance than authors that double his scientific production. That is because 100% of his articles are above ASI≥50 reflecting high quality papers. Perez R., has both high scientific oeuvre and at the same time high quality papers. Seventy seven percentile of his papers are in ASI≥50. In addition, author Delafuente J., who has an average scientific production, has high quality papers with 83% of his articles above ASI≥50. This result suggests that PRI is not as sensitive to the number of articles of an author as the H or G indexes are. In addition, no correlation between the number of papers and PRI exists r_s=0.071, p=0.734. This suggests that the PRI focuses on the quality of the papers as defined by the citations they received compared to the rest of the articles published in the same year in the journal in which they appeared. This measure is not sensitive to the age of the publication or the length of the author’s career.

When the ASI index is used to assess the author’s citation-based performance, we found many insightful results. For example, the contrast between two authors who have high scientific production: Villalonga R. and Perez R. The first produced 36 papers, but none of them has PRI at the 99th percentile, while for Perez R., 5 of his 31 papers are at the 99^th percentile. Thus, 5 of his papers were top cited within the journals the year they were published.

Even more interesting is the performance of Vazques A.M. This author published only 15 papers, but three of them are above the 99 percentile, that is the top cited within the journal the year they were published. However, his overall citation score is almost 3 times lower than the top author Villalonga R, while his H-index is only nine which is much lower than Villalonga’s H-index 16. This researcher is certainly an author of high standing as 20% of his papers are among the most cited ones of the journal the year they were published.

The Relative ASI also adds interesting insights. For example, the author Martin C. has a low number of papers, H-index below the median, and a relatively short career (seven years below the median) but his RASI is 100. This suggests that all his papers are above the 50 percentile, or he has a high citation based-performance. In addition, Fragoso A. and Silva R. are not highly productive authors and their citation rate is below the median but their papers are above the median of the Percentile Rank Indicator. The results corroborate that the relative ASI does not correlate to the number of papers as described by Finch (2010) for ASI. This result suggests that the Relative ASI overcomes this drawback.