Measuring, Rating, Supporting, and Strengthening Open Access Scholarly Publishing in Brazil

Latin America was an early leader in the movement to provide online open access to the research and scholarship published in peer-reviewed journals. In their analysis of journals indexed in Scopus, Miguel et al. found that 74% of Latin American journals were OA (compared to a global proportion of 9%) (Miguel, Chinchilla-Rodriguez & Moya-Anegón, 2011). Abadal (2012) found that, for a subset of Ulrich’s directory, Brazil has the highest proportion of OA of peer-reviewed active journals (67%) of any country; Haider (2005) ranks Brazil third among OA publications in the world, and in 2013, only the US had more OA titles than Brazil in the Directory of Open Access Journals (DOAJ). Latin American editors and researchers have found over time that open access offers a means of reaching the global research community with their research and scholarship (Alperin, Fischman & Willinsky, 2011; Fischman, Alperin, & Willinsky, 2010). At the same time, the growth and visibility of open access raised concerns over the quality open access journals across the region (Mann et al, 2009; Sandes-Guimarães & Costa, 2012).

As leader in scholarly publishing and open access in Latin America, Brazil offers an excellent case study on the extent, systems, and quality of open access publishing in a region in which the open access model dominates. This paper synthesizes and analyses the available data on Brazilian scholarly journals as a way of reviewing the state of knowledge on each of these issues, including the systems behind the data. Given the degree to which the open access publishing model is growing in prominence in other parts of the world, Brazil offers lessons about the role of federal agencies in using open access to strengthen the national research culture, which should be of widespread interest for those concerned with the improved access and quality of scholarly publishing (Jamali & Nabavi, 2015; Laakso et al., 2011).

Brazil’s Journal Rating System

Brazil has a research university system composed by a minority of public institutions accounted for the majority of the scientific research produced in the country. This Brazilian federal system and the national higher education strategy have been focused on improving scientific research quality by investing in public institutions programs, while private institutions are responsible for absorbing the expanding enrollment in undergraduate courses.

As observed by Alperin (2013), the Brazilian public higher education system has solid graduate programmes with qualified professors and produce a significant amount of research in a sufficient internal market to exchange ideas in locally academic journals. National policies have incentivised research and scholarship at these public universities, increasing the number of doctorate degrees, students and professors collaborating with international universities. The country consequently has shown considerable growth in the number of scientific papers published, having a large body of journals, therefore can be considered as a regional leader in terms of scientific publication in Latin America.

The Brazilian government funds the production of academic journals and researchers, increasing the number of published journals and attracting visibility to the Brazilian science. The two agencies in charge of allocating the majority of research funding, CAPES (from the Portuguese acronym for the Federal Agency for Support and Evaluation of Graduate Education) and CNPq (the National Council for Scientific and Technological Development) have focus on strengthening graduate education and national research culture. The national agencies have also implemented a comprehensive graduate programme valuation scheme. CAPES is also the agency responsible for the sistematic evaluations of the Brazilian graduate programmes, assigning a 7-point scale score based on criteria such as publishing in high-prestige journals, receiving international scholars, producing patents and organising international conferences. The Serials Qualis rating system established by CAPES is a crucial part of this evaluation system.

The Serials Qualis (also known as just Qualis) is the name adopted for the set of procedures used by CAPES for stratification the quality intellectual output of graduate programs. It evaluates and rates global and national journals with the purpose of assessing the Brazilian graduate education publication system. It provides a rating list of scientific journals that can be used by graduate programs for evaluating dissemination possibilities of their scientific production. The premise of Qualis is to estimate the quality of the articles based on the analysis of the quality of the scientific journals where they have been published. Qualis employs multiple committees of Brazilian researchers to rate periodicals from both Brazilian and foreign publishers, grouped into nine disciplinary fields (agricultural sciences, bioscience, health sciences, engineering, exact sciences, arts and linguistics, human sciences, social sciences and multidisciplinary disciplines). The function of Qualis is to be an evaluation tool for institutional scientific production of authors linked to the formal system of Brazilian graduate programs. The scientific production is assessed indirectly by ratings in a scientific journals list, categorized and prioritized by the ad-hoc committees, therefore we have to consider contextual variables of theses evaluation procedures. Qualis represents a complex process evaluation and it is not intended to be a generalized expression of the journals quality. This is not Qualis purpose, although it may be a significant and representative sample of the Brazilian publishing potential. We must then consider the scope and meaning of the hierarchies established in the classification of journals in the Qualis list, thus the same journal published within an area of knowledge can generate a relevant surplus, receiving and publishing articles from authors from other areas of knowledge without incurring inconsistency.

Our analysis of open access journal quality both reviews and relies on aspects of the CAPES Serials Qualis rating program. While not without its critics (Rocha-e-Silva, 2009), Qualis ratings make it possible to see where open access serials fit into the Brazilian scholarly publishing landscape.

One serial can be rated in several areas, with different ratings in each, expressing the serial’s relative value in each area of evaluation. Qualis classification criteria vary somewhat according to field, with some common to all fields, mainly those based on citation indicators, such as the Impact Factor and the Scopus h-index (Figure 1). Rating criteria also include the number of indexing and abstracted services in which the journal is included, number of different author affiliations, presence of international co-authors, the use of blind peer-review system, and wide serial dissemination (Capes, 2013). This is consistent with other Latin American Science Councils, which are increasingly relying on bibliometrics in assessing the research quality of journals for their respective nations (Alperin, Fischman, & Willinsky, 2011).

Despite the fact that area coordinators have autonomy to establish their own criteria, all Qualis committees have to observe three generic proportion rules when rating serials, defined by the CAPES Technical-Scientific Council. The main rule is that the percentage of A ratings (A1 and A2 combined) in a given area should be no more than 25% of the total serials in that area, and the percentage of ratings A1 should be less than the percentage of A2. The total percentage of A1, A2 and B1 ratings in a given area should be no more than 50% of total serials in the area (Andrade & Galembeck, 2009; Mafalda et al., 2015). Combined, these three rules mean than no more than half the journals can be rated in the top three categories, and the higher categories have to be more selective than those below it.

Figure 1
Formally stated Qualis criteria for rating journals by nine disciplinary areas Source: Capes. Evaluation Board: Areas Documents and Committees. 2013. Translated from Portuguese by authors.

Research Methods

To establish the number, proportion, and quality of open access Brazilian journals, we first sought to assess the comprehensiveness of the Qualis list by comparing it with other serials directories and bibliographical database indexes (Table 1). We collected data from the institutions websites and created a dataset containing several journals characteristics. The main source of data was WebQualis, the online web-based system from CAPES, used to access data from Serials Qualis. The CAPES evaluation system produces a list of periodicals from Brazilian and foreign publishers with ratings attributed by committees composed of academics from all fields of science. The Qualis rating classes are in an 8-point grading scale, with letters and numbers, A1 being the highest grade, attributed to top journals in each evaluation area, followed by A2, B1, B2, B3, B4, B5 and C being the lowest level with zero weight, attributed to serials considered non scientific journals. For our analysis purpose, we coded the ratings classes with numbers from 8 to 1, 8 corresponding to the A1 level and 1 to the C level. If the serial was not in Qualis the serial code rate was defined as 0 (zero).

This analysis was conducted from February 2013 to October 2013, and as such it captures a moment in time. In 2013, Qualis rated 24,406 serials, including both active and non-active titles. Of these, 32% were Brazilian and 68% non-Brazilian. Since a serial can be rated in multiple disciplinary areas (once by each area committee), the number of Qualis ratings was 87,178 (of which we used the highest rating in our analysis).

The objective of crossing data from several datasets was to establish a reasonable estimated number of OA journals in Brazil. First we face the question of how many serials are published in Brazil and we have to deal with the various challenges at arriving at the total number of periodicals published in the country. In some directories and indexers websites data were unavailable for download, so we contacted these institutions directly to obtain the data. After collecting data from the institutions websites, we combined all datasets to the list of journals extracted from Webqualis. As data extraction criteria for world serials and union of databases, we used variables such as country, institution, name, language, ISSN and e-ISSN. Three datasets were created, one with Qualis ratings, one with all Qualis serials (Brazilian and international serials rated in Qualis) and the third one with Brazilian serials only. This last dataset contained periodicals from all datasets joined by ISSN and eISSN (from Qualis plus other directories and indexers).

Next steps were data cleaning and browsing websites to confirm journals online. Data cleaning was done with the junction of print and online versions of the same serial (by ISSN, e- ISSN, Name and Publisher). The indiscriminate use of ISSN and e-ISSN numbers for the same journal in different datasets duplicated the journal on the final database. Extensive checking and data cleaning was performed, with the exclusion of duplicated journals and the reunion of the paper and online version of the same version in single cases. Data from Serials Qualis had several cases with two different ratings for printed and online versions of the same journal, and in this case, the higher value attributed was selected.

There were several ceased or merged periodicals on datasets. We excluded ceased, merged, suspended and non-Brazilian serials. Additionally, we excluded publications considered as nonperiodicals (proceedings, magazines, directories, newspapers and any other type of non-periodical). Several serials did not have ISSN or e-ISSN numbers. ISSN numbers and serials without ISSN were identified through a web-search and those not considered journals were deleted from the database. After all the process of cleaning and checking data, we created variables from others, to characterize each serial (highest rating, sum of ratings, maximum ratings, presence in indexers, etc).

Table 1

Data Sources Used to Assess Comprehensiveness of Qualis Journal List

It has been observed that Qualis database contains a number of inconsistencies (Lopez, 2001). We found, for example, duplicate ISSNs, in which two different journals were registered with the same ISSN in Qualis (for examples, see Appendix A, Figure A1). The Qualis database also contained several cases in which the print and online versions of the same journal received different ratings, and in such cases (as with a title receiving ratings in multiple areas), the higher value was used. For example, the print edition of Cadernos de História da Educação, published by Universidade Federal de Uberlândia, received an A2 rating, while the online version was ranked B3. In 2013, the ratings were both changed to A2 in education, but a discrepancy remains in the journal’s rating in the interdisciplinary area (where the print version is rated B1 and the online version B2) (Appendix A, Figure A2).

We sorted journals based on Serials Qualis ratings. Qualis periodicals list reflects a number close to the real number of periodicals currently existing in Brazil. However, Qualis journal list is incomplete, since it is only updated annually and may fail to include a title for a number of reasons. A web-based research performed in other databases completes the list of journals obtained from Qualis.

To study the Qualis rating of open access journals, a random sample of 450 Brazilian journals was created from the Qualis list. The sample was stratified, with 50 randomly selected journals from each of the eight Qualis rating classes (A1 to C), plus 50 journals that were not found in Qualis. These 50 journals not listed in Qualis were also randomly selected from a list of 1,143 serials obtained from other serials directories and bibliographical database indexes, presented on Table 1.

Whether a journal was OA, had an embargo, charged submission or processing fees was checked manually by the researchers. (While DOAJ and Ulrich identify open access titles, their coverage of Brazilian journals was inadequate). For each, we checked the first four open access features described in the Open Access Spectrum (Table 2), readers’ rights, reuse rights, copyrights and author posting rights.

To check the Platform Systems most used by Brazilian Journals and verify the proportion of open access titles supported by the journals use of grant-supported software systems, we crosschecked information from original databases. We made a Venn diagram to compare serials in SciELO a successful 15-year Brazilian project of OA initiative engaged with the international movement of OA to scientific information (Packer & Meneghini, 2007), with serials from RedALyC, another OA journal publishing platform based in Mexico. And serials using the most used open source software Open Journal Systems (OJS), developed by the Public Knowledge Project, with support from the Brazilian Science and Technology Institute (IBICT).

Table 2

The Open Access Spectrum Used to Assess the Journal’s Open Access Status

Source Adapted from PLOS, 2013.

Results

A. The Database Count of Brazilian Serials

While the Qualis dataset lists 7,825 Brazilian serials (32% of all the serials in Qualis), only 6,634 titles proved to be valid journals, after removing duplicate, ceased, merged, suspended serials and non-periodicals publications (Table 3).

This is twice the number of journals listed for Brazil in the Latindex Directory, the region’s most comprehensive index service, and one in which Brazil accounts for 16%. Only 269 Brazilian journals were found in Scopus and 118 in the Web of Science (both around 1% of their total journal count), reflecting the much-noted under-representation of Latin American journals in external databases (Alperin, 2014; Cetto & Alonso-Gamboa, 2008).

Table 3

Number of and Proportion of Brazilian Serials by Major Journal Datasets

Notes a Filtered excluding non-periodicals.

b Approximate.

c Excluding duplicate, ceased, merged, suspended, and non-periodicals publications.

By comparing the Brazilian journals listed in Qualis and in the other indexes and services, an additional 1,143 journals were identified that were not in Qualis (Table 4). Of these, only 405 (35%) are available online, with the majority of them new journals, while the remaining 738 (65%) are only to be found in print or CD-ROM formats. This meant that according to relevant database sources, Qualis was perhaps missing 57 active and current journals, which is less than 1% of its filtered collection of 6,634 titles.

Qualis is clearly the most comprehensive, if not entirely complete, source for judging the state of Brazilian scholarly publishing. By the same token, it is fair to say that CAPES is doing a good job of reviewing the nation’s scholarly journal literature.

Table 4

Status of Brazilian Journals Not Found in CAPES Serials Qualis

B. Brazilian Serials Ratings

The Qualis ratings of Brazilian journals places 75.8% of the titles in the A and B categories, with less than 2% of the journals judged to be in the highest category of A1 (Table 5).

Table 5

Number and Proportion of Ratings of Brazilian Journals in CAPES Serials Qualis

The Qualis ratings fall well short of the CAPES guidelines (discussed above), which permit 25% to be awarded A’s – with the 2013 ratings at 5.0% – while allowing 50% to be awarded to A1- B1, compared to the 11.5% awarded. The message to the journal community is that there is plenty of room to grow in terms of improved quality. At the same time, the Qualis committee found that less than a quarter of the Brazilian titles fail to meet the basic requirements of offering qualified editorial oversight, submission rules, authors affiliation, institutional affiliation, editorial council, bilingual description, ISSN number, periodical publishing, peer-review and publication by an educational institution or scientific society.

By assigning an 8-point scale to the Qualis A-C ratings (with A = 8 and C = 1), we were able to average ratings and compare Brazilian journals, which had an average rating of 2.8 (SD = 1.7), and international journals in Qualis which averaged 5.1 (SD = 2.1). The international journals in Qualis are those in which Brazilian scientists publish their research papers and, as such, are more likely to be higher quality journals in the Global North, with better research in Brazil being sent abroad to be published. We were also able to compare fields (Figure 2). Among Brazilian journals, mutltidisciplinary journals received higher ratings on average than other Brazilian journals. In the international comparison, the biggest gaps exists in the biological sciences, with the average for Brazilian journal dropping below 2.0 in their rating, compared to a 4.8 average rating for the international journals in the Qualis set. This suggest that the best work in biology goes out of country to be published, with room for a concerted effort on the part of the journals and federal agencies to attract, promote and serve well high quality submissions.

Figure 2
Distribution of Brazilian and Non-Brazilian serials Qualis ratings by área

C. The Proportion and Quality of Open Access Journals in Brazil

To establish the proportion of open access journals published in Brazil, we used a stratified sample (described above) and manually verified what access model the journal was employing. In the resulting sample of 450 Brazilian journals, 68% proved to OA, a proportion similar to that found by Abadal in his analysis of Ulrich’s database of journals (2012). The proportion deemed “open access” includes serials that were free to read online, whether with an embargo period for new issues (10%) or not (58%). The remaining 32% had some form of fee-based access.

However, if one considers the top two classes of journals (A1-2), the proportion that are OA (immediate and embargoed) jumps to 95%. In fact, we found that in all cases, the higher the rating class, the higher the proportion of serials providing free access to articles (Figure 3). The best journals in Brazil are far more likely to be open access. Without being able to determine cause and effect in this association, we can least confirm that open access is not a journal quality issue in Brazil, confirming what has been found for Ibero-American countries more generally, and without this level of analysis [18]. What is also encouraging about the quality of open access journals is that this researh is being consulted by the public, with 20-25% of all article views coming from nonuniversity affiliated individuals reading for personal or professional practice reasons (Alperin, 2015).

Figure 3
Proportion of Open Access journals published in Brazil by Qualis Rating NQ = Not in Qualis.

It should be noted that the open access journal copyright and self-archiving policies in the sample differed on PLOS’s Open Access Spectrum (2013). Of the A1-B2 journal subsample, 85% (169/200) of the journals lack explicit self-archiving policies either on their website or in SHERPA/RoMEO (a database of such policies). Of the 31 with explicit archiving policies, 19 adopt the blue policy (authors can archive only the post-print version), ten the green policy (archive preand post-version of the article) and only two had a restrictive policies (yellow or white).

In addition, 30% of the journals rated A1-B2 do not appear to include information on copyright ownership in their website. The lack of information on copyright policies in journals appears to be widespread in Latin America, where a study of 292 journals from across the region found that a third of journals lacked any such information, although in some Latin American jurisdictions authors retain copyright by default [20]. Only 20% of the journals in this sample made explicit mention of author’s retaining copyright, while the remaining journals assert copyright of the published works. These numbers are again similar to the Cerda Silva and Lara study that found that 44% of journals assert copyright of the works they publish (Cerda Silva & Lara, 2011).

In relation to licenses, 112 (56%) of journals in the sub-sample of 200 do not specify a Creative Commons (CC) (or similar) license (including those for which there is no license information at all). These journals do not technically meet the full definition of Open Access, and may be said to be public access journals. The remaining 88 (44%) do specify a CC license, where CC-BY-NC is the most commonly found (22% of the journals in the subsample), followed by CCBY (16%) and CC-BY-NC-ND (6.5%). Similarly, not all the journals that we coded as OA follow the same business or even access models. The Arts, for example, were more likely to use embargo periods for open access content (Table 6).

Table 6

Proportion of Open Access Journals by Full Sample and by A1-B2 Journals

Note Includes sample from all Qualis levels and from those journals not in Qualis. Fields are not mutually exclusive and one serial can be rated in more than one field and was considered in the respective field.

As well, variations exist in the proportion of OA among the disciplines, with Agricultural Sciences, Engineering, Exact and Health Sciences exhibiting the higher proportion of OA journals in the sample, and for the sub-sample of top-tier journals.

The business models are also varied among the Brazilian open access journals. Only 8 (4%) of the highest rated journals (based on the sample of 200 A1-B2 journals) charged submission fees prior to the acceptance of a paper, while 16 (8%) levied “article processing charges” (APC), which is lower than the proportion of journals (26%) that charge in the international list of the Directory of Open Access Journals (Morrison et al., 2015). The business model of the Brazilian journals appears to be based on subsidies from partner institutions, grants from financing agencies, and subscriptions for the print version of the journals (Sandes-Guimarães & Diniz, 2014).

D. Platform Systems used by Brazilian Journals

Brazil’s high proportion of open access titles is supported by the journals use of grantsupported software systems. Foremost among them is SciELO, which began in Brazil to support OA in 1998 (Packer & Meneghini, 2007; Packer et al., 2014). SciELO provides full open access to 1,030 journals from across Latin America, and more recently, South Africa (Packer, 2009). A second regional OA journal publishing platform is RedALyC, based in Mexico, having started a few years after SciELO with a focus on the Social Sciences.

RedALyC currently indexes close to 1,000 journals from across the Latin America. Both systems have served to professionalize journals from the region, and to strengthen the culture of openness in the region (Alperin, Fischman & Willinsky, 2011; Guedon, 2008). In a sense, SciELO and RedALyC have been among the most visible Latin American contributions to a globalized science composed mainly of international commercial publishers (Vessuri, Guedón & Cetto, 2013).

In addition, Márdero Arellano (Márdero Arellano, 2008) and Cyzyk and Choudhury (2013) describe proprietary and open source software systems that provide management and publishing platforms. Brazil was an early adopter of the open source (free) software Open Journal Systems (OJS), developed by the Public Knowledge Project, with support from the Brazilian Science and Technology Institute (IBICT), which translates, distributes, and provides training for the platform, which it has renamed SEER, from the Portuguese acronym (IBICT, n.d.).

In 2014, 1,747 journals were using OJS in Brazil or 26% of country’s journals (Sandes- Guimarães & Costa, 2012). Among the journals rated A1-B2, the use of OJS increases to 54% (656/1,220). The use of OJS among journals also overlaps with their use of RedALyC and SciELO, suggesting the strength of the infrastructure for the national and global presence of Brazilian research through open access publishing systems (Figure 4).

Figure 4
Top 4 tiers Qualis serials in SciELO and RedALyC and using OJS.

Implications

In determining the extent and quality of open access among Brazilian journals, it first needs to be recognized how fluid and difficult it is to track the current state of the journal market. To Qualis’ credit, and despite a number of inevitable data errors and inconsistencies, the Brazilian government has established what appears to be the most comprehensive list of scholarly journals for the country. The Qualis list, along with its rating system, enables a number of important points to be made about the place of open access in Brazilian scholarly publishing. The fact that the proportion of open access journals goes up with higher rated journals points, to above the 90% level for A1, points to the integral role of open access in quality publishing, as it increases global access and readership, for Brazilian journals.

At the same time, it needs to be noted that the Brazilian academic community can be judged to be unsatisfied with the quality of the country’s journals. This can be seen in both its under utilization of the highest quality ratings and in its submitting work to higher rated journal abroad. In terms of what can be done to improve the perceived and actual quality of Brazilian journals, this study suggests a need for strategies to encourage better publishing within Brazil.

One such strategy may be to make more of the relative rarity of the APC among Brazilian open access journals. Consider Brazil’s biology journals, to take a field with room for growth in quality, and the policy implications. The current rating disparity encourages researchers to publish abroad. In doing so, this work is either inaccessible in Brazil without a subscription or the work faces an APC fee prior to publication, given that APCs prevail among biomedical open access journals in the Global North, reducing the amount that can be spent on research in Brazil. In response to this situation, the Brazilian government may, for example, opt to work further with SciELO and the journal community as a whole to support Brazilian journal quality, enhance journal and article metrics, and improve publishing platforms. The goal would be to make Brazilian journals more attractive for the best research and a wider readership. Attracting better work is key to improving the quality of the journal, editors, reviewers, board members, and readership.

This approach is in marked contrast to having Qualis send the signal to the academic community that Brazil has good journals but a smaller proportion of them than Brazilian researchers can find abroad. This only serves to perpetuate the current situation. This study establishes, for example, that the country’s leading journals are committed as a whole to open access and are doing so largely without the reliance on APCs that otherwise reduces the amount spent on the actual research conducted in Brazil.

The most favorable assessment to international journals in Qualis list can be attributed to several factors. It could be occasionally case of deficit in the quality of national journals, but it seems more to be related with the effectiveness of policies that promote the internationalization of Brazilian science, encouraging the publication in international journals. Brazil is an example in Open Access with public universities funded by the government, creative journals with small budgets, no interest by international commercial publishers, pressure for impact in citations and the work of national agencies, but still has a research evaluation system pressuring researchers for publication in international traditional non-OA journals. The premise of measuring the quality of the articles based on the quality of the journals should be changed. We emphasize that the Brazilian educational policy should further enhance publications in open access, recognize and value its OA publishing system supported directly or indirectly by graduate programs and societies.

CAPES needs to recognize that establishing a rating scheme based on widely accepted scholarly criteria is only the first step in improving the state of scholarly publishing and what it gives back to the world. It has to seize opportunities to recognize and promote within the academic community the value of publishing in Brazilian journals. This need not hamper anyone’s freedom to publish where they see fit. Rather, it enables authors to make informed decisions about where they place their work as a matter of extending the contribution of their work both globally and regionally, and in light of both scholarly and public use. It also enables CAPES to direct more of its considerable investment in Qualis to strengthening Brazil’s existing global leadership in making universal access to research and scholarship.

References

Abadal, E. (2012). Challenges for open access journals: Quantity, quality and economic sustainability. Hipertext.net, 10. Retrieved 7 November 2015 from http://www.upf.edu/hipertextnet/en/numero-10/challenges-for-open-access-journalsquantity- quality-and-economic-sustainability.html

Alperin, J. P. (2013). Brazil’s exception to the world-class university movement. Quality in Higher Education, 19(2), 2013: 158-172. http://dx.doi.org/10.1080/13538322.2013.802573

Alperin, J. P. (2014). South America: Citation databases omit local journals. Nature, 511(7508): 155–155. http://dx.doi.org/10.1038/511155c

Alperin, J. P. (2015). The Public Impact of Latin America’s Approach to Open Access (Doctoral dissertation, Stanford University). http://purl.stanford.edu/jr256tk1194

Alperin, J. P., Fischman, G. E., & Willinsky, J. (2011) Scholarly communication strategies in Latin America's research-intensive universities. Revista Educación Superior y Sociedad, 2011; 16(2). https://purl.stanford.edu/fj828hg2133

Andrade, J. B., & Galembeck, F. (2009). QUALIS: Quo Vadis? Química Nova, 32(1), 5. http://www.scielo.br/scielo.php?script=sci_arttext&pid= S0100-40422009000100001 http://dx.doi.org/10.1590/S0100-40422009000100001

Capes. (2013). Evaluation Board: Areas Documents and Committees. Available: http://www.avaliacaotrienal2013.capes.gov.br/documento-de-area-e-comissao.

Cerda Silva, A., & Lara, J. C. (2011). Políticas Editoriales de Publicaciones Académicas en Línea en Latinoamérica, 28. https://www.derechosdigitales.org/wp-content/uploads/Informe- Politicas_Editoriales_de_Publicaciones1.pdf

Cetto, A. M., & Alonso-Gamboa, J. O. (2008). Scientific periodicals in Latin America and the Caribbean: Global Perspective. Interciencia, 23(525): 84–93.

Cyzyk, M., & Choudhury, S. (2008). A survey and evaluation of open-source electronic publishing systems. University, Baltimore, MD: Sheridan Libraries, Johns Hopkins University. Retrieved March 6, 2013 from https://wiki.library.jhu.edu/display/epubs

Fischman, G., Alperin, J., & Willinsky, J. (2010). Visibility and quality in Spanish-language Latin American scholarly publishing. Information Technology and International Development, 6(4). http://itidjournal.org/itid/article/view/639

Guedon, J. C. (2008). Mixing and matching the green and gold roads to open access—Take 2. Serials Review, 24(1): 41-51. http://dx.doi.org/10.1016/j.serrev.2007.12.008

Haider, J. (2005). The geographic distribution of open access journals. ICML 9, World Congress on Health Information and Libraries. Salvador/Bahia, Brazil. Retrieved 7 November 2015 from http://lup.lub.lu.se/luur/download?func=downloadFile&recordOId=1549482&fileOId =3738628

IBICT. (n.d.). Lista de Portais e Periódicos que usam o SEER. Retrieved 11 November 2015 from http://seer.ibict.br/index.php?option=com_content&task=view&id=505&Itemid=144

Jamali, H. R., & Nabavi, M. (2015). Open access and sources of full-text articles in Google Scholar in different subject fields. Scientometrics, (July 23): 1-17.

Laakso, M., Welling P., Bukvova, H., Nyman, L., Björk, B. C., & Hedlund, T. (2011). The development of open access journal publishing from 1993 to 2009. PLOS One, 6(6): e20961. http://dx.doi.org/10.1371/journal.pone.002096

Lopez, A. P. A. (2001). Será que o Web Qualis é o melhor indicador? Retrieved 12 August 2015 from http://metodologiaci.blogspot.com.br/2011/02/sera-que-o-web-qualis-e-omelhor. html

Mafalda de Oliveria Amorim, K., Degani-Carneiro, F., Da Silva Avila, N., & Jose Marafon, G. (2015). Evaluation Systems of Scientific Journals in Latin America. In J. P. Alperin & G. E. Fischman (Eds.), Made in Latin America (English Ed.). Buenos Aires: CLACSO.

Mann, F., Von Walter, B., Hess, T. Y., & Wigand, R. (2009). Open access publishing in science: Why it is highly appreciated but rarely used. Communications of the ACM, 52(3): 135-139. http://dx.doi.org/10.1145/1467247.1467279

Márdero Arellano, M. A. (2008). Critérios para a preservação digital da informação científica. (Unpublished doctoral dissertation.) Universidade de Brasília.

Miguel, S., Chinchilla-Rodriguez, Z., & Moya-Anegón, F. (2011). A new approach to scientific visibility from the standpoint of access. Journal of the American Society for Information Science, 62(April), 1130-1145. http://dx.doi.org/10.1002/asi.

Morrison, H., Salhab, J., Calve-Genest, A., & Horava T. (2015). Open access Article Processing Charges: DOAJ Survey May 2014. Publications, 3: 1-16. http://www.mdpi.com/2304- 6775/3/1/1, http://dx.doi.org/10.3390/publications3010001

Packer, A. L. (2009). The SciELO Open Access: A Gold Way from the South. Canadian Journal of Higher Education/Revue canadienne d’enseignement supérieur. 39(3): 111-126. http://ojs.library.ubc.ca/index.php/cjhe

Packer, A. L., Cop, N., Luccisano, A., Ramalho, A., & Spinakorgs, E. (Orgs.) (2014). SciELO – 15 Anos de Acesso Aberto: Um estudo analítico sobre Acesso Aberto e comunicação científica. Paris: UNESCO. http://dx.doi.org/10.7476/9789237012376

Packer, A. L., & Meneghini, R. (2007). Learning to communicate science in developing countries. INCI, 32(9): 643-647. http://www.scielo.org/local/content/pdf/032.pdf

PLOS. (2013). How Open is it? Open Access Spectrum. Retrieved 5 September 2013 from http://www.plos.org/ about/open-access/howopenisit/

Rocha-e-Silva, M. (2009). O novo Qualis, ou a tragédia anunciada. Clinics 64(1): 1-4. Available: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1807-59322009000100001.

Rodrigues, R. S., & Abadal, E. (2014). Ibero-American journals in Scopus and Web of Science. Learned Publishing, 27(1). http://dx.doi.org/10.1087/20140109

Sandes-Guimaraes, L. V., Costa, S. M. S. (2012). Brazilian scientific journals that use the Open Journal Systems (OJS): A quality analysis. JISTEM - Journal of Information Systems and Technology Management Revista de Gestão da Tecnologia e Sistemas de Informação, 9(1): 61-88. http://dx.doi.org/10.4301/S1807-17752012000100004

Sandes-Guimaraes, L. V., Diniz, E. H. (2014). Gestão de periódicos científicos: estudo de casos em revistas da área de Administração. Revista de Administração, 49(3): 449-461. http://dx.doi.org/10.5700/rausp1160

Vessuri, H., Guedon, J.C., & Cetto, A.M. (2013). Excellence or quality? Impact of the current competition regime on science publishing in Latin America and its implications for development. Current Sociology. http://dx.doi.org/10.1177/0011392113512839

Appendix A

Figure A1
Screenshots of Qualis classification, with examples of the same journals with two ISSNs and of difference journals with the same ISSN.

Figure A2
Screenshots of Webqualis online Cadernos de História da Educação Qualis showing variations of ratings for the same journal.