Technical and artistic production by brazilian postgraduate courses with emphasis on Social Sciences and Humanities

Produção técnica e artística por cursos brasileiros de pós-graduação com ênfase em Ciências Sociais e Humanas

Producción técnica y artística de posgrados brasileños con énfasis en Ciencias Sociales y Humanidades

Concepta McManus concepta@unb.br

Universidade de Brasília, Brasil

Abilio Afonso Baeta Neves abiliobneves@gmail.com

Universidade de São Paulo, Brasil

Carlos Henrique de Carvalho carloshcarvalho06@yahoo.com.br

Faculdade de Educação, Brasil

This paper aims to look at the production profiles from Brazilian postgraduate courses, especially in the Social Sciences (SS), Humanities (H), Letters, Literature & Arts (LLA), and examine for whom they are producing this knowledge. It looks deeper into the results from McManus and Baeta Neves (2021), identifying sources of funds and other differences between areas of knowledge. Overall, the abbreviation SSH will be used to contemplate the three areas together.

It is more complicated to define and collect information on research’s social and political impacts when compared to bibliographic production and in the Social Sciences (SS), Humanities (H) and Letters, Literature & Arts (LLA) than in the hard sciences (McManus; Baeta Neves, 2021). Most authors agree that assessing these impacts is complex (Bastow; Dunleavy; Tinkler, 2014). Authors argue that impact should be measured as the process rather than the outcome (Molas-Gallart; Tang, 2011). Differences exist between what is produced within each area of knowledge and why and how this information is produced (Ochsner; Hug; Daniel, 2013). Pedersen, Grønvad and Hvidtfeldt (2020) note the methodologically solid pluralism needed for analysing these areas, which affects the outputs. According to Prewitt, Schwandt and Straf (2012), the information produced in all science fields needs SS to explain whether, how and why that knowledge is used.

Chen et al. (2015) stated that bibliographic methods could not fully capture the variety of products in SSH scholarship. Benneworth and Jonbloed (2010) found the valorisation of a universities’ worth solely through patenting, licensing, spin-off formation and technology transfer as a worrying development, limiting the impact of Social Sciences and Humanities (SSH). This is partly because models for the hard sciences are generally not transferable to SSH. Universities’ main stakeholders include the international scientific community, industry, politics, the public sector, and the general public (Jongbloed; Enders; Salerno, 2008), which demand a return from their investments. In this sense, it is more challenging to show the impact of SSH as it tends to be more time-dependent and sometimes less tangible than the (Science, Technology, Engineering and Mathematics) STEM areas. According to Benneworth and Jongbloed (2010), the universities do not promote the values of the SSH, especially in comparison to STEM areas. Nevertheless, the knowledge produced by these areas is used by government, business organizations, and cultural sector as well as information and technology industries (Olmos-Peñuela; Castro-Martínez; D’Este, 2014).

According to Kenyon (2014), implicit evaluations and comparative judgments without explicit criteria are fertile ground for inconsistency, arbitrariness and bias across a broad class of domains (e.g., Greenwald; Kreiger, 2006; Uhlmann; Cohen, 2005). Holbrook, Barr and Brown (2013) name 56 possible measures of research impact, with Esko, Tuunainen and Miettinen (2012) stating that quantitative indicators cannot capture the whole array of social and cultural impacts of research. Spaapen and Van Drooge (2011) proposed that impact in SSH should be measured through productive interactions, direct or personal; indirect through texts or artefacts; and financial through money or “in kind” contributions. Sovacool (2014) show that a broader pool of expertise is needed to understand how human behaviour affects demand and uptake of technologies, with SSH being marginalised. Pimenta (2016) lists various technologies in the SSH that are the product of research that are not bibliographical. These include applications for mobile phones, georeferenced systems, technological mediation in the information-communicational phenomenon, infographics, data visualisations, digital convergence, communication and multimodal traffic of information, interfaces and ethics of informational design in public access, among many others.

Felt (2014) also show that SSHs are portrayed as crucial for attaining the innovation goals yet are conceptualised as the junior partners; the leading role remains with science and engineering. Esteves, Franks and Vanclay (2012) highlight the need for culture, community, power, human rights, gender, justice, place, resilience and sustainable livelihoods as part of social impact assessment (SIA). They state that it is incumbent on SIA practitioners to educate proponents, regulators, and colleagues about these concepts and embed them into practice norms.

The Brazilian postgraduate system was created to qualify human resources and produce scientific and technological knowledge to allow the country’s industrial expansion (Baeta Neves, 2020; Moreira; Velho, 2008). In the context of the University Reform of 1968 (Silva, 1977; Durham, 2004), graduate studies serve to institutionalise research at Brazilian universities.

Debates about the emergence of new ways of producing knowledge and changing paradigms in Science and Technology (S&T) (Moreira; Velho, 2008) focus on the evaluation model. The main challenges of this system are changes in political, economic and social contexts, different from those in which postgraduate studies have been organised to date in Brazil. Moreira and Velho (2008) also point out that the acceleration of globalisation boosted the idea that complete, multi-institutional innovation systems should link science, technology, government and industry. In this context, human resource training has become even more critical to increase the country’s competitiveness, training highly qualified researchers from universities and the graduate system to work in scientific production and new cutting-edge technologies. Thus, the concern with science and technology’s environmental, social, economic, and political impacts must be part of human resource training for research in today’s society (McManus; Baeta Neves, 2021). The principles that guide the organisation and evaluation of postgraduate courses (PG) in Brazil still retain typical features of a simplistic design. Supovitz (2009) showed that accountability responds to a real need in the educational system to demonstrate that public money is spent wisely. High-risk testing encourages educators to align curriculum, standards and assessments.

When parameterising postgraduate products (which facilitates and speeds up the evaluation process) that do not adequately contemplate the different disciplinary profiles (McManus; Baeta Neves; Maranhão, 2020; McManus et al., 2021), academic institutionalisation would lead to a drop in the creation of relevant knowledge and a homogenised scientific production. As the quantity of output is pursued and articles published in scientific journals are prioritised, the result would be institutionalised disciplines that lead to partial, ultra-specialised and limited scope results. This would be detrimental to long-term research, as well as the construction of more complex theories and comprehensive explanations (Marenco, 2014), with wider scope and impact.

The limitation of knowledge within pre-determined assessment areas remains the majority (Maranhão, 2010), and we need more researchers’ openness to overcome the paradigm of disciplinary fragmentation (Schmitt et al., 2006). The Brazilian postgraduate system (Moreira; Velho, 2008) is based on principles of strictly academic recognition of the quality of courses, whose rewards are associated, among other criteria, with publications and the production of knowledge (Vogel; Kobashi, 2015) according to a linear pattern, from basic to applied science and, then, to development and production (Nobre; Freitas, 2017). This has caused a certain apparent homogeneity between the programs and an impatient search for concepts instead of accurate qualitative results. In addition (Bueno, 2015), under these conditions, showed that the Program’s productivity is revealed as a quantitative production of publications in the channels evaluated as valid by the formulators of CAPES’s evaluation standards, and efficiency becomes synonymous with the achievement of these standards. Therefore, the differentiated use of the resources for this production, notoriety or public effect of what they produce is not considered.

For Moita (2002), the ways currently used to evaluate the performance of postgraduate programs have been based on measurements conditioned to pre-fixed standards by the agencies and institutions that evaluate them. These hinder the potential efficiency existing in the group and the peculiarities they contemplate. According to Davyt and Velho (2000) and Baeta Neves and McManus (2020), what seems to have been maintained throughout this time is the notion that only scientists themselves can evaluate the work of their colleagues. Part of these objections is due to a lack of information on the parameters conventionally used in the academic evaluation process and responsible for institutionalising a discipline (Altman, 2012). Lopes et al. (2011) show that the internal collaboration of the graduate course in the country is a good indicator of the course grade.

It can be claimed, when prioritising quantitative indicators, such as the number of published articles, citations or impact factors, CAPES’ institutional evaluation would end up encouraging the quantity of scientific production at the expense of its quality or originality (McManus; Baeta Neves, 2021). It may also promote subterfuge to artificially increase these rates, such as self-citation, cross-reference or replication of the same production (Mishra et al., 2018).

Shifts in research policy and reductions in government spending have meant that research, which was historically viewed as a social endeavour for the public good, is being recast in terms of productivity, economic efficiency, accountability, and value for money. Such shifts are part of a broader “audit culture” which has emerged in Higher Education Institutions (HEIs) (Shore; Wright, 2015). A new social contract has now arisen between science and state, which revolves around accountability, relevance and value (Demeritt, 2010). Therefore, research evaluation has become prominent in many industrialised countries (Guena; Martin, 2003). These research evaluation systems, which were once associated with assessing scientific output, are now increasingly associated with assessing societal impact as governments endeavour to ensure academic research is accountable and beneficial to society. With the emergence of societal impact assessments, a situation has emerged in which evaluators have no experience in applying this new, untested and unclear criterion (Derrick; Samuel, 2016; Samuel; Derrick, 2015).

Van Leeuwen (2013) states that current bibliometric tools are not suitable to assess the actual research production of scholars in SSH. This author suggests that a broader focus on a wider variety of publication types is necessary, and these are not always intended for a purely academic audience. In this sense, much bibliographic production may have a technical use, or technical production published in bibliographic form (Aquino; Cortese; Shibao, 2019). Spaapen and Van Drooge (2011) show a need to narrow the gap between research and impact, focusing on productive yet reliable interactions. Accepted indicators are not available for social impact assessment. These authors recognise problems with the lack of quantitative data, lack of consensus on what data to collect, audience definition and expectations, attribution of impact and temporality. Olmos-Peñuela, Castro-Martínez and D’Este (2014) show that use of knowledge produced in universities in higher when there is interaction with external stakeholders.

According to Henshall (2011), research leads to a wide range of benefits or types of “payback”. According to these authors, much research leads, by subsequent application of the knowledge it generates, to significant benefits for society beyond that which the knowledge itself brings to science. It can be challenging to understand the ways and extent to which any individual project, researcher, or programme of research does this, even looking back with the wisdom of hindsight. It is even more challenging to predict, looking forward without the wisdom of hindsight, the characteristics of projects, researchers, or programmes that are likely to be associated with the greatest payback or impact. Volberda et al. (2010) show that 75% of successful innovation depends on social innovation, such as new forms of organising work and relations, and only 25% on R&D and new knowledge. In this case, the evaluation procedure should recognise that research, to achieve social impact, must cross disciplinary boundaries and those of other professional expertise and social domains.

New research assessment frameworks increasingly include previously untested criteria, considering how research is considered excellent or relevant outside the academic sphere (Derrick; Samuel, 2016). Donovan (2011) stated that metrics-only approaches employing economic data and science, technology and innovation indicators are in the past. Best practices combine relevant qualitative and quantitative indicators to gauge research’s broader social, environmental, cultural and economic public value. The author states that it is often impossible to connect research and “impact” outcomes in a maze of complex social interactions and serendipitous turns. So, a cruder approach may be necessary (Martin, 2011). Molas-Gallart and Tang (2011) encourage “contribution not attribution” evaluations, while impact need not be conceived of purely in terms of economic returns but can embody broader public value in the form of social, cultural, environmental and economic benefits. Methods such as the Payback Framework (Donovan; Hanney, 2011) and SIAMPI (Social Impact Assessment Methods for research and funding instruments through the study of Productive Interactions between science and society) or “Participatory framework” (Rowe; Frewer, 2005) have the flexibility to accommodate these and many other impact domains. Muhonen, Benneworth and Olmos-Peñuela (2020) state that the interactions and the changes they mediate need to be monitored. They define 12 pathways in which SSH can impact societal change through dissemination, co-creation, reacting to societal change, and driving societal change. Productive interactions are seen as exchanges between researchers and stakeholders whereby the knowledge produced is scientifically robust and socially relevant (Spaapen; Van Drooge, 2011). According to these authors, productive interactions tell us how researchers communicate with their environment (direct, indirect and financial). This broadens the impact of research and how it affects the public.

Rowe and Frewer (2005) ’s study on the role of public engagement in regulating the flow of knowledge from scientists to societal users introduced the concepts of public communication, public consultation and public participation. These authors recognised about 100 different manners of public engagement with scientists did not discuss the mechanisms involved between engagement activities and impacts (Muhonen; Benneworth; Olmos-Peñuela, 2020). Evaluation should look at small but necessary steps in achieving social impact and can be seen as intermediate indications of de facto impact. According to Klautzer et al. (2011), while some principal investigators (PIs) could identify specific impacts of their research, PIs generally thought they had incrementally influenced policy and informed the policy debate. Some consequences may be inaccessible to evaluation, and some evaluations may occur too early or too late to capture the impact of research on a constantly changing policy environment. Pedersen, Grønvad and Hvidtfeldt (2020) list various types of impact, such as academic, policy, social, educational, cultural and economic, while Olmos-Peñuela, Castro-Martínez and D’Este (2014) show that much interaction by research groups is through consultancy and contracts, resulting in research reports and other types of material but not necessarily published scientific papers. In light of this, we attempted to identify products of postgraduate courses in Brazil, highlighting for by whom they were financed and therefore their sphere of impact.

Data from this study came from the Sucupira database (2013-2016) and available in excel format at Dados Abertos CAPES (CAPES, s./d.). This data is collected annually by Capes (Coordination for Improvement of Higher Education Personnel linked to the Ministry of Education - MEC) and includes information from the postgraduate courses such as localization, institution, lecturers and students. The available production by the postgraduate programs in terms of bibliographical, technical and artistic works is generally imported directly from the Curriculum Lattes from all master and doctorate postgraduate lecturers and programs in Brazil. This agency is responsible for funding and evaluating postgraduate education. CNPq (National Council for Scientific and Technological Development, linked to the Ministry of Science, Technology, Innovation and Communications - MCTIC) maintains the Lattes Curriculum used to feed the Sucupira database on an annual basis by the coordinators of postgraduate programs in Brazil. Individual lecturers (as well as students, technicians etc.) maintain the Lattes database up-to-date, but only lecturer information is imported into Sucupira. Scholarships, courses/programs, lecturers, students and production in artistic, technical and bibliographic databases were downloaded from Sucupira for the period from 2013-2016 (last whole evaluation period). In theory, all activities by postgraduate lecturers in Brazil should be registered on the Lattes platform and therefore imported into the Sucupira platform. The platform uses data in three broad categories: bibliographic, artistic and technical. This paper looks mainly at the latter two, although a subdivision of books and chapters is highlighted insomuch as it refers to research reports and didactic material and (Olmos-Peñuela; Castro-Martínez; D’Este, 2014). Artistic production looks at the cultural area, including scenic arts, music, visual arts, and other cultural and artistic productions. Technical production includes Patents, Computer programs, Technological products, Consultancy, advisory, processes and techniques, didactic or instructional material, editing, interviews, round tables, programs or commentaries in the media, scientific reports, social networks, blogs, websites, organisation of events, evaluation of journal papers, short courses, participation in examining boards and lectures given. Information also includes source of funding of these services and products.

Capes collects data in nine knowledge areas - Agricultural, Health (Medical), Engineering, Biological, Exact & Earth, Applied Social Sciences (SS), Humanities (H), Linguistics, Letters & Arts (LLA) as well as Multidisciplinary. The latter treats postgraduate courses with actions in more than one area of knowledge and is subdivided into Biotechnology, Environmental Sciences, Teaching, Materials and Interdisciplinary.

Technical services were divided into political and socio-economic as shown in Reale et al. (2018). The data was cleaned by the authors by correcting the spelling and names of agencies. Project and scientific paper revisions for scientific journals and funding agencies were not considered in this analysis and so removed. Definitions were made for whom the service was intended - federal - separating ministries and the ministry of education due to the volume (MEC), state or municipal governments, as well as international organisms such as Food and Agriculture Organization (FAO), World Health Organization (WHO), United Nations (UN) etc., internationally on the political side and businesses, private individuals, confessional institutions, cultural centres (such as museums, theatres), private individuals, S-System, communications networks (not including interviews), and social organisms such as trade unions. It should be noted that the Lattes curriculum does not classify this information and the separation made by the authors.

The paper also looks at books and book chapters, which although bibliographical in nature may be of technical use. Data on reviews of papers or abstracts for journals, congresses, funding agencies etc., were not considered in this analysis as they have been published elsewhere (McManus; Baeta Neves, 2022).

The funder of the technical work was used to form word clouds. These consider the number of times the identification of who funded the research appears in the data bank. Subdivisions of technical and artistic (and therefore used in the construction of word clouds) are defined by the Sucupira platform.

Statistical analyses included correspondence analyses using SAS® (Statistical Analysis System Institute, Cary, North Carolina). Correspondence analyses (PROC CORRESP) were carried out with all data and by area of knowledge and funder of the technical work and also separated if the funder was socio-economic or political.

While exact and life sciences productions are concentrated in scientific journals and congresses, SSH tends to be more diversified, with a high output of technical services. This implies that the super valuation of one type of production over the rest may be prejudicial to these knowledge areas. Life and Exact sciences produce relatively more in congresses and journals than SS, H and LLA (Graphic 1). Annex 2 summarises different production types by area of knowledge, expanded from Graphic 1. Humanities, SSA and especially LLA, are most active in the visual arts (Table S2).

Graphic 1:
Overall Production per lecturer in postgraduate courses in Brazil by area of knowledge (Sucupira 2013-2016)
Source: Elaborated by authors from Sucupira data (2013-2016).

These differences in concentrations in different production types reflect different priorities for knowledge areas and need to be adequately understood before measuring their impact. As can be seen in Graphic 2, Social Sciences (SS), Humanities (H), Letters, Literature and Arts (LLA) are responsible for almost all productions except publications in scientific journals and events.

Graphic 2:
Percentage of the total of all products in postgraduate courses in Brazil that is from Social Sciences, Humanities, Letters, Literature and Arts (Sucupira 2013-2016)
Source: Elaborated by authors from Sucupira data (2013-2016).

The SSH areas tended to carry out more services than other areas (Graphic 3). This may be due to the importance of these services for the areas in question. It should be remembered that although this information was collected from the Sucupira database used to evaluate the postgraduate courses in Brazil, this information is also used by lecturers for career advancement or for the attainment of resources for research, among others. All areas highly interact with the business sector (Graphic 4). Agricultural sciences and engineering showed a higher percentage of work for businesses while SS, H and LLA more for governments (local, state or federal).

Graphic 3:
Percentage of services per funder
Source: Elaborated by authors from Sucupira data (2013-2016).

Medical and Applied SS have more international funding, as well as humanities (Graphic 3 and 4). This is probably because international organisations such as the World Bank, Unesco and other international organisations are of quality due to their international agenda and Brazilian production in this area.

Types of products such as Apps, Newspaper articles, patents, maps and translations differ between areas of knowledge (Table 1). Many apps are seen in the Exact & Earth, Engineering and Multidisciplinary Sciences, while newspaper articles were concentrated in the Humanities and Applied Social Sciences. These latter areas also saw more translations as well as LLA. Patent numbers were high in medical and multidisciplinary sciences.

Graphic 4:
Percentage of services per area of knowledge
Source: Elaborated by authors from Sucupira data (2013-2016).

Table 1:
Types of technical production by area of knowledge from Sucupira database (2013-2016)

Source: Elaborated by authors from Sucupira data (2013-2016).

In the political realm (Graphic 5 and 6), Agrarian and Health Sciences were close to international institutions because of their relationship with WHO and FAO agencies. SSA was linked to state organizations while LLA to municipal and the Ministry of Education (MEC). Humanities were linked to museums, confessional orders, and individuals of the private sector in the socio-economic realm, while SSA and LLA to businesses and social organizations. The knowledge area in annex 1 shows individual correspondence analyses. For the humanities, politics and international studies show higher correspondence with cultural and federal activities, while, as expected, religion and theology with confessional institutions. Geography shows higher correspondence at the state level, while education is together with MEC. In the social sciences, Law is linked to both federal and state governments and social organisations. Architecture shows more at the municipal level while planning, economics and administration is connected to the business sector, the S_System and MEC. In LLA, linguistics shows correspondence with the private sector, Arts with cultural activities and Letters with social organisations and other ministries.

Graphic 5:
Correspondence analyses for political
LLA: Linguistics, Literature and Arts; SSA: Applied Social SciencesSource: Elaborated by authors from Sucupira data (2013-2016).

Graphic 6:
Correspondence analyses for socio-economic production
LLA: Linguistics, Literature and Arts; SSA: Applied Social SciencesSource: Elaborated by authors from Sucupira data (2013-2016).

Funding or supporting production related to art and culture is varied (Figure 1). Applications for computer or cellphone and books and chapters showed funding from government agencies such as Capes and CNPq. Note the demand for maps made by SEEG (System for Estimates of Emissions and Removal of Greenhouse Gases) and WRI (World Resources Institute). Culture has the support of the Rouanet Law (Brasil, s./d.) (that supports cultural activities in Brazil) and SESC and numerous other entities, mediating funding for music and the arts. Other funding sources may be available but not registered in the Lattes database.

Figure 1:
Word clouds for funders of different types of production in Brazilian
Source: Elaborated by authors from Sucupira data (2013-2016).

The production of books and chapters (Graphic 7 and 8) in some cases may be considered technical production, such as manuals, professional works and didactic material. This definition was given by the lecturer when filling in his/her curriculum. While bibliographical in nature its use may be more technical. The production of books and chapters is similar by area of knowledge. While most areas produce these documents from their research projects, Human, SS, LLA and multidisciplinary also produce much didactic and technical material, reinforcing the need for production in Portuguese, with 86,9% in Portuguese, 11% in other languages (mainly English) and 2% in more than one language. There was little difference between languages for the different types of production, varying from 79% in Portuguese for translations to 90,6% for Professional works.

Graphic 7:
Number and types of chapters by area of knowledge
Source: Elaborated by authors from Sucupira data (2013-2016).

Graphic 8:
Number and types of books by area of knowledge
Source: Elaborated by authors from Sucupira data (2013-2016).

Archambault and Larivière (2011) showed three limitations of bibliometrics in the analysis of the impact of SSH. They show that assessing the impact of the humanities applied social sciences and linguistics, letters and arts, as previously stated, must also include the socio-economic and political dimensions. Reale et al. (2018) stated that political impact refers to the transfer of research findings to the political sphere to inform decision-making or policy design. On the other hand, social impact refers to how an action from a policy or a civil society-led action has contributed to improving identified social challenges. Thereby, by identifying the studies’ funders in the present paper, we can see who the potential users of this information are and the possible impact. According to Kenyon (2014), SSH products include books, conference presentations, policy-writing, legal opinions, creative performances and gallery shows, all of which may be expressions of research output in SSH disciplines (Graphics 1 and 2). The diversity of outputs from SSH was highlighted by Olmos-Peñuela, Castro-Martínez and D’Este (2014), and engagement of these areas in consultancy, contract research, and training can result in products, although bibliographical (didactic material, research reports), may not be citable as seen here. This can be analysed in the light of national data, even if they present problems and inconsistencies (McManus; Baeta Neves, 2021).

According to Reale et al. (2018), impact is often understood as a change that research outcomes produce upon academic activities, the economy, and society at large. Social Sciences and Humanities (SSH) because of their organisational and epistemic characteristics and the type of outcomes that differentiate them from the Science, Technology, Engineering, and Mathematics (STEM) disciplines (Bastow; Dunleavy; Tinkler, 2014). This was also seen here and in McManus and Baeta Neves (2021). Reale et al. (2018) also noted that in both political and social impact of SSH research and, to some extent, scientific impact, there was an increasing trend toward responding to the demand to create new opportunities for participation and public engagement of researchers and stakeholders. The present study shows the importance of government and political organs in using information produced by the different areas of knowledge (Graphics 3, 4, 5 and 6).

Aiello et al. (2021) identified strategies contributing to achieving the social impact of research projects. These included a clear focus on social impact and the definition of an active strategy for achieving it; a meaningful involvement of stakeholders and end-users throughout the project lifespan, including local organisations, underprivileged end-users, and policy-makers who not only are recipients of knowledge generated by the research projects but participate in the co-creation of knowledge; coordination between projects’ and stakeholders’ activities; and dissemination activities that show valuable evidence and are oriented toward creating space for public deliberation with a diverse public. Results in the present study show that international, national and local governments and public and private organisations use SSH products (Graphics 5 and 6; Figure 1). Identification of funders can also help to identify possible end-users. While the financing of book and chapter production by CAPES and CNPq is in line with these agencies being the main funders of post-graduate education in Brazil, they also were main funders for computer apps, as by-products of research projects. Other productions show financing from both governmental and private sources. These include short courses from schools and states, cultural activities (music, art) by museums, SESC (Social Service maintained by the Commerce Sector), Galleries and Universities, publications in local and national newspapers and magazines as well maps by carparks, developmental agencies such as WRI or Mapbiomas, and geological services such as CPRM.

According to Benneworth and Jongbloed (2009), universities have generally become a representative for the “arts” community in governmental discussions. According to these authors, the strategy has become essential in positioning universities as important cultural actors. While there might be a lot of rhetoric, these authors state that valorisation comes through engagement with community stakeholders. Figure 1 shows the funders for scenic and visual arts in Brazil count on support from museums, theatres, and federal and local governments.

Graphics 1 and 2 show the importance of books for SS, H and LLA areas. This is a tradition in Humanities in western science. Books in these areas are not necessarily solely academic, being of historical, cultural or local interest (Seoane; Rodríguez, 2015) and therefore may not be citable (Graphics 7 and 8). Nevertheless, some attempts have been made to verify the impact of books through Book Citation Counts, of which few Brazilian are registered, for example, in international databases such as SciVal (based on Scopus from Elsevier) or Incites (based on Web of Science, from Clarivate Analystics). Library Holdings (WorldCat), Book Reviews (Google Scholar) or Other Qualitative Indicators such as SearchBox Book reviews may be used. Still, data on Brazilian books in these databases are scarce, maybe because of the language barrier or because they are not used as citable references but rather as didactic or historical sources or for the general public (Sivertsen; Larsen, 2012). This use cannot be seen as less important than citable scientific papers as it may cater to cultural peculiarities of students, encourages learning through activities contextualized in the environment, and contribute to the preservation of local heritage (Seoane; Rodríguez, 2015). Use of this type of material also varies by subject area (Cooke; Rosenthal, 2011; Roesnita; Zainab, 2005). As such, SSH tends to have a more context-sensitive and nuanced understanding of impact (Pedersen; Grønvad; Hvidtfeldt, 2020), including book production for general audiences. McManus et al. (2022) show that bibliographic production in SSH have wider impact than citation or bibliographic analyses, especially those analysed using bases such as Web of Science and Scopus. SSH also has many more research and technical reports, manuals and didactic material than other areas (Graphics 1, 7 and 8). Neylon, Willmers and King (2014) also showed that these types of bibliographic production are not always cited, but also what is cited (such as scientific papers in journals) may not be used.

It is possible to determine what an area of knowledge produces in Brazil. This would mean modifying the Curriculum Lattes in Brazil to collect this information more efficiently (McManus; Baeta Neves, 2021). Keeping this curriculum up-to-date is an annual obligation for all lecturers and students in postgraduate programs. It is also used to request grants and scholarships and for career advancement. Therefore, it is a powerful tool for measuring the political and socio-economic impact of research in Brazil but needs to be modified for this end.

Pedersen, Grønvad and Hvidtfeldt (2020) indicate that there is considerable room for researchers, universities, and funding agencies to establish impact assessment tools directed toward specific missions while avoiding catch-all indicators and universal metrics. This is in line with Davies et al. (2021), who states that citation counts fail to capture how research can affect policy, practice, and the public, with the real impact of SS, H and LLA frequently being overlooked. Even within areas of knowledge, no consensus exists (Belfiore; Bennett, 2009). Here, (Graphic 2) SSH accounts for more than 50% of Brazilian postgraduate courses’ technical and artistic output.

It should be noted that a change is underway in Brazil, with pressure being put on these areas to move away from the traditional manner of production (technical production, books, chapters, congresses) and into the publication of scientific papers, especially in international journals, as happens in other areas of knowledge. As discussed above, this is not a simple task. The evaluation of SS, H and LLA needs to be better understood and examined so that the relevant products from these areas is not lost. Nevertheless, quality and relevance must be maintained when publishing scientific papers (McManus; Baeta Neves; Maranhão, 2020).

This paper aims to look at the production profiles from Brazilian postgraduate courses, especially in the Social Sciences (SS), Humanities (H), Letters, Literature & Arts (LLA), and examine for whom they are producing this knowledge. It looks deeper into the results from McManus and Baeta Neves (2021), identifying sources of funds and other differences between areas of knowledge. The type of production is different in humanities, social sciences and Literature, Letters & Arts compared with other fields of knowledge. Except for articles in journals and congresses, the production of these areas is predominant in Brazilian postgraduate studies. The evaluation of these different areas of knowledge cannot exclude artistic and technical production, as they are a significant part of their impact (political and socio-economic). The funding sources for these products also vary by product and subarea of knowledge.

To Capes for funding (finance code 001).