1 INTRODUCTION

Co-production contributes to public administration as it creates a link between citizen participation and the production of public services. Such a link has a two-fold transformative function, it works both for the citizens and for the quality of the services they provide (Salm & Menegasso, 2010). Brandsen and Pestoff (2006) observed that the actors involved in the production process exchange system components.

Based on the fact that this production process entails new forms of management that rely on a greater articulation between the State and society and creates opportunities and challenges to managers, it is crucial to study and debate this exchange between the actors involved. Some authors (Verschuere et al., 2012; Pestoff, 2012; Vanleene et al., 2015) recognize that citizens need to participate in the provision of a good part of the services. In this sense, creating a scale of typologies of co-production can be useful for managers to improve the efficiency of service provision. Then, this study was motivated by the following research question: how can we measure the type of co-production according to the degree of citizen involvement?

This question is due to the empirical observance of the phenomenon of co-production, increasingly frequent in society, relying on the essential elements raised by Schommer et al. (2011), such as the strategy of production of public goods and services in networks and partnerships, and the mutual engagement of governments and citizens, either individually or around organizations. Then, this article aims to create a scale of typologies of co-production, following the methodology proposed by Costa (2011). Typology encompasses the classifications, models and types of co-production existing in the literature, and it was chosen due to the existence of previous studies that have proposed various typologies.

The work of Salm and Menegasso (2010), on a conceptual model for the co-production of public services relying on the typologies of participation served as a reference for constructing the scale and for indicating the research question. In addition, we sought to fill a knowledge gap with a quantitative study (Verschuere et al., 2012; Pestoff, 2012; Chaebo & Medeiros, 2017; Nabatch et al., 2017) on the types of co-production, advancing beyond conceptual models. Thus, as we built an instrument to measure typology consisting of 19 items divided into five dimensions, we have produced a scale with a great potential to contribute to studies on co-production.

We have built an unprecedented scale that relies predominantly on qualitative research and case studies. This enables comparing different cases and experiences of co-production in different areas. Also, we expect to provide subsidies for managers to create and implement public policies to be developed jointly with society. Although there exists a study dealing with the scale of co-production in the Brazilian context (Bezerra et al., 2022), our study differs from it since those authors point out the aspects that motivate citizens to participate in the co-production of public services by taking part in Civil Society Organizations (CSOs) while we point out the type (level) of co-production based on citizen involvement. That is, they point out the why whereas we point out how citizen participation occurs.

Given the arguments presented about co-production and its potential to be explored as a management strategy, this research contributes with an empirically tested and validated instrument. Our work seeks to provide subsidies for managers to know the typologies of co-production set to solve various social problems, with the involvement of the population. We expect that in the future qualitative studies can address historical, economic, political, social, and cultural aspects to help explain the processes adopted in the creation of the items of the scale proposed in this work. We also expect that quantitative or mixed studies can use our scale to study specific groups or contexts.

The research presents its contributions, both theoretical and practical, as it delivers a scale capable of measuring the various ways citizens see co-production, and with the potential to contribute to future studies. In the theoretical field, the scale can assist researchers in quantitative, qualitative or mixed studies in advancing the field of co-production. In the practical aspect, the scale can be useful to public and/or social managers who seek to base management on the type of co-production to be implemented. Therefore, adopting a typology model makes it possible to compare different cases and experiences of co-production, and it contributes to improve evaluation, transparency, and communication. As for future studies, it should be noted that the typology scale can be used alone or combined with the motivation scale developed by Bezerra et al. (2022).

2 THEORETICAL FRAMEWORK

2.1 Co-production of the public services

Co-production is a broad concept that can take on different meanings and definitions in different contexts (Brandsen & Pestoff, 2006; Verschuere et al., 2012). In this work, we have focused on the definition that addresses the role of citizens and their relationships with the State in the provision of public services (Bovaird, 2007; Brandsen & Pestoff, 2006). The theoretical relevance of the discussion on co-production is verified in the growing number of studies on the subject in recent decades (Bezerra et al., 2022; Osborne et al., 2016; Pestoff, 2012; Vanleene et al., 2015). Therefore, Verschuere et al. (2012) consider that the theoretical understanding of co-production is important in several aspects. In turn, Chaebo and Medeiros (2017) concluded that the literature fails to articulate the concepts of co-production.

The term co-production is associated with different phases of the public policy cycle. According to Pestoff et al. (2013), co-production can serve as an “umbrella” for various types of participation of citizens or CSOs, from the creation to the delivery of public services. For this reason, Table 1 presents a timeline for the concept of co-production, adapted from the original texts, which shows its evolution over time.

Source: Adapted from Schommer and Tavares (2017) and Chaebo and Medeiros (2017).

The State recognizes the complexity of contemporary public problems and admits that their solution involves multiple forms of expertise (Schommer et al., 2011). Co-production innovated public administration (Dos-Reis & Isidro-Filho, 2019), addressed issues related to society in general and promoted participation and, therefore, citizenship.

The proposal is for services to be no longer delivered by professionals or public agents alone but, rather, co-produced by users, citizens and CSOs (Age & Schommer, 2017). However, it is worth noting that the strategy of co-production does not apply to all types of public services nor is it the panacea for all social problems.

It can be concluded that the concept of co-production is broad and complex. According to Lotta (2017), co-production is treated in different ways, according to the objective of the person who creates the concept. It also gets political interference and varies in form and degree of intensity. At times, there is an economic-financial bias, at other times, a social bias. Some currents in the literature see it only in the implementation of services, while others extend the understanding to all phases of the public policy cycle. Thus, these are points that need further discussion to create a more consistent theoretical framework on the topic. The same breadth and complexity can be observed when discussing typologies of co-production, as noted below.

2.2 Types of co-production

Recognizing the variety of concepts and typologies is key to achieve a greater coherence and consistency on the topic. Then, according to Nabatchi, et al. (2017), adopting a typology model facilitates comparing different cases and experiences of co-production, and it contributes to improving evaluation, transparency, and communication.

According to Lotta (2017), the existing classifications of co-production found in the literature stem from the breadth of its concept in the search for a better empirical and theoretical understanding. Similarly, Schommer and Tavares (2017) found that several authors propose models and types of co-production.

The work or Whitaker (1980) presented a typology for co-production of public services that is considered a classic in the subject. It was one of the first studies on the subject and presents three different forms of citizen action in co-production, classified as follows: by requesting assistance from service agents; by cooperating with service agents; and by negotiating with the public body providing services.

In addition, Brudney and England (1983) also proposed a relevant typology for the co-production of public services, classified into individual, group and collective. Their publication served as a starting point for other researchers, who adopted the typology they presented to propose new models or even their reformulation and expansion (Nabatchi et al., 2017).

Salm and Menegasso (2010) also proposed models of co-production of public services based on typologies of participation. To do this, they based themselves on three works on citizen participation, namely, by Arnstein (1969), Pretty (1995) and White (1996), as well as studies on the co-production of public services. Then, they related the two things and created a conceptual model of co-production of public services.

Nevertheless, Brandsen and Honingh (2016) set out to distinguish different types of co-production, based on a conceptual analysis of the classical definitions of the subject. According to those authors, identifying the various typologies is essential to make the research on co-production more comparable. The classification takes into account the nature of the service (core or complementary) and the degree of citizen participation (implementation or design and implementation). Given the variety and complexity of the models of co-production, there is no single model or typology for co-production.

In this sense, it is worth highlighting the work of the Brazilian authors Salm and Menegasso (2010), who considered the participation and involvement of citizens to different degrees to propose a conceptual model of co-production. However, as they propose a conceptual model of co-production of public services based on works on citizen participation, Salm and Menegasso (2010) expose a relevant fragility. Participation and co-production are different concepts. It is known that all co-production is a form of participation, but not all participation is a form of co-production. The concept of citizen participation is broader.

Moreover, Salm and Menegasso (2010) recognize the limitations of their study and leave it up to those interested in the theme to improve the model. There is a knowledge gap that can be explored to deepen the studies on the types of co-production in Brazil. In this sense, this research advanced in relation to that one by proposing a model based on typologies of co-production.

2.3 Conceptual model of typologies of co-production of public services

The research began with a qualitative phase by conducting a systematic literature review (SLR) (Gouveia et al., 2023). Then, the development of a new measurement instrument took into account the classifications, models and types of co-production existing in the literature. This was fundamental for constructing the conceptual model of typologies of co-production.

After analyzing the works surveyed in the SLR, we opted for creating our own model of typology of co-production, since according to Gouveia et al. (2023) there is no adequate model in the literature with the potential to measure this construct. Creating the model is fundamental to achieve the objective proposed in this paper, as it highlights the dimensions of the typology into which the items will be grouped for later developing the scale of typologies of co-production. It is worth noting that the conceptual model precedes the construction of the scale. The conceptual model gives the theoretical support for the final instrument, into which the items will be grouped.

The proposal of a new typology took into account the degree of involvement of individuals in co-production activities, which, inspired by Arnstein's ladder (1969) and the conceptual model of Salm and Menegasso (2010), is a possibly measurable variable, which allows developing a scale. The idea of involving citizens to different degrees is a challenge and an innovation for public administration, as it contemplates the understanding of co-production under the “umbrella” of social participation.

Then, we designed a four-level scale (Table 2) that varies as citizens play an increasingly active role in the implementation of public services in partnership with the State. Therefore, the proposed model acknowledges co-production as an integrative and democratic management strategy, and perceives citizens as indispensable elements for public policies to be successful.

This model shows four categories and the existence of thresholds among them, which do not always correspond to the complexities of the co-production processes. In fact, there may be other typologies with more or less “pure” characteristics, the distinction between which is not as clear and "pure" as presented here. This sheds light on an issue that gets little attention in the literature (Brudney & England, 1983; Strokosch, 2013; Brandsen & Honingh, 2016), which is that, in fact, the involvement of citizens in the activities of co-production of public services is gradational.

As a result, it is worth mentioning that knowing the typologies can be useful to managers, since it allows them to identify the different forms of co-production and to select the one that best aligns with the objectives proposed by the public management, and functions as a management strategy that meets the population's demand for a greater share of management, considering the initiative of the State to employ co-production.

The types of co-production of public services are expected to vary depending on the nature of the services, which generates an appropriate type of co-production according to the degree of citizen engagement. Therefore, this model is meant to take a further step towards shedding light on a proposal for a model of co-production that results from the interest and effort of researchers on this topic.

3 METHODOLOGY

This research aims to create a scale of typologies of co-production of public services. First, we conducted a SLR (Gouveia et al., 2023) which analyzed 105 works from October to November/2019. As for the inclusion criteria: studies had to be published in the form of a scientific paper; works had to be available in full; innovative proposals of typologies; studies considered a classic in the area. The following databases were used: Web of Science, Scopus, Scientific Electronic Library Online-SciELO, Scientific Periodicals Electronic Library-Spell and the Capes Journals Portal.

The next step consisted of constructing the scale of typologies of co-production of public services. At that moment, the ten steps for building scales proposed by Costa (2011) were observed. The scale construction model, represented in Figure 1, was developed based on theoretical principles and foundations on the subject and aligns with the so-called classical test theory. Thus, the instrument was created by following the steps presented in Figure 1.

The fieldwork and application of the research instrument occurred entirely online, mainly due to the social isolation motivated by the coronavirus pandemic. The survey was applied to randomly chosen citizens, who could collaborate by answering the questionnaire. The questionnaire was developed as a Google Form and shared on WhatsApp.

It is worth highlighting that ethics was a concern during this study. Throughout the research, the authors sought to adopt an impartial and neutral attitude towards the decisions inherent in the investigative process. Ethics warrants validity and reliability to the results, which are fundamental aspects in scientific works, especially regarding the use of data collected through questionnaires answered anonymously.

4 CONSTRUCTION OF THE SCALE AND DISCUSSION OF THE RESULTS

After the qualitative phase came the process of generating the items that were used to measure the construct. Based on the SLR, resulting from Step 1, we proceeded to items prospection and face and content validation. Fourteen experts collaborated in this phase, including M.Scs, Ph.Ds, and professionals in the areas of Public Management, Statistics and Business Management, all of whom experienced in creating measurement instruments.

In Step 2, Costa (2010) explains that the more abstract the construct, the greater the number of items to be generated. This is due to the concern of defining items for each of the aspects of the construct and not running the risk of failing to adequately cover all these aspects. Since the construct proposed here presents dimensions, items generation was systematized by the dimensions defined for the construct. Items generations considered literature evaluation, qualitative-exploratory procedures and consulting specialists. Still, Costa (2010) recommends generating as many items as possible, considering the adherence to the definition of the construct and its dimensions.

For this, the preliminary instrument was sent to the experts via e-mail to be analyzed. They were sent the list of the items divided by dimension accompanied by the respective description of what is intended to be measured by those indicators grouped into factors. The evaluation was meant to verify whether the item was clear, relevant, representative, and suitable to the typology construct. Thus, the face and content validation instrument had five degrees (1 - very bad to 5 - very good).

After that, the points raised by the experts were brought to a meeting to be analyzed by the members of an academic research group; the items were adjusted for the first data collection. This led to a proposal with 28 items divided into four dimensions: initial, basic, intermediate and advanced, according to Table 3.

The collection instrument used in this research used a 10-point Likert scale for verification. It goes from 1 to10, and the respondent must bear in mind that 1 indicates total disagreement and 10 means total agreement. According to Costa (2011), when items are measured by more than seven points, scale reliability is increased. This is the reason why the 10-point scale was chosen; in addition, the two ends of the scale are farther away, which facilitates respondents' understanding.

Initially, the instrument was applied to any citizen with an exploratory purpose, to get to know the field and later make the appropriate adjustments, making it a non-probabilistic sample. Data were first collected from November 18 to 29, 2020 and retrieved 295 valid responses. This quantity meets the recommendation of Costa (2011), that the number of responses should be at least ten times as big as the number of items in the measurement instrument.

After that, the scale was cleaned. This basically consists in analyzing the results of the first collection by using the techniques of correlation analysis, exploratory factor analysis, and reliability analysis. In the end, the instrument is expected to be improved by excluding the items that did not fit the proposed dimensions or even deciding for maintaining a certain item after changing its wording to test it in the second round of data collection.

First, the validity of the 295 responses was verified with the help of Microsoft Excel. Then, the IBM SPSS 21 software (Statistical Package for Social Sciences) results showed that the values initially found were within the acceptable range according to the literature.

We also looked at the outliers, that is, values above the top threshold or below the bottom threshold. The cases appointed for each item were excluded and replaced by the mean when factor analysis was applied. These limits are indicated by SPSS 21 in the form of box-plot graphs as outliers, which, according to Kline (2004), need to be evaluated before conducting the analysis.

One of the possibilities recommended by the literature is exclusion. We found that each item had a different number of outliers. So, for all items to have the same number of respondents, the excluded items were replaced by the mean, which is a widely used estimation method. Because each item presented few cases of outliers, the substitution by the mean did not generate great distortions, according to the sample size, nor did it bias the subsequent analyses.

After that, the database was ready for exploratory factor analysis (EFA). The first collected data were analyzed by using all the items grouped altogether.

After defining such parameters, the suitability of the sample was checked by using the KMO test (0.868). Subsequently, Cronbach's alpha (0.913) was extracted as a way to verify the reliability of the questionnaire used in the research. Thus, the SPSS results were confronted with the items' exclusion criteria adopted in this research, according to Table 4.

According to the reference thresholds of the previous table, we found that some items needed to be excluded to improve the instrument for the second round of data collection. A total of seven items were excluded (IT-4, AV-2, IN-1, BS-6, AV-5, BS-4 and IT-2). It is worth mentioning that after each item was excluded, in the order presented above, the remaining items were processed again on the SPSS. Then, the model showed a total explained variance of 61.857% for five dimensions.

The commonalities, factor loadings, KMO (0.852) and Cronbach's alpha (0.893) were found to present acceptable values, according to the parameters set for this research. Eventually, we had a leaner instrument, theoretically capable of measuring the typologies of co-production of public services. However, Costa (2011) states that after new field activities are conducted, the scale needs to be cleaned again for improvement.

As stated earlier, the EFA suggested a more consistent and suitable five-dimension model. The factor of the initial type of co-production was divided in two to create the preliminary dimension, which precedes the initial. The new factor corresponds to the first stage in the scale of the typologies of co-production of public services. At this level, citizens' involvement with the delivery of public services is nearly symbolic, since it is not effective and it is limited to monitoring and agreeing with the actions, with no power of intervention.

Then, items IN-2, IN-3, IN-4 and IN-5 were grouped into the preliminary factor and items IN-6, IN-7 and IN-8 remained in the initial dimension. In turn, items IT-6 and IT-7 were grouped with the items of the advanced type of co-production. This could happen because it is an instrument that seeks to measure a scale of citizen involvement in co-production activities. Since the two items were in the transition between the intermediate and advanced factors, the factor loadings were higher in the last dimension and they were renamed as AV-7 and AV-8, respectively.

In addition, items were created in the preliminary, initial and intermediate factors, for the dimensions of the instrument for the second round of data collection to have an equal number of items. So, items IN-2, IN-3, IN-4 and IN-5 were renamed as PR-1, PR-2, PR-3 and PR-4, respectively, and were joined by a new item, PR-5, regarding the preliminary dimension. Items IN-9 and IN-10 were added to the initial dimension while items IT-8 and IT-9 were added to the intermediate dimension. After these adjustments, we had the instrument that was applied in the second round of data collection, as shown in Table 5.

After all, the final instrument has 26 items divided into five dimensions. Like the first questionnaire, this one was also created as a Google Form and was sent out to the general public from 04 to 21 January 2021 via the WhatsApp instant messaging application.

On that occasion, 301 valid answers were obtained, which satisfied Costa (2011), in that the number of answers should be at least ten times as big as the number of items in the questionnaire. The data collected in the second round were analyzed according to the same procedures of the first round, which closed the confirmatory factor analysis.

First, the results were found to be within the reference range for this research. Next, the outliers were excluded and replaced by the mean for the factor analysis to be run. Then, after the exploratory steps, the confirmatory factor analysis (CFA) could be run.

The data from the second round of collection were analyzed with all items grouped on the IBM SPSS software. The adequacy of the sample was verified through the KMO test (0.912), and the Cronbach's alpha (0.911) was extracted, which attests to the reliability of the questionnaire applied in this research.

Next, the results from the SPSS were compared to the criteria for excluding items adopted in this research, which were previously explained. Primarily, it was identified that some items needed to be excluded. Seven items were excluded in the following order: PR-1, AV-1, AV-8, BS-7, IT-8, IT-9 and BS-1; each time one item was excluded, the data of the remaining items were processed again on the SPSS.

Then, after seven exclusions, the model showed a total explained variance of 73.753% for five dimensions. After the proper procedures of analysis and cleaning of the scale, it was found that the commonalities, factor loadings, KMO (0.918) and Cronbach's alpha (0.910) presented acceptable values, except for the commonality observed in item PR-4, which needed to be rounded up (0.500) to adapt to the reference range established for this research.

With this, we reached the final standardized solution of the CFA, defined by the causal relationships. According to Costa (2011), the following step is to analyze the validity and reliability of the final scale. This is an important stage in the creation and development of scales since it will reveal whether the instrument is valid and reliable.

Next, we checked the convergent and discriminant validities of the measurement model by using the criterion proposed by Steenkamp and Van Trijp (1991), who state that the convergent validity of a construct can be verified by examining the factor loadings of the indicators in the latent variable. The construct has convergent validity when the factor loadings are strong (> 0.50) and significant (C.R. Value > t_critical,a).

Convergent validity was verified for the “intermediate”, “initial”, “advanced”, “basic” and “preliminary” dimensions, since all the indicator variables presented significant and strong factor loadings, according to the definitions of Hair, Black, Babin, Anderson & Tatham (2009). Composite reliability is confirmed with values over 0.70; in turn, the extracted variance is confirmed with values over 0.50.

This way, the five factors were found to be reliable, considering that their respective composite reliability indicators presented values over 0.70. In addition, an extracted variance over 0.50 was observed for each dimension, which means that the convergent validity for the factors proposed in this study was met with no need for adjustments.

Next, we dealt with the discriminant validity of the scale, which verifies the difference between one factor and the others. To do this, the criterion used was that suggested by Fornell and Larcker (1981), which consists in comparing the variance extracted of the factor with the shared variance (the squared multiple correlation, R2) of the other factors. Discriminant validity is set if the variance extracted for the construct (diagonal in Table 6) is greater than the shared variance.

All the variances extracted are greater than the variances shared by the dimensions. Therefore, the factors are said to have discriminant validity. Thus, validity (both convergent and discriminant) and reliability can be confered on the final scale.

Next, the indicators obtained were compared to the reference range recommended by the literature, specifically with regard to model fit, as illustrated in Table 7.

In general, the indices presented values within those set for the research, according to Table 6. The exception was the relative fit indicators - TLI (0.666) and CFI (0.729) – which showed values close to those suggested in the literature.

However, this did not affect the acceptance of the final model, since, according to Marôco (2010), the reference thresholds of the fit indices are general guidelines rather than final rules. In addition, Hu and Bentler (1998) highlighted those aspects such as adequacy and interpretability of parameter estimates, and model complexity can be critical in deciding on the validity of a model.

Then, taking all these issues into account, the results are considered acceptable and, therefore, the model was accepted. This indicates that the items analyzed form a scale that is fit to measure the typology of co-production of public services.

Once the necessary adjustments were made, a reliable instrument with 19 items was obtained, divided among five factors: “preliminary”, “initial”, “basic”, “intermediate” and “advanced”, as shown in Table 8.

The “preliminary" factor is meant to measure actions that characterize the first level of citizen involvement with the delivery of public services. It is rather symbolic, with little effectiveness and power of intervention. Citizens are informed and consulted on the implementation of a given public policy, and interests may even compromise but without community participation in the decision-making process.

In turn, the “initial” factor, idealized by Alford (1998) and Pestoff (2006), tries to measure the degree of consultation with the citizens regarding the provision of information, showing tolerance or consent towards possible actions of the State in the delivery of public services. Unlike the "preliminary" dimension, here citizens can start to become aware of their roles in society, as transformative agents, by engaging in co-production activities in partnership with the State.

Then, the “basic” factor, created by Etgar (2008), is meant to measure how much citizens can exert, through councils, a certain influence over public agents’ decisions, even though decisions remain in the hands of State agents. The basic typology advances in relation to the previous dimension by guaranteeing quotas or places for citizens in specific groups of society. There is some appreciation and encouragement for the involvement of citizens in the delivery of public services.

The “intermediate” factor, advocated by Brandsen and Pestoff (2006), verifies the power shared by the State and citizens towards delivering more efficient and effective public services. Here the decisions are split and, according to Alford (2002) citizens are seen as partners of the State and they get and offer resources for the delivery of public services.

Finally, the "advanced" factor tries to measure citizens' degree of empowerment to meet the interests of the community. According to the ideas of Cooper & Kathi (2005), the entire community is meant to get involved to keep society mobilized. Citizens can make contact with external institutions to obtain the resources and technical advice they need but they still control the use of resources.

This way, we concluded the construction of the scale of typologies of co-production by delivering a consistent measurement instrument, which can be seen as an advance in the analysis and verification tools on the theme studied herein. Measurement is important in the context of knowledge production, as it serves professional and academic purposes, and it aims to clarify the observed phenomenon.

5 FINAL REMARKS

The objective of this article was achieved: to create a scale of typologies of co-production of public services. The process of creating the scale was complex and followed the steps suggested by Costa (2011). We started off from a theoretical model with 28 items divided into four dimensions and ended up with an instrument consisting of 19 items divided into five dimensions. This is an unprecedented scale with great potential for contribution to studies in co-production, which predominantly relies on qualitative research and case studies.

The final instrument has been tested and empirically validated. Managers interested in adopting co-production as a management strategy can apply this instrument to a group of citizens and obtain, as a result, the type of co-production that is appropriate for that community. Then, they will be able to create a more precise public policy, with regard to the engagement of citizens in the delivery of the proposed services. This can generate benefits that exceed the economic aspect, especially in the promotion of citizenship.

In addition to its empirical contributions, this research also raised theoretical implications by filling the gaps pointed out in the literature, which relates two or more of the “concepts” presented herein, among which are typologies (levels and forms). Therefore, this research may be useful in future studies on co-production of public services.

As to recommendations for future studies, we suggest conducting a more in-depth qualitative analysis of the items excluded during the process of constructing the instrument for measuring typologies of co-production. We also suggest applying the resulting instrument to a selected group of citizens or a specific locality. These studies are believed to be able to contribute with necessary adjustments to the presented scale.

A limitation of this research lies in the time factor and the samples obtained in the two rounds of data collection, considering that data were collected exclusively online with the help of Google Forms and the WhatsApp instant messaging application. Respondents did not necessarily belong to a specific group or area, and the questionnaire was applied to random citizens.

Therefore, we did not intend to exhaust the subject or present a definitive answer to the research problem. This study begins a discussion, which deserves qualitative deepening by bringing to light the historical, economic, political, social and cultural factors that have shaped Brazilian society over the years. It should be an investigation that takes into account the entire context determining the process of shaping national citizenship.

REFERENCES

Age, L. M., & Schommer, P. C. (2017). Coprodução de serviço de vigilância sanitária: Certificação e classificação de restaurantes. Revista de Administração Contemporânea, 21(3), 413-434. https://doi.org/10.1590/1982-7849rac2017170026.

Alford, J. (1998). A public management road less travelled: Clients as co-producers of public services. Australian Journal of Public Administration, 57, 128-137. https://doi.org/10.1111/j.1467-8500.1998.tb01568.x

Alford, J. (2002). Why do public-sector clients coproduce? Toward a contingency theory. Administration & Society, 34(1), 32-56. https://doi.org/10.1177/0095399702034001

Bezerra, J. C. V., Cavalcante, C. E., & Gouveia, A., Júnior. (2022). Motivação para coprodução do bem público: Proposta de uma escala. Administração Pública e Gestão Social, 14(1), 1-18. https://doi.org/10.21118/apgs.v14i1.11607

Bovaird, T. (2007). Beyond Engagement and participation: User and community coproduction of public services. Public Administration Review, 67, 846-860. https://doi.org/10.1111/j.1540-6210.2007.00773.x

Brandsen, T., & Honingh, M. (2016). Distinguishing different types of coproduction: A conceptual analysis based on the classical definitions. Public Administration Review, 76, 427-435. https://doi.org/10.1111/puar.12465

Brandsen, T., & Pestoff, V. (2006). Co-production, the third sector and the delivery of public services. Public Management Review, 8(4), 493-501. https://doi.org/10.1080/14719030601022874

Brudney, J. L., & England, R. E. (1983). Toward a definition of the coproduction concept. Public Administration Review, 43, 59-65. https://doi.org/10.2307/975300

Chaebo, G., & Medeiros, J. (2017). Participação direta em coprodução é democrática? Controle vetorial da dengue em Campo Grande – MS. Cadernos Gestão Pública e Cidadania, 22(71), 1-20. https://doi.org/10.12660/cgpc.v22n71.61067

Cooper, T. L., & Kathi, P. C. (2005). Neighborhood councils and city agencies: A model of collaborative coproduction. National Civic Review, 94, 43-53. https://doi.org/10.1002/ncr.82

Costa, F. J. (2011). Mensuração e Desenvolvimento de Escalas. Rio de Janeiro: Editora Ciência Moderna.

Dos-Reis, M. C. A., & Isidro-Filho, A. (2019). Inovação em serviços e a coprodução no setor público federal brasileiro. Administração Pública e Gestão Social, 12(1). https://doi.org/10.21118/apgs.v12i1.5481

Etgar, M. (2007). A descriptive model of the consumer co-production process. Journal of the Academy of Marketing Science, 36, 97-108. https://doi.org/10.1007/s11747-007-0061-1

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39-50. https://doi.org/10.2307/3151312

Gouveia, A., Júnior, Bezerra, J. C. V., & Cavalcante, C. E. (2023). Tipologias de coprodução do bem público: Estado da arte e agenda de pesquisa. Gestão & Regionalidade, 39(116). https://doi.org/10.13037/gr.vol39n116.7667

Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2009). Análise multivariada de dados. (6a ed.). Porto Alegre: Bookman Editora.

Hu, L.-t., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424-453. https://doi.org/10.1037/1082-989X.3.4.424

Kline, R. B. (2004). Principles and practice of structural equation modeling (2nd ed.). New York: The Guilford Press.

Lotta, G. S. (2017). Redes e parcerias em políticas sociais: Novos arranjos institucionais de coprodução de serviços nos municípios brasileiros. Brasília, DF: Enap.

Marôco, J (2010). Análise de equações estruturais: Fundamentos teóricos, software & aplicações. Report number, Pêro Pinheiro.

Nabatchi, T., Sancino, A., & Sicilia, M. (2017). Varieties of participation in public services: The who, when, and what of coproduction. Public Admin. Rev., 77, 766-776. https://doi.org/10.1111/puar.12765

Osborne, S. P., Radnor, Z., & Strokosch, K. (2016). Co-production and the co-creation of value in public services: a suitable case for treatment? Public Management Review, 18(5), 639-653. https://doi.org/10.1080/14719037.2015.1111927

Ostrom, E. (1996). Crossing the great divide: Coproduction, synergy, and development. World Development, 24(6), 1073-1087. https://doi.org/10.1016/0305-750X(96)00023-x

Parks, R. B., Baker, P. C., Kiser, L., Oakerson, R., Ostrom, E., Ostrom, V., Percy, S. L., Vandivort, M. B., Whitaker, G. P., & Wilson, R. (1981). Consumers as coproducers of public services: Some economic and institutional considerations. Policy Studies Journal, 9, 1001-1011. https://doi.org/10.1111/j.1541-0072.1981.tb01208.x

Pestoff, V. (2006). Citizens and co-production of welfare services. Public Management Review, 8(4), 503-519, https://doi.org/10.1080/14719030601022882

Pestoff, V. (2012). Co-production and third sector social services in Europe – Some crucial conceptual issues. In V. Pestoff, T. Brandsen & B. Verschuere (Eds.), New public governance, the third sector and co-production (Chap.2). London: Routledge. https://doi.org/10.1007/s11266-012-9308-7

Pestoff, V., Brandsen, T., & Verschuere, B. (Eds.). (2011). New public governance, the third sector, and co-production (1st ed.). Routledge. https://doi.org/10.4324/9780203152294

Salm, J. F. (2014). Coprodução de bens e serviços públicos. In: R. F. Boullosa (Org.). Dicionário para a formação em gestão social (pp. 42-44). Salvador: CIAGS/UFBA.

Salm, J. F., & Menegasso, M. E. (2010). Proposta de modelos para a coprodução do bem público a partir das tipologias de participação. Anais do XXXIV Encontro Científico de Administração da Associação Nacional de Pós-Graduação e Pesquisa em Administração - ANPAD. Rio de Janeiro, Brasil, 34.

Schommer, P. C, Andion, C. M., Pinheiro, D. M., Spaniol, E. L., & Serafim, M. (2011). Coprodução e inovação social na esfera pública em debate no campo da gestão social. In P. C. Schommer & R. F. Boullosa (Orgs.), Gestão social como caminho para a redefinição da esfera pública (pp. 31-70). Florianópolis: Udesc Editora.

Schommer, P. C., & Tavares, A. O. (2017). Gestão social e coprodução de serviços públicos. Fortaleza: Fundação Demócrito Rocha/UANE/BID/STDS-CE.

Steenkamp, J. B. E., & Van Trijp, H. (1991). The use of lisrel in validating marketing constructs. International Journal of Research in Marketing, 8, 283-299. https://doi.org/10.1016/0167-8116(91)90027-5

Strokosch, K. (2013). Co-production and innovation in public services: can co-production drive innovation? In S. P. Osborne & L. Brown (Eds.), The Handbook of Innovation in Public Services(pp. 375-389). Massachussets, USA: Edward Elgar Publishing Limited. https://doi.org/10.5902/1983465963947

Vanleene, D., Verschuere B., & Voets, J. (2015). The democratic quality of coproduction: A theoretical review and initial research design. Paper presented at the EGPA Conference, Toulouse, France.

Verschuere, B., Brandsen, T., & Pestoff, V. (2012). Co-production: The state of the art in research and the future agenda. Voluntas, 23(4), 1083-1101. https://doi.org/10.1007/s11266-012-9307-8

Whitaker, G. P. (1980). Coproduction: Citizen participation in service delivery. Public Administration Rev., 40(3), 240-246. https://doi.org/10.2307/975377