Local Government Performance: Evaluating Efficiency, Efficacy, and Effectiveness at the Basic Education Level

Marco Antonio Catussi Paschoalotto

Universidade do Oeste Paulista, Brazil

João Luiz Passador

Universidade de São Paulo, Brazil

Cláudia Souza Passador

Universidade de São Paulo, Brazil

Pedro Henrique de Oliveira

Universidade de São Paulo, Brazil

According to the World Bank (2018), in contrast to any other policy, education is extremely powerful, as it improves freedom and benefits, as well as provides a better life to the members of a society. Although over recent years access to education has increased, there is seen a significant worldwide disparity in its delivery and effectiveness, mainly due to poverty, sociodemographic conditions, and disability (World Bank, 2018). In the search for solutions to such problems, it is necessary to improve elementary schooling, the outcomes and environment for education, along with its equality (United Nations Children’s Fund [UNICEF], 2017).

Therefore, for governments to achieve social improvement goals set through improved education, improvements to the efficiency, efficacy, and effectiveness of national educational systems are necessary (Rosser & Sulistiyanto, 2013; UNICEF, 2017; World Bank, 2018). In our case, Brazil has a socially unequal basic education system, according to Organization for Economic Co-operation and Development (OECD) (2018), with a low level of expenditure per student against OECD countries, with different salaries paid to teachers and number of students per state, as well as one of the highest disparities in income found among OECD countries and partners.

The main challenge facing the Brazilian basic education system is how policymakers and practitioners can improve the quality and impartiality concerning outcomes through better evaluations and governance (OECD, 2015). The problem in Brazil exacerbates further, as shown in the state of São Paulo: despite being the richest state in Brazil (Instituto Brasileiro de Geografia e Estatística [IBGE] 2010; Campoli, Ferraz, & Rebelatto 2019), it has the title of being one of the least efficient states in Brazil in terms of education. While spending more money than any other, it still demonstrates the same or even worse results than other poorer states.

In the Brazilian context, according to the Federal Constitution (Brazil, 1988), municipalities are responsible for executing the educational policies from nursery until the first part of the elementary school - grade five. Nevertheless, this decentralized model has caused some problems due to the incapacity on the part of municipalities to attend to educational demands (Fabrino, Valle, & Gomes, 2014). Consequently, there is an academic and practical gap in the public management performance area, at the local government performance level, when studying efficiency, efficacy, and effectiveness on a road map conception, starting from efficiency, passing to efficacy, and finishing with effectiveness at the basic education public school on a subnational level (Avellaneda & Gomes, 2015; Olvera & Avellaneda, 2019).

To collaborate in finding solutions to the gaps and problems mentioned above, our research question is ‘How socioeconomic conditions influence local government performance (efficiency, efficacy, and effectiveness), at the basic education level?’ To answer this question, we have analyzed the influence of socioeconomic conditions on local government performance of the 3Es (efficiency, efficacy, and effectiveness), at the basic education level. Therefore, through use of our study, we are stimulating advances concerning public administration studies, mainly in area of local government performance theory, by use of the 3Es analysis, while at the same time covering two periods of political mandates.

Public management performance, at local levels, has discussed the influence of sociodemographic, economic, and cultural factors concerning efficiency, efficacy, and effectiveness in municipalities (Avellaneda & Gomes, 2015; Puppim, 2017). The debate has advanced to the performance in central social policies, linked to these factors, where one finds studies based on educational efficiency, efficacy, and effectiveness (Olvera & Avellaneda, 2019).

The study by Theunissen, Bosma, Verdonk, and Feron (2015), based on contributing factors in primary schools in the Netherlands, proved that sociodemographic factors are influencers in the dropout rate and efficiency associated with public schools. In support of the study by Theunissen et al. (2015), Bastos, Bottan, and Cristia (2016) discuss how the expansion in primary schools can collaborate with education coverage, under the condition that there exists better quality in the action taken by the school. Along these lines, Carnoy et al. (2017) also debate the influence of sociodemographic conditions on the effectiveness of schools in Brazil, where each state has its own profile and route toward achieving a more effective education.

H1: The sociodemographic and economic conditions have a positive impact on an efficiency model at the basic education level in the Brazilian municipalities (Carnoy et al., 2017).

On the other hand, a study developed in Ghana, a developing country, supports the idea that there is no correlation between sociodemographic and economic characteristics when dealing with educational performance in terms of efficacy and effectiveness (Amankwaa, Agyemang-Dankwah, & Boateng, 2015). Additionally, Marin, Peuker, and Kessler (2019) include psychological factors, aggregated with sociodemographic conditions, in student performance, such as alcohol consumption and pattern behaviors.

The study by Birchler and Michaelowa (2016), complementing the previous studies, discusses the relationship between economic conditions and enrolment in public primary schools, which represents a connection between a higher financial profile and a more efficient school. Likewise, the economic and efficiency indicators can impact upon the competencies developed during the school period, which results in an improvement in the efficacy and effectiveness of the school (Aesaert et al., 2015).

In addition, according to Pholphirul (2017), there is a positive relation between social status and enrollments in pre-primary education, which later affect the effectiveness of these students in reading skills, science, and mathematics. However, it is essential to emphasize equity strategies in schools, working together in the improvement of efficiency and effectiveness (Lincove, 2006).

Furthermore, studies from all over the world validate the connection between efficiency, efficacy, and effectiveness. In Denmark, Norway, and Sweden, researchers have shown that the national educational systems have improved their efficiency through technologies and resources (Imsen, Blossing, & Moos, 2017; Yoshida & Van der Walt, 2017). In Tunisia, research discussed a possible non-correlation between better resources and student performance, even though it pointed to poverty as a factor of inefficiency (Ramzi, Afonso, & Ayadi 2016). It is also worth mentioning the movement in South Africa, where more efficient municipalities in public education have not provided a more effective society (Monkam, 2014).

In the case of Brazil, Diaz (2012) determines that spending more resources in public schools does not guarantee achievements, as shown through a multilevel analysis study in Brazilian municipalities; in other words, more economic resources do not necessarily lead to an improvement in public education. Moreover, Rocha, Oliveira, Duarte, Gadelha, and Pereira (2017) arrived at the same results in another study based on public schools in Brazilian municipalities, where using more resources without better management practices does not generate a more effective and efficient system. Finally, Lauro, Figueiredo, and Wanke (2016) represent efficiency as vital for socioeconomics and management in terms of improving student performance in public schools.

H2: Socioeconomic conditions and an efficient model can introduce a more efficacious basic level of education to Brazilian municipalities (Lauro et al., 2016).

According to Si and Qiao (2017), contributions linked to structure and number of students per teacher can have a positive impact on performance at the basic education level. However, these contributions need to be administered correctly on a public administration level, in order not to cause under-investment, missed training teaching programs, and inequality in school infrastructure (Lu, Li, & Wu, 2015). Therefore, school efficiency in primary education provides favorable conditions for increasing the achievement of the students, through good management (Si & Qiao, 2017).

H3: Socioeconomic conditions, with an efficient and efficacious model, can provide more effectiveness at the basic education level in Brazilian municipalities (Si and Qiao, 2017).

To reach an effective public education system, it is necessary to administrate all resources with a multi-perspective view, through a social profile and economic and social policies (Adu-Gyamfi, 2014). Through a more profound investigation, Ramos et al. (2018) concluded that efficiency and effectiveness were seen as non-motivational aspects by teachers, but fundamental toward developing a good job. It is important to underline the influence of management in primary school performance, as it has the potential to shift the actual status quo to a higher level (Driessen, Agirdag, & Merry, 2016).

Subsequently, understanding which variables influence effectiveness in public schools is essential to the academic world, as is the social and economic status of the family of the pupil (Groot-Reuvekamp, Ros, van Boxtel, & Oort, 2017; Martini & Fabbris, 2017). In corroboration with this discussion, Saminathen, Låftman, Almquist, and Modin (2018) and Yogo (2017) showed that the effectiveness of public education is dependent on characteristics of the parents, as well as on the access society has in participating in school decisions.

In Finland, recognized for its educational system through the implementation of the Basic Education Act, the government has improved efficacy in distributing resources between schools and, consequently, effectiveness (Pulkkinen & Jahnukainen, 2016). The study of Craigwell, Bynoe, and Lowe (2012) also demonstrated the impact of the Caribbean government and its capacity to change national education. In Brazil, a country with education inequality (Campoli et al., 2019), municipality expenditures in education can improve the effectiveness in public schools, and as a result, enhance education across the country (Fabrino et al., 2014).

H4: There is education inequality between Brazilian municipalities among the socioeconomic conditions of efficiency, efficacy, and effectiveness, at the basic level of education (Campoli et al., 2019).

H5: There is a positive and significant correlation between the sociodemographic conditions and efficacy, and between sociodemographic conditions and effectiveness, at the basic level of education in Brazilian municipalities (Fabrino et al., 2014).

Hence, after discussions concerning local government performance of public education and the points of influence (Adu-Gyamfi, 2014; Aesaert et al., 2015; Carnoy et al., 2017; Olvera & Avellaneda, 2019), we can create a conceptual model to test Brazilian local governments at the basic education level, where sociodemographic and economic conditions could play a role in impacting the efficiency, efficacy, and effectiveness of Brazilian municipalities (Garcia, Prearo, Romero, Secco, & Bassi, 2016; Lauro, Figueiredo, & Wanke, 2016; Matias, Quaglio, Oliveira, Lima, & Bertolin, 2018; Theunissen, Bosma, Verdonk, & Feron, 2015; Vinha, Karino, & Laros, 2016).

Figure 1.
Hypotheses structure

Finally, in our paper, we are using the following concepts of efficiency, efficacy, and effectiveness (Garcia et al., 2016; Matias et al., 2018; Vinha et al., 2016): (a) Efficiency also means cost-benefit, through which we can produce more with fewer resources - in our case, enrollments, students, and time; (b) Efficacy expresses the achievement of goals - in our case, the approval rate; (c) Effectiveness demonstrates a higher impact concerning municipality performance - in our case, basic education index.

In order to provide a better comprehension, we divided our study by section according to the hypothesis, as public education inequality, linked to hypothesis 4, better condition - better effectiveness, linked to hypothesis 5, and socioeconomic conditions, efficiency, and effectiveness issues, related to hypotheses 1, 2, and 3.

Public education inequality (hypothesis 4)

As noted in Table 2, the municipalities in the state of São Paulo, Brazil, are heterogeneous concerning economic conditions and efficiency. The standard deviations attest to the affirmation - GDP per capita (13,468.06), average student per teacher (4.12), average for period duration (5.68), and approval rate (0.51). Another negative aspect is the average approval rate (5.06), remaining close to the average value of evaluations, between 0 and 10.

Table 2
Descriptive statistics

Nevertheless, a high mean is seen for the two socioeconomic indicators, two efficiency indicators, and the effectiveness indicators. The GDP per capita (21,610.92), the energy consumption ratio (0.83), the number of students per teacher (8.62), and the average duration period (14.68) are higher than the national mean (IBGE 2010; Inep, 2019). The average of the effectiveness indicator Ideb-AI (5.68) is above the Ideb-AI national goal of 3.8; 4.2 and 4.5. (Inep, 2019).

Therefore, through the above data, the authors support hypothesis 4: there is a public education inequality in the municipalities of São Paulo State (Campoli et al., 2019). With these results, we can discuss the extent of the disparity in the socioeconomic conditions and the public education at the basic education level in municipalities of the São Paulo State, even after having shown to have the second best score on the human development index among Brazilian states (Campoli et al., 2019; IBGE 2010).

Better conditions - Better effectiveness (hypothesis 5)

As shown on Table 3, there is a positive and significant correlation between the sociodemographic indicators of efficiency (0.152**; 0.133**; 0.089**; 0.096**; and 0.129**), efficacy (0.079**), and effectiveness (0.102**). At the same time, there is a negative and significant correlation between population and efficacy (-0.076**), and between energy consumption ratio and efficiency (-0.133**) and effectiveness (-0.164**).

Table 3
Correlation matrix

Note.** Correlation is significant at the level 0.01.* Correlation is significant at the level 0.05.

The economic condition has a positive and significant correlation with efficacy (0.085**) and effectiveness (0.103**), but a negative and significant correlation with efficiency (-0.105**). Regarding the main dimensions, one can detect a positive and significant correlation between average numbers of students per teacher and efficacy (0.126**), as well as a negative and significant correlation between average duration of school period and efficacy (-0.075**). Similarly, the same correlation standards can be seen between the efficiency indicators and effectiveness (0.483** and -0.097**, respectively). Finally, there is a positive and significant correlation between efficacy and effectiveness (0.222**).

With a more in-depth search, we can highlight that:

1. 1. The high positive and significant correlation between the number of students per teacher and the basic education index (0.483**), as well as between the approval rate and the basic education index (0.222**) (Lu et al., 2015). The first analysis relating students per teacher to the basic education index may occur due to the economic capacity of bigger cities, which have bigger classes, but also better conditions to study (Lauro et al., 2016).
2. 2. The positive and negative, both are significant in the relationship between socioeconomic conditions and efficiency, efficacy, and effectiveness (Lauro et al., 2016).

As a result, we can partially support hypothesis 5, which has a positive and significant correlation between efficacy/effectiveness and sociodemographic conditions (Fabrino et al., 2014), due to the contrary and significant correlation with the population (-0.076**) and the energy consumption ratio (-0.164**).

These points allow for the discussion and the support of the possibility that school resources and performance can influence the effectiveness of a given school within that municipality (Lauro et al., 2016; Lu et al., 2015; Si & Qiao, 2017).

Socioeconomic conditions over efficiency and effectiveness (hypotheses 1, 2, and 3)

The first point highlighted here is the values of R² and adjusted R² of the three regression models. The low values (0.330 and 0.280 - efficiency, 0.101 and 0.094 - efficacy, 0.305 and 0.299 - effectiveness, respectively) may also report a low capacity of the indicators, thus explaining the performance in public schools. However, here we are working with macro indicators, which justifies the acceptance of the low values for R² and adjusted R² (Allan, Mooney, & Ling, 2018).

Table 4
Standardized coefficients (beta) for the four equations

Note.*** p < .001.** p < .05. * p < .01.

The first model represents the attempt to explain the efficiency, represented by the average duration of the school period. The results of R² (0.330) and adjusted R² (0.280) show the low capacity of the socioeconomic indicators in explaining efficiency in the first elementary public schools. Hence, we can support partially the hypothesis that not all socioeconomic indicators have a positive effect on the efficiency of municipalities and consequently on Brazilian public schools (Carnoy et al., 2017).

We can also point out that the coefficients for the rate of urbanization (0.159***) and GDP per capita (0.121***) are positive and statistically significant at 0.001. In addition, the coefficient of the residential energy ratio is negative (-0.111**) and statistically significant at 0.05. So, an increase of 1 percent above the mean rate of urbanization (0.8532) would generate 0.15 percent of the rise in the average duration of school period (1.3 minutes). Similarly, the increase of 1 percent above the GPD per capita mean (R$ 216.10) causes 0.12 percent of the rise in the average duration of the school period (1.05 minutes). Therefore, a more prosperous municipality with a higher GDP and possibly a better urbanization capacity could improve school student hours, hence its efficiency (Carnoy et al., 2017; Imsen et al., 2017; Yoshida & Van der Walt, 2017).

In the second model, we achieved the lowest R² (0.101) and adjusted R² (0.094) over the three models, therefore hypothesis 2 (socioeconomic conditions with an efficiency model are able to provide a more efficacious performance) is shown as false (Amankwaa et al., 2015). However, we can highlight that the coefficients for the rate of urbanization (0.311***) and the coefficient of the average duration of the school period (0.098***) are positive and statistically significant at 0.001. In contrast, the coefficient of the residential energy ratio is negative (-0.376***) and statistically significant also at 0.001. Thus, an increase of 1 percent above the mean rate of urbanization (0.8532) would bring a 0.31 percent rise in the approval rate (0.01). Likewise, the increase of 1 percent above the average duration of the school period mean (8.4 minutes) generates 0.09 percent of growth in the approval rate (0.004). These individual results can advance in the local government performance studies, since the municipalities being more urbanized and more efficient, measured herein by school period, could in fact be more effective (Imsen et al., 2017; Lauro et al., 2016; Yoshida & Van der Walt, 2017).

Finally, the third model with R² of 0.305 and adjusted R² of 0.299 best explains the results. Thus, hypothesis 3 is partially supported: a more effective model can be provided by better socioeconomic, efficiency, and efficacy indicators at the primary education level (Si & Qiao, 2017). The third model shows a positive and statistically significant level at 0.001 for the rate of urbanization coefficients (0.221***), the average duration of the school period (0.139***), and the approval rate (0.403***), and at 0.05 for the GDP coefficient per capita (0.081**). There is also a negative and statistically significant level of 0.001 for the coefficient of residential energy ratio (-0.271***).

Moreover, the last multiple of linear regression shows that an increase of 1 percent above the mean rate of urbanization (0.8532), the average duration of the school period (8.4 minutes) and an approval rate of 0.05, brings an increase of 0.22 (0.012), 0.14 (0.007), and 0.40 (0.022) percent to the basic education index, respectively. The aforementioned results bring about the discussion about the importance of a more efficient and effective municipality, creating as such an increase in its effectiveness, and perhaps in society (Fabrino et al., 2014; Si & Qiao, 2017).

Therefore, with the three runs of regression analysis, we were able to partially support the influence of the socioeconomic conditions in public management performance at the local level, measured here through the level of basic education in municipalities (Puppim, 2017; Olvera & Avellaneda, 2019). A more significant influence was seen for the urbanization rate indicators in all the 3Es, while GDP per capita in efficiency and effective performance and the educational indicators (average duration and approval rate) were significant in the efficacy and effectiveness models (Carnoy et al., 2017; Si & Qiao, 2017).

Our research has five main findings. First, we have a public education inequality between the São Paulo State municipalities, with high values of standard deviation among GDP per capita, Average Stu/Tea (Pre.S), Average Dur. Per. (AI) and Approval Rate (AI). Second, we have shown a positive and significant correlation between better social conditions or schools and better performance in efficiency, efficacy, and effectiveness in municipalities of public schools in the state of São Paulo.

Third, some of the socioeconomic conditions have a positive and significant impact on efficiency, as the urbanization rate and GDP per capita indicators generate a greater duration of the elementary school period. Following this, our fourth finding is that greater urbanization and more time spent in schools have a positive and significant impact on the approval rate indicator, representing the efficacy dimension. The positive influence of the rate of urbanization in the efficiency, efficacy, and effectiveness at the basic education level can be found and discussed in the Brazilian educational studies, by comparing rural and urban areas. Finally, our last finding shows that social conditions, efficiency, and efficacy can provide a more effective public school system, represented by the positive and significant impact of the rate of urbanization, the average duration of the school period, and the approval rate with the basic education index.

The limitations of this study include the historical data series in Brazil (short time), the sample size, and the representativeness of the variables to explain the phenomenon. The data series problems in Brazil within social policies are conditions and not a limitation, like quality, discontinuation, and missing data in municipalities; however this affects the inclusion of new indicators, as enrollments and evasion rate. Additionally, to represent the dimension of ‘efficiency’ we used just one indicator, elementary school period, which could bias the analysis.

For future studies, it would be interesting to enlarge and divide the sample according to population ranges, as well as expanding the years of analysis, with at least one more year of examination. Moreover, other variables could appear to broaden the explication capacity of the model. Furthermore, to represent the dimension with more accuracy, the principal component regression analysis could be used, thus creating new factors with the corresponding indicators.

Therefore, our study has made significant advances in the literature concerning public management performance at a local level, particularly pertinent to the municipalities in the state of São Paulo, Brazil, embedded at the basic education level. The conclusion is that we can answer our main questions: ‘How socioeconomic conditions influence local government performance (efficiency, efficacy, and effectiveness), at the basic education level?’ A: Social conditions can have more influence in efficiency, efficacy, and effectiveness, while economic conditions can have a more significant relationship with efficiency of local governments.