The Good, the Bad, and the Disruptive of Performance-Based Financing on the Mozambican Health System: Results from a process evaluation

O Bom, o Mau e o Desequilíbrio do Financiamento Baseado em Desempenho no Sistema de Saúde de Moçambique: resultados de uma avaliação de processo

Lo bueno, lo malo y lo perturbador del financiamiento basado en el desempeño en el sistema de salud de Mozambique: resultados de una evaluación del proceso

This work is licensed under Creative Commons Attribution 4.0 International.

Performance-based financing (PBF) has characteristically been defined as financial incentives for health workers and facilities based on performance on pre-determined measures (Kalk; Paul; Grabosch, 2010; Bonfrer et al., 2014). However, a broader definition has been proposed which more accurately describes PBF as a reform that targets and interacts with the entire health system (Renmans et al., 2017; Meessen et al., 2017; Shroff; Bigdeli; Meissen, 2017; Fretheim et al., 2012; Renmans et al., 2016a). Through this lens, PBF is an approach that links health facilities and provider payments to the achievement of predefined services and quality measures, but also a mechanism that establishes greater institutional autonomy and governance structures in order to secure a division of functions (purchaser, provider, verifier) (Renmans et al., 2017). In practical terms, the PBF toolk its instruct that schemes be designed to include features that emphasize institutional autonomy, clarified roles (contracting –in), a focus on result-based planning, community participation, division of functions, intensified monitoring, and specific accountability arrangements (Renmans et al., 2017; Fritsche, 2014; Health, 2012; The Aidstar-Two Project, 2011; Tonnen; Van der Wal, 2012). Although every PBF scheme has different objectives and is implemented in varying contexts (Renmans et al., 2017; Witter et al., 2012; De Allegri et al., 2018) a majority incorporate most or all of the aforementioned features. However, PBF is still largely considered a program or intervention by the broader development community and policymakers. Because of this, among the 40+ PBF schemes globally, most remain in pilot or subnational program status rather than an institutionalized health sector reform (Bertone; Lagarde; Witter 2016; Bonfrer et al., 2014; Peerenboom et al., 2014; Basinga et al., 2011b; Meessen et al., 2017; Shroff; Bigdeli; Meissen, 2017).

Despite the recent surge of interesting PBF evalutations, the impact on reported outcomes varies immensely between programs and contexts, and we still know too little about the mechanisms that lead to changes in outcomes (Renmans et al., 2017; Soucat et al., 2017; Eijkenaar et al., 2013). The PBF enthusiasts have heralded the approach as the starting point for larger health sector reform, hypothesizing that PBF can enhance service delivery, health information systems, supply chain management, financial management, and governance (Meessen; Soucat; Sekabaraga, 2011; Meessen et al., 2017; Shroff; Bigdeli; Meissen, 2017; Paul; Renmans 2017). Skeptics argue that PBF can lead to perverse incentive structures and negative side-effects on non-targeted services and should not be viewed as a “magic bullet” to address all that ails poorly performing health systems (Ireland; Paul; Dujardin, 2011; Renmans et al., 2016b; Paul; Renmans, 2017).

This debate has inspired a growing body of literature exploring the results of PBF and how it operates (Witter et al., 2012; Turcotte-Tremblay et al., 2015; Renmans et al., 2016a; Dale, 2014; Paul; Renmans, 2017). Much of this evidence focuses on assessing the impact of PBF on health outputs and outcomes (Basinga et al., 2011a; Borghi et al., 2015; Kambala et al., 2017; Binyaruka et al., 2015; Engineer et al., 2016; Matsuoka et al., 2014; Rudasingwa; Soeters; Basenya, 2017; Rudasingwa; Soeters; Bossuyt, 2014; Skiles et al., 2015;Van de Poel et al., 2016; Rajkotia et al., 2017). While it is important to know whether an intervention has led to gains in healthcare utilization and health outcomes, we also need to understand how PBF processes and factors affect the entire health system, an area that remains understudied (Witter et al., 2013; Meessen et al., 2017; Renmans et al., 2017).While a handful of studies have attempted to explore the effect of PBF on key elements of the health system, such as service delivery, provider motivation, information systems, or access to essential medicines (Matsuoka et al., 2014; Huillery; Seban, 2014; Chimhutu; Lindkvist; Lange, 2014; Lohmann et al., 2017b), few studies have adopted a comprehensive approach to explore the effects of PBF on the health system.

Against this backdrop, this paper explores the wider health system effects of a PBF intervention in the two program regions of Gaza and Nampula. We gathered detailed information on health workers’ perceptions of the effects – both good and bad – of the PBF program across the health system. We used the framework developed by Witter et al. (2013) which adapted health system building blocks defined in the World Health Organization’s Health Systems Framework to better capture the interactions and changes PBF created at a health facility level in Mozambique (Organization; Witter et al., 2013). In so doing, this study contributes evidence on the perceived effects of PBF on the health system, focusing particularly on how and whythese changes occurred.

In Mozambique, the PBF implementer (Elizabeth Glazer Pediatric AIDS Foundation (EGPAF)), financed by the US President’s Emergency Plan for AIDS Relief (PEPFAR) through the Centers for Disease Control and Prevention (CDC), initiated a PBF scheme to accelerate the achievement of maternal and child health (MCH) and HIV-focuses health outcomes in two provinces, Nampula and Gaza. The program began with 19 health facilities in 2011 and was incrementally phased up to 138 health facilities as of mid-2015.

The program established separation of functions. The implementer served as the purchasing agent and fund-holder, while verification of quantity and quality of care is co-led by the Provincial Health Office (DPS) and implementer field staff through quarterly visits to each PBF facility (public government-managed workers).The package of services under the PBF scheme is linked to health facility performance across 21 clinical indicators including prevention of mother to child vertical transmission (PMTCT), pediatric HIV, adult HIV care and treatment, tuberculosis, and maternal and child health (MCH). The scheme also includes three quality checklists on hygiene and sanitation, maternal and child health (MCH), and HIV/AIDS. Each facility was mandated to establish a managerial committee in charge of planning and managing PBF funds with guidance from the managing NGO (implementer) and district health office (DPS).

Of the performance payments to facilities, 40% is allocated for reinvestment into the health facility per a pre-approved business plan, while 60% is distributed as salary top- ups to all health facility staff according to pre-determined criteria, including tenure at the facility, level of education, attendance, and staff performance review. On average, staff bonuses represent 15-25% of a monthly salary for health staff with the highest baseline salaries or those working in larger facilities, and 30-50% of the monthly salary for lower- level staff with low baseline salaries or for rural staff in small facilities.

To gain an in-depth understanding of the perceptions of health workers and facility managers of the effects of PBF, we used a qualitative research design using in-depth semi-structured interviews and focus group discussions. We triangulated the findings with quantitative verification data. We measured the effect that the PBF program had on the health system, namely service delivery, human resources for health, health financing and governance.

We defined these four health system pillars using Witter’s framework (Witter et al., 2013). Witter et al. (2013) posits that the interaction between incentives, institutional changes, accountability efforts, and supporting mechanisms for health workers impacts the organization and performance of the health system (Witter et al., 2013).The resulting framework articulates that all health system building blocks are likely affected by the introduction of PBF, but for simplicity modifies the six into four health system pillars: service delivery (including commodities), human resource, governance (including information systems) and health financing. More recently, the boarder definition and conceptual discourse posited by Renmans et al. (2017) and Soucat et al. (2017) these four of those categories in their definitions of PBF (Soucat et al., 2017; Renmans et al., 2017). The health workers were asked to describe changes they observed within the health facility since the introduction of PBF and to reflect on the positive, negative, or null effects of PBF across the four pillars. Data was collected at 23 health facilities in Nampula (12) and Gaza (11), 73% of which were primary health centers and 82% which were classified as rural or peri-urban. We selected a range of facilities based on size, geographic location (rural, peri-urban, urban), and facility performance data (verified indicator quantities and quality checklist scores). Each facility had participated in the PBF program for a minimum of 6 quarters (18 months).

All interviews were conducted and transcribed in Portuguese, then imported into ATLAS.ti Version 1.0.14 for thematic analysis. Interviews were coded line-by-line and analyzed by the two researchers to identify recurrent themes and variations across responses. Initial themes and sub-codes were organized based on our theoretical model (Figure 1).

Figure 1
Conceptual Results Framework for Mozambique’s PBF Health System Effects
Caption: Theoretical model adapted from Witter et al., 2013. Abbreviations: HIS, Health Information System; HRH, Human Resources for Health

Magnitude of effect is a proxy measure to estimate the perceived effect of each change and its attribution to PBF. Magnitude was estimated by the frequency of the code and a coding analysis exercise which assessed and scored each quotation by attribution of the change to PBF (positive and negative, and strength on a three-point scale), saliency and depth (explanation of the change, specificity of the results and self-perceived strength of the change compared to before PBF or in a facility without PBF), other explanations or causes of the observed change (other interventions, contextual changes). While we are aware of the concerns and publications on causality of qualitative work, this analytical technique was used to help organized the top perceived influences of PBF, what was observed and discussed the most by health providers and managers (Maxwell, 2004). We acknowledge that the perceived changes were influenced by PBF, but that contextual components, environmental readiness and other interventions contributed to the changes observed by respondents. Parsing apart attribution to PBF is not achievable with this particularly study design.

In each facility, health workers (nurses, medical technicians, and physicians), community health workers and facility administrators were sampled based on their exposure to PBF (minimum of nine months) and availability given we collected data through unannounced facility visits during normal clinic hours. Sixty health workers and facility managers were interviewed individually and a total of 30 health workers participated in focus group discussions (Table 1). Health workers typically participated in the PBF program for an average of 9.5 quarters (2.4 years). Semi-structured interview and group discussion guides were used during the interviews, allowing for open dialogue and discussion about the effects of PBF on health system components including service delivery, financing, governance, human resources, access to medicines, and information systems. Data was collected by two Mozambican researchers between November-December 2015.

Table 1
Profile of Study Participants

Abbreviations: Q, quarters, equivalent of 3 months.

Caption: The majority of respondents were medical technicians, who assume some managerial role, in addition to diagnosis, treatment, and counseling of all patients, but particularly HIV patients. The second most common respondents were nurses, falling within three professional designations. A head nurse is typically in charge of many of the managerial activities including the completion of the patient registry books. General nurses concentrate on providing wide range of care prescribed to patients by clinician. MNCH nurses focus on providing preventative and curative care to maternal and child patients, including antenatal care, assisting in deliveries, immediate post-partum care, and contraceptive counseling and prescription. The main function of the lay counselors or Activistas is to provide counseling and referral information for patients in their assigned communities, with special emphasis on HIV+ patients. Information included access points to community support groups (GAACs) and helping patients to find pharmacies to access ARVs. Facility management activities are shared between the facility-based cadre, particularly when the facility is small. The three facility managers listed in Table 1 had no clinical activities and were employed by peri-urban large hospitals.

Health workers cited feeling a greater sense of coordination in the care they were providing to patients, particularly maternal patients, mothers living with HIV, and infants born to mothers living with HIV. Integration or continuity of care was described as ensuring that patients who were already at the facility received all services or counseling they were eligible for, referred to by one health workers as ‘patient double dipping’.

Integration of PBF services for mothers without HIV included: antenatal care (ANC) visits, facility delivery, post-partum consultation, and family planning (FP) counseling, in addition to timely vaccination for the newborn within 9-12 months and nutritional counseling. For mothers living with HIV, PBF incentivized integrated care through the testing of their partner, antiretroviral (ARV) prophylaxis, antiretroviral therapy (ART) initiation during pregnancy, polymerase chain reaction (PCR) testing within four to eight weeks of birth, rapid HIV testing within nine to 12 months, ART initiation for newborns 0-23 months, and FP counseling after birth. The integration of care was an initiative discussed at monthly or quarterly PBF meetings.

The results of the impact evaluation show significant increases in patients receiving care for PBF indicators compared to control groups (magnitude of effect). Most notably for the services that facilities reported integrated services (Rajkotia et al., 2017).

Figure 4
Measuring Impact (% change from baseline) for PBF Indicators

Caption: Results are derived from the impact evaluation completed in 2014 (full results in Rajkotia et al., 2017). Three indicators were dropped from the impact evaluation, which are not displayed here. Magnitude scores were constructed by clustering the magnitude of average treatment effect estimates. A multi-period difference-in-differences (MDID) was applied on the matched sample to obtain average treatment effects

As more donors and country governments in LMICs start embracing PBF as an approach to boost performance in the health sector, there have been repeated calls for rigorous evaluations to grow the evidence base on PBF (Eldridge; Palmer, 2009;Witter et al., 2012; Renmans; Holvoet; Criel, 2017; Renmans et al., 2016b; Paul; Renmans, 2017).

While the body of research on PBF has grown significantly in the past few years, recent reviews of the literature have found that, firstly, the evidence about the impact of PBF programs on health system outputs and health outcomes is mixed and, secondly, a lot remains unknown about how PBF acts on the health system (Witter et al., 2012; Renmans et al., 2016a; Paul; Renmans, 2017). This study speaks to the second observation, using a PBF program in Mozambique to understand how PBF influences actors, institutions, and processes that comprise the health system. The approach we adopted of interviewing health workers to capture their perceptions is similar to what others have used to explore key aspects of PBF programs, such as the implementation process and the effect of PBF on staff motivation (Bhatnagar; George, 2016; Kalk; Paul; Grabosch, 2010). Our application of evaluating PBF across the entire health systems pillars, as defined by Witter et al. (2013), about the changes attributed to the PBF program is novel. While other articles have documented the perceived effects of PBF interventions on individual building blocks like service delivery, (Bonfrer; Van de Poel; Van Doorslaer, 2014; Chimhutu et al., 2015; Skiles et al., 2015; Flink et al., 2016; Kalk; Paul; Grabosch, 2010; Paul, 2009; Bhatnagar; George, 2016) human resources for health,(Matsuoka et al., 2014; Engineer et al., 2016; Lohmann et al., 2017a; Lohmann et al., 2018) and health information system, (Rusa et al., 2009; Kalk; Paul; Grabosch, 2010) no study to date has used a comprehensive health systems lens to systematically explore the perceived effects of PBF on all pillars.

The results from the study show that health workers reported PBF as having had good, bad and disruptive perceived effects on all pillars at variable magnitudes. Overall, the PBF scheme was perceived to have had the greatest effect in the areas of service delivery, health workforce, and information systems. It had more modest effects on health financing and governance, and no effect on the supply of pharmaceutical commodities (by design).

In the area of service delivery, health workers believed the PBF program has resulted in an expansion of service coverage, which is not surprising given that an impact evaluation for the PBF program in Mozambique found increases in a range of health outputs, (Rajkotia et al., 2017) and other evaluation of PBF pilots that showed similarly positive perceived effects on healthcare utilization (Basinga et al., 2011a; Bonfrer et al., 2014; Engineer et al., 2016). Health workers felt the PBF program had strengthened integration of care and the nature of the interaction between health workers and patients. This is similar to findings in Rwanda and Cambodia, where PBF schemes led to an improvement in the interaction between health workers and patients (Kalk; Paul; Grabosch, 2010; Khim; Annear, 2013; Skiles et al., 2015). What emerged most notably is how much health workers appreciation for the infrastructure improvements that health facilities have been able to undertake thanks to the PBF scheme. Health workers were largely unaware of the PBF quality components, including the checklists themselves, and the payment formula based on the facility’s quality score. Quality checklists also need to be more process driven, measuring multiple components of quality, including process and outcomes of care. Understanding advances in maternal and child health quality efforts may be a great source of cross-learning, especially because most PBF programs focus on quality of care in MCH (Gergen et al., 2017; Josephson et al., 2017).

In the domain of human resources for health, the PBF payments reportedly increased health worker motivation. This finding is in line with other studies that showed documented positive responses by health workers to the increase in their remuneration (Kalk; Paul; Grabosch, 2010; Bertone; Lagarde; Witter, 2016; Skiles et al., 2015; Lohmann et al., 2017b, Lohmann et al., 2017a). PBF was also credited for improving staff retention rates, as was the case in Cambodia (Matsuoka et al., 2014). However, health workers in Mozambique expressed concerns about the fairness of how the PBF funds were distributed amongst health workers, which echoes concerns by health workers in Tanzania (Chimhutu et al., 2016), Cambodia (Khim; Annear, 2013), DRC (Fox et al., 2014) and Benin (Paul; Sossouhounto; Eclou, 2014). Unlike other PBF programs, where the incentive to improve performance led managers to increase training and supportive supervision for health workers (Bhatnagar; George, 2016), study participants in Mozambique reported receiving limited clinical training and supportive supervision external to the PBF verification and managerial committee meetings. This finding reiterates the need to integrate broader supervision, such as supportive supervision mechanisms, within the PBF design to further enhance facility and provider performance, clinical adherence, patient-provider communication and respectful care. PBF did not change the composition within the facilities, which is likely due to the fact that distribution of health workers is decided by the provincial and central government based on catchment population estimates.

The PBF program strengthened the health information system at the facility-level. Similar positive effects of the PBF program on health information captured at the facility level have been reported in Burundi (Peerenboom et al., 2014). However, these improvements came at a cost. Faced with a trade-off between delivering more clinical services and spending time maintaining the register books, health workers reported shifting their focus from the former to the latter because of the PBF verification. There were also notes on discrepancies between verification and reported numbers. Health workers were not concerned about the discrepancies because although penalties were intended to be applied, no health facility ever received penalties during the program’s tenure. Instead the verified would note it and warn the health facility manager. This is similar to Rwanda, where Kalk; Paul; Grabosch (2010) found evidence of task-shifting towards record maintenance, and in Burkina Faso (Kalk; Paul; Grabosch, 2010; Turcotte-Tremblay et al., 2017)

Further, the PBF program was credited with improving some governance and managerial practices at the facility-level, especially frequent business planning meetings. However, the absence of monthly PBF performance review meetings and the perceived unfairness in incentive distribution made staff and team relations tense at some facilities. Similar to the experience in Cambodia, the PBF and operational cost disbursement were faster than the MoH budget endowment, ensuring timely disbursement of the incentives on the basis of their performance and strengthened capacity for financial management (Matsuoka et al., 2014; De Allegri et al., 2018). Although the governance structures and processes at the facility level were changed, they could be improved significantly using a more participatory, transparent approach (De Allegri et al., 2018; Seppey et al., 2017).

With respect to health financing, health workers credited the PBF program with increasing the fiscal autonomy. This resulted in greater planning at the facility-level on how the PBF funds would be reinvested into the facility (De Allegri et al., 2018). However, these decisions were made largely by the facility managers, with little input from the health workers. Most health workers interviewed were absent from the allocation and resource mobilization processes but keenly aware of the results of these investments, namely the structural and equipment changes. This concept of performance-based financing was broadly and shallowly understood by nearly all health workers, and the link between taking action and planning for increasing service in order to get a bigger bonus was not as clear. This is a core gap in the theory of change and operationalization of PBF, workers and managers both need to understand that performance is directly tied to payments in order to driven performance. Understanding the payment formula would be the first step, however there is significant heterogeneity in the payment formulas, as well as complexity in the calculations. The methodology adopted in this study of capturing information on the PBF health system perceived effects through semi-structured interviews with health workers has a number of limitations. First, respondents were selected in collaboration with facility management, which could have led to a bias sample of well-adjusted or positive health workers. Moreover, the sample only focused on the supply-side and did not capture beneficiary perceptions or validate any of the changes health providers attributed to PBF. Although the evaluation was conducted by an independent evaluator (ThinkWell), the implementer and occasionally DPS accompanied our data collectors to the facility to facilitate the introduction and coordination of the interviews. The interview guide was open-ended, but all health workers were probed on changes attributed to PBF for all pillars to ensure we were able to capture the perceived effects across the entire system but could have introduced biasin health worker responses.

The changes described demonstrate that PBF has the potential to go well beyond health worker motivation and enhance service delivery to broadly strengthen the health system. However, to achieve its potential, a number of things need to happen. First, program designers and implementers will need to think more critically about how a combination of PBF incentives and technical support can catalyze and institutionalize managerial governance changes at the facility level. Managerial changes and organizational culture should incorporate managerial autonomy to ensuring that the facilities are properly staffed, have adequate supplies, and completed registers. Moreover, program implementers should consider a more robust and active long-term sustainability strategy to institutionalize the program into provincial and national health offices while decreasing control by an external NGO. The Mozambican PBF program failed at numerous level of institutionalization and gaining buy-in from government and other actors, and subsequently the program was ended in 2017 (Gergen; Falcao; Rajkotia, 2018), which reflects the experience documented across many programs (Shroff; Bigdeli; Meissen, 2017; Seppey et al., 2017). Second, the program must be designed such that an increasing proportion of the health facility budget is output-based versus input-based to keep the focus on performance. Third, strengthening the participation of the supply-side could support the strength of PBF’s positive effects on the health system.

Disclaimer: The findings and conclusions in this report are those of the author(s) and do not necessarily represent the official position of the Centers for Disease Control and Prevention

Ethical Trial Registration: Ethical approval for the study protocol, data collection instruments and informed consent forms was obtained from the Ethics Review Committee of the Centers for Disease Control and Prevention (CDC) [IRB 2015-190], and the Ethics Review Committee of the Mozambique Ministry of Health.

Author Contributions: Conceived and designed study: JG, YR. Performed data collection: JG. Analyzed the data: JG, NR. Wrote the paper: JG, NR,YR.

Conflicts of Interests: None

We would also like to thank all health care providers, managers and community health advocates who took the time to speak with our team and helped to inform these findings.