When government wastes benefits from a project: The case of Line 3 of Light Rail in Guadalajara, Mexico

Esta obra está bajo unaLicencia Creative Commons Atribución-SinDerivar 4.0 Internacional.

Choosing the best quality projects, which offer the best value for society and impact economic growth, should be the fundamental objective of every public investment program (European Commission, 2014). In low- and middle-income countries, where resources are scarce, and needs are significant, a cost-benefit analysis (CBA) represents an essential tool for efficiency in the use of public funds (Robinson et al., 2019) because it allows valuing the project’s flows of benefits and costs along a time horizon, signaling if the project is profitable for society. The question is what happens when the CBA initially reveals that the project will produce positive net benefits to society, however, even before it begins to run, those benefits have already vanished?.

This question reveals that a promising CBA is a necessary, but insufficient condition for a successful public investment. As we know, invest[1]ment projects have a life cycle and CBA is just a part of the first stage (project identification and preparation). Ineffective political decisions may affect investments negatively during the subsequent stages: a) the project implementation, and b) the project’s operational life. This paper focuses on the project implementation stage because managers usually fail to control inputs and capital investments costs (Isham and Kaufmann, 1999), eroding the project’s profitability, as is the case analyzed here.

Although unanticipated cost risings are not uncommon in significant infrastructure and transport projects (Independent Evaluation Group, 2012), there is also a complementary problem in certain CBAs, where costs are systematically underestimated (Skamris-Holm and Flyvbjerg, 1997; Flyvbjerg et al., 2003a; Flyvbjerg et al., 2003b., De Rus-Mendoza et al., 2006; Van Wee, 2007; Flyvbjerg, 2009; Nicolaisen and Driscoll, 2014; Hickman and Dean, 2018), generating significant costs overruns during the construction phase. Unfortunately, these cost overruns are more frequent in developing countries due to weak governance institutions.

When the project arrives at its implementation phase, cost overruns are mainly due to failures in management and execution capacities. These failures result from a lack of strategic planning, monitoring, coordination, training, or commitment to the project (Ika et al., 2012; Lagarda et al., 2018). All these failures seem to be present in the case of Line 3 of Light Rail of Guadalajara (from now on called ‘Line 3’), whose nominal cost was duplicated (in domestic currency) and began to operate almost four years after the planned date of completion.

Reviewing the literature on light rail cases, it would be a mistake to dismiss the case of Line 3, considering that cost overruns are commonplace worldwide. Andric et al. (2019) analyzed 102 major infrastructure projects in Asia, finding that, in fact, there are cost overruns (mean = 26.2%) in a variety of projects, but they also found several cases of cost underrun (mean = 12.2%). The latter illustrates that projects can indeed aspire towards waste-minimization strategies, especially when one considers high opportunity costs in developing countries.

On the other hand, another issue to analyze in light rail projects is the lack of demand when projects operate. In many projects, the ridership analysis shows light rail to achieve considerably fewer passengers than forecast and thus lower benefits (Flyvbjerg et al., 2006; Flyvbjerg, 2007a and 2007b; Winston and Maheshri, 2007; Carpinteiro and Petersen, 2014; and Van der Bijl et al., 2018).

The general objective of this work is to show, through several scenarios, how applying simple modifications to the official CBA incorporating actual ex-post costs and benefits, we discover a significant deterioration in the project’s net social benefits, even before Line 3 began its operational life. This paper contributes to studying efficiencyin public investment in developing countries, where there is a lack of evidence. Particularly, there is very limited ex-post monitoring of transport projects and, consequently, there is too little learning from experience on the application of CBAs (Hickman and Dean, 2018). This paper is, to the best of our knowledge, the first ex-post analysis for a major transport project in Mexico and other Latin American countries. Docuenting and analyzing the failures in developing this kind of project will supply insights for better public investment decision-making in developing countries. For example, Aguiar et al. (2017) point out for the case of Brazil, the biggest economy in Latin America, that “it is not clear for Brazilian society which development projects had positive performances and which were considered feasible or infeasible.” This statement seems to apply to most Latin American countries, except Chile, where a National Investment System privileges CBA use.

According to this ex-post evaluation, this light rail project has performed worse than the average international figures reported by several authors concerning the two most critical issues related to this kind of project: cost overruns and overestimated demand. Scenarios analyzed in this work show that failures in managing and executing Line 3 resulted in, at least, a loss of 431 million dollars in net social bene- fits (in present value), but if we compare the ex-post results to ex-ante studies, this difference can escalate to more than 1 billion dollars. This loss means wasting scarce social resources that could have been better used to solve other needs.

Section 2 reviews the literature on ex-post evaluations in urban rails, including failure to forecast ridership and costs. Section 3 describes the characteristics of the project. Section 4 briefly explains the approach of CBA. Section 5 explains the methodology for the ex-post analysis. Section 6 reviews several versions of the official CBA, indicating its chronology and main results. Section 7 explains the scenarios constructed for ex-post analysis, including its results. Finally, section 8 discusses the results and conclusions.

To know if an infrastructure project is successful, it is not enough that its construction phase is finished and the project has begun to operate; also, it would be desirable to elaborate an ex-post cost-benefit analysis. The broader objective of the ex-post evaluation is to learn from the experience, i.e., the re-examination of the case studies focused on elaborating those recommendations that may increase the capability of the projects to achieve their identified goals (Maffii and Parolin, 2012) and, ultimately, avoid Pareto-inefficient allocation of resources. Some progress has been achieved following this path; for example, The EVA-TREN research project, supported by the European Commission, was purposely targeted at improving the ex-ante appraisal practices for assessing large transport infrastructure projects through the ex-post analysis of several case studies.¹ The ex-post evaluations should also have a central role in improving the process of infrastructure planning (Short and Kopp, 2005; EVA-TREN, 2008; Grimaldi et al., 2014; Meunier and Welde, 2017).

Several light rail cases show favorable ex-post evaluations, e.g., the urban rails from San Francisco and Chicago in the U.S. (Winston and Maheshri, 2007), and the new Bergamo tram in Italy (Grimaldi et al., 2014). However, there are also many cases where investments in urban rail do not produce desirable effects on social welfare. That is the case of over 20 systems in U.S. cities where the share of urban travelers is declining (Winston and Maheshri, 2007). When reviewing the cumulated literature about light rails (and other trains) it seems evident that a key factor to project success is the obtention of a satisfactory level of demand to reduce the average total costs. It is worth emphasizing that a sufficient level of demand generates enough social benefits to compensate social costs of the project. In particular, as shown by Winston and Maheshri (2007), the lack of demand has been the main problem in most urban rail projects in the U.S., where urbanization and employment patterns have reduced ridership. This problem is especially true in New York, Washington, Boston, Atlanta, Philadelphia, and Los Angeles.

According to the analysis of 44 urban rail projects in several countries elaborated by Flyvbjerg et al. (2006) and Flyvbjerg (2007a and 2007b), the analysis of ridership shows urban rail to achieve considerably fewer passengers than forecast and thus lower revenues: actual traffic is on average 51% lower than forecast traffic, and 75% of these projects have actual traffic that is at least 25% lower than forecast traffic. Moreover, 25% of projects have actual traffic at least 68% lower than forecasted. In some extreme cases, demand underestimation has ultimately led to suspend a project after it was already operating. For example, in Spain, a new tramway opened in Vélez-Málaga in 2006 but was shut down six years later. The tram project in Jaén operated for only two weeks in 2011. In both cases, demand turned out to be significantly lower than anticipated (Van der Bijl et al., 2018). Carpinteiro and Petersen (2014) also documented the lack of demand in several Spaniard light rail projects, observing only 50% of the forecasted demand.

The other relevant problem from light rail projects is cost overruns. Flyvbjerg et al. (2003a), Flyvbjerg et al. (2003b), and Flyvbjerg (2007b) analyzed 44 urban rail projects (18 located in North America, 13 in Europe, and 13 in developing nations), compared to 214 other transportation infrastructure projects. Urban rail has the largest cost escalation with an average of 44.7% in constant prices (standard deviation = 38.4), followed by bridges and tunnels with 33.8%, and finally roads with 20.4%. For urban rail, 75% of projects have at least 33% cost escalations. Among urban rail projects, 25% have at least 60% cost escalations. These studies did not find any statistical difference in cost overruns between urban, high-speed, and ordinary rail. Recently, Andric et al. (2019), studying 102 major infrastructure projects from Asia, confirmed that rail projects are most likely to overrun budget (mean = 26.2%). Lee (2008) and Huo et al. (2018) obtained a similar conclusion for South Korea (where all rail projects had a maximum cost overrun of 50%) and Hong Kong (average cost overrun for rail = 34.8%), respectively. Flyvbjerg et al. (2003b) and Bhargrava et al. (2010) found that cost escalation is strongly dependent on the length of the implementation phase. Cantarelli et al. (2012) concluded that the size of cost overruns varies with location. For urban rail projects, Dutch projects performed considerably better, with projects having significantly lower cost overruns in real terms (11%) compared to projects in other northwest European countries (36%) and other geographical areas (40%). Meunier and Welde (2017) found that 75% of Norwegian road and rail projects have had final costs lower than the approved budget (average cost underrun = 7%).

Overall, the ridership overestimation and the cost overruns reveal a lack of reliable knowledge of risks. Although many urban rail projects fail to control and manage those risks, evaluating the social resources lost during the process through ex-post analysis is not common practice. Ex-post analysis documents a need for implementing institutional checks and balances that would change the current practices of cost underestimation and revenue overestimation to ones where empirically-based risk assessment and management are effectively used in all phases of the project development process (Flyvbjerg, 2007b).

The Project is located in the West part of Mexico in the Metropolitan Area of Guadalajara (MAG), the second most populated city in this country (about 5.2 million people in 2020), and now constituted by nine municipalities. The municipality of Guadalajara, inside MAG, is the capital of the state of Jalisco. The mobility of the inhabitants of this metropolitan area is mostly secured by private owned vehicles (2.4 million vehicles in 2018) and more than 5 thousand buses. It is worth noting that for the most part the organization and administration of these buses are not publicly centralized, like those in large cities in Europe for instance, but rather the bus lines are subcontracted and the vehicles owned by private entities, often competing against each other for the extra passenger, in accordance to government regula[1]tions with regards to a broad framework of fares and other specifics with poor oversight for efficiency and safety. The service provided is thus very deficient because there are no regular schedules, information about routes is not well established, nor disseminated to the public; the buses are often overcrowded and drivers overlook established stops. In short, according to SENERMEX (2019, p. 67), there are 93 bus routes with already high use along the tracks of the Line 3 project. It is worth noting that among the chaotic bus system there are just three efficient and ordered means of mass transport, Lines 1 and 2 of Light Rail and one Bus Rapid Transit (BRT) line. Due to the population growth and the lack of efficient passenger transport systems, MAG faces soaring traffic jams and increasing travel times. According to the project’s official CBA (SENERMEX, 2019, p. 60), in 2018, the average speed of public transport was about 10 miles/hour, and the generalized costs of travel (GCT)² along the project route had a yearly value of 760 million dollars. There is a projection about the growth of GCT of 3.4% per year in 2018-2026 (considering the project’s absence) (SENERMEX, 2019, p. 115-116). As we know, these problems produce externalities, mainly pollution (MAG air is currently 4.5 times above the WHO annual air quality guideline value).³ It is within this context that the federal bureau specialized in infrastructure construction (la Secretaría de Comunicaciones y Transporte – SCT) received Jalisco’s government proposal for the execution of Line 3.

Line 3 is 20.74 kilometers (12.88 miles) long (74.2% on an elevated viaduct, 14.9% underground, and 10.8% on the surface). It crosses the three most populated municipalities of the MAG, from Zapopan, in the North-West, to Tlaquepaque, in the South-East, crossing through down- town (Guadalajara) (see figure 1). Line 3 features 18 stations: 13 elevated and 5 underground; and, according to the official CBA, it would transport 227 thousand passengers per day (SENERMEX, 2019, p. 207). There were four different versions of the official CBA. The first one was presented in 2013, stating that the construction would begin during that year and become operational in 2017. Eventually, the project’s construction began in August 2014, and the project began to operate during September 2020; consequently, the project implementation lasted more than six years!

Figure 1
Location of Line 3 in the metropolitan Area of guadalajara
***

Objectives of the project. – According to the official CBA (SENERMEX, 2019, p. 41), “…is to increase society’s welfare, through the implementation of a massive transport system in the form of a light rail... This social improvement will be reached by diminishing present and future travel times and vehicle operating costs from public and private trans- port systems. Additionally, the train procures security on travels and generates less pollution than present modes of transport.”

Main costs of the project. – Studies, investment in infrastructure (liberation of ways, construction of the track, train control platform, high-voltage and traction substations, tunnels and stations), trains, costs for discomfort during construction, and operation and maintenance costs.

Main benefits of the project. – Diminishing GCT, liberation of resources, and end-of-useful-life salvage values.

CBA implies valuing the project’s flows of benefits and costs along a planning horizon, recognizing that social benefits and costs do not always coincide with private pecuniary benefits and costs (Harberger, 1976). Another concern of CBA is to appraise those benefits and costs for which there is no market value (Layard, 1972). For example, in the case of Line 3, as pointed at the end of the section 2, there were identified costs of travel and costs for discomfort. Both refer to goods (time and comfort) for which there is no explicit market.

Profitability measures are calculated when benefits and costs have been identified and valued along a time horizon. These measures determine, with a certain degree of confidence, if an investment project will supply positive net benefits to society, and if that project is more convenient than other alternatives. This approach is used in many countries, including the European Union and the United States, to identify those investment projects with the most outstanding social profitability.

An investment project’s main profitability measures are the net present value (NPV) and the internal rate of return (IRR). We may define NPV as follows:

where:

(NB)t net benefits (total benefits minus total costs) during a single periodt

r social discount rate

t number of time periods

A project that presents an NPV > 0shows positive social profitability; otherwise, the project does not increase society’s welfare.

On the other hand, IRRis calculated such that theNPV of investment yields benefits equal to zero. Thus, if IRR > r, the project is profitable; otherwise, the project will not be convenient. It is important to note that the official social discount rate is 10 percent in Mexico. Thus, any IRR lower than 10 means the project is not profitable.

The ex-post evaluation is based on the ex-ante evaluation, as suggested by Morín and Alvarado (2017), in which the project’s engineering studies purportedly support cost estimates. This work takes data from officialCBA and incorporates actual ex-post costs and benefits, intro- ducing updated figures based on specific public facts and information. We followed four main steps:

i. Replicating the ex-ante evaluations. There were four official CBA. Once those documents were obtained and comprehensively analyzed, we replicated the profitability results for each one, focusing particularly on NPV and IRR.

ii. Making a detailed search of possible factors that affected the project’s costs and benefits. According to Morín and Alvarado (2017, p. 4), the ex-post evaluations are intended to “identify those factors that influenced in a positive or negative way in compliance with the objectives set for the project during the pre-investment stage, ...” These possible changes were based on the review of official information regarding the project’s final costs, other complementary costs reported by the local press, reports about the project’s ridership (one year after it began operations), and changes based on the delays of the project not considered by the last official CBA.

iii. Formulating scenarios applying changes based on factors found during step ii, upon the most recent official evaluation cash flow. This process allows bestowing more realism to the evaluation results. We obtained the NPV and the NPV/investment cost ratio for each scenario.

iv. Once step iii results are obtained, establish comparisons against the ex-ante results.

v. The results obtained in steps i to iii will allow a global evaluation of the project, in terms of what Morín and Alvarado (2017, p. 16-18) call “efficiency analysis” and “efficacy analysis”. The former focuses on the degree to which the activities were executed in accordance with the established costs and deadlines and if the components were obtained in accordance with their initial conceptualization. The latter seeks to analyze whether the project reached or is in the process of achieving the socioeconomic profitability that was raised in the ex-ante evaluation.

In the case of Mexico, the Department of the Treasury (la Secretaría de Hacienda y Crédito Público – SHCP) requests a CBA for any public investment project requiring federal funding for its planning and execution. When the project has been registered in the SHCP project portfolio, a CBA document is valid for three years. However, if the project goes longer than this period, the CBA must be updated. Another reason for updating the study is to allow investment costs to grow more than 10%. Due to these reasons, during the six years of the Line 3 implementation, four official CBAs were presented.

CBA studies presented to the SHCP must follow certain guidelines. In the case of massive transport systems, Vázquez and Morín (2018) propose a methodological guide, which establishes the studies of traffic and transport engineering that must be carried out, especially with the intention of calculating the GCT of the project. In addition, the components of the document are explained, such as the analysis of the current situation, the analysis of the situation without the project, the analysis of the situation with the project, and the socioeconomic evaluation. It is worth mentioning that the ex-ante evaluations of Line 3 do comply with the structure and components established by the indicated guide.

Table 1 shows some profitability indicators from the four Line 3 CBAs (SENERMEX, 2013, 2017, 2018, 2019). Investment figures and NPVare shown in dollars and Mexican pesos, taking into consideration the monthly average exchange rate at the approximate time when each CBA document was introduced to the SHCP project portfolio. It is relevant to point out that according to the four CBA documents, the project’s nominal initial investment increased 76% in domestic currency. Nevertheless, that investment increase, measured in dollars, was about 20%. These contrasting figures reveal that the exchange rate depreciation can explain some less than two-thirds of the total investment increase considered by CBA documents. We have three comments related to this issue: i) This type of project has a substantial share of imported supplies in a country like Mexico, especially the part related to rolling stock and the equipment to drill tunnels. However, the construction supplies are mainly domestic; we cannot establish a straightforward relationship between investment costs and exchange rate. ii) If the government had been executed the project according to the original schedule, it would not have been so exposed to the exchange rate fluctuations. For example, in July 2013, the average exchange rate was 12.96 Mexican pesos per dollar. This rate was lower to 16 pesos per dollar until mid-2015. In contrast, in 2019, the average exchange rate was over 19 pesos per dollar. iii) Cost raising in domestic currency was not (mostly) a consequence of inflation. The cumulative inflation rate in construction materials was about 26% during the implementation stage.⁴ Cost increase in constant prices (considering 2019 = 100) was about 60.5%. This cost increase in domestic currency is clearly greater than the average international cost overrun from urban rails (45%) reported by Flyvbjerg et al. (2003a), Flyvbjerg et al. (2003b), and Flyvbjerg (2007b). According to these studies, Line 3 is located in the highest quartile of cost overruns. Line 3 cost overrun is also notably greater than the average cost overrun from rail projects in Asia (26%) reported by Andric et al. (2019).

Table 1
Profitability indicators from Line 3 CBAs

(1) Official CBA documents do not show a precise elaboration date. They show an elaboration year, but it can be wrong. For example, on the cover of the last version (presented in 2019), it says “June 2018”, the same as the previous version.(2) Average exchange rates correspond to the following dates:BCA1 (October 2013), BCA2 (August 2017), BCA 3 (April 2018), and BCA 4 (march 2019).(3) The official social discount rate is 10 percent in México.

The first CBA was presented in 2013 and determined an NPV of 557 million dollars, representing 40% of the nominal investment.⁵ The second CBA was presented in 2017 and calculated an NPV of 219 million dollars, meaning 16% of the nominal investment. The third one was presented in 2018 and determined an NPV of 155 million dollars, representing 10% of the nominal investment. The last CBA was presented in 2019 and calculated an NPV of just 92 million dollars, less than 6% of the nominal investment. The last figure is the most revealing indicator of the project delays and cost overruns impacts. We remark that all these figures come from official CBAs, with no modification and without questioning its methodology or procedures.

As mentioned in section 4, this work aims to take figures from official CBA and incorporate actual ex-post costs and benefits. For this, we defined six scenarios that take into account the same cash flow stated in the last CBA (see Table 2). Then, we introduced some changes based on specific public facts and information. The following scenarios were established (see Table 3):the last CBA(see Table 2). Then, we introduced some changes based on specific public facts and information. The following scenarios were established (see Table 3):

Table 2
Benefits and costs from official CBA (2019) (million dollars)

Source: SENERMEX (2019: 214).

1) Updating of investment costs. – According to the last official CBA, when concluding the project’s implementation (supposedly at the end of 2019), Line 3’s total investment costs would be 31,143 Mexican million pesos, i.e., 1,618 million dollars – considering the average exchange rate in March 2019. However, Line 3 was ultimately concluded during mid-2020 (it started operations on September 12th, 2020). When finishing the construction, the federal government reported a total investment cost of 35,054 Mexican million pesos.⁶The difference between both figures was about 207 million dollars. In order to measure the impact of cost overrun on project profitability, we added these additional costs to the cash flow in 2020.⁷ Furthermore, we subtracted the proportional part of unspent operation costs during 2020. The NPV decreased to just 13.7 million dollars, with an IRR of 10.09%. NPV/ nominal investment ratio was less than 1% (0.75%).

If we add these extra 207 million dollars to the 1,618 million dollars accounted for in March 2019, we have a revised total investment cost of 1,825 million. This final figure means that cost overrun, measured in dollars, compared to those costs from the original CBA, was about 35%. Figure 2 shows investment costs from the first CBA and this ex-post scenario.

Figure 2
Total investment costs from first CBA (2013) and ex-post Scenario 1
ource: Own calculations based on SENERMEX (2013) and (2019), and the Funding Calendar of the project. Notes: (1) Each yearly figure was converted from mexican pesos to dollars using the average exchange rate corresponding to July. (2) VAT (16% in Mexico) was subtracted from nominal quantities.

2) Reduction of benefits in 2020. – In addition to those changes included in scenario 1, we considered that the last CBA scheduled the beginning of operations for Line 3 in January 2020. As we already mentioned, Line 3 started to operate during September 2020; thus, the project lost 70.8% of its benefits in that year. We subtracted those benefits without modifying any other variable. As a result, the NPV turned negative in almost 22 million dollars (IRR = 9.84%). NPV/investment cost ratio was reduced to -1.2%. This finding means that just updating benefits (in 2020) and investment costs, the Line 3 project is no longer profitable.

3) Repairing costs for alternative roads and costs for pedestrian adaptations. – Additional to those changes included in scenarios 1 and 2, this scenario includes costs for road repairs used as alternative routes during the construction period. Repairing costs were 19.5 million dollars, and the local government applied them at the end of 2019. In this scenario, we also included pedestrian adaptations’ costs to facilitate access to some Line 3 stations. These adaptations were made involving 10.6 million dollars. As a result, the NPV was reduced to -37.5 million dollars (IRR=9.73%). NPV/invesment cost ratio was reduced to -2.0%.

4) Exclusion of costs and benefits from another project. – In addition to the changes included in scenarios 1 to 3, this scenario excludes costs and benefits from an underpass for vehicles built at one point in the Line 3 route.⁸ It is relevant to point out that the underpass was not necessary for the passing of Line 3’s track (at this point, Line 3 runs on an elevated viaduct), but to mitigate traffic car in a conflictive cross point. The official CBA added costs and benefits from this underpass because it was built using the Line 3 budget, but this procedure is incor- rect, considering they are different projects. Let us remember the prin- ciple known as “separability of projects,” which states that if we have costs and benefits from two (or more) independent projects, we must evaluate each project independently, not together. As a result, the NPV was reduced to -139 million dollars (IRR = 9.00%). NPV/investment cost ratio was reduced to -7.6%.

5) Additional annoyance costs. – In addition to the changes in scenarios 1 to 4, we included additional annoyance costs during the project’s construction. According to the official CBA (SENERMEX, 2019, p. 52): “[annoyance] costs are due to a minimal affectation because construction procedures have followed night work strategies or have been developed during low demand hours...” As these costs were considered “minimal,” project evaluators assigned them just a value equivalent to 1.5% of vehicles’ total travel costs during the four hours of maximum traffic on weekdays. Indeed, those costs were not negligible. The city lived in continuous traffic chaos for almost five years, and some areas suffered recurrent flooding during rainy seasons due to Line 3 construction works, although these latter costs are very compli[1]cated to estimate, and we only mention them. The actual annoyance costs were multiplied several times, but it is impossible to know its total amount; we supposed, conservatively, that annoyance costs doubled (i.e., 3% of the considered travel costs) those estimated in the official CBA. We obtained an NPV of -197 million dollars (IRR = 8.62%). NPV/investment cost ratio was reduced to -10.8%.

6) Reduction of benefits due to overestimation of demand. – In addition to those changes included in scenarios 1 to 5, we included reductions in benefits due to lower-than-forecasted demand in ex-ante evaluations. According to CBAs, Line 3 would have an average daily demand of about 227 thousand passengers. After one year of operations, the operative enterprise reported a daily average ridership figure of just 67.8 thousand, i.e., 29% of forecasted demand (Carapia, 2021). After almost two years of operations, the effective demand had grown up to 50% of the original forecasts (Rodríguez, 2022). This scenery was included at the end because Line 3 began to operate in 2020 and, during the first months, the Covid-19 pandemic had probably contributed to reducing demand. However, at the end of 2021, the director of the operative enterprise pointed out that the forecasted goal will be reached “probably between the next 5 and 10 years” (Carapia, 2021); implicitly accepting that original demand was overestimated. In this scenery we tried two situations: a) that the original goal can be reached in 5 years (30% in 2020-2021, 50% in 2022, 60% in 2023, 75% in 2024, 90% in 2025, and 100% from 2026); b) that the goal can be reached in 7 years (30% in 2020-2021, considering a 10% annual increase, and 100% from 2028). We obtained an NPV of -431 million dollars (IRR = 7.16%) in the former. In the latter, the NPV was reduced to -500 million dollars (IRR = 6.72%), and the NPV/investment cost ratio was reduced to minus 27.45%. Of course, it could also happen that the forecasted demand will never be fulfilled, reducing the project’s ex-post profitability ever more.

In Table 4, results from scenario 6 are compared to those established by CBAs. Comparing the former results to those from the first CBA, there is a loss in social profitability of almost 1 billion dollars, and IRR was reduced to almost one-third. According to Isham and Kaufmann (1999), significant differences in projects’ rates of returns from three- to-seven percentage points, if economy-wide, can add up to a very significant difference in the aggregate growth rate of the country; Line 3 project suffered a loss of over 10 percentage points in IRR. Moreover, the first CBA stated that the profitability of Line 3 would be over 40% of investment costs. Ex-post scenario 6 indicated that the Line 3 project profitability was minus 27% compared to investment costs.

Table 3
Characteristics of ex-post scenarios and their profitability indicators

Note: The official social discount rate is 10 percent in México.

Table 4
Profitability indicators from official CBAs and ex-post Scenario 6

Note: The official social discount rate is 10 percent in México.

The results obtained in the ex-post scenarios allow a global evaluation of the project according to its efficiency and efficacy (Morín and Alvarado, 2017). In the first case, we can affirm that the project was inefficient since it was not executed according to the established deadlines and costs. In the second case, we can affirm that the project was ineffective since it did not reach the expected socioeconomic profitability.

In Mexico, we cannot say that CBA is used as a guide to define public investment projects. As happens in other countries, politicians consider different criteria to determine public investments, especially commitments to social and political groups (Eliasson and Lunberg, 2012, Annema et al., 2017). When politicians have defined the project, and if the investment requires federal funds, it must present one CBA document proving the project is profitable for society (i.e., it has a positive NPV). This CBA must compare its results against an alternative project, whose costs were obtained in a parametric fashion as part of its methodology. Thus, the CBA shows some possible project advantages versus a counterfactual case, but not really versus a set of feasible alternativeprojects. This limitation arises because the CBA is not analyzed within a general multi-year transport plan, which involves many related projects; there is no such plan in Mexico. Moreover, CBAs do not display incentives for an adequate forecast of the main factors for projects’ successes. In Line 3 case, this light rail has performed worse than the average international figures reported by several authors concerning the two most critical issues related to this kind of project: having 62% of cost overruns (in constant prices) and 50-70% of overestimation demand.

Furthermore, in this paper, we have seen that information and results from CBAs may have a little (or null) impact on the adequate execution of projects, even considering that there are official methodologies such as Vázquez and Morín (2018), and even if the studies are structured according to these methodologies.⁹ Why does this happen? Line 3 was built during a period of macroeconomic stability in Mexico, and any lack of funds to execute the project or lack of supplies or specialized equipment were never reported. Another relevant issue is that there was no social opposition because people recognized the need for better public transport. There were no rejection attitudes from local government authorities either, and there was the previous experience of having built Lines 1 and 2 in the same city.

The Line 3 project was executed by a federal bureau specialized in infrastructure construction. Why were there so many delays in the construction of Line 3? Last CBA (SENERMEX, 2019, p. 8-29) includes an explanation for costs increases; many of them are correlated with these delays: a) project modifications (of trains’ specifications, relocation of stations, protection of historic buildings), b) elements not provided for in the original construction catalog, c) unplanned removal of facilities and monuments, d) payment of financial interests due to outstanding payments, e) delays in the deed of land, f) late delivery of works, g) delay in the release of construction areas. It is worth highlighting that the document indicates the “technical” reasons for cost overruns, but even so, they represent deeper causes. Some of them related to insufficient planning, unlike various European cases where the cause of project failure was excessive planning (van der Bijl et al., 2018).

The literature on institutional gaps gives us important clues to explain cases such as Line 3. Institutions establish restrictions and incentives over individual or collective actions (North, 1990). Institutions also explain some inertias or “traps” affecting the efficiency of social actions, such as largescale transport projects. This kind of trap leads to practices where some actors have incentives to perform inefficiently to benefit from delays and cost overruns without suffering sanctions. This behavior is sometimes borderline corruption, although this is difficult to prove for an external party. Another observed effect of this type of trap is the lack of local authorities’ management capacity. This inability made it impossible to speed the construction process, or take practical actions to reduce annoyances caused by the project. Technical reasons for cost overruns and delays in Line 3 seem to be rooted within formal institutions, including property rights, legal framework, bureaucratic functions of government, and the power distribution among different government levels (Williamson, 2000). As it is known, the success of public investment projects is contingent on the quality of these institutions (Burnside and Dollar, 2004; Dollar and Levin, 2005) and favorable political, eco, social, and cultural conditions (Ika et al., 2012).

In countries like Mexico, institutional change focusing on transparency and accountability is necessary, but proposals to enhance future projects must consider institutions at their adequate level. For example, if in the official Line 3 CBA evaluators admitted planning failures and many other mistakes, and if there was not an appropriate government management capacity to solve and sanction those problems, then an institutional change that allows a better management framework for assigning responsibilities, coordinating functions, and penalizing negligence and omissions is necessary. All these elements can be analyzed as part of governance institutions, but governance cannot be improved without a solid legal framework (Canitez et al., 2019).

These issues remind us that institutional changes can take a lot of time, unless stakeholders accelerate them. In public transportation projects, users are the primary stakeholders. If society can demand major public accountability and better governance effectively, these elements can increase government efficacy. Project coordination and management could be enhanced by establishing impact evaluations (Legovini, 2010) that register and make procedures and ex-post project results public knowledge. Establishing stakeholder committees that participate in impact evaluations could be a pressure mechanism to improve public investment management offices’ performance.

Additional recommendations imply that the federal bureau specialized in infrastructure construction (now named la Secretaría de Infraestructura, Comunicaciones y Transporte – SCT) make the following elements mandatory to every major infrastructure project: 1) Establishing probability distributions for each type of project to compare each specific project with the reference class distribution (Reference Class Forecasting method). This practice allows for correct identification, reducing, and managing risk (Flivbjerg 2007a). 2) Implementing a methodological exercise analyzing all probable causes for cost overruns on each type of project. Ahiaga-Dagbui et al. (2017) indicated that understanding cost overrun is not the ability to list or rank factors, but the capacity to analyze connections, interactions, and plausible causal combinations. 3) Forecasts should be made subject to independent peer reviews and made available to the public as they are produced, including all relevant documentation (Flivbjerg 2007a).

Line 3 represents a sad story of social benefits gone to waste. Scenarios analyzed in this work show that failures in managing and executing Line 3 resulted in, at least, a loss of 431 million dollars in net social benefits (in present value), but if we compare the ex-post results to ex-ante studies, this loss can escalate to almost 1 billion dollars. Those social resources were wasted, generating huge annoyances for city inhabitants. In the end, Line 3 will be useful to many users, but society will never recover the lost social resources.

JEL Classification:: D61, H54, L92, O54, R42