Localizing the Sustainable Development Goals: A case for investment in data and impact evaluation

2.1Policy coherence

To accelerate the necessary progress on SDG implementation a systems approach needs to be in place where there is a strong commitment to SDG localization evident in collaboration at all levels of government. Policy integration has been earmarked as a critical enabler for achieving the SDGs [14]. Achieving a policy coherence at all levels is not an easy task. Horizontal and vertical intersectoral integration needs to be developed by adjusting existing or adopting new institutional structures and decision-making processes [6].

In order to create a conducive environment for achieving the SDGs the efficiency and effectiveness of governance systems need to improve through national policies and coordination mechanisms as well as measures that can be used to achieve a specific SDG target [15] suggested practical guidelines for local stakeholders, entailing four initial steps for mainstreaming the SDGs: (i) developing a participatory process; (ii) setting the local agenda; (iii) planning for implementation; and (iv) monitoring and evaluation. To achieve policy coherence local authorities can link different development plans to the SDGs or systematically align local action and policies with the SDGs and address all local action with a development element [16].

2.2Stakeholder engagement

The 2030 Agenda for Sustainable Development recognizes that no actor can achieve the scale and ambition of the SDGs. Therefore, it requires wide stakeholder engagement, including civil society, the private sector, academia, philanthropic organizations, research foundations and technical experts [17]. Stakeholder engagement calls for the active engagement of all partners in providing inputs to problemsolving and decisionmaking. It also entails enabling public oversight, transparency and accountability of local governments. For example, the use of social media platforms can increase the active participation of civic society and citizens and therefore lead to design of new polices and the delivery of public services by local governments. Similarly, private sector involvement in public infrastructure projects through public-private partnerships, in addition to providing the necessary financial resources for the project, can also reshape business practices and promote inclusive development [17]. However, changes in legal and regulatory environment are needed to enable this collaboration. In addition, the private sector needs to look beyond financial results and incorporate other metrics to measure social impact which requires the support of local governments.

The notion of equal participation must also be factored in any discussion on the implementation of the SDGs at the local level. Therefore, a stronger presence of women in local government will ensure more effective and full participation politically, socially and economically, which will have a major and very positive impact on local policy-making and decision-making In addition, young people will inherit the development challenges. Youth participation also reflects local diversity in terms of socio-economic, religious, ethnic, cultural and gender dimensions of local development. There is potential for including youth in the political process through different engagements, such as lobbying and activism rather than elections [10].

2.3Decentralization

The level of political and fiscal decentralization is crucial in achieving an enabling institutional environment that strengthens the ability of local governments to set priorities in line with the SDGs. However, many decentralization attempts have been afflicted by insufficient management and administrative capacity. An enabling environment should constitute the basis for the devolution of power and enable local government to discharge its obligations, efficiently and effectively. Consideration must be given to fiscal transfer instruments ensuring that local planning and budgeting feed into the national budgeting and planning process [18].

In addition, fully integrating the SDGs at local levels requires subnational governments to have the public sector capacity to engage in planning, monitoring and evaluating of SDG progress. Improving the quality of local human resources, management and technical capacities of local governments to mobilize local resources, deliver services and engage citizens in planning and decision-making is crucial in promoting ownership and facilitating sustainable development and ultimately, enhancing service delivery at the local level.

2.4Data ecosystem

In addition to investing in human resources’ capacity, strengthening statistical capacities is key to effective SDG localization. Capacity to collect, monitor and analyse data on various indicators are critical elements of successful SDG implementation. In addition to methodological difficulties in measuring some of the SDG indicators, there are also capacity challenges. While at the national level, national statistical offices are usually responsible for collecting data, at the local level, the management systems either do not exist or are not suitable for using quantitative indicators to measure progress. Therefore, significant investment in data management systems is needed accompanied by investments in capacity to analyse and share data among different levels of governments. This would allow local governments to monitor progress on the SDGs and become more transparent and accountable. This will require not only significant financial outlays, but also a strong cooperation between different stakeholders such as local and national governments, NGOs, civil society and the private sector. Having accurate statistics is a necessary condition to collect, monitor and evaluate SDG indicators. In addition, it is also critical that different stakeholders can provide inputs when adapting indicators to the local level. Furthermore, to encourage a wider participation it is necessary that information collected at the local level is included in nationallevel reporting and in SDG progress reports.

2.5Financing development

In addition to statistical capacity and broad stakeholder engagement, local governments need to have the financial capacity to take on increasing responsibilities. Even before the COVID-19 pandemic, local governments faced challenges in meeting recurrent expenditure and make long-term investments in support of sustainable and inclusive development [19]. While greater decentralization has led to increasing responsibilities of local governments, in many cases an increase in budgets has not followed suit. A recent survey by the Emergency Governance Initiative encompassing 33 territories from 22 countries across all continents found that in addition to having insufficient budgets, many local governments have been grappling with an overreliance on central governments, lack of access to new sources of revenue, limited autonomy to change/introduce taxes, and low levels of taxpayer compliance. The ongoing pandemic has exacerbated some of these challenges because local governments have had to increase the budgets of their social, health and public services and to support local businesses that were unable to trade due to the pandemic. Therefore, it is to be expected that infrastructure development and interventions to promote gender equality and investment in education will be postponed.

Given the limited fiscal capacity of local governments, the Financing for Development: Progress and Prospects 2018 report underscores the importance of: (i) the strengthening of national and subnational institutional frameworks to include formal and clear assignment of functions and revenue generation responsibility; (ii) transparent and accountable financial management; (iii) incentives to function effectively; and (iv) predictable and flexible intergovernmental transfers. This would allow local governments to respond to local needs better.

To facilitate the attainment of the SDGs at local levels local governments should be able to raise revenues through tax collection and from new sources outlined in Table 1 that complement traditional forms of financing and can create new partnerships that can leverage the expertise for achievement of the SDGs [19]. The new forms of financing are especially important for developing countries and subnational governments whose local budgets are being affected by the pandemic. In addition, the availability of new forms of financing can enable local governments to take an integrated approach to development and strengthen multistakeholder engagement [17].

To effectively finance development, the budgeting process at the national and local levels needs to be aligned with development priorities. Preferably, the SDGs are incorporated within local government budgets. Further, public expenditure capacities need to improve via financial and technical support provided by national governments or international organizations. The reliability and timeliness of fiscal transfers must be increased to support localization of SDGs, especially in the most remote areas or regions affected by natural disaster or conflicts. There should be mechanisms in place that determine allocations to prevent inequality in the access and distribution of funds from national to local governments.

While COVID-19 poses a challenge, it also provides an opportunity to re-emphasize the importance of the SDGs and to put in place monitoring and evaluation procedures that would allow to identify which SDGs have been hit the hardest and therefore need support. Recovery thus represents an opportunity to redirect development that relies on principles put forward in the 2030 Agenda.

3.1SDG measurement at the local level

The core principle of the SDGs to leave no one behind requires disaggregated data (by age group, ethnicity, income group, and at different geographical levels), particularly for local development planning and programme implementation. Since most of the SDG indicators are not routinely collected within the national statistical system, local information could be obtained either through primary data collection or from existing large-scale (household) surveys. Also, ideally, panel data would be available to monitor progress over time. However, until such data ecosystems are in place, progress on the SDGs is measured by representative household surveys.

Primary data collection would have the advantage of permitting more tailored questionnaires that focus particularly on the measurement of SDG indicators. When secondary data sets are used, the appropriateness depends on the purpose of the statistical analysis. Should the aim be to learn about the effects of local spending on local SDG progress within a single country, then focus should be placed on the various data sets available within the respective country. In contrast, to estimate impacts for several countries, there is preference for household surveys conducted in several countries with the same or very similar methodology to permit cross-country comparability of the estimates. Wellknown examples of these multipurpose large-scale household surveys are Demographic and Health Surveys (DHS), Multiple Indicator Cluster Surveys (MICS), Living Standard Measurement Surveys (LSMS) and others which have been implemented in many countries and several points in time with similar methodology. However, current general purpose national household surveys have shortcomings: they have not been tailored particularly to the SDG indicators and would therefore also introduce measurement error. These surveys have often been conducted only very infrequently, which precludes timely measurement of progress or lack of progress in certain sectors. The household survey should ideally be repeated every two or three years to measure progress more quickly.

In addition, the empirical analysis requires data on all villages, counties or districts where outcome data is available for (at least) two points in time. Since datasets such as MICS, LSMS and DHS have been developed as multi-indicator nationally representative surveys, they will not include every county. In addition, for each new wave, these surveys normally draw new samples of enumeration areas, i.e. they do not target the same counties/districts. From each wave of MICS, LSMS and DHS a data set with countylevel averages of outcome variables would thus have to be constructed.

Nationally representative data sets are usually not geared towards very precise measurement at the local level. One solution is to apply techniques such as small area estimation techniques to improve the measurement of SDG indicators at the local level. The technique improves the level of granularity without necessarily collecting additional data in the field. This methodology integrates existing surveys with other auxiliary datasets such as census or administrative data and capitalizes on the strengths of each source. Survey data allow for the collection of detailed information on household characteristics but not the collection of information from every unit of its target population. In contrast, census data may completely enumerate all units within the target population but may lack detailed information about the characteristics of the unit. Small area estimation therefore yields estimates that are reliable even at granular geographical locations (municipality) for which the survey was not originally designed. There are two approaches used to generate small area estimates: a ‘direct approach’, which uses observations for the small area gathered via nationwide survey; and an ‘indirect approach’, which uses observations and auxiliary data in mathematical and statistical models. These models provide the functional relationship between the data obtained directly from the nationwide survey and a set of auxiliary data.

Small-area estimation techniques could also combine data from other sources in order to increase precision of local SDG indicators. For instance, one might have detailed data sets on sales of contraceptives for each pharmacy in the country and combine this with DHS data on family planning to make better model-based local prediction for family planning. Alternatively, one might have very detailed data on violence reported to the police and combine with DHS data for better local predictions on gender-related violence.

Another alternative is to use phone surveys that could also support such small area estimations, e.g. higher-frequency intermittent phone surveys. Phone surveys alone, however, are prone to attrition and non-response, as well as selection bias due to incomplete coverage. In addition, SDG indicators required face-to-face interviews due to their complex definitions.

However, it is possible to consider combining phone surveys with the household survey so that the face-to-face household survey would be considered the error-free data and the phone survey the response- and selection-bias affected sample. At each point of time when both surveys are conducted, correction factors are computed and applied to the phone survey data so that the corrected phone survey data would produce the same numbers. In these points of time, when only the phone surveys are conducted, the correction factors are applied. The phone survey could be conducted every six months to obtain updated measurements at higher frequency.

3.2Local government data

Although household surveys focus on outcome variables in terms of the SDGs, they do not include where investments have taken place. Therefore, existing secondary data on a community or district level on local development investments are required.

Ideally, the number of investments dedicated to development by various social sectors (education, health, etc.) for each county/district would be measured Investments into development could stem from three sources:

Most likely, data on private investments by county will not exist. Therefore, the focus is on public investments at the district/provincial level, from national budgets and/or from international development assistance sources.

Although nationallevel data have been harmonized according to the System of National Accounts and have become widely available for empirical analysis, this is less the case with local government expenditure. If at all, public figures were collected for the official statistical reporting at the provincial level, but often not at the district or municipality level.

It would be ideal to know the amount of investment for each county by various social sectors (education, health, etc.). It would be expected that investments into for example education are closely aligned with SDG targets related to education. In recent years, there have been efforts in various countries to improve the accuracy of sub-nationallevel data.

However, local government expenditures are not typically aligned with the national systems of accounts and public financial classification system. For example, if the public financial system has an account number for funding for education, which includes primary, secondary and university funding, then local government spending would capture total education funding and it would not be possible to separate investments into schools from investments in university education. However, it would be difficult to compare such analyses between countries it different countries use different classification schemes for public funds.

In addition, it would be desirable to capture funds that actually have been disbursed (i.e. actual expenditures) in a given year and not only allocated to a particular project; i.e. for large infrastructure investments, the public budget usually allocates total required funds for planning but disbursement may be several years later Availability of such data is countryspecific. There are two types of public budgets: ex ante planned public budgets for the next fiscal year, i.e. the budget approved by parliament for spending by the government. These may be public, but are only planning figures, i.e. maximum budget ceilings for spending and public debt. On the other hand, realized spending would be more appropriate, but rarely available. Given these caveats, local government spending may have to be aggregated over several years.

One of the major requirements for local government expenditure is similar to the one discussed above for SDG indictors at the local level and refers to geographical distribution. The measurement of local government spending for the geographic unit would require that spending data can be precisely allocated to each unit. In addition to detailed levels of local government spending data that are separated according to categories of spending, data would be needed on actual (not planned) national government expenditure corresponding to different geographical units. For example, if investment in hospitals in a specific county/municipality falls under national or federal government spending, ideally these expenditure/investments made at lower levels should add up to total national government investments. These data would allow a more robust analysis on how government expenditure contributes to the 2030 Agenda for Sustainable Development at the local level. This is because it would minimize the measurement errors and would allow policymakers to monitor the progress against different indicators and when necessary, adjust government expenditure to areas where progress is not sufficient or not in line with national targets.

3.3Models to evaluate SDG localization

In addition to having appropriate and reliable data disaggregated at local level, for example, the district, county and village level, it is necessary to carefully choose the appropriate modelling approach that allows to identify the causal effects as opposed to pure correlations or predictive relationships. In the absence of controlled intervention, analysis will mostly rely on secondary data available from household surveys with the same or very similar methodology to permit cross-country comparability of the estimates. In addition, using these surveys limits the number of SDG indicators that can be obtained because they are not designed for precisely measuring.

The above caveats limit the type of econometric models that can be used to evaluate the impact of local government expenditure on SDG outcomes. The baseline econometric equation is as following:

Where Yi,t is a vector of SDG outcome variables at the local level and Xi,t is past local government expenditure spending. Xi,t may be aggregate local spending or a combination of local spending on various expenditure categories, such as health and education.

While this model is simple to estimate, it presents various issues. First, it is highly plausible that some other factors may be driving local government expenditure and affecting SDG outcomes at the same time. Therefore, this would lead to omitted variable bias, thus violating the statistical properties of the estimator. Second, measurement would lead to biased and less statistically precise estimates. The extent of bias depends on the quality of data used to measure local government expenditure as well as SDG outcomes. As mentioned, both the construction of SDG indicators and local government expenditure require sufficient sample size, i.e. the number of geographical districts available to estimate the model.

While it is generically unknown which variables could be confounders, as a minimum there may be concern that past values of SDG indicators could have affected spending patterns, while expecting the past SDG situation to be correlated with the future SDG situation. Therefore, the previous model is augmented by including past values of SDG outcomes and other control variables that may influence current SDG outcome.

In the case study described in more detail in Section 5 data were used from two waves of the Demographic and Health Surveys 2017 and 2013, which allowed to control for past values of SDG outcomes and add control variables from 2013. The estimated model takes the following form:

where Xi,2014-2016 is the sum of all local spending over 2014 to 2016, i.e. Xi,2014-2016=Xi,2014+Xi,2015+Xi,2016.

The choice between using aggregate data and more detailed disaggregation of expenditures by sector is driven by how well the data source can be used to construct accurate SDG outcomes and local government expenditure data as well as the sample size. Having a small number of geographical units and estimating a model with several categories of local government expenditure or disentangling them according to temporal variation would result in less precise estimates. Summing up the local expenditures over several years helps to average out part of the measurement error.

In addition to linear models, it is possible to incorporate non-linearities. However, sample size might be too small to permit extensive specification modelling together with control variable selection. Controlling for confounders is a key priority from a statistical point of view.

If panel data is accessible, then the estimation of panel data models and difference in difference (DiD) models could shed more light on the impact of local government expenditure on SDG outcomes in specific geographical units. However these models are only feasible when both current and past SDG outcomes are measured in the same way. This requires that construction of SDG indicators over time is based on the same questionnaire, interviewing technique, survey methodology and sampling approach. The basic structure of these models consists in allowing that geographical units with different levels of local spending may differ in some unobserved way but that this difference is constant over time. In addition, if in time considerable effort has been made to collect better data on local government expenditure, there may be a limitation to using fixed effects approach for panel data. In these cases, it is possible to resort to a controlling-for-confounders approach described above.

Finally, with respect to observational data, the previous models examined the relationship between local government spending and SDG outcomes in the same geographical unit, e.g. district, county or village. The analysis could be extended to analyse whether there are externalities/spillovers to adjacent geographic areas. Spatial econometric models can be added to learn about spillovers to neighbouring districts, e.g. whether investments in education in some districts also affect other districts.

5.1Identification and measurement of SDG indicators

The 2013 and 2017 NDHS for the Philippines were used to estimate SDG indicators or their proxies at the provincial level, including three special areas (Pateros, Isabela City and Cotabato City).

Both NDHS employed a stratified two-stage design. The Philippines is divided into 17 administrative regions. Each region is further divided into provinces, which are divided into cities, municipalities and barangays. Barangays are classified as either rural or urban.

The 2013 NDHS was stratified by region and each region was stratified by rural and urban barangays. This resulted in 34 strata, and the survey was designed to provide representative results at this level. Within each stratum, enumeration areas were implicitly stratified by province. In total, the 2013 NDHS included 14,804 household respondents and around 16,437 women respondents. This resulted in an average of 176 household respondents per sampled province. However, because provinces were only implicitly stratified, the number of household respondents per province varies widely. Ten percent of sampled provinces include fewer than 39 household respondents and one province only includes 16 household respondents. Two of the smallest provinces were not sampled (Batanes and Camiguin). There are no estimates for Davao Occidental since it became a separate province from Davao del Sur only after sampling for NDHS 2013 was completed.

In comparison, 2017 NDHS employed a much larger sample than 2013 NDHS (27,496 household respondents), and the sample was stratified across a much larger number of sampling domains. Each province (80 provinces plus the new province of Davao Occidental) constituted a domain as well as 33 highly urbanized cities and three special areas (Pateros, Isabela City, and Cotabato City). This resulted in 117 sampling domains.

For several provincial-level estimates, more than one sampling domain were combined. On average, each province included 316 household respondents with 90 percent of provinces including more than 255 household respondents. The province with the lowest number of household respondents included 116 household respondents.

The 2013 and 2017 NDHS allowed to estimate the number of SDGs provided in Table 2. Although, local level poverty indicators for SDGs are already available for many cities and municipalities in the Philippines, they represent a very small proportion of 231 indicators used to monior the progress towards sustainable development.

For this analysis, we focus on 13 SDG indicators, for which the estimates across 74 provinces are provided in Table 3.

5.2Local government expenditure

To obtain data on local government expenditures, the authors used SREs published by the Department of Finance’s Bureau of Local Government Finance. The SREs include Total Current Operating Expenditures for each province and for each municipality in each province. Expenditures are categorized as (i) General Public Services; (ii) Education, Culture and Sports/Manpower Development; (iii) Health, Nutrition and Population Control; (iv) Labour and Employment; (v) Housing and Community Development, vi) Social Services and Social Welfare (vii) Economic Services; and (viii) Debt Services. Focus was placed on fiscal years 2011 to 2016. For these years, General Public Services typically accounted for the largest share of reported expenditures followed by Economic Services and Health, Nutrition, and Population Control; Labour and Employment account for the smallest share of reported expenditures.

5.3Applied methodology

The primary goal of the pilot study was to explore data availability in existing surveys and seek to provide proxy indicators that would reliably reflect official SDG indicators as well as to explore the relationships between SDG indicators and local government expenditures. While several estimation techniques were applied, these results are best understood within the framework of exploratory analysis rather than as a rigorous attempt to establish causal inference.

6.1Key recommendations