Using machine learning to make government spending greener

2.1Green spending and deforestation

Deforestation plays a critical role in the aforementioned planetary crises, as forests foster biodiverse ecosystems and combat climate change through carbon sequestration. This role, coupled with good data coverage and availability from Global Forest Watch on deforestation across the globe, made it an ideal target variable for the analysis, acting as a proxy for environment-related outcome indicators.

The goal of the analysis was to predict deforestation rates using data on spending and fiscal decisions. Lacking access to national governments’ historical and disaggregated budgets, we were limited to publicly available datasets. In this regard, ODA was an ideal proxy, comprising a completely transparent and public component of governments’ spending due to flows originating in donor countries. Furthermore, ODA flows are broken down by economic sector and environmental marker targeted [5].

Of course, ODA or government spending is not the only factor impacting deforestation rates. An all-encompassing model perfectly capturing the diverse components and their interplay affecting deforestation would be impossible to construct. However, there are characteristics which could make it more or less likely for ODA in a given country to have a greater impact on deforestation. In particular, large ODA inflows relative to gross national income (GNI) in a country increases the likelihood that a model based on ODA would be able to capture important elements affecting deforestation, underpinned by the knowledge that lower income or less economically developed economies are often tempted to exploit their natural resources and assets to act as a fiscal buffer [6].

These assumptions guided the selection of the six countries examined in the analysis; each coupled high deforestation rates with elevated levels of net ODA to GNI ratios. Figures 1 and 2 show the cumulative treecover losses, in percentage, since 2000 and net ODA to GNI ratios for the six countries, respectively. Each country had lost at least seven per cent of its 2000 treecover by 2020, ranging all the way up to 24 per cent for Madagascar. Over the period from 2000 to 2019, each country also averaged at least the low income country average for net ODA as a percentage of GNI, as in the case of Haiti, ranging up to more than four times that average in the case of Liberia. The following sections will briefly explore the specific environmental and fiscal circumstances of each country.

Figure 1.

Cumulative tree cover loss since 2000.

Figure 2.

Net ODA received as a percentage of GNI.

2.2Machine learning

Machine learning is an expansive field and the information presented in this section is only a casual introduction to the topic. Specific algorithms will be discussed in further detail in the next section. For more information on machine learning in general, see [36]. For a more comprehensive background, see [37] or [38].

Machine learning refers to the study and application of algorithms whose crucial characteristic is the ability to improve their predictive performance automatically through the use of data. I.e., no human has to explicitly program or give instructions to the algorithm in order for it to produce its predictions or output. This learning and improvement process is referred to as “training” the model using existing or historical data. Machine learning is broadly generalizable into two areas: supervised learning and unsupervised learning. Supervised learning refers to algorithms and applications where both model inputs and outputs are provided to the model in training. That is, the output of the model is an existing variable with data available at the time of training. An example of a supervised machine learning application would be predicting sales of a company based on its advertisement spending. Unsupervised learning refers to the situation where the target variable is unknown or undefined. This paper is concerned with supervised machine learning, as the application has a well-defined target variable, explained in Section 3.

The first step in a supervised machine learning application is defining and gathering input and output variables. Input variables are the data the model will use to try and predict the output variable. The output, or target variable, is the variable of interest. The model is then trained on data where both the input and output variables are present. In this way, the model can establish the relationship between input and output. The ultimate goal is to generate predictions of the output variable in cases where its value is not known. This is referred to as inference. However, before the inference step, it is important to assess the performance of the model to make sure that it can generate good predictions. For this, it is necessary to test the model and generate predictions on input data where the output is known. This should not be done on the same data the model was trained on, however. This can lead to the issue of overfitting, where a machine learning model may pick up on noise in the data and subsequently produce very accurate predictions on data it was trained on, but generalize very poorly to new data it has not seen before. To ensure against this, the model is typically tested on a test set, consisting of data the model has never seen before, i.e., data it was not trained on. In the test set, both input and output variables are present. In this way, the test input data can be used to generate predictions which can then be compared with the actual values of the output variable. Good accuracy between these test predictions and the actual values can then give a better indication of the model’s performance.

This process is perhaps best clarified by way of example. Say a company is looking to predict its revenues based on its advertisement spends. It has three advertisement channels, TV, radio, and online. It has five years of monthly data on how much it spends on each of these channels. These are the input variables. The output variable is monthly revenue, for which it also has the same five years of monthly data as the input variables. Keeping in mind that it needs test data to assess its model on, it sets aside the last year’s worth of data for testing. This 80-20 per cent split of training and test data is a commonly used rule of thumb ratio in machine learning. It then trains a model using the first four years’ worth of data, where the algorithm automatically determines the relationship between the input and output variables. The company then gives the trained model the last year’s input data, generating predictions for monthly revenue. These predictions are compared with the actual monthly revenue for the last year. If these predictions are sufficiently accurate, the company can finalize their model and begin using it for inference. With their trained model, they can now do things like scenario analysis, where they see how different advertisement allocations affect revenue in order to maximize the expected return of their spending, and revenue forecasting. For instance, if they have a good idea of the advertising budget available to them over the next year, they can then generate a forecast of revenues based on those figures. This is just one example of a machine learning application, but this approach can be generalized to many different situations.

3.1Data

Data for the analysis came from two principal sources: the Organisation for Economic Co-operation and Development (OECD) for information on ODA and aid activities, and Global Forest Watch for information on deforestation rates. Two separate series were collected by country from the OECD: ODA by economic sector [39], and aid activities targeting Global Environmental Objectives [40], a.k.a., aid activities by Rio Marker [5]. From Global Forest Watch, the ”global annual tree cover loss” series was used [17]. All series were published at an annual frequency and transformed to year-over-year growth rates. Data were transformed to growth rates for two reasons. First, to provide a clearer semantic interpretation of the data and improve comparability and interpretation across countries. Data in hectares needs to be contextualized to each individual country to be able to be understood. Is predicted deforestation of 50 000 hectares a lot or a little? For a country like St. Vincent and the Grenadines, this is a massive amount of deforestation, greater in fact than the size of the entire country, while for a country like Brazil, this represents less than 0.005 per cent of its land area. Data in hectares needs to be contextualized not only across countries, but also temporally. 50 000 hectares of deforestation is not a large number relative to the size of Brazil, but how does this compare to last year? Is the figure moving up or down, by how much? The growth rate transformation addresses both of these aspects. Second, the transformation improves the performance of many machine learning approaches by scaling all input and output variables to a smaller range. Hectares and dollars can occupy ranges from thousands to millions or billions, but growth rates tend to be much more constrained, rarely extending much beyond ± 300 per cent in the dataset. See section ”Scaling of data in neural network models” of [41] for more information on the impact of scaling data for neural networks in particular. The final dataset consisted of annual observations dating from 2006 to 2019 for 6 countries: the Democratic Republic of the Congo, Haiti, Liberia, Madagascar, Solomon Islands, and Zambia.

3.2Methodologies

Long short-term memory artificial neural networks (LSTM)

LSTMs are a variant of artificial neural network (ANN). Over the past decade, ANNs have garnered much interest due to their impressive performance in applications such as image recognition, autonomous driving, and natural language processing. ANNs are composed of interconnected layers of nodes. These layers receive input, transform that input by multiplying it by coefficients and running it through activation functions, before finally generating an output or prediction. Coefficients are randomized initially but are then adjusted via a process called back propagation. Back propagation involves defining a cost function, calculating gradients with respect to this function, and then adjusting the coefficients in the direction of minimizing error. In a feedforward multilayer perceptron (MLP), the simplest type of ANN, information flows through this network unidirectionally, from beginning to end. See [47] or [48] for more information on ANNs.

The unidirectional nature of feedforward ANNs means they are not always best-suited to deal with information with a temporal element. Recurrent neural networks (RNN) address this issue by introducing feedback loops to the network architecture. This makes them well-suited to applications with a time element, such as speech processing. For more information on RNNs, see [49] or [50]. RNNs can have trouble modelling long term temporal dependencies due to the issue of vanishing or exploding gradients [51]. LSTMs solve this issue by introducing a memory cell, along with an input, output, and forget gate. Gradients are subsequently able to travel through the network unchanged, allowing the LSTM to preserve longer term dependencies than RNNs. For more information on the architecture of LSTMs, see [52] or [53]. LSTMs have been shown to perform well in economic applications, such as in nowcasting global trade or US GDP [54, 55].

Figure 3.

Year-over-year deforestation rates in the Democratic Republic of the Congo. Note: results shown using the LSTM algorithm and ODA by economic sector input variables.

3.3Model training

In order to train and assess the models, the data were split into a train and test set. The train set consisted of data dating from 2006 to 2015, while the test set dated from 2016 to 2019. The logic of having a train and test set was to ensure the robustness of the models. A model may perform well on data it has been trained on, but may generalize poorly to new, unseen data. By evaluating the models on the test set, we can be sure that the model is not overfit to the train data and can perform adequately into the future on new data.

The test set was also used to determine hyperparameters. Hyperparameters are parameters which determine the macrostructure of a machine learning algorithm. This differs from the model’s coefficients, which are determined in the course of training or fitting the model to the data. Hyperparameters are set before a model is trained. In the case of a simple decision tree, hyperparameters may correspond to characteristics such as the max depth of the tree, i.e., the number of splits a tree can have. This can be unlimited, where splitting continues until all observations reside by themselves on a unique leaf, or can be limited to just two in the extreme, where the initial data is split into just two separate groups. Depending on the algorithm, hyperparameters can have a large impact on the predictive performance of a model. As such, they often need to be tuned. That is, define a cost function, such as mean absolute error (MAE) for a regression application, then train the model with various combinations of hyperparameters, and finally assess performance based on the cost function. The hyperparameters of the best-performing model are then used for the final model. In the ideal case, the train set is further split into a validation set for determining hyperparameters, but due to the low number of observations available in the data and the illustrative nature of the analysis the test set was used here.

In terms of input variables, three separate sets of variables were examined. The first corresponded to including only ODA by economic sector; the second to aid activities by Rio Marker; the third to the combination of both previous sets of variables. The best-performing models in terms of MAE and root mean square error (RMSE) for each country are presented in the next section.