Soul of a new machine: Self-learning algorithms in public administration

1.Introduction

The arrival of digital technologies in general, and that of self-learning algorithms in particular, has given rise to much scholarly debate about its implications for the public domain, e.g. with regard to policy analysis, policy making and governance (e.g. Danaher et al., 2017; Gil-Garcia et al., 2018; González-Bailón, 2013; Hilbert, 2016; Janssen et al., 2015; Just & Latzer, 2017; Katsonis & Botros, 2015; Kosorukov, 2017; Landsbergen & Wolken, 2001; Todorut & Tselentis, 2018; Yeung, 2018, and others). Digitization, as an overarching term for various techniques and methods, is often seen as a useful tool that could improve policy analysis and decision making. An oft-repeated argument is that digitization can unlock insights from big data sets at speeds unattainable for human operators. As such, it could simultaneously offer unprecedented access to information to guide governmental action, and make sense of a daunting abundance of data about the state of society. It could be used to model and simulate decisions to explore possible outcomes of certain policies (Janssen et al., 2015), or to predict certain societal dynamics (e.g. crime in public transportation, see Kouziokas, 2017) so that authorities can deploy their resources more efficiently. No wonder then, that great things are expected from such techniques.

However, there is a growing body of literature that demonstrates that there are severe and persistent issues with the use of big data and self-learning algorithms in public administration. As will be elaborated later in this paper: such algorithms are partial and subject to self-confirmation, which may have real and sometimes dire consequences (e.g. Bellovin, 2019). For example, software used in Florida to assess the likelihood of sentenced criminals to become repeat offenders only predicted 20% of such occurrences correctly, and it was particularly likely to falsely predict black defendants as future criminals (at a twice higher rate than white defendants), while severely underestimating the likelihood of white defendants to become repeat offenders (Angwin et al., 2016). Naturally, a self-learning machine may be corrected using additional search and classification rules to counter bias (e.g. Caliskan et al., 2017) but the development and successful implementation of such a technique is strongly dependent on many factors. Besides, bias is only one of the major challenges in getting self-learning algorithms to work in public administration. There are also many practical, operational, legal and ethical questions (e.g. Mittelstadt et al., 2016) surrounding those techniques when used in policy analysis and policy making.

There appears quite some optimism, and also some hype, about the promises of digitization and the use of self-learning algorithms in public administration (e.g. Agarwal, 2018; Corvalán, 2018; Keast et al., 2019; Maciejewski, 2017). The many serious concerns that are starting to surface in literature provide ample reasons for a more critical approach (see e.g. Wirtz et al., 2019, 2020). For example, one could reconsider the ways in which administrations are organized, and the ways in which administrators use digital technology (e.g. Giest, 2017; Lindgren et al., 2019; Veale & Brass, 2019), as well as consider new practices for transparent data management and review (e.g. Janssen et al., 2020; König & Wenzelburger, 2020). The logic behind such considerations is that the machine is becoming increasingly enmeshed in the practices of policy analysis and policy making. Consequently, it has a real, tangible effect on the formulation and execution of policies (e.g. Kolkman, 2020; Valle-Cruz et al., 2020). Indeed, any enmeshed technology has transformative capacities (Dolata, 2013), which implies that such a technology may gain agency in that enmeshment with policy making. This point is not novel in itself but is often discussed at a conceptual level. How it works within the machine itself is less articulated.

While there is a growing body of literature on what digitization could mean for policy making – as summarized above – and a vast literature on the technicalities of self-learning algorithms (e.g. Conway & White, 2012; Flach, 2012; Mitchell, 2013), there is scant literature that bridges the two realms (see e.g. Sun & Medaglia, 2019; Valle-Cruz et al., 2020). The goal of this article is to open the machine’s black box in order to understand and critically examine how self-learning algorithms gain agency by transforming raw data into policy recommendations that are then used by administrators, and how this impacts policy making. Such a critical reflection is necessary because scholars in public administration should not only get a better understanding the implications of those techniques, they should also gain an understanding of the machine’s main operations in order to understand how those techniques work. To this end, the paper follows the process of transforming large data volumes into policy recommendations. In doing this, I draw from various bodies of knowledge, most notably science and technology studies, information theory, and the sociology of technology. Mackenzie’s ethnographic studies of digitization and machine learning (most notably Mackenzie, 2015, 2017) are central to this argument. I attempt to link these various literatures to the study of public administration and public policy, in the hope to enhance those literatures with some of the principles of machine learning. The references may also be used as pointers for public administration scholars who wish to do more in-depth research in this direction.

I start with a brief explanation of what is meant by the enmeshment of the machine in policy making. Since ‘digitization’ is often used as an umbrella term, it tends to obfuscate various differences between related techniques. It is therefore necessary to discuss three main aspect of digitization, namely big data (as volume as well as diversity), algorithms (to sort, structure and synthesize data) and machine learning (to develop policy recommendations with a large degree of automated autonomy). The final part of the paper is dedicated to an in-depth and critical examination of how these three aspects impact policy making. I argue that these aspects lead to five major concerns that hamper the seamless enmeshment of machine and administration as advocated by certain authors. First, self-learning algorithms require dichotomization of data at various levels to produce output, which means that the machine transforms data before it gives a recommendation. Second, machine output is likely to be biased because of the way the algorithms were trained. Third, machine learning may lead to normalization, as such confirming that the machine was correct even if it has generated biased output. Fourth, machines may learn from data but unlearning it is much more difficult, even though such unlearning is important in generating more accurate production. Fifth, the machine can’t print intelligible output by itself. Taken together, these five points imply that humans are poor monitors when using machines in policy making. Machine learning may possibly have something to add to public administration but an uncritical embrace of the technology is not justified.

2.Enmeshment

Public policies are developed within networks of actors (e.g. Wachhaus, 2009). Actors in such networks can be individuals but more often concern collectives as actors such as a Ministry, a government agency or a stakeholder group (Klijn & Koppenjan, 2015). Digital technologies can also be considered as actors in such governance networks. The technology derives its actor quality not only from its own capabilities but above all from the way it interacts with other actors in the network – in the same way that e.g. a group of stakeholders can gain agency from its interactions with public officials. It is in networks that agency is created.

The idea that agency is a network attribute is articulated most prominently in Actor-Network Theory or ANT (Latour, 1991, 2005; Law, 1992; Venturini et al., 2017). Naturally, ANT resonates strongly in the digital age where social life has become increasingly dependent on all sorts of digital technologies (e.g. Bächle, 2016; Bellanova, 2017; Haque & Mantode, 2013; Schmidgen, 2011; Stanforth, 2007). Thinking in terms of actor-networks that include actors of any type (not just humans) is somewhat underrepresented in policy and governance theories (Ludmilla et al., 2014; O’Brien, 2015). In ANT, all technologies can have actor qualities, even simple technologies such as the hotel room key in Latour’s 1991 example. While above I wrote about digitization in broad terms, for the present purpose I focus on self-learning algorithms that are used to generate predictions. Since it encompasses a variety of computational techniques that are combined in order to achieve learning and prediction, I refer to this actor as the ‘machine’ in the network. Importantly, this goes beyond the use of computers for e.g. registering and keeping track of data (although data management can be part of the machine) or programs made to streamline public service delivery.

The example of predicting whether convicted people may become repeat offenders as given above serves to demonstrate how much agency a machine may achieve. In that particular case, it recommended which convicts could be considered eligible for parole, a recommendation that was usually followed-up. Thus, the machine developed agency. Another example concerns the predictions of livestock disease outbreaks (Kroschewski et al., 2006). Here, the machine calculates the likelihood of such diseases spreading from farm to farm. Using input such as contagiousness of the disease, density of the area etc., it recommends different scenarios. For example, it may recommend quarantining a farm or an entire region, or even the destruction of all animals in the infected area. As with the example of repeated crime prediction, the machine gains agency if its recommendations are acted upon. This is a recurring theme when it comes to the role of the machine in policy making, even though that role is still poorly understood (Janssen et al., 2020; Lindgren et al., 2019; Valle-Cruz et al., 2020).

The machine is an assemblage of hardware and software, of external input and self-generated learning mechanisms, of predefined schemes for structuring data and autonomous generation of recommendations. It is a set of different technologies that combine with human input and subsequent action to generate a certain outcome, such as a decision to turn down a request for probation or the decision to clear a farm of live-stock. This agency is characterized by invisibility (human operators at least partially unaware of how data is processed and recommendations are made) and impact (the machine’s output has an actual outcome on the real world). Although I won’t deploy the entire apparatus of ANT in this paper, I use the main idea as a search light to discuss how the machine operates and interacts with human operators to bring about policies. I focus on self-learning algorithms as arguably the most far-reaching role a machine can obtain in policy making. In literature, various terms such as machine learning, artificial intelligence and automation are often used interchangeably (see e.g. Etscheid, 2019) so there is a need to first clarify the main techniques and concepts, and to map how they relate (see also boyd & Crawford, 2012; Manovich, 2012). These are: big data, algorithms and machine learning. Of the three, big data may be the most loosely defined.

3.Big data

Big data concerns data that is not only characterized by its volume but above all by its unsorted type diversity as well as granular diversity. While conventional policy analysis works with theoretical frames and sets of assumptions about relationships between variables to collect and categorize key data such as census data or data for certain socio-economic key variables, big data sets are principally unsorted and lacking in predefined structuring (Manovich, 2012). In such large, diverse and unstructured data sets, each utterance is considered (potentially) valuable data and each piece of information forms a variable, regardless of its form (Mackenzie, 2015). The key to working with this daunting abundance of information is categorization, i.e. the sorting and labelling of every piece of data such that those pieces can become related through statistical operations. Since every piece of data becomes a variable, the entire data set forms a very-high dimensional space where countless pieces of data are related to countless pieces of data, i.e. many vectors are formed within this space (Mackenzie, 2017). The question which variables are to be related to others is determined in the statistics used (Hastie et al., 2009). Compare this with more conventional approaches to policy analysis, in which the data space is predefined by a limited set of variables and their relationships, usually in the shape of correlation. In big data, the properties of the vector space may, and usually does, change because of the sorting and labelling that takes place Big data spaces contain all contextual, indexical, symbolic or lived differences in data (Mackenzie, 2015). By implication, the data set is dynamic. That is: new data can enter or leave the space continuously. The new data is not merged with a given pre-defined causal structure. Instead, it may change the causal structure if the new data or the discarding of old data provide reasons for doing that. As such, new data becomes part of the vector space and the causal structure may change continuously.

It is important to note that the diversity of the data in big data sets doesn’t only stem from the nature of the input data and the continuous flow of new data (and discarding of data considered no longer relevant in the light of what has been ‘learnt’) but also from the juxtaposition of entire but seemingly incompatible data sets. The ‘remixing’ (Mackenzie, 2017) of different types of data and of various collections or sets of data is one of the key aspects of big data, as is the transformation of entire data sets when combined with other sets in various ways (Mackenzie, 2012).

The data itself is transformed upon entering the vector space. The machine operates by the grace of digitization. This requires data to become encoded into bits. The encoding essential for the operation of the machine but it also causes dichotomization of the data at the micro level (Mackenzie, 2017). Data that may appear as gradients to the naked eye needs to be cut up into discrete values before it can be put to work. Naturally, dichotomization of data itself is nothing new. Academic and policy researchers do it all the time whenever they decide that a certain observation falls into one category or another one. The same goes for policy makers when they try narrowing the complexity of real-world issues into categories that can be processed in bureaucracies, even if it is understood that such simplification violates the actual complexity of those issues (Boisot, 2004, 2006; Boisot & Child, 1988; Gerrits, 2012). The difference between such instances on the one hand, and dichotomization of big data on the other, is that the dichotomization of all data is a necessary step before it can be processed to form a vector space in the latter.

The dichotomization of data takes place at various level. At the micro level, it is encoded in bits. Above that level, each piece of data is classified into (emerging) categories (Mackenzie, 2017). The exact consequences of this dichotomization for encoding and vectorization are hard or perhaps even impossible to assess on a case by case basis. This is not much of an issue in instances of discrete data but becomes more pressing when the data is ambiguous and open to multiple interpretations. Data that is not easily classified is still forced into a category, or it may become a new category on its own. Either way, the ambiguities that are real to social data are hard to deal with in big data sets.

Exactly how data as variables in the vector space relate will emerge once sufficient data have been collected and labelled – which is why such data sets tend to be enormous. Naturally, it is considered impossible to sort those data manually and to discern patterns that matter. This where algorithms and machine learning come into play.

4.Algorithms

If data forms into vector spaces, algorithms can be said to inhabit those spaces (Mackenzie, 2017). In their very basic form, algorithms are nothing but if-then rules applied to the data in order to form the vector space by classifying, structuring and relating said data (Cormen et al., 2007), e.g. the rule that if new data appears similar (in properties) to data that is already classified, the new data will be classified in the same way. Machines may feature many of such algorithms. They can be complicated, and they can be combined at will. Many decisions in conventional policy analysis can be considered algorithmic, too, for example when all instances having a certain set of attributes are considered to fall under the scope of a particular rule. However, there are some important differences, in particular when it comes to the number and diversity of algorithms that can be combined, and the speed with which the data can be processed.

When it comes to the algorithms deployed in big data sets, a principal distinction can be made between reactive systems, i.e. algorithms that trigger an automated response; and pre-emptive systems, i.e. algorithms that utilize historic data to infer predictions about future behaviour (Yeung, 2018). An example of the first would be a speed camera monitoring car drivers on a road. Once someone drives faster than the pre-set limit, it will register that driver as an offender. An example of the algorithms that generate predictions – which is what I’m after in this article – would be an algorithm that sorts through (seemingly) unrelated data to establish vectors in order to predict an outcome. An example of this is China’s ‘Situation-Aware Public Security Evaluation’ (SAPE) platform that was developed for the prediction of terrorist attacks (Wu et al., 2016). This machine combines different data from different sources, including (but not limited to) money transfers that appear irregular in size and in sender-receiver patterns, and overseas calls by citizens with no relatives outside of China (Gallagher, 2016). This data is collected on a daily basis and the results are compared to similar patterns in such data preceding terrorist attacks, as registered in the Global Terrorism Database.11 The outcomes are tailored with the help of data from over 10.000 ‘public security events’ as registered by Chinese provinces, in order to account for regional differences. In fitting the curve to the data, authorities may be able to predict that people with certain characteristics are be more likely to engage in acts of terrorism.

The initial sorting of the data can be done manually in order to provide the machine with an anchoring point about what constitutes a fit. This is called supervised learning. A sample from an existing data set may be assessed, sorted and categorized by human operators, giving the machine some basis for the accurate processing of the rest of the data. The remaining sorting, categorization and relating of data is done by algorithms that become more able as more data is processed and checked against what has happened in the real world. A simple algorithm can be told to label all instances of a particular word in communications as a possible indicator for social security fraud, and another one to check if those words correspond with actual fraud as detected in the real world. The data can be matched to pre-defined data and the outcomes can be checked and adjusted in the light of known outcomes. Over time, the algorithms can be made to learn that certain instances in the data, and the way they relate, also co-occur with given outcomes. As such, there is not necessarily a need for continuous human oversight. If algorithms are capable of going through this entire process from sorting to predicting all by themselves, this is called machine learning.

5.Machine learning

Machine learning enables the machine to develop categories and labels for data all by itself, as such actively sorting and relating data without much prior instruction as how this should be done exactly. In other words, the machine will try out in what ways the best fit with real world outcomes can be created. This is called unsupervised learning. The basic principle of (unsupervised) machine learning constitutes a positive feedback loop. The data are labelled and related, and the outcomes are then tested to see if the sorting and structuring have generated the correct prediction. If not, the data will be related repeatedly until its output starts to approach known reality, i.e. fit has been reached. Once the resulting predictions are confirmed, the machine will be better able to sort new incoming data and entire data sets. In other words: the more a machine knows, the more it can know, i.e. generalization through mobilization (Mackenzie, 2015). The inclusion of additional data may improve the capacity of the machine to learn and to get better at sorting data and predicting outcomes. An evolutionary approach sees the machine pitching alternative, competing algorithms that label and sort the data in different ways and check their predictions against outcomes. The algorithm or combinations of algorithms approximating known reality the best will be kept and the other ones discarded (Salcedo-Sanz et al., 2014). Not only will this enable the machine to make better predictions, it will also become increasingly more efficient at making such predictions. It actively selects and shapes the algorithms that work the best, i.e. it is capable of enhancing its own learning capacities.

While this certainly looks impressive, there is no black magic involved in this process. Machine learning runs on a collection of known statistical techniques to do the labelling and sorting (Hastie et al., 2009). The apparent magic derives from the speed with which these enormous amounts of data are labelled, sorted, tested, and resorted and relabelled until they produce output that appears to get closer to reality. The important point is that it is impossible for human operators to track and trace how the machine traversed the highly-dimensional vector space in order to come up with a given output (Latour et al., 2012; Mackenzie, 2015; Mittelstadt et al., 2016). That is, the self-selection of algorithms on the basis of the machine’s learning curve is invisible to the human operator. In that sense, the machine is indeed a black box, the capacities of which are to be assessed by its output, i.e. its capacity to predict, but not necessarily by the way it achieves its predictive capacity.

The best way of telling that the machine has learned is by looking at its ability to generalize (Burrell, 2016). There are two issues with this generalization. First, the resulting model may adapt itself too closely to the current data set and subsequently fail to generalize (excessive fit), or may not be complex enough, subsequently representing too little and performing poorly in generalization (underfit). Again, this is not dissimilar from what administrators also do when they try to match real-world issues to predefined bureaucratic categories (Boisot, 1998) but the speed at which it happens is unmatched, and the impromptu flexibility imposed on existing categories is virtually non-existent in bureaucracies. Second, the learning works well as long as the object it is learning about remains more or less static. A static object allows the machine to fine-tune its model and to become increasingly good at making predictions. However, every change in the object of interest requires a new iteration and possibly a change of the predictive model. By implication, machine learning has a hard time keeping up with the complexity of social reality (Mackenzie, 2017). Naturally, this also goes for humans (Ang, 2011). The difference here is that machines can iterate at a much higher rate than humans can do. Regardless, Mackenzie and others are correct in saying we are still far off self-learning algorithms that respond adequately to the fluidity of human complexity.

6.Transformations

Following the process from the processing of raw data to policy recommendations, it appears that the data is transformed in many and profound ways before it reaches the policy maker’s desk. These transformations are non-trivial in the sense that they alter the lived experience into analytical and bifurcating units, which is not a difference in degree but a difference in kind (Savage, 2009). Data is dichotomized at the micro-level and vectorized in as many ways as is necessary in order to produce an output that appears sensible. The inclusion of different types of data in one data set (e.g. quantitative gradients vs. qualitative gradients, categorical vs. ordinal, etc.) requires transformations before these data can vectorized, can be made to relate. Aggregated and transformed data are restructured in all different types of arrays or schemata such as dendrograms, trees, scatter plots and NK-models. Every traversing of the (dichotomized) data, every production of curves approximating fit involves a transformation. As such, there is a considerable difference between the real world and the representation thereof as generated by the machine – even if its models and predictions appear to resemble social reality. It is this altered reality that is used for guidance in their policy making. Naturally, the machine may learn from its own mistakes by developing and then selecting competing algorithms for their best performance. But even that can be considered a process related only indirectly to human operations. There is no exaggeration in saying that there is considerable autonomy in, and subsequent agency of, the machine (Stampfl, 2013). At the same time, however, machine learning would be a dead artifact were it not for the ways in which it is deployed by human operators in general (e.g. Markham, 2013; Matzner, 2019; following ANT), and by administrators in particular (e.g. Bellanova, 2017).

Admittedly, the discussion above only scratches the surface of how the machine operates in generating predictions on the basis of self-learning algorithms because I can only cover a tiny fraction of the techniques used. However, the main point is that transformations are real. The remainder of the article continues to follow the process and examines how that transformation links to practices in public administration, at which point the machine attains full agency.

7.The machine enmeshed in public administration

The use of computers in policy making goes a long way back. Early versions saw computers computing the input given by human operators, for instance to assess the possible effects of a certain policy measure given known facts as collected and structured by those operators (e.g. Kaufmann, 1968). Although becoming increasingly advanced over time, these models can be considered conventional in that the input is pre-structured in sets of variables and the relationships between them on the basis of the operators’ prior knowledge of the subject matter. In those instances, the computations are essentially passive. That is: the models produce outputs in exactly the same was they were told to produce. Algorithms are in place – otherwise there would be nothing to compute with – but they are not self-learning algorithms so it is not about machine learning and big data sets. The juxtaposition of those two make the difference between a machine that produces a complicated, but essentially traceable output, and a machine that produces outputs no longer (directly) traceable for human operators. In fact, machine learning also means that the type of output generated may change over time as new data enters the vector space.

Machines of the latter kind are becoming increasingly popular in administration and policy making, and the number of applications seem to grow year by year (see Yeung, 2018; for an overview). One could argue that any policy recommendation that is enacted by policy makers allows the machine to gain agency because it has an impact on the real world. Establishing this fact is a first step. In the following, I will examine that enmeshment more critically. There are five main critical concerns that come with this enmeshment.

First, the data transformations as discussed above mean that, contrary what is often believed, the machine is not generating a true representation. The dichotomization of data when it is classified and clustered can be a clumsy affair. For example, Ku and Leroy (2014) demonstrated that a human expert could be more accurate than a machine that was trained to generate automated classifications of anonymous crime reports. The machine struggled to see differences in between two types of crimes if the reports were highly similar in other aspects, which is that kind of ambiguity that a human expert (a crime analyst in this case) has no trouble in dealing with (Ku & Leroy, 2014). While the machine was faster, the expert was more precise.

Second, the machine is prone to bias (Kolkman, 2020). This can happen in both supervised learning, if the trainers confirm the bias knowingly or unknowingly, and in unsupervised learning. Prominent and pressing examples can be found in predictive policing. For example, Ferguson (2017) showed how an operator can distort the output of the machine if it is told to correlate poor neighbourhoods with crime rates. While certain neighbourhoods may co-occur with high crime rates, the actual dynamics that produce the crime rates remain invisible. All that the machine achieves is to make it seem as if the people living in that neighbourhood are more likely to commit crimes. This may also happen in unsupervised learning. The nature of self-learning algorithms is such that they need historical data to develop an explanatory model of the subject matter, thus confirming existing biases more than discovering new causal relationships. If the machine recommends police to patrol a certain area more heavily – therefore increasing the likelihood that a larger portion of all people arrested are from that area – the machine will train itself that it has made the correct prediction even if it is a heavily biased prediction. After all, self-learning algorithms are seeking increased fit and not a particular policy outcome, such as a just policing system.

Third, and following from the previous point, the machine’s predictions may lead to normalization of a situation because humans act upon the recommendations (Coglianese & Lehr, 2017). As such, there may be a convergence between human behaviour and machine-generated predictions (Mackenzie, 2015). While the machine itself runs on a feedback loop between the computation of predictions and the matching of those predictions to reality (establishing fit), there is a second feedback loop that runs between the generated recommendations and the conforming behaviour of humans. The predictive policing example given above illustrates this. Ultimately, all machine learning is geared towards ordering, transforming, and shaping unstructured data in such a way that it can detect patterns that would neither be visible to the naked eye nor accessible through conventional statistical methods used in isolation with more limited data sets (Mackenzie, 2015). Some of the obvious errors can be corrected (e.g. prohibiting the machine to use the label ‘ethnicity’ when traversing crime statistics), provided that the human operator can be vigilant enough. The keyword, then, is traceability (alternatively: followability; being intelligible). One can, and should, ask how machines arrive at their recommendation (Coglianese & Lehr, 2017) but this may be extremely complicated and in many cases impossible. The weak spot may not rest with the machine itself – it just does what it can – but in how humans interact with machines (Gross, 2013, 2015; Mcsherry, 2005; Pu & Chen, 2007). Even if the machine could share the reasons for its recommendation, there is no guarantee that human operators would be able to understand the reasons given.

Fourth, while much attention is given to how the machine can learn, ‘unlearning’ is considerably less developed (Bourtoule et al., 2019). The machine’s algorithms are trained on existing data sets. As mentioned before, these data bases may be dynamical with new data added continuously. But while older data can be discarded, the machine cannot stop knowing what it has gathered from those older data. That is, the older data may remain present in the shape of how the algorithms are trained even when the original data on which the machine was trained is no longer present. Among others, it implies that a request to pull data (e.g. under the General Data Protection Regulation (EU) 2016/697; or GDPR) does not mean that the machine has forgotten what the original data meant. This can be dealt with in various ways, ranging from discarding the machine’s algorithms and retraining them from scratch using new data, to marking data such that one can determine the ways in which algorithms were affected by that particular piece of data. All of those options are inconvenient and require considerable work. For example, marking data requires the operator to understand the importance of each data point in constructing the final model, which is a tall order in big data sets (Bourtoule et al., 2019).

Fifth, policy makers essentially deal with machines that do not know how to print an intelligible, followable output suitable for the human operator requiring that information (Norman, 1989). This is already an issue when the machine works with a crisp database (Beierle et al., 2003; Clancey, 1983; Puppe et al., 2013) but becomes even more complicated when the database is ambiguous and the information needs not clearly defined a priori (Mast et al., 2016), and the dichotomization is applied autonomously – as is the case with the machines described above. On top of that, the ex-post explanation is still an aggregate of various algorithms are human operators are unlikely to observe the machine working through each bit of data. There is ample evidence that humans perform poorly in the role of monitor. Getting the machine enmeshed has the advantage of analysing heaps of unstructured data that cannot be processed by humans alone. The disadvantage is that it induces passivity because humans will no longer actively be involved in structuring data and creating outputs. Such passivity impacts awareness to such an extent that humans may not be comprehend the output even if was produced in a comprehensible way (Dixon & Wickens, 2006; Endsley, 1995, 1996). Moreover, information is irretrievably lost if no initial attention is paid (Peterson, 1985) and humans have struggle to process complex information, regardless of how it is produced and presented (Gerrits, 2012).

The enmeshment of machines and human operators stem from the interaction between the both, where machines build on prior human knowledge, which then leads to real-world consequence, and, subsequently, more ‘learning’ on behalf of the machine. The five main concerns highlight how that enmeshment can also create warped realities in public administration. This means that an uncritical embrace of the machine in public administration is not warranted.

8.Conclusions: The machine and its administration

While the machine has already gained agency in policy making because of its autonomy in developing recommendations from unsorted data, we are still a long way off building a seamless mesh of humans and machines (Pantic et al., 2006). Some authors (e.g. Coglianese & Lehr, 2017) have argued that legal authority and accountability still rests with humans as they are the ones that make the actual decisions following recommendations by the machine. As such, the decisions are to be subjected to the usual requirements for sound decision making, including due diligence. From that perspective, the machine doesn’t change existing questions about transparency and accountability, it only adds a novel technical layer (König & Wenzelburger, 2020). But that is only half of the story. I argue that such legal aspects are difficult to uphold if the administrator can’t retrace the operations that lead to the machine’s recommendation. Certainly, an administrator can choose to follow or ignore a recommendation but won’t be able to state the reasons of ignoring or following it, outside of one’s own consideration regarding justice, fairness, representativeness etc. because the machine’s workings remain a black box. In other words, the fact that there are laws that apply to the administrator in order to maintain accountability doesn’t solve the problematic deus ex machina. As such, I fully agree with Valle-Cruz (2020) that administrations should not embrace machine learning uncritically.

While data can certainly make sense of itself, as per Anderson’s famous opinion piece (2008), this not necessarily produce sensible and traceable outputs that could be used for policy making. A machine to structure heaps of data into patterns by means of statistics but such patterns may not reveal actual truths, let alone present something that policy makers can follow blindly. Data volume does not equate objectivity, complete datasets are as difficult to deal with as are incomplete ones, and vectors in the data are not necessarily insights. Besides, contexts remain as important as ever. Last but not least, all machine learning needs to negotiate the same gap between intension (the attributes an object must feature in order to fit a concept) and extension (the class of objects referred to) that any policy research encounters. That is: more abstract concepts generate generic statements that apply to many cases but without specific detail, and more precise concepts generate more precise outcomes that cover fewer instances (Boisot, 1998; Toshkov, 2016). Consequently, its reliance on sheer volume may render the machine less competent in the face of complex reality.

Contemporary big data and machine learning repertoires can be useful in digging up patterns as long as the transformations that take place inside of the machine are understood. Ultimately, those transformations are simplifications of complex, real-world problems. The most pressing problem lies in the machine’s opacity – that derives from the self-learning and self-selecting of unsorted and highly diverse data – when rendering a recommendation that leads to a decision that leads to a material change. It can’t be expected from administrators to first develop deep knowledge about machine learning in order to work with the recommendations that the machine has generated. Likewise, it can’t be expected that the machine will function as an accountable (and perhaps even better) partner in administrative decision making (Hofstetter, 2014). Of course, the machine will become rather more enmeshed than less, which implies that the challenges identified here are likely to become more prominent in the future. The positive feedback loop central to machine learning and the normalization of situations when machine learning is enacted in actual policy analysis and policy making means that scholars are looking at a reality partially generated by the machine itself. With the machine in the loop, reality becomes recursive. Administrations better be prepared for this.