The Barcelona declaration for the proper development and usage of artificial intelligence in Europe

1.Motivation

2.The declaration

This section reprints the complete text of the declaration. A summary is provided in Table 1.

Barcelona declaration for the proper development and usage of artificial intelligence in Europe

1. Scope. AI is a collection of computational components to build systems that emulate functions carried out by the human brain. The field started in the mid-nineteen fifties and has since gone through cycles of promise, enthusiasm, criticism and doubt. At this moment we see a strong wave of enthusiastic adoption of AI in many areas of human activity.

We distinguish between knowledge-based AI and data-driven AI.

Knowledge-based AI has shown to be most successful in intellectual tasks, such as expert problem solving, whereas data-driven AI is most successful in tasks requiring intuition, perception, and robotic action. The full potential of AI will only be realized with a combination of these two approaches, meaning a form of hybrid AI [12].

2. Investment. The current surge of interest and application of artificial intelligence (AI) is without precedent. There is a growing consensus that AI is of huge importance for the future economy and functioning of European society. AI is now understood to be a powerful, novel way to link producers and consumers, and a novel way to add value to products, build new ones, and improve production processes. Moreover AI can help to introduce more efficiency and quality into bureaucratic procedures and give greater access to knowledge and creativity for all. We therefore call upon European funding agencies and companies to invest in the development of AI at a scale which is adequate for the challenge, and in such a way that ALL European regions and citizens can profit. This investment should target the creation of a complete ecosystem with a network of high-end research labs with sufficient structural (as opposed to project-based) funding, diffusion of AI techniques to form a significant number of ‘AI engineers’, and proper conditions and stimuli for successful AI entrepreneurship. Of particular importance is the development of open resources, such as corpora, ontologies and software frameworks, that should be available as the common infrastructure on which specific applications get built. Because many of these resources are specific to individual languages and cultures, it is important that Europe invests in their development, partly to make applications accessible and adapted to all European regions. Europe currently lags behind other economic areas in the investment in AI and the time for a very significant scale-up is now.

3. Prudence. The leap forward in AI has been caused by a maturation of AI technologies, vastly increased computing power and data storage, the availability of delivery platforms through the internet, and an increased willingness of many economic actors to try out the technology for their own application domain. But we must be aware of the limitations. Many fundamental problems of artificial intelligence are not yet solved and will require radical breakthroughs. Solving them will require substantial long-term research efforts. The application of AI also demands very stringent prerequisites; otherwise the results will be disappointing and potentially very harmful. For example, the application of knowledge-based AI requires the availability of human expertise and sufficient resources to analyze and model it. The application of data-driven AI requires enough high quality data and careful choices of which algorithms and parameters are appropriate in each case. These application prerequisites need to be investigated and spelled out in much more detail so that those responsible for applying AI can exercise the necessary prudence.

4. Reliability. All artificial systems that are used in our society have to undergo tests to determine their reliability and security. So it is normal that the same is done for AI systems, particularly in domains like medicine or autonomous robots. Although verification and validation procedures have been developed for knowledge-based systems in the nineteen-eighties and nineties, they are still lacking for data-driven AI. Sure, at the moment machine learning practices make a distinction between an example data set used for training and a test set used to gauge in how far a system has reached adequate levels of performance, but there is still a significant difference between a test set and actual testing in real world conditions. Moreover, once adequate verification and validation methodologies are available, we will need a network of agencies in European countries (or a central European agency) that use them. They should become the authority to validate AI applications before they are put into widespread usage. The European Parliament has recently decided to create an agency for robotics and AI and this agency could potentially take up this task.

5. Accountability. When an AI system makes a decision, humans affected by these decisions should be able to get an explanation why the decision is made in terms of language they can understand and they should be able to challenge the decision with reasoned arguments. This is particularly important in domains such as decisions on loans, legal decisions (for example about granting parole55), insurance, taxation, etc. AI systems, particularly those based on data-driven approaches, are currently unable to provide this kind of explanation. Their decisions are based on a large set of statistically derived network parameters. Techniques to make sense of these parameters are in their infancy and will probably require a combination of knowledge-based and data-driven AI. Nevertheless we should not allow widespread application usage without a solution to the accountability problem, and accountability should become a precondition for deployment.

6. Identity. There is a growing worry about AI chat-bots or other kinds of automatic messaging systems operating on the Internet and in social media, designed for the manipulation of political opinion, disinformation through the propagation of false facts, extortion, or other forms of malicious activity that is dangerous for individuals and destabilizing our society. These chat-bots pretend to be human and do not give away the identity of those behind them. The use of AI has made these chat-bots sufficiently realistic that unsuspecting users are not able to make a distinction and get mislead. A possible solution to this issue is to demand that it is always clear whether an interaction originates from a human or from an AI system, and that, in the case of an artificial system, those responsible for it can be traced and identified. This solution could possibly be implemented by a system of water marking and become mandatory in Europe.

7. Autonomy. AI systems have not only the capacity to make decisions. When they are embedded in physical systems, such as self-driving cars, they have the potential to act upon their decisions in the real world. This understandably raises questions about safety and about whether autonomous AI will not at some point overtake humans. Although some of these worries belong more in the domain of science fiction than reality, the proper circumscription of autonomous intelligent systems is an important challenge that must be addressed. It is necessary to have clear rules constraining the behavior of autonomous AI systems, so that developers can embed them in their applications. It is also necessary to clarify who is responsible for failure – as is indeed the case with all products.

8. Maintaining Human Knowledge. The undeniable enthusiasm for AI gives sometimes the impression that human intelligence is no longer needed and it has lead some companies to fire employees and replace them by AI systems. This is a very serious mistake. All AI systems critically depend on human intelligence. Knowledge-based systems model the knowledge and insight of human expertise and data-driven AI systems rely critically on data of human behavior. It follows that human expertise should continue to be taught, developed and exercised. Moreover in almost any area, human expertise still far outstrips artificial intelligence, particularly for dealing with cases that have not appeared in the example data sets from which AI systems are learning.

We believe that AI can be a force for the good of society, but that there is a sufficient danger for inappropriate, premature or malicious use to warrant the need for raising awareness of the limitations of AI and for collective action to ensure that AI is indeed used for the common good in safe, reliable, and accountable ways.

Barcelona, 8 March 2017

The list of current signataries is available through the website:

https://www.iiia.csic.es/barcelonadeclaration/

It is still possible to sign the declaration through the same site.

3.Follow up

After the event in Barcelona, the declaration was spread through various AI research channels and public media. It was integrated in various discussions on the future governance of AI in Europe, for example, at a hearing in Brussels of the EU political Strategy Center.66 The declaration was also discussed at various AI schools and fora, such as the 2017 edition of the Interdisciplinary College IK in Guenne, Germany.

In general, the declaration contributed to raise awareness and has given additional impetus to initiatives by governments and law makers in many European countries, such as the Netherlands [20], Belgium [23], Denmark [24], the UK [1], a.o. In some cases, the recommendations of the declaration were explicitly referred to in parliamentary hearings [13]. In addition, the European Commission initiated in the spring of 2018 a High-level Expert Group on Artificial intelligence77 as a steering group of the newly formed European AI alliance,88 which includes stakeholders ranging from industry to policy makers and academics.

So, although the landscape of AI in Europe is rapidly changing through all these discussions and activitites, the issues raised in the declaration remain highly relevant. The remainder of this section highlights some of them.

1. There is an even greater need today to clarify what we mean by AI when discussing legal and ethical issues. The first item of the declaration was intended to circumscribe the field. This has become even more urgent. At the moment AI is increasingly being used as an umbrella term for a wide range of techniques that used to be classified under operations research, pattern recognition, information retrieval, data analytics, business modeling, statistical analysis, etc. The term data science is currently used for these activities and is indeed much more appropriate. AI is also used for developments in digital media, particularly social media, even though its role is in many cases non-existent.

When AI is interpreted too broadly like this, there is a risk, for the field of AI itself, to be blamed for malicious applications, business practices or societal phenomena, such as fake news, hate speech or cyber crime, even if no AI is involved at all. The declaration therefore proposed to focus only on ethical issues as related to AI in the narrow sense. And even if we maintain this restriction, we need to be more precise whether we are talking about knowledge-based AI or data-oriented machine learning, because the legal and ethical issues for both approaches are quite different. For example, the topic of explainability was already an important component of knowledge-based systems built in the 1980’s and adequate approaches have been developed and used extensively [16], whereas explanation is highly problematic for current machine learning techniques such as deep learning and it is still very unclear how it could be achieved [2].

2. The plans for supporting the development and deployment of AI still have to become concrete in Europe. Since march 2017, the call for greater and above all more stable investment in European AI, as proposed in the second recommendation of the Barcelona declaration, has been ‘heard’ in several European countries, giving rise to considerable optimism. The Macron 1.5 billion Euro action plan for AI in France [26] has already been mentioned, but there are also plans being drawn by the Merkel government in Germany (anounced for autumn 2018), and by several smaller countries. Moreover in april 2018 most of the member states of the European Union have signed a Declaration of Cooperation on Artificial Intelligence (AI) in order to combine and coordinate their efforts.99

All of these initiatives are very welcome but they are statements of intent and concrete actions with a direct impact on the deployment of AI, or, just as important, on the creation of stable funding for AI research and education in Europe are still rare. One positive sign was a first 20 mi EU call within the European H2020 framework program (with deadline march 2018) that explicitly targeted the stimulation of European AI research in line with what was proposed in the Barcelona declaration, namely the creation of a European ecosystem and a platform in which European actors could share resources in the form of machine learning algorithms, data sets, knowledge bases, ontologies, lexicons, grammar models, etc. The AI4EU consortium has been selected and activity is planned to start in January 2019. Moreover Cecile Huet from the Future and Emergent Technologies office (DG CNECT) outlined the Artificial Intelligence Strategy for Europe at IJCAI-2018 in Stockholm, which foresees an important increase of opportunities for AI research (70%), mainly because a large number of calls will include the possibility to integrate AI. Of particular significance is that now “AI-oriented proposals in any basic or applied research domain are welcome” for the ERC calls. Beyond the Horizon 2020 program, it is foreseen that the next Multi-Annual Financial Framework earmarks 2.5 billion EU for AI.

3. We need more rather than less prudence with respect to what AI can do. The current hype in AI is largely fueled by anouncements and demonstrations by American tech companies, by web-based blogs and news media that amplify expectations or prototypes, by very rapid publishing (for example through blogs or arXiv) without the reviewing process characteristic of normal scientific communication, and, to some extent, by a lack of cognitive science background by newcomers to the field, who appear unaware of the enormous complexity of human knowledge and knowledge processing and therefore heavily underestimate the challenges of ‘real’ AI. Moreover there is the old AI disease of reading too much into the performance of an AI system, for example assuming that an artificial agent has intentions like deception, whereas there is only the appearance of deceptive behavior to the human observer without any explicit intentional goal to deceive by the agent.

A typical example is a recent hype episode about an experiment carried out by Facebook researchers on the acquisition of skills in negotiation, with the acquisition of language skills as a secondary needed competence [15]. The paper was published on arXiv which ensures rapid dissemination in the machine learning community, and on a company blog,1010 which ensures that the experiment is picked up by the media. So far so good.

However, a report in the media, on the website Fast Company, did not discuss the negotiation experiment itself but focused entirely on the acquisition of language skill with the headline: “AI Is Inventing Languages Humans Can’t Understand. Should We Stop It?”, commenting “Researchers at Facebook realized their bots were chattering in a new language. Then they stopped it.”1111 Indeed, non-English dialogs started to be produced such as this one:

Bob: you i everything else....

Alice: balls have a ball to me to me to me to me to me to me to me.

Although the researchers never mentioned anything about stopping the experiment for this reason.

The phenomenon of novel language emergence is in itself interesting, particularly to those in the AI research community that have been studying for decades the cultural evolution of language through a wide range of agent-based experiments, including with embodied robots [22]. But then, web media, blogs, and newspapers picked up the theme of self-generated language and elaborated only on the potential dangers with stories that became progressively more and more scary. For example, the UK tabloid The Sun reported: “Facebook shuts off AI experiment after two robots begin speaking in their OWN language only they can understand” and quotes Kevin Warwick as “anyone who thinks this is not dangerous has got their hand in the sand”. The Sun adds: “The incident closely resembles the plot of The Terminator in which a robot becomes self-aware and starts waging a war on humans.”1212 Newspapers all over Europe picked up the story, all adding their own twists and exagerations and confronting AI researchers with this supposed step towards AI disaster and urging politicians to stop this madness.

The Facebook researchers involved surely did not intend this media storm but these stories are the ones that stick into the public understanding of AI. Clearly much greater care needs to be taken in communicating AI experiments. Otherwise the lack of prudence will without doubt lead to a new AI winter as the expectations and scare stories currently being created by overzealous media are impossible to fulfill and they overpower the more reasonable statements that most AI researchers tend to make.

4. No mechanisms for certification of AI systems have been put in place yet. The European parliament already voted in early february civil law rules on robotics, which also are highly relevant for AI and recommended the European Commission to set up an agency that would be tasked with certification.1313 But certification of AI is easier said than done. There are already some small-scale European initiatives exploring whether specific AI algorithms1414 are performing as they are claimed to be and there are various initiatives for value-based design, some reflecting the conclusions [7].

All this work builds up possible experience that can lead to certification procedures but there is still a considerable road to travel before this will be as uncontroversial as certifying a new refrigerator or new medicine.

5. Accountability through explainable AI and legal personhood. Accountability of AI systems continues to be a cause of great concern. It covers two aspects: being able to get an explanation to understand how a decision was reached, and being able to identify who is ultimately responsible for a mishap. Accountability should not be relegated to regulations to be applied once the AI system has been deployed, instead it must betaken into account at design time. Explainability is necessary not only because transparent explainable AI systems allow users to trust the systems, but also because it is a crucial element for accountability. Indeed, explanations should not just be traces but justifications. A type of justification could be, for instance, a contrastive explanation. That is, answering the question ‘why output A instead of output B?’. Relevant explanations allow to inspect why and how an AI system came to its conclusions and to locate possible errors and biases in the design of the systems. Besides being contrastive, explanations should also be selective, that is they should focus on the most relevant features that led to the output. Such rich explainability scenarios will require the capability of counterfactual reasoning [19], and are not only very much needed in data-driven AI, due to its black box nature, but also in knowledge-based AI.

6. The Identity of demonstrated AI systems keeps getting blurred. The sixth recommendation of the declaration argued that AI systems should make it very clear that they are artificial rather than human. The Turing test is in that respect misleading – because it suggests that fooling humans into believing that an artificial system is indistinguishable from a human is the ultimate goal of AI. It is not the goal of AI and should never be. AI can be useful without this kind of deception, particularly because sooner or later human users detect the deceipt anyway.

Here is a recent example where this recommendation has been violated. In may 2018 Google demonstrated a speech understanding system called Google Duplex that is claimed to be able to hold a conversation over the phone for ordering a reservation in a restaurant or make similar appointments for services [14]. This result is quite interesting although the boundaries of the system’s performance are not very clear and some have even questioned whether the demo was in real circumstances. But the fact that Google Duplex tried to con humans into believing that the conversation was not with a machine, created an immediate backlash and a promise that in the future the system would identify itself.1515 Moreover some observers were quick to realise the abuse that could be made of this technology. Here is for example a typical reaction (posted on the Google Blog anouncing the Duplex demonstration).

A possible response to avoid these problems is to legally require that any AI system should make it explicitly clear upfront that it is an artificial system, so that human users can also shield themselves from calls by such systems.

7. Most issues related to the autonomy of AI remain open. The question how much autonomy should be given to an AI system is for many applications, such as weapon technology or autonomous cars, of primordial importance. There are two avenues with which this issue is being approached. The first one is to create rules of governance and a legal framework that is both a guideline for developers and a mechanism by which those impacted negatively by the technology can seek redress. There is a considerable amount of recent work in this area with very specific proposals being developed in several European countries. For example, the German Ministry for Transport and Digital Infrastructure has already issued a set of ethical guidelines for the design and deployment of autonomous cars which is now legally binding [8]. The main points are: (i) Autonomous driving is ethically not only justified but obligatory because it can lead to a general decrease in the number of accidents. (ii) Saving human life has always a higher priority than avoiding material damage. (iii) There should be no discrimination with respect to saving life. (iv) It must always be clear who is responsible, human or machine. (v) All data must be stored to clear up future misunderstandings and these data must be accessible and under the control of the driver. A similar initiative in France is expected to issue guidelines by 2019. But it would obviously be better if there is a European initiative on this matter to avoid divergences between member states. The second avenue is to integrate moral decision-making and legal rules in the behavior of the AI system itself. Here also there is a considerable history of prior discussion [27] and occasional technical work. [9] that requires however much deeper exploration before being usable in practice.

8. Maintaining Human Knowledge. AI systems critically depend on human intelligence but also humans can greatly benefit when teaming up with AI systems. The development of AI should be human-centered, that is we should shift the focus from machines replacing human workers to tools, assistants, and in the long term peers that will help, complement and leverage humans in performing tasks and taking decisions leading to better results and higher quality of jobs. To achieve the ‘peer’ level of collaboration, humans and machines will have to share goals and cooperate synergistically towards their fulfillment. Synergy is due to the complementary strengths of humans and machines. Humans, for instance, are much better than machines at adapting to unforseen changes when performing a task and dealing with unexpected and uncertain situations in general. AI systems are better than humans in aspects such as recognizing patterns, memorising and analysing large amounts of data and information. There are already many examples that show the synergetic advantage of human-machine co-working: An excellent example is the work Combining Deep Learning with a human pathologist for Identifying Metastatic Breast Cancer [28]. In this study, based on several hundred cases, a top human pathologist achieved a percentage of error of 3.4%, a deep learning system an error of 7.5% but the combination of both reduced the error to only 0.52%. Therefore, the emphasis should be on how machines and humans can be co-workers instead of machines replacing humans.

On the other hand, the discussion on automation and employment is erroneously centered on only the number of jobs. Instead it should be focused on the changing nature of work. Some tasks have been, and will continue to be, automatized but the number of jobs where the majority, or all, of its tasks can be automatized is not as large as some studies say. There are studies in different European countries that show how robotization has in fact increased the global number of jobs, creating jobs with higher quality and better paid. On example is a study in Catalonia done at the Catalan Open University with several hundred SMEs from 2002 to 2014 [25].

A similar study done by the Centre for European Economic Research in Mannheim, Germany, also shows that automation resulted in an overall increase in (better paid) jobs in Germany between 2011 and 2016. This does not mean that we should not be concerned by the effects of AI and automation in general on employment (not only robots in manufacturing but also clerical work and professional services) but we should focus our concerns and find solutions to the problem of the changing nature of jobs, train workers to face this challenge and ensure that automation does not increase inequality in society.

4.Conclusions

This paper contributes to ongoing discussions in Europe related to the ethical issues of AI. We focused on the ‘Barcelona Declaration for the Proper Development and Use of AI’, which was launched in the spring of 2017, and discussed some of its ramifications. Given the public interest in AI and the eagerness of many organisations, both private companies and governmental institutions, to develop applications that affect people in their daily lives, it is important that the AI community, encompassing application developers as well as researchers, engages in open discussions, partly to avoid over-expectations with an unavoidable backlash later and partly to avoid improper usage of AI that causes unneeded negative side effects and undue human suffering. At the same time, we must realize that no set of rules or in-built technological constraints can ever avoid malicious use by unscrupulous actors. The ultimate responsibility always lies with humans, both as designers and as users, and they should be held accountable.