Open government data (OGD) can enable outbound open innovation (OI) that is beneficial to society. However, innovation barriers hinder OGD users from generating value. While previous studies have detailed a large number of such barriers, little is known of how different types of OGD users are affected, and when the barriers appear in their innovation processes. To this end, this paper describes a case study of distributed service development in the Swedish public transport sector. The contribution to extant research is twofold. Firstly, based on an inductive analysis, three OGD user archetypes are proposed: employees, entrepreneurs and hobbyists. Secondly, the study finds that the significance of distinct innovation barriers varies across phases of the services’ lifecycles and depending on the OGD users’ motivation, objective, pre-conditions and approach. Drawing on these insights, we propose that OGD initiatives aimed at facilitating outbound OI to a greater extent should address the barriers that appear during diffusion of innovations, the barriers that are not directly related to the OGD provision, and the barriers that are experienced by non-obvious OGD user groups.
Open data has been defined as “publicly available data that can be universally and readily accessed, used and redistributed free of charge” (Verhulst & Young, 2016, p. 5). The attention to how public organizations’ release of such data can trigger socially beneficial service development is on steady increase. The rise of the approach rests on two key movements (Huijboom & Van den Broek, 2011). Firstly, the release of open government data (OGD) tallies with growing inclusion philosophies such as new public governance (cf. Osborne, 2006) and openness (cf. Peters & Britez, 2008), which sees citizens as co-producers of public policies and demands public access to official information. The availability of OGD is for example believed to increase government transparency and trust (Bertot et al., 2010) as well as citizen participation and collaboration (Neuroni et al., 2013). Secondly, the OGD movement is also related to the idea of data as a tremendous resource that yet is largely untapped (Dietrich et al., 2009). Provision of OGD is believed to create prospective for vast economic growth (Borzacchiello & Craglia, 2012), with supposed effects such as new economic opportunities and increased solving of public problems (Verhulst & Young, 2016).
However, a large share of the anticipated societal value of OGD is still yet to be realized (Davies et al., 2013). Both OGD providers and users are facing barriers, hindering sustainable innovation ecosystems to emerge (Zuiderwijk & Janssen, 2014). While extant research has identified multiple impediments for both OGD release (e.g. Barry & Bannister, 2014) and adoption (e.g. Janssen et al., 2012), studies examining barriers during later phases of the service lifecycle are scarce. Hence, little is known about what obstacles OGD users (also referred to as third-party developers and external innovators in literature) encounter in their attempts to deploy services from OGD. Indeed, “understanding data users‘ needs and nurturing their engagement to drive value creation still does not appear to be a priority for many governments” (Ubaldi, 2013, p. 42). Furthermore, how effects of innovation barriers vary across different types of OGD users is poorly understood. In line with recent calls for studies on links between OGD user types and the effects of OGD use (Safarov et al., 2017), we argue that OGD-related research must seriously consider the heterogeneity among OGD users. Users are not homogenous in terms of for instance resources and knowledge. In fact, a diverse crowd can generate a wider variety of solutions, and removing barriers to non-obvious actors is an important design issue in crowd-based innovation practices (Jeppesen & Lakhani, 2010). Ubaldi (2013) notes that understanding and fostering data use, rather than merely providing access, is critical to create value from OGD. Better knowledge in this area could, among other things, enhance prospects to boost, predict and govern the outcome of OGD initiatives aimed at stimulating innovation from distributed OGD users. Accordingly, this paper explores the following research questions:
(i) RQ1: How do innovation barriers affect the use of OGD in different phases of the service lifecycle?
(ii) RQ2: How do perceptions of innovation barriers vary across different types of OGD users?
We address the research questions through an exploratory case study of 19 OGD users’ efforts to develop and operate services based on public transport data provided through an OGD marketplace (cf. Smith et al., 2016a) in Sweden. By qualitatively analyzing interview transcripts, we first identify what types of barriers they perceive as impeding. Then, in order to distinguish differences across OGD user types and across service lifecycle phases, we use an inductively developed categorization of OGD users and a circular model that describes five generic phases of service lifecycles as analytical tools.
In the following, a brief overview of related research is first provided in order to position the paper, to introduce the utilized analytical tools and to enable a discussion on how the findings in the paper contribute to extant knowledge. Next, the research setting and method are described. The perceived innovation barriers in each phase of the service life cycle are then outlined, followed by portrayals of identified OGD user archetypes and their experiences of innovating with OGD. Subsequently, the paper discusses implications for ensuring effectiveness and sustainability of OGD initiatives aimed at catalyzing distributed innovation . Finally, the last section concludes.
2.1Non-pecuniary outbound open innovation from OGD
OGD is the subset of open data that is produced by publicly owned organizations. Two distinctive features make the release of OGD particularly promising. Firstly, public organizations create and collect vast amounts of data in many different domains (Janssen et al., 2012). Thus, they hold potential to single handedly create a large solution space for innovation. Secondly, as an integral part of public operations, taxpayers have paid for the creation and collection of OGD. Accordingly, compared to other forms of open data, OGD constitute a shared resource that offers value beyond what is captured from the original intended use (Jetzek et al., 2014).
While acknowledging that OGD also is used directly (cf. Davies, 2010), we argue that services or products that transform the data into practical knowledge is a cornerstone in making OGD useful for citizens. Thus, in our view, aggregation and refinement of OGD is required to reap the proposed societal value. We consider this activity as innovation, that is “an intentional and proactive process that involves the generation and practical adoption and spread of new and creative ideas, which aim to produce a qualitative change in a specific context” (Sørensen & Torfing, 2011, p. 8). Further, we also argue that fruitful cultivation of OGD implies that public organizations must look beyond their organizational borders and involve external actors in the innovation process. Hence, we adopt an open innovation (OI) lens to position the case study.
Chesbrough (2003) identified a paradigm shift away from closed innovation models, and denoted the movement OI. He moreover summarized its emphasis as “the use of purposive inflows and outflows of knowledge to accelerate internal innovation, and expand the markets for external use of innovation, respectively” (Chesbrough, 2006, p. 2). Gassman and Enkel (2004) explored three types of openness: inbound, outbound and coupled (see Fig. 1). In the inbound form, the focal organization opens its organizational borders to purposively insource external ideas and technologies in order to enhance its products’ values. In the outbound form, the initiator instead outsources its internal resources, and allows external actors to refine, exploit and bring them to market. The third form is labeled coupled. It is a combination of the inbound and outbound processes, often characterized by the formation of enduring alliances with complementary external partners. Dahlander and Gahn (2010) moreover distinguished between pecuniary and non-pecuniary forms of OI, that is whether or not the initiator seeks to reap immediate financial rewards or indirect benefits.
The release of OGD to accelerate innovation from distributed OGD users is grounded in the strategy of revealing internal resources to the external environment. These internal resources are generally made accessible without any immediate financial compensations, but with the hope of indirect benefits. Therefore, we consider such practices as non-pecuniary outbound types of OI. Accordingly, we position our case study as a non-pecuniary outbound OI in the public sector, focusing on the barriers that OGD users experience when innovating with OGD.
2.2Barriers to OGD driven innovation
Although outbound OI from OGD is promising and theoretically sound, the practical impact so far remains limited. Both OGD providers and users face a number of barriers in generating value. Extant research on open data has often either focused on barriers for providers, users or the role of intermediaries in mitigating them. For example, Barry and Bannister (2014) propose a typology of barriers for data providers, Zuiderwijk et al. (2012) identify 118 impediments and emphasize how open data policies neglects users despite them being the actual value creators, Kaasenbrood et al. (2015) and Maccani et al. (2015) both identify factors influencing the adoption of open data by private organizations, Hjalmarsson et al. (2014) provide a framework for barriers experienced in relation to OI contests, and, Smith et al. (2016a) as well as Jetzek (2016) both suggest design considerations for open data intermediaries. In a comprehensive working paper, Ubaldi (2013) moreover shows that successful open data initiatives require a holistic view. The paper lists policy, technical, economic and financial, organizational and cultural, as well as legal challenges.
For the purpose of this study, the classification of innovation barriers suggested by Jansen et al. (2012) is adopted as it provides an appropriate level of abstraction and has proved useful in a large number of studies. In this classification, six interrelated categories of barriers towards open data provision and adoption are identified: institutional, task complexity, use and perception, legislation, information quality, and technical.
In sum, extant research has identified, described and categorized a large number of innovation barriers. Still, the issue of how their impacts vary, between different types of OGD users and across their services’ lifecycle phases, remain largely uncharted.
Extensive research on why people contribute to open source projects has in short come to the following conclusions: motivations differ extensively between individuals (Juell-Skielse et al., 2014), the ‘open’ ideology fuels development (Shah, 2006), open source developers are both intrinsically (e.g. Ghosh, 1998) and extrinsically motivated (e.g. Kuan, 2001), participation is often driven by factors similar to other social movements (Hertel et al., 2003), and the motivations varies over time (Shah, 2006). In contrast, research on motivation among open data users is scarce. In a rare study, Davies (2010) finds that they are mostly male originating from smaller businesses in the private sector, the public sector or academia. Davies (2010) moreover identifies six sets of drivers for open data use: understanding the government, creating technological innovations, seeking recognition or profit, improving government through digitalization, solving specific problems and providing novel public sector services. Antikainen et al. (2010) discover that contributors to OI communities value intangible factors such as community cooperation, support, skills development and amusement, over monetary gains. Ståhlbröst and Bergvall-Kåreborn (2011) add that open data users’ motivation is influenced by both community type and technology adoption type. Zuiderwjk et al. (2015) state that performance expectancy, effort expectancy, social influence, facilitating conditions and voluntariness of use together account for 45% of variability in intention to use open data (for any purpose). Nevertheless, thorough descriptions of open data users are missing in general, and in particular little is known of who the OGD users are, what motivate them to develop services and how they go about for doing so. Moreover, the understanding of how their characteristics correlate with their experiences of innovation barriers is low. Consequently, in this case study, we first set out to identify a topology for different types of OGD users, in order to enable an analysis of how their characteristics affect perceptions of innovation barriers throughout their innovation processes.
Trafiklab was used as basis for the case study. It is an OGD marketplace in Sweden providing open public transport data to distributed OGD users (Smith et al., 2016a). Three regional public transport authorities distribute their application programming interfaces (APIs) through the marketplace. Their overarching objective for sharing their data is to facilitate development and diffusion of digital travel services that are beneficiary for their end-users (i.e. public transport users). Smith et al. (2016a) identify four key elements of the setup of the marketplace; a technical platform that manages access to APIs and back-ends other digital services, a website that forms the digital front-end of the marketplace, and support services and knowledge sharing activities for both OGD providers and users.
Three principle reasons prompted the decision to use Trafiklab as case for studying OGD users experiences of innovation barriers. Firstly, the Swedish public transport sector is actively seeking new cost-effective measures to boost external digital innovation that contribute to public transport growth. Trafiklab is an important initiative in these efforts and gathers a relatively large community (about 3000 members at the point of our data collection). Accordingly, our research not only has empirical relevance, but the size of the community also enabled access to a heterogeneous set of OGD users working on distinct types of projects. Secondly, Trafiklab has received both considerable media attention and several industry awards for their work (e.g. ‘most innovative public sector e-service’ in Sweden in 2012). Therefore, we judge Trafiklab to be a noteworthy OGD initiative that has reached some maturity in their work with innovation barriers and can provide insight into non-obvious challenges that OGD users face. Thirdly, we had an established contact with the manager for Trafiklab, who expressed interested in identifying potential improvements for Trafiklabs. As a result, we were able to gain access to OGD users that used Trafiklab’s services.
The empirical core of the study is narratives gathered in interviews with OGD users that either had developed or were developing services based on OGD provided through Trafiklab. 19 semi-structured interviews were conducted during spring 2015. The average interview time was 57 minutes, and all interviews were recorded and transcribed verbatim. Forty-four Trafiklab members that had signaled their willingness to participate in discussions regarding Trafiklab’s development were contacted, and 19 were willing and able to share their experiences. All 19 interviewees were males based in Sweden. Their ages ranged from the late 20s to the late 50s. The interviews, which primarily were conducted over the phone, were of explorative nature. Thus, they followed an adaptable and flexible interview guide. The guide contained five overarching topics: background information, motivations for developing services through Trafiklab, service development process, experiences of using Trafiklab and Trafiklab’s areas of improvement.
The analysis procedure was inspired by the method described in Charmaz (2006). As a first step, the first author performed an initial inductive coding of the interview transcriptions. This resulted in approximately 1400 quotations and 350 thematic codes.
Secondly, we collectively utilized a directed qualitative content analysis approach to distinguish OGD user archetypes among the interviewees. Criteria for the selection were developed from the codes that concerned pre-conditions, opinions, motivations, innovation processes, strategies and objectives. By crossing of which of the criteria that the different interviewees fulfilled and then performing a simple, manual pattern analysis, three OGD user archetypes were identified; employees, entrepreneurs and hobbyists.
Thirdly, using a focused coding technique, the first author developed 38 refined codes from the approximately 450 quotes that regarded either experiences of problems during the innovation process or requests for supplementary functionality. These refined codes described unique innovation barriers that the interviewees had experienced. The identified barriers were clustered into the categorization adopted from Janssen et al. (2012).
Fourthly, we applied the ITIL framework to match the 38 identified barriers with the distinct parts of the service innovation process that they primarily related to. Figure 2 depicts the framework, which is a generic loop model for services’ lifecycles that has become the de facto standard in the software industry (Hochstein et al., 2005). It consists of five core phases: strategy, design, transition, operation and continual improvement. The strategy phase involves determining objectives and performance requirements for serving end-users and a particular market space, and to identify, select and prioritize opportunities. In the design phase, the strategy is translated into a design specification. The transition phase encompasses building, validating, and deploying the designed service to end-users. The operation phase includes deliverance and maintenance of the service. Finally, the continual improvement phase focuses on identifying and managing incremental improvements to the service (Taylor, 2007). Thus, the ITIL framework provided a structure for our analysis of how innovation barriers affect the use of OGD in different phases of the service lifecycle (RQ1).
Fifthly, we utilized the code document table tool in Atlas.ti to develop a so-called cross-tabulation matrix. The matrix mapped the 38 identified barriers upon which of the three identified OGD user types that had expressed them. This enabled an analysis of how perceptions of innovation barriers vary across different types of OGD users (RQ2).
4.Results and analysis
4.1Experiences of barriers throughout the service lifecycle
In the following section, the perceived innovation barriers in each phase of the service life cycle are outlined. A summary of the main barriers experienced in each phase is provided in Table 1, while the full list of 38 identified barriers can be found in the appendix.
|Main innovation barriers|
|Strategy phase||Lack of cooperation, communication and service license agreements with OGD providers|
|Design phase||Lack of data, impeding data formats and high task complexity|
|Transition phase||Lack of time to develop services, and difficulties in developing services with unique value propositions and in gaining end-users|
|Operation phase||Lack of communication from OGD providers, poor support, slow data provision and low back-end reliability|
|Continual improvement phase||Lack of understanding of how the services could be improved and difficulties in motivating efforts to do so.|
Strategy phase. OGD users experienced difficulties in communicating with, understanding, influencing and getting help from OGD providers and potential business partners. Thus, it was difficult for them to foster and nurture service strategies where they could control their project processes and outcomes. The absence of service level agreements (SLAs) moreover made it difficult to establish sustainable business models. This was partly due to uncertainty on the long-term consistency of the data provision, but also because they could not guarantee a level of service to the end-users of their services. The lack of SLAs impeded end-users’ willingness to pay, an issue that was particularly detrimental as the potential end-user pool was perceived as limited, the digital service market as fairly saturated and the general cost acceptance among end-users as low.
The quality is vital, and they still lack a SLA for that regarding this data, or for any data that goes through Trafiklab that I know of. There is no guarantee. Instead it’s best effort. Best effort is damn hard for an individual or a small business to make money from. It’s only hobby projects that will be pursued, if they cannot guarantee the quality. – IP16 (translated)
Design phase. The most prominent barrier during the service design phase was the overall lack of OGD. OGD users requested more, richer and increasingly diverse data in order to broaden the range of possible services. 17 out of 19 interviewees emphasized this issue. The cumbersome registration process and the difficulty in getting an overview of the APIs’ content further augmented the perceived lack of data access.
Trafiklab provides public transport data. At some point, you run out of ideas. You cannot do much more than present the public transport. I believe that, in order to get a little more dynamic in the data they provide and somewhat greater diversity among the apps, or whatever innovation one can develop, they need to take in more data, but still data that concerns traffic. – IP18 (translated)
Task complexity barriers were likewise noticeable during the design phase. Designing digital services require a wide spectrum of skills, such as developing system architectures and service wireframes (i.e. skeletal frameworks of the user interfaces). As a result, many of the often programming-oriented OGD users were forced out of their comfort zones. OGD users less acquainted with digital development were on the other hand often repelled by the technical and opaque appearance of the marketplace.
The format of the APIs provided by the studied OGD marketplace was a major barrier, hindering OGD users from designing services as they wished. It did not permit downloading raw bulk data or subscribing to information updates. The effect of this was twofold. First, the format did not facilitate system architectures that stock parts of the intelligence internally – increasing OGD users’ dependency on data provision reliability and lowering the scalability of the ecosystem (see service strategy barriers). Second, the lack of access to raw data moreover restricted the OGD users to design services within the OGD providers’ envisioned scope.
If I provide a product like a spoke wrench for bikes, you cannot do much else [than adjusting and replacing spokes]. But if I provide a hammer, you can suddenly use it for much more than I originally intended. …You might create artwork, take down a wall, or whatever. – IP5 (translated)
Transition phase. The transition phase was home turf for a majority of the OGD users. Yet, the lack of documentation, example code, support, and test facilities added intricacy to the task. Several expressed frustration with having to deal with issues related to data access at the expense of the problems they actually are interested in, for instance making use of the data. The task complexity was further augmented upon nonexistent standards, fragmentation of resources and lack of mapping tables. This made it hard to combine information from several APIs. As both the community’s functionality and activity was fairly low, the OGD users also had a hard time trying to benefit from others’ knowledge or experiences to overcome these impediments.
Many station names are cut off, there are columns with false data and mapping tables are missing between different APIs. Therefore, you have to develop such mapping tables yourself if you want to combine different types of data. – IP4 (translated)
Another salient barrier category was use and participation. OGD users struggled to motivate, find and finance the required time for service development. As writing the code represented the lion share of many of the projects, the time barrier was particularly apparent during the service transition phase. However, some also struggled with the time for publishing the services, especially as many predicted the monetary benefits to be small.
At the minute, time is a major barrier for me. This is something I at least must develop into a proof of concept on my own time. Then I might be able to show it to my company, and they can then feel that; this is a good idea, it’s within our sphere and we can invest in it. – IP11 (translated)
Most OGD users moreover experienced problems with establishing and communicating unique value propositions for both end-users and business partners. Thereupon, they were stuck in a chicken or egg type of dilemma – not able to establish either the partnerships needed in order to reach end-users or end-user bases that would make the services attractive for potential partners. Overall, the OGD users found it very difficult to market their services and to reach end-users.
Sure, there’s a huge need [for my service]. The only problem is to reach out in the roar of 1 million other apps available on App store. – IP1 (translated)
Operation phase. The operative phase was hampered by the slow data provision and the low back-end reliability. These barriers made it hard for OGD users to deliver accurate information in time through their services. This issue was particularly troublesome since the end-users tended to blame the OGD users when the service quality was low, regardless of the quality of the data provision. The lack of communication and poor support from OGD providers moreover obstructed the OGD users’ abilities to understand current and future statuses of the data provision as well as to solve all the various issues that came up during the operation.
If you have a product, which is up and running, but then stops working – then you don’t want to send a letter with the post. It takes a lot of time and you don’t know if the receiver checks his or her post box. He might even be on vacation, and then you don’t know who to send the letter to. They ought to be more active in some way, as they gain extremely much from things that we develop for free. – IP5 (translated)
Continual improvement phase. End-users were perceived to supply very little feedback in general, and the critique that came through to OGD users was seldom constructive. Thus, it was difficult for the OGD users to assess the services’ level of usability and performance, that is to understand how the services could be improved. It was moreover difficult for the OGD users to motivate putting effort into updating their services, since publishing them often was the ultimate goal of their projects. Furthermore, very few of the OGD users earned any money from their services. So, for most OGD users there were no monetary incentives either.
The barrier for me is that I only do the things for myself, because I enjoy tapping on the keyboard and see if something happens. I’m not an entrepreneur. I don’t find it enjoyable to turn it into a product and launch it. I usually only do things half-done, as it’s only in the beginning it’s fun. – IP9 (translated)
4.2User archetypes and their experiences
The 19 interviewed OGD users were categorized as employee, entrepreneur or hobbyist. The categories differ in motivations, objectives, pre-conditions and approaches as well as in experiences of innovation barriers, see Table 2.
|Motivation||Primarily extrinsic||Extrinsic and intrinsic||Primarily intrinsic|
|Objective||Fulfill work tasks||Develop services to gain attention and/or earn money||Explore data, have fun and solve problems|
|Pre-condition||Service project is part of employment||Service project is part of entrepreneurial activities||Service project is part of leisure activities|
|Approach||Pursue clearly guided projects sequentially||Pursue industrial projects based on opportunities||Pursue several explorative projects simultaneously|
|Experience of innovation barriers||Encounter high task complexity barriers during design and transition phases, and low use & participation barriers during all phases||Encounter high task complexity barriers during the transition phase, and high use & participation barriers during all phases||Encounter high information quality barriers and low technical barriers during the transition phase|
Employees. OGD users categorized as employees utilized OGD as a part of their employment, in order to fulfill pre-defined work tasks. In relation to the OGD marketplace, their tasks were often to integrate information from the provided APIs into existing or developing services. The employees commonly worked as developers for IT consultancy firms that deploy several projects simultaneously, but also worked in various roles for businesses that focus on single projects. The employees were primarily driven by the outcome of their projects, that is to fulfill managements’ and/or clients’ expectations. Thus, they were generally fairly focused and goal-driven – predominantly concentrating on getting access to and decoding the data they need. Therefore, they neither valued nor participated in knowledge sharing activities. Seven of the 19 interviewees were catalogued as employees.
On average, the employees dwelled less on barriers, compared to the two other OGD user archetypes – both in terms of quantity and emphasis. However, being solely focused on getting hold of the OGD, the employees’ major concerns were the scope and quality of the data provision service. They tended to expect the same professional approach from OGD providers and intermediaries as they experience from their other suppliers, partners and clients. Hence, they were surprised and disappointed with the level of help in terms of documentation, example code and support. It impeded their efforts of designing and developing services. Lack of data and the data format moreover hindered them from both developing strategies and designing services as they wished.
Overall, employees’ innovation efforts and their services’ lifecycles were a lot less concentrated around and dependent on the innovation ecosystem surrounding the studied OGD marketplace. Instead, their ideas, project financing, business partners and feedback frequently originated from other innovation ecosystems.
Entrepreneurs. OGD users categorized as entrepreneurs utilized OGD to pursue service development projects aimed at either monetizing or at gaining attention and reputation. Their projects were more often than not based on a combination of an identified gap in the existing range of services and a novel service opportunity offered by the data provision of the OGD marketplace. While their primary focus commonly was on one project at a time, the entrepreneurs often had several project ideas in their pipelines, awaiting the right timing to pursue them. Both internal and external driving forces motivated them, that is the challenge of developing services and businesses as well as the potential gains of the project outcome. The entrepreneurs were generally interested in all the OGD marketplace’s activities, where they often used knowledge-sharing activities to promote their projects and themselves. Five of the 19 interviewees were catalogued as entrepreneurs.
The entrepreneurs were the most outspoken OGD user category, having endless of experiences, opinions and suggestions regarding the OGD marketplace that they wanted to share. As the employees, entrepreneurs were limited by the lack of data and the lack of support. However, the entrepreneurs were to a greater extent also hindered by the inherent complexity of the task, as they often were operating on their own and not always as acquainted with writing code as the other user types. They frequently mentioned struggling with making sense of the fragmented documentations and overviewing what information the APIs contained. Moreover, the entrepreneurs also had bigger problems with allocating time during all phases of the service lifecycle, compared to the other archetypes.
Institutional and legislation barriers were particularly detrimental for the interviewed entrepreneurs. Many of them developed services that use the OGD marketplace as the core data provider. As a consequence, their services were highly dependent on the data provision of the OGD marketplace, but as a minor player, they still experienced immense difficulties when trying to gain insight to and influence the activities of the OGD providers.
Hobbyists. OGD users categorized as hobbyists utilized OGD to pursue their hobby projects. Internal driving forces typically motivated them. Their primary objectives when interacting with the OGD marketplace were thus often to explore the published data, to have fun and to solve problems. These problems mostly related to either the hobbyists’ everyday issues or to purely technical challenges. The hobbyists pursued several smaller development projects simultaneously, often at an irregular pace. Most of these projects will never be finished and the hobbyists’ objectives were seldom to publish the services they developed, as they predominantly were interested in the early phases of the service lifecycle. Seven of the 19 interviewees were catalogued as hobbyists.
The hobbyists were very different from the other two types of OGD users in terms of objective and approach. Being mainly interested in exploring the data, ‘playing with it’ and using it to solve self-experienced problems, the hobbyists were not too concerned about matters such as contractual issues or about the barriers that appear during the later stages of the service lifecycle. Further, being both more flexible and more programming-oriented than the other OGD user types, the hobbyists were not as frustrated with specific technical barriers during the transition phase. However, the hobbyists were also troubled by the lack of data and struggled with finding time, just as the entrepreneurs.
All interviewed hobbyist developed services that used the OGD marketplace as the core data provider for their services’ functionalities. Still, being less focused on publishing their services, institutional and legislation barriers were not as central for them, as for the entrepreneurs. Instead, the hobbyists spoke more about the underpinning values of disclosing OGD than the two other groups. As a consequence, their frustration in regards to tedious authorization procedures and limited data sets was higher. Lastly, the hobbyists were in general also more interested in co-creation and discussions with other OGD users. Thus, the lack of community functions was particularly unsatisfactory for them.
5.1The dynamic effects of innovation barriers
Extant research has identified a vast number of innovation barriers that impede release and adoption of OGD. However, how effects of innovation barriers vary across distinct OGD user types and activities in innovation processes has received little attention. To this end, we studied a specific set of OGD user, service developers. We conducted an inductive analysis to identify distinct types of OGD users in this group, and we traced how effects of innovation barriers vary across OGD user types and across phases in their innovation processes. Our findings provide a number of insights on the dynamic effects of innovation barriers.
The analysis reveals that barriers hinder OGD users’ innovation efforts throughout the entire service lifecycle. This finding challenges the use of open data production, distribution and consumption (e.g. Zuiderwijk & Janssen, 2014) as model when considering OGD impediments. Instead, it suggests that also later service lifecycle phases must be considered in order to facilitate throughput in terms of innovation. Moreover, the analysis illustrates that OGD users face different types of barriers in different lifecycle phases. For instance, the lack of data, the impeding data format and the high task quality were perceived to be most challenging in the design phase, while the operation phase rather was impeded by lack of communication from data providers, poor support, slow data provision and low back-end reliability. This finding suggests that an analysis of which phases that primarily hamper OGD users in a given innovation ecosystem can inform the process of choosing appropriate measures for catalyzing its throughput. Several of the identified innovation barriers, such as lack of transparency and influence are also social rather than technical in nature. Hence, the analysis enforces earlier reports that the mere provision of data does not generate value on its own (e.g. Janssen et al., 2012), and highlights the importance of going beyond technical aspects when ensuring that the possibilities for translating OGD into useful outcomes is available for the widest possible range of OGD users (cf. Gurstein, 2011). Furthermore, the interviewees expressed experiences of a multitude of innovation barriers that are not directly related to the OGD provision. Examples include lack of domain knowledge during the strategy phase, impeding client requirements during the design phase, insufficient programming skills during the transition phase, time constraints during the operation phase and lack of feedback during the continual improvement phase. This finding emphasizes the importance of utilizing a holistic innovation ecosystem perspective (cf. Adner & Kapoor, 2010) when analyzing OGD users’ possibilities to reap benefits from OGD.
The analysis of OGD user archetypes extends earlier findings on the diversity among the service developing crowd (e.g. Zuiderwijk, 2015) and variations in reasons for using OGD (e.g. Davies, 2010) as it provides a more granular examination of these issues. The paper portrays how OGD users differ in terms of motivations, pre-conditions, approach and objectives, and suggest three archetypes based on these criteria: employees, entrepreneurs and hobbyists. These character traits suggest that the different types of OGD users can play different roles in the evolvement of OGD innovation ecosystems (cf. Styrin et al., 2017). For example, due to their motivations and goals, entrepreneurs seem more likely to pursue high profile projects utilizing provided OGD as core. These projects can be used as best practice and for increasing the public awareness of the innovation ecosystem as well as the perceived attractiveness of participating in it. Employees, on the other hand, have unique abilities to integrate the provided OGD into solid services that origin from other innovation ecosystems. Thereupon, they are also important for the expansion of the ecosystem. Lastly, being data-driven, focused on multiple projects, and interested in knowledge sharing activities, hobbyists are important for acquiring continuous activity and cross-border knowledge transfer in the innovation ecosystem (cf. Smith et al., 2016b).
This study also adds to previous literature on impediments for OGD use (e.g. Zuiderwijk et al., 2012) by demonstrating that the OGD users’ motivations, pre-conditions, approach and objectives have an impact on the experiences of innovation barriers. Motivation: Employees, driven by extrinsically factors, were in general less focused on the process as such. Thus, impediments affecting the experience of it, for instance the unappealing appearance of the data provision website, were less hampering for them. On the other hand, compared to the intrinsically motivated hobbyists, both the employees and the entrepreneurs regularly put higher emphasis on impediments affecting the solution space for services, such as the limiting data format. Pre-conditions: The employee driven projects were pursued as part of principal occupations. The employees’ commitment was therefore, at least partially, ensured through their employment contracts. As an effect, the employees were less likely to be hampered by internal use and participation barriers than hobbyist and entrepreneurs, who often developed services on their spare time. Approach: The employees had clearer guidelines and frameworks for their projects, compared to entrepreneurs and hobbyists who often pursued project upon opportunity. Hence, their flexibility in overcoming technical barriers such as nonexistent standards was in general lower. Objective: Both the business oriented employees and the entrepreneurs were more likely to be hampered by legislative barriers, such as lack of SLAs, than hobbyists. This was probably largely due to their strong outcome orientation and expectations on extrinsic rewards, while hobbyist were less committed to publishing and reaping rewards from their services.
For policy-makers and practitioners, the analysis of experiences among the three OGD user archetypes implies that OGD stakeholders such as OGD providers and policy-makers can influence the mix of OGD users in the innovation ecosystem by choosing which types of barriers to work on. For instance, efforts aimed at lowering task complexity barriers during the transition phase can substantially enhance the attractiveness of participating in the ecosystem for employees and entrepreneurs, but would probably not affect the appeal for hobbyists, as they mainly struggle with information quality barriers during this phase. Likewise, use and participation measures would probably impact the willingness and possibility to participate for entrepreneurs, but would not affect employees, as their devotion is regulated by their contracts.
As discussed earlier, different types of OGD users can play different roles in the evolvement of the innovation ecosystem. Innovative developments moreover often happen at the intersection of fields (Hargadon & Sutton, 1997) when innovators manage to connect seemingly unrelated fields, problems, or ideas. Diversity as a driver of innovation has been referred to as the ‘Medici effect’ and points to how innovation often arises from combinations of knowledge and experiences across contexts rather than from isolated ingenuity (Johansson, 2006). A diverse crowd with large coverage provides the ability to turn distant search into local (i.e. leveraging specialists from a wide set of contexts without associated costs being incurred), and to assure the sustainability and innovation potential of the ecosystem. Thus, the palette of actions aimed at attracting OGD users to OGD innovation ecosystems should be organized to support diversity.
The diversity argument is also relevant for the data provision as such. For example, the utilized data format and the setup of the studied marketplace were perceived to hamper what type of services that could be developed. In order to attract a large and diverse crowd and to ensure scalability, it is important to expand the solution space and to lower the dependability as much as possible. For this reason, flexible, call-based data formats and a system design that allows for bulk-download of data seems preferable.
Furthermore, the cross-tabulation of innovation barriers and the service lifecycle phases suggests that measures aimed at boosting non-pecuniary outbound OI practices in the public sector based on OGD should not exclusively focus on initial and technical data provision impediments, although these barriers are highly important during the design and transition phases. Other types of innovation barriers impede OGD users’ ability to sustainably develop, launch and maintain services, thus calling for supplementary actions.
For open government and open data researchers, this paper calls for increased focus on the innovation ecosystem for OGD users. Previous literature on OGD enabled innovation time and again fail to acknowledge OGD users’ reality in the sense that it merely focus on the data supply side of their innovation processes, ignoring their relations with end-users, clients and business partners. In other-words, less data centric and more user centric OGD research is needed.
This study was confined both geographically (Sweden) and to a particular domain (public transport). Additional studies are needed in order to establish the generalizability of the findings and for widening the understanding of OGD users’ experiences of barriers when innovating with OGD. Moreover, further comparison with studies on OI in the public sector in general (e.g. Feller et al., 2011; Gascó, 2017; Mergel, 2015; Mergel & Desouza, 2013), and barriers to such practices in particular (e.g. Mergel, 2017; Smith et al. 2018), is needed in order to establish if the findings are transferable to other forms of non-pecuniary OI in the public sector.
The outcome of the study is naturally also influenced by the chosen research method. The choice of adopting a categorization for barriers from Janssen et al. (2012) ruled and limited the scope of the analysis. The appropriateness of the ITIL service lifecycle model (Taylor, 2007) as a framework for describing the innovation processes of OGD users also needs further assessment. Despite associating the expressed barriers with different phases of the service lifecycle, it should moreover also be emphasized that some of them are present during several lifecycle phases.
Lastly, the boundaries between the proposed OGD user categories are blurry. Moreover, they neither represent the full spectrum of OGD users, nor display the full complexity in terms of relevant aspects. As such, the categorization merely provides a sensitizing device (Weick, 1976) that illustrate that differences within the OGD user group exist, and that these differences, among other things, impact their experiences of innovation barriers.
This paper sets out to explore how innovation barriers affect the use of OGD in different phases of the service lifecycle and how the perceptions of the barriers vary across different types of OGD users. To this end, it finds that innovation barriers hamper OGD users’ innovation efforts throughout their services’ lifecycles, and that the barriers that hinder them from creating value are both social and technical in nature. Moreover, only a subset of the identified barriers is directly related to the data provision. The paper also finds that OGD users differ in motivations, pre-conditions, approach and objectives, and that the experiences of and abilities to overcome different types of innovation barriers are influenced by these characteristics. Consequently, compared to extant literature, the findings suggest that OGD initiatives aimed at facilitating non-pecuniary outbound open innovation to a greater extent should address the barriers that appear during diffusion of innovations, the barriers that are not directly related to the OGD provision, and the barriers that are experienced by non-obvious OGD user groups.
We wish to thank the interviewees for devoting their time and for sharing their valuable knowledge. We also want to acknowledge Trafikverket (the Swedish Transport Administration) for funding the research project “Increased Digitization and the Modern Transport Administration” that this paper is a product of. Further, we have also received financial support for our research from the Marianne and Marcus Wallenberg Foundation as well as from Västra Götalandsregionen and K2 – The Swedish Knowledge Centre for Public Transport.
Adner, R., & Kapoor, R. (2010). Value creation in innovation ecosystems: How the structure of technological interdependence affects firm performance in new technology generations. Strategic Management Journal, 31(3), 306–333. https://doi.org/10.1002/smj.821.
Antikainen, M., Mäkipää, M., & Ahonen, M. (2010). Motivating and supporting collaboration in open innovation. European Journal of Innovation Management, 13(1), 100–119. https://doi.org/10.1108/14601061011013258.
Barry, E., & Bannister, F. (2014). Barriers to open data release: A view from the top. Information Polity, 19(1, 2), 129–152. https://doi.org/10.3233/IP-140327.
Bertot, J. C., Jaeger, P. T., & Grimes, J. M. (2010). Using ICTs to create a culture of transparency: E-government and social media as openness and anti-corruption tools for societies. Government Information Quarterly, 27(3), 264–271. https://doi.org/10.1016/j.giq.2010.03.001.
Borzacchiello, M. T., & Craglia, M. (2012). The impact on innovation of open access to spatial environmental information: a research strategy. International Journal of Technology Management, 60(1/2), 114. https://doi.org/10.1504/IJTM.2012.049109.
Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis (Introducing Qualitative Methods Series). London: Sage Publications.
Chesbrough, H. (2003). The era of open innovation. MIT Sloan Management Review, 44(3), 35–41.
Chesbrough, H. (2006). Open innovation: a new paradigm for understanding industrial innovation. In Open innovation: Researching a new paradigm pp. 1–12). Oxford: Oxford University Press.
Chesbrough, H., Vanhaverbeke, W., & West, J. (2014). New frontiers in open innovation. Oxford: Oxford University Press.
Dahlander, L., & Gann, D. M. (2010). How open is innovation? Research Policy, 39(6), 699–709. https://doi.org/10.1016/j.respol.2010.01.013.
Davies, T. (2010). Open data, democracy and public sector reform. A look at open government data use from data.gov.uk (Master of Science dissertation). Oxford: Oxford Internet Institute. Retrieved from http://www.opendataimpacts.net/report/.
Davies, T., Perini, F., & Alonso, J. M. (2013). Researching the emerging impacts of open data: ODDC conceptual framework (ODDC Working Paper No. 1). Retrieved from https://idl-bnc-idrc.dspacedirect.org/bitstream/handle/10625/56313/IDL-56313.pdf?sequence=1.
Dietrich, D., Gray, J., McNamara, T., Poikola, A., Pollock, P., Tait, J., & Zijlstra, T. (2009). Open data handbook. Cambridge: Open Knowledge International. Retrieved from http://opendatahandbook.org.
Feller, J., Finnegan, P., & Nilsson, O. (2011). Open innovation and public administration: transformational typologies and business model impacts. European Journal of Information Systems, 20(3), 358–374. https://doi.org/10.1057/ejis.2010.65.
Gascó M. (2017). Living labs: Implementing open innovation in the public sector. Government Information Quarterly, 34(1), 90–98. https://doi.org/10.1016/j.giq.2016.09.003.
Gassmann, O., & Enkel, E. (2004). Towards a theory of open innovation: three core process archetypes. Presented at the R&D management conference (RADMA), Lisbon, Portugal July 6–9.
Ghosh, R. A. (1998). Interviews with Linus Torvalds: What motivates software developers. First Monday, 3(3).
Gurstein, M. B. (2011). Open data: Empowering the empowered or effective data use for everyone? First Monday, 16(2).
Hargadon, A., & Sutton, R. I. (1997). Technology brokering and innovation in a product development firm. Administrative Science Quarterly, 716–749. https://doi.org/10.2307/2393655.
Hertel, G., Niedner, S., & Herrmann, S. (2003). Motivation of software developers in Open Source projects: an Internet-based survey of contributors to the Linux kernel. Research Policy, 32(7), 1159–1177. https://doi.org/10.1016/S0048-7333(03)00047-7.
Hjalmarsson, A., Johannesson, P., Jüll-Skielse, G., & Rudmark, D. (2014). Beyond innovation contests: A framework of barriers to open innovation of digital services. In Proceedings of the European Conference on Information Systems (ECIS) 2014. 9–11 June, Tel Aviv.
Hochstein, A., Zarnekow, R., & Brenner, W. (2005). ITIL as common practice reference model for IT service management: formal assessment and implications for practice. In 2005 IEEE International Conference on e-Technology, e-Commerce and e-Service (pp. 704–710). IEEE. https://doi.org/10.1109/EEE.2005.86.
Huijboom, N., & Van den Broek, T. (2011). Open data: an international comparison of strategies. European Journal of ePractice, 12(1), 4–16.
Janssen, M., Charalabidis, Y., & Zuiderwijk, A. (2012). Benefits, adoption barriers and myths of open data and open government. Information Systems Management, 29(4), 258–268. https://doi.org/10.1080/10580530.2012.716740.
Jeppesen, L. B., & Lakhani, K. R. (2010). Marginality and problem-solving effectiveness in broadcast search. Organization Science, 21(5), 1016–1033. https://doi.org/10.1287/orsc.1090.0491.
Jetzek, T. (2016). Managing complexity across multiple dimensions of liquid open data: The case of the Danish Basic Data Program. Government Information Quarterly, 33(1), 89–104. https://doi.org/10.1016/j.giq.2015.11.003.
Jetzek, T., Avital, M., & Bjorn-Andersen, N. (2014). Data-driven innovation through open government data. Journal of Theoretical and Applied Electronic Commerce Research, 9(2), 100–120. https://doi.org/10.4067/S0718-18762014000200008.
Johansson, F. (2006). The Medici effect: what elephants and epidemics can teach us about innovation. Boston: Harward Business School.
Juell-Skielse, G., Hjalmarsson, A., Johannesson, P., & Rudmark, D. (2014). Is the public motivated to engage in open data innovation? In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 8653, pp. 277–288). https://doi.org/10.1007/978-3-662-44426-9_23.
Kaasenbrood, M., Zuiderwijk, A., Janssen, M., de Jong, M., & Bharosa, N. (2015). Exploring the factors influencing the adoption of open government data by private organisations. International Journal of Public Administration in the Digital Age, 2(2), 75–92. https://doi.org/10.4018/ijpada.2015040105.
Kuan, J. W. (2001). Open source software as consumer integration into production (Working paper). SSRN. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=259648.
Maccani, G., Donnellan, B., & Helfert, M. (2015). Exploring the Factors that Influence the Diffusion of Open Data for New Service Development: An Interpretive Case Study. In ECIS 2015 Completed Research Papers. 27–29 May, Munster. Retrieved from http://aisel.aisnet.org/cgi/viewcontent.cgi?article=1126&context=ecis2015_cr.
Mergel, I. (2015). Opening government: Designing open innovation processes to collaborate with external problem solvers. Social Science Computer Review, 33(5), 599–612. https://doi.org/10.1177/0894439314560851.
Mergel, I. (2017). Open innovation in the public sector: drivers and barriers for the adoption of Challenge.gov. Public Management Review, 1–20. https://doi.org/10.1080/14719037.2017.1320044.
Mergel, I., & Desouza, K. C. (2013). Implementing open innovation in the public sector: The case of Challenge. gov. Public Administration Review, 73(6), 882–890. https://doi.org/10.1111/puar.12141.
Neuroni, A. C., Riedl, R., & Brugger, J. (2013). Swiss Executive Authorities on Open Government Data – Policy Making beyond Transparency and Participation. In 2013 46th Hawaii International Conference on System Sciences (pp. 1911–1920). IEEE. https://doi.org/10.1109/HICSS.2013.19.
Osborne, S. P. (2006). Ed. The new public governance? Public Management Review, 8(3), 377–387. https://doi.org/10.1080/14719030600853022.
Peters, M. A., & Britez, R. G. (2008). Open education and education for openness. Sense Publishers Rotterdam.
Safarov, I., Meijer, A., & Grimmelikhuijsen, S. (2017). Utilization of open government data: A systematic literature review of types, conditions, effects and users. Information Polity, (Preprint), 1–24. https://doi.org/10.3233/IP-160012.
Shah, S. K. (2006). Motivation, governance, and the viability of hybrid forms in open source software development. Management Science, 52(7), 1000–1014. https://doi.org/10.1287/mnsc.1060.0553.
Smith, G., Ofe, H. A., & Sandberg, J. (2016a). Digital Service Innovation from Open Data: Exploring the Value Proposition of an Open Data Marketplace. In 2016 49th Hawaii International Conference on System Sciences (HICSS) (pp. 1277–1286). https://doi.org/10.1109/HICSS.2016.162.
Smith, G., Hjalmarsson, A., & Burden, H. (2016b). Catalyzing Knowledge Transfer in Innovation Ecosystems through Contests. In AMCIS 2016 Proceedings. 11–14 August, San Diego.
Smith, G., Sochor, J., Karlsson, I.C.M. (2018). Public-private innovation: Barriers in the case of mobility as a service in West Sweden. Public Management Review, in press. https://doi.org/10.1080/14719037.2018.1462399.
Sørensen, E., & Torfing, J. (2011). Enhancing collaborative innovation in the public sector. Administration & Society, 43(8), 842–868. https://doi.org/10.1177/0095399711418768.
Ståhlbröst, A., & Bergvall-Kåreborn, B. (2011). Exploring users motivation in innovation communities. International Journal of Entrepreneurship and Innovation Management, 14(4), 298–314. https://doi.org/10.1504/IJEIM.2011.043051.
Styrin, E., Luna-Reyes, L., & Harrison, T. M. (2017). Open data ecosystems: an international comparison. Transforming Government: People, Process and Policy, 11(1). https://doi.org/10.1108/TG-01-2017-0006.
Taylor, S. (2007). The official introduction to the ITIL service lifecycle. London: The Stationary Office.
Ubaldi, B. (2013). Open government data: Towards empirical analysis of open government data initiatives (OECD Working Papers on Public Governance No. 22). OECD Publishing. Retrieved from http://dx.doi.org/10.1787/5k46bj4f03s7-en.
Verhulst, S., & Young, A. (2016). The Global Impact of Open Data – Key Findings from Detailed Case Studies Around the World. Sebastopol: O’Reilly Media Inc. Retrieved from http://www.oreilly.com/data/free/the-global-impact-of-open-data.csp.
Weick, K. E. (1976). Educational organizations as loosely coupled systems. Administrative Science Quarterly, 1–19. https://doi.org/10.2307/2391875.
Zuiderwijk, A. (2015). Open data infrastructures: The design of an infrastructure to enhance the coordination of open data use (Doctoral dissertation). Delft University of Technology. Retrieved from http://repository.tudelft.nl/view/ir/uuid:9b9e60bc-1edd-449a-84c6-7485d9bde012/.
Zuiderwijk, A., & Janssen, M. (2014). Barriers and development directions for the publication and usage of open data: A socio-technical view. In Open Government (pp. 115–135). New York: Springer.
Zuiderwijk, A., Janssen, M., Choenni, S., Meijer, R., Alibaks, R. S., & Sheikh_Alibaks, R. (2012). Socio-technical impediments of open data. Electronic Journal of eGovernment, 10(2), 156–172.
Zuiderwijk, A., Janssen, M., & Dwivedi, Y. K. (2015). Acceptance and use predictors of open data technologies: Drawing upon the unified theory of acceptance and use of technology. Government Information Quarterly, 32(4), 429–440. https://doi.org/10.1016/j.giq.2015.09.005.
Appendix: Cross-tabulation of perceived barriers with lifecycle phases and OGD user types
|Barrier||Problem||Category||Mentions of barrier|
|Lack of transparency||Understand data providers’ motives and plans||Institutional||X|
|Lack of influence||Influence the data provision||X||X|
|Lack of domain knowledge||Understand needs and requirements within the domain||Task complexity||X||X||X|
|Lack of market knowledge||Identify gaps in the existing range of services||X|
|Limited market||Few potential end-users||Use and participation||X|
|Low willingness to pay||Many end-users required to build sustainable businesses||X|
|Lack of cooperation||Establish partnerships with data providers and partners||X||X|
|Lack of SLA||Secure long-term reliable data provision||Legislation||X|
|Procurement policies||Collaborate with public organizations||X|
|Lack of scalability||Secure long-term reliable data provision||Technical||X|
|Impeding client requests||Design thriving services||Institutional||X|
|Registration process||Understand what the APIs contain and getting started||Task complexity||X|
|Lack of GUI skills||Design wireframes for user interfaces||X|
|Unappealing appearance||Get interested in developing services based on OGD||Use and participation||X|
|Lack of data||Access information needed to develop services||Information quality||X||X||X|
|Limiting data format||Design services outside of the intended solution space||Technical||X||X||X|
|Barrier||Problem||Category||Mentions of barrier|
|Poor documentation||Understand APIs content and how to process them||Task complexity||X||X||X|
|Lack of code examples||Understand how to process APIs||X||X||X|
|Poor test facilities||Test and troubleshoot services||X|
|Lack of community||Benefit from other OGD users’ knowledge and experiences||X||X|
|Lack of coding skills||Interact with APIs and write appropriate code||X|
|Low value for end- users||Stand out from other services (unique selling propositions)||X|
|Lack of exposure||Marketing and establishing end-user bases||X||X||X|
|Lack of financing||Finance the time required to develop and publish services||X|
|Lack of open practices||Collaborate with other OGD users||X||X|
|Poor data quality||Develop trustworthy services||Information quality||X||X||X|
|Data provision restrictions||Comply with marketplaces’ standards||Legislation||X||X||X|
|Nonexistent standards||Combine multiple APIs||Technical||X||X|
|Lack of mapping tables||Combine multiple APIs||X||X|
|Intermediate position||Communication and support for service quality problems||Institutional||X|
|Lack of time||Find time to maintain services|
|(also relevant for transition phase)||Use and participation||X||X|
|Lack of communication||Understand current and future state of data provision||X|
|Poor support||Solve various issues when operating services (also relevant for transition phase)||X|
|Slow data provision||Deliver up-to-date information in time||Information quality||X|
|Low back-end reliability||Achieving high service reliability||Technical||X||X|
|Continual improvement phase|
|Lack of feedback||Understanding service usability and performance||Task complexity||X|
|Lack of motivation||Motivation for improving and updating services (also relevant for transition and operation phases)||Use and participation||X||X|
|Changing back-end||Service reliability||Technical||X||X|