Whose open data is it anyway? An exploratory study of open government data relevance and implications for democratic inclusion

3.1E-democracy

Digital government and ICTs have long been regarded as promising tools for enhancing citizen-state relations (Morgeson III et al., 2011; Janssen et al., 2017; Tolbert & Mossberger, 2006). Following the rise in internet use and the development of new ICTs in the early 2000s, many assumed that new technologies would transform the ways that the public participates. There was hope that governments’ use of ICTs would help to strengthen democracy by increasing opportunities for those who traditionally participate and mobilizing new groups (Norris, 2001). The term e-democracy was quickly coined as a sort of umbrella term to describe the many ways that governments used ICTs to improve democratic outcomes (Garson, 2006), including using ICTs to inform, consult, and communicate with citizens (Coleman & Norris, 2005; Lee et al., 2011).

Distinct from e-government, e-democracy is aimed at empowering citizens to use new technologies to participate more meaningfully in government decision-making and policy-making processes (Hujran et al., 2020). Nonetheless, e-democracy is an intentionally broad concept in order to encompass a wide range of activities and initiatives in which digital tools are leveraged to enhance democratic participation (Coleman & Norris, 2005). This includes open government programs. With citizens “at the heart of open government” (Wirtz et al., 2019, p. 566), OGD initiatives have become obvious choices for democratic governments eager to facilitate a more deliberative and participatory democracy. In contrast to voting in periodic elections, increasing access to government data is thought to provide citizens with the information needed to actively participate in a number of ongoing governance processes (Attard et al., 2015; Harrison et al., 2012), thus, enhancing democratic systems.

Yet, scholars continue to debate whether OGD indeed empowers citizens to participate (Linders, 2012) or if OGD initiatives have failed to encourage meaningful citizen participation. To this end, scholars have argued that technology adoption is not enough to advance the kinds of democratic values in question (Yu & Robinson, 2011). Evans and Campos (2013) argue that most open government initiatives place too much emphasis on data delivery, including the technical aspects of the platforms, which distracts from the need to provide citizens with the necessary contextual information to make sense of and the data in order to use it appropriately. Similarly, Wirtz et al. (2017) contend that governments have neglected the crucial importance of accessibility and usability for ensuring equal access to OGD. For participatory democracy to materialize in the desired manner, it will take more than simply introducing new technology. Citizens must possess a degree of knowledge about the issues, the systems they are embedded in, and the nature of the policymaking process to engage productively and meaningfully in deliberative processes. Additionally, the data provided should be relevant to citizens and the decision-making processes they are encouraged, even expected at times, to participate in.

3.2The democratic divide

The digital divide, which largely refers to disparities in access to and use of technology among marginalized groups, has been widely discussed in the digital government literature. However, the democratic divide is also salient in this discussion as it describes the “differences between those who do and do not, use the panoply of digital resources to engage, mobilize, and participate in public life” (Norris, 2001, p. 4). In other words, of those who have access to and use the internet, who uses it to participate and who does not? (Min, 2010; Nam, 2011). Thus, the democratic divide poses questions concerning increased access to information via internet-based applications such as open data portals and whether it indeed enhances democratic participation by empowering new groups of citizens, or if it remains another tool for those who would otherwise participate anyway. Like many ICTs used by governments in efforts to enhance citizen participation, there is a real possibility that OGD is not actually making participation processes more inclusive. In fact, e-democracy research provides evidence of cases in which digital tools have failed to engage individuals and groups outside of the already actively engaged (Kreiss, 2015; Jensen, 2006; Dahlberg, 2001).

3.3Who uses open government data and for what purpose?

Zuiderwijk et al. (2015) define open data use as any activity taken by a person or organization to view, understand, analyze, or visualize a dataset provided by a governmental organization. OGD use is thought to provide a wide range of stakeholders, including individuals, firms, scientists, and journalists, with many benefits (Charalabidis et al., 2018). Dawes et al. (2016) outline three stakeholder groups who are either involved in the implementation or active users of OGD. The first group is the governments and organizational leaders who are often tasked with promoting open data initiatives and ultimately responsible for the implementation and management of OGD. The second group is direct open data users, which tend to be advocates of transparency initiatives such as OGD, as well as data analysts and developers who leverage open data to create new applications, innovations, and public interest technologies. The third group consists of those who are beneficiaries of the products and services developed out of OGD. More recently, in perhaps the most comprehensive systematic literature review on open data utilization to date, Safarov et al. (2017) reviewed these conceptualizations among others in the literature and identified six categories of OGD users: citizens, businesses, researchers, developers, 224 non-governmental organizations, and journalists.

Similarly, the researchers identified eight purposes, or uses, of OGD: innovation, data analytics, decision-making, anti-corruption, smart city, new services, research, and hackathons/competitions (Safarov et al., 2017). Further, Wijnhoven et al. (2015) found that motivations for using OGD vary based on the context and task at hand. Using insights from studies of the open source software community, they also highlight the influence of individuals’ internal motivations including the desire to learn, solve their own problems, or simply have fun (Von Krogh et al., 2012). Schmidthuber et al. (2017) also found that in the context of open government tools, citizens were more likely to participate in problem-solving activities and provide input and feedback when they perceived benefits from participating.

Like other open government initiatives, OGD also aims to promote collaboration among stakeholders. An emerging means by which this occurs is through data collaboratives which help to facilitate the sharing and use of data to address social problems (Susha et al., 2018). Collaboration, in this sense, occurs between government departments and agencies (Yang et al., 2014; Gil-Garcia et al., 2009; Yang & Maxwell, 2011) and even across sectors (Susha et al. 2018). Nonprofits, non-governmental organizations, and voluntary organizations may be motivated to participate in these efforts in order to share and obtain resources that help them fulfill their mission.

Lastly, a key aspect of the Obama administration’s open government directive was an emphasis on interoperability which would enable more efficient data sharing across governments and sectors increasing transparency, accountability, and opportunities for collaboration. In a comprehensive literature review on open government and democracy, Hansson et al. (2015) propose that the emphasis on interoperability in the U.S. can be explained by the prominent role that private actors and NGOs play in governing the public sector. Furthermore, rising interest in co-creation, co-design, and co-production in the context of open government and OGD has emphasized the multi-stakeholder interest in OGD (McBride et al., 2018). Specifically, co-creation projects using OGD require collaboration among local authorities, community-based groups, academics, industry, and volunteers (Khayyat & Bannister, 2017) in order to leverage data to create valuable products and services. OGD-enabled collaboration is expected to produce both social and economic benefits for stakeholders and society at large by encouraging the use of open data in ways that stimulate innovation and economic growth. Positive economic impacts can occur by way of businesses using OGD to evaluate potential investments (Janssen et al., 2012), from developers leveraging traffic and geographical data for the creation of new apps and services (Bertot et al., 2014), or by facilitating open innovation (Linaker & Runeson, 2020) and social innovations by empowering civic hackers through organized hackathons (Young & Yan, 2017).

In any case, for OGD to be a tool for democracy and not just economic benefit, governments must think about inclusive means of encouraging citizen participation by providing data that is useful not just to the private sector, nonprofits, and government agencies, but of interest and relevance to all citizens. For citizens to meaningfully participate, simply disclosing information about government is not necessarily enough (Nikiforova & McBride, 2021; Piotrowski & Liao, 2012); the data provided should be useful for informing meaningful engagement in deliberative activities.

4.1Quality data

There are many factors that impact OGD’s ability to enhance democratic outcomes once adopted, but the quality of OGD is paramount. Quality is a dynamic concept but the degree to which data is indeed usable is often considered a factor because usability has such a big impact not only on whether stakeholders use open data as intended but whether the values OGD is intended to promote are realized. Usability has been discussed quite a bit in the literature but mostly from a technical aspect with ease of access and navigation (Cucciniello et al., 2012; Pina et al., 2010; Pina et al., 2007) and quality of data (Bertot et al., 2014; Arrard et al., 2015; Janssen et al., 2017; Zhu & Freeman, 2019; Ojo et al., 2018; Vetro et al., 2016) being necessary, but still overlooked, conditions of usability.

Usability can also refer to how easy it is for users to understand, interpret, and comprehend the information provided (Zhu & Freeman 2019; Karr, 2008; Caba Perez et al., 2005). A common issue with many open data portals is that they neglect to provide the necessary context and guidance on how to use the information provided (Evans & Campos 2013). As a result, users must expend valuable time and energy determining the relevance of the data and whether or not it is usable. Thus, even if OGD is error-free, timely, and complete, the availability of relevant and useful data remains particularly critical for its use (Belhiah & Bounabat, 2017; Belhiah et al., 2015; Evans & Campos 2013; Pipino et al., 2002; Wang & Strong, 1996).

4.2Relevant and useful data

Previous studies have also emphasized the importance of providing communities with the information they actually care about like data on crime, public health, and the environment (Ojo et al., 2018). Ultimately, the information provided should provide value to its users. It is one thing to allow increased public access to government data, but it is quite another thing to publish data that is valuable to the public and that individuals are interested in using (Attard et al., 2015; Harrison et al., 2012; Lee & Kwak, 2011). Furthermore, the Open Data Barometer (2017), which develops principles for open and accessible data and ranks governments accordingly, believes that open data is a right for all and, as such, governments should provide the public with data that people need and can easily use.

Fitness for use is considered a critical factor for data to be relevant (Juran, 1974), and thus usable (Attard et al., 2015). In other words, data should be appropriate for and applicable to whatever the intended goal is. This involves acknowledging the data’s context (Ruijer et al., 2017; Wang & Strong, 1996). For instance, if the goal is to encourage citizen participation, governments should provide citizens with contextually relevant information necessary for participating in a meaningful and informed manner. If it is feedback that is desired, governments should provide sufficient guidance to the public on how to go about submitting feedback, in what form, and where to find any additional information that may be useful for helping citizens participate in this way.

5.1Data

In addition to providing streamlined access to all published datasets, the Socrata platform allows governments to publish detailed and comprehensive metadata with each dataset including a description of the dataset, the category it belongs to, the agency that published it, when the dataset and its metadata were last updated, and how frequently the data are updated. Data owners and publishers (cities, agencies, departments, etc.) can also create custom fields for any additional details it chooses to disclose. For this study, the metadata for each publicly available dataset was collected using a web scraping package in Python. The Python code was programmed to crawl through the pages of NYC Open Data in order to collect the information on the landing page of each individual dataset. This resulted in an original dataset containing the metadata for all 2605 datasets published on the portal.

To answer the research questions, I conducted a content analysis of the 2605 datasets. Since this was an exploratory study, the aim was to note patterns and themes in the metadata that could help with first understanding the categories of data published and which agencies and their functions were publishing the data. Then I used the themes and patterns to qualitatively analyze the datasets in order to determine the most prevalent types of information and their relevance to five main stakeholder groups, as well as the open government principle (transparency, accountability, or participation) that the dataset promoted. As illustrated in Table 1, the analysis focused on specific fields of the metadata: the name of the dataset, the description provided of the data, the name of the agency that published the data, the category the agency assigned it to, and the tags used by the agency. Below, I expand on how I conducted the analysis.

5.2Data category

For the most part, the data owners (agency, department, etc.) categorized the datasets according to the ten categories listed in Table 2. However, 68 datasets (2.6%) were not categorized by the data owner, and 41 datasets (1.5%) were miscategorized. In the latter, a dataset was considered incorrectly categorized if the category was in conflict with what the name and description suggested, if it appeared to be a subcategory of another category (e.g. NYC Big Apps as a sub-category of “Business”), or if it was generally more appropriate for a different category. For example, a dataset titled “Businesses Receiving Training Fund Awards” was categorized as “City Government” but the agency that published the data was the “Department of Small Business Services” and the description provided noted that it was a list of businesses that received the award. The agency’s focus, the name given to the dataset, and its description suggested that “Business” was a better fit for the data than “City Government.”

5.3Agency function

There were 85 total unique data owners listed. These included large agencies such as the “Department of Parks and Recreation” or “Human Resources Administration” as well as commissions and advisory boards (e.g. Commission on Women’s Issues), public development authorities (e.g. New York City Housing Authority), and various offices and divisions within city agencies. The 85 unique data owners were then sorted into 9 broad agency functions (see Table 2) according to the types of tasks and duties it typically performs or is generally responsible for overseeing. “Other” was listed for eight datasets. A closer look at the metadata showed that these eight datasets were owned by organizations outside of the city including community associations and collaboratives.

5.4Type of information

Scholars argue that governments should use OGD as an opportunity to provide the public with data related to the core activities of government including the processes, procedures, and decisions made (Janssen et al., 2017). Since understanding the degree to which governments are indeed leveraging open data portals to provide this type of information was one of the motivations of this research, it was important to go beyond the categories self-selected by the data owners. The data categories provided were often very broad and did not offer much insight into what kind of information was actually provided in the data. Likewise, the names and categories given could be vague, ambiguous, and misleading, even if unintentional.

Understanding the type of information provided on open data portals is important for a few reasons. First, transparency research would suggest that it is not just access to information about education, in and of itself, that is valuable, but it is perhaps access to performance information about local schools, more specifically, that enhances transparency. Similarly, citizens may not be interested in information about “city government,” generally speaking, but may desire to know more about the city’s plan for broadband adoption, including opportunities to participate in decision-making processes concerning the plan. Lastly, different groups of stakeholders have different interests and information needs so understanding what type of information is actually provided on open data portals can help with understanding its relevancy to the many potential user groups.

Therefore, a more detailed content analysis of the dataset names, descriptions, and tags provided by the data owners was necessary for identifying the types of information the datasets contained in an effort to determine what the data could potentially be used for, and by whom.

5.5User groups

Dawes et al. (2016) argued that the success of open government initiatives rests on the ability of governments to account for the many competing demands of a diverse group of stakeholders including users, providers, and broad communities. However, understanding the many nuances of the data itself and the various ways that data can be used and by whom can be a challenge. Nonetheless, the literature provides insight into some of the main user groups of OGD. Using this literature as a framework, the results of the content analysis helped to determine potential users of the information provided. The user group analysis focused on analyzing the available datasets according to the relevance of the information provided to five main groups of OGD users identified in the literature: citizens, private sector, nonprofit sector, researchers, and internal city agencies.

For example, data that included information about public services, participation processes, and decision-making were coded as relevant to citizens. There is a dearth of research on which types of data citizens actually find relevant and most likely to use,22 so the data were coded according to a few specific criteria; mainly, whether the datasets provided information that could help citizens find out about city services, programs, performance, and policies. Any dataset that was of interest, involved, or advanced the mission of actors and organizations in the private sector was coded as private sector and, likewise, nonprofit sector, for those that concerned nonprofits. For example, information about contracts and bids are typically of interest to private companies that routinely submit bids to win contracts or find out who won a particular contract. Similarly, the city contracts out many human and social services to nonprofits, so data about service contracts, clients served, and information about social services are likely of interest to nonprofits and civil society organizations.

Many of the dataset descriptions mentioned specific research studies and partnerships between the city and external researchers. These data were either provided by or ostensibly, of interest to, researchers, and were coded as such. Lastly, since administrative agencies are not just data producers but data users as well (Dawes et al., 2016), datasets that agencies could use internally for agency-related tasks and in decision-making processes were coded as city agencies.

Appendix A provides more details on this process, including examples of datasets that were coded as relevant to each user group.

It is important to note that these categories are not mutually exclusive. Many of the datasets that were relevant to the private sector, for instance, were also relevant to nonprofits. Similarly, there was a great deal of overlap between datasets that were relevant to city agencies and the four other stakeholder groups.

5.6Open government principles

Lastly, using the open government principles of transparency, participation, and accountability to guide the analysis and coding, the datasets’ names, descriptions, and tags were also used to understand how the available data promoted or contributed to transparency and accountability, as well as citizens’ participation in democratic activities. Here, the focus was on identifying information that could increase transparency by providing insight into the inner workings of the city (Matheus & Janssen, 2020; Meijer et al., 2012) as well as data that provided citizens with relevant information for monitoring the activities of the city for purposes of holding the city and its agencies accountable for outcomes such as performance, progress toward stated goals, and generating public value (Gains & Stoker, 2009; Bovens, 2007; Shktabur, 2007; Meijer, 2006). Datasets coded as “particiption” explicitly referenced city-led engagement activities, information concerning formal venues of participation, and any information that would ostensibly assist citizens in participation in decision-making and other democratic activities. As with the user group analysis, it is important to note that there was often overlap. Many datasets were compatible with more than one of the principles.

5.7Measuring open data use

Open data use was measured using the number of views and downloads of each dataset. These data points were included in the metadata provided for each dataset and are updated routinely or as needed. The count of views refers to the number of times a user visits a dataset’s landing page through the portal. A view is counted any time a user visits the page, views the data table embedded on the web interface, or engages with the visualization tools (if available). Socrata, Inc. provides a number of visualization tools within the platform to allow users to build their own graphs, charts, and visualizations with the data as well. Any time a user visits a dataset’s page and interacts with any of the visualization components, it is counted as a view. Thus, views are a more passive form of use, as users are only observing and interacting with the data within the platform’s web interface. On the other hand, downloads represented the number of times a dataset had been downloaded for offline use by a user. Datasets can be downloaded in machine-readable file formats (CSV, XML, shape file, etc.) made available by the data owner. The extra step of downloading a dataset to one’s device for offline use suggests an intent to use the dataset for some purpose. Thus, downloads could be considered a more active form of open data use.

6.1What were the main types of data published?

At a broad level, Table 3 shows the breakdown of the number of datasets per category (provided by the data owner). Education was the most prominent category with over 1000 datasets (38.93%). These datasets mainly concerned school performance information including standardized test scores, disciplinary reports, and graduation rates. The next most prevalent category was City Government with 638 total datasets (24.49%). This data mainly consisted of budget documents, bids and contracts, agency reports, and strategic plans. Social Services data made up almost 8% of the datasets, followed by Transportation (6.22%), Environment (4.72%), Housing & Development (6.41%), Public Safety (3.3%), Business (3.03%), Health (2.8%), and Recreation (2.42%).

Table 3 shows the breakdown of the number of datasets by the agency and its function. As shown in Tables 3 and 4, there was quite a bit of overlap between the category of data and the data owner’s (i.e. agency, department, etc.) function. In other words, unsurprisingly, data categorized as “Transportation” was often published by transportation-related bodies. Likewise, data categorized as “Social Services” were published mostly by health and human services agencies. Therefore, since the focus of this research was mainly on understanding the type of data that was published, the remainder of the analysis focused less on the data owner (agency function) and more so on how the data were categorized by the data owner.

The content analysis of dataset descriptions highlighted six prominent topics represented among the datasets: performance information, budgets and financial reports, contracting and procurement, crime and public safety, elections and campaigns, and demographic data. The results of the content analysis revealed that an overwhelming majority of the datasets published to NYC Open Data were performance reports and other types of performance information (860 datasets). Second to performance information was demographic data (353 datasets) which also included U.S. census data and reports. Third, were budgets, including revenue and expenditure reports from agencies. These documents accounted for 166 total datasets. Table 5 shows the number of datasets for each of the topics identified. As shown, topics were not mutually exclusive with several datasets compatible with more than one type of information (e.g. “performance information” and “crime and public safety data”).

The dataset names, descriptions, and tags were analyzed further to identify datasets that promoted and/or contributed to the open government principles of transparency, accountability, and participation. This analysis identified 2535 (97.31%) datasets that promoted transparency and 1115 (42.8%) datasets that either promoted accountability measures or could otherwise be used to hold the city accountable for its performance, decision-making, or other policy-related outcomes. Whereas, only 189 (7.26%) datasets concerned citizen participation activities. The types of participation-related information provided was mostly geared toward formal city-led processes such as participatory budgeting, public meetings, and community advisory boards. However, information about general civic engagement activities such as lists of volunteering opportunities and activities and events that were more community-focused events (e.g. community-gardens) were also present.

Overall, the analysis revealed that an overwhelming majority of the data published on NYC Open Data was aimed at promoting transparency. Specifically, transparency of government performance, budget and financial information, and strategic planning. However, less than 10% of all of the datasets published directly informed or involved citizen participation activities. At the same time, as the transparency literature suggests, the amount of data available on the city’s performance, decision-making, planning, and policy outcomes can still enable citizens to monitor and evaluate how the city is performing. In doing so, citizens can use this information to monitor the activities of the city and hold the city accountable, if necessary. This suggests a potentially powerful indirect effect of the data available.

6.2What can this tell us about open government data as a tool for democratic participation?

Scholars posit that the success of OGD rests on government’s ability to account for the many competing demands of a diverse group of stakeholders including users, providers, and communities (Dawes et al., 2016). The benefits of OGD are expected to generate social benefits, including a more informed and engaged citizenry. However, OGD is also promoted as an innovative tool for generating economic benefits. This then begs the question of which outcomes and groups of stakeholders are most reflected in the types of data that governments choose to publish. As discussed earlier, for OGD to promote democratic participation, governments must publish data that is inclusive of the interests of diverse groups of stakeholders. Further, available data should be sufficient for informing meaningful participation. Therefore, the availability of data that is relevant and useful to citizens is especially important if OGD is to empower and enable citizen participation in democratic activities.

The analysis found that 722 (27.72%) of the datasets provided information that was of relevance to citizens. Note that this is different from information that directly informs or involves participation activities (discussed above) and that some datasets were relevant to two or more user groups. An overwhelming majority of the information coded as relevant to citizens was about government programs and services, including contact and directory information for agencies and/or public officials, information about and for community advisory boards, participatory budgeting processes and outcomes, and 311-related information. In contrast, 734 (28.18%) of the datasets were of relevance to private sector actors and firms as evidenced by datasets that concerned real estate transactions and development projects, information for and about business improvement districts, construction projects including bids, requests for proposals, and permits, and transportation-related information, especially for taxi companies and For-Hire-Vehicles. Additionally, 421 (16.16%) of the datasets were of relevance to nonprofits. The data mostly involved social services-related information, including information about and for nonprofit providers of public programs and services, grants, site locations, directories, as well as data about community demographics and public health, which is often used by nonprofits to assess community needs. Similarly, 239 (9.17%) datasets contained data collected by the city for research purposes, by external researchers, or relevant to those conducting research. For example, data from surveys, environmental studies, and city collected data about water and air quality were included in this group. Lastly, 1472 (56.51%) of the data published to the portal was of relevance to the city agencies, themselves. These were datasets that contained information that helps city agencies and officials advance the public interest, provide public services, and, otherwise, do their job. For example, there were a number of “trackers” that agencies used for project management purposes including capital projects, managing work orders, responding to citizen complaints, and even overseeing the city’s open data projects. Additionally, much of this data was relevant to internal human resources matters as evidenced by employment codes, salary schedules, pension benefits, as well as information related to agency-specific tasks, and key performance indicators in adherence with the mayor’s performance plan.

6.3What can the number of dataset views and downloads tell us?

To gain a better understanding of the data’s relevance and usefulness to the five user groups, it was beneficial to look at the number of views and downloads of the datasets to determine the extent to which users were engaging with the data. When it comes to engagement with the data, datasets relevant to the private sector, researchers, and city agencies were viewed the most, overall. Whereas, the highest number of downloads were of datasets relevant to the private sector and researchers. This suggests that these groups were frequently downloading datasets to use for specific purposes offline; perhaps to analyze for research projects or use for software development projects.

Since there was a lot of overlap in terms of user-group relevance for many of the datasets, it was helpful to look closer at the number of datasets in which each of the user groups was the sole audience. In other words, the datasets that appeared relevant to only one group, rather than to multiple groups. Table 5 includes the breakdown of the number of datasets and mean views and downloads for those datasets that were coded as relevant to only one user group. Here, we see that the number of datasets specific to internal city agency business (465 datasets) was much higher than the others and these datasets also had the highest mean views. This suggests that agencies may be using the open data portal to routinely look up information they need (i.e. for internal decision-making, service provision, etc.) or reference one of the many performance trackers present. Whereas, the datasets specific to researchers had the highest mean downloads suggesting that researchers are indeed using the portal to access datasets for research purposes.

The results of this descriptive analysis suggest that not only was the supply of citizen-relevant datasets lower than for other user groups such as private sector firms and internal administrative agencies, but it seems that the demand was too. This was evident by the relatively low amount of views and downloads for datasets compared to researchers, private firms, and internal administrative agencies, as well.

While it is not possible to see exactly who is downloading the datasets due to the proprietary nature of the software, these descriptive results suggest that users, regardless of affiliation with any particular stakeholder group, are using specific datasets that fulfill particular functions more than others. In all, the datasets published disproportionately advantaged private interests and the agencies themselves. When it comes to engagement with the data, the mean views and downloads, especially, suggested that researchers are some of the most active users of OGD. Of course, it is still likely that these data are being used for research purposes that generate public value for citizens. Nonetheless, these findings highlight potential barriers to inclusion when it comes to OGD-enabled citizen participation if the information citizens need to participate meaningfully is not available or accessible.