Census metadata driven data collection monitoring: The Ethiopian experience

1.Introduction

By definition, a population and housing census is an enumeration of the total population of a country, which provides data on numbers of people, their spatial distribution, age and sex structure, their living conditions and other key socioeconomic characteristics.

Such data are critical, inter alia, to national and sub-national development planning, monitoring progress for the Sustainable Development Goals (SDGs), distribution of infrastructure and social welfare programs, election planning and needs analysis.

While national registry systems are evolving world-wide, and in some countries replacing reliance of censuses, for the majority of countries, especially in most developing countries, the population and housing census remains the primary source of data on the size and spatial distribution of the population and its related characteristics, and this is likely to remain the case for the foreseeable future [1].

However, it is widely recognised that conducting a population and housing census is one of the most expensive and complex data collection operations, which includes a series of many interrelated activities.

A review of the 2010 Population and Housing Census World Program of the United Nations (UN) carried out by the United States Census Bureau (USCB) identified the main characteristics of the censuses taken under the 2010 round and the challenges and successes faced by the countries [2]. It highlighted that cost issues remain the principal challenge for governments (67% of all countries interviewed).

Further challenges refer to: (i) timely dissemination of data; (ii) assurance of data quality; (iii) decrease in response rates.

Such drivers have motivated most National Statistical Agencies and the international statistical community to investigate alternative ways of implementing the census and to take advantage of the new and evolving technologies available especially for constructing census maps, capturing and validation of data and dissemination of results.

The 2010 census round pioneered a variety of data sources and collection methods, introducing a wide range of technology solutions, from the use of the internet, laptops, tablet computers and other handheld electronic devices to Geographic Information Systems (GIS) and scanning and recognition systems.

The mentioned review of the 2010 Census Round reported that, as for data sources (where the data come from), a traditional census with full field enumeration was the main source of census data for 83 percent of the countries interviewed.

Face-to-face interviewing using a paper questionnaire was the main enumeration method (72 percent). However, most of the countries that used paper questionnaires, expressed an interest in using laptops, handheld electronic devices or the Internet in their 2020 census enumeration.

During the 2020 census round planning, this interest in the adoption of electronic data collection has been confirmed.

Many statistical agencies plan to use mobile technology for data capture (as the sole data collection method, or adopting a mixed-mode approach), changing from paper to electronic questionnaires (Computer Assisted Personal Interviewing – CAPI) on handheld electronic devices.

Such decisions affect the entire census lifecycle and makes it possible to improve the quality and timeliness of the entire census operation. With proper planning, governance and vision, it can also help to improve efficiency in terms of cost savings, provided the costs of the electronic equipment are carefully estimated and considered in carrying out ex-ante cost-benefit analyses.

As mobile and wireless technologies continually improve and become more affordable, reliable, powerful and user-friendly, CAPI is expected to become the standard in field-based data collection even in those countries where access to infrastructure for information and communication technologies is limited.

African countries are such a case. According to a preliminary report on the status of country preparedness for the 2020 Round of Population and Housing Census in Africa, in 2017 more than half of the African countries (about 57%) had decided to use CAPI in census taking [3].

Africa is very committed to taking part in the 2020 Round of Population and Housing Censuses. The vast majority of countries in the region have started preparing their national censuses. Some countries, for example Malawi and Kenya, have already completed the enumeration phase. Several other countries, including Ethiopia, are currently at a very advanced stage in their census preparatory activities.

To some extent, the impetus was created by the success of the 2010 round of censuses in Africa where 47 out of the 54 countries participated [4, 5, 6].

However, a number of countries needed, and in some cases still need, external support to mobilize appropriate financial resources, even for funding the initial start-up costs, and technical assistance to reinforce their capacity to manage the new technologies and successfully carry out all the phases of the census, through the dissemination and use of the resulting data.

Institutional partnerships at global, regional and national levels, mainly lead by the UN Population Fund (UNFPA) and the UN Economic Commission for Africa (UNECA), have been aimed at filling the national capacity gaps and ensuring the effective modernization of census operations.

In line with this approach, Italy supported the Ethiopian census through a project of technical assistance financed by the Italian Agency for Development Cooperation (AICS) and implemented by the Italian National Institute of Statistics (Istat) in partnership with the Ethiopian Central Statistics Agency (CSA), which is responsible for conducting the upcoming population and housing census.

In Ethiopia, census data will be collected mostly digitally, using tablets running the Android operating system. As a CAPI software system, CSA has chosen the Census and Survey Processing System (CSPro), the public domain software package developed by USCB [7].

CSA also decided to develop a monitoring system to receive regular reports on the progress of the enumeration activities and analyze (preliminarily) the quality of collected data.

This paper shows how Istat has supported CSA in designing and implementing such a monitoring system, which has been fully integrated with the census data collection process managed by CSPro.

The following paragraphs provide a description of the data collection process and the implemented architecture in the framework.

The Ethiopian Census is proposed as a case study to describe the main features of the system implemented to support data collection and fieldwork processes. The strong cooperation between Istat and the USCB, facilitated by the coordination and guidance provided by UNFPA, made such results possible.

2.Ethiopia Central Statistics Agency: Istat capacity building support

Capacity Building and Institutional Strengthening are integral parts of the technical support that Istat provides to local partner institutions in the framework of its cooperation projects. They represent the link between project outputs and sustainability and ensure that the improvements made are absorbed and maintained beyond the life of the project.

As for concrete implementation, these results are achieved putting a significant emphasis on working together, through coaching and advice activities aimed at transferring the know-how of Istat experts to their counterpart, introducing innovations in the partner institution’s processes and supporting the local staff with the overall goal of development of the targeted areas. Comparative analysis is part of the coaching activity in order both to present methodologies and practices which guarantee compliance with international standards and to identify specific strengths and weaknesses of the current systems.

The process of developing and strengthening the institutional capacity is done not only by transferring the methodologies and techniques appropriate for the national context, but also by introducing the corresponding tools and increasing the training component of the cooperation activities.

For a long time Istat has been experimenting with the development and use of generalized software for statistical production processes implemented by adopting open source instruments, also to meet need to make the software portable everywhere and reusable in different Countries requiring technical support [8].

This general approach has also been adopted to support CSA in preparing for the Fourth National Population and Housing Census.

The project, lasting 24 months (from June 2016 to June 2018), was structured into various components, one of which directly focused on establishing the IT infrastructure for the Census, including setting up the system for data collection monitoring.

Senior and junior experts from Istat, with the proper professional and technical requirements, carried out the activities as per the work-plan agreed with CSA.

For each mission, Terms of Reference (ToR) were prepared by Istat and agreed to in writing by CSA. Proper attention was paid to encourage CSA to identify, in advance, the census team (management, IT /GIS experts and statisticians) to be involved in the project, considering that appropriate commitment and availability of the officials and staff is essential to ensure effectiveness, ownership and sustainability of results.

Plenary meetings involving top and middle management were organized at the beginning and at the end of each mission, to share the progresses made. Meetings were also attended by representatives from UNFPA and other International development partners supporting the census.

Cooperation with other actors was constantly sought, both to create synergies and exploit experiences, and to avoid overloading staff with conflicting activities.

During the missions, specific training activities were organized. These activities followed mainly two objectives: i) to provide conceptual and methodological frameworks according to international standards and recommendations. As for the IT component, basic and advanced training on relational database, MySQL and Java language were delivered; ii) to transfer specific and technical know-how; these activities included trainings on census data architecture and on the use and maintenance of the monitoring system.

In all cases, a tailor-made mix of training methodologies were used, including both theoretical knowledge, concepts, models and analysis of case studies, practical examples and exercises.

In order to ensure continuity of on-going activities, Istat experts also provided their counterparts with remote support and assistance in between missions, both by e-mail and by using specific tools for sharing software.

Specific care has been devoted to the drafting and sharing of documentation for all the activities:

As for the software described in the next paragraphs, a User Guide was also released.

All these documents represented key deliverables of the project and remained as a reference for CSA in the future, serving the sharing and wide dissemination of innovations and therefore their re-use, ownership and sustainability.

3.Data collection process

Ethiopia has planned to conduct a paperless population and housing census, using mobile devices [9]. Mobile data capture offers a new set of capabilities that can improve the overall quality of a census, for example mobile devices allow: (i) accessing GPS data; (ii) displaying households on a map; (iii) real-time processing of collected data. Further, as data from each device are routinely sent to a central database, they can be elaborated to evaluate, in near real-time, the progress of the enumeration activities, e.g. identify which areas have already been covered at different geographic levels and produce early reports on population structure. Having access to this information in real-time allows adjustments to be made to field operations during the data collection to improve efficiency, coverage and data quality.

CSA decided to adopt CSPro, the public domain software package developed by USCB, to manage data collection. CSPro’s advantages include a wide familiarity and acceptance by a large number of users (it has been used in over 160 countries for censuses and surveys), power and flexibility to handle complex questionnaires and workflows, a full suite of tools for data capture, editing and tabulation and no license fees.

Being a free off-the-shelf solution, it requires the statistical agency only to invest in increasing the capacity of its own personnel that will be using the software. These skills and knowledge will be then reused in other survey operations beyond the census.

On the other hand, CSPro offers limited features related to fieldwork monitoring.

Microdata collected by enumerators are stored in plain text files. This means that, as the number of returned questionnaires increases, it becomes difficult and takes a very long time to extract and elaborate real-time information from the database. In addition, it does not offer a user-friendly interface for monitoring.

The software developed by Istat fills these gaps, ensuring an almost real-time monitoring of the primary data collection process managed by CSPro [10].

In this section, we provide a high-level description of the data collection process implemented for the Fourth National Population and Housing Census in Ethiopia. The proposed approach is quite general and can be applied to support Censuses in many countries, assuming that the CSPro software is used to collect microdata.

To describe the process we used the ArchiMate modelling language, i.e. a formal language implemented in the context of Enterprise Architecture Standards, to describe processes, actors and the input/outputs involved in the process chain [11, 12].

Figure 1.

GSBPM 4.3 ‘Run collection’ process (business view).

The process described in the paper corresponds to sub-phase 4.3 ‘Run collection’ of the GSBPM ‘Collect’ process [13]. Firstly, a description of the data collection process at the ‘Business level’ is given, i.e. a description of ‘what’ has been implemented, without providing any technical detail (the ‘how’ perspective that will be described in the following section). In the approach the coarse-grained GSBPM 4.3 ‘Run collection’ is split in three business processes (see Fig. 1).

This business process is composed of a set of sub-processes that allow the creation/loading/updating of reports during the enumeration phase, which will be described in the following section. It is important to stress that to display the reports it is necessary to implement a software component, the dashboard, which allows accessing the content of the reports and displaying them on tables, charts and maps. The dashboard is a key element in the fieldwork monitoring activities.

4.Data collection architecture

In order to implement the data collection process described above, it is necessary to design a metadata driven architecture, which allows generating the microdata database and the dashboard reports, parsing questionnaire metadata (CSPro dictionaries).

In this section, a brief description is presented on ‘how’ the GSBPM 4.3 ‘Run collection’ process was implemented.

4.2Data transformation

Cspro2sql is the application component ‘implementing’ the data transformation process displayed in Fig. 2. Cspro2sql is an open source software released under the EUPL license, developed by Istat in the framework of the Capacity building project in Ethiopia. The main functionalities offered by the software are: i) parsing of CSPro dictionaries; ii) creation of the questionnaire relational database; iii) loading of questionnaires; iv) report generation.

A complete guide for cspro2sql (engines, parameters and settings) is available on the web page of the project [14].

Figure 3.

Dashboard home page. The home page of the dashboard contains a map displaying the status of the enumeration. On the right side, the following progress indicators are displayed: i) total households returned from the field (ENUMERATION); ii) total expected households from the listing questionnaire (LISTING); iii) ratio of household, listing totals (COVERAGE). Data displayed on the map has been randomly generated.

4.3Fieldwork monitoring

This process allows generating reports to monitor the progress of enumeration activities and analyze (preliminarily) the quality of collected data. Reports can be classified as follows:

• 1) Progress: reports that provide coverage information at different levels of geography. Generally, the coverage, at each geographic level, is obtained by taking the ratio between the aggregated data collected from the field (household questionnaire) and the aggregated listing data (used as a benchmark). In addition, if the number of households from a pre-census cartographic field operation is available, it may also be used as the denominator to calculate coverage.

• 2) Analysis: in order to monitor the quality of the data collected from the field, the system provides a set of reports on questionnaire variables, e.g. average household size, average age (male/female), sex distribution (male/female). These reports are automatically generated by parsing the domain specific metadata described in the previous section. The current version of the software analyzes only the sex, age, and religion variables. Future releases of the software will analyze a wider range of variables.

• 3) GIS: integration of ‘tabular’ reports with GIS maps is of utmost importance, as it allows users to instantly check the progress of enumeration activities and to identify which areas have not been covered. The current version of the dashboard provides a map report displaying enumeration coverage at the region level (Fig. 3). Further, in pilot surveys, it is possible to enable a report displaying a marker for each household interviewed.

The list of sub-processes needed to generate these reports is displayed in Fig. 4. The application components ‘implementing’ these processes are cspro2sql and the dashboard.

Figure 4.

Fieldwork monitoring process: the upper part of the image corresponds to the ‘business view’ of the ‘Fieldwork monitoring’ process. The lower part shows the ‘application components’ that implement all sub-processes. More specifically this process is ‘realized’ by cspro2sql and the dashboard.

The dashboard is a web application implemented using open source Java frameworks. This application allows displaying the reports generated by cspro2sql in tabular format, on charts and on maps (see Fig. 3). Istat developed the dashboard in the framework of the capacity-building project in Ethiopia. A detailed description of the architecture and of the main functionalities of the dashboard is available on the web page of the project [15].

5.The Ethiopian Census

This section provides a high-level description of the work done by Ethiopia Central Statistics Agency (CSA) in the context of the Fourth Population and Housing Census.

As already pointed out, Ethiopian census data will be collected almost all digitally, using tablets running the Android operating system. About 180,000 devices will be used during the enumeration phase: this number of devices adds a dimension to the complexity of the census exercise and provides an idea of the scale of fieldwork operations.

Here the main focus is on two key aspects: the questionnaire design and the application of GIS technologies to data collection. Such activities have required a full commitment by CSA staff, outside of the framework of the Italian support.

5.1The questionnaire

The Fourth Ethiopian Population and Housing Census has two main questionnaires

• 1) Household listing questionnaire: a short questionnaire used to number and list all houses and households in each enumeration area three days before the census date.

• 2) Household questionnaire: the main census questionnaire that collects data on household members socio-demographic characteristics and houses amenities and facilities. This questionnaire contains nine sections such as enumeration area identification, type of residence and housing unit identification, details of household members (socio-demographic variables like sex, age, relationship to the head of household, religion, ethnicity, disability, migration, marital status), education and information technology, economic characteristics, fertility and child mortality, death in the household last 12 months, emigration and housing questions.

Figure 5.

Architecture for mobile census enumeration area mapping.

The data will be collected using a CAPI data collection application implemented in CSPro running on Android tablets. The application was developed by CSA programmers with the guidance of the US Census Bureau. The application has three components, the menu application, the household listing application and the main questionnaire application. The menu application helps supervisors and enumerators perform their assigned task like registering supervisors and enumerators, assigning enumeration areas, collecting household listing data, individual information and housing data. It also generates different types of reports for enumerators and supervisors which are used to monitor the progress of the census in the field. . The listing and household applications are able to collect data about identification of houses, purpose of the housing unit, residential and non-residential houses, households engaged in agricultural practices, number of usual household members by sex. The literal question text of the two questionnaires are prepared in six languages including English and five local languages (Amharic, Affan Oromo, Tigrigna, Somali and Affar). In order to control data quality on the spot, the application contains quality controls (range and consistency checks) as well as reading latitude and longitude from the tablet’s GPS for each housing unit and household during listing and the main census. The national census is designed to cover all population groups dwelling in the country including nomadic populations according to the UN principles and recommendations. In order to ensure the enumeration of the nomadic population, the census date is selected based on their availability in their usual stationed area in consultation with regional states and local leaders. The application is also able to transfer data and any updates between enumerator’s tablets, supervisor’s tablets and the server at headquarters. Assignments, data and any updates are transferred between enumerator and supervisor tablets using Bluetooth while supervisors receive application updates and send data to the headquarters server through a dedicated network (VPN) for the census only using a special SIM card. To receive data at headquarters, CSA uses a web application called CSWeb, that allows users to securely transfer cases (questionnaires) and files between client devices running CSPro and a web server.

6.Conclusions

The proposed architecture provides a simple solution for monitoring electronic data collection operations, particularly in cases where the technical and financial resources to implement such a system from the ground up are lacking.

Compared to the traditional paper-based approach used for previous population censuses in Ethiopia, the proposed system provides more timely and accurate monitoring of field activities while reducing the workload of field supervisors. With paper-based data collection, monitoring is accomplished by having field supervisors fill out summary sheets which are then physically sent up the supervisory chain until they reach headquarters. This summary of data is manually aggregated at different coordination and supervisory levels and finally used to compute indicators at the national level. This approach not only requires significant work on the part of field staff but also introduces delays and possible errors as the information is transmitted from the field to headquarters. The proposed electronic system provides more timely and accurate data which allows supervisors to intervene early as soon as problems in the field are detected.

The integrated architecture described in this paper meets the following requirements: (i) it is generalised, i.e. applicable to different cases without the need to develop ad hoc code; (ii) it does not require financial resources to be acquired.

The implemented software has been used in Malawi and Kenya, to monitor census fieldwork. Potentially, it could be reused to support the census data collection phase in other countries where CSPro will be used as an application for electronic questionnaires.

The requirements above underpin the sustainability of the solutions that donors and their implementing agencies, such as Istat and the USCB, design to support statistical agencies of partner countries.

Furthermore, the results of the work jointly carried out by CSA, Istat and USCB reaffirm the value of institutional partnerships and the need for strong cooperation and coordination among partners, for improved synergies, to avoid duplication of investments, and above all support evidence based decision.