Gender-responsive remote data collection, analysis, and use during crisis: UN Women’s Rapid Gender Assessments (RGAs)

1.1COVID-19 is not gender-neutral

While COVID-19 pandemic was a universal experience, it has disproportionate impact on women and may have even re-opened some aspects of gender gaps that have already been closed [1]. Economically, women were the first to bear the effects of job loss since COVID-19 affected more women-dominated sector and women-dominated industries (such as tourism, service, and education) in both formal and informal sectors. The lockdowns and mobility restrictions of the pandemic also put a toll on women’s social lives. Prior to the pandemic, women were already spending three times more hours than men on unpaid care work [2]. With pandemic-related lockdowns affecting work and school set-up, this situation worsened as more household members stayed at home and women were expected to take on the increasing unpaid care and domestic work. For victims of violence against women, this adds risk as their perpetrators may be at home with them. Lastly, women have particular medical care and services needs11 and access may be difficult during COVID-19.

1.2Standard data collection for collecting gender data was constrained

Baseline and new data were needed to provide immediate gender-sensitive recovery response and to form evidence-based policies for the lingering impacts of the pandemic. However, baseline gender data were very little. On average, data are missing for 67% of SDG gender indicators [3] prior to the pandemic. For new data, COVID-19 limited the data collection efforts of gender data producers. The standard face-to-face data collection used in official statistics was not possible due to closure of some national statistics offices (NSOs),22 mobility restrictions, and to not mitigate health risks to both the interviewers and respondents.

But gender data necessity called for data collection innovations. National statistical agencies, development partners, as well as international organizations, have explored alternative ways of collecting data, particularly, the use of telephone- and web-based surveys to measure the gendered impacts of COVID-19. UN Women is one of the first international organizations who pursued remote data collection to measure the socioeconomic impacts of COVID-19 and the impacts of the pandemic on violence against women through a total of 78 national Rapid Gender Assessment Surveys (RGAs) which will be further discussed in Section 2.

1.3Remote data can address certain gender data gap but comes with additional layers to address gender sensitivity

While differences in women and men’s experience and perspective must always be taken into account in all stages of any survey design, added attention should be given to remote data collection.

First, in terms of reaching the sample respondents, there is digital divide between women and men. Only 57% of women were using the Internet compared to 62% of men [4]. While women’s mobile ownership is at 83%, women were still 7% less likely to own mobile phone than and the gap is further driven by range of factors such as location, literacy, age, disability, and employment [5]. In implementing remote data collection, data producers should be mindful that no one is left behind.

Second, there are many sensitive gender questions that cannot be simply included in a remote survey without any safety protocols in place. Moreover, in standard face-to-face (F2F) data collection, the interviewer has more chance to build rapport and gain the trust of women respondents. In phone surveys, there is a little window to do that. There is also not much chance to gather information that are usually being observed in standard F2F data collection such as presence of other people in the location of the respondent who may hear the conversation. Thus, proper ethical methodology should be added to assure and ensure the safety of women in answering these questions.

Lastly, lessening the respondent’s burden is more critical in phone surveys. Women, for example, who are usually doing unpaid care and domestic work may find the call taxing. At any moment, the respondent can just drop the question in remote data collection. In order to guarantee that enough information will be collected, questionnaire designers should be conscious of the order and phrasing of the questions.

1.4Objectives of the paper

This paper presents UN Women’s experience in intentionally applying a gender-responsive approach in remotely conducting rapid gender assessment surveys (RGAs) across regions. Section 2 provides more background information on UN Women’s RGAs. Section 3 specifies the gender-responsive approaches in various stages of data collection. Based on the learnings, recommendations were provided in Section 4 to further improve gender-responsive remote data production, analysis, and use. While this paper was produced in the context of remote gender data collection during COVID-19, some learnings and recommendations may also apply to producing gender-responsive data in general and to other crisis and emergency situations.

2.1RGAs on the socioeconomic impacts of COVID-19 (RGAs on SE)

In collaboration with governmental entities (particularly NSOs and national women’s machineries), national partners, and other development partners, UN Women produced much-needed gender data from 65 RGAs on the socioeconomic impacts of COVID-19 (RGAs on SE) between March 2020 and March 2022.

RGAs on SE collected information from over 150 thousand women and men, allowing gendered comparison on five key gender areas: a) participation in the workforce; b) unpaid care and domestic work; c) emotional and physical well-being; d) access to goods and services; and e) relief and social protection measures. To minimize respondent burden, RGAs on SE asked direct questions on relative changes (i.e. increase, decrease, does not change) instead of exact measures of the impact of COVID-19.

Various data collection approaches were used in RGAs in SE, but majority were through computer-assisted telephone interviews (CATI) wherein respondents were selected through Random Digit Dialing (RDD). For web-based RGAs, two major selection were conducted. In some countries, a text message was sent to randomly selected mobile phone numbers with a link to a web-based survey while in others, “web-based system delivering anonymous opt-in surveys to random Internet users browsing online was employed [6].”

In a publication compiling and analysing the pooled results [6], RGAs on SE corroborated previous findings that women were disproportionately suffered the socioeconomic impacts of COVID-19. Women were found to be more likely losing jobs and reducing work hours while at the same time, increasing intensity of care and domestic work. Consequently, they were less likely to receive relief or social protection, either from governments or non-government organizations (NGOs).

2.2RGAs on the impacts of COVID-19 on violence against women (VAW RGAs)

UN Women led the implementation of the RGAs that were specific to the impacts of COVID-19 on violence against women. Implemented in 13 countries (Albania, Bangladesh, Cameroon, Colombia, Côte d’Ivoire, Jordan, Kenya, Kyrgyzstan, Morocco, Nigeria, Paraguay, Thailand and Ukraine) between April and September 2021, RGAs on the impacts of COVID-19 on violence against women (VAW RGAs) covered responses of over 16 thousand women on their experience of violence, feelings of safety, and perception of VAW.

VAW RGAs were successfully conducted using CATI, but recognizing the sensitivity of the matter and possible presence of perpetrator at home, ethical and safety protocols were put in place. This includes ensuring that women were in a safe location (e.g. they are alone and no one can hear the responses), knows how to use a safe word in case needed, and speakerphone or call recording were turned off. Questions were asked in an indirect manner and responses were mostly limited to short neutral responses. At the end of the interview, women were provided list of local VAW helplines.

Aside from the 13 new nationally-representative data on violence attesting that VAW has indeed intensified during the pandemic [7], VAW RGAs produced evidence-based guidance on remote VAW data collection [8] and technical reports documenting the learning of each country at all stages of survey implementation.

2.3Timely gender data can transform lives of women

Gender data have tangible and measurable impact on the lives of women. Data from RGAs were used to develop gender-responsive programmes to counter socioeconomic impacts of COVID-19. In Maldives [9], women working in the informal sectors have been reached through inclusion in Income Support Allowance because of RGA results. The RGA in Albania [10] revealed that 67% of women loss income from farming and this drove redesigning farm subsidies to better reach women. The RGA findings on lost jobs and decreased income in Colombia [11] had justified programs prioritizing women such as the Formal Employment Support Program which gave out 50% subsidy to women compared to 40% for men and the Colombia’s Public Employment Service which placed around 300,000 women in formal employment during COVID-19.

RGA findings were also utilized to inform national plans and policies aimed at improving women’s wellbeing. In Colombia [11] and Chile [12], results on unpaid work informed upcoming national care policies. Meanwhile, in Jordan [13], the addendum to the 2020 National Strategy on Women in Jordan benefitted from some findings in the RGAs.

Results from the RGAs were also used to prepare for more gender-responsive data collection and activities. In Samoa and Philippines [6], the RGA methodology were referenced in national surveys on impacts of COVID-19. In countries like Albania [10] and Senegal [14], RGA findings served as a major tool in resource mobilization.

3.1Fostering inclusive collaboration

UN Women partnered with different agencies in implementation of the RGAs, such as government (particularly NSOs and MoWs), NGOs, civil society organizations, development partners, and partner agencies. Inclusive collaboration contributed majorly to the promotion of the use of gender data in strategy, advocacy, and programme formulation. It also contributed to three key outcomes.

First, it resulted to comprehensive identification of gender data needs coming from different disciplines and points-of-view. Gender cuts across various topics and in measuring the gendered impacts of the pandemic, there were several data producers and users involved in addition to national women’s machineries who usually formulate gender policies. The discussions allowed for richer analysis of different aspects of women’s lives that were intended to improve the quality of lives of women.

Second, it guaranteed that were no duplication of efforts. It was necessary to coordinate with NSOs, the official source of gender statistics and usually provides approval of non-official surveys in the countries. Non-duplication of efforts was beneficial to women respondents to avoid respondent fatigue in case of re-selection. This was a particular potential problem for smaller countries with few enumeration areas.

Lastly, it promotes sharing of innovative methodologies among data producers, learning from the challenges, and adopting the best practices in ensuring the safety of women in remote data collection. In designing the questions on employment in RGAs on SE, UN Women consulted with International Labour Organization (ILO). VAW RGAs learned from the experience of World Bank in using indirect methodologies to capture women’s sensitive experience.

3.2Leaving no woman behind in sample design

RGAs were conducted with individuals as units of analysis. This guaranteed equal selection of women and men in sampling compared to using household which only selects one household member and is usually biased towards the head of the household.

To further commit to leaving no woman behind, RGAs also designed its sample for phone surveys to capture differences among women groups. This was done through quota sampling for age, geographic locations, and if applicable, socioeconomic characteristics.

VAW RGAs was able to generate statistics for older women (60+) which was not usually covered in health surveys where usual VAW statistics were generated. However, it did not come easy. There was difficulty in completing the required number of sample for older women. To resolve this, innovations in sampling frame sources was applied with the aid of database from previous surveys of the partner survey research firm of respondents who have indicated they can be contacted in future similar surveys.

While there is inherent bias in sampling for phone- and web-based data collection due to mobile and internet access as discussed in Section 1.3, the sampling quota (for phone surveys) and weighing strategy (for web surveys) employed in RGAs to generate national representative data were based from official statistics from NSO such as censuses and national household surveys.

3.3Training for gender-responsive data collection44

At least two modules were added in RGAs to ensure that the interviewer trainings highlight the essentials of collecting data from women remotely.

First, in training interviewers for remote data protocols and preparing them from points of difficulties. This includes topics on building rapport with respondents given the little window of opportunity, mitigating refusals by giving all survey and privacy information and rescheduling, minimizing non-response to questions, and rigorous application on safety and ethical protocols.

Second, a sensitivity training on handling gender topics and issues covered in the survey must be included in the survey. For VAW RGAs, interviewers were trained to conduct the survey professionally but still with empathetic neutrality by “not make assumptions, and should overcome their own prejudices and stereotypes, if any, with regard to women who experience violence [6].”

3.4Designing a questionnaire with least burden to women

The instrument for the RGAs was kept short and concise (approximately 15 minutes) to not put additional burden to respondents, particularly to women who bears more unpaid domestic and care work. In one region, the East and Africa region, where it was necessary to cover more items, the RGAs were conducted in two phases.

The RGA questionnaires prioritized items that were intended to be use and structured the questions for easy implementation. Responses were also limited to single-word neutral responses to protect women’s response on sensitive item. Questions were translated to local language for ease of communication with respondents and response categories were adjusted to local context as needed.

Pre-testing the RGA questionnaires proved to be helpful in ensuring that the questions were clearly conveyed and that the technical aspects such as filter questions were working well to avoid any issues like asking unnecessary questions.

3.5Ensuring safe participation of women in the surveys

While gender data is a necessity during crisis, guaranteeing women’s safety should rank higher than this. RGA implementation centered on keeping women’s safety all through-out the interview. Similar to F2F surveys, a proper introduction was given detailing the contents of the survey and the name of the organization, how the respondent was selected, privacy and confidentiality information, and agreeing to consent agreement. In sensitive surveys like VAW RGAs, the ethical and safety protocols are usually shared in the introduction section. During the surveys, interviewers must strictly follow the instructions they learned during the training, particularly in being proactive to women’s needs. In implementing RGAs, a woman can request the interviewer to stop at any time she feels uncomfortable, and the interviewer can must honor this. At the end of the surveys, particularly for surveys tackling women issues, a list of helplines and other information may be given to the respondents.

While RGAs have been respondent-centric, the surveys have also observed the safety of women interviewers, who might also be vulnerable in some topics discussed. In an as-needed basis, counselling, check-ins, and support were available for VAW RGAs interviewers [8].

3.6Going beyond sex-disaggregated statistics

One assumption about gender statistics is that it is simply sex-disaggregated data. To ensure that policies to be generated from RGAs will leave no woman behind and vulnerable women groups are being reached, RGAs provided analysis beyond sex and produced multi-disaggregated statistics on age, marital status, and presence of children in the household. This revealed unique and varying experiences of women that can guide multi-level gender-responsive actions. For example, in both RGAs on SE and VAW RGAs, women living with children were seemed to be at disadvantaged. Women living with children were more likely to lose their job and see increase in unpaid care and domestic work than men [6]. In terms of experience of violence, about 1 of 2 women living with children experienced violence or know a woman who has [7].

RGAs was conducted in multiple countries. While global reports were generated based on pooled estimates, analyses reported were always mindful that there were cultural differences among women and the level of providing information remotely varies.

3.7Promoting gender data use to all types of prospective users

RGA results are currently available at the UN Women Data Hub.55 The dedicated webpages also include useful materials such as questionnaire, data dictionary, and statistical/technical notes. For RGAs on SE, the harmonized microdata is available for most countries. While these promote accessibility, UN Women ensured that the microdata and other documents produced were anonymized and confidential.

The RGA dashboards were meant for a general audience wherein uptake was not definite. In addition, the dashboards were only made available after the survey. However, to intentionally reach prospective users, irrespective whether F2F or remote surveys, promoting gender data use should be done as early as the planning stage wherein intended data use will play a big role in designing the survey questionnaire instead of the other way around.

Despite a substantive list of demonstrated use cases in Section 2.3, intended use cases were not systematically covered in RGAs on SE. In contrast, VAW RGAs have been mindful of use cases in all stages of the survey starting during the seeking of commitments with national partners, internal policy discussion with UN Women’s Ending Violence Against Women section, monthly updates provided to regional and country offices, and a series of briefings with UN Women colleagues and national partners once data became available.

4.1Monitor intended and actual gender data use

Data use should not just be an afterthought. It would be ideal to ensure that prior to data production, intended use will already be identified. That way, gender data would not be put in a waste and the potential of it in transforming the lives of women will be realized. In monitoring intended use cases, this paper recommends that as much as possible, all gender data users (this includes not only the NSOs and national machineries for women’s advancement but also national and international partners) be included in the discussion to ensure the comprehensiveness of list of intended data use and that all specific policy and programmatic gaps are at the tables.

There is also a need to systematically monitor how the data were actually being used. This will demonstrate that value of the gender data produced and can be a guiding mechanism for future gender-responsive data collection efforts and other interventions.

4.2Mainstream gender in statistical operations for measurement of crisis remotely

The results of RGAs and other studies conducted during COVID-19 and other crisis had reflected how women were unreasonably affected during crisis. This paper recommends mainstreaming a gender-responsive approach in statistical operations whether in F2F or remote data collection and whether crisis-based or regular operations.

This goes beyond collecting sex and other intersection variables, but a) incorporating variables on understanding how women were being affected by the topic being covered and b) in practice, ensuring women’s unique considerations on participating in surveys. This way, gender data are readily available to inform evidence-based gender-responsive policies.

This applies not only to stand-alone surveys, but also putting gender perspective on implementing modular surveys or reprocessing existing survey data. In modular surveys, one point that needs to be thought through is the sampling integration, particularly when the module is to be attached to household surveys (see Section 3.2 on why individual level analysis is required). In terms of reprocessing existing survey data, to ensure that the data that will be generated is of value to women and matches to actual data needs, the following should be looked into: a) geographic coverage, b) target population, c) disaggregation level, and d) relevant reference period.

4.3Support the NSOs in exploring alternative gender data collection methods

While many organizations have already explored the use of remote data collection and backed its advantages, a plethora of questions still exist on how the results compare to the standard way of doing F2F surveys in low- and middle- income countries,66 how sampling can further be improved to cover all population groups, and if it can be integrated in regular survey implementation.

The NSOs and other actors in the national statistical systems can play a huge role in answering these questions. Currently, some NSOs do not have the technical capacity in performing remote data collection. In most NSOs, official statistics uses sampling frames coming from F2F data collection which covers every member of the population. Their expertise on sampling can be fuelled to design a nationally representative sample of individuals that can be reached remotely while still abiding by the privacy and confidentiality agreements. Integration of various sampling frames may also be possible.

In terms of other needs, NSOs may need further support to jumpstart the exploration of this alternative data collection effort. Remote data collection requires a different knowledge set and equipment. International agencies can partner with NSOs on building capacity and exchanging skills set to improve remote gender-responsive data collection, analysis, and use.