Developing a future pipeline of applied social researchers through experiential learning: The case of a data fellows programme

2.1Q-Step programme – teaching ‘quant skills’

The aim of the UK-wide Q-Step programme was to create a step-change in teaching quantitative research methods to social science undergraduates, as a ‘systematic response to the shortage of quantitatively skilled social science graduates.’ [1]. David Willetts, the former government minister who announced the launch of the programme, referred to the need for ‘STEM skills in the social sciences’ [2]. The seventeen Q-Step Centres funded through the programme across the UK were encouraged to be experimental and innovative in their approaches to developing undergraduates’ ‘high-level quantitative skills’ [3, p. 1]. Although the Centres took differing approaches several common themes emerged. These included: developing new data-driven curricula; understanding how to develop and improve the quantitative skills pipeline from undergraduate to postgraduate study, and careers; and addressing the need for more practice-based opportunities for students to grapple with the realities of data analysis. Research work placements and internships, and summer schools, sprang up across the Centres as a way to address the need to learn how to do data analysis outside of the classroom. Practising data skills in real world contexts and developing, within and outside the classroom, the pipeline of graduates able to undertake quantitative social research, came to characterise the Q-Step programme.

Grundy [3, p. 5] provides a summary of the background to and inception of the Q-Step programme, together with a snapshot of the multiple outputs across the centres drawing on the mid-term external evaluation of the programme. He further expands on the programme’s definition of ‘quant skills’ which includes the ability to:

The work reported in this paper focuses largely on the ‘quant skills’ listed above. We use a broad definition to capture the wide range of analytical and research skills required to undertake quantitative social research. This paper cannot reflect on how these skills were taught across the Centres, although this forms the basis of a forthcoming external evaluation of the programme. Rather we seek to illustrate how one of the Q-Step Centres addressed the need to develop data and statistical skills training in its undergraduate student population, and how this was made possible through the introduction of a work placement – data fellows – initiative. Importantly, this involved building on a teaching approach that taught these skills through a social science subject lens thereby equipped students with the knowledge, skills and confidence to apply for these experiential learning opportunities.

University of Manchester’s Q-Step Centre and the data fellows programme

The University of Manchester’s Q-Step Centre provides the case study upon which this paper is developed. Being just one of the seventeen Q-Step Centres we make no claim regarding the generalisability of the findings, offering it rather as an example of what was achieved at one research-intensive UK university, and thus illustrating what is possible when universities work in partnership with industry.

Our response to developing quantitative social scientists at the University of Manchester is two-fold. We aim:

In Section 2 of this paper we provide some examples of how we teach data analysis in three of our degree courses to achieve our aims. Elsewhere we discuss the approach we have developed to teaching quantitative research methods that are embedded in substantively led social science [5, 6]. Approaches developed by others in teaching quantitative social science (and indeed social science research methods more generally) at different levels indicate this is a growing, but under-researched, area [7, 8, 9]. There is a strong literature on how quantitative skills and statistical reasoning are taught, although this appears most prominently in STEM and educational (rather than social science) spaces. For example, the International Association for Statistical Education’s (IASE) Statistical Education Research Journal (SERJ) provides papers by academics and practitioners spanning more than a quarter of a decade and the Guidelines for Assessment and Instruction in Statistics Education (GAISE) report provides comprehensive guidelines and posits a blueprint for teaching statistics [10]. Moreover, the IASE’s International Statistical Literacy project11 which was originally established in 1994 (as the World Numeracy Program) to ‘spread quantitative skills around the world in areas and populations … that could benefit from increased knowledge of numbers and their applications, with particular regard to statistics’ is an educational initiative that shines a light on the pedagogy of statistics. Education in general, and STEM education in particular, thus has a rich history in teaching and reflecting on the acquisition of quantitative skills. There is also a vast literature on experiential learning, especially in vocational teaching (see [11, 12]). Koparan and Guven show how experiential learning approaches, such as project-based learning, can result in success in teaching statistics (specifically data representation) in the secondary school classroom [13]. There is a paucity of research however examining how quantitative skills relevant to applied social research are acquired in the workplace; that is to say, how experiential learning supports the acquisition of data and statistical skills in the workplace. This paper sets out to address one way in which we can address this gap, through providing examples which evidence how a data fellows programme equips students with the skills and experience to enter graduate careers which require training in quantitative research methods and data analysis.

Here we provide a summary of the data fellows programme, with a more thorough exposition available in ‘Work placements, internships and applied social research’ [14]. The University of Manchester’s Q-Step Centre established a paid internship programme, through which social science undergraduates studying data analysis units as part of their degree courses could optionally apply to undertake a summer work placement in a host organisation. We opened the programme in 2013, with 19 students participating in the summer of 2014. By the summer of 2019 we had placed a total of 236 data fellows in over sixty host organisations. The students were paid the living wage to undertake these data-driven research projects which were co-designed with the host organisation (we call them hosts, rather than employers, as the students became casual employees of the University for the duration of the internship). This required significant input from the data fellows programme lead, but the return on investment has proven to be encouraging, as this paper shows.

One measure of success for the data fellows programme is the establishing of partnerships and graduate employment opportunities between the university and organisations undertaking applied social research (see also Section 4). Eight of the ten case studies in Carter’s book [14] are taken from the programme, with vignettes provided from early career researchers in social research roles. Examples of organisations that have hosted our students include government departments (including the Office for National Statistics, the Department for Education and the Home Office), think tanks and data consultancies, market and social research agencies (e.g. Ipsos) and polling companies (e.g. YouGov), charities and voluntary organisations (AgeUK, Respect), media organisations (BBC, the Times) and The World Bank. These hosts are selected to reflect the types of organisations we want our undergraduates to acquire experience in, and taking the broad definition of ‘quant skills’ as presented earlier, we focus on projects that will enable them to apply the skills they have learned in the classroom and PC lab. After all, statistics and data analysis require practitioners as well as theoreticians. We were mindful of creating practical experiences, to build on the taught components of the research methods curriculum and help develop graduates who will consequently be able to evidence their skills, in part gained through experience in the workplace, and therefore be in a strong position to enter the labour market. A framework for supporting data fellows to elicit and capture analytical and research, and professional, skills is provided by Carter (14, Chapters 7 and 8).

2.2Data and statistical literacy skills needs

The data fellows programme described is set firmly against the backdrop of a global need for data and statistical skills development. The most obvious example of this is the UN Sustainable Development Goals22 (SDGs), which have established the need for a data literate global citizenship, and we report elsewhere on how we are exploring the potential of the data fellows model to develop skills capacity to deliver on the SDGs [15].

There is clearly a societal need for data and statistical skills development, globally, nationally, regionally and locally. Our response to this need is to align a social science university education with the public, private and not-for-profit sectors to support skills training. The capacity development is also spearheaded by the IASE’s ISLP,33 a global project to coordinate efforts to improve statistical literacy through education. Much human capital investment is needed if we are to deliver on the skills development required to achieve these ambitious global goals.

The UK National Data Strategy (NDS) [16] makes reference to a ‘data skills taskforce’ which contributes to ‘the data skills ecosystem in order to improve the leadership and facilitation of new and better collaborations between industry, the public sector, universities and institutes’ [16]. The NDS also uses a broad definition of data skills, noting that basic data literacy should be included, which requires ‘some knowledge of data uses, some ability to assess the quality of data and its application, and the skills to conduct basic analysis’ – a definition which accords with the ‘quant skills’ one given above. Rather than attempt to define precisely the terms data and statistical literacy or skills, in this paper we adopt these looser and pragmatic definitions. Our teaching and the skills our data fellows acquire and develop in the workplace create a self-reflective iterative feedback loop which enables us to be responsive to industry needs, whilst creating tailored opportunities for our students building on how we teach them.

2.3Experiential learning

We noted above that whilst data analysis is a practical skill set, publications reflecting the teaching of it at least in the social sciences seem not to have permeated the literature on experiential learning. We argue here that this is a gap that needs to be filled and propose that experiential learning provides a useful framework for the teaching of quant skills, especially when delivered through work-placed project-based experiences.

Carter (14, Chapters 1 and 2) covers the theoretical background to experiential learning and its relevance to applied social research as well as the benefits to different stakeholders (that align with those acknowledged in the National Data Strategy cited above). In his typology for early career learning, Eraut [17, p. 187] provides a potentially valuable framework for monitoring and assessing work processes and actions relevant to the acquisition of data analytical skills. These include, for example, problem solving and trying things out (as work processes with learning as a by-product) and reflecting and learning from mistakes (as learning actions located within work or learning processes). This typology is sufficiently general to encompass the sorts of data skills training covered through the data fellows programme, but also with its focus on early career learning is valuable to the development of our students. As such this provides a useful high-level framework which informs the tools created to assist data fellows to baseline and monitor their analytical and research skills development. A framework that evaluates more precisely the data and statistical skills required of early career applied social researchers would be extremely valuable in helping support the development of specific skills and competencies acquired through the workplace. The UK’s ESRC (Economic and Social Research Council) has commissioned a project to explore the skills needs of social sciences to support data-driven research which will report shortly and inform future skills training for social researchers [18]. We need research on this especially as it relates to the acquisition of data skills in the workplace, The UK Government Public Sector Data Science Skills Capability report [19] is a step in the right direction although it fails to make reference to the subject expertise that social scientists bring to their research. This paper raises the question of what we can do, as a social science community, to ensure that quantitative research skills acquired in the workplace can enhance understanding of the sorts of experiences that educators need to develop for their students. We argue here that this is an under researched area and we need to improve our understanding of how it can contribute to developing the pipeline of applied social researchers that forms the focus of this paper.

3.1Three example course units

Three course units are discussed in this section taken from Politics and International Relations, Criminology and Sociology degree offerings.

3.2Summary of teaching approach to quant skills

These examples provide information to not only show the teaching approach we adopt, but also provide information about the skills and statistical techniques we teach, the different types of data sources students are introduced to, and the modes of delivery and types of assessments we employ.

There is much commonality between these three course units. For example, they all build students’ confidence up gradually, taking them from exploratory and descriptive statistics through to the building blocks for inferential statistics. They all use real world data (from sources relevant to the subjects being taught). They are all ‘topic driven’, in other words they aim to answer research questions of substantive interest to the students studying on a particular degree course (in these three examples, politics and international relations, criminology and sociology). That said there is also variation in the software taught (Excel, SPSS and R are mentioned for quantitative analysis with NVivo for qualitative analysis). Different students will therefore be equipped to different degrees with these skills and software tools by the time they apply for a data fellow position.

The modes of teaching also cover the lecture format (this is becoming a point of discussion where much of our teaching has moved online due to the COVID-19 pandemic), as well as lab classes and tutorials. We are increasingly using the term ‘whole class teaching’ to reflect that what we have previously called lectures are the sessions where we expect the whole class to be present. However, unlike the days when these classes were delivered in a lecture theatre, with data analysis training we are beginning to deliver these sessions after the practical lab classes, which are often online and asynchronous. There is a whole other paper to be written to explore post-pandemic teaching models for data analysis, but our experiences have not only forced us to pivot to online (and hybrid) teaching but consider this as an opportunity for continuing in this manner.

Data sources as we have shown reflect the subject nature of the course unit. Note that in almost all cases the data are taken from nationally representative surveys or studies. It is not stated in the course unit summaries given, but the datasets used in teaching are likely to have been pre-prepared either by the lecturer or the service from which they were downloaded. For example, the UK Data Service99 provides teaching datasets which are subsets of surveys. No third-year courses were included in the three examples, but at that stage students would be expected to move into studying inferential statistics and are more likely to be asked to create the subset of the sample to be used for their analysis. In the first two years we do not do much data cleaning – in large part due to the very restricted time we have with undergraduates to cover the statistical methods in lab classes – as outlined here.

It should be noted that the examples included in this paper are from course units taught in the first two years of the degree course. The students are introduced to many different types of data. Whilst it is the case that some nationally representative surveys (such as the British Social Attitudes Survey) and some locally collected data (e.g. from Greater Manchester Police Force) are used in teaching, and in some cases social media data are scraped from the web and analysed. We note here that the emphasis in the first two years is not on Official Statistics, but rather on developing awareness of relevant data (and skills) for the degree subject. For example, the first-year politics course included above includes a practical session where students are introduced to a wide range of data sources, including official statistics. Students in this session are given an inventory of data sources including links to websites for: the Office for National Statistics (ONS); the Scottish Government Statistics; Northern Ireland Statistics and Research Agency (NISRA); Open Data; and the Electoral Commission and others. Students, as part of this session, select a website and develop a research question reflecting the data that could be obtained and downloaded to answer it. The Criminology course included in this paper uses a sample from the Crime Survey for England and Wales (CSEW) which meets Official Statistics requirements1010 (but as presented above it also uses other data too). And a third-year demography course introduces students to different sources of official statistics including registration data and population estimates from which students produce fertility measures, for instance. Our purpose in this paper was to illustrate the types of data literacy skills social science students are taught, within the context of their substantive subject, to enable them to understand these sufficiently to apply these in the workplace (see Section 3) with a range of different data sources (including official statistics) and to scaffold their learning in their final year of their degree.

A final note on data sources is worth emphasising to acknowledge that these students are studying (and will graduate with a degree in) social science, not statistics. By embedding the statistical and data skills training in the course units we find that students are motivated to learn, seeing the techniques and tools as a necessary building block to enable them to do this. Gunn has critically evaluated this as a successful strategy for methods teaching in the social sciences [20] and other social science educators, such as Scott Jones and Goldring, have developed and evaluated their own teaching methods [21].

The purpose of this section was primarily to uncover the sort of teaching we undertake in the first two years of our degree courses, to prepare students to think about applying to undertake a data fellowship. We have not included more advanced statistical training here, though they have the opportunity to pursue this in their third year, and it is important to point out that students are able to take more theoretical statistical courses as they progress through their degree, and select quantitative methods pathways should they choose to focus on this. Our aim remains always to demonstrate the practical application of the techniques to real world complex social research.

The next section moves on to explore how the learning from these teaching units has direct relevance to the workplace. Chapters 3 and 4 in Carter [14] addresses how a student from a social science degree can prepare for and be successful in an interview to gain an applied work placement experience.

4.1Which organisations in which sectors host data fellows?

Since beginning the initiative, we have hosted almost 240 students in around fifty organisations. Note that the programme did not run in 2020 due to the COVID-19 pandemic. Table 1 categorises these hosts into six groups: polling/data companies; media organisations; Universities; Local government; National government; and not-for-profit organisations. The majority of host organisations we find projects with are public sector or not-for-profits with far fewer in the private sector. In part this reflects the networks and connections we have as a social science team. It also reflects the type of organisations that attribute value to what we teach our students, and indeed who employ quantitatively trained social science graduates. We have designed and developed the programme in response to the demand from applied social research and data analytics organisations.

The types of projects students undertake varies widely, but it is imperative that the host organization can offer a data-driven research project of up to 8 weeks duration. Projects are often defined at the start (when students are applying) and then change as the student progresses, but it is important that there is sufficient scoping and bounding of the project to enable us to advertise it to eligible students so that they can apply. An example of how the projects are advertised to students can be found on the website.1111

Figure 1.

Data fellows projects: by host organisation type and year.

Table 1 and Fig. 1 show the number of student projects between 2014 and 2019 by host organisation type (as described above). Note the numbers do not correspond exactly to the numbers of students placed as some organisations deployed more than one student on a project.

Table 1 and Fig. 1 show that the types of organisations that have engaged most with the data fellows programme over the period reported are national government departments, not-for-profit organisations and polling and data companies who have collectively provided around 70% of the projects. Universities have hosted a healthy 18% of the projects (which cover both academic research and professional services activity), with the remaining ones being in media organisations and local government. These last two categories are interesting as data journalism is still maturing (the media companies have small teams who work with data) and local government departments have suffered from disinvestment in analytical teams and their capacity to host a student is limited, at least compared to their national government peers who work in much larger analytical teams.

These summary data are valuable in providing information regarding the types of organisations that offer projects to students to enable them to undertake short term data-driven applied social research in the workplace. Whilst we do not routinely collect information from the hosts to elicit their feedback, we do know – as they return year upon year to engage with the data fellows programme – that they cherish the opportunity to host a student. The main benefits to the organisations of providing these opportunities are covered in Carter [14, pp. 39–40].

It is important to note here that the data fellows work is not formally assessed, and the experience does not contribute to a student’s final grade. Nonetheless, year after year students comment that the internship was one of the most beneficial parts of their degree experience. For further evidence to capture the benefits to data fellows we developed short films, which are available on the Q-Step website, reflecting students’ experiences.1212 These short videos prove popular for students who contemplate applying for a placement, as well as at University open days to attract students to apply to our courses. We do however insist upon ongoing self-reflection to help the data fellows process their learning. The self-reflection framework used is covered in detail in Carter (14, Chapter 6). We also require all data fellows to produce a poster when they return to university in their final year. These are all displayed the Q-Step website under the ‘Student Stories’ section (link provided above) and is the best way to showcase the vast range of projects and data analysis that the data fellows undertake. The poster provides input into a student half-day conference which serves as a celebration event which we run in the autumn after they complete their work placement experience. Each event has a prize-giving element in which we award an audience-choice and a judges’ panel prize to the best poster and the best three-minute presentation (for those students who select to enter this part of the conference). We list here the titles of the celebration events in recent years.

The next sub-section provides examples of some of the outputs that organisations benefit from through students undertaking these data fellowship opportunities.

4.2Student outputs created through the data fellows programme

Students can produce a lot in just eight weeks. The four examples included here are indicative of what the data fellows have produced over the period since the programme started. They are included to show what these social science undergraduates have the capacity to offer host organisations, in supporting them to deliver on their quantitative research activities. We find out about these outputs not because we ask students, but because their reflective pieces, which they submit three times whilst on placement, alert us to the work they are undertaking and the outputs they are creating. Through establishing a closed social-media networking group whilst the students are on placement, we manage to maintain contact with the majority of former data fellows once they graduate; this can prove beneficial in tracking the career trajectories of students.

In summing up this sub-section it is important to note that most of these outputs are incidental rather than designed in at the outset of the placement. Nevertheless, these four examples indicate the potential for the data fellows programme to build on the students’ taught skills (as covered in Section 3) by providing opportunities to engage in applied social research at the level and standard expected by the host organisation. In other words, the students start their placements as novice social researchers, but in many cases transform into early career applied social researchers. The skills and methods we teach them position them well to make this transition, and they return to their final year supremely well-equipped, and highly motivated, to undertake quantitative social research projects, having gained an insight into professional roles and careers that they could undertake.

4.3A case study of a former data fellow’s career trajectory

To bring this section together, we now report what is possible upon graduating with a social science degree that teaches quantitative research methods and provides experiential learning opportunities through a data fellows work placement. The case study that follows, we call them Sam (not their real name), is not intended to reflect a typical experience and certainly is not suggested as a representative example of the graduates of the programme. The intention rather is to illustrate that social science graduates, with the types of teaching we have developed, coupled with acquisition of real world data skills gained through the workplace, have the potential to enter careers in which these skills are sought after.