The Development of a Multi-Dimensional Coding System to Categorise Negative Online Experiences Including Cyberbullying Behaviors Among Adolescents with Lower Socioeconomic Status
Abstract
This original paper, based on data from the Erasmus+Blurred Lives Project, presents a new multi-dimensional categorisation model to describe negative online experiences, including forms of cyberbullying, based on a study of internet usage among over N = 2,500 adolescents with lower socio-economic status (SES) backgrounds across five European countries. The paper first sets out the rationale for the development of a new coding system, before describing the current study and nature of the survey data collected. There follows a description of the development of the new system and the series of reliability checks undertaken by the research team (N= 11, from 5 countries) and of the refinements made to the categories and codes. The resulting coding system is presented with consideration of the strengths and limitations, and description of two early pilot studies which have successfully adopted the new system.
Introduction
The pace of technological progress in recent years, and in particular the growing availability and popularity of computers, smart phones, tablets, and gaming consoles, has allowed individuals to connect with each other in real time, in a variety of ways, via the Internet, 24 hours a day, seven days a week (Ofcom, 2023a; Scheithauer et al., 2021). Studies have shown that children and young people, as young as three years-old, use a range of devices to access video sharing platforms, live streaming apps, messaging sites, and watch television programmes and movies (Ofcom, 2023b). While it is acknowledged that children and young people enjoy the many benefits of Internet access and social media platforms (Costabile & Spears, 2012; Purdy et al., 2023), the Internet also exposes them to a wide range of negative online experiences, including cyberbullying, grooming, sexting, and blackmail/sextortion (Notar et al., 2013). While negative online experiences such as cyberbullying or online hate speech and the effects of these constructs on the development of children and adolescents have been studied intensively in recent years, other negative online experiences, such as name-calling, threats, unwanted contact etc. –and in particular their joint appearance –have been studied less intensively (Skogen et al., 2023). The analysis of the full range of negative online experiences is important to discern trends and identify appropriate interventions to protect, support, and educate children and young people. For example, negative online experiences are positively associated with symptoms of depression and anxiety, and are negatively associated with mental well-being in general (Skogen et al., 2023; Smahel et al., 2020). Such analysis enables a clearer understanding of the range of behaviours experienced, and allows the phenomenon to be investigated more validly and reliably. Moreover, such a clear understanding of the cyberbullying and other negative online experiences allows investigation into, for instance, gender-specific differences, poly-victimisation and differences in incidence by age, ability, or social/ethnic background.
Studies have shown that people with low socioeconomic status (SES) compared with people with high SES spend more time on screen-based activities, and that children from families with lower SES have more often access to media devices in the bedroom (Skogen et al., 2022). According to the Routine Activity Theory (e.g., Festl & Quandt, 2013) more intensive use of the Internet is related to greater victimisation. Accordingly, Skogen et al. (2022) found that low SES was associated with more negative experiences on social media in adolescence, even after adjusting for age, gender, country of birth, type of study, and amount of social media use. However, according to Skogen et al. (2022) in a summary of the literature, there are also study results that do not demonstrate that children and adolescents from families with low SES report more negative online experiences. A further challenge is identified by Smahel et al. (2020) who note that SES is “difficult to measure in a way that is consistent across European countries” (p.131) and explain that this has hindered cross-cultural analysis, despite the undoubted importance of this form of difference. In summary, therefore, it can be said that there remains a lack of research into negative online experiences that explicitly focuses on adolescents with low SES and, in particular, that attempts to draw international comparisons.
Categorisation Approaches Utilised for Negative Online Behaviours
Existing categorisation approaches utilised for negative online behaviours have tended to focus primarily on the behaviour itself and can therefore be described as mono-dimensional. The best known such taxonomy was created by Willard (2007), who lists different forms of cyberbullying and cyberthreats as flaming, harassment, denigration, impersonation, outing and trickery, exclusion, cyberstalking, and cyberthreats. Other risks are also listed including sexually related risks, unsafe online communities, dangerous online groups, online gaming, unsafe personal disclosure, and addictive access. How such experiences occur, and their frequency, the scale of the problem, where experiences occur (e.g., the platform), who did it, relationships, role of bystanders, why it happened, and impact and reporting are also briefly discussed. However, Willard (2007) acknowledges that, at the time of writing 17 years ago, academic research about cyberbullying and cyberthreats was in its infancy and had not been carried out “with much depth or quality” (p. 27). Consequently, Willard notes that the primary source of her categorisation was “an informal qualitative analysis of news reports and privately reported incidents, visits to online communities, and consultations with other professionals who focus on Internet use concerns” (p. 27). While Willard claims that she did also consult some academic research on traditional bullying and the limited research available on cyberbullying and that she also spoke to school counsellors and psychologists, she acknowledges that “this approach does not meet the standards of academic research” and that as a result her findings should be considered “tentative, subject to further study and clarification” (p. 27).
Kowalski et al. (2008, 2012) later extended Willard’s 2007 taxonomy by adding “happy slapping” (physical attacks which are filmed and then shared) and “sexting” (the sending of nude or partly nude images), however more than a decade later, many of the (then) most popular social media platforms, such as Bebo and Formspring, discussed by Kowalski et al. (2012), no longer exist, highlighting how quickly the (social) media landscape and associated behaviours evolve.
Nocentini et al. (2010) and Menesini et al. (2011) focused on the incident itself, with consideration for messages, violent images, intimate images, unpleasant images, impersonation, silent phone calls, and exclusion types. Gahagan et al. (2016) identified nine types of social media-based cyberbullying: negative words, messages, photos, comments, posts, posting without permission, continual, public, and private. In addition, more platform-specific types of negative online behaviour such as hacking of accounts, creation of false profiles, and pages created to harass a victim, and the visibility of such interactions to the victim’s social network circle (Brody & Vangelisti, 2017).
More recent studies (within the past five years) which have sought to measure and assess negative online experiences have so far also proven to be heterogeneous, with the research sample often differing considerably, and with wide variety in the number and type of experiences described and reported. One such example is provided by Gainsbury et al. (2019) who developed an original author-generated scale to assess common “negative online consequences”. A total of 1,773 Australian adults participated in the survey and indicated (‘yes’/’no’) whether they had experienced any of a range of listed negative online experiences in the previous year. The list, based on a review of the literature as well as negative online experiences identified from previous studies, included a total of 25 items such as “personal abuse (e.g., target of negative comments, abuse, cyberbullying, trolling, accessing disturbing content), theft of personal information (e.g., personal or financial accounts hacked, personal or financial details stolen), fraud (e.g., shopping-related, application for credit in your name), spam, malware, phishing, and scams (e.g., ransomware, romance, advance fee fraud, refund fraud)” (p. 1237).
For instance, cyberbullying on WhatsApp was investigated among 4,477 elementary, middle and high school students in Israel (Aizenkot & Kashy-Rosenbaum, 2021). A 10-item questionnaire was used in which four different types of cyberbullying were identified: verbal violence (offensive responses, insults, mocking, curses, threats), visual violence (posting offensive photos without the individual’s consent), group violence and selectivity (forced removal from a group, denied entry to a group, group rejection), and participation avoidance due to fear of offensive responses. Participants were invited to report how often the experience had occurred, but notably, the authors only counted more than one exposure to the offensive behaviour as constituting cyberbullying, drawing on the definition of Tokunaga (2010) which emphasises the repeated nature of the behaviour as follows: “Cyberbullying is any behavior performed through electronic or digital media by individuals or groups that repeatedly communicates hostile or aggressive messages intended to inflict harm or discomfort on others.” (p.278).
In another example, in their Norwegian study which recruited 3,253 high school students (aged 16+) in the municipality of Bergen, Skogen et al. (2023) presented participants with eight statements about negative experiences on social media (with responses ranging from ‘Never’ to ‘Very often’) and combined these into two composite measures: ‘Unwanted attention from others’ (3 statements: “I get unwanted attention from strangers”; “I receive unwanted nude pictures/ sexualised content”; “I am asked to send nude pictures/ sexualised content”) and ‘Negative acts and exclusion’ (5 statements: “Others share pictures/videos of me against my will”; “I get negative comments on what I post”; “I receive unpleasant or hurtful messages”; “Others say/post negative things about me”; “I feel excluded from groups/chats”).
In a further example, Kimball (2023) reports on a study of the online experiences of 1,059 children and adolescents (aged 5–21) in the New York City area (mean age = 12 years). The quantitative and qualitative responses were then used to develop a “hierarchy of offenses” (p.12) where “first-tier acts of malice” included demands for nude photos and exposure of personal information, “second-tier acts of malice” included faking identities and the absence of consent, and “slights” included unsolicited attention, exclusion from groups and making fun of someone. This hierarchy serves as a guide to the “outrage threshold”, the point at which victims would choose to report to the platform or adults in authority. Almost all respondents said that they would report “first-tier acts of malice” while there was much greater variability in reporting experiences of “second-tier acts of malice” or “slights” where the intent to harm may have been unclear and the degree of malice arguable.
In their study of positive and negative online experiences and loneliness among Peruvian adolescents during the Covid-19 lockdown, Magis-Weinberg et al. (2021, cf. Uchino et al., 2023) used the Online Social Experiences Measure (OSEM) scale (Kent de Grey et al., 2019). Divided into two subscales, OSEM measured the independent effects of positive and negative online social experiences using eleven items, five of which measured positive experiences (e.g., “I feel valued by people in my social media,” “People in my social media give me advice”) with the remaining six measuring negative experiences (e.g., “People in my social media make me feel like I do not belong,” “People in my social media treat me badly”). The study also assessed the frequency of the online experiences on a 5-point scale (1 = never to 5 = very frequent).
Thus, the prevalence rate of negative online experiences varies considerably across studies due to (among others) definitional, methodological, and sampling differences. While most taxonomies have focused on the nature of the behaviour, some have sought to classify the incidents by the communication medium. An early, evidence-informed attempt to categorise the emerging phenomenon of cyberbullying by medium was made by Smith et al. (2008) who admits that at the time “research on cyberbullying is at an early stage” (p. 376). Smith et al. (2008) focus on a particular range of cyberbullying incidents which were (necessarily) limited by the technology available at that time. There is therefore a focus on cyberbullying via a total of seven “media”: through text messages, pictures/photos or video clips, phone calls, emails, chat room comments, instant messaging, and websites. Smith et al. (2008) also asked their survey respondents about who had cyberbullied them (their gender and class), whether they had reported the cyberbullying, and to describe its impact. There was an interest too in whether the cyberbullying had taken place on computers or on mobile phones. There was no mention of tablets, social media, or online gaming at that time. Other more recent studies have investigated cyberbullying on social media (Brody & Vangelisti, 2017; Whittaker & Kowalski, 2015), video sharing platforms such as YouTube (Kyriacou & Zuin, 2016), and via online gaming (Common Sense, 2019 McInroy & Mishna, 2017).
Other studies have chosen to focus on the motive. Scheithauer et al. (2021) summarize the state-of-the-art and conclude that motives for cyberbullying can vary substantially. Studies conducted so far investigated, e.g., internal issues, such as revenge, boredom, jealousy, and more external issues such as appearance or degrading of minorities as motives for cyberbullying.
A further shortcoming is that existing approaches to investigating negative online experiences, such as online bullying, are often highly conceptual, including being informed by offline experiences, or practical, and based on motives, different behaviours, social roles, victim characteristics, or platforms, with very few being empirically validated (Scheithauer et al., 2021; Smith et al., 2008). As Scheithauer et al. (2021) conveyed, multi-country research on negative online experiences (including online bullying) has increased over recent decades, with quantitative studies on the rise, but qualitative studies on the decline.
There has been much debate on definitions and categorisation (e.g., what is and isn’t cyberbullying), assumptions made including on power-relations, and criteria used for analysis. A need for more qualitative and mixed-methods (as opposed to quantitative alone), multi-dimensional research approaches was also discussed, that capture data that is more reflective of the inherent complexities involved in understanding such phenomena, including with respect to poly-victimisation. Such approaches would allow trained researchers to code comprehensively negative online experiences that would be reported by the participating adolescents who have been exposed to the negative online experiences. Such understanding requires taking a more holistic approach with consideration for new perspectives e.g., a child-centred approach, and the social, physical, institutional, and community dimensions, including interactions between cyberbullies, victims, peers, teachers, and administrators, as opposed to focusing on any particular group (Li, 2010). As Horton (2016) noted, this would amount to investigating negative online experiences using a wide-angle lens. This is especially true, since children are dynamic, social beings who interact with others to construct joint meanings within a given context (Greig et al., 2007), and context can vary greatly.
The Present Study
The development of a new coding system for adolescents’ negative online experiences derives from data from the Blurred Lives Project - a cross-national, co-participatory exploration of cyberbullying, young people, and socio-economic disadvantage (see project website https://www.ou.nl/en/web/blurred-lives). This was a two-year project (2017–2019) funded by Erasmus + under Key Action 2: Strategic Partnerships for School Education and involved five European partners from Northern Ireland, England, Germany, Italy, and the Netherlands. The Blurred Lives Project uniquely explored the online experiences of (mostly) 14-to-16-year olds in schools in disadvantaged urban areas in each of the partner countries as well as setting out to understand more about adolescent internet use and online experiences. The project also aimed to facilitate and promote pupil voice through the co-creation of resources and/or recommendations for teachers, pupils, parents/carers, and social networking providers (for further details, see Hamilton et al., 2020; Purdy et al., 2021; Mameli et al., 2022; Willems et al., 2023).
The four central objectives of the two-year Blurred Lives Project were as follows:
• To empower young people aged 14–16 to better understand, prevent, and combat cyberbullying.
• To develop the most useful support materials for teachers, parents/carers and young people (aged 14–16) for signalling, preventing and combating cyberbullying, and to make recommendations to social networking providers.
• To determine how the young people understand, experience and respond to the present conditions of cyberbullying.
• To determine the relationship between socio-economic disadvantage and young people’s understanding and experiences of cyberbullying and their responses/coping strategies.
The aim of the present focused study was to determine the nature and frequency of negative online experiences in adolescents from lower socio-economic backgrounds (SES) from five European countries. Adolescents with low SES in general show poorer cognitive development, lower academic achievement, and language, memory, or socioemotional processing deficits (American Psychological Association, 2017), resulting in reduced capacity to complete questionnaires. Survey procedures and methods must therefore be adapted to the specific situation of the population under investigation (cf., Emery et al., 2023) which is why we considered –in the present study - providing support during the survey in class, and conducting surveys with a range of closed and open-ended questions wherever possible, rather than using exclusively quantitative methods surveys.
However, at the time of planning and implementing the Blurred Lives Project, there were no standardized, quantitative or qualitative methods available (as outlined above) that could be used to comprehensively record negative online experiences - and no methods that were specifically tailored to the needs of adolescents from lower socio-economic backgrounds. Thus, the second aim was to develop a coding system, which can be used to categorize responses in qualitative surveys. In doing so, we wanted to take into account the special characteristics and prerequisites of samples of adolescents from lower socio-economic backgrounds.
Additionally, as Scheithauer et al. (2021) note, of the many different approaches to categorising cyberbullying which have been developed over the past two decades, many have emerged from practical or theoretical considerations and have not been replicated empirically. Moreover, there are many different possible dimensions to the phenomenon of cyberbullying, which confirms the importance of developing an appropriately multidimensional categorisation model. Such an up-to-date, empirically tested, multidimensional categorisation model of cyberbullying has not been available until now.
Methods
Sampling and Recruitment Process
The research was conducted in accordance with the Declaration of Helsinki, and approved by the university’s ethics committee in each respective country as follows: Approval of the ethics committee at the Open Universiteit in the Netherlands was received on June 6, 2018, under reference number: U2018/03921/MQF; approval of the Ethic committee of the University of Bologna in Italy was received on 5 May 2018, under reference number: n.68251; approval of the Ethics Committee of the Department of Education & Psychology at Freie Universität Berlin in Germany was received on 16 October 2018, under reference number: 191/2018; approval of the Research Ethics and Integrity Sub-Committee of the Department of Psychology, Goldsmiths, University of London was received on 29 June 2018, under reference number: 1393; approval of the Research and Ethics Committee, Stranmillis University College, Belfast, Northern Ireland was received on 12 March 2018, under reference number: 2018-03PURDY.
The need for ethical approval in five institutions led to some minor but unavoidable differences in question design between countries e.g., in Germany participants were not asked to provide details of their ethnic background.
At the outset of the project, each partner country agreed to aim for 500 survey responses from (mostly) 14-to-16-year olds in urban schools with above-average levels of socioeconomic disadvantage. How this was measured differed between partner countries. For instance, in England and Northern Ireland, the survey was distributed to schools where 40% or more of the pupils were entitled to free school meals; in Germany, schools were chosen according to the Berlin social structure atlas; and in Italy and the Netherlands, urban vocational schools were selected which traditionally have higher levels of pupils from socioeconomically disadvantaged backgrounds.
The number of schools and pupils varied by country, depending on the size of the respective schools and the response rates. In total, 2,655 pupils completed the online survey, with 89.7% aged 14 to 16 years.
Online Survey Design
In the absence of any pre-existing categorisation models or question schema to draw on to explore the negative online experiences of adolescents with lower SES, and to provide optimal opportunities for flexibility (not restricting responses to pre-ordained categories) and accessibility (allowing freedom of expression), an original and bespoke questionnaire was designed by the project team (cf. Emery et al., 2023).
The online survey asked a series of questions on demographics (e.g., age, gender, ethnic origin [except for German sample], special educational needs, social background etc.), and questions which explored participants’ Internet usage. In three sections, adolescents were asked to think back over the previous couple of months and then to consider:
• First, whether any nasty or unpleasant experiences had happened to them and if so, to describe their own worst negative online experience (i.e., their experience as a victim);
• Second, whether they had seen or heard of any nasty or unpleasant experiences happening to someone they know well and if so, to describe what might have been the worst one (i.e., their experience as an online witness or bystander);
• Third, whether they themselves had done any nasty or unpleasant things to someone on the internet, and if so, to describe what might have been the worst one (i.e., their experience as a perpetrator).
These three open-ended questions provided a large sample of qualitative data which presented an immediate challenge in terms of analysis, given (as outlined above) the lack of recent coding systems to analyse adolescents’ negative online experiences, and in particular the lack of coding systems which extended beyond the details of what happened (see for example, Smith et al., 2008; Willard, 2007). Consequently, this formed the basis for the development of a new coding system for negative online experiences (focusing on cyberbullying in this case), with respect to complex multi-dimensional data.
Coding Team
Each of the five partner counties enlisted at least two team members to work independently to code the open-ended qualitative data (resulting in N = 11 coders). The coding teams were as follows: Northern Ireland (2 males), England (1 male, 1 female), Germany (1 male, 1 female), Netherlands (1 female, 1 male), Italy (3 females). In each case this comprised the experienced country lead researcher and another (often less experienced, e.g., pre- or post-doctoral researcher) colleague. Given that the responses were in different languages (English, Italian, German, and Dutch), it was felt to be too difficult to code data from another country’s data sample. Cross-country ratings were therefore not realized.
Results
Initial Coding Process
An initial reading of the data among the coders resulted in a list of 4 macro-categories, exploring the nature of the negative online experience (WHAT happened?), the online platform where the negative experience happened (WHERE did it happen?), the motivation or pretext for the negative online behaviour (WHY did it happen?) and, finally, the person(s) responsible for the behaviour (WHO did it?). Under each of the four macro-categories, researchers from all five countries provided examples from their respective samples to populate the subcategories in an iterative manner (see Fig. 1).
Figure 1
Summary of Categorisation Framework with Four Macro-Categories, Each with Respective Sub-Categories.
Inter-Rater Reliability Procedure
To further refine the categorisation model, checks for inter-rater reliability were run before applying the categorisation model to the full dataset. This method is widely used in qualitative social science particularly for cross-national datasets (Armstrong et al., 1997; Iwarsson et al., 2005), to identify problems of definition, interpretation, or translation that might lead to significant variations in the use of the individual codes contained within the categorisation model.
The analysis involved inter-rater reliability of the dichotomous assessment of the 54 items included in the categorisation model. Inter-rater reliability was calculated in terms of agreement in pairs of raters, with research teams from each of the five countries independently rating a random sample (N = 30) of cases. These were taken solely from the England and Northern Ireland cohorts, as English was the common working language of the research team.
Agreement between pairs of raters was calculated using Cohen’s kappa statistic. Kappa corrects for chance agreement but can only be used to assess the agreement between two raters. The kappa values were interpreted following Altman’s (1990) guidelines: values < 0.20 = poor, 0.21-0.40 = fair, 0.41-0.60 = moderate, 0.61-0.80 = good, and 0.81-1.00 = very good agreement. As Cohen’s kappa could only assess the reliability of agreement between pairs of raters comparing national datasets, we then also used Fleiss’s kappa, which is recommended to assess the reliability of agreement between multiple raters. Our first assessment returned an overall kappa value of 0.35, as well as individual kappa values for each of the items contained in the categorisation model and for each of the national teams. This assessment allowed the research team to conduct a re-appraisal of the model, adapting ambiguous language within individual codes, re-ordering certain codes, and removing others that appeared to be producing unreliability. For example, we chose to include an expanded definition of ‘grooming’ to help raters make a more accurate judgement for item 1.18 (see Fig. 2 below). It should be noted that some codes that were not used as they did not come up in the sample were retained in the categorisation model. Similarly, some codes which are prone to low reliability scores such as those that were only used once or general ‘other’ codes (e.g., 1.29, 2.5) were also retained (see Fig. 2 below).
Figure 2
Full Categorisation Framework with Four Macro-Categories, Each with Respective Sub-Categories..
Following this secondary development stage, and satisfied that the categorisation model was sufficiently robust to be applied to the whole of the cross-national dataset, the research proceeded with a full coding procedure, but included a random sample within four of the five national sub-sets which repeated the dichotomous assessment exercise (Italy n = 111, Netherlands n = 48, England n = 36, Northern Ireland n = 30) with a new random sample of cases. Cohen’s kappa was then calculated for each sample and averaged to return a value of 0.51, indicating a significant improvement from “fair” agreement to “moderate” agreement. Part of this effect can be accounted for by the process of development itself, which could be understood as a kind of rater training. Analysis of individual items confirmed that those that had been adapted from the previous assessment exercise had vastly improved kappa values, but that general ‘other’ codes and single-use codes continued to bring down the average kappa value.
Resulting Coding System
The final coding system, which can be downloaded from the project website1 , comprises a total of four macro-categories, each further divided into subcategories as follows:
• Macro-category 1 considered the type of online experience (WHAT happened?) and included a list of 30 diverse subcategories or codes (see Fig. 2) including “Offensive or malicious comments were made about me or sent directly to me”, “I was asked to send nude/sexual images of myself or to expose myself online”, “Someone tried to groom me”, and “Someone tried to blackmail me (e.g., for money).”
• Macro-category 2 explored the type of online platform (WHERE did it happen?) and comprised six options (see Fig. 2) including “On a public social media site, e.g., Instagram, Facebook, Snapchat”, “On a private chat/group”, “Via direct message/chat to one other person (one to one only)”, and “On a gaming site / games console”.
• Macro-category 3 coded data which referred to the motivation or pretext (WHY did it happen?). It featured ten subcategories (see Fig. 2) including “Related to my physical appearance (including smell)”, “Related to my race, ethnicity”, and “Related to my physical or learning disability.”
• Macro-category 4 examined who the person(s) responsible for the negative online experiences (WHO did it?). This was further divided into a total of eight subcategories (see Fig. 2) including “Definite stranger e.g. A guy popped up and ... ”, “Member of peer group (friends/ former friends/ fellow pupils/ team mates), “Ex-boyfriend/ girlfriend”, and “Family member”.
Handling Data Using the Blurred Lives Coding System
Given the magnitude of the sample and complexity of the coding system, the following discussion will focus solely on the questions related to online victimisation. Adolescents were first asked whether anything nasty or unpleasant had happened to them on the Internet over the previous couple of months. A total of 24.5% of the respondents said ‘yes’ (N = 639). There were significant differences by country (ranging from 36.9% of German adolescents to 11.3% of Dutch adolescents), and also by gender (30.2% of girls compared to 18.6% of boys). When analysed by country and gender, the differences were greater still, ranging from victimisation rates of 46.2% among German female adolescents to just 4% of Dutch male adolescents.
The subsequent survey question asked respondents to explain in their own words what (if any) was their worst online experience over the previous couple of months. Using the new coding system, the data was coded against each of the four macro-categories as described above. In each case the raw number of responses fell below the requirement for statistical significance for further comparison of subgroups (e.g., gender/country).
In terms of macro-category 1, types of negative online experience (WHAT happened?), the results highlight that, in respect to the sub-sample (n = 639) of those who said that they had a negative experience, the most commonly experienced subcategory was “Offensive or malicious comments were made about me or sent directly to me” (22.7%, N = 145), followed by “I was sent inappropriate content whilst online (e.g., pictures/films with sexual/violent content)” (5.3%, N = 34), “Definite but unspecified nasty experiences (e.g., I was cyberbullied)” (4.7%, N = 30), and “I was asked to send nude/ sexual images of myself or to expose myself online” (3.0%, N = 19).
Under macro-category 2, types of platform (WHERE did it happen?), the results highlight that, where mentioned at all, the most common online platform for online negative experiences to take place was “On a public social media site, e.g., Instagram, Facebook, Snapchat” (13.9%, N = 89), followed by “On a gaming site / games console” (5.8%, N = 37), “Other” (4.1%, N = 26), “On a private chat/group” (3.8%, N = 24), and “Via direct message/chat to one other person (one to one only)” (2.5%, N = 16).
The data were next coded using macro-category 3 which examined the motivation or pretext (WHY did it happen?). The number of responses which provided information relating to motivation was low, but based on the available data, the most common motive was identified to be “Related to my physical appearance (including smell)” (3.4%, N = 22), followed by “Related to my race, ethnicity” (1.6%, N = 10), and “Related to my sexuality (LGBT), e.g., homophobic, transphobic” (0.8%, N = 5).
Finally, in respect of macro-category 4, the person(s) responsible (WHO did it?) the results highlighted that the most common response was “Unspecified, e.g., Someone ... / People ... ” (20.0%, N = 128), followed by “Member of peer group (friends/ former friends/ fellow pupils/ team mates)” (10.3%, N = 66), and “Definite stranger e.g. A guy popped up and ... ” (6.9%, N = 44).
Discussion
In the present study, we developed a coding system to determine the nature of negative online experiences among adolescents and to describe the experiences more precisely, which can be used to categorize responses in qualitative surveys. In doing so, we took into account the special characteristics and prerequisites of samples of adolescents from lower socio-economic backgrounds and investigated adolescents from five European countries: England, Germany, Italy, Netherlands, and Northern Ireland.
After a modification and comprehensive reliability analysis, categories remained which, from our point of view, allow a satisfactory allocation of the answers received in a qualitative survey to the individual categories. Overall, the eleven coders were thus able to summarize the comprehensive, qualitative data in a manageable time in such a way that further, also quantitative analyses are possible. The development and reliability checking of this new multi-dimensional coding system for negative online behaviours among adolescents from lower socio-economic backgrounds is both unique and timely. It also offers researchers a useful, convenient single tool to explore a broader range of dimensions than is currently available in any other existing taxonomy, and to analyse not just the behaviour but also where it happened (e.g., on a public social media site or on a gaming site), what the motive or pretext was (e.g., related to gender, race/ethnicity) and who was responsible for the behaviour (e.g., member of peer group, definite stranger, ex-boyfriend etc.).
The results of this study show that almost a quarter (24.5%, N = 639) of the adolescents in this SES sample (mostly aged 14-16 years) reported that something nasty or unpleasant had happened to them online over the previous couple of months, and that there was a much higher victimisation rate among girls (30.2%) than boys (18.6%). Occurrence rate was highest in the German sample and lowest in the Dutch sample, although this may be explained by differences in identification of SES samples between countries.
Comparison with other studies is problematic, given the definitional, methodological and sampling differences outlined above. For instance, when compared with two recent studies, the occurence rate appears lower than the 50% reported in a recent Australian study (eSafety Research, 2021) of non-SES 14-17-year olds (though with a timeframe of the previous six months rather than two months), but higher than the occurence rate of 18.5% among a non-SES sample of 14-18-year olds in Northern Ireland (Purdy et al., 2023).
Limitations
The still preliminary nature of our results, which we have obtained with the coding system and presented here, must certainly be viewed critically. For example, it is to be expected that many of the behaviours and other related information in the other three categories will be mentioned more frequently and more clearly when asked about them in the qualitative survey. In our study, we developed the coding system and assigned answers AFTER the qualitative survey which asked just one question of the respondents (to describe what happened in their worst online experience over the past two months). In future studies, the four category questions should therefore be asked more clearly from the outset in the qualitative surveys. Future studies can then also examine the differences between the countries included here in the frequency etc. of negative online experiences in more detail and check whether these are real differences or sample differences and differences in response behaviour, which would then have to be taken into account in surveys.
While the multi-dimensional nature of the negative online behaviour was of interest from the data that emerged from the Blurred Lives Project, the retrospective analysis highlighted the shortcomings of the single open-ended question that was asked in the survey. In this instance, those adolescent participants who had reported that “nasty or unpleasant things had happened to you on the Internet” were asked to provide more detail based on the following instruction: “If Yes, please think of what might have been the worst one, and write below what happened”. Consequently, many of the responses quite naturally focused (as requested) on what had happened, with relatively few providing information on the other three key dimensions (where it had happened, why it had happened, and who did it?).
However, the significance of this original multi-dimensional coding system is that it provides a framework for future research studies to ask respondents more specifically from the outset to provide details of their negative online experiences under each of the four key macro-categories, using the subcategories developed above. If we had had the current multi-dimensional taxonomy from the outset of the Blurred Lives Project, there is no doubt that we would have obtained much more comprehensive qualitative data which would have shed additional light on the multi-dimensional nature of the behaviours reported. As it was, many of the open-ended responses simply reported what happened and provided no additional information at all.
Another challenge encountered through the development of the current multi-dimensional coding system was the relatively low inter-rater reliability scores obtained, even after several rounds of coding in pairs. While there was evidence of an improvement in the inter-rater reliability scores through the various rounds of coding as the codes were refined, the final ‘moderate’ reliability scores reflect perhaps the sheer breadth and complexity of the negative online experiences, as well as some uncertainty when coding responses which, to some coders, appeared as silly or joke answers, or were unclear and/or incomplete. In short, the reliability checks undoubtedly led to a more robust coding system, but coding adolescents’ open-ended responses remains a challenging and highly subjective undertaking.
The final and obvious limitation is that this new coding system, like those that have preceded it, will have a limited lifespan as the range of online devices, platforms, apps and experiences means that the potential for negative online experiences (and victimisation) is evolving and expanding all the time. Further and continuous refinement of the model, based on future studies, will be necessary to future-proof the taxonomy and the possible sub-categories listed.
Outlook
Since it was developed, the multi-dimensional coding system has been trialled in two studies, each of which will be outlined briefly below.
In Northern Ireland, the coding system was integrated into the online survey that was completed by 6,481 children and young people aged 8-18 as part of the ‘Growing Up Online in Northern Ireland’ study, funded by the Safeguarding Board for Northern Ireland and conducted by a research team from Stranmillis University College, Belfast (Purdy et al., 2023). One version of the survey was administered to 8-to-13-year olds (with amendments made to the list of behaviours in macro-category 1 to ensure age appropriateness) and another fuller version was given to 14-to-18-year olds. The survey links were distributed to all schools in Northern Ireland via the Education Authority and remained open for a period of four weeks, from 6th February to 6th March 2023. The coding system was also subject to adaptation by the funder who requested that some additional negative online behaviours were added, e.g., “I saw or was sent content promoting self-harm”, “I saw or was sent content promoting suicide”, “I saw or was sent content promoting eating disorders”. In terms of the other macro-categories, there was however very close adherence to the new multi-dimensional coding system outlined above. The outcome of the trial was very successful in that, in contrast to the Blurred Lives Project where the open-ended responses had focused mainly on what had happened (and little else), the series of four questions covering each of the key aspects (What happened? Where did it happen? Why did it happen? Who did it?) elicited a high response rate across each dimension, thus enhancing the validity and reliability of the resulting data.
The study “Negative Online Experiences at Universities” (“Negative Online-Erfahrungen an Hochschulen”, NOAH) at Freie Universität Berlin is investigating what negative online experiences students have in the university context and whether and how often this includes cyberbullying. The aim of this study is to record students’ experiences, understanding and perception of cyberbullying and other negative online experiences and to use this information to develop recommendations for dealing with these phenomena at universities. In January 2023, psychology students at Freie Universität Berlin conducted a pilot project, led by Prof. Dr. Herbert Scheithauer, as part of a seminar with a research focus at Freie Universität Berlin, in which all students in their third semester in the Department of Education and Psychology who are at least 18 years old were able to participate. The items of the four macro-categories were transferred into a quantitative online questionnaire for this purpose. For each online experience ticked by the participants, the frequency of occurrence and a classification according to the item content of macro categories 2-4 was also investigated.
Conclusions
In this paper, we have presented a unique and timely new multi-axial, multi-dimensional categorisation model for a wide range of negative online experiences (including cyberbullying) for adolescents from lower SES backgrounds. We have also presented and discussed our work to empirically test and replicate the coding system which was developed on the basis of a series of refinements based on inter-rater reliability tests across five countries (using a pair of coders per country).
Moreover, we have reported on two recent examples where the categorisation model has been used in pilot studies and where early feedback has been extremely positive. While any coding system relating to online experiences will need to be agile and adaptable (especially in terms of the subcategories offered), the principle of a four-dimensional categorisation model represents a significant advance on previous, largely outdated uni-dimensional models.
We would suggest in conclusion that the new approach offers much to researchers, policy makers and practitioners seeking to explore the wide and complex range of negative online behaviours experienced by adolescents from lower SES backgrounds.
References
1 | Aizenkot, D. , Kashy-Rosenbaum, G. ((2021) ). Cyberbullying victimization in WhatsApp classmate groups among Israeli elementary, middle, and high school students. Journal of Interpersonal Violence, 36: (15-16). https://doi.org/10.1177/0886260519842860. |
2 | Altman, D. G. (1990). Practical statistics for medical research. Chapman & Hall/CRC Press. |
3 | American Psychological Association (2017). Education and socioeconomic status. APA.https://www.apa.org/pi/ses/resources/publications/education. |
4 | Armstrong, D. , Gosling, A. , Weinman, J. , Marteau, T. ((1997) ). The place of inter-rater reliability in qualitative research: An empirical study. Sociology, 31: (3), 597–606. https://doi.org/10.1177/0038038597031003015. |
5 | Brody, N. , Vangelisti, A. L. ((2017) ). Cyberbullying: Topics, strategies, and sex differences. Computers in Human Behavior, 75: , 739–748. https://doi.org/10.1016/j.chb.2017.06.020. |
6 | Common Sense. (2019). The common sense census: Media use by tweens and teens, 2019. https://www.commonsensemedia.org/research/the-common-sense-census-media-use-by-tweens-and-teens-2019. |
7 | Costabile, A. , Spears, B. (2012). The impact of technology on relationships in educational settings. Routledge/Taylor and Francis Group. |
8 | Emery, L. F. , Silverman, D. M. , Carey, R. M. ((2023) ). Conducting research with people in lower-socioeconomic-status contexts. Advances in Methods and Practices in Psychological Science, 6: (4), https://doi.org/10.1177/25152459231193044. |
9 | eSafety Research (Ed.). (2021). The digital lives of Aussie teens - February 2021. Online:https://www.esafety.gov.au/sites/default/files/2021-02/The%20digital%20lives%20of%20Aussie%20teens.pdf. |
10 | Festl, R. , Quandt, T. ((2013) ). Social relations and cyberbullying: The influence of individual and structural attributes on victimization and perpetration via the Internet. Human Communication Research, 39: (1), 101–126. https://doi.org/10.1111/j.1468-2958.2012.01442.x. |
11 | Gahagan, K. , Vaterlaus, J. M. , Frost, L. R. ((2016) ). College student cyberbullying on social networking sites: Conceptualization, prevalence, and perceived bystander responsibility. Computers in Human Behavior, 55: , 1097–1105. https://doi.org/10.1016/j.chb.2015.11.019. |
12 | Gainsbury, S. M. , Browne, M. , Rockloff, M. ((2019) ). Identifying risky Internet use: Associating negative online experience with specific online behaviours. New Media & Society, 21: (6), 1232–1252. https://doi.org/10.1177/1461444818815442. |
13 | Greig, A. , Taylor, J. , MacKay, T. (2007) Doing research with children (2nd ed.). Sage.https://methods.sagepub.com/book/doing-research-with-children. |
14 | Hamilton, J. , Purdy, N. , Willems, R. , Smith, P. K. , Culbert, C. , Brighi, A. , Fiedler, N. , Guarini, A. , Mameli, C. , Menin, D. , Scheithauer, H. , Völlink, T. ((2020) ). Using the Quality Circle approach to empower disadvantaged youth in addressing cyberbullying among adolescents: An exploration across five European countries.(Special Issue). Pastoral Care in Education, 38: (3), 254–272. https://doi.org/10.1080/02643944.2020.1788127. |
15 | Horton, P. ((2016) ). Portraying monsters: Framing school bullying through a macro lens. Discourse: Studies in the Cultural Politics of Education, 37: (2), 204–214. https://doi.org/10.1080/01596306.2014.951833. |
16 | Iwarsson, S. , Nygren, C. , Slaug, B. ((2005) ). Cross-national and multi-professional inter-rater reliability of the Housing Enabler. Scandinavian Journal of Occupational Therapy, 12: (1), 29–39. https://doi.org/10.1080/11038120510027144. |
17 | Kent de Grey, R. G. , Uchino, B. N. , Baucom, B. R. , Smith, T. W. , Holton, A. E. , Diener, E. F. ((2019) ). Enemies and friends in high-tech places: the development and validation of the Online Social Experiences Measure. Digital Health, 5: . https://doi.org/10.1177/2055207619878351. |
18 | Kimball, H. (2023). Helping teens make their online spaces safer: Perspectives from the Online Use and Reporting Survey (OURS). Child Mind Institute.https://childmind.org/wp-content/uploads/2023/10/CMI_Online-Reporting_Report_5197.pdf. |
19 | Kowalski, R. M. , Limber, S. P. , Agatston, P. W. (2008). Cyberbullying: Bullying in the digital age. Wiley-Blackwell. |
20 | Kowalski, R. M. , Limber, S. P. , Agatston, P. W. (2012). Cyberbullying: Bullying in the digital age (2nd ed.). Wiley-Blackwell. |
21 | Kyriacou, C. , Zuin, A. ((2016) ). Cyberbullying and moral disengagement: An analysis based on a social pedagogy of pastoral care in schools. Pastoral Care in Education, 34: (1), 34–42. https://doi.org/10.1080/02643944.2015.1134631. |
22 | Li, Q. ((2010) ). Cyberbullying in high schools: A study of students’ behaviors and beliefs about this new phenomenon. Journal of Aggression, Maltreatment & Trauma, 19: (4), 372–392. https://doi.org/10.1080/10926771003788979. |
23 | Magis-Weinberg, L. , Gys, C. L. , Berger, E. L. , Domoff, S. E. , Dahl, R. E. ((2021) ). Positive and negative online experiences and loneliness in Peruvian adolescents during the COVID-19 lockdown. Journal of Research on Adolescence, 31: (3), 717–733. https://doi.org/10.1111/jora.12666. |
24 | Mameli, C. , Menabò, L. , Brighi, A. , Menin, D. , Culbert, C. , Hamilton, J. , Scheithauer, H. , Smith, P. K. , Völlink, T. , Willems, R. A. , Purdy, N. , Guarini, A. ((2022) ). Stay safe and strong: Characteristics, roles and emotions of student-produced comics related to cyberbullying. International Journal of Environmental Research and Public Health, 19: (14), 9324025. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9324025/. |
25 | McInroy, L. B. , Mishna, F. ((2017) ). Cyberbullying on online gaming platforms for children and youth. Child and Adolescent Social Work Journal 597–607. https://doi.org/10.1007/S10560-017-0498-0. |
26 | Menesini, E. , Nocentini, A. , Calussi, P. ((2011) ). The measurement of cyberbullying: Dimensional structure and relative item severity and discrimination. Cyberpsychology, Behavior, and Social Networking, 14: , 267–274. https://doi.org/10.1089/cyber.2010.0002. |
27 | Nocentini, A. , Calmaestra, J. , Schultze-Krumbholz, A. , Scheithauer, H. , Ortega, R. , Menesini, E. ((2010) ). Cyberbullying: Labels, behaviours and definition in three European countries. Australian Journal of Guidance and Counselling, 20: (2), 129–142. https://doi.org/10.1375/ajgc.20.2.129. |
28 | Notar, C. , Padgett, S. , Roden, J. ((2013) ). Cyberbullying: A review of the literature. Universal Journal of Educational Research, 1: (1), 1–9. https://doi.org/10.13189/ujer.2013.010101. |
29 | Ofcom (2023a). Online nation 2023 report. Ofcom.https://www.ofcom.org.uk/__data/assets/pdf_file/0029/272288/online-nation-2023-report.pdf. |
30 | Ofcom (2023b). Children and parents: Media use and attitudes. Ofcomhttps://www.ofcom.org.uk/__data/assets/pdf_file/0027/255852/childrens-media-use-and-attitudes-report-2023.pdf. |
31 | Purdy, N. , Hamilton, J. , Smith, P. K. , Culbert, C. , Scheithauer, H. , Fiedler, N. , Brighi, A. , Mameli, C. , Guarini, A.-L. , Menin, D. , Völlink, T. , Willems, R. (2021). Using focus groups and quality circles to enable pupil voice in European teenagers from socioeconomically disadvantaged backgrounds. In L. Schiamberg & M. Wright (Eds.), Child and adolescent exposure to online risks (pp.403-423). Elsevier. |
32 | Purdy, N. , Ballentine, M. , Lyle, H. , Orr, K. , Symington, E. , Webster, D. , York, L. (2023). Growing up online: Children’s online activities, harm and safety in Northern Ireland –an evidence report. Stranmillis University College/Safeguarding Board for Northern Ireland. https://www.stran.ac.uk/growingup-online-in-ni/. |
33 | Scheithauer, H. , Schultze-Krumbholz, A. , Pfetsch, J. , Hess, M. (2021). Types of Cyberbullying. In P. K. Smith & J. O’Higgins Norman (Eds.), The Wiley Blackwell handbook of bullying: A comprehensive and international review of research and intervention (Volume 1, Chapter 7, pp. 120-138).Wiley Blackwell. |
34 | Skogen, J. C. , Bøe, T. , Finserås, T. R. , Sivertsen, B. , Hella, R. T. , Hjetland, G. J. ((2022) ). Lower subjective socioeconomic status is Associated with increased risk of reporting negative experiences on social media. Findings from the “LifeOnSoMe”-study. Frontiers in Public Health, 1749: , 1–12. https://doi.org/10.3389/fpubh.2022.873463. |
35 | Skogen, J. C. , Andersen, A. I. O. , Finserås, T. R. , Ranganath, P. , Brunborg, G. S. , Hjetland, G. J. ((2023) ). Commonly reported negative experiences on social media are associated with poor mental health and well-being among adolescents: Results from the “LifeOnSoMe”-study. Frontiers in Public Health, 11: , 1192788. https://doi.org/10.3389/fpubh.2023.1192788. |
36 | Smahel, D. , Machackova, H. , Mascheroni, G. , Dedkova, L. , Staksrud, E. , Ólafsson, K. , Livingstone, S. , Hasebrink, U. (2020). EU Kids Online 2020: Survey results from 19 countries. EU Kids Online.https://doi.org/10.21953/lse.47fdeqj01ofo. |
37 | Smith, P. K. , Mahdavi, J. , Carvalho, M. , Fisher, S. , Russell, S. , Tippett, N. ((2008) ). Cyberbullying: Its nature and impact in secondary school pupils. Journal of Child Psychology and Psychiatry, 49: , 376–385. https://doi.org/10.1111/j.1469-7610.2007.01846.x. |
38 | Tokunaga, R. S. ((2010) ). Following you home from school: A critical review and synthesis of research on cyberbullying victimization. Computers in Human Behavior, 26: (3), 277–287. https://doi.org/10.1016/j.chb.2009.11.014. |
39 | Uchino, B. N. , Jordan, K. D. , Smith, T. W. (2023). Positive and negative online social experiences and selfrated health: Associations and examination of potential pathways. Health Psychology. Advance online publication.https://dx.doi.org/10.1037/hea0001338. |
40 | Whittaker, E. , Kowalski, R. M. ((2015) ). Cyberbullying via social media. Journal of School Violence, 14: , 11–19. https://doi.org/10.1080/15388220.2014. |
41 | Willard, N. (2007). Cyberbullying and cyberthreats: Responding to the challenge of online social aggression, threats, and distress. Research Press. |
42 | Willems, R. A. , Smith, P. K. , Culbert, C. , Purdy, N. , Hamilton, J. , Völlink, T. , Scheithauer, H. , Fiedler, N. , Brighi, A. , Menin, D. , Mameli, C. , Guarini, A. ((2023) ). Internet use and perceived parental involvement among adolescents from lower socioeconomic groups in europe: an exploration. Children, 10: , 1780. https://doi.org/10.3390/children10111780. |
Bio Sketches
Noel Purdy is Director of Research and Scholarship and Director of the Centre for Research in Educational Underachievement at Stranmillis University College, Belfast, Northern Ireland. His main research interests are in educational disadvantage and underachievement, special educational needs and pastoral care, with a particular interest in addressing on- and offline bullying in schools.
Herbert Scheithauer is Professor for Developmental and Clinical Psychology at Freie Universität Berlin, Berlin, Germany, and Head of the Unit “Developmental Science and Applied Developmental Psychology”. His research interests are on bullying, cyberbullying, and the development and evaluation of preventive interventions.
Jonathan Harris is a political geographer at King’s College London, following posts at Stranmillis University College and Cambridge University. His research uses mixed methods to examine the intersections of power and space in diasporic, postcolonial and educational settings.
Roy A. Willems is Assistant Professor at the Department of Psychology at the Open Universiteit, Heerlen, The Netherlands. His research focusses on understanding and reducing stigma and bias-based bullying. He is an expert in systematic intervention development and evaluation and has successfully implemented several evidence-based interventions. The goal of his work is to contribute to a more inclusive society.
Consuelo Mameli is Associate Professor in Developmental and Educational Psychology at the Department of Education Studies “G. M. Bertin”, University of Bologna, Italy. Her main research interests are in student well-being, teacher –student interactions, student engagement and agency, and bullying and cyberbullying in schools.
Annalisa Guarini is a Full Professor of Developmental and Educational Psychology at the Department of Psychology “Renzo Canestrari”, University of Bologna and Director of the SERES Psychological Service. She focuses her research on language, number and cognitive skills in typical and atypical populations and on bullying and cyberbullying phenomena, with particular attention to prevention at school.
Antonella Brighi is Full Professor of Developmental and Educational Psychology at the Free University of Bolzano, Italy and Dean of the Faculty of Education. She leads international and national research projects on bullying and cyberbullying in schools and in juvenile justice contexts. Her research focuses on risk and protective factors from bullying and cyberbullying, the impact of violence on mental health, the design and validation of intervention programs for the prevention of violence in school and online and for the promotion of well-being, from childhood to adolescence.
Damiano Menin is assistant professor at the University of Ferrara. His scientific background includes research on motor behaviour and development in early infancy, and peer relationships and well-being in adolescence, with an emphasis on bullying, cyberbullying, school climate, positive peer relationships, as well as on multilevel risk and protective pathways to peer aggression and its psychological, emotional and social consequences.
Catherine Culbert is a PhD student at the Division for Health Research, Lancaster University, England. Her main research interests are cyberbullying, mental health and neurodevelopmental disorders among school-aged children.
Jayne Hamilton is a trained primary school teacher and PhD student at Queen’s University Belfast, Northern Ireland. Her thesis is a qualitative study, focusing on the evolving cultural identity of children in Northern Ireland. Her research experience is wide, ranging from systematic reviews to qualitative reflexive methodologies and her research interests lie in pastoral care, educational interventions and social justice issues.
Trijntje Völlink is an assistant professor at the Open University of the Netherlands. Her scientific background encompasses research on the determinants of cyberbullying and evaluation studies related to interventions aimed at preventing and addressing cyberbullying. Currently, her primary focus centres around qualitative studies exploring the determinants and mechanisms of coaching in a natural environment.
Mark Ballentine is a Research Fellow at Stranmillis University College, with a background in STEM and education research, and teaching engineering, mathematics and digital design at HE level. Mark also has many years of experience volunteering in the community, including working with disadvantaged youth and adults.
Nora Fiedler is a Post-Doc Researcher at the Department of Education and Psychology at Freie Universität Berlin, Germany. Her scientific background includes research on school shootings and bullying with an emphasis on early, indicated preventative approaches and crisis management. Other research interests are teacher well-being, mindfulness-based interventions, and experiences of migration and parenting practices of refugee fathers in Germany.
Peter K Smith is Emeritus Professor of Psychology at Goldsmiths, University of London. His most extensive research has been on bullying in schools, where he has led research projects including a COST Action on Cyberbullying. He jointly edited (with James O’Higgins Norman) The Wiley Blackwell Handbook of Bullying: A Comprehensive and International Review of Research and Intervention (Wiley-Blackwell, 2021).
