You are viewing a javascript disabled version of the site. Please enable Javascript for this site to function properly.
Go to headerGo to navigationGo to searchGo to contentsGo to footer
In content section. Select this link to jump to navigation

Review of methods to identify the critical job-tasks undertaken by the emergency services

Abstract

BACKGROUND:

The roles of the emergency services are challenging and often physically demanding. Readiness to meet these challenges and demands is a fundamental requirement for staff to deliver their roles safely and effectively. Furthermore, employers are required by law to demonstrate every reasonable effort to protect their staff from undue risk of work-related injury. Implementing Physical Employment Standards (PES) enables employers to assign staff to roles for which they are physically-suited whilst contributing to such duty-of-care. However, for PES to be successful and legally-defendable, standards must reflect the demands of those job-tasks which are truly critical to the readiness of these services.

OBJECTIVE:

To determine whether a standardised approach to identifying critical job-tasks exists in the development of PES for the emergency services.

METHODS:

Studies which conducted analysis of job-tasks to develop PES within the emergency services were reviewed.

RESULTS:

Forty-two reported studies (i.e. records) met the inclusion criteria. Methods to determine job-tasks varied but were typically criteria-based incorporating one, or all, of 9 reported techniques. Methods were subjective and based upon reference to past or present job-task performance.

CONCLUSION:

Correctly determining critical job-tasks is essential for effective, legally-defendable PES. A standardised method to define job-tasks remains to be established.

1Introduction

The concise Oxford English Dictionary defines an emergency as ‘a serious, unexpected, and often dangerous situation requiring immediate action’ (Tenth edition [2002], page 466). Alternative definitions describe incidents (events or situations) which may have already caused (or have a high probability of causing) an immediate threat to life, health/welfare, property, or the environment, including acts of war or terrorism [1]. The Civil Contingencies Act 2004 [1, 2] is an Act of Parliament in the United Kingdom (UK) that establishes a coherent framework for emergency planning and response. Other nations have similar policy and strategies (such as USA’s Homeland Security Act of 2002 [3, 4]). Common to all nations’ strategies is the nature of the emergency services as a collaboration of multiple agencies who are prepared to deliver the necessary immediate response to a wide range of incidents. The UK’s emergency response and recovery strategy [2] was used to define and categorise the roles emergency services are required to deliver. Category 1 first responders (i.e. organisations expected to arrive first on the scene of an incident) comprise: Police services; Fire and Rescue authorities; Paramedic / health agencies; Maritime and Coastguard Agency; local authorities; and Environment Agency. Category 2 responders concern the wider resilience community (including health professionals, transport providers, highways agency, telecommunications, Health and Safety Executive etc.). In the UK the military Armed Forces are mandated to support category 1 and 2 roles via the Military Aid to the Civil Authority (MACA) [5] as indeed the USA’s military personnel may similarly undertake the roles of first responders (akin to the UK’s category 1 response). For the purpose of this review the emergency services were classed as the category 1, or first, responders.

The roles of the emergency services are challenging and often physically demanding with a risk of injury [6]. Statistics for the UK Fire and Rescue Services reported annual (2018 [7]) response to 564,827 incidents (involving 167,150 fires) ranging from false alarms to major incidents. Manning strength at the start of the reporting year was 40,964 full time staff of which 33,049 were employed as firefighters [6]. By the end of the year 4,425 staff had left the service. Injuries sustained by firefighters in a similar period included 2,523 cases of which 1,071 injuries had been sustained during emergency operations.

In order to assure the necessary manning strength with which to deliver the operational capability there is a need to ensure that incumbents are able to undertake their roles safely and effectively. Furthermore, in order to satisfy employment law and to increase the size and scope of the population from which applicants may be recruited there is a need to diversify the workforce in terms of ethnicity, gender, age and ability. An example from the Fire and Rescue Service (England) reported [6] that female firefighters accounted for only 5.2% of incumbents and that only 3.9% of all firefighters represented ethnic minority groups. Methods to assess applicants and incumbents for their readiness to meet the physical demands of roles in the emergency services continue to evolve. Physical Employment Standards (PES) describe the level of physical performance that must be demonstrated during such assessments in order to meet the requirement for employment. In order to be defended in-Law, PES must reflect the actual demands of conducting job-tasks upon which operational success relies (i.e. the discrete actions or tasks which are undertaken as part of a job within the role of the emergency service).

The focus for this review were the methods used to identify those job-tasks which are critical to the emergency services, and upon which PES have been (or will be) established.

1.1The purpose of Physical Employment Standards (PES): Why develop them?

Tipton et al. [8] state that the purpose of establishing a PES is to ensure that workers can complete the critical (in terms of performance and productivity) and generic (i.e. common or regular) tasks associated with their job without undue stress to themselves and to others working with them. Such purpose suggests that the physical demands of work may not be achieved by all people and that there is an associated risk of injury which may, within reason, be managed to its irreducible minimum. However, legislation mandating that employers enforce policies for the provision of health and safety at work (i.e. duty of care) and equal opportunities (without discrimination) have likely been the underlying drivers for PES since the 1970’s [9–11]. Recognising the risk of work-related injury and the need to maintain a safe, effective workforce of sufficient size, employers have increasingly invested in the development of PES as part of their selection process and to inform the training of recruits and incumbents. Efficiently placing people to the work for which they are best physically suited has been the premise of strategies to promote cost effectiveness in recruitment (as it is hoped that staff successfully complete their training at the first attempt, progress to deliver their roles without injury and remain in-service for the duration of their career). Increased legal scrutiny (since 2015) of the rationale for the exclusion (by many Armed Forces of various nations) of women from employment in the ground close combat roles emphasised the need for evidence-based PES.

1.2The process for developing PES: How to develop them?

There are a number of well-established frameworks for the design and development of PES. The fundamental process for developing PES has been the topic of numerous technical events since the 1990’s [12–18]. Whilst researchers in Canada responded to legal scrutiny by establishing Bona Fide Occupational Requirements (BFOR) [12] other authors continued to report a similar process for developing physical fitness standards [19]. Whilst Gledhill et al. [12] had described the development of a BFOR in 12 stages and Taylor and Groeller [20] offered their generic planning model (incorporating trade- and task-analyses), the 6-stage process described by Tipton et al. [8] has been most often cited for PES:

  • Stage 1: Establish the critical job-tasks (i.e. job-analysis);

  • Stage 2: Determine the method of best practice for undertaking the critical job-tasks;

  • Stage 3: Agree the criteria-for, and acceptable minimum level of, job-task performance;

  • Stage 4: Determine the physical demands of the critical job-tasks;

  • Stage 5: Determine a reasonable maximum permissible relative workload; and

  • Stage 6: Production of a valid minimum occupational fitness standard.

Common to all approaches reported in the evidence-based literature is the need to correctly identify the critical job-tasks at the very start of the process. However, varied criteria have been reported for determining critical job-tasks and the methods used to acquire the evidence (i.e. qualitative and quantitative data).

1.3The need to standardise the PES process

The precedent has been set within occupational research for use of the 6-stage process in developing PES [8, 21–24]. However, similar precedent is less evident with regard to the methods for conducting the initial stages of this process which are often contested in the Court of Law and must withstand legal scrutiny [24]. Reference to job analyses, task analyses, and trade analyses are commonplace for stage 1 in the process but they describe very different methods and data. Recognising the need for standardisation, researchers have proposed methods for each stage [22–24]. Payne and Harvey [23] provided their (often cited) comprehensive framework for the design of physical employment tests and standards, in which they discussed job-task analyses (considering component tasks and analysing the mode, frequency, duration, intensity and work: rest ratio). However, the description, by Tipton et al. [8], of job-task analysis as objectively subjective, highlighted the complexity of techniques that have been used to investigate this first stage in the PES process. Methods to select (and acquire data from) reliable, experienced, informed expert practitioners of the critical job-tasks have been suggested by leading researchers of PES [22, 24]. In 2016 an international delegation of these PES experts (representing 9 nations) formed a North Atlantic Treaty Organization (NATO) research task group [25] with the objective of designing an international standardisation agreement defining the methods to use when developing PES for combat roles.

1.4Job-task analysis

A job-task analysis (sometimes known as a job analysis, task analysis or work analysis) has been defined [21, 26] as the process for establishing an accurate accounting of the tasks or activities that take place in a job. No published guidelines exist to instruct researchers on how to conduct job-task analyses [21, 27] and there is no single correct method that fits all requirements. Hardison et al. [21] recommended that the choice of method may necessarily vary according to the intended purpose of the analysis (e.g. to develop PES or to review doctrine). They [21] added that the centrality of a job-task analysis in defending the use of a selection system or PES could not be overstated. There was a fundamental requirement to correctly identify and characterise the job-tasks which were evidently critical to the success of the emergency service. Furthermore, the job-task analysis was considered to be fundamental to ensuring that the standards for an occupation were valid predictors of critical job requirements (such assurance was considered essential in order to withstand legal scrutiny [28]). Landy and Vasey [29] reported that plaintiffs would typically assert that there was a fatal flaw in the job analysis techniques, analyses, results, or inferences. Thus, the choice of method used to define the content of the job would be vital in addressing some of the criticisms that might be raised. Larsen and Aisbett [27] asserted that given the importance of reporting accurate and legally defensible job-task analyses it was imperative to critically evaluate methods and to provide a best-practice approach for future research.

Hardison et al. [21] reviewed several methods used to develop PES for the Armed Forces. The techniques that were cited when conducting job analyses included; [a] analysis of documentation such as job descriptions, training manuals and task inventories (i.e. the employer’s list of tasks that were in-scope for specific jobs); [b] review of relevant scientific literature and professional competencies; [c] conduct of site visits to observe, and to interview, incumbents performing their jobs; [d] interviews with instructors and job-supervisors; [e] judgement panels comprising Subject Matter Experts (SME); and [f] on-line staff surveys. Zumbo [30] suggested criteria to apply when selecting SMEs as suitably qualified and experienced professionals. Use of technology (where participants rated the performance of scenario-based tasks that were observed via recorded imagery) enabled Siddall et al. [31] to acquire real-time data from anonymized e-voting systems within their expert judgement panels. Methods were cited which acquired both qualitative and quantitative data [8] and included ratings of task importance, difficulty, intensity, duration and frequency.

1.5Objective of this study

To identify the methods by which critical job-tasks have been determined when developing PES within the emergency services and to assess whether a standardised approach may exist.

2Method

A review of the evidence-based literature was conducted using six information systems: (a) Web of Science v5.30; (b) National Center for Biotechnology Information’s PubMed; (c) Google Scholar; (d) United Kingdom’s (UK) defence research database known as ATHENA; (e) Defense Technical Information Center (DTIC ® which is an open-access repository managed by the United States of America’s Department of Defense); and (f) Scientific publications available from Australia’s Defence Science and Technology (DST) Group website. Searches were limited to documents that had been reported between 01 January 1976 and 31 July 2018. Search terms (i.e. keywords) used in this study were identified following a preliminary search of the literature (using PubMed). Assurance that the keywords were appropriate and inclusive was assumed following a further search of the literature which successfully acquired several publications (of varying citation) that were known to meet the study’s inclusion criteria. The keywords used in this study were: Physical Employment Standard (PES); job-task; job analysis; review; emergency services; method; first responder; physical performance; emergency services; fire; ambulance; paramedic; police; government; military and Bona Fide Occupational Requirement (BFOR). To find additional studies, the reference lists of the articles obtained were also investigated. Only information that was reported by each study was included in this review. No additional information was sought from the authors of the reported studies. An initial search was conducted to determine whether such a review had already been reported in the open literature.

2.1Eligibility for the review

Titles and abstracts of the acquired studies were assessed for their eligibility against the following criteria:

a. Qualification for review required the following conditions to be met (i.e. inclusion criteria):

  • Reported within the search dates (i.e. between 01 January 1976 and 31 July 2018);

  • Written in English;

  • Studies reporting methods to identify essential, or critical, physically-demanding job-tasks within the emergency services;

  • Novel studies at their first reporting with the objective to inform the development of PES at the time of reporting or in the near future;

  • Evidence of peer-review;

  • Context relevant to PES (keyword search).

b. Records were excluded if they were:

  • Classified documents which the owner would not authorise for release;

  • Instruction manuals or reference documents;

  • Not applicable to the emergency services;

  • Unable to identify one or more critical job-tasks;

  • Lacking in sufficient detail to describe the techniques used in the method;

  • Reviews of, or proposals for, ‘best-practice’ when developing PES which did not report previously unreported data from a novel study;

  • A duplicate report of a study which had appeared in another publication;

  • Incomplete or draft manuscripts and documents which had not undergone peer-review.

The above criteria formed a checklist which was used by two of the authors of this review to independently assess the studies for their eligibility. Only those studies which were assessed by both authors as eligible were included in the review.

2.2Processing the reported methods

Details of the methods reported were entered into a Microsoft Excel workbook. Each reported method was assigned to one of the following {1} to {9} categories:

  • {1} Surveys

  • i.e. questionnaires, focus groups, interviews, workshops etc;

  • {2} Facilitated expert judgement panels

  • e.g. military judgement panels;

  • {3} Review of policy and doctrine

  • including training manuals;

  • {4} Historical analysis

  • i.e. Use of data which reported the course of action, and performance, of the emergency services when managing past incidents. This included incidents reports, after-action-reviews and recorded lessons;

  • {5} Case law

  • Considering litigation and any legal precedent concerning the legitimacy of defined job-tasks as essential, or critical, to the role of the emergency service (and underpinning a legally-defendable PES);

  • {6} Observation and analysis of real-life work and/or training exercises

  • Use of research staff to observe and record details of tasks undertaken by trained incumbents conducting their role in a real, or realistic (i.e. training), scenario;

  • {7} Job descriptions and performance appraisals

  • i.e. Definitions used by the emergency service to describe specific jobs and the criteria that incumbent peers (and managers) apply to assess the quality of staff performance

  • {8} Consultation with Subject Matter Experts (SME) and stakeholder panels

  • e.g. Access to appointed liaison officers and stakeholder working groups that are empowered to provide evidence-based advice to the study team (distinct from {1} or {2} by the assignment of a specific SME advisor to the project and/or by direct involvement with the employer’s working group or committee)

  • {9} Other reported methods

  • e.g. Mathematical modelling, role-play and research staff conducting the training courses intended to prepare personnel for a role within the emergency service

A process for assessing (e.g. scoring) the quality of implementation for each reported method was not conducted.

2.3Analyses

The methods that were reported by each of the studies which successfully met the eligibility criteria were recorded by their category ({1} to {9}) and quantity (i.e. the number of methods used from 1 [minimum] to 9 [maximum]). The frequency with which each individual method was reported (between studies) was calculated as a proportion (% ) of the total number of eligible studies in the review. The Microsoft Excel data analysis histogram tool was used to determine the frequency (calculated as a proportion [% ] of the eligible studies) with which multiple methods had been reported. The statistical mode was calculated for the number of methods reported by the studies (data were not adjusted for heterogeneity).

3Results

3.1The literature search

Use of the keywords in the initial search identified 617 records. No previously reported review was identified which sufficiently met the objective of this present study. A further 6 records were acquired from other (manual) sources which included proceedings from scientific conferences. Hence, a total of 623 records were assessed for their eligibility against the inclusion/exclusion criteria. Figure 1 presents the outcome of each stage of the assessment process which resulted in the independent assessments by 2 of the authors of this present study concurring that 42 of the records qualified for this review.

Fig.1

Results of each stage of the process in assessing eligibility for inclusion in this present review.

Results of each stage of the process in assessing eligibility for inclusion in this present review.

3.2Findings from the studies that were reviewed

The number and type of methods used by each of the 42 records included in this review have been described in Table 1. The value for the mode number of methods or techniques reported by a record was 4 (methods reported ranged from 1 [least] to 9 [most]). Twenty-five records reported using 4 or more method categories (Fig. 2), whilst 2 of the remaining 17 records relied exclusively upon the use of surveys (i.e. applied only 1 method). Within the 42 studies that were reviewed the use of survey methods were evident in 38 (90% ) cases whilst consultation with Subject Matter Experts (SME) and stakeholder panels (empowered to make decisions regarding the emergency service), i.e. {8}, was cited in 34 (81% ) cases (Fig. 3).

Fig.2

Use of methods to identify critical job-tasks in the emergency services (as reported by the [n = 42] studies in this review).

Use of methods to identify critical job-tasks in the emergency services (as reported by the [n = 42] studies in this review).
Table 1

Methods (reported by 42 studies) to identify critical job-tasks in the development of

Qualifying studyParticipant populationEMERGENCY SERVICEMethods to identify job-tasks used in PES (see * key)Additional informationConsidered future job-tasks? (Y/N)Nation
123456789 (Other)
Blacker et al. [35]Ground Close CombatMILITARY (COMBAT)xxxxxxxxReview of mission tasks lists. Y UK
Milligan [43]Maritime Coast Guard Agency(NAVY)xxxxxxxNUK
Nevola et al. [44]BAA fire fightersFIRExxxxxDeveloped a list of core operational tasks.Incumbent and instructor-informed job performance criteria. Y UK
Nevola et al. [45]Police (Home Office)POLICExxxxxxIdentified core role related patrol tasks. Used Police competency framework. Y UK
Nevola et al. [46]Royal Air ForceAIRxxxxxxNUK
Gruse et al. [47]Explosive Ordnance Disposal (EOD)SPECIALISTxxxxxxAccess to task inventories.NUSA
Gledhill and Jamnik [48]FirefightersFIRExxxxxxNCAN
Myers et al. [49]All Arms ArmyARMYxxxxxNUK
Taylor et al. [50]Urban FirefightersFIRExxxxxNAUS
Foulis et al. [51]Army Combat ArmsMILITARY (COMBAT)xxxxxMilitary Occupational Specialisations (MOS) lists.NUSA
Kilding and Fordy [52]ArmyARMYxxxxxNNZ
Siddall et al. [31]Firefighters and commandersFIRExxxxxNUK
Reilly [53]Royal National Lifeboat Institute crewNAVYxxxxxNUK
Vine et al. [54]Royal Marine CommandoMILITARY (COMBAT)xxxxxNUK
Allen et al. [55]Armed ForcesMILITARY (COMBAT SUPPORT)xxxxPanel of extramural expert advisors in PES.NCAN
Blacker et al. [56]Fire and RescueFIRExxxxConsultation with policy makers.NUK
Deakin et al. [57]Armed ForcesMILITARY (COMBAT SUPPORT)xxxApplied previous definitions of critical tasks (used in PES).Developed common emergency tasks.NCAN
Doyle et al. [58]Defence ForceMILITARY (COMBAT SUPPORT)xxxxNAUS
Gumieniak [59]Wildfire fightersFIRExxxDeveloped scenarios.NCAN
Marshall-Mies et al. [60]Department of Defense adjudicatorsSPECIALISTxxxxDeveloped task list (database).NUSA
Milligan et al. [61]Offshore Wind IndustryINDUSTRYxxxxNUK
Prusaczyk et al. [62]Sea, Air, Land (SEAL) personnelNAVYxxxxNUSA
Rayson et al. [34]ArmyARMYxxxxNUK
Silk et al. [63]PoliceSPECIALISTxxxxApplied delphi method.NAUS
Sothmann et al. [42]FirefightersFIRExxxxTask inventory included cognitive requirements.NUSA
Fischer et al. [64]ParamedicsPARAMEDICxxxNCAN
Middleton et al. [65]Royal Australian NavyNAVYxxxNAUS
Redmond et al. [66]Infantry Indirect Fire BatteryARMYxxxMilitary Occupational Specialisations (MOS) lists. Y USA
Silk and Billing [67]Defence Force Dismounted Assault TaskMILITARY (COMBAT)xxxNAUS
Robson et al. [68]Tactical Air Control Party and Air Liaison OfficersSPECIALISTxxx Y USA
Jamnik et al. [69]Correction OfficersPOLICExxxApplied delphi method to responses structured into emergency scenarios.NCAN
Doolittle et al. [70]Line WorkersSPECIALISTxxxNCAN
Osborn [71]PolicePOLICExxResearchers conducted the incumbent training course to familiarise with job-tasks.NUSA
House et al. [72]Royal Navy DiversNAVYxxNUK
Stein et al. [73]Special Weapons And Tactics (SWAT)SPECIALISTxxSME moderator of information.NUSA
Stuart-Hill et al. [74]Marine Search and RescueNAVYxxNCAN
Stevenson et al. [75]Armed ForcesMILITARY (COMBAT SUPPORT)xxNCAN
Smith [76]Royal NavyNAVYxxNUK
Anderson et al. [77]PolicePOLICExxNCAN
Reilly et al. [78]Royal National Lifeboat Institute beach lifeguardsNAVYxMathematical modelling.Theoretical analysis of time requirements.NUK
Anderson et al. [79]PolicePOLICExNCAN
Brown and Fallowfield [80]Royal NavyNAVYxAnalysis of a generic task list.NUK

* Key (methods): {1} Surveys (questionnaires, focus groups, interviews, workshops); {2} Facilitated expert judgement panels (including military judgement panels); {3} Review of policy and doctrine (including training manuals); {4} Historical analysis of past reported emergencies; {5} Case law; {6} Observation and analysis of real-life work and/or training exercises; {7} Job descriptions and performance appraisals; {8} Consultation with Subject Matter Experts (SME) and stakeholder panels; and {9} Other reported methods.

In 22 (52% ) cases the studies described observing incumbents conducting their service roles, i.e. {6}, (either during operational duty or scheduled training exercises) as well as seeking assurance by cross-reference to policy and doctrine, i.e. {3} (established tactics, techniques and procedures for use in both training and emergency scenarios).

The miscellaneous methods which were categorised within {9} included: (a) research staff undertaking the training of the emergency service (to gain understanding and practical experience of the tasks and their associated physical demands); (b) use of mathematical modelling techniques; (c) development of operational scenarios in which the critical job-tasks were conducted; (d) development, or review, of operational task lists; and (e) reference to critical job-tasks which had been defined by previous research. Thirteen (31% ) records cited the use of task lists developed by, or on behalf of, the emergency service (e.g. Military Occupational Specialisations [MOS], and Mission Essential Task Lists [METL]). Only 5 (12% ) records sought information to describe how job-tasks may be conducted in future scenarios (obtained by method category {1}, {8} or {9}), or acquired guidance from SMEs regarding strategic plans for the immediate future of the emergency service (e.g. the planned introduction of new equipment and operating procedures). Records (n = 37 [88%]) tended to limit methods to acquiring evidence from past or present operations and doctrine.

Fig.3

Number (and proportion) of the 42 studies reporting use of multiple methods to identify critical job tasks in the emergency services.

Number (and proportion) of the 42 studies reporting use of multiple methods to identify critical job tasks in the emergency services.

4Discussion

The results support the position that no standardised method or criteria have been reported to determine critical job-tasks for the purpose of developing PES in the emergency services. However, these results were limited to inclusion of only those documents baring the security classification which permitted access by the aforementioned methods. Authors of the records included in this review described constraints in the availability of (and access to) information managed within (or on behalf of) the employing emergency service which limited their methods. Military and security services (particularly maritime) accounted for the overriding majority of records which were eligible for this review whilst certain elements of the emergency services (e.g. environment agency etc.) were notable by the lack of published information to describe their job-tasks. Method categories {4} and {5} (i.e. historical analyses and case law, respectively) received fewest citations within the reported methods. Whilst completed legal cases challenging PES within each emergency service may have been comparatively few it was surprising that analyses of empirical data from operations were rarely considered (despite surveys typically requesting incumbents to provide subjective ratings of their operational performance) [81].

4.1Standardising the identification of critical job-tasks: Factors to consider

Dukalskis and Beadle [32] suggested that there were many commonly used methods for systematically obtaining details regarding a job, task or role (i.e. job analysis [33]). However, the choice of method varied and depended upon the type of information that was hoped to be gained and the purpose of the investigation. It was evident that the variance between services regarding the availability of appropriately managed information and data was an important factor in the cases that were reviewed (e.g. MOS; competency frameworks; training manuals; incident reports and investigations; incident lessons; performance databases; continuous professional development records etc.). Hence, any future intent to standardise the methods for determining critical job-tasks [25] (and their performance criteria) must encourage employers to maintain robust and accessible information management systems. Operational data are needed to understand the determinants of success and failure when describing job-task performance.

It is likely that PES will be subjected to further legal challenge in the future. Subsequent reviews of this nature should consider the methods that were used to establish critical job-tasks in cases where PES was successfully defended. Where such methods are found to be consistently successful in their legal defence, they should be incorporated as best practice when developing PES.

Establishment of an international standard (for developing PES) will facilitate data sharing and help to determine reliable success criteria when setting standards or cut-off scores in performance assessments. However, a paradox may exist where the intent to reflect the reality of conduct on operations is in conflict with the method of best practice as advocated in doctrine and underpinning PES. An example of a physically demanding job-task has included the replacement of a wheel on a truck [34]. Although doctrine may describe procedures for loosening the wheel nuts using upper body muscle strength, in practice applying body weight to the wrench (by standing on it) may reflect the reality of common practice on operations and result in a lesser requirement for upper body muscle strength. Implications for strict adherence to doctrine (in such cases) when setting PES need to be understood and explained to the employer.

4.2Implications of strategic plans on the priority and performance of future job-tasks

It can take several years to develop PES [24, 34, 35]. Strategic review of the roles of the emergency services (including procurement of new equipment, infrastructure, protective clothing and operating procedures) is typically conducted within a similar timescale [36–38]. Therefore, criteria used to develop PES must remain relevant to the operational roles for which they are expected to represent else they risk being out-of-date as soon as they are implemented [82]. When designing a BFOR, Gledhill et al. [12] stated that the job-task analysis needed to understand the evolution of the job both past and future as well as considering the changing nature of the job (including legitimate variations in how the job-tasks may be achieved). Authors of guidance concerning the process for developing BFOR and PES suggest the need to consider future tasks but they do not offer any specific methods for achieving this. Only 5 of the 42 records included in this review considered the implications of future developments in their critical job-tasks. However, employers undertaking strategic review of the roles in the emergency services typically apply techniques such as Wargaming [39] and red teaming to assess resilience and to investigate future ways of operating. Wargaming roles [39] and scenarios [83] planned for the future workforce (an established scenario-based model where the outcome and sequence of events affect, and are affected by, the decisions made by SME players) was not evident in any study in this review. Wargaming facilitates realistic role-play for specific scenarios [83] which may allow incumbents to repeatedly practice common incidents as well as those emergencies which may occur only rarely in a typical career. Existing guidance for conducting job-task analyses when developing PES [40] does not consider the techniques (such as Wargaming) that are increasingly used by these employers.

Reference to the emergency services describes the collaboration of multiple agencies responding to the same incident. However, PES may differ between services despite the similarity in the emergency job-tasks they conduct. Although some differences may be reasonably defended, there remains a risk of legal challenge which would be wise to anticipate and address [84]. Furthermore, emergency services rarely, if ever, deploy individuals to manage an incident. However, PES are applied to individual performances. Consideration should be given to determinants for effective team working as well as individual competence.

4.3Potential opportunities provided by emerging technology

PES that are valid are those which correctly distinguish between people who are likely to be able to perform the requirements of the job from those who are not. Hardison et al. [21] suggest that without careful implementation and ongoing monitoring and updating, even well designed standards will fail to screen individuals appropriately if the testing is done improperly or as occupational tasks and equipment change over time. Emerging technology is increasingly enabling personnel to be monitored in ways that had not previously been possible. Wearable sensors and stand-off sensing continue to develop rapidly and may soon enable the concept of rolling validation (i.e. on-going evaluation of the physical demands of job-tasks) to be achieved [85]. GPS-based techniques for conducting notational analysis of individuals operating in large complex teams is becoming increasingly available to the emergency services. Process may endure but the methods must progress with such advances in technology in order to benefit from greater ability to monitor true work performance and to improve the sensitivity and specificity of PES.

4.4Employment standards an holistic approach

In 1999 Dukalskis and Beadle [32] developed the FLAG system (a system for conducting a job analysis based upon Fleishman’s algorithm [41]). Their system recognised that jobs are not purely physical in nature but require timely and effective coordination of various competencies (e.g. cognitive and physical). Future standards may need to consider an holistic approach [86] to assessing job-task performance rather than discriminating individual competencies in isolation. Sothmann et al. [42] included cognitive tasks within their job-task inventory as did Padula et al. [86].

4.5Recommendations

The results suggest that there is a need for the emergency services to:

  • 1 [a] Maintain robust and accessible information with which to describe their critical job-tasks;

  • 2 [b] Adopt a consistent approach (i.e. international standardisation) to the development of performance standards which accurately reflect the combination of competencies required to safely and successfully conduct critical job-tasks;

  • 3 [c] Implement employment standards which incorporate the requirements of job-tasks that have been assessed to be critical to both present and likely future operational scenarios (e.g. informed by techniques such as wargaming).

Research to inform techniques which develop and assure employment standards should consider emerging advances in practical, pervasive monitoring technologies (including smart technologies/environments and wearable systems). There is also a need to develop reliable methods to assess the contribution (and performance) of individuals whilst they work in teams during real-time operations. Methods must be applicable to all emergency services and must be readily integrated within their operational systems (without unduly adding to the demands of conducting the job-tasks). Data from real-time operations (undertaken by the emergency services) remain an enduring and critical requirement in the assurance of evidence-based employment standards.

5Conclusion

Correctly determining critical job-tasks is essential for legally-defendable PES. No consistent method was evident to determine critical job-tasks for those studies included in this review. Criteria used to establish critical job-tasks were based upon subjective judgement of past and present performance. A standardised method to define job-tasks when developing PES remains to be established and should include techniques used to inform future roles and strategic force development (e.g. wargaming).

Conflict of interest

None to report.

Acknowledgments

This paper, and its presentation at the 3rd International Conference on Physical Employment Standards (18 July 2018, Portsmouth, UK), was funded by the UK Ministry of Defence Chief Scientific Advisor’s Future workforce and human performance programme (with kind permission of the DSTL programme manager, Mrs Jess Onslow).

References

[1] 

HM Government Acts of Parliament (UK). Civil Contingencies Act 2004 (Chapter 36). Available from: https://www.legislation.gov.uk/ukpga/2004/36/contents

[2] 

Cabinet Office. Emergency response and recovery. Available from: https://www.gov.uk/guidance/emergency-response-and-recovery. 2013.

[3] 

The White House (USA). Homeland Security Presidential Directive/ HSPD–8-National Preparedness. Office of the Press Secretary. 2003.

[4] 

107th Congress. The Homeland Security Act (HSA) of 2002. Public Law 107-296, 116 Stat. 2135. 2002.

[5] 

Cabinet Office. Operations in the UK: The defence contribution to resilience. 2007. Available from: https://www.gov.uk/government/publications/operations-in-the-uk-the-defence-contribution-to-resilience.

[6] 

Home office. Fire and rescue workforce and pensions statistics: England, April 2016 to March 2017 (2nd edition). Statistical Bulletin 18/17. 2018. Available from: https://www.gov.uk/government/statistics/fire-and-rescue-workforce-and-pensions-statistics-england-april-2016-to-march-2017.

[7] 

Home Office. Fire and rescue incident statistics: England, year ending March 2018. Statistical Bulletin 16/18. 2018. Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/732555/fire-and-rescue-incident-march-2018-hosb1618.pdf.

[8] 

Tipton MJ , Milligan GS , Reilly TJ . Physiological employment standards I. Occupational fitness standards: Objectively subjective? Eur J Appl Phys. 2013;113(10):2435–46.

[9] 

Equality Act 2010 (chapter 15). 2010. UK. Available from: https://www.gov.uk/guidance/equality-act-2010-guidance.

[10] 

The European Economic Communities Equal Treatment Directive (76/207/EEC). 76/207/EEC. 1976. Available from: http://www.pfc.org.uk/caselaw/The\%20EEC\%20Equal\%20Treatment\%20Directive\%201976.pdf.

[11] 

The European Economic Communities Council Directive. Introduction of measures to encourage improvements in the safety and health of workers at work. 89/391/EEC.1989. Available from: http://csdle.lex.unict.it/Archive/LW/EU\%20social\%20law/EU\%20legislation\%20in\%20force/Directives/20110715-095610\_89\_391\_EC_Council\_Directive\_enpdf.pdf.

[12] 

Gledhill N , Bonneau J , Salmon A , editors. Proceedings of National forum on Bona Fide occupational requirements. Toronto. 2000.

[13] 

Bentley M , Greeves JP , Myers SD , Wickes SJ , editors. Proceedings of the International Uniformed Services Symposium (IUSS). Report reference QinetiQ/CHS/ CAP/HP010497 (ISBN 0-9541283-1-1). 2002.

[14] 

Taylor NAS , Billing DC , editors. Physiological and physical employment standards. Proceedings of the first Australian conference on physiological and physical employment standards, Canberra. (ISBN 978-1-74128-220-7). 2012.

[15] 

Petersen SR , Anderson GS . The Second International Conference on Physical Employment Standards: An International Perspective. Appl Phys Nutr Metab. 2016; 41(6):iii–iv.

[16] 

Milligan GS , Blacker SD , Siddall A , Brown PEH , Izard R , editors. Proceedings of the 3rd conference on Physical Employment Standards (ISBN: 978-1-5272-2601-2). 2018.

[17] 

Nindl BC , Sharp MA . Third International Congress on Soldiers’ Physical Performance: Translating State-of-the-Science Soldier Research for Operational Utility. J Strength Cond Res. 2015;29(Suppl 11):S1–3.

[18] 

Nevola VR . Fitness for work - The model employee. Proceedings of the International Uniformed Services Symposium (IUSS). (ISBN 0-9541283-1-1). 2002.

[19] 

Constable SH , Palmer B , editors. The process of physical fitness standards development. Human Systems IAC State of the Art Report, Ohio, USA. 2000.

[20] 

Taylor NA , Groeller H . Work-based physiological assessment of physically demanding trades: A methodological overview. J Phys Anthr Appl Human Sci. 2003;22(2):73–81.

[21] 

Hardison CM , Hosek SD , Bird CE . Defining Physical Standards for Physically Demanding Jobs: A Review of Methods. RAND National Defense Research Institute report number RR-1340/1-OSD. Draft prepared for the Office of the Secretary of Defense. 2015.

[22] 

Milligan GS , Reilly TJ , Zumbo BD , Tipton MJ . Validity and reliability of physical employment standards. Appl Phys Nutr Metab. 2016;41(6 Suppl 2):S83–91.

[23] 

Payne W , Harvey J . A framework for the design and development of physical employment tests and standards. Ergonomics. 2010;53(7):858–71.

[24] 

Reilly TJ , Gebhardt DL , Billing DC , Greeves JP , Sharp MA . Development and Implementation of Evidence-Based Physical Employment Standards: Key Challenges in the Military Context. J Strength Cond Res. 2015;(29 Suppl 11):S28–33.

[25] 

North Atlantic Treaty Organisation (NATO) Human Factors and Medicine Research Task group 269. Combat integration – Implications for physical employment standards. NATO Science and Technology Organization. 2016. Available from: https://www.sto.nato.int/search/Pages/activities_results.aspx?k=269&s=Search%20Activities.

[26] 

Pearn M , Kandola R . Job analysis: A managers’ guide. 2nd edition. Inst Pers Dev. 1995.

[27] 

Larsen B , Aisbett B . Subjective job task analyses for physically demanding occupations: What is best practice? Ergonomics. 2012;55(10):1266–77.

[28] 

Hogan J , Quigley AM . Physical standards for employment and the courts. American Psychologist. 1986;41(11):1193–217.

[29] 

Landy FJ , Vasey J . Job Analysis: The Composition of SME Samples. Personnel Psychology. 1991;44(1):27–50.

[30] 

Zumbo BD . Standard-setting methodology: Establishing performance standards and setting cut-scores to assist score interpretation. Appl Phys Nutr Metab. 2016;41(6):S74–S82.

[31] 

Siddall AG , Stevenson RDM , Turner PFJ , Stokes KA , Bilzon JLJ . Development of role-related minimum cardiorespiratory fitness standards for firefighters and commanders. Ergonomics. 2016;59(10):1335–43.

[32] 

Dukalskis L , Beadle I . The development of a Tri-Service job analysis tool: Interim report. Defence Evaluation and Research Agency report number DERA/CHS/MID/WP990008.1999.

[33] 

Dale S . The Development of a Tri-Service Job Analysis Tool: A Scoping Study. Defence Evaluation and Research Agency report number DERA/CHS/HS3/TR97132. 1997.

[34] 

Rayson MP , Bell DG , Holliman DE , Llewellyn MGM , Nevola VR , Bell RL . Physical Selection Standards for the British Army Phases 1 and 2. Army Personnel Research Establishment report number 94R036. 1994.

[35] 

Blacker SD , Myers SD , Nevola VR , Rayson MP , Walker E , Coakley S , Hale B , Cordell NJ , Knapik J , Gebhardt D , Billings A , McDevitt C , Monkhouse J . Technical Report: TIN 3.179 Development and Validation of Role Related Gender Free Physical Employment Standards for Ground Close Combat Roles, in the British Ministry of Defence – Phase 1. Defence Human Capability Science and Technology Centre report. 2017.

[36] 

The Chief Fire Officers Association and the Local Government Association. A Service for the Future - 670,000 Opportunities to Add Value, UK. 2015. Available from: www.cfoa.org.uk/download/58994.

[37] 

Greater London Authority. The future of the London Ambulance Service - A strategic review. (ISBN 978-1-84781-481-4). 2011. Available from: www.london.gov.uk.

[38] 

HM Government. National Security Strategy and Strategic Defence and Security Review - A Secure and Prosperous United Kingdom 2015: (ISBN 9781474125963). 2015. Available from: https://www.gov.uk/government/publications/national-security-strategy-and-strategic-defence-and-security-review-2015.

[39] 

UK Ministry of Defence. Wargaming handbook. Development, Concepts and Doctrine Centre (DCDC). 2017. Available from: www.gov.uk/mod/dcdc.

[40] 

Rayson MR . Job analysis. Chapter 3 in the process of physical fitness standards development. Constable SH and Palmer B , editors. Human Systems IAC State of the Art Report. Ohio, USA. 2000.

[41] 

Fleishman EA , Quaintance MK . Taxonomies of human performance. Academic Press, New York. 1984.

[42] 

Sothmann MS , Gebhardt DL , Baker TA , Kastello GM , Sheppard VA . Performance requirements of physically strenuous occupations: Validating minimum standards for muscular strength and endurance. Ergonomics. 2004;47(8):864–75.

[43] 

Milligan GS . Fitness Standards for the Maritime and Coastguard Agency and the Oil and Gas Industry. PhD Thesis, University of Portsmouth, UK. 2013.

[44] 

Nevola VR , Coyles VR , Myers SD . Role-related physical fitness for fire-fighters within the British Airports Authority. QinetiQ report number QINETIQ/KI/CHS/CR032106. 2003.

[45] 

Nevola VR , Collins SM , Puxley KP , Bentley M . Annex C to QINETIQ/CHS/MC/CR020250 (Physical demands analysis of the operational Police Constable role: Results). QinetiQ report number QINETIQ/KI/CHS/TN022176. 2002.

[46] 

Nevola VR , Puxley MP , Messer PM , Roberts AJ , Collins SM . Development of operationally related physical fitness standards for the Royal Air Force (Phase 1 report). QinetiQ report number QINETIQ/KI/CHS/TR030168. 2003.

[47] 

Matthew Gruse M , Logan R , Flerlage E , Kilby E , Cupples J , Hale K , Baumgartner N . A physical demands analysis to establish critical physical tasks for US Air Force explosive ordnance disposal and battlefield airmen occupations. J Sci Med Sport. 2017; 20(S156).

[48] 

Gledhill N , Jamnik VK . Development and validation of a fitness screening protocol for firefighter applicants. Can J Sport Sci. 1992;17(3):199–206.

[49] 

Myers SD , Rayson MP , Reed PW , Nevola VR , Blacker SD . TIN 3.223 Women in Ground Close Combat (WGCC) All Arms Physical Employment Standards (PES) Phase 1 - Final Report. Defence Human Capability Science and Technology Centre report number 0-DHCSTC-I307344_P_T3_223/002. 2018.

[50] 

Taylor NA , Fullagar HH , Mott BJ , Sampson JA , Groeller H . Employment Standards for Australian Urban Firefighters: Part 1: The Essential, Physically Demanding Tasks. J Occup Environ Med. 2015;57(10):1063–71.

[51] 

Foulis SA , Redmond JE , Warr BJ , Zambraski EJ , Frykman PN , Sharp MA . Development of the Occupational Physical Assessment Test (OPA T) for Combat Arms Soldiers. U.S. Army Research Institute of Environmental Medicine report number Tl6-2. 2015.

[52] 

Kilding H , Fordy G . Development of a Land Combat Fitness Test for the New Zealand Army. Defence Technology Agency report number 388 NR 1661. 2014.

[53] 

Reilly TJ . Fitness Standards for Royal National Lifeboat Institution (RNLI) Lifeboat Crew. PhD Thesis, University of Portsmouth, UK. 2007.

[54] 

Vine CAJ , Rue CA , Walker EF , Coakley SL , Lee BJ , Doherty J , Flood TR , Rayson MP , Knapik JJ , Gebhardt D , Nindl B , Brown PEH , Jackson S , Greeves JP , Myers SD , Blacker SD . Development of a swimming representative military task for the Royal Marine commandos’ physical employment standards. Proceedings of the 3rd conference on Physical Employment Standards. (ISBN: 978-1-5272-2601-2). 2018. pp. 80.

[55] 

Allen CL , Nottrodt JW , Celentano EJ , Hart LEM , Cox KM . Development of occupational physical selection standards (OPSS) for CF - Summary Report. DCIEM report number 84-R-57. 1984.

[56] 

Blacker SD , Rayson MP , Wilkinson DM , Carter JM , Nevill AM , Richmond VL . Physical employment standards for UK fire and rescue service personnel. Occ Med. 2016;66:38–45.

[57] 

Deakin JM , Pelot R , Smith JT , Weber CL . Development of a Minimum Physical Fitness Standard (MPFS) for the Canadian Armed Forces. Queen’s University, Ergonomics Research Group Selection and Maintenance Standards Division. 2000.

[58] 

Doyle T , Billing D , Drain J , Carr A , Ham D , Fogarty A , Carstairs G , Silk A , Best S , Tofari P , Savage R , Lewis M . Physical employment standards for Australian Defence Force employment categories currently restricted to women – part A: Physically demanding trade tasks. Defence Science and Technology Organisation report number DSTO-CR-2011-0377. 2012.

[59] 

Gumieniak RJ . Establishing a legally defensible physical employment standard for Canadian wildland firefighters. PhD Thesis, University of York, Toronto. 2017.

[60] 

Marshall-Mies J , Rigg K , Harding F . Job/Task Analysis for DoD Adjudicators Phase A: Job Tasks for DoD Adjudicators. Office of Naval Research report number 88-04. 1989.

[61] 

Milligan GS , O’Halloran J , Tipton MJ . A critical task analysis for technicians in the offshore wind industry. Proceedings of the 3rd conference on Physical Employment Standards (ISBN: 978-1-5272-2601-2). 2018. pp. 42.

[62] 

Prusaczyk WK , Stuster JW , Goforth HW , Sopchick Smith T , Meyer LT . Physical demands of US Navy Sea-Air-Land (SEAL) operations. Naval Health Research Center report number 95-24. 1995.

[63] 

Silk A , Savage R , Larsen B , Aisbett B . Identifying and characterising the physical demands for an Australian specialist policing unit. Appl Ergonomics. 2018;68:197–203.

[64] 

Fischer SL , Sinden KE , MacPhee RS , Ottawa Paramedic Service (OPS) Research Team. Identifying the critical physical demanding tasks of paramedic work: Towards the development of a physical employment standard. Appl Ergonomics. 2017;65:233–9.

[65] 

Middleton K , Carstairs G , Billing D , Carr A , Lee-Bates B , Caputi P , Linnane D . Development of Physical Employment Standards for the Royal Australian Navy: Validation of Identified Whole-of-ship Tasks. Defence Science and Technology Organisation report number DSTO-TN-1376. 2014.

[66] 

Redmond JE , Foulis SA , Warr BJ , Sauers SE , Walker LA , Canino MC , Hydren JR , Zambraski EJ , Frykman PN , Sharp MS . Development of a physical employment testing battery for Infantry soldiers: 11B Infantryman and 11C infantryman – Indirect Fire. Us Army Research Institute of Environmental Medicine report number T16-10. 2015.

[67] 

Silk AJ , Billing DC . Development of a valid simulation assessment for a military dismounted assault task. Mil Med. 2013;178(3):315–20.

[68] 

Robson S , McCausland TC , Cerully JL , Pezard S , Nahom LR , Beyene M . Evaluating an operator physical fitness test prototype for tactical air control party and air liaison officers. RAND Corporation. (ISBN: 978-0-8330-9944-0). 2018.

[69] 

Jamnik VK , Thomas SG , Shaw JA , Gledhill N . Identification and characterization of the critical physically demanding tasks encountered by correctional officers. Appl Phys Nutr Metab. 2010;35(1):45–58.

[70] 

Doolittle TL , Spurlin O , Kaiyala K , Sovern D . Physical Demands of Lineworkers. Proceedings of the Human Factors Society 32nd Annual Meeting. 1988;632–6.

[71] 

Osborn GD . Validating physical agility tests. Police Chief. 1976;43(1):43–5.

[72] 

House CM , Lunt H , Green A , Tate R . Determination and description of the most physically demanding generic tasks performed by military divers. Institute of Naval Medicine report number 2007.019. 2007.

[73] 

Stein JA , Hepler TC , Cosgrove SJ , Heinrich KM . Relationship between self-reported combat-relevant task frequency, difficulty, and importance. Proceedings of the 3rd conference on Physical Employment Standards. 2018; pp. 32.

[74] 

Stuart-Hill LA , Ryan P . Job demands analysis for the Royal Canadian Marine Search and Rescue volunteers of British Columbia. Proceedings of the 3rd conference on Physical Employment Standards. (ISBN: 978-1-5272-2601-2). 2018. pp. 44.

[75] 

Stevenson JMJ , Bryant T , Andrew GM , Smith JT , French SL , Thomson JM , Deakin JM . Development of Physical Fitness Standards for Canadian Armed Forces, Younger Personnel. Canadian Journal of Sport Science. 1992;17(3):214–21.

[76] 

Smith C . An ergonomic analysis of Royal Navy firefighting. Institute of Naval Medicine report number 98031. 1998.

[77] 

Anderson GS , Plecas DL , Segger T . Police Officer physical ability testing: Re-validating a selection criteria. International Journal of Police Strategies and Management. 2001;24(1):8–31.

[78] 

Reilly TJ , Wooler A , Tipton M . Occupational fitness standards for beach lifeguards. Phase 1: The physiological demands of beach lifeguarding. Occ Med. 2006;56:6–11.

[79] 

Anderson GS , Girard G , Plecas D . Re-evaluation of the physical abilities readiness evaluation: Phase 1 - Task analysis. Proceedings of the 1st Australian conference on physiological and physical employment standards. 2012; pp. 18.

[80] 

Brown PEH , Fallowfield JL . The design and validation of a strength-based Royal Navy Fitness test: Part 1 - identification of the most critically demanding generic tasks performed on board Royal Navy ships. Institute of Naval Medicine Report No. 2012.045. 2012.

[81] 

Sharp MA . Physical Fitness and Occupational Performance of Women in the U.S. Army. Work. 1994;4(2):80–92.

[82] 

Zana J-P , Meyer J-P , Turpin-Legendre E . The physical workload analysis method : Application of ergonomics standards for companies of any size. Work. 2012;41(Supplement 1):5749–51.

[83] 

Blacklock RE , Reilly TJ , Spivock M , Newton PS , Olinek SM . Standard Establishment Through Scenarios (SETS): A new technique for occupational fitness standards. Work. 2015;52(2):375–83.

[84] 

Harley A , James C . Fire-fighters’ perspectives of the accuracy of the Physical Aptitude Test (P.A.T) as a pre-employment assessment. Work. 2006;26(1):29–35.

[85] 

Tegtmeier P . A scoping review on smart mobile devices and physical strain. Work. 2018;59(2):273–83.

[86] 

Padula RS , Rosina JN , Cabral CMN , de Freitas SMSF , Chiavegato LD . The influence of the tasks characteristics in physical performance and psychosocial aspects of workers. Work. 2012;41(Supplement 1):4813–6.