Patient and Public Involvement and Engagement in the Development of a Platform Clinical Trial for Parkinson’s Disease: An Evaluation Protocol
Abstract
Background:
Patient and public involvement and engagement (PPIE) in the design of trials is important, as participant experience critically impacts delivery. The Edmond J Safra Accelerating Clinical Trials in PD (EJS ACT-PD) initiative is a UK consortium designing a platform trial for disease modifying therapies in PD.
Objective:
The integration of PPIE in all aspects of trial design and its evaluation throughout the project.
Methods:
PwP and care partners were recruited to a PPIE working group (WG) via UK Parkinson’s charities, investigator patient groups and participants of a Delphi study on trial design. They are supported by charity representatives, trial delivery experts, researchers and core project team members. PPIE is fully embedded within the consortium’s five other WGs and steering group. The group’s terms of reference, processes for effective working and PPIE evaluation were co-developed with PPIE contributors.
Results:
11 PwP and 4 care partners have supported the PPIE WG and contributed to the development of processes for effective working. A mixed methods research-in-action study is ongoing to evaluate PPIE within the consortium. This includes the Patient Engagement in Research Scale -a quantitative PPIE quality measure; semi-structured interviews -identifying areas for improvement and overall impressions of involvement; process fidelity- recording adherence; project documentation review – identifying impact of PPIE on project outputs.
Conclusions:
We provide a practical example of PPIE in complex projects. Evaluating feasibility, experiences and impact of PPIE involvement in EJS ACT-PD will inform similar programs on effective strategies. This will help enable future patient-centered research.
INTRODUCTION
Patient and public involvement and engagement (PPIE) is increasingly globally recognized as a crucial factor in bringing key benefits to the entire life cycle of research projects [1]. PPIE ensures research is relevant to patients and addresses their needs and priorities. Furthermore, inclusive, sensitive research design increases accessibility, and thereby has a positive impact on research deliverables [2]. To harness these benefits and ensure the success and relevance of research, strong PPIE is becoming a key requirement for charity and government funders as well as policy makers [3].
For clinical trials in particular, success is contingent on the willingness of participants to contribute towards research. Failure to engage with trial participants significantly impacts research delivery. As many as 44% of trials do not reach their recruitment target [4]. Those that do are often not representative of patient populations that will receive treatments thereby risking generalizability of findings and leading to suboptimal treatment guidelines and care through lack of evidence [5].
Parkinson’s disease (PD) research faces particular challenges as this slow, progressive neurodegenerative disease affects primarily an elderly population prone to additional co-morbidities, who are underrepresented in trials [6–8]. Recruitment and retention of elderly populations is often prohibited through trial design, such as restrictive eligibility criteria, length of follow up, choice of outcome measures and assessment schedules [9, 10]. Attrition in trials lies between 10–20% [4]. In particular for PD, retention has been demonstrably linked to trial design rather than treatment tolerability or safety [7, 11]. Engagement with patients from the inception of research is fundamental for overcoming barriers to participation and leads to more inclusive studies, enhancing participant experiences [12, 13].
When executed well, effective collaborative relationships with patient and their care partners have the potential to be beneficial for all project stakeholders enhancing knowledge exchange and trust, and leading to better research experiences [14]. Evaluations of public involvement in health research are rare; a systematic scoping review conducted by the PARADIGM consortium in 2020 mapping evaluation methodology, identified only 91 articles across the entirety of health research. Of those articles only 16 involved patients as partners and only 24 focused on clinical trials [14]. Furthermore, impacts of patient partnerships are rarely measured in terms of project outcomes [15].
Common practical challenges to involving patients in co-design include power imbalances, insufficient training, restrictive research time pressures, overburdening of volunteers and involvement of volunteers at research stages where impact is limited [14, 16, 17]. Several frameworks and guidance on good quality public involvement exist from funders, government bodies and initiatives [3] to help mitigate these challenges. For example, initiatives such as NIHR INVOLVE have led to the development of a framework providing UK national guidance on how to successfully integrate the public in research projects [18]. The Patient Focused Medicines Development (PFMD), a global multi-stakeholder collaboration, has proposed a meta-framework, outlining quality guidance for patient engagement activities [19]. A call to action by Meinders and colleagues within the Journal of Parkinson’s Disease in 2022 highlights that practical examples of how to implement such guidance and frameworks alongside methodology for evaluating their impact are needed to encourage their integration into projects [20].
The Edmond J. Safra Accelerating Clinical Trials for Parkinson’s Disease (EJS ACT-PD) initiative is a UK based consortium delivering a 3-year program to design a multi-arm, multi-stage platform trial investigating disease modifying therapies for PD [21]. The initiative has involved people with Parkinson’s (PwP) and care partners within a co-design partnership from the beginning to support and advise in all aspects of protocol development as well as trial delivery decisions.
Understanding the challenges affecting quality and effectiveness of public involvement in co-design, the EJS ACT-PD initiative has co-developed processes to mitigate these and facilitate engagement with its PPIE contributors. A comprehensive evaluation of these processes was integrated into the initiative from the start to allow the evaluation of PPIE involvement quality alongside its impact on the project. This is rarely reported but important, not only to build an evidence base of the practical challenges and facilitators of implementing PPIE frameworks in projects, but also to create an understanding of where PPIE can add most value [22, 23].
Here we share our co-designed processes for patient involvement as well as our protocol for evaluating PPIE within the EJS ACT-PD initiative.
MATERIALS AND METHODS
Formation of the PPIE working group
The PPIE working group is chaired by a PwP (Kevin McFarthing (KMF)) with 10 years of activity in PD advocacy and a strong research interest. The working group chair was involved throughout the formation of the PPIE working group, giving guidance on recruitment of PwP and care partners, desirable attributes of candidates and selection of applicants.
UK based PwP and care partners were recruited to the PPIE working group via advertisement through UK Parkinson’s charities, investigator PPIE groups as well as the UK based PwP and carer contributors to a Delphi study in trial design [24]. The invitation included a short description of project background, remit of involvement as well as details about reimbursement for time spent on the project. Interested PwP and care partners were required to complete a short application form which asked for their reasons for wanting to join the initiative and any relevant experience as well as demographic information including gender, ethnicity, age, year of diagnosis and geographic location. Applications were reviewed by the project leads and the chair of the PPIE working group. PPIE contributors were selected to join the EJS ACT-PD initiative based on enthusiasm, experience and demographic characteristics with the aim of recruiting 10 PPIE representatives overall. The background information given to applicants and the application form can be found in Supplementary Material 1. Two to three PPIE representatives advise on each of the five working which consist of about 12–15 members. Working groups are multi-professional, with each profession or specialism having 2 or 3 representatives, tailored to the working groups remit, so that the PPIE ‘voice’ would be similar to that of other individual professional contributions.
Setting aims and values of patient and public involvement
The EJS ACT-PD initiative was launched in June 2021 and is a 3-year program that brings together national and international experts to develop a protocol and sustainable delivery strategy for a multi-arm, multi-stage platform trial for disease modification therapies in PD.
PPIE working group terms of reference were reviewed and agreed with PPIE working group members at the beginning of the program to ensure both project leadership and the PPIE team were comfortable with their roles and aims. An evaluation of PPIE impact was included within the terms of reference of the group to ensure auditable documentation processes were put in place from the outset in line with the evaluation’s goals allowing dependable, confirmable, transferable and reliable data collection and analysis (Supplementary Material 1) [25].
The PPIE working group within the EJS ACT-PD consortium
The EJS ACT-PD initiative comprises five working groups addressing treatment selection, outcome measures, funding and sustainability, infrastructure and trial design.
To seamlessly integrate PPIE within the project, two or three PPIE contributors are members of each of five working groups (Fig. 1). All of the PPIE contributors come together to form a sixth working group (the PPIE working group) alongside other stakeholders including representatives from patient organizations (Parkinson’s UK and Cure Parkinson’s), trial delivery experts and researchers with a particular interest in patient involvement. This provides a rich environment for discussions to gain a wider perspective on topics. Professional members of the PPIE working group help guide discussion, provide trial communications and trial delivery expertise, as well as access to wider patient and care partner networks where required. The group is additionally supported by the EJS ACT-PD core project team including administrative staff, a project manager, a research fellow and research assistant.
Fig. 1
The EJS ACT-PD consortium structure. The consortium consists of five working groups each charged with one aspect of the trial’s development: Trial design, treatment selection, outcome measures, infrastructure, funding, and sustainability. Two to three PwP/care partners together support each of the five working groups. Together with charity representatives as well as trial delivery experts they form a sixth patient and public engagement working group. The chairs of each working group and the project leads form the steering group. A core team consisting of a project funded administrator, project manager, and research staff overseen by the project leads support the working groups in delivering their objectives.
Chairs of all working groups, including the PPIE working group chair, and project leads form the EJS ACT-PD project steering group. This allows PPIE representation to directly impact decision making.
Development of processes for effective working
Processes for effective working including a training program were discussed and co-developed with PPIE contributors within the first PPIE WG meetings to meet the specific needs of the group. To achieve this, working group members drew on their prior experiences in PPIE, discussing what had previously worked well for them and how this could be adopted by EJS ACT-PD. Processes for effective working are reviewed annually. Reviews included requests for informal feedback on the group’s working, unsolicited feedback from working group members or the review of results from the PPIE evaluation. Feedback is discussed with a view to making positive changes to the workings of the PPIE working group. Suggested changes are additionally reviewed by the core team and project leads to assess their effects and acceptability to the wider consortium after which changes are implemented.
Development of the PPIE Evaluation study
An evaluation study to assess PPIE within the EJS ACT-PD consortium was co-designed with professional consortium members as well as PPIE contributors to maximize acceptability and raise awareness of the PPIE evaluation study. A sub-group including the PPIE working group chair (KMF), a PPIE representative (Antony Morgan (AM)), and a professional working group member (Natasha Ratcliffe (NR)) led by Marie-Louise Zeissler (MLZ), met to outline the priorities of the PPIE evaluation. Based on these priorities a study protocol was co-developed with the group and reviewed by the wider PPIE WG for comments prior to submission to the University of Plymouth Faculty Research Ethics and Integrity Committee. Methodological aspects of the interview component of the study were further refined with a project external qualitative researcher (Nancy R. LaPelle (NRL)). Additional PwP perspective within the evaluation study is provided by a project external PwP (Elaine Cowd (EC)) who reviews all participant facing materials and advises on interview scripts and data presentation.
RESULTS
Working group composition
At the time of writing (November 2023), the working group had been supported by 15 volunteers including 11 PwP and 4 care partners. Other (non-core team) working group members included 2 Trial delivery experts, 3 researchers with a special interest in PPIE and 2 charity representatives. Of the volunteers 9 PwP and 4 care partners agreed for their demographic information to be included within this publication. Details can be found in Table 1.
Table 1
Demographics characteristics of all volunteers who contributed to the project (Nov 2023 cut-off) and agreed to contribute demographic information for publication
| People with Parkinson’s (N = 9) | Care Partners (N = 4) | |
| Age (Median (Range)) | 63 (48, 77) | 75 (49, 80) |
| Male (%) | 44.4% | 25% |
| Years since diagnosis (Median (Range)) | 8 (3, 15) | – |
| Active caring role (%) | – | 50% |
| Years caring (Median (Range)) | – | 7 (5, 16) |
| Years in Education (%) | 19 years or over – 88.9% | 19 years or over – 100% |
| 18 years – 11.1% | ||
| White British (%) | 77.8% | 75% |
Co-designed processes for effective working
To build patient and care partner understanding and facilitate communication of PPIE recommendations across the project, the following EJS ACT-PD processes were co-developed with PPIE contributors and are embedded within the project:
1. PPIE working group feedback is an early agenda item at all meetings.
2. A movement break is scheduled within the agenda of all meetings.
3. Post-meeting debriefs between PPIE representatives and the working group chairs take place at the end of each working group meeting and are scheduled into the overall meeting time within agendas. They ensure a shared understanding of discussions and clear objectives for wider PPIE consultation as they arise.
4. Co-designed Standard Reporting Forms are completed as part of the post-meeting debrief (Supplementary Material 2). The forms record concerns or discussion points relevant to the PPIE group, matters to be discussed with the PPIE working group, or matters that may need wider PwP input from outside the initiative. Once matters have been discussed within the PwP working group, responses from the PPIE working group to these queries are recorded within the same form. Thus, the reporting forms facilitate communication between working groups and PPIE members, supporting group input into issues arising. Standard Reporting Forms are completed with administrative support during the debrief conversation and after PPIE working group meetings and made available as part of meeting enclosures to support transparency.
5. Monthly PPIE forums provide opportunities for informal education and themed, in-depth discussions.
6. Post forum feedback surveys allow for session evaluation as well as identification of future topics.
7. A WhatsApp group exclusive to PwP and care partner representatives creates a safe space for sharing experiences and ideas. Topics arising from this are fed back by the PPIE working group chair via a standing agenda item during PPIE working group meetings.
8. To allow for wider input into the project and enhance representation of underserved and ethnically diverse groups, the EJS ACT-PD initiative set up two additional advisory groups. Topics for consultation with these two groups can be specified in the standard reporting form together with working group chairs or by request from the PPIE working group directly. Outcomes of discussions are fed back to the working groups as appropriate. A wider network of 34 individuals recruited via Parkinson’s UK and Cure Parkinson’s networks which includes people with Parkinson’s and their care partners who have consented to be consulted on an ad-hoc basis with short surveys or focus group requests. This wider network serves as a rapid response pool to canvas wider PPIE opinion without the delay inherent in going through charity facilitated channels; A community advisory panel including members of the public with lived experience of a chronic illness was brought together to advise on matters of communication, recruitment and retention and accessibility. This panel meets every two months and was especially recruited to represent ethnic minority backgrounds and individuals from geographically remote locations.
The PPIE Evaluation study
The PPIE working group evaluation subgroup identified three key aspects that the PPIE evaluation should address: evaluating the impact of PPIE contributions, ensuring processes are working and using the evaluation to help improve PPIE within the project. We therefore embedded a mixed methods research-in-action study within the program with the following main objectives: a) assess whether PPIE contributions have an impact on the project outputs including the trial’s protocol and delivery plan; b) highlight barriers and facilitators to contribution and effective involvement based on PFMD PPIE quality criteria; c) assess whether processes to facilitate PPIE contribution are working; and d) document the best strategies for effective involvement following an action research approach where outcomes are directly translated into process improvements and their impact is explored. The ongoing study evaluates perspectives of both researchers and PPIE contributors, acknowledging that successful partnership involves all stakeholders [26].
The study commenced in November 2022 and follows a longitudinal, mixed methods approach [23] based on action and evaluation research principles [27–30] as outlined in Fig. 2.
Fig. 2
Study overview. a) The evaluation study integrates both quantitative and qualitative data capture elements allowing for an ongoing evaluation of PPIE within the EJS ACT-PD initiative. The rigorous, systematic and iterative evaluation framework will allow PPIE to be continually adapted to improve PPIE within EJS ACT-PD and to inform best practice of PPIE in relation to existing PPIE quality guidance. b) Timeline of evaluation processes throughout the EJS ACT-PD project.
Ethics approval
Ethical approval for this study was granted by the University of Plymouth Faculty Research Ethics and Integrity Committee (Ref: 3085).
In brief, operational aspects are being evaluated annually through an evaluation of process fidelity. Bi-annual administration of a modified Patient Engagement In Research Scale (PEIRS) allows quantitative comparison of professionals’ as well as patient and care partner views on PPIE [31, 32]. Semi-structured interviews are conducted midway through and at the end of the program by a qualitative researcher independent from the EJS ACT-PD program. This will give insights into acceptability of processes and perceived impact on project outputs, as well as provide opportunities to identify and address areas requiring improvement. Reports of critical findings and actionable results are disseminated to the consortium to support improved PPIE experience and contribution. Improvements will be implemented and their effect investigated sequentially throughout the project.
Study participants and recruitment
At the time of writing, the consortium comprised 92 members including 72 professional members, 12 PPIE contributors, six core team members and two project leads. Recruitment is ongoing throughout the project timeline to enable new consortium members to join the evaluation. Consortium members are sent the participant information sheet by email and invited to complete an online registration survey if willing to participate. The study is additionally promoted through consortium newsletters and at working group meetings. Consent for the modified PEIRS survey as well as an expression of interest to take part in interviews are requested during registration. Willingness to participate has no impact on members’ roles within the consortium and this is clarified during the recruitment process. Participation is voluntary and, in case of PPIE contributors, separate from their remunerated advisory role in the EJS ACT-PD consortium. To protect anonymity, all participants are assigned a unique identifier on enrolment.
Modified PEIRS surveys
The modified PEIRS surveys are open to all EJS ACT-PD consortium members at each data collection time point. We aim for 80% of consortium members from each of the six working groups to complete each survey round. Since consortium membership fluctuates throughout the duration of the project, new consortium members will be allowed to enter the study at any stage and recruitment to the evaluation study will remain open throughout the duration of the project.
Semi-structured interviews
We aimed to recruit 20 individuals (10 PPIE contributors and 10 professional members) to participate in interviews from those who expressed an interest to participate upon study registration. Potential participants were sent the study participant information sheet and consent statement for the interview part of the study and invited to a video conferencing call to discuss the study and provide consent. This was recorded and signed by the researcher via screen-share. A signed copy of the consent form was emailed to the participant during the meeting. Participants were then put in contact with the qualitative researcher to arrange the interview.
Potential interview participants were selected out of those who express interest to ensure representation from each working group and reflecting low and high satisfaction with the program’s PPIE as indicated by top and bottom quartiles of PEIRS scores. This ensures the full range of experiences is reflected within this qualitative study component. However, we recognize that the available participant pool for selection is small and therefore likely to represent a convenience sample. Participants were asked to consent to both interviews, however it is anticipated that some may leave the EJS ACT-PD project during the course of the study. Should this occur, we aim to replace these participants from those who expressed interest in interviews ensuring similar selection characteristics as much as possible.
Data collection
Study registration survey
To enter the study, participants complete an online registration form collecting contact information, membership category (PPIE contributor, working group member, chair or EJS ACT-PD administration team), their working group membership (PPIE, trial design, infrastructure, outcome measures, treatment selection, funding and sustainability) and demographic information including gender, age, ethnicity and self-reported experience of PPIE involvement outside EJS ACT-PD. Participants are able to express their interest in the interview study and are asked to give consent to their details being shared with the qualitative researcher.
Modified PEIRS-22 survey
PEIRS-22 is a validated 22-item tool [32] reduced from the original, unvalidated 37-item scale [31]. It has been developed to measure quality of engagement from a patient perspective covering 7 domains: procedural requirements, convenience, contributions, team interactions, support, feeling valued and benefits. Minor modifications were made to apply the scale to researchers asking them about their perceptions of patient/care partner involvement. In addition, the baseline questionnaire contained all items of the 37-item PEIRS “procedural requirement”-domain asked once in the context of the EJS ACT-PD program start and once in context of the study baseline time point. Inclusion of these additional items was pertinent to capture views on understanding and set-up of the project and therefore formed a useful addition to the baseline questionnaire. Modifications were discussed with the authors of the scale and permissions for their use was granted. Due to copyright, the exact wording of modifications will not be disclosed.
Participants are completing the PEIRS-22 in online survey format every 6 months throughout the program duration [32]. At each timepoint, participants are asked to self-report meeting attendance frequency to give an indication of the participants’ exposure to EJS ACT-PD working groups. Participants are also given the opportunity to provide open text responses within each survey domain to allow them to expand on their experiences.
Semi structured interviews
Interviews are carried out to gain a deeper understanding of participants’ experiences of PPIE involvement within EJS ACT-PD to a) understand feasibility and effectiveness of processes b) gain insights into facilitators and barriers to PPIE involvement in research design and c) understand participant perceptions of PPIE impact on the program. Interviews are conducted by a qualitative researcher external to the EJS ACT-PD project to ensure uninhibited responses from participants and to reduce chances of introducing confirmation bias during the interview.
Participants are invited to two semi-structured interviews: one at 18 months from EJS ACT-PD initiation and one at 36 months (project end). The first set of interviews has been concluded and focused on how working processes might be improved; the second interview will include reflection on the impact of changes introduced as a result of analyses of initial interview findings. Interview guides have been co-developed and finalized with the qualitative researcher (NRL), with PEIRS domains as well as the PFMD guidelines for good quality patient engagement [19] being used as a framework for their development (Table 2). Initial results from the modified PEIRS surveys also informs interview content. The qualitative researcher carrying out the interviews as well as interviewees have access to their PEIRS survey answers as well as the interview questions in advance of interviews. PPIE contributors recommended that this enables them to give considered responses during the interview especially for those struggling with recall due to their PD.
Table 2
Considerations for interview guide development
| Topic guide considerations | PEIRS domains | PFMD guidelines for good quality patient engagement |
| Overall impressions of PPIE involvement in project | ||
| Project set up | Procedural requirements (1) | Share purpose (1) |
| Roles and responsibilities (4) | ||
| Representativeness of stakeholders (3) | ||
| Facilitating contributions | Convenience (2) | Capacity and capability for engagement (5) |
| Contributions (3) | ||
| Team interactions (4) | Respect and accessibility (2) | |
| Support (5) | Transparency in communication and documentation (6) | |
| Project impact | Feeling valued (6) | Continuity and sustainability (7) |
| Benefits (7) |
Numbers in brackets indicate the item number for PEIRS domains and PFMD guidelines respectively.
Each interview is digitally recorded via video-conferencing software such as Zoom or equivalent for transcription purposes. Interviews are then transcribed verbatim by a sponsor-approved transcription service, anonymized and analyzed by the qualitative researcher.
Evidencing process fidelity
To assess fidelity of deployment of processes developed to support PPIE contribution, adherence to operational aspects developed to support PPIE contribution is reviewed. This includes EJS ACT-PD meeting agendas and minutes to confirm inclusion of a PPIE working group feedback item. PPIE debrief forms and meeting minutes are evaluated to assess whether meeting debriefs 1) took place, 2) included all relevant parties, 3) debrief items were discussed in relevant meetings (by tracking agenda items and presence of topics within minutes), 4) outcomes were recorded within the form. A data capture proforma can be found in Supplementary Material 3.
Document review to assess impact
To capture impact of PPIE contribution, we will conduct a document review including meeting minutes, debrief forms and reconcile suggestions/contributions with project outputs, including discussion papers, the final protocol, any publications and project delivery plans. We will use NVivo software to cross-reference text from meeting documentation (agenda, minutes, debrief forms) and project outputs (protocol and trial delivery plan) to record the topics discussed by the PPIE working group, whether these were brought back to the other 5 working groups and what effect they had on project outcomes. The software will allow the creation of a detailed record for analysis.
Data analysis
Data analysis for this study will be carried out sequentially and follow an iterative approach integrating three aspects of data capture (Interviews, PEIRS surveys and process fidelity) (Fig. 2a) to: a) determine whether PPIE working group contributions had an impact on the project outputs including the trial protocol and delivery plan; b) highlight barriers and facilitators to contribution and effective involvement, exploring these under the framework of PFMD PPIE quality criteria; and c) document the best strategies for effective involvement following an action research approach where outcomes are directly translated into improvements and their impact is explored.
The final study report will include a reflexivity statement for transparency to document how the researchers’ project roles, personal backgrounds and experiences, preconceptions and viewpoints may have influenced the generation and analysis of data.
Modified PEIRS survey
We will quantitatively explore PPIE involvement by PEIRS domain: procedural requirements, convenience, contributions, team interactions, support, feeling valued and benefits. We will compare overall and individual domain scores between professional consortium members and PPIE contributors and by working group, and also explore changes over time.
Baseline PEIRS questionnaire analysis will be used to further inform interview guide development. Analysis will be performed as outlined by the scale authors. Each item is rated on a 5-point Likert scale from ‘strongly agree’ to ‘strongly disagree’, which is scored 4 to 0 respectively and represented as percentages of total questionnaire or total domain scores.
Statistical analysis will be carried out using SPSS software. We will employ the Wilcoxon rank-sum test to compare differences between working groups, as well as compare professional member and PPIE contributor experiences at each survey time point. We will compare within group changes over time using the Wilcoxon signed-rank test for comparison of the first two timepoints and the Friedman test to allow inclusion of more than two timepoints [33].
Inductive content analysis will be conducted on open ended free text data to identify themes and subthemes by a single reviewer. Content analysis facilitates the reporting of common issues mentioned in data, allowing the quantitative reporting of themes. It is appropriate for analysis of survey data since detailed context that allows the researcher to infer meaning is lacking [34].
Semi structured interviews
Interim interviews have already been conducted and an initial analysis leading to action points has taken place. An inductive thematic analysis is performed in the first instance [35]. In a second step, emerging themes are mapped to the patient engagement quality criteria developed by the Patient Focused Medicines Development (PFMD) collaboration. This process identifies whether elements of PPIE involvement incorporated within the EJS ACT-PD initiative map to guidance on good practice, processes have been effective to achieve these and how participants think these could be improved moving forward.
A random sample of two interviews of research professionals and two interviews of PPIE contributors is co-analyzed by a second qualitative researcher to cross-validate emerging themes. This second qualitative researcher is also external to the EJS ACT-PD project team to guarantee anonymity of participants. Outcomes of the analysis are fed back to interview participants in a member checking process to improve accuracy, credibility and validity of results.
A record of the analysis process is kept through transparent recording of raw data association with themes, subthemes and participant categories (working group association and project role).
Evidencing process fidelity
Percentage completion of processes is recorded. We will examine trends over time using descriptive statistics and data visualization. Process fidelity reviews highlight whether processes are being implemented.
Document review to assess impact
We will use the following main aims as a framework for analysis, gaining an understanding of:
1) Whether PPIE contributions were listened to in meetings, achieved by triangulating minuted PPIE working group discussions, points raised within standard reporting forms. We will record the topics and whether these resulted in decisions or discussions minuted within the other 5 working groups.
2) The impact of PPIE on project outputs, achieved by correlating minuted working group decisions recorded in 1) with the final protocol and trial delivery plan.
3) Topics where PPIE had most impact; we will categorize topics identified in 1) and 2) into where PPIE opinion aligned with other stakeholders, opinion diverged but had no effect, achieved a compromise or were taken on board without compromise.
4) Barriers and facilitators to achieving impact
Reasons for achieving or not achieving impact as categorized in 3) will be documented.
We will use NVivo software to support this process as this will provide a detailed record of this complex analysis.
DISCUSSION
Patient and public involvement is becoming a key requirement for health research regulatory and funding bodies [3]. Despite this, clinical trials rarely report on PPIE [36]. In addition, the sharing of learnings derived from evidence of effective involvement especially in the co-design of complex projects is limited [14, 37]. This makes it challenging for researchers to effectively incorporate public involvement in their projects. Studies that try to implement best practice and evaluate their effect are therefore needed [19]. The EJS ACT-PD initiative has set up a consortium structure that maximizes involvement of PPIE in all aspects of the project. Supportive processes were co-developed with PPIE representatives to facilitate their roles within the project and an evaluation study is underway to evaluate the success of these strategies.
Achieving a representative PPIE group was challenging and a limitation to our work. Due to the nature of the work and the level of engagement required all PPIE volunteers were highly educated. Furthermore, to achieve co-design it is important to involve volunteers in discussions and within the decision making process. Providing sufficient training opportunities for specialist topics is therefore crucial. All of this takes considerable time and commitment from volunteers which is difficult to achieve for some such as those who are older, PwP with longer disease duration or those actively caring for someone with PD. Furthermore, although the PPIE working group did include volunteers who were not from White-British backgrounds this was by no means representative of ethnic minority groups in England. It is important to ensure diverse voices are included in the development of trials as barriers to research participation are often dependent on an individuals background. Acknowledging the limited diversity within the core PPIE working group, two additional advisory groups were set up, the EJS ACT-PD Wider Network and Community Advisory Panel to address two separate but important needs: broader imput that is timely—circumventing waiting times associated with charity outreach, and addressing ethnic and geographic diversity—which was hard to achieve across existing working groups within the timescales of initial project set-up due to a lack in existing networks within the UK.
We will evaluate the effectiveness and impact of PPIE both through formal research processes such as interviews and surveys and through reviews of project output and documentation. A rigorous, systematic evaluation framework was embedded into the EJS ACT-PD initiative from the outset which is important to enable adequate data capture to answer research questions [25, 30].
The evaluation was co-developed with EJS ACT-PD members, including PPIE representatives from the PPIE working group, to ensure the acceptability of this approach, achieve stakeholder buy-in and encourage retention of participants throughout this longitudinal study. Update reports and resulting adjustment of processes will be clearly communicated with consortium members to ensure that efforts of participation are rewarded with actionable results and improvements.
In terms of the evaluation there are several limitations. In our protocol we decided to take a formal approach, consenting participants to a research study. Others have argued that evaluations of involvement should be carried out in a collaborative manner with patients as partners without formally consenting participants to emphasize a collaborative approach. One benefit of holistic assessment that does not require consent is that all aspects and people can contribute to the evaluation without introducing selection bias [16]. However, this is difficult to achieve as interpretation of co-development is not clear cut and such a proposal may be against regulatory and ethical standpoints of participating institutions. Furthermore, even within a collaborative framework, principles of free choice to participate in activities still apply and it can therefore be reasoned that similar biases could be introduced. Our purposive sampling strategy, which takes into account satisfaction with PPIE by selecting interview participants with high and low PEIRS scores, ensures that positive as well as negative experiences are captured. However, selection will be limited by consortium size. Interviews and analysis are carried out by an external qualitative researcher to preserve confidentiality of interviewees. Due to resource constraints, only a subset of transcripts will be co-analyzed by a second external qualitative researcher which impacts on the rigor of analysis. To mitigate this a member checking process will be conducted whereby interviewees review interview findings and ensure their views and experiences are represented.
The PEIRS tool is only validated to capture PPIE contributors’ views and therefore our modifications to allow its administration to researchers may impact on the scale’s sensitivity to change. Insights into the appropriateness of using this tool to capture researchers’ experience of PPIE will be an interesting additional outcome of this study.
The restriction of the document review to formal project outputs and meeting documentations forms another limitation to this study since informal communications via e-mail and other direct messaging platforms may be missed. However, the reporting structures and processes integrated within the project, such as agenda items and reporting forms, which include the capture of informal feedback requests should mitigate this challenge.
A published report of the study will be submitted to a peer-reviewed journal and reported following GRIPP2 (Guidance for Reporting Involvement of Patients and the Public 2) guidelines to improve quality, transparency, and consistency with other PPIE evaluation efforts [38]. A final report of the main findings will be shared with the EJS-ACT PD consortium as well as all participants, including those that contributed to the study but left the project.
Our holistic approach will ensure that practical insights of involvement are captured alongside the impact on final outcomes. The longitudinal aspect of this study is key, as it acknowledges the dynamic nature of long duration projects and allows the capture and impact of these changes to PPIE involvement quality. In particular, the repeatedly administered PEIRS survey together with process fidelity and document review will allow us to triangulate impacts of procedural changes including whether changes implemented between analyses had an impact on PPIE quality and meaningfulness.
The evaluation of impact on project outcomes is valuable not only to provide evidence for such impact in general, but also to pinpoint topics where PPIE input has most benefit, as this will facilitate targeting of PPIE efforts in projects where resources are constrained. The results of the evaluation will therefore provide much needed evidence on effective strategies to achieve good quality patient engagement.
ACKNOWLEDGMENTS
We would like to thank all PPIE working group members past and present including those who wished not to be named as co-authors for all their hard work and dedication that they have given to this project.
FUNDING
This work was conducted as part of the Edmond J Safra Accelerating Clinical Trials in Parkinson’s Disease (EJS ACT-PD) Initiative, which is funded by the Edmond J Safra Foundation.
CONFLICT OF INTEREST
CBC, TF, ELL and KMF are Editorial Board members of this journal, but were not involved in the peer-review process nor had access to any information regarding its peer review.
DATA AVAILABILITY
Data sharing is not applicable to this article as no datasets were generated or analyzed during this study.
SUPPLEMENTARY MATERIAL
[1] The supplementary material is available in the electronic version of this article: https://dx.doi.org/10.3233/JPD-230444.
REFERENCES
[1] | Aiyegbusi OL , McMullan C , Hughes SE , Turner GM , Subramanian A , Hotham R , Davies EH , Frost C , Alder Y , Agyen L , Buckland L , Camaradou J , Chong A , Jeyes F , Kumar S , Matthews KL , Moore P , Ormerod J , Price G , Saint-Cricq M , Stanton D , Walker A , Haroon S , Denniston AK , Calvert MJ , TLC Study Group ((2023) ) Considerations for patient and public involvement and engagement in health research. Nat Med 29: , 1922–1929. |
[2] | Crocker JC , Ricci-Cabello I , Parker A , Hirst JA , Chant A , Petit-Zeman S , Evans D , Rees S ((2018) ) Impact of patient and public involvement on enrolment and retention in clinical trials: Systematic review and meta-analysis. BMJ 363: , k4738. |
[3] | Hoddinott P , Pollock A , O’Cathain A , Boyer I , Taylor J , MacDonald C , Oliver S , Donovan JL ((2018) ) How to incorporate patient and public perspectives into the design and conduct of research. F1000Res 7: , 752. |
[4] | Walters SJ , Bonacho Dos Anjos Henriques-Cadby I , Bortolami O , Flight L , Hind D , Jacques RM , Knox C , Nadin B , Rothwell J , Surtees M , Julious SA ((2017) ) Recruitment and retention of participants in randomised controlled trials: A review of trials funded and published by the United Kingdom Health Technology Assessment Programme. BMJ Open 7: , e015276. |
[5] | Rochester L , Carroll C ((2022) ) Implications of research that excludes under-served populations. Nat Rev Neurol 18: , 449–450. |
[6] | Vaswani PA , Tropea TF , Dahodwala N ((2020) ) Overcoming barriers to Parkinson disease trial participation: Increasing diversity and novel designs for recruitment and retention. Neurotherapeutics 17: , 1724–1735. |
[7] | McGhee DJM , Ritchie CW , Zajicek JP , Counsell CE ((2016) ) A review of clinical trial designs used to detect a disease-modifying effect of drug therapy in Alzheimer’s disease and Parkinson’s disease. BMC Neurol 16: , 92. |
[8] | Hart RG , Pearce LA , Ravina BM , Yaltho TC , Marler JR ((2009) ) Neuroprotection trials in Parkinson’s disease: Systematic review. Mov Disord 24: , 647–654. |
[9] | Thake M , Lowry A ((2017) ) A systematic review of trends in the selective exclusion of older participant from randomised clinical trials. Arch Gerontol Geriatr 72: , 99–102. |
[10] | Carroll CB , Zajicek JP ((2011) ) Designing clinical trials in older people. Maturitas 68: , 337–341. |
[11] | Kehagia AA , North TK , Grose J , Jeffery AN , Cocking L , Chapman R , Carroll C ((2022) ) Enhancing trial delivery in Parkinson’s disease: Qualitative insights from PD STAT. J Parkinsons Dis 12: , 1591–1604. |
[12] | Witham MD , Anderson E , Carroll C , Dark PM , Down K , Hall AS , Knee J , Maier RH , Mountain GA , Nestor G , Oliva L , Prowse SR , Tortice A , Wason J , Rochester L , INCLUDE writing group ((2020) ) Developing a roadmap to improve trial delivery for under-served groups: Results from a UK multi-stakeholder process. Trials 21: , 694. |
[13] | Adrissi J , Fleisher J ((2022) ) Moving the dial toward equity in Parkinson’s disease clinical research: A review of current literature and future directions in diversifying PD clinical trial participation. Curr Neurol Neurosci Rep 22: , 475–483. |
[14] | Vat LE , Finlay T , Jan Schuitmaker-Warnaar T , Fahy N , Robinson P , Boudes M , Diaz A , Ferrer E , Hivert V , Purman G , Kürzinger M-L , Kroes RA , Hey C , Broerse JEW ((2020) ) Evaluating the “return on patient engagement initiatives” in medicines research and development: A literature review. Health Expect 23: , 5–18. |
[15] | Boylan A-M , Locock L , Thomson R , Staniszewska S ((2019) ) “About sixty per cent I want to do it”: Health researchers’ attitudes to, and experiences of, patient and public involvement (PPI)-A qualitative interview study. Health Expect 22: , 721–730. |
[16] | Bird M , Ouellette C , Whitmore C , Li L , Nair K , McGillion MH , Yost J , Banfield L , Campbell E , Carroll SL ((2020) ) Preparing for patient partnership: A scoping review of patient partner engagement and evaluation in research. Health Expect 23: , 523–539. |
[17] | Price A , Albarqouni L , Kirkpatrick J , Clarke M , Liew SM , Roberts N , Burls A ((2018) ) Patient and public involvement in the design of clinical trials: An overview of systematic reviews. J Eval Clin Pract 24: , 240–253. |
[18] | Crowe S , Adebajo A , Esmael H , Denegri S , Martin A , McAlister B , Moore B , Quinn M , Rennard U , Simpson J , Wray P , Yeeles P ((2020) ) “All hands-on deck”, working together to develop UK standards for public involvement in research. Res Involv Engagem 6: , 53. |
[19] | Deane K , Delbecque L , Gorbenko O , Hamoir AM , Hoos A , Nafria B , Pakarinen C , Sargeant I , Richards DP , Skovlund SE , Brooke N ((2019) ) Co-creation of patient engagement quality guidance for medicines development: An international multistakeholder initiative. BMJ Innov 5: , 43–55. |
[20] | Meinders MJ , Donnelly AC , Sheehan M , Bloem BR ((2022) ) Including people with Parkinson’s disease in clinical study design and execution: A call to action. J Parkinsons Dis 12: , 1359–1363. |
[21] | Foltynie T , Gandhi S , Gonzalez-Robles C , Zeissler M-L , Mills G , Barker R , Carpenter J , Schrag A , Schapira A , Bandmann O , Mullin S , Duffen J , McFarthing K , Chataway J , Parmar M , Carroll C , EJS ACT-PD Consortium ((2023) ) Towards a multi-arm multi-stage platform trial of disease modifying approaches in Parkinson’s disease. Brain 147: , 2717–2722. |
[22] | Blackburn S , McLachlan S , Jowett S , Kinghorn P , Gill P , Higginbottom A , Rhodes C , Stevenson F , Jinks C ((2018) ) The extent, quality and impact of patient and public involvement in primary care research: A mixed methods study. Res Involv Engagem 4: , 16. |
[23] | Littlewood DL , Quinlivan L , Steeg S , Bennett C , Bickley H , Rodway C , Webb RT , Kapur N ((2021) ) Evaluating the impact of patient and carer involvement in suicide and self-harm research: A mixed-methods, longitudinal study protocol. Health Expect 24 Suppl 1: , 47–53. |
[24] | Zeissler M-L , McFarthing K , Raphael KG , Rafaloff G , Windle R , Carroll CB ((2023) ) an international multi-stakeholder Delphi survey study on the design of disease modifying Parkinson’s disease trials. J Parkinsons Dis 13: , 1343–1356. |
[25] | Lincoln YS , Guba EG ((1982) ) Establishing dependability and confirmability in naturalistic inquiry through an audit. In 66th Annual Meeting of the American Educational Research Association. |
[26] | Staley K ((2015) ) “Is it worth doing?” Measuring the impact of patient and public involvement in research. Res Involv Engagem 1: , 6. |
[27] | Clegg S , Hardy C , Nord W ((1996) ) Action research for the study of organizations. In Handbook of organization studies, Clegg S Hardy C NordW, eds. Sage, Beverly Hills, p. 17. |
[28] | Allen P , Black N , Clarke A , Fulop N , Anderson S ((2004) ) Studying the Organisation and Delivery of Health Services: Research Methods, Routledge, p. 172. |
[29] | Huxham C , Vangen S ((2003) ) Researching organizational practice through action research: Case studies and design choices. Organ Res Methods 6: , 383–403. |
[30] | Powell RR ((2006) ) Evaluation research: An overview. Libr Trends 55: , 102–120. |
[31] | Hamilton CB , Hoens AM , McQuitty S , McKinnon AM , English K , Backman CL , Azimi T , Khodarahmi N , Li LC ((2018) ) Development and pre-testing of the Patient Engagement In Research Scale (PEIRS) to assess the quality of engagement from a patient perspective. PLoS One 13: , e0206588. |
[32] | Hamilton CB , Hoens AM , McKinnon AM , McQuitty S , English K , Hawke LD , Li LC ((2021) ) Shortening and validation of the Patient Engagement In Research Scale (PEIRS) for measuring meaningful patient and family caregiver engagement. Health Expect 24: , 863–879. |
[33] | Marino MJ ((2018) ) Chapter 3 - Statistical analysis in preclinical biomedical research. In Research in the Biomedical Sciences, Williams M, Curtis MJ, Mullane K, eds. Academic Press, pp. 107–144. |
[34] | Vaismoradi M , Turunen H , Bondas T ((2013) ) Qualitative descriptive study. Nurs Health Sci 15: , 398–405. |
[36] | Braun V , Clarke V ((2006) ) Using thematic analysis in psychology. Qual Res Psychol 3: , 77–101. |
[37] | Brennan J , Poon MTC , Christopher E , Fulton O , Porteous C , Brennan PM ((2023) ) Reporting of PPI and the MCID in phase III/IV randomised controlled trials-a systematic review. Trials 24: , 370. |
[38] | Gray E , Amjad A , Robertson J , Beveridge J , Scott S , Peryer G , Braisher M , Pugh C , Peres S , Marrie RA , Sormani MP , Chataway J ((2023) ) Enhancing involvement of people with multiple sclerosis in clinical trial design. Mult Scler 29: , 1162–1173. |
[39] | Staniszewska S , Brett J , Simera I , Seers K , Mockford C , Goodlad S , Altman DG , Moher D , Barber R , Denegri S , Entwistle A , Littlejohns P , Morris C , Suleman R , Thomas V , Tysall C ((2017) ) GRIPP2 reporting checklists: Tools to improve reporting of patient and public involvement in research. Res Involv Engagem 3: , 13. |
