With a little help from our friends : Collaborative research partnerships in three workplace-based occupational disease research projects

BACKGROUND: Research conducted in collaboration between academic and non-academic partners (known as integrated knowledge translation [iKT]) in the field of occupational health and safety needs to be evaluated. OBJECTIVE: This study examined three collaborative workplace-based intervention projects that focused on reducing exposure to occupational carcinogens. Practice, policy and advocacy intermediary organizations partnered with multidisciplinary groups of researchers. This evaluation study sought to understand the characteristics of successful and unsuccessful iKT partnerships from the perspective of the intermediaries. METHODS: Researchers conducted face-to-face interviews with 21 intermediaries and used a thematic-driven “framework analysis” method to analyze the interviews, based upon an evolving conceptual framework. RESULTS: Seven enablers and barriers of collaboration were identified. Enablers included having: adequate capacity; defined project roles; the right partners; an inclusive project leader; mutual respect; good communication; and shared values and priorities. Lacking these was considered a barrier. Seven outcomes were identified as: improved relevance and quality of the research; learning about each others’ “world”; building contacts; improved use of research in practice and policy; dissemination of the research; development of trust and goodwill; and continued collaborations. CONCLUSIONS: Recommendations for future collaborative studies include: spend time defining roles, responsibilities, and expectations; ensure practitioners have the time and resources, and the commitment to the project; and choose representatives from the organizations with the necessary skills or decision-making mandate.


(Integrated) knowledge translation
The field of knowledge translation confirms that "research to practice" is difficult to achieve in all fields including healthcare, education, and the social sciences.The gap between research findings in the field of occupational health and safety, and their use by practitioners also continues to be a challenge.The research on knowledge translation that has examined methods to bridge this gap has highlighted the benefits of the collaboration of researchers and practitioners/partners during the research process.The Canadian Institutes for Health Research, Canada's largest federal funder of health research has taken the lead in advocating for this collaborative method of research.They call it "Integrated Knowledge Translation" (iKT) [1,2].The research on iKT has found that the collaborative research model: 1) improves the likelihood that the researchers and practitioners/partners will increase their understanding and value of each others' world [2][3][4][5][6]; 2) rebalances what is considered 'expertise' [4]; 3) increases the networking across the groups and therefore the spread of knowledge [3][4][5]7]; and 4) results in findings that are more significant than either group could individually contribute [4].
Despite this growing literature, how to achieve a well-integrated collaboration between researchers and practitioners/partners has not been clearly defined.There is yet to be a blueprint on which to base iKT activities.The literature has been criticized for not including sufficient details of the process [8].The practitioners/partners are often only involved to a limited extent in the research [9].There is no underlying theory that reinforces the research; this can be a barrier to cross-project learning [2].The evaluation of the implementation of iKT is often considered anecdotal and hence inadequate [2].

Intermediaries as knowledge users and research partners
Another issue worth highlighting is that the practitioners who become partners on an iKT study can include anyone with a stake in the potential findings and who could use the findings.They could be decision-makers that come from the community, industry, clinicians, government policy-makers, or the media [10].Hence using the catch-all-phrase, "partners", is appropriate.In healthcare, the partners in a research study could be the patients, the clinicians, the hospital administrators, or healthcare policy-makers.In occupational health and safety, the partners in a research study could be the workers, supervisors, managers, owners, health and safety associations, industry organizations, organized labour, or government policy-makers.The partners who join a research team could represent themselves or represent their organizations.
In many projects, the ability to reach the end-users of research findings (patients or workers, in the two fields respectively) is a logistical challenge.For expediency's sake, it makes more sense for researchers to have research partners who are "intermediaries"organizations, associations, societies, or unions who have existing networks of relationships with the endusers and ways of communicating with them.These intermediaries are trusted and are considered credible communicators.They understand the context and cultures of the end-users (processes, political and social ties, limitations, and infrastructure).Importantly, they are able to adapt the research findings to the endusers' needs [2].
Despite these advantages, there has been limited examination in the field of occupational health and safety of how health and safety organizations, industry associations, policy-makers, suppliers of goods and services, membership organizations, educational organizations, government agencies, or others [11] can be partners and collaborators in research.
Responding to this need, nearly a decade ago, based on research conducted in the early 2000's (published in 2010), members of this article's research team attempted to evaluate the involvement of intermediaries as research partners in three workplace-health interventions [12].The intermediaries were four Ontario-based health and safety associations.The analysis was conducted in the early years of iKT.It explored whether the researchers and the intermediary associations had found the collaboration worthwhile and worth repeating.The associations as partners on the study helped the researchers recruit the workplaces, navigate the workplaces' hierarchies, territorial barriers, and organizational politics, and gain the cooperation of the workers.The study concluded that the relationship was worth repeating, and that future efforts should focus on building rich relationships through "frequent interactions, exchanges of information, and the opportunity to work on mutual projects".It also identified the need for future research to clarify the collaborative model.This 2010 study helped inform the approach taken by the authors in their subsequent collaborative research projects.However, the 2010 study did not fully answer the question "What are the elements of the iKT collaborative partnership of researchers and knowledge users that could predict success?"These are the questions that this study attempts to answer.

Expanding our knowledge of the role of intermediaries
This present study has attempted to respond to the identified shortcomings in the larger iKT literature and to build on the 2010 study with intermediaries.The present study examines the iKT collaborations of researchers and intermediary practice partners (IPP) in three projects: one provincial and two national.These three workplace-based iKT projects were respectively named Sun Safety at Work Canada, the Burden of Occupational Cancer, and Completing the Picture (Table 1 offers a comparison of the three studies).The common objective of the projects was examining ways to prevent or reduce occupational exposure to carcinogens.The studies were multidisciplinary and had researchers from multiple research centres.However, they were either led or co-investigated by researchers from the Occupational Cancer Research Centre (OCRC), a provinciallyfunded research centre based at Cancer Care Ontario.
The three projects varied in duration from 2.5 to four years.They differed in the number of researchers and IPPs (from one organization to 12).They have all recently completed their funding cycles.The IPPs were involved from the inception of the studies through to their evaluation but were not necessarily involved at every stage of the process.
A decision was made by the researchers to focus this study on qualitative feedback from the IPP representatives about their experiences on the three projects.A decision was made not to examine the experience of the researchers, but to focus only on the intermediaries' perspectives (this is a difference from the 2010 study that included both perspectives).Through 22 one-on-one interviews and one twoperson interview, the researchers sought to examine the nature of the iKT activities/collaborations that facilitated the partnership; the enablers and barriers; the iKT approaches that were adopted; and the impact the partnership had on the intermediaries' dissemination of the research findings.
The research questions that framed this study are: • What are the characteristics of successful and unsuccessful iKT partnerships?• What are the outcomes of iKT partnerships?
• What are the factors that researchers might consider at the outset of an iKT partnership so as to facilitate success?

Sun Safety at Work Canada
The aim of the Sun Safety at Work Canada project was to enhance sun safety for outdoor workers.The funder was the Canadian Partnership Against Cancer.The cross-Canada project team that led the project included 5 researchers, 9 intermediary practice partners (IPPs), and a total of 15 staff were engaged over the 2.5 years of the project.(The project has been described in more detail in three manuscripts [13][14][15].) The project's IPPs had different mandates, but their focus was broadly on preventing occupational disease or preventing skin cancer in particular.Although the IPPs were categorized as policy or practice, individuals often spanned multiple roles.In the project, the IPPs helped recruit the workplaces and supervise the project's knowledge brokers.When findings became available, the IPPs integrated those into their dissemination/diffusion activities.The IPPs helped design the evaluation strategy including the baseline assessments of the workplaces (study sites), facilitated data collection, helped with creating resources for the workplaces, helped develop a project website and materials for the website, and set up presentations with stakeholder groups.The researchers conducted the qualitative and quantitative data analysis and wrote the reports and manuscripts.
The team met face-to-face four times.They also met monthly by teleconference for one hour in the first six months and then monthly for 1.5 hours.Subcommittees were formed for tackling certain aspects of the project and they met by teleconference weekly or bi-weekly.

Burden of Occupational Cancer
The aim of the Burden of Occupational Cancer project was to assess the number of cancers, cancer deaths and associated economic costs occurring in Canada due to exposure to workplace carcinogens.This four-year project was funded by the Canadian Cancer Society (CCS).A condition of funding was that a division of CCS should be a partner in the study.CCS's Ontario Division became the IPP on the study.The project has been described in more detail in a report [16], and a manuscript [17].
The research question (What is the burden of occupational cancer?) was developed by researchers based in four provinces.The project was informed by CCS's wish to know more about occupational and environmental carcinogens as part of its prevention and lobbying initiatives.From four to eight practitioners from CCS were identified as the primary IPPs for the project.The grant application included a detailed plan for iKT, including an evaluation of the partnership.
Initially, the researchers and a group from CCS prioritized 44 occupational carcinogens and 24 different kinds of cancer.The IPPs were not involved in choosing the study's methodology or in conducting the data analysis.The IPPs took the lead on disseminating the project's findings and using the findings in policy advocacy.They were kept up to date during the project's four years through regular (bi-monthly) informal breakfasts and lunch meetings, as well at more formal meetings that occurred about four to six months apart.There were also conference-style meetings attended by CCS employees from across Canada.At these conferences (one a year), the researchers presented emerging findings.

Completing the Picture
The aim of the Completing the Picture project was to determine the feasibility of integrating questions about work and work exposures into routine primary healthcare.This three-year project was funded by CCS and the Canadian Institutes for Health Research.Six community Health Centres (CHCs) in Ontario were chosen as the primary health care settings.Two provincial occupational health and safety associations became the IPPs: the Occupational Health Clinics for Ontario Workers, and the Public Services Health and Safety Association.The project's grant proposal's knowledge translation strategy included collaboration with the partners, intensive engagement with the CHCs, and a commitment to evaluate the knowledge utilization of the CHCs.
The pilot study has been described in more detail in two manuscripts [18,19].The research question and the project's design were formulated in consultation with the two IPPs.The grant was written by the researchers and received sign-off and letters of support from the IPPs.The budget covered the expenses that might be incurred by the partners for their involvement with the study.
This project was less explicitly laid out as an iKT partnership, since emphasis was given to the relationship building between the knowledge broker on the research team and the CHCs.However, the two IPPs helped recruit the CHCs to the study.They helped develop study tools.In the early stages of the project, the researchers and IPPs met once per month, and then several times per year in later stages at each of the IPP's individual workplaces.
The IPPs were not involved in the day-to-day implementation at the CHCs, data collection or analysis.At the time the members from the two IPPs were interviewed, the project was not totally complete and only preliminary findings were available.
Hence the IPPs had not initiated any dissemination activities.

Ethics
Ethics approval from the Waterloo Region Community Research Ethics Office was granted for the Sun Safety at Work Canada and the Completing the Picture projects, as well as for this sub-analysis of the three projects' iKT relationships.Ethics approval for the Burden of Occupational Cancer study was obtained from the University of Toronto Research Ethics Board.The individual participants (IPPs) consented to having their data published once all identifying information was removed.

Study design
This study is based upon phenomenology theory and understanding.It is a contained piece of condensed study based on the experiences gained from three individual projects, that seeks to understand one phenomenon, from the perspective of the participants, based upon a conceptual framework [20].We used this approach in order to gain a deeper understanding of the dynamics of the experience of the IPPs during their participation in these collaborative research partnerships.

Conceptual model
The conceptual model for this project's evaluation of iKT was adapted from Gagliardi et al.'s (2016) "Summary of IKT approaches, influencing factors, and outcomes" [2].This model is an outcome of the authors' scoping review on iKT in healthcare.They included all constructs that were identified in their scoping review and noted that some were less strongly represented in the literature than others.Their model depicts any iKT approach as having Enablers, Barriers and Conditions that precede and influence it.Examples of enablers in the model include having capacity or infrastructure, having clear expectations and responsibilities, and having high level recognition and commitment.Examples of barriers in the model include not having time for iKT, having differing timing, and having differing values.The iKT approaches are the activities undertaken to support iKT such as writing evidence briefs, making a web portal, and consultation.The conditions might include the complexity of the health services issue, involvement in different stages of the project, and the current phase of the project.The iKT approach leads to Outcomes, which include relevance of the research, quality of the research, and learning about research.To review the full list of constructs, (see Fig. 2).
Congruent with Gagliardi et al.'s (2016) model, this study identified a wide range of issues concerning enablers, barriers, conditions, iKT approaches, and outcomes of iKT partnerships.In the same way as the Gagliardi et al.'s model reflected a scoping review of iKT in healthcare, we have used some elements of that model, modified other elements, and omitted those which were deemed not relevant for this study (see Fig. 1).

Data collection
Interviews were conducted face-to-face from June 2015 to May 2016.The 21 interviewees were the partners on the three projects -seven from each project.All IPPs who were not researchers were included regardless of their level of involvement in their project.Interviewees were categorized as policy-makers (2), practitioners (17), or both (2).The two in the "both" category had practitioner and policy-maker roles in different organizations.The experience of participants in the study ranged from less than five years to over 30 years in private, not-forprofit, and government organizations.Roles spanned from lower-level professionals up to Vice President and President/CEO levels.
The interviews were conducted wherever the IPPs felt most comfortable; most often in their office.Sometimes interviews were conducted at a café or restaurant.All the interviews were one-on-one with one exception (two participants).All participants agreed to be audio recorded with one exception.Audio recordings were transcribed verbatim.Interviews lasted from 30 minutes to one hour, with a total of almost 13 hours of audio recordings.
An interview guide was developed according to an initial conceptual model.As the study progressed, the interview guide evolved so as to explore emerging themes.The conceptual model has also gone through a number of iterations as we have come to learn more about the characteristics of successful collaborative researcher-practitioner partnerships.Figure 1 reflects this emerging knowledge.The interviews focused on the enablers and barriers, conditions, and outcomes of the iKT approach.They specifically explored

Data analysis
The interviews were listened to multiple times to understand tone and to gain a deeper understanding of meaning.Transcripts were thematically coded in NVivo10 using constructs in the conceptual model as a priori codes.A "framework analysis" method was used to analyze data in order to generate findings that would be more applicable to practice change [21].A matrix was developed that organized the three projects along the horizontal axis and the codes along the vertical axis.Data were added to the matrix in the appropriate cell corresponding to the study and the code.This analysis method is supported by Miles, Huberman and Saldaña [22] as it helps with distilling the data and organizing it for ease of comparison (e.g., some codes were not strongly supported across all three projects, and sometimes new themes emerged).As new themes emerged, new codes were added to the matrix allowing for an inductive analysis.In a secondlevel analysis, our main themes were compared to the variables identified by Gagliardi et al. (2016) in their review of iKT in healthcare [2, Fig. 2].

Results
The following results from this overview of the iKT collaborative partnership across three research projects has been divided into two large categories: 1) Enablers and Barriers, and 2) Outcomes.Seven enablers and barriers for researcher-practitioner partnerships, and seven outcomes from the researcherpractitioner partnerships emerged strongly in this study.

Enablers and barriers
The seven enablers and barriers that emerged included having: adequate capacity; defined project roles; the right partners; an inclusive project leader; mutual respect; good communication; and shared values and priorities.Lacking these was considered a barrier.

Capacity (money and time)
Having funding available to engage in the research projects was very important.The IPPs said that their individual organizations needed the research grant money to buy out their time to dedicate to the projects.

exclusively. This partnership allowed us to do that."
In one project, having funding earmarked for the project enabled the partner organization to keep the project within their mandate.This was especially important since the organization was undergoing significant internal change and there were competing claims on resources and priorities.

"I think [money] helps . . . . I think money drives focus and money drives through dedicated investment and enables resources and accountability."
A few partners said that funding was needed to continue the project's work after the official grant funding ended.However, the majority thought they had the resources they needed to continue their commitment as disseminators and advocates of the research findings.

"[Our organization] will always keep those documents up to date because it's part of the core work that we do. We don't need any grant money to keep that information out there because this is what we do."
Regardless of the money to support partner organizations' activities, a number of the partners mentioned how little time they had to dedicate to working on the projects.Many did this work "off the side of their desks".This ended up affecting how involved they could be while the studies were being conducted.
"I always like to be more involved but it's always a question of workload.So, I like to hear lots about [the] research but [I] understand it's not our primary role."

Project management/role
Many of the interviewees said they experienced some confusion during the project regarding how it was progressing and what their contribution to the project ought to be.In general, partners wanted a clearer long-term vision, more updates, and a better understanding of what would be expected of them.

"Months and months can go by [after the research submission] before the actual [grant] approval so, I think it needs to be reiterated at the beginning of the project: 'Remember this is what we submitted, and this is what we said. Does this still make sense? Can you still play the role? Who is going to be [involved]? Are you still committed?' All of those pieces."
In some cases, the perceived lack of project organization left the partners feeling negative.
"Sometimes the materials would be sent the morning of a meeting or the day before which did not really offer a lot of time to review and to be able to provide useful input.Sometimes that comes across as, 'Okay, we are working on this and we do not really want your input but here it is.'And I know that is likely not the case but that is certainly how it can be perceived." For the most part, the partners contributed in ways that aligned with their expertise or with their role in their respective organizations.None of the interviewees expressed a need to contribute to aspects of the project that were beyond their area of expertise.
"Well, I think it's very nice to partner with organizations who will stick to their scope . . .Although some of the partners found the changes in research direction that took place during the study frustrating, many of them appreciated that the project needed more flexibility as it progressed.
"I thought we would be putting together a work plan, working towards something, but there never really was a formal plan . . . .I've seen the value of being able to use this data in many ways . . . .I've actually really liked the way it's gone and the flexibility that it's provided in terms of its application to many different facets of the work that we do.So, on the one hand we can used it to help advance policy, [but] we can also use it to secure corporate partnership."

Right partners involved
A strong theme was the importance of inviting the right people to be part of the project team.This included people whose mandates allowed them to fully contribute to the project, as well as senior decision-makers with the authority to focus their organization on the research, and support sustainability.There were a few instances where interviewees felt that the wrong person from the organization had been included in the project, and that this had affected the organization's ability to use the knowledge effectively.

"One of the keys to success of these types of partnerships is everyone's mandate and who has the authority to affect change in that mandate or give it credence, right? So you did this work but what was the uptake? What was the outcome?"
In the two projects that did not include government decision-makers as partners, the value of including policy decision-makers as part of the research team was not clear.Some thought it would be better to only involve government as an IPP once results were finalized.Another thought that it would be difficult to engage a policy maker sufficiently without overwhelming them with work."I would say [policy makers] were probably missing but were they needed? . . .They can say so much . . .but the broader priorities of the organization take precedence over what they might say . . . .Also, how do we want to present it to government? . . .We can be engaging with bureaucrats but we can also engage with the elected officials and sometimes there are other means of getting policy changes." The importance of having a continuity of individuals as part of the team was also highlighted.One IPP organization experienced a high level of turnover in the key position that was meant to link the researchers with the practitioners in their organization.During the study, there had been at least four people delegated to fulfil this role.This contributed to reduced continuity in the project from the partners' viewpoint."There's a lot of turnover happening in the NGO sector . . . .Who maintains that relationship, that contact?How do you transfer projects when somebody leaves?If that position is not being replaced immediately, when that person leaves, how are they transferring that project onto somebody else?"

Relationship with the project leaders
Many of the partners had worked with the researchers in the past.In a few cases, the pre-existing relationship helped the organizations make the decision to sign on as a partner.

. . I'm a big fan."
There were a few comments about how the leadership could be improved, such as more clearly defining who the leader was, and better delegation of staff working on the project."Sometimes it felt as though [the project lead] might not have been trusting or as aware or maybe just as regularly informed by [their] staff.So, there were those instances when we felt like, 'Oh, how did you not know that we were doing this?' Or, 'Why are you asking these questions now?' . .

Mutual respect
Many partners acknowledged that mutual respect is important for a successful partnership.

"I would say some of the keys to a good partnership are to be able to communicate and to be able to understand each other's point of view and respecting each person's unique differences but also knowing when to leverage their strengths."
Largely the partners felt respected and that their input had been valued throughout the project.

"I'd say the working committee's been very inclusive in their approach. They've always wanted to listen to people's perspectives. I wouldn't say I've felt any level of detachment at any point in the project."
However, some of the partners in one project felt that their input was not being valued and used as much as it should have been, especially during the initial stages of the project."In the beginning it was a little rough.I won't lie . . . .We were isolated, and they kept forgetting about us . . . .They would have calls . . .and they'd be talking about all this stuff; we'd go, 'When did that happen, who decided that, no one asked us about this!' And they were like, 'Oh yeah, we had a meeting yesterday in our office,' or 'We went for coffee,' or 'We had lunch.'And it was like, 'Oh' . . . .Initially we were so pissed off!Like they didn't ask us our opinion and we have opin-

ions. They decided this and didn't ask us about anything!"
Respect for the other's knowledge and expertise extended to gratitude for being able to benefit from the specialized knowledge and skill that each partner brought to the table.Partners also knew their own specialized skill set and what they were able to bring to the table.

"When I met [the project lead] . . . he gave us an overview of the project, and I expressed interest.
Could we tap into it?Both from the angle of being involved in the project and contributing to it but also being able to utilize the resources? . . .The work being done by the group, that's been a great boon to us because they did all that review stuff that we normally do on our side.It just allows us to be lazy basically!In a really good way . . . .The stuff that's been developed is great stuff.We can tap into that and utilize that." A significant enabler that emerged was the need for mutual respect for the different ways that practitioners go about their day-to-day business in comparison to researchers.This led to some difficulties with alignment of the partners and the researchers, especially at the beginning of the partnerships.For example:

"It is more like oil and water. It can exist in the same bucket but you never fully integrate. "There was definitely a bit of a bump, bumping along you know, when you realized that you don't share a lot of the same vocabulary and you don't share a lot of the same working priorities and all those kind of things"
The partners gave many examples of the differences between the research world and the practice world.For example, those who had a marketing focus wished the research findings would be presented in simplified language to make communication and advocacy easier and more effective.

"In practice, we are only skimming the surface . . . . It is great that there is a lot more depth to the information but what we need is only this [top layer] . . . . The depth probably is meaningful to other researchers or future research but as far as in the practice, we just need the top-level information."
Some mentioned the difference in work speed between researchers and practitioners; practitioners highlighted that research moved slower than practice.
"It was funny because when the project was going on, oftentimes the researchers would say, 'Yeah, yeah, we'll have a number by say, June', and some of our research colleagues here . . .would laugh and say, "There is no way it's going to be done by June, it's going to be done months later than that!"And they were right, every time."Despite these inherent differences, many thought that those from each world could still work together.The partners were used to different ways of working and to different types of expertise.For example, within the Burden of Occupational Cancer study, estimates of the number of cases of mesothelioma from asbestos exposure at work turned out to be smaller than expected, and therefore less compelling.The partners knew, from their experience in policy advocacy, that they needed to strategize and find the best way to communicate the numbers.

Communication
Each project experienced challenges with communication.There was always room for more communication between the researchers and the IPPs.In the Sun Safety at Work project, the partners were spread across the country.A big theme in these interviews was the value of the face-to-face meetings in comparison to the conference calls.Many acknowledged that face-to-face meetings are preferable, but difficult to fund and organize given the geographical distance.The partners noted that conference calls also have their challenges (like missing body language, facial expressions, or side conversations), but given those challenges, they worked well.

"Sometimes it's hard to tell what people are actually feeling about a thing . . . . I could hear [people on the phone] saying something but you kind of get a sense that there's something else going on in the room or there's a facial expression that you can't pick up on because you're not there . . . . I really enjoyed it when we could come together. I always save my really nitty gritty hard questions for when I'm going to see people because I find it a lot easier to chat things out and you're able to read people and see what they really mean. It's hard on the phone. I like meeting in-person a lot better, but it's just been hard to do that too often."
The IPPs on the Burden of Occupational Cancer project did appreciate the efforts that the principal investigators (PIs) made to maintain strong communication (they scheduled regular working breakfasts and lunches together).However, there were a few miscommunications between the IPPs and the researchers.There was some confusion regarding who was supposed to be leading the planning of conferences.Also, the IPPs wished they had received more regular updates on the process and a clearer understanding of when the research findings could be shared publicly.They were anxious to use the burden estimates for occupational asbestos exposure in their advocacy work.
"There was a time early on [when we said], 'When can we use these [asbestos] numbers because we would like to start incorporating them?' and the PI was like, 'Well they're still not final . . .you have to wait.' From a research perspective, I recognize that you need to have these numbers complete and validated.[They were] more like, 'We're showing you these [other] numbers.Here's the diesel.Here's the wood dust.You can start using these in your work.'We didn't have the appetite or the resource really to take on those new areas."Some of the IPPs on the Completing the Picture project felt a bit disconnected from the project.However, others felt that the meetings had been adequate since they did not have the time to meet more frequently.
"By virtue of the partnership I always felt that we'd be kept on track.So there wasn't a point where I was thinking, 'Oh, we should be meeting more'.Probably because we were juggling so many different other things.I don't think it had occurred to me . . . .If this] had been one of a lot fewer items that we were working with maybe I would be thinking, 'Oh we should have met more."'

Shared values & priorities
Finally, a major enabler of the relationship between researchers and partners was whether they shared a vision of the importance of the project.The projects were important to the IPPs because the projects' mandates fell within their organizations' priorities, or because the partners' stakeholders (workers or patients) wanted the research to be done, or because their stakeholders valued evidence-based practice and investment in research.Some individual interviewees stated that their project's focus went beyond their organizational mandate and was actually a personal passion.
"This particular project, it was one that we were absolutely proud to be a part of and felt that it was just giving us more capacity.In fact, it was fulfilling our mandate in a very thoughtful way with very skilled operators.It allowed us to hit areas we would have liked to be hitting but just hadn't gotten to."However, in one case, a partner organization was undergoing structural change and organizational priorities were changing.This became a barrier to the relationship.These partners did not know whether the project work would continue because they did not know whether their mandate would continue to include occupational cancers.

"It's a little unclear as to where things will go in the future because we have undergone such massive restructuring here with new leadership. We've had this win [with policy change] but it's hard to say whether our priorities will stay focused on that or not because of all the changes happening. So, I don't think it's because of anything related to [the researchers]. I think it's just an independent factor that's happening here that might influence moving forward." 4.2. Outcomes
There were tangible outcomes of the iKT partnerships that were mentioned by the IPPs.The outcomes that were highlighted included: 1) improved relevance and quality of the research findings; 2) enhanced learning about the others' "world"; 3) enhanced access to other information and contacts; 4) improved use of research in practice and policy; 5) dissemination and diffusion of the research findings; 6) development of trust and goodwill; and 7) continued collaboration on new projects.

Relevance/quality of research findings
The Sun Safety at Work project produced many communication tools outside of traditional research papers, such as workplace-focused sun safety resources, a project website, and presentations for workplaces and industry.The IPPs were involved in developing these and were proud of them."It's incredible how much work is being pumped out in the time that I've been on the project.That always amazes me how much a small group is capable of . . .[I'm] looking forward to the final result of the website.It will be great to see something from the very beginning idea through to fruition." The Burden of Occupational Cancer could have been overwhelmed by the number of exposures and the number of cancers to include in the research, but the IPPs were able to limit both so that the results of the study would be relevant to their membership as well as to policy-makers.
"There was a big list of carcinogens to take a look at so we had to help reduce that list to a more workable list and one that would make sense in terms of our prevention exercises, initiatives, and public policy or advocacy initiatives"

Learning about the other's "world"
The partners appreciated learning about the research content area and the difficulties of achieving workplace change.The IPPs also reported that they had expanded their knowledge about occupational health and safety, occupational cancer, and the field of knowledge translation.

"I learned more about occupational cancers and the types of carcinogens that are frequent within the workplace as well as the number of people exposed to them and the types of cancers they can lead to and the whole process of how we figure that out... I have learned a lot about occupational cancers and it's really opened up my eyes to the research that's happening in that area."
Although the majority of the IPPs had worked with researchers before (or had a research background themselves), many said they still learned something new from partnering on their research project.The process of partnering taught the IPPs about researchers' work style, as well as the limitations and constraints that researchers face.

"Actually, I think [partnering on this research project] improved our relationship because we understood sort of what was involved in the process, how it can be cumbersome to get access to
data, sometimes the data was not enough -those sorts of things.I think it really, probably created greater respect for the work that researchers do." They also felt the learning had been a two-way street, and that the researchers had also learned what the partners needed.
"We both learned from each other, I think there was a lot of back and forth, you know, a lot of give and take, understanding what we need, what they need . . . .I think [the researchers] probably learned how simplified we may need things to be! So, we might have this great, very detailed, comprehensive backgrounder or summary and then we would take it to three facts [laughs], and have [the bigger document] in the background."

Access to other information/contacts
The Sun Safety at Work project was the only project that brought together partners from organizations across Canada.These partners were thankful for the opportunity to network with and learn from each other.They planned to leverage those contacts again in the future.Further, this networking allowed partners in British Columbia and Nova Scotia to build capacity around occupational sun safety.
"Obviously we loved the whole aspect of having a Canada-wide initiative.But [we] also realized if we're going to do something provincially we want to try and set up something here where we can partner with people."

Use of research in practice/policy
It is often difficult to directly link research results to a policy initiative [23][24][25] since policy and advocacy work happens within a larger system with multifaceted and interacting forces.However, the partners felt that the projects had played a positive role in the ultimate outcome of certain advocacy and policy work.An example from each project follows.
One of the project partners of the Sun Safety at Work project worked for WorkSafeBC, the British Columbia provincial governmental body responsible for both the implementation and enforcement of occupational health and safety legislation.Through this partner's involvement in the project, Work-SafeBC integrated sun safety into their occupational disease strategy for 2016-2018.
The findings from the Completing the Picture indicated significant challenges with incorporating work exposure screening into the daily practice of primary healthcare.One of the project's partners, the Occupational Health Clinics for Ontario Workers, has been leading a provincial Ministry of Labour initiative on preventing occupational disease.The initiative's Occupational Disease Action Plan has a goal to: "Develop a strategy to embed 'Occupation' and potentially other links to work into [patients'] Electronic Medical Record (EMR)" [26].This goal may be shelved or adapted at least partially as a result of the project's findings.
Thirdly, there is also some evidence that the Burden of Occupational Cancer project's estimates have been useful in policy change.Estimates of the burden of mesothelioma and lung cancer resulting from occupational asbestos exposure were leveraged by the project's partner and the media to support a campaign for a national ban on the use of asbestos.In December 2015, the Canadian government announced that it would ban the sale of asbestos by 2018 (Grant, 2016).As one of the partners said: "We did have some success.The asbestos numbers have helped to bring about policy change at the national level . . . .I would say having that big win at the national level in terms of asbestos was a big accomplishment and the Burden grant helped to push that forward.I mean there was a lot of momentum.I wouldn't say it was, you know, solely because of that, but I think it helped push the evidence forward to the federal government that change needed to happen."

Dissemination/diffusion of research
At the time of the interviews, the Sun Safety at Work Canada and the Burden of Occupational Cancer had produced research results.Partners from both of these projects reported that their organizations had disseminated the research results during the course of the study.As part of the required reporting for its funder, the Sun Safety at Work Canada project team tracked its stakeholder engagement activities.Table 2 shows a breakdown of these activities.
Dissemination of the results from the Burden of Occupational Cancer project focused on policy change that would be led by the partner/funder organization.As mentioned, the findings were used by the CCS to advocate for a ban on asbestos use in Canada.They also used the radon estimates to advocate for radon testing.Employees from other regions of CCS (i.e., other provinces) were flown to Toronto twice for meetings on the preliminary Burden study results.Additionally, four conferences were organized to spread the emerging findings from the study to relevant stakeholders (health and safety advocates, industry, organized labour, practitioners, and other researchers).Findings have been presented at numerous conferences, have been discussed at many stakeholder meetings, and have been reported in the media.
The project partners on these two projects reported that they continued to disseminate findings after the project's funding ended, although it has been difficult to maintain good metrics on these dissemination/diffusion activities.

Development of trust and goodwill
Almost all the partners directly said that they had a positive experience being part of the research project and that their relationships with the researchers had developed and grown."It did feel like a partnership.I was always happy to say this is who we work with and this is what we do and how we're doing it, and we're funding part of this and it's great.So I think it was a positive relationship experience, and relationship building experience." However, not all the relationships had the same positive level of progression.In the Sun Safety at Work Canada project, the researchers were mostly based in Ontario.The IPPs felt that the researchers had closer relationships with the IPPs based in Ontario than with IPPs located in the other provinces.Due to the geographical distance, fewer in-person meetings, and less opportunities to work together on all of the different aspects of the project, the IPPs in British Columbia, Nova Scotia and Alberta found the relationship with researchers more difficult to develop.
" [Partnering] helped develop the relationships.It deepened other relationships that pre-existed but it was really the local ones where we were seeing each other . . . the relationships were growing because of the project but also because we were seeing each other.So, the folks out west and the folks out east were names and they were voices but I don't have a sense that I really have a rela-

Continued collaboration on new projects
The IPPs agreed that, if the chance arose, they would partner again with the researchers.In some cases, it was clear that the relationship between the two parties (researchers and IPPs) was a driving factor in this openness to continue to collaborate."There are some established relationships and friendships that have emerged that are really going to be longstanding.So [there's] the opportunity to continue to move this work forward and to have people to go to with questions or to mobilize people across the country.I think there is a bit of a foundation that's been started . . . .I don't think [these relationships are] going to go away.And if there are opportunities for additional work, collaborative work, I hope we'll be able to rally the troops again." A few said that future partnerships would depend on what the research project was and what their organization's priorities were and whether the two aligned.These comments were likely influenced by the fact that their organizations were undergoing significant change.As these IPPs said, it was unclear what their organization's priorities would be in the future."It's going to take that alignment of sort of [our] priorities and the opportunity that would [be] presented.So at the moment, we're undergoing a massive amount of change.Our priorities are changing and in flux."

Discussion
From the perspective of researchers seeking partners for a research project, this study aimed to understand the elements of the iKT collaborative partnership of researchers and knowledge users (our IPPs) that could predict success.The study considered indicators of a successful partnership as both an improved spread or use of knowledge in practice or for policy change, and as positive outcomes from the partnership such as trust and lessons learned.From these results, the researchers now propose four suggestions for how iKT partnerships should be structured to set them on the path for success.

Negotiating aspects of engagement
The Canadian Institutes for Health Research says that ideally, a collaborative iKT research study, should include the knowledge users in every aspect of the research process: 1) research question generation; 2) selection of methodology; 3) the intervention and implementation; 4) data collection; 5) data analysis and interpretation; 6) writing manuscripts; and 7) dissemination/diffusion [1].However, the three projects in this study did not include the partners in all stages of the research process.Boundaries were set at the outset by the researchers, and these boundaries were never challenged by the IPPs.Overall, the IPPs were happy with the scope and level of their involvement.Although perhaps not an explicit reason for setting these boundaries at the projects' outsets, it may have been that the IPPs believed that they lacked the requisite expertise to engage in all aspects of the research process.For example, the skills possessed by epidemiologists that enable them to select and adapt a methodology for estimating the social and economic burden of occupational cancer, and then carry out the analysis, is not a skill set that practitioners necessarily have, nor is it one that is easily taught.
Although full involvement in these very technical and specialized decisions might not be feasible, a level of involvement in the choice of methodology and data analysis is still possible.For example, formative qualitative findings could be shared with practitioners with relative ease, and the practitioners' contribution to the interpretation of the meaning of these results would be valuable.However, what we heard from the IPPs on all three projects was that these tasks were outside their scope of practice both personally and in their organizational role.Partners thought it would be beneficial to learn these skills, but it was not feasible given all their other priorities outside of the project.Thus, in these three iKT projects, there was a balance between inclusion and practicality.The partners were welcome to be a part of the full research process, but it was understood that sometimes they would select activities where they thought their expertise would be most valuable.For example, the partners were more willing to help with identifying major elements in existing sun-safety materials, writing, editing, or contributing to plain-language communications, engaging their network to recruit workplaces and then later disseminate findings, and helping to give presentations to workplaces on occupational health and safety or occupational disease issues.

Recommendation: Researchers and knowledge users should spend time at the beginning of a project partnership outlining their expectations of their roles and involvement.
There should be a discussion about making contributions in aspects that are outside of their realms of expertise, and the benefits that such an involvement could bring (i.e. a chance to build capacity and provide input on the results).

Investing time in the project
One of the biggest barriers that the IPPs encountered regarding their full participation within the research process was time.Research is the primary focus for the researchers, but it is not for the knowledge users.All of the IPPs mentioned they felt they were restrained in their ability to fully participate in their project (or hire someone who could).Despite this, many were still able to participate regularly on their projects.For others, the timing of the project was unfortunately not auspicious.In these cases, the IPPs were in organizations that were undergoing disruptive internal change that precluded them focusing fully on the study.
Recommendation: When considering intermediary organizations to engage as partners, consider these questions: Is this the right time for the organization to invest their time and resources in an iKT partnership?Do they have the time and resources available?

Role demands
Higher-level decision makers generally have greater demands on their time, and therefore might be less able to dedicate adequate time to an external research project.Thus, available time and resources might affect which person the organization delegates as the key liaison with the project.However, it is important to consider the role and decision-making power of the individual chosen from the organization and whether their responsibilities align with the role that the project needs them to fill.If the goal of including a specific organization is to have them make significant changes within their organization or to advocate for change externally, then it is important to have a higher-level decision-maker on the team.However, if the purpose of engaging a particular organization is to harness a skill or technical ability, then a different type of employee should be targeted as the representative partner.
Recommendation: Consider the role the partner organization is to play on the project, and approach someone within the organization with an appropriate mandate to be the partner.

Partners who believe in the project
One of the research projects was attempting to address a problem that some of the IPPs thought was nearly impossible to address.The project's goal-to find a way to systematically collect information on workplace exposures-was meant to help find a solution to a problem that the partner organization(s) had grappled with in their work.Although this research project was extremely relevant to the intermediary organization(s), some of the partners thought the study's goal had been tried before and failed, and hence was likely to fail again.Although they thought the goal was desirable, they struggled to buy into the project because they did not believe that it could achieve results.They were doubtful that the problem would or could ever be solved.Thus, their commitment to the research process was conflicted.
Recommendation: When considering which organizations to engage as partners, consider these questions: Even if the partners are willing to engage in the project, do they have the commitment to the project?By this we mean, how dedicated will the partners be to the project?How involved do they want to be? Do they think their investment of time and resources will reap proportional benefits?Do they think the goals of the project are achievable?

Limitations
This evaluation of the three researcher-IPPs collaborative partnerships has limitations.Only the perspective of the IPPs is represented and not that of the researchers.The researchers wanted to learn more about iKT research partnerships with the hope of improving their approach to project partnerships in the future and contributing to the literature on iKT.The researchers wanted to interview the IPPs about their experience in the partnership-what worked well, and their suggestions for improvement.However, when this idea was first proposed at the initiation of one of the studies, the newly-recruited IPPs were reluctant.Although it was eventually decided amongst the researchers and the IPPs that this partnership evaluation and the interviews would go forward, some of the initial hesitancy was still present.
Some of the IPPs' reluctance to have researchers interview them may be because it could challenge the equality in the partnership between the researchers and IPPs.Equality is an important part of iKT partnerships.While the researchers followed the necessary ethical procedures and ensured that each of the interviewees' views would be kept confidential, the proposal to interview the IPPs tested the established trust between the researchers and the IPPs.Although the researchers feel confident that this trust was never broken and has actually grown through the process of partnering, this initial concern is important to mention.Social desirability bias was also a concern in this study.The researchers wondered if the interviewees might feel obligated to respond more positively than they actually felt.All but two of the interviews were completed by EH, who reports to DMK. (DMK completed the remaining two).DMK was a main research partner on all three of the projects and has professional relationships with the partners.Although the participants interviewed by EH were assured that neither the recordings nor the transcripts would be shared with DMK, the researchers suspect that interview responses were skewed to the positive.One of the research authors on this manuscript, based upon her experience working on one of the projects, was surprised that the responses were not more negative.

Conclusion
IKT partnerships are considered by many to be an important and effective method for translating research to practice.However, few have evaluated such partnerships to understand what factors predict success or failure.This study looked at three iKT partnerships to understand what worked and did not work.This study builds upon a summary of iKT approaches, influencing factors, and outcomes outlined by Gagliardi et al. (2016), and on a previous study that the authors conducted on a researcher-intermediary partnership.The study offers recommendations for those who might be considering taking an iKT approach to their research.In conclusion, the study found that iKT partnerships can achieve notable knowledge use and dissemination outcomes, can build valuable relationships between the researchers and practitioners, and these strong relationships can be harnessed for future research collaborations.

Fig. 1 .
Fig. 1.An analysis of the preconditions, approaches, enables and barriers that lead to successful outcomes for Integrated Knowledge Transfer (Adapted from Gagliardi et al., 2016).project leadership, trust, relationships, money, role definition, what worked well and not so well in the partnership, and what the partners learned from the experience.

Table 1
Comparison of three workplace-based integrated knowledge transfer research projects . Having people perform different roles in a partnership is the way it should be and it's nice to be able to collaborate and share but having clearly defined, 'This is what we're going to do and this is what we would like to do,' is I think the best-case scenario.""I felt like I was involved enough.Not that I didn't enjoy it, but again my role is very functional.So in terms of being more involved in any of the research, it wouldn't necessarily fit in to my role."

"
Going into [the project] we pretty much had established the relationship in my view.I was very familiar with your work, with the type of people that you were, your passion for it . . . .Of course we're going to at every opportunity partner with you guys."Partners spoke about the project leadership's openness to everyone's ideas and input."[The project lead] is very open-minded, very friendly, very easy to engage with.S/he's responsive and respectful.That's important in providing the kind of leadership s/he has to provide.At the same time, s/he's not a doormat . . . .S/he has [their] beliefs in how things need to move forward and has the evidence to back it up.So you respect what s/he's coming forward with.And if s/he doesn't agree with a position s/he's comfortable saying that and why.So it's been a nice balance."Partners also thought that the project leaders were credible, based on their previous research experience or the organizations from which they came."[They're] not there working for industry.[They're] independent, really.And where do you find that?You don't . . . .You can't find that anywhere.So yeah, we would latch on . .

Table 2
Summary of stakeholder engagement activities for the Sun Safety at Work Canada project up to September 30, 2016