You are viewing a javascript disabled version of the site. Please enable Javascript for this site to function properly.
Go to headerGo to navigationGo to searchGo to contentsGo to footer
In content section. Select this link to jump to navigation

A Systematic Review of Outcome Reporting, Definition and Measurement Heterogeneity in Non-Muscle Invasive Bladder Cancer Effectiveness Trials of Adjuvant, Prophylactic Treatment After Transurethral Resection




Heterogenous outcome reporting in non-muscle-invasive bladder cancer (NMIBC) effectiveness trials of adjuvant intervention after transurethral resection (TURBT) has been noted in systematic reviews (SRs). This hinders comparing results across trials, combining them in meta-analyses, and evidence-based decision-making for patients and clinicians.


We aimed to systematically review the extent of reporting and definition heterogeneity.


We included randomized controlled trials (RCTs) identified from SRs comparing adjuvant treatments after TURBT or TURBT alone in patients with NMIBC (with or without carcinoma in situ) published between 2000–2020. Abstracts and full texts were screened independently by two reviewers. Data were extracted by one reviewer and checked by another.


We screened 807 abstracts; from 15 SRs, 57 RCTs were included. Verbatim outcome names were coded to standard outcome names and organised using the Williamson and Clarke taxonomy. Recurrence (98%), progression (74%), treatment response (in CIS studies) (40%), and adverse events (77%) were frequently reported across studies. However, overall (33%) and cancer-specific (33%) survival, treatment completion (17%) and treatment change (37%) were less often reported. Quality of Life (3%) and economic outcomes (2%) were rarely reported. Heterogeneity was evident throughout, particularly in the definitions of progression and recurrence, and how CIS patients were handled in the analysis of studies with predominantly papillary patients, highlighting further issues with the definition of recurrence and progression vs treatment response for CIS patients. Data reporting was also inconsistent, with some trials reporting event rates at various time-points and others reporting time-to-event with or without Hazard Ratios. Adverse events were inconsistently reported. QoL data was absent in most trials.


Heterogenous outcome reporting is evident in NMIBC effectiveness trials. This has profound implications for meta-analyses, SRs and evidence-based treatment decisions. A core outcome set is required to reduce heterogeneity.


This systematic review found inconsistencies in outcome definitions and reporting, pointing out the urgent need for a core outcome set to help improve evidence-based treatment decisions.



Description of the condition

Bladder cancer is the 6th commonest male, and 17th commonest female cancer globally, with the highest incidence rates being observed in Europe and North America [1]. The disease is categorised into two broad stage groupings, non-muscle invasive (NMIBC) and muscle-invasive (MIBC) bladder cancer. Most cases (75– 85%) present as NMIBC and these patients typically have a higher long-term survival and a lower cancer specific mortality compared to those with MIBC [2].

NMIBC is defined as tumour(s) confined to the mucosa or invading the lamina propria [3]. Using the TNM staging system, they are classified as Ta-T1 or Tis (or Cis) N0 M0 [4]. NMIBC tumours may be graded using the WHO 1973 or WHO 2004 grading systems – both indicating worse prognosis with increasing grade. Most patients diagnosed with NMIBC is initially treated conservatively (sparing the bladder) with curative intent by transurethral resection of bladder tumour (TURBT). NMIBC is seen as a chronic disease requiring frequent follow-up and repeated TURBTs, making it the most expensive of all cancers to treat from diagnosis to death [5– 8] with additional productivity losses and informal care costs [9]. Cumulative costs of care are especially high in intermediate- and high-risk NMIBC due to higher risk of progression to MIBC requiring definitive treatment [7].

Given the high recurrence rates and the risk of progression to MIBC, NMIBC treatment usually involves adjuvant intravesical instillations with che-motherapy or immunotherapy. The timing, treatment duration, and choice of agent for intravesical therapy is guided by a risk categorisation system which is based upon clinical and pathological factors [3]. For instance, evidence from high quality systematic reviews and meta-analyses shows that a single immediate post-operative instillation of chemotherapy (IPOIC) is well tolerated and clinically effective in reducing recurrences in low risk patients [10– 12]. The European Association of Urology (EAU)[3] and the National Institute for Clinical and Healthcare Excellence (NICE) [13] both recommend that eligible patients receive IPOIC. It is considered cost effective for the NHS [13]. Intermediate risk patients may also be given repeated chemotherapy instillations, but their timing and frequency remains undefined [14]. It is recommended that high risk patients are treated with intravesical bacillus Calmette-Guerin (BCG) immunotherapy or be considered for immediate cystectomy [3]. Five-year recurrence and progression rates for patients with stage Ta-T1 bladder cancer treated with 1 to 3 years maintenance BCG are 28– 51% and 7– 20% , respectively [15].

Why it is important to do this review

Inconsistent outcome reporting (different outcomes in different trials) and variability in outcome reporting (same outcomes reported, but different definitions used) become acutely evident when many bladder cancer trials are included in systematic reviews of intervention effectiveness [16– 18]. Outcome reporting heterogeneity has been highlighted as a concern within evidence-based medicine generally, [19– 22] and has been emphasised as an area for improvement in NMIBC trials by the International Bladder Cancer Group [23]. Heterogeneous outcome reporting and the potential for selective outcome reporting bias in NMIBC trials hinder comparing and contrasting the results of individual trials as well as the publication of unbiased systematic reviews and meta-analyses of the evidence base. As a consequence, making evidence-based recommendations in clinical practice guidelines, translating them into health care policy, and decision-making by clinicians and patients are all hampered.

Developing core outcome set (COS) a solution to reduce outcome heterogeneity, selective outcome reporting bias, and helps to ensure that all trials contribute useable information to the evidence base. A COS is an agreed standardised collection of outcomes which should be measured and reported, as a minimum, in all trials for a specific clinical area [22]. Our group has registered a bladder cancer COS development project (B-COS) with the Core Outcome Measures for Effectiveness Trials initiative COS register (, with the intent to create separate COS for three broad categories of disease: NMIBC, MIBC, and metastatic BC. Within each COS we define the scope with regards to the applicable populations and treatments. After defining the scope of a COS, the next step is to identify existing knowledge regarding outcomes. To meet this requirement, we have aimed to systematically review the outcomes reported in NMIBC effectiveness trials. Our systematic review protocol was registered with PROSPERO ( The reviews for the other parts of the project will be reported separately as will the subsequent phases of the COS development projects, involving qualitative interview studies with patients, and consensus studies with key stakeholders such as patients and healthcare professionals using Delphi methods to come to consensus on the core outcomes to be measured in future bladder cancer effectiveness trials and audits.


Aims and objectives

The aim was to systematically review outcomes reported in NMIBC effectiveness trials of adjuvant, prophylactic treatment after TURBT.

The objectives were to systematically review:

  • 1. Outcomes reported

  • 2. Outcome definitions (including time points)

  • 3. Outcome assessment methods

Eligibility criteria

Types of studies

We included phase III randomised controlled trials (RCTs) comparing different adjuvant instillation treatments after TURBT or trials with TURBT alone as a control arm. We limited to RCTs included in systematic reviews of intervention effectiveness as a pragmatic and efficient way to identify studies and overview potentially important outcomes. This is a strategy that has been used in published systematic reviews of outcome reporting heterogeneity where the aim is to overview outcome reporting heterogeneity rather than to find every outcome previously reported [22, 24]. All pre phase III trials and all non-randomised designs were excluded. Studies reported only as abstracts were excluded a priori because it was unlikely that all outcomes would be reported in the abstract, and that they would also not provide enough information on the definition and measurement of outcomes reported.

Types of participants

We included studies with adult (≥18 years) males and females with histologically confirmed urothelial NMIBC, stage Ta or T1 N0 M0, with or without carcinoma in situ (CIS), and all tumour grades (using any grading system). Studies including paediatric patients and patients with MIBC, clinical N + or M + were excluded unless outcomes were separately reported and defined for NMIBC patients.

Types of interventions and comparators

We included RCTs comparing any type of intravesical adjuvant prophylactic treatments after TURBT and RCTs comparing intravesical treatment after TURBT versus TURBT alone. Studies of oral vitamins or mineral supplements were excluded.

Types of outcomes

We report on all outcomes related to clinical effectiveness including, for example, outcomes related to recurrence, progression, survival and cause of death, local and systemic adverse events and quality of life/patient reported outcomes. Outcome definitions, timepoints, and assessment methods are also reported.

We do not report any estimates of treatment effect for any individual trials and there was no attempt to synthesise aggregated quantitative data.

Literature search

The literature search was undertaken by an exper-ienced information specialist (CY) using the search criteria specified in Appendix 1. Medline, Embase and Cochrane Database of Systematic Reviews (CDSR) were searched for relevant systematic re-views. We also hand-searched the reference sections of relevant international clinical practice guidelines. We restricted to systematic reviews and RCTs published after 2000 to reflect outcomes reported in the current clinical practice. We excluded non-English studies as a pragmatic consideration due to resource restrictions.

An update search was done on 15th January 2020.

Data collection and analysis

Selection of studies

Following de-duplication, at least two review authors (DC, SM, SS, IO, EV, RC) independently screened the titles and abstracts of identified systematic reviews for eligibility. The full texts of all potentially eligible publications were retrieved and screened independently by two review authors (DC, SM, SS, IO, EV, RC) using a standardised form, linking together multiple records of the same study in the process. Any disagreements were resolved by discussion or by consulting a senior review author (RS). Once the list of systematic reviews meeting the inclusion criteria were finalised, a second screening process was initiated whereby the studies included in the systematic reviews were screened against our inclusion criteria. Where lists of studies excluded from the systematic reviews were available, we also screened these in case the studies had been excluded for not reporting on outcomes of interest. In such instances the trial may still have met inclusion criteria for our review. The study selection process is described in the PRISMA flow diagram (Fig. 1) [25].


Preferred Reporting Items for Systematic Reviews (PRISMA) diagram of studies. SR, systematic review; RCT, randomized controlled trial.

Preferred Reporting Items for Systematic Reviews (PRISMA) diagram of studies. SR, systematic review; RCT, randomized controlled trial.

Data extraction and management

A standardised data extraction form was developed and piloted. One review author extracted data and a second review author checked data extractions for accuracy (DC, SM, SS, IO, EV, RC). Any disagreements were resolved by discussion or by consulting a third review author.

Data that were extracted included: the study design; countries and institutions where the data were collected; dates defining start and end of patient recruitment and follow-up; how intervention comparator groups were formed; participant demographic and clinical characteristics; eligibility criteria for participants; the numbers of participants who were included in the study, assigned to each intervention comparator group; description of interventions; study funding sources; and ethical approval. All primary and secondary effectiveness outcomes reported, their definitions, and any outcome measurement instruments used were extracted verbatim.

Assessment of risk of bias in included studies

Risk of bias assessment is not necessary for systematic reviews undertaken for COS development. Some outcomes may be at risk of detection bias depending on whether they are relatively subjective or objective. Although these aspects were extracted under the ‘definition’ or ‘measurement’ fields in the data extraction form, this is out of the scope of this phase of our project. They will be investigated in a subsequent phase whereby we will assess the psychometric properties of the various outcome measurements and seek consensus on the most appropriate and feasible definitions and measurements [26, 27].

Data synthesis

Verbatim outcome names were recoded to common names. This was done by categorising outcomes referring to the same underlying constructs under a common term. For example, “survival rates”, “overall survival”, “number of deaths at median follow up” and “mortality rate” all refer to the concept of ‘overall survival’ and were coded as such. The outcome and domain coding process was inductive and iterative. Coded outcomes were further grouped in broader domains using the standardised Williamson and Clarke Taxonomy (W/C Taxonomy) [28].


Characteristics of the included studies

Our initial search for relevant systematic reviews yielded 807 abstracts, of which 639 remained after removing duplicates. In total, 100 full-text SRs were assessed and 19 SRs, including 14 meta-analyses, were included. Four SRs included only previously identified RCTs and these SRs were not utilised further (Supplemental Table 1). From 15 SRs published between years 2010– 2018, 106 full-texts of RCTs were screened and 57 eligible RCTs were finally included (see PRISMA flow diagram, Fig. 1).

An overview of the included studies’ populations, stage and grade, instillation treatments and number of outcome domains reported is shown in Table 1. Overall, 32 studies included patients with papillary only tumors, while 25 studies included a mixed population of patients with CIS with/without papillary tumors. There were 11 “single-instillation” trials, 12 “single instillation followed by induction course” trials, 27 “maintenance instillation” trials and 7 trials comparing instillations with different schedules.

Table 1

Baseline characteristics of the study population, instillation treatments and outcome domains

Bladder cancer morphologyStudiesPOPULATIONINSTILLATION TREATMENT (S-single; I- induction; M-maintainance)Number of outcomes domains (n/10)
pTapT1CIS as authors have reported
low gradehigh gradelow gradehigh gradeprimary cissecondary cisconcomitant cis
CIS+/- PAPILLARYLamm 2000xxxxxxM7
Palou 2001G3G3xxM6
Au 2001xxxxxxI2
Sekine 2001xxxxxxxI5
Martinez-Pinneiro 2002G2G3G1G2G3xxI7
Di Stasi 2003xxxxM7
Kaasinen 2003xxxxxxxM5
Martinez-Pinneiro 2005G3G3xxI4
de Reijke 2005xxxxxxxM5
Di Stasi 2006G2G3xM6
Gårdmark 2007xxxxxxM4
Cai 2008G2G3G2xM4
Neple 2010xxxxxxM2
Porena 2010G3G3xxM3
Koga 2010xxxxM5
Gülpınar 2012xxxxxxI4
Järvinen 2012xxxM5
Sengiku 2013xxxxxxxI3
Inamoto 2013xxxxxxxI4
Rentsch 2014xxxxxI5
Hemdan 2014G2G3xM4
Martinez-Pineiro 2015G1G2 + cisG3G1G2 + cisG3xI vs M5
Solsona 2015G1 + cisG2G3G1G2G3xxI5
Arends 2016xxxxxxxM4
Nakai 2016xxxxxxI vs M5
PAPILLARYKaasinen 2000G1G2G1G2M2
Bilen 2000xI4
Van der Meijden 2001xxxxM4
Nomata 2002G1G2G1G2M2
Okamura 2002xxxxS3
Rajala 2002xxxxS1
Kuroda 2004G1G2G1G2M4
Koga 2004xxxxI vs M2
Mitsumori 2004xxxxI2
Cheng 2005xxxxM5
Vijjan 2006xxxxI5
Hinotsu 2006xxxxI vs M3
Barghi 2006G1G2G1S3
Ojea 2007G2G1G2M4
El-Ghobashy 2007G1G2G1G2S4
Agrawal 2007xxxxM2
Friedrich 2007xxxxM2
Hendricksen 2008xxxxM3
Berrum-Svennung 2008G1G2G1G2S3
Isbarn 2008xxxxI vs M2
Böhle 2009xxxxS4
Gudjonsson 2009G1G2G1G2S1
Järvinen 2009xxxxM5
Seretta 2010G1G2G1G2I vs M3
Sylvester 2010xxxxM6
De Nunzio 2011G1G2S4
Di Stasi 2011xxxxS5
Hinotsu 2011xxxxI vs M4
Oddens 2013xxxxM6
Huang 2015xxxxM4
Onishi 2017G1G2G1G2S3
Bijalwan 2017xxxxS3

In all studies, patients were followed up at regular intervals in the same and largely accepted manner: urinary cytology, cystoscopy and if necessary, by taking biopsies from the urinary bladder [3].

Heterogeneity in outcome reporting, detection, and definitions

The outcomes were organised into the 10 domains in the W/C taxonomy [27]: “recurrence”, “progression”, “treatment response” (for CIS), “cancer-specific survival”, “overall survival”, “adverse events”, “completion/adherence”, “treatment failure/change of treatment”, “quality of life” and “health economics” (Table 2).

Table 2

Outcome domains reported in 57 randomised controlled trials classified using the Williamson and Clarke Taxonomy

RecurrenceProgressionTreatment response (for cis)Overall survivalCancer-specific survivalAdverse eventsCompletion/adherenceTreatment failure/change of treatment reported (RC,RT)Quality of lifeHealth Economics
CIS+/- PAPILLARYLamm 2000xxxxxxx
Palou 2001xxxxxx
Au 2001xx
Sekine 2001xxxxx
Martinez-Pinneiro 2002xxxxxxx
Di Stasi 2003xxxxxxx
Kaasinen 2003xxxxx
Martinez-Pinneiro 2005xxxx
de Reijke 2005xxxxx
di Stasi 2006xxxxxx
Gårdmark 2007xxxx
Cai 2008xxxx
Neple 2010xx
Porena 2010xxx
Gülınar 2012xxxx
Järvinen 2012xxxxx
Sengiku 2013xxx
Inamoto 2013xxxx
Rentsch 2014xxxxx
Hemdan 2014xxxx
Martinez-Pineiro 2015xxxxx
Solsona 2015xxxxx
Arends 2016xxxx
Nakai 2016xxxxx
PAPILLARYKaasinen 2000xxNAx
Bilen 2000xxNAxx
Van der Meijden 2001xxNAxx
Nomata 2002xNAx
Okamura 2002xxNAx
Rajala 2002xNA
Kuroda 2004xNAxxx
Koga 2004xNAx
Mitsumori 2004xNAx
Cheng 2005xxNAxxx
Vijjan 2006xxNAxxx
Hinotsu 2006xxNAx
Barghi 2006xxNAx
Ojea 2007xxNAxx
El-Ghobashy 2007xxNAx
Agrawal 2007xNAx
Friedrich 2007xNAx
Hendricksen 2008xxNAx
Berrum-Svennung 2008xxNAx
Isbarn 2008xNAx
Böhle 2009xxNAxx
Gudjonsson 2009xNA
Järvinen 2009xxNAxxx
Seretta 2010xxNAx
Sylvester 2010xxNAxxxx
De Nunzio 2011xxNAxx
Di Stasi 2011xxNAxxx
Hinotsu 2011xxNAxx
Oddens 2013xxNAxxxx
Huang 2015xNAxxx
Onishi 2017xxNAx
Bijalwan 2017xxNAx
TOTAL (n/%)56/57 (98%)42/57 (74%)10/25(40%)19/57 (33%)19/57 (33%)44/57 (77%)10/57(17%)21/57(37%)2/57 (3 %)1/57 (2 %)
Number of individual verbatim outcomes35201410113614622

As seen in Table 2, tumor related outcomes such as recurrence (98%), progression (74%), treatment response (in CIS studies) (40%), and adverse events (77%) were frequently reported across studies. However, overall (33%) and cancer specific (33%) survival, treatment completion (17%) and treatment change (37%) were less often reported. Quality of Life (3%) and economic outcomes (2%) were rarely reported.

Tumor related outcomes

The heterogeneity in the definition and reporting of recurrence and progression in studies that recruited patients with papillary tumors only, and also treatment response in patients with CIS with or without papillary tumors, are shown in Tables 3 and 4, respectively.

Table 3

Definitions and reporting of bladder cancer recurrence and progression in RCTs with papillary-only tumors. RFS, recurrence-free survival; NMIBC, non-muscle invasive bladder cancer; MIBC, muscle-invasive bladder canvcer; PFR, progression-free survival; DFR, disease-free survival

Time to recurrenceRFSRecurrence rate (%)2 yr recurence-free rates (%)3 yr recurence-free rates (%)Recurrence per year (n)NMIBC recurrence (pTa,pT1)Early recurrence (0-2 yrs)Late recurrence (>  2 yrs)50 % recurrence-time (days)Recurrence index/100 patients per monthRecurrence-rate reductionT-stageGradeMIBCMetastasesProgression rateProgression rate at the time of first recurrenceTime to progression to MIBCTime to progression to distant metastasis5-yr PFRProgression as the first event included as recurrence?Progression-free survival
Kaasinen 2000xxx
Bilen 2000xx≥pT2xx
Van der Meijden 2001xxxxxxx
Nomata 2002xNA
Okamura 2002xxxx
Rajala 2002xxNA
Kuroda 2004xxNA
Koga 2004xxxNA
Mitsumori 2004xxNA
Cheng 2005xxxxxxx
Vijjan 2006xxTa->T1, T1->MIBCx
Hinotsu 2006xxxx≥pT2xx
Barghi 2006xxx
Ojea 2007xxxxxxx
El-Ghobashy 2007xxxxx≥pT2xx
Agrawal 2007xNA
Friedrich 2007xxxxNA
Hendricksen 2008xxx≥pT2, cisxxxxxx
Berrum-Svennung 2008xxx
Isbarn 2008xxx
Böhle 2009xxxxx
Gudjonsson 2009xxx
Järvinen 2009xx≥pT2xxxx
Seretta 2010xxxxx≥pT2xx
Sylvester 2010x≥pT2xxxxx
De Nunzio 2011xx (0-1yrs)x (>  1yr)x≥pT2x
Di Stasi 2011xx≥pT2xxxx
Hinotsu 2011xxx≥pT2xxxx
Oddens 2013xx5yr DFR≥pT2, cisxxxxx
Huang 2015xxxNA
Onishi 2017xxxnot specifiedxxx
Bijalwan 2017xxTa->T1LG->HGx
Table 4A

Definitions and reporting of A) recurrence, B) progression and C) treatment response in patients with CIS with or without papillary tumours

Study IDRecurrence rate (%)Disease-free intervalRecurrence-free survivalTime to recurrence5-year Recurrence-free survival (%)1-year Recurrence-free survival (%)Interval before recurrenceRecurrence rate at 5 yearsRegression of grade/stage (%)Worsening-free survival (mo, %)Low-grade relapse (n)High-grade superficial relapse (n)
Lamm 2000xxxx
Palou 2001xxx
Au 2001xx
Sekine 2001
Martinez-Pinneiro 2002xxxx
Di Stasi 2003xxx
Kaasinen 2003x
Martinez-Pinneiro 2005xx
de Reijke 2005
di Stasi 2006x (for cis pts)x
Gårdmark 2007
Cai 2008xxxx
Neple 2010xx
Porena 2010xx
Koga 2010x
Gülpınar 2012xxx
Järvinen 2012xx
Sengiku 2013xx
Inamoto 2013xx
Rentsch 2014x
Hemdan 2014xx
Martinez-Pineiro 2015xxx
Solsona 2015x
Arends 2016x
Nakai 2016x
Table 4B


Study IDT-stageGradeMIBCMetastasesProgression rateTime to progressionProgression-free timeProgression-free survival5 year progression-free survivalOther?Not definedProgression as the first event included as recurrence?
Lamm 2000≥pT2
Palou 2001xxx
Au 2001NA
Sekine 2001xxx
Martinez-Pinneiro 2002≥pT2xxxxfor cis -> extravesical extension
Di Stasi 2003xx
Kaasinen 2003≥pT1xx
Martinez-Pinneiro 2005≥pT2xx
de Reijke 2005≥pT2x
di Stasi 2006xxxx
Gårdmark 2007Ta->T1;xx (stage)NA
Cai 2008xx
Neple 2010x
Porena 2010xNA
Koga 2010xxx
Gülpınar 2012xx
Järvinen 2012≥pT2xxxx
Sengiku 2013NA
Inamoto 2013x
Rentsch 2014xxxx
Hemdan 2014xxxFree of progression (%)
Martinez-Pineiro 2015xxxx
Solsona 2015xxxx
Arends 2016xx
Nakai 2016xxx
Table 4C


Carcinoma in Situ Response
Study IDComplete responsePartial response no Cis, but Ta/T1 persistsNo change Cis or Ta/T1 persists
No Cis, no Ta/T1at 9 mo no progression; at 12 mo no recurrenceComplete response in cis patientsTime to recurrence in complete respondersRecurrence after complete response (%)First recurrence type after complete response (papillary, cis, papillary + cis)
Lamm 2000xx
Palou 2001
Au 2001
Sekine 2001xx
Martinez-Pinneiro 2002
Di Stasi 2003x
Kaasinen 2003x
Martinez-Pinneiro 2005
de Reijke 2005xxxx
di Stasi 2006x
Gårdmark 2007
Cai 2008x
Neple 2010
Porena 2010
Koga 2010x
Gülpınar 2012
Järvinen 2012
Sengiku 2013x
Inamoto 2013
Rentsch 2014
Hemdan 2014
Martinez-Pineiro 2015
Solsona 2015
Arends 2016x
Nakai 2016


Recurrence was reported in 56 (98%) of 57 trials (Tables 1,3,4), with 35 different verbatim names (Table 5), often related to the definition. The definition of recurrence was missing in 8/56 (14%) studies and in the others, variations of the percent of recurrences at a given time point or as a time to event outcome were used, but no consistent way of defining and measuring recurrence was used overall. Furthermore, in studies that reported both progression and recurrence, progression as the first event was regarded as a recurrence event in 12 studies and in 34 others it was not.

Table 5

Verbatim outcome name and definition heterogeneity

Time to recurrencemedian time to first recurrencetime to initial recurrenceearly recurrence (0– 2 years) late recurrence (>  2 years) recurrence-free survivalrelapse-free survival recurrence-free period disease-free time disease-free survival first bladder recurrence first TCC recurrence Recurrence-free interval disease-free interval Recurrence rate (%) recurrence rate and recurrence free rate at different time points: 1,2,3,5,10 yrs incident of recurrence probability of recurrence at 5,10 and 15 yrs Recurrence risk (at 3 year and 5 year) NMIBC recurrence (pTa, pT1) Recurrence per year recurrence rate per year tumor per year rate Recurrence index/100 patients per month Number of recurrences 50 % recurrence time (days) Recurrence-rate reduction absolute recurrence risk reduction Recurrence-free ratenon-recurrence rate relapse-free patients 1-year recurrence free survival 5-year recurrence free survival Tumor size of first recurrenceTime to progression Time to first progression Time to progression in stage Time to progression to MIBC Time to progression in grade Time to progression to distant metastasis Time to distant metastasis Progression-free survival 5-year progression-free survival Progression-free at 5 years Worsening free survival Progression-free time Duration of progression-free interval Progression rate Disease progression rate Free of progression Progression-free survival rate Rate of progression to T2 or grater Progression rate at the first recurrenceSurvival rate Overall survival Overall mortality Survival Cancer deaths Death Survival time5-year overall survival Time to death by any cause Duration of survival Causes of deathsCancer-specific survival rate Cancer deaths Cancer-specific death rate Disease-specific mortality Death from bladder cancer Cancer specific survival timeTime to death due to bladder cancer Cancer-specific survival Cause-specific survival Disease-specific survival Disease-free survival Cancer-specific survival timeStandardised outcome term Local toxicity Systematic toxicity Allergic reactions (dermatological) Constitutional symptoms Laboratory abnormality Treatment interruption Death due to toxicity Definitions/Instruments used CTCAE v3.0 (Common Terminology Criteria of Adverse) NCI-CTC v2.0 (National Cancer Institute-Common Toxicity) WHO toxicity grading scale WHO-ART (1979 WHO Adverse Reaction Terminology)complete response rate complete response at 3 months and 1 year response to treatment efficacy of induction therapy number of instillations need to achieve a complete response complete response rate in patients with CIS or concomitant carcinoma in situ (pTa or pT1) treatment efficacy in patients with CIS overall complete response ratecompletion rate (all planned instillations in the cycle were administered) performance rate (at least one of the three planned instillations in the cycle was administered) completion and performance rate at 3, 6, 12 and 18 months% of patients that received all 8 scheduled courses during 3 years% of patients completed the planned 2-year programmed treatment% of patients that received maintenance therapy for 6 months and 18 monthsthe number of institutions number of patients receiving 6-11 instillations number of patients receiving <  6 instillations number of patients received induction treatment maintenance (long-term arm): number of patients complete six instillations and also 1 yr, 2 and 3 yr of maintenance therapy median number of instillations probability of non-cessation of instillations (%) at 6, 12 and 15 months number of patients received at least 4 instillations as induction course number of patients that received fewer than the planned 28 instillatons of the protocol


Of 57 studies, 42 (74 %) reported bladder cancer progression. Definition for progression was given in 41/42 (97%) studies with a large variability in definition. A common threshold for “progression” was≥pT2 in 16 (38%) studies, with 2 of them also classifying CIS as a progression. As an example of inconsistency in verbatims used, “progression to MIBC” was used in the definition in 31/42 (74%) studies, with 22 of those further including metastases. Ta->  T1 and T1->  MIBC were considered progression in 4/42 (9%) studies (Tables 3 and 4).

Treatment response

Treatment response in patients with CIS was reported in 10 (40%) of 25 studies (Table 2). There was heterogeneity in what time-point was considered to assess the response to treatment. de Reijke et al defined and reported “complete response”, “partial response”, “no change” and “progression” [29]. The rest of the studies reported only complete response to treatment.

The time-point to assess complete response varied largely, ranging from 3 months from enrollment up to 12 months.

Eight different outcomes were included in the “Treatment response (for CIS)” domain (Tables 4 and 5).

Treatment relapse after complete response was described in three trials (Table 4).


A survival outcome was reported in 44/57 (40%) of studies; equally common were cancer-specific survival and overall survival, each reported in 19 (33%) studies. Ten and eleven different verbatim names were used to report overall survival and cancer-specific survival, respectively (Tables 2, 5).


Adverse events (AEs) were heterogeneously defined. In 12 of the 44 studies (27%) reporting AEs, there was no definition of an AE, and overall 24 different definitions/instruments were used. Studies reporting AEs used unique systems to categorise the type of AE or grade the severity of the AEs, and made no reference to a standardised reporting system. Across 10 studies, 3 standardised AE reporting instruments were used, but these did not include some of the most relevant AEs for intravesical instillations:

  • NCI-CTCAE (Common Terminology Criteria of Adverse Events),

  • WHO toxicity grading scale,

  • WHO-ART (1979 WHO Adverse Reaction Terminology)

Adverse events were further grouped in numerous ways, e.g. local or systemic toxicity, constitutional symptoms, laboratory abnormality, death, and treatment interruption due to AEs (Table 5). Detailed lists of how AEs were described and reported are provided in Supplementary Table 2.

In 25 of the 44 studies (57%) specific AEs were not listed; instead, authors reported either only local toxicities, or major/severe/more common side-effects or AEs that resulted in treatment interruption. Five of these 25studies did not report the list of individual toxicities at all; instead, authors presented only the frequency and percentage [n (%)] of any AEs which occurred. Furthermore, poor treatment compliance related to AEs was not consistently reported.


Adherence to completion of all planned instillations was at least partially reported in 10/57 (17%) studies: six studies concerning maintenance instillations, two “induction course” studies, and two studies comparing induction to maintenance. None of the single-instillation studies reported completion rates. Four studies gave a comprehensive overview of the reasons for treatment discontinuation. The author definitions for instillation treatment completion are reported in Table 5.


21/57 studies (37%) reported treatment failure and/or the need to change from instillations to a different treatment. 21 studies specified the treatment that was given after instillations were discontinued:

  • radical cystectomy (RC) (14/21 studies)

  • RC and/or radiotherapy (RT) (4/12 studies)

  • TURBT (1/21)

  • RC, TURBT + RT, chemotherapy (1/21)

  • “non-allowed instillations” (1/21)


Two studies measured and reported patient experience during the instillations; Koga et al by measuring Qol, and Huang et al by evaluating instillation related pain/irritation [30, 31].

In the study by Koga et al, Qol was assessed according to the Japanese version of the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-Core 30 (EORTC QLQ-C30) v2.0. QoL was assessed before induction therapy, after the 5th instillation of induction therapy, 4 weeks after the completion of induction therapy, and 14 months after randomization [30].

Huang et al evaluated the effect of hyaluronic acid in reducing pirarubicin instillation related side-effects. A visual analog scale (VAS) was used daily to evaluate pain [31].


Only one study evaluated the costs related to the treatment. Berrum-Svennung et al randomized BC patients to one instillation of epirubicin or placebo after TURBT and evaluated cancer recurrences. They also calculated the cost of delivering a single instillation during the initial treatment and as first recurrences occurred [32].


This is the first study to systematically and comprehensively overview the extent of outcome reporting, measurement, and definition heterogeneity in the setting of adjuvant treatments for NMIBC.

Recurrence was frequently reported in the included RCTs; yet, some studies did not define it. In those that did, there was variability in the names that were used, the definitions and the reporting. Most concerning, however, was the variation in how progression was handled in the analysis of recurrence. In studies where progression as the first event was counted as a recurrence, the measure provided is qualitatively and quantitatively different from those where recurrence was more narrowly defined as the re-appearance of a non-muscle invasive tumour. To overlook this subtlety runs the risk of not comparing like with like across studies, or statistically pooling aggregated results in a potentially misleading way.

Progression was also frequently reported, but again the definitions were inconsistent across trials. Worsening of the disease leads to a change in treatment strategy, and that was also inconsistently reported. It is also crucial to point out whether assessment of progression has been made based on imaging (e.g. CT or MRI), TURBT or radical cystectomy. As only four studies gave a comprehensive overview of reasons to change the treatment strategy, there is a high risk of getting misleading results. If prior to progression, patients die due to an unrelated cause, or undergo cystectomy (for example due to recurrent high grade T1 disease), then the progression rates at specific time points will be different according to whether the death and cystectomy have been counted as a competing risk (cumulative incidence function) or simply as censored (Kaplan-Meier curve). Equally important is to highlight how patients are followed for the efficacy outcomes in case the treatment has been stopped due to side-effects. There may also be a difference in outcomes according to whether the results are reported in all randomized patients (intent to treat analysis) or only in eligible patients who have been treated according to the protocol (per protocol analysis).

Treatment response in patients with CIS in specific was evaluated and reported in only 40% of studies. The rest of the studies recruiting patients with CIS considered CIS as papillary tumors, and reported only recurrence or/and progression. However, CIS additional diagnostic challenges and may have a very different disease course than papillary tumors do: as such, separate approaches to measure and define their outcomes should be applied [23].

The most heterogenous outcome was AEs, evident in the many categorizations and instruments used to record AEs, and in the system level subgroupings chosen by trialists. Unfortunately, many of these were not optimal for instillation-related AEs. Whilst in some instances it may be possible for systematic reviewers to recode lists of AEs (if they are provided) to a common standardized toxicity classification system, this is a poor excuse for lack of standardization in primary trials and needlessly adds time and complexity to the critical interpretation of the evidence base. Poor treatment compliance reporting is likely to confound other cancer related outcomes such as recurrence, progression and overall survival.

Perhaps the most alarming finding is that QoL is conspicuously missing. Instillation treatments are demanding for patients and it would be very important to understand all the consequences (both oncological and QoL-related) for patients before the decision about treatment is made. A recent investigation of QoL in bladder cancer patients compared to a matched sample of older adults without bladder cancer in a US population found significant declines in health-related QoL (HRQoL) scores over time in the physical, mental and social components of the SF-36 [33]. The EORTC Quality of Life Group also developed an externally validated QLQ-BLS24 questionnaire for NMIBC [34]. In a systematic review, Mason and colleagues used the COSMIN checklist to evaluate the psychometric properties of PROMs used in bladder cancer populations, of which two of the 15 included PROMs were NMIBC-specific (QLQ-BLS24 and CAVICAVEMNI) [35, 36]. Of note, they found that no existing PROM stood out as the most appropriate measure of QoL in any bladder cancer populations and although further validation studies are required generic PROMs, cancer-generic PROMs and bladder cancer-specific PROMs will currently provide the most robust picture. This is a very important study to a subsequent phase of our COS development as most existing cancer COS have included QoL and it is anticipated NMIBC patients will also prioritise this, encompassing urinary, bowel and sexual function, as a critically important outcome domains.

Without having included NMIBC patients in a qualitative study of their experiences of bladder cancer and its treatments, it cannot yet be known which outcomes are of most importance to them, or if they are adequately captured in current trials, but it is discouraging that so few trials routinely include patient reported outcome measures (PROMs).

Health economics was considered in only one RCT, which calculated costs of single instillation [32]. Bladder cancer, especially NMIBC, contributes significantly to healthcare costs due to intense surveillance strategies and its potential to recur and progress [8, 37]. This should be considered when treatments and outcomes are compared.

Kamat et al provided recommendations on NMIBC intervention trial designs, eligibility criteria, and ‘clinically meaningful’ effect size thresholds for outcomes [23]. Likewise, Lamm et al suggested a change in definition for progression in NMIBC [38]. These initiatives are important to bear in mind for subsequent phases of our project. Once the outcomes considered core by all stakeholders (e.g. patients, urologists, oncologists, nurses, payers, methodologists) are known (i.e. what to measure)[22] then we will turn attention to definitions and measurement tools (i.e. how to measure) [39] whilst again including key stakeholders. Importantly, these initiatives, in conjunction with ours, show that there is an acknowledgement of problems with the evidence base and a desire to do improvements.


The decision to exclude phase I and II trials (phases before determining the therapeutic effect of the drug) and to exclude all non-randomised designs may have limited the chance to capture longer-term and patient reported outcomes relating to function and QoL. However, in subsequent phases of the project, such as in Delphi survey and consensus meetings, participants will have an opportunity to propose ‘new’ outcomes not already considered for prioritisation, therefore we consider that the risk of having missed outcomes is minimal, and that we have carried out a pragmatic trade-off against the resource implication of including all study designs.


We have shown that there is inconsistency in outcome reporting and variation in definitions in randomized trials comparing adjuvant treatments in NMIBC patients. This situation makes comparing the results of individual studies difficult, and makes their statistical combination challenging, impossible, or inappropriate; hence, providing summaries of the evidence which are, at best, unwieldy and at worst misleading, making evidence-based treatment recommendations difficult. A core outcome set, incorporating the views of a variety of stakeholders such as urologists, oncologists, methodologists and, most importantly, patients, is urgently required.


The authors report no funding.


Erik Veskimae - performance of work; interpretation or analysis of data; writing the article.

Selvarani Subbarayan – performance of work; interpretation or analysis of data; writing the article.

Riccardo Campi - performance of work; interpretation or analysis of data; writing the article.

Domitille Carron - performance of work; interpretation or analysis of data; writing the article.

Muhammad Imran Omar – conception; performance of work; interpretation or analysis of data; writing the article.

Cathy Yuan- performance of work; interpretation or analysis of data; writing the article.

Konstantinos Dimitropoulos - performance of work; interpretation or analysis of data; writing the article.

Mieke Van Hemelrijck - interpretation or analysis of data; writing the article.

Richard T. Bryan - interpretation or analysis of data; writing the article.

James N’Dow - interpretation or analysis of data; writing the article.

Marek Babjuk - interpretation or analysis of data; writing the article.

J. Alfred Witjes - interpretation or analysis of data; writing the article.

Richard Sylvester – conception; performance of work; interpretation or analysis of data; writing the article.

Steven MacLennan – conception; performance of work; interpretation or analysis of data; writing the article.


This study, as a literature review, is exempt from any requirement for Institutional Review Board approval. No human or animal research was involved in the elaboration of this manuscript.


Erik Veskimae – Has no conflict of interest to report

Selvarani Subbarayan – Has no conflict of interest to report

Riccardo Campi - Has no conflict of interest to report

Domitille Carron - Has no conflict of interest to report

Muhammad Imran Omar – Has no conflict of interest to report

Cathy Yuan- Has no conflict of interest to report

Konstantinos Dimitropoulos - Has no conflict of interest to report

Mieke Van Hemelrijck - Has no conflict of interest to report

Richard T. Bryan - Reports other from Janssen EMEA, grants from UroGen Pharma, grants from QED Therapeutics, outside the submitted work

James N’Dow - Has no conflict of interest to report

Marek Babjuk - Has no conflict of interest to report

J. Alfred Witjes - Has no conflict of interest to report

Richard Sylvester – Has no conflict of interest to report

Steven MacLennan – Has no conflict of interest to report



A systematic review of outcome reporting, definition and measurement heterogeneity in Non-Muscle Invasive Bladder Cancer effectiveness trials of adjuvant, prophylactic treatment after transurethral resection

Search strategies

OVID Medline Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Ovid MEDLINE(R) Daily and Ovid MEDLINE(R) 1946 to Jan 15, 2020.

( n  = 341)

  • 1. exp Urinary Bladder Neoplasms/

  • 2. ((bladder or vesical) adj3 (cancer * or carcin * or malign * or tumor * or tumour * or neoplasm * or papilloma)).tw,kw.

  • 3. exp Carcinoma, Transitional Cell/

  • 4. (transitional cell adj3 (carcinoma * or cancer * or tumor * or tumor *)).tw,kw.

  • 5. (NMIBC and bladder).tw,kw.

  • 6. or/1– 5

  • 7. ((transurethral or trans-urethral) and resect * and bladder).tw,kw.

  • 8. (TURBT or TUR or TURB).tw,kw.

  • 9. exp Prophylactic Surgical Procedures/

  • 10. exp Chemotherapy, Adjuvant/ or exp Chemoradiotherapy, Adjuvant/

  • 11. (adjuvant or prophylaxis or prophylactic or prevent * or intravesical or intra-vesical or instillation *).tw,kw.

  • 12. or/7– 11

  • 13. 6 and 12

  • 14. randomized controlled

  • 15. controlled clinical

  • 16. random *.mp.

  • 17. placebo.ab.

  • 18. drug therapy.fs.

  • 19. trial.ab.

  • 20. groups.ab.

  • 21. or/14– 20

  • 22. exp animals/ not

  • 23. 21 not 22

  • 24. 13 and 23

  • 25. exp meta-analysis as topic/

  • 26. exp Meta-Analysis/

  • 27. (Systematic review or meta-analysis).tw,kw. or (Medline or Embase or Pubmed or Cochrane or literature search or literature review).ab.

  • 28. or/25– 27

  • 29. 24 and 28

  • 30. Congresses as Topic/ or “Journal: Conference Abstract”.pt.

  • 31. 29 not 31

Embase <  1974 to 2020 January 15 >  (via Ovid):∥( n  = 375)

  • 1 exp bladder tumor/

  • 2 ((bladder or vesical) adj3 (cancer * or carcin * or malign * or tumor * or tumour * or neoplasm * or papilloma)).tw,kw.

  • 3 exp transitional cell carcinoma/

  • 4 (transitional cell adj3 (carcinoma * or cancer * or tumor * or tumor *)).tw,kw.

  • 5 (NMIBC and bladder).tw,kw.

  • 6 or/1– 5

  • 7 exp transurethral resection/

  • 8 ((transurethral or trans-urethral) and resect * and bladder).tw,kw.

  • 9 (TURBT or TUR or TURB).tw,kw.

  • 10 exp prophylaxis/

  • 11 exp cancer adjuvant therapy/

  • 12 (adjuvant or prophylaxis or prophylactic or prevent * or intravesical or intra-vesical or instillation *).tw,kw.

  • 13 or/7– 12

  • 14 6 and 13

  • 15 Randomized controlled trial/

  • 16 Random$.ti,ab.

  • 17 randomization/

  • 18 intermethod comparison/

  • 19 placebo.ti,ab.

  • 20 (compare or compared or comparison).ti.

  • 21 ((evaluated or evaluate or evaluating or assessed or assess) and (compare or compared or comparing or comparison)).ab.

  • 22 (open adj label).ti,ab.

  • 23 ((double or single or doubly or singly) adj (blind or blinded or blindly)).ti,ab.

  • 24 double blind procedure/

  • 25 parallel group$1.ti,ab.

  • 26 (crossover or cross over).ti,ab.

  • 27 ((assign$ or match or matched or allocation) adj5 (alternate or group$1 or intervention$1 or patient$1 or subject$1 or participant$1)).ti,ab.

  • 28 (assigned or allocated).ti,ab.

  • 29 (controlled adj7 (study or design or trial)).ti,ab.

  • 30 (volunteer or volunteers).ti,ab.

  • 31 human experiment/

  • 32 trial.ti.

  • 33 or/15– 32

  • 34 (random$ adj sampl$ adj7 (“cross section$” or questionnaire$1 or survey$ or database$1)).ti,ab. not (comparative study/ or controlled study/ or randomi?edcontrolled.ti,ab. or randomly assigned.ti,ab.)

  • 35 Cross-sectional study/ not (randomized controlled trial/ or controlled clinical study/ or controlled study/ or randomi?edcontrolled.ti,ab. or control group$1.ti,ab.)

  • 36 (((case adj control$) and random$) not randomi?ed controlled).ti,ab.

  • 37 (nonrandom$ not random$).ti,ab.

  • 38 “Random field$ ”.ti,ab.

  • 39 (random cluster adj3 sampl$).ti,ab.

  • 40 (rat or rats or mouse or mice or swine or porcine or murine or sheep or lambs or pigs or piglets or rabbit or rabbits or cat or cats or dog or dogs or cattle or bovine or monkey or monkeys or trout or marmoset$1).ti. and animal experiment/

  • 41 Animal experiment/ not (human experiment/ or human/)

  • 42 or/34– 41

  • 43 33 not 42

  • 44 14 and 43

  • 45 exp “systematic review”/

  • 46 exp meta analysis/

  • 47 (Systematic review or meta-analysis).tw,kw. or (Medline or Embase or Pubmed or Cochrane or literature search or literature review).ab.

  • 48 or/45– 47

  • 49 44 and 48

  • 50 conference

  • 51 Conference

  • 52 50 or 51

  • 53 49 not 52

Cochrane Database of Systematic Reviews< 2005 to January 15, 2020 >  (via Ovid):

( n  = 91)

  • 1 ((bladder or vesical) adj3 (cancer * or carcin * or malign * or tumor * or tumour * or neoplasm * or papilloma)).tw,kw.

  • 2 (transitional cell adj3 (carcinoma * or cancer * or tumor * or tumor *)).tw,kw.

  • 3 (NMIBC and bladder).tw,kw.

  • 4 1 or 2 or 3

  • 5 ((transurethral or trans-urethral) and resect * and bladder).tw,kw.

  • 6 (TURBT or TUR or TURB).tw,kw.

  • 7 (adjuvant or prophylaxis or prophylactic or prevent * or intravesical or intra-vesical or instillation *).tw,kw.

  • 8 or/5– 7

  • 9 4 and 8


The authors have no acknowledgements.




Richters A , Aben KKH , Kiemeney LALM . The global burden of urinary bladder cancer: an update. World J Urol. 2020;38(8):1895–904. doi:10.1007/s00345-019-02984-4


van Rhijn BWG , Burger M , Lotan Y , et al. Recurrence and Progression of Disease in Non– Muscle-Invasive Bladder Cancer: From Epidemiology to Treatment Strategy. Eur Urol. 2009;56(3):430–42. doi:10.1016/j.eururo.2009.06.028


Babjuk M , Burger M , Compérat EM , et al. European Association of Urology Guidelines on Non-muscle-invasive Bladder Cancer (TaT1 and Carcinoma In Situ) - 2019 Update. Eur Urol. 2019;76(5):639–57. doi:10.1016/j.eururo.2019.08.016


Brierley J , Gospodarowicz M , Wittekind C . TNM Classification of Malignant Tumours, 8th Edition. Wiley-Blackwell; 2017.


Botteman MF , Pashos CL , Redaelli A , Laskin B , Hauser R . The health economics of bladder cancer: A comprehensive review of the published literature. PharmacoEconomics. 2003;21(18):1315–30. doi:10.1007/BF03262330


Riley GF , Potosky AL , Lubitz JD , Kessler LG . Medicare payments from diagnosis to death for elderly cancer patients by stage at diagnosis. Med Care. 1995;33(8):828–41. doi:10.1097/00005650-199508000-00007


Mossanen M , Wang Y , Szymaniak J , et al. Evaluating the cost of surveillance for non-muscle-invasive bladder cancer: an analysis based on risk categories. World J Urol. 2019;37(10):2059–65. doi:10.1007/s00345-018-2550-x


Svatek RS , Hollenbeck BK , Holmäng S , et al. The Economics of Bladder Cancer: Costs and Considerations of Caring for This Disease. Eur Urol. 2014;66(2):253–62. doi:10.1016/j.eururo.2014.01.006


Leal J , Luengo-Fernandez R , Sullivan R , Witjes JA . Economic Burden of Bladder Cancer Across the European Union. Eur Urol. 2016;69(3):438–47. doi:10.1016/j.eururo.2015.10.024


Sylvester RJ , Oosterlinck W , Holmang S , et al. Systematic Review and Individual Patient Data Meta-analysis of Randomized Trials Comparing a Single Immediate Instillation of Chemotherapy After Transurethral Resection with Transurethral Resection Alone in Patients with Stage pTa-pT1 Urothelial Carcinoma of the Bladder: Which Patients Benefit from the Instillation? Eur Urol. 2016;69(2):231–44. doi:10.1016/j.eururo.2015.05.050


Perlis N , Zlotta AR , Beyene J , Finelli A , Fleshner NE , Kulkarni GS . Immediate post-transurethral resection of bladder tumor intravesical chemotherapy prevents non-muscle-invasive bladder cancer recurrences: an updated meta-analysis on 2548 patients and quality-of-evidence review. Eur Urol. 2013;64(3):421–30. doi:10.1016/j.eururo.2013.06.009


Abern MR , Owusu RA , Anderson MR , Rampersaud EN , Inman BA . Perioperative intravesical chemotherapy in non-muscle-invasive bladder cancer: a systematic review and meta-analysis. J Natl Compr Cancer Netw JNCCN. 2013;11(4):477–84. doi:10.6004/jnccn.2013.0060


Bladder cancer: diagnosis and management of bladder cancer: © NICE (2015) . Bladder cancer: diagnosis and management of bladder cancer. BJU Int. 2017;120(6):755–65. doi:10.1111/bju.14045


Sylvester RJ , Oosterlinck W , Witjes JA . The Schedule and Duration of Intravesical Chemotherapy in Patients with Non– Muscle-Invasive Bladder Cancer: A Systematic Review of the Published Results of Randomized Clinical Trials. Eur Urol. 2008;53(4):709–19. doi:10.1016/j.eururo.2008.01.015


Cambier S , Sylvester RJ , Collette L , et al. EORTC Nomograms and Risk Groups for Predicting Recurrence, Progression, and Disease-specific and Overall Survival in Non-Muscle-invasive Stage Ta-T1 Urothelial Bladder Cancer Patients Treated with 1-3 Years of Maintenance Bacillus Calmette-Guérin. Eur Urol. 2016;69(1):60–9. doi:10.1016/j.eururo.2015.06.045


Hernández V , Espinos EL , Dunn J , et al. Oncological and functional outcomes of sexual function– preserving cystectomy compared with standard radical cystectomy in men: A systematic review. Urol Oncol Semin Orig Investig. 2017;35(9):539.e17–539.e29. doi:10.1016/j.urolonc.2017.04.013


Veskimäe E , Neuzillet Y , Rouanne M , et al. Systematic review of the oncological and functional outcomes of pelvic organ-preserving radical cystectomy (RC) compared with standard RC in women who undergo curative surgery and orthotopic neobladder substitution for bladder cancer. BJU Int. 2017;120(1):12–24. doi:10.1111/bju.13819


Bruins HM , Veskimae E , Hernandez V , et al. The impact of the extent of lymphadenectomy on oncologic outcomes in patients undergoing radical cystectomy for bladder cancer: a systematic review. Eur Urol. 2014;66(6):1065–77. doi:10.1016/j.eururo.2014.05.031


Williamson PR , Altman DG , Blazeby JM , et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132. doi:10.1186/1745-6215-13-132


Williamson P , Altman D , Blazeby J , Clarke M , Gargon E . Driving up the Quality and Relevance of Research Through the Use of Agreed Core Outcomes. J Health Serv Res Policy. 2012;17(1):1–2. doi:10.1258/jhsrp.2011.011131


Tunis SR , Clarke M , Gorst SL , et al. Improving the relevance and consistency of outcomes in comparative effectiveness research. J Comp Eff Res. 2016;5(2):193–205. doi:10.2217/cer-2015-0007


Williamson PR , Altman DG , Bagley H , et al. The COMET Handbook: version 1.0. Trials. 2017;18(Suppl 3):280. doi:10.1186/s13063-017-1978-4


Kamat AM , Sylvester RJ , Böhle A , et al. Definitions, End Points, and Clinical Trial Designs for Non-Muscle-Invasive Bladder Cancer: Recommendations From the International Bladder Cancer Group. J Clin Oncol Off J Am Soc Clin Oncol. 2016;34(16):1935–44. doi:10.1200/JCO.2015.64.4070


Bruce I , Harman N , Williamson P , et al. The management of Otitis Media with Effusion in children with cleft palate (mOMEnt): a feasibility study and economic evaluation. Health Technol Assess. 2015;19(68):1–374. doi:10.3310/hta19680


Moher D , Liberati A , Tetzlaff J , Altman DG , PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. doi:10.1371/journal.pmed.1000097


Prinsen CAC , Vohra S , Rose MR , et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” – a practical guideline. Trials. 2016;17(1):449. doi:10.1186/s13063-016-1555-2


Mokkink LB , Terwee CB , Patrick DL , et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539–49. doi:10.1007/s11136-010-9606-8


Dodd S , Clarke M , Becker L , Mavergames C , Fish R , Williamson PR . A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery. J Clin Epidemiol. 2018;96:84–92. doi:10.1016/j.jclinepi.2017.12.020




Koga H , Ozono S , Tsushima T , et al. Maintenance intravesical bacillus Calmette-Guérin instillation for Ta, T1 cancer and carcinoma in situ of the bladder: Randomized controlled trial by the BCG Tokyo Strain Study Group: Maintenance intravesical BCG. Int J Urol. 2010;17(9):759–66. doi:10.1111/j.1442-2042.2010.02584.x


Huang W , Wang F , Wu C , Hu W . Efficacy and safety of pirarubicin combined with hyaluronic acid for non-muscle invasive bladder cancer after transurethral resection: a prospective, randomized study. Int Urol Nephrol. 2015;47(4):631–6. doi:10.1007/s11255-015-0940-1


Berrum-Svennung I , Granfors T , Jahnson S , Boman H , Holmäng S . A Single Instillation of Epirubicin After Transurethral Resection of Bladder Tumors Prevents Only Small Recurrences. J Urol. 2008;179(1):101–6. doi:10.1016/j.juro.2007.08.166


Smith AB , Jaeger B , Pinheiro LC , et al. Impact of bladder cancer on health-related quality of life. BJU Int. 2018;121(4):549–57. doi:10.1111/bju.14047


Blazeby JM , Hall E , Aaronson NK , et al. Validation and Reliability Testing of the EORTC QLQ-NMIBC24 Questionnaire Module to Assess Patient-reported Outcomes in Non– Muscle-invasive Bladder Cancer. Eur Urol. 2014;66(6):1148–56. doi:10.1016/j.eururo.2014.02.034


Mason SJ , Catto JWF , Downing A , Bottomley SE , Glaser AW , Wright P . Evaluating patient-reported outcome measures (PROMs) for bladder cancer: a systematic review using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. BJU Int. 2018;122(5):760–73. doi:10.1111/bju.14368


Abáigar-Pedraza I , Megías-Garrigós J , Sánchez-Payá J . Cuestionario de calidad de vida para pacientes con cáncer de vejiga no músculo invasivo. Actas Urol Esp. 2016;40(4):251–7. doi:10.1016/j.acuro.2015.12.001


Cox E , Saramago P , Kelly J , et al. Effects of Bladder Cancer on UK Healthcare Costs and Patient Health-Related Quality of Life: Evidence From the BOXIT Trial. Clin Genitourin Cancer. 2020;18(4):e418–e442. doi:10.1016/j.clgc.2019.12.004


Lamm D , Persad R , Brausi M , et al. Defining progression in nonmuscle invasive bladder cancer: it is time for a new, standard definition. J Urol. 2014;191(1):20–7. doi:10.1016/j.juro.2013.07.102


COSMIN handbook.