Individualised Predictions of the Survival Benefit Due to Adjuvant Therapy in a Randomised Trial of Sorafenib after Nephrectomy for Localised Renal Cell Carcinoma

INTRODUCTION

‘What are my chances’, and ‘how much better will they be with a trial treatment’ are pivotal questions in clinical practice and research, but surprisingly little has been published regarding how doctors think and talk about prognosis and the potential benefits of adjuvant therapy. A prognosis is a forecast of the likely course and outcomes of a condition. The prognoses of a medical condition with and without a proposed treatment are the crucial elements for clinical decision-making in individuals, and for the design of randomised trials that inform these decisions. The difference between the prognoses with and without a proposed treatment reflect the effect of that treatment, referred to hereafter as the potential treatment benefit.

In oncology, prognoses and the potential survival benefits of treatment are typically expressed differently in discussions about early stage cancers treated with curative intent compared with advanced cancers treated with palliative intent. In early stage cancers treated with curative intent, prognoses and potential survival benefits are often expressed in terms of survival rates, typically at five years, e.g. the percentages expected to survive five years or longer. In cancers treated with palliative intent, prognoses and potential survival benefits are usually expressed in terms of median survival times: the length of time beyond which half are expected to survive [1, 2]. The optimal methods to think and communicate about prognosis and the possible benefits of adjuvant treatments remain unclear [3].

The role of adjuvant systemic therapy after nep-hrectomy for renal cell carcinoma (RCC) at intermediate or high risk of recurrence remains controversial. The aim of adjuvant therapy is to improve disease-free survival (DFS) and overall survival (OS) by pre-venting or delaying cancer recurrence. Five trials of vascular endothelial growth factor receptor tyrosine kinase inhibitors (VEGF-TKI) as adjuvant therapy following nephrectomy for intermediate and high risk renal cell carcinoma were activated from 2006 to 2012, following multiple randomised trials demonstrating the activity and effectiveness of these drugs in advanced RCC [4–7].

We sought clinicians’ predictions of prognosis and the potential benefit of adjuvant therapy for each individual patient they enrolled in the SORCE trial of adjuvant sorafenib (NCT00492258) to improve understanding of how oncologists think about prognosis and potential treatment benefit in the adjuvant setting.

METHODS

We conducted an observational sub-study nested within SORCE, an international, double-blind, pla-cebo-controlled, three arm, randomised phase 3 trial comparing observation (placebo), adjuvant sorafenib for one year, and adjuvant sorafenib for three years, after resection of localised RCC with a risk of recurrence categorised as intermediate or high according to the Leibovich score. The SORCE trial recruited 1711 participants from the United Kingdom (UK), France, Denmark, Belgium, the Netherlands and Australia from 2007 to 2013 and observed no differences in DFS or OS [8].

The clinicians participating in this sub-study were the medical oncologist and referring urologist of each patient recruited to SORCE at any site in Australia, and at selected sites in the UK. The clinicians were asked to complete a one page questionnaire for each patient they recruited to SORCE, before the patient had started their blinded study treatment (see Fig. 1). The questionnaires were completed prior to the release of results from any of the trials of VEGF-TKI as adjuvant therapy. The questionnaires asked for predictions of each patient’s prognosis, with and without adjuvant sorafenib, using two types of question, one based on survival times, the other based on survival rates.

Fig. 1

Questions completed by clinicians at baseline for each patient regarding their predicted prognosis with and without adjuvant sorafenib.

The survival time questions first asked clinicians to estimate ‘this patient’s life expectancy without adjuvant sorafenib’ in years, and then to estimate ‘the absolute increase in this patient’s life expectancy with one year of adjuvant sorafenib’ from a series of discrete options ranging from 0 months to more than 15 years. The survival rate questions first asked clinicians to estimate ‘this patient’s chance of living 5 or more years without adjuvant sorafenib’ as a percentage, and then to estimate ‘the absolute increase in this patient’s chance of living 5 years or more with one year of adjuvant sorafenib’ from a series of discrete options ranging from 0% to greater than 50%.

Our aim was to study clinicians’ predictions of prognosis and potential treatment benefits, not to test their background knowledge about prognostication or the trial, so we provided information about estimating survival and the assumptions used in the statistical design of the trial on the back of the one page questionnaire (Supplementary Appendix Figure 1). This background information included the life expectancy in years of Australians who have reached various ages,[9] and the median survival times corresponding to various five-year survival rates assuming an exponential survival distribution.

We used the following non-parametric statistics to summarise the distributions of clinicians’ estimates: median, interquartile range (IQR, 25th to 75th percentiles), and interdecile range (IDR, 10th to 90th percentiles). We used Spearman’s rank correlation (r_s) to assess associations, and Wilcoxon’s signed rank test to assess differences between paired observations. We assumed declining exponential survival distributions (that is, a constant hazard rate) during the period of interest to transform prognoses elicited as survival times into prognoses expressed as survival rates, and to calculate hazard ratios (HRs) reflecting the potential benefits of adjuvant treatment with sorafenib in relative terms. To assess the plausibility of the elicited survival benefits expressed as HRs, we compared them with a benchmark of 0.75, the HR for DFS hypothesized in the SORCE trial protocol, based on an improvement in three year DFS from 63.5% to 71%. We expected that plausible benefits in OS would be smaller than those hypothesised for DFS, and therefore would be reflected by HRs for OS that were larger (less extreme, and closer to 1.0) than the 0.75 hypothesized for DFS.

Each site had approval to conduct the study from its Human Research Ethics Committee or Institutional Review Board, and all participants provided written informed consent. This sub-study was led and conducted by the Australian and New Zealand Urogenital and Prostate Cancer Trials Group (ANZUP) and the National Health and Medical Research Council of Australia Clinical Trials Centre (NHMRC CTC) at the University of Sydney, in collaboration with the Medical Research Council Clinical Trials Unit at University College London (MRC CTU at UCL), UK.

RESULTS

Clinicians from 20 sites in Australia and 12 sites in the UK completed questionnaires for each of their patients recruited to the SORCE trial. Sixty-one medical oncologists provided predictions of prognosis for 216 of their participating patients. The median number of completed questionnaires per medical oncologist was 2 with an interquartile range (IQR) of 1 to 5, and a range of 1 to 18. The majority of questionnaires were completed by medical oncologists in Australia (157, 73% of the total). Seventeen urologists provided predictions of prognosis for 39 patients who had a paired prediction of prognosis from their medical oncologist.

Predictions of prognosis without adjuvant sor-afenib varied widely, as expected. Medical oncologists’ predictions are shown in Figs. 2A, 2B. The distribution of their predicted 5-year survival rates without adjuvant sorafenib were roughly symmetrical with a median of 60%, IQR of 45% to 70%, and IDR of 25% to 80%. The distribution of their predicted median survival times without adjuvant sorafenib were skewed to the right (towards longer survival times), with a median of 7 years, IQR of 5 to 12 years, and IDR of 3 to 15 years. The relationship between predictions of 5-year survival rates elicited directly by asking for estimates of 5-year survival rates, versus indirectly, by transforming estimates of survival time in years into estimates of 5-year survival rates assuming an exponential survival distribution, is shown in Fig. 3. The correlation between these two methods of eliciting prognosis was substantial (r_s = 0.62, p < 0.0001) with no appreciable systematic difference in magnitude between them (p = 0.6).

Fig. 2

Distributions of medical oncologists’ predicted survival rates and survival times without adjuvant sorafenib, and predicted improvements with adjuvant sorafenib. A. Predicted chance of living 5 years or more without adjuvant sorafenib (n = 179). B. Predicted life expectancy without adjuvant sorafenib (n = 177). C. Predicted absolute increase in the chance of living 5 years with adjuvant sorafenib (n = 212). D. Predicted absolute increase in life expectancy with adjuvant sorafenib (n = 214).

Fig. 3

Relationship between the paired predictions of prognosis without sorafenib based on the predicted chance of living 5 years and the predicted life expectancy in years expressed as survival a 5-year survival rate.

Predictions of the potential benefits of adjuvant sorafenib also varied widely, as expected. Medical oncologists’ predictions are shown in Figs. 2C, 2D. Their predictions of the absolute improvements in 5-year survival rate with one year of sorafenib were skewed to the right (towards larger improvements) with a median of 10%, IQR of 5% to 18%, and IDR of 4% to 50%. Their predictions of the absolute improvements in life expectancy with one year of sorafenib expressed as survival times were also skewed to the right (towards larger improvements) with a median of 1 year, IQR 0.75 to 5 years, and IDR of 0.5 to 10 years.

To compare predictions of potential treatment benefits based on absolute improvements in survival times versus absolute improvements in survival rates, we again transformed predictions elicited as improvements in survival times into improvements in 5-year survival rates by assuming an exponential survival distribution, see Fig. 4. There was moderate correlation between these paired estimates (r_s = 0.54, p < 0.0001). However, predictions of the absolute potential treatment benefit with adjuvant sorafenib were larger when elicited using improvements in survival rates rather than survival times (median improvement in 5-year survival rate of 7% vs 6% respectively, p < 0.0001).

Fig. 4

Relationship between the paired predictions of absolute treatment benefit with adjuvant sorafenib based on the predicted chance of living 5 years and predicted life expectancy in years expressed as survival rates.

The relationship between medical oncologists’ predictions of 5-year survival rates without adjuvant sorafenib versus the absolute improvement with 1 year of adjuvant sorafenib is shown in Fig. 5A. There was moderate correlation between these paired estimates (r_s = –0.34, p < 0.0001). The relationship between medical oncologists’ predictions of median survival time without adjuvant sorafenib versus the absolute improvement with 1 year of adjuvant sorafenib is shown in Fig. 5B. There was moderate correlation between these paired estimates (r_s = 0.32, p < 0.0001).

Fig. 5

Relationship between predicted survival without adjuvant sorafenib, and predicted absolute benefit of adjuvant sorafenib (A. as a prediction of chance of living 5 years or more and B. as a prediction of life expectancy).

The predicted absolute benefits of adjuvant sorafenib for each individual patient were transformed into predicted relative benefits expressed as HRs, using the standard assumptions of proportional hazards and exponential survival distributions. The paired predictions of the relative benefits with adjuvant sorafenib (n = 172) elicited as improvements in survival time versus survival rates are shown in Fig. 6. The correlation between the predictions elicited as survival times versus rates was moderate (r_s = 0.41, p < 0.0001), but lower than expected given that they are estimating the same effect. Moreover, the predicted relative benefit of adjuvant sorafenib was substantially larger when elicited as an improvement in survival rates than when elicited as an improvement in survival times (median HR of 0.76 vs 0.83 respectively, p < 0.0001). We specified a priori that the predicted benefit of adjuvant sorafenib on overall survival in individual patients should be smaller (less extreme and closer to 1.0) than the benchmark HR of 0.75 for DFS hypothesized in the trial design. However, the proportions of HR for OS predicted by medical oncologists for individual patients that were less extreme (closer to 1.0) than the target hazard ratio for DFS hypothesized in the trial design was 51% for predictions elicited as improvements in survival rates, and 65% for predictions elicited as improvements in survival times.

Fig. 6

Relationship between the predicted relative benefits (HR) elicited as improvements in survival time versus survival rates.

There were 39 patients with paired predictions from their medical oncologist and their urologist. Urologists predicted substantially larger benefits with adjuvant sorafenib than medical oncologists when elicited as improvements in survival rates (median absolute benefit in 5-year survival rate of 15% vs 5%, p = 0.009, median relative benefit HR 0.55 vs 0.76, p = 0.02), but not when elicited as improvements in survival times (median absolute increase benefit in survival time of 2 years vs 3 years, p = 0.77, median relative benefit HR 0.76 vs 0.80, p = 0.89).

DISCUSSION

The benefits of adjuvant sorafenib predicted by clinicians for their individual patients were larger when elicited as improvements in survival rates than when elicited as improvements in survival times. The predicted benefits of adjuvant sorafenib based on survival times were more consistent with the target HR in the statistical hypothesis of the trial design. Urologists predicted substantially larger benefits with sorafenib than medical oncologists when based on survival rates (median absolute increase of 15% vs 5%, p = 0.009, median relative benefit of HR 0.55 vs 0.76, p = 0.02). Estimates of survival without adjuvant therapy based on survival rates and survival times were consistent with one another (median chance of surviving 5 years or more of 60% vs 61%, p = 0.6).

We were surprised to find that the predicted benefit of adjuvant therapy was substantially larger when elicited as an improvement in survival rate, the standard metric for expressing outcomes in the adjuvant setting, than when elicited for the same patient as an improvement in survival time, the standard metric for expressing outcomes in advanced cancer. We were also surprised that predictions of the same treatment effect elicited with these two methods were only moderately correlated (r_s = 0.41, p < 0.0001). These findings suggest that clinicians may have difficulty comparing, interpreting and predicting potential treatment benefits expressed in terms of survival rates, survival times, and hazard ratios. It also sup-ports the findings of other studies that physicians’ decisions about treatment recommendations are inf-luenced by whether the treatment effects are exp-ressed in relative versus absolute terms [10–12].

The wide distribution of predictions of survival time and survival rate without adjuvant therapy was expected, as individual patients were of differing ages and co-morbidities, as well as of differing baseline risks of cancer recurrence. We expected that the predicted absolute benefits of adjuvant treatment would be related to the risk of recurrence, with a bigger absolute benefit in survival rate for those at higher risk (lower survival rate without treatment); and, that predicted relative benefits would be approximately constant, as is generally assumed in the des-ign and analysis of research reporting on survival outcomes [13, 14]. There was moderate correlation between the predicted survival without treatment and the paired predicted absolute treatment benefits for survival time (r_s = 0.32, p < 0.0001), moderate correlation in the opposite direction for survival rate (r_s = –0.34, p < 0.0001), and a wide range of predicted relative benefits. Although the direction of correlation was as expected, the magnitude of correlation was less than expected. One possible explanation for this is that the clinicians’ predictions of potential treatment benefit were more influenced by the duration of adjuvant treatment than by the risk of recurrence in an individual patient.

The SORCE trial was powered to detect a 7.5% absolute increase in 3 year disease DFS from 63.5% to 71% (HR 0.75). Effects on overall survival are likely to be less extreme (i.e. HR closer to 1.0). Predictions of the potential benefit of adjuvant sorafenib based on survival times were smaller (HR less extreme, closer to 1.0), and therefore more consistent with the hypothesised effect size in the trial design, than predictions based on improvements in survival rates. This was surprising as we expected predictions of treatment benefit to be more accurate and plausible when based on survival rates because these are commonly used to analyse, report, and discuss treatment benefits in the adjuvant setting. Median survival times are rarely reached, reported, or discussed in the adjuvant setting. However, clinicians in our study were likely to overestimate the benefits of adjuvant therapy when expressed as an improvement in survival rates, whereas their estimates were more plausible when expressed as an improvement in median survival times. This may be because seemingly small improv-ements of a few percent in survival rates at times sooner than the median survival time translate into seemingly large improvements in median survival time. Improvements expressed in survival times also facilitate consideration of the actual duration of adjuvant therapy and the likely durability of its effects. These findings imply that researchers and clinicians should consider the potential survival benefits of adj-uvant therapies expressed as improvements in both survival times and survival rates when designing, reporting, and discussing clinical trials of adjuvant therapy.

Urologists in our study predicted that adjuvant sorafenib would have substantially larger benefits than did the medical oncologists who saw the same patient when the benefits were elicited as improvements in survival rates, but not survival times. The numbers of urologists and paired observations were small, and self-selected rather than random, so this observation should be taken as hypothesis-genera-ting, and as worthy of further study because patients who may be suitable for adjuvant therapy are usually seen and spoken with by a urologist first, before referral to a medical oncologist for further consideration of adjuvant treatment. The opinions and advice of both urologists and medical oncologists are likely to be influential in decision making [15, 16]. Patients given differing opinions about adjuvant therapy by their surgeons and medical oncologist about adjuvant therapy may find it harder to make decisions, and may be more vulnerable to decisional regret.

The main strengths of this study are its prospective design and conduct within a randomised controlled trial. The questionnaires were completed before the release of results from any of the trials of VEGF-TKI as adjuvant therapy, so clinicians were not influenced by these results. Its main limitation is the selection bias inherent in eliciting answers from clinicians participating in a randomised controlled trial whose views might differ systematically from those of other clinicians. The numbers of urologists, and of paired observations, were too small for definitive conclusions. Our results included a few extreme predictions that cast doubt on those respondents’ understanding of the questions. However, we doubt that those who chose to provide answers were less knowledgeable than those who chose not. Our assumption of constant hazard rates over the period of interest is a simplification that is often used to calculate sample sizes for clinical trials, and while unlikely to be exactly true, is also unlikely to affect comparisons between treatment groups.

Clinicians should take account of these potential cognitive biases when discussing adjuvant therapy with patients, and be wary of overestimating and/or overstating the potential benefits of adjuvant therapy. Others have shown that patients prefer explanations of the absolute benefits of a treatment (given as natural frequency, e.g. number per hundred or as a percentage) rather than as a relative risk reduction, or number needed to treat [11, 16]. Until it is known whether patients prefer information about the benefits of adjuvant treatment to be expressed in survival rates or survival times, clinicians should consider explaining both. For example a potential benefit could be expressed as an improvement of approximately m months longer life expectancy in someone expected to live y years without adjuvant therapy, as well as an improvement of x per hundred people surviving 5 years from y per hundred without adjuvant therapy to z per hundred with adjuvant therapy. Graphical representation of this information may help facilitate understanding [17, 18].

In conclusion, the predicted benefits of adjuvant therapy with sorafenib in individual patients were larger when elicited as improvements in survival rates than in survival times. The predictions of benefit eli-cited as improvements in survival rate were larger than plausible in comparison with the effects hypothesized in the trial design. Clinicians may have diff-iculty comparing, interpreting and predicting potential treatment benefits expressed in terms of survival rates, survival times, and hazard ratios. Clinicians and researchers should be wary of possible biases in how they think, and presumably talk, about prognosis and the potential benefits of adjuvant therapy.

FUNDING

This work was supported by funding from Cancer Australia for ANZUP and the NHMRC Clinical Trials Centre. The SORCE trial was funded globally by Cancer Research UK, Bayer Pharmaceuticals and the Medical Research Council UK, and partly supported in Australia and New Zealand by an untied grant from Bayer Australia. Bayer had no input into the design, conduct, analysis, or reporting of this study. ID Davis is supported by an NHMRC Practitioner Fellowship (APP1102604). NJ Lawrence received support from an NHMRC Postgraduate Scholarship (Australia), Goldman Sachs New Zea-land Fellowship (New Zealand) and a Top-Up PhD Scholarship from Sydney Catalyst (University of Sydney and Cancer Institute NSW, Australia). The NHMRC Clinical Trials Centre is supported by an Australian NHMRC Program Grant (APP1037786).

AUTHOR CONTRIBUTION

NJ Lawrence: Concept and design, Interpretation and Analysis of data, Drafting and Revising the manuscript, Final approval, A Martin: Conception and design, Interpretation and Analysis of data, Drafting and Revising the manuscript, Final approval, ID Davis: Acquisition of data, Interpretation of data, Revising the manuscript, Final approval, S Troon: Acquisition of data, Revising the manuscript, Final approval, S Sengupta: Acquisition of data, Revising the manuscript, Final approval, E Hovey: Acquisition of data, Revising the manuscript, Final approval, X Coskinas: Acquisition of data, Revising the manuscript, Final approval, R Kaplan: Acquisition of data, Revising the manuscript, Final approval, B Smith: Acquisition of data, Revising the manuscript, Final approval, A Ritchie: Acquisition of data, Revising the manuscript, Final approval, A Meade: Acquisition of data, Revising the manuscript, Final approval, J Goh: Acquisition of data, Revising the manuscript, Final approval, H Gurney: Acquisition of data, Revising the manuscript, Final approval, M Harrison: Acquisition of data, Revising the manuscript, Final approval, K Fife: Acquisition of data, Revising the manuscript, Final approval, T Eisen: Acquisition of data, Revising the manuscript, Final approval, P Blinman: Acquisition of data, Revising the manuscript, Final approval, MR Stockler: Concept and design, Acquisition of data, Interpretation and Analysis of data, Drafting and Revising the manuscript, Final approval.

CONFLICT OF INTEREST

ID Davis is or has been within the last two years a member/ chair of following industry advisory boards: Astellas, AstraZeneca, Bayer, Bristol Myers Squibb, Ipsen, MSD, Novartis, Pfizer, Roche. No personal remuneration is received for any of this work. All payments or honoraria are paid directly to ANZUP Cancer Trials Group. ID Davis is a director and chair of the Board of ANZUP Cancer Trials Group. No personal remuneration is received for any of this work. ANZUP led the SORCE trial in Australia with funding support from Bayer Australia. ANZUP has funding agreements to support clinical trials in renal cell carcinoma from MSD, Amgen and BMS. ANZUP has funding from Cancer Australia through its Support for Cancer Clinical Trials Program.

S Sengupta is a member of the Board of ANZUP Cancer Trials Group. No personal remuneration is received for this work.

E Hovey is a member of the Janssen Advisory Board and Bayer Advisory Board for prostate cancer.

J Goh is a member of the BMS Advisory Board, and speaker’s bureau /engagement for Ipsen and MSD.

H Gurney is a member of Advisory Boards for Bayer, Roche, Pfizer, BMS and MSD.

K Fife is a member of the following advisory boards: Esai, Ipsen, Roche, Novartis, Merck, Pfizer, Eusa, BMS and has received speaker fees/ consultancy for BMS, Pfizer, MSD and Ipsen. Institutional Research Funding: Roche, MSD, Exelixis.

T Eisen declared current employment with Astra Zeneca and Vice President of Astra Zeneca. Research grants from Astra Zeneca, Pfizer, Bayer to Dr Eisen, and from Astra Zeneca to institution.

MR Stockler declared research grants from Astellas, Bayer, Astra Zeneca, Pfizer, Celgene, BMS, Me-rck, and Amgen all to institution.

All other authors declared no conflict of interest.