Advertisement

Comparing symptom reporting by prostate cancer patients and healthcare professionals in the international multicentre REQUITE study

Open AccessPublished:November 25, 2022DOI:https://doi.org/10.1016/j.radonc.2022.11.015

      Highlights

      • One of the largest prostate cancer cohort studies on symptom agreement.
      • Agreement was better for observable than subjective treatment-related symptoms.
      • Fatigue had a negative impact on the agreement.
      • Patients usually graded their symptoms more severely than healthcare professionals.
      • PROs should complement symptom assessment by healthcare professionals.

      Abstract

      Introduction

      Previous studies showed that healthcare professionals and patients had only moderate to low agreement on their assessment of treatment-related symptoms. We aimed to determine the levels of agreement in a large cohort of prostate cancer patients.

      Methods

      Analyses were made of data from 1,756 prostate cancer patients treated with external beam radiotherapy (RT) and/or brachytherapy in Europe and the USA and recruited into the prospective multicentre observational REQUITE study. Eleven pelvic symptoms at the end of RT were compared after translating patient-reported outcomes (PROs) into CTCAE-based healthcare professional ratings. Gwet's AC2 agreement coefficient and 95% confidence intervals were calculated for each symptom. To compare severity of grading between patients and healthcare professionals, percent agreement and deviations for each symptom were graphically depicted. Stratified and sensitivity analyses were conducted to identify potential influencing factors and to assess heterogeneity and robustness of results.

      Results

      The agreement for the 11 pelvic symptoms varied from very good (AC2>0.8: haematuria, rectal bleeding, management of sphincter control) to poor agreement (AC2≤0.2: urinary frequency fair, proctitis and urinary urgency). Fatigue had a negative impact on the agreement. Patients tended to grade symptoms more severely than healthcare professionals. Information on sexual dysfunction was missing more frequently in healthcare professional assessment than PROs.

      Conclusion

      Agreement was better for observable than subjective symptoms, with patients usually grading symptoms more severely than healthcare professionals. Our findings emphasize that PROs should complement symptom assessment by healthcare professionals and be taken into consideration for clinical decision-making to incorporate the patient perspective.

      Abbreviations:

      CTCAE (Common Terminology Criteria for Adverse Events), EBRT (External beam radiotherapy), EORTC (European Organisation for Research and Treatment of Cancer), ADT (Hormone therapy), LENT (Late Effects of Normal Tissue), PRO (patient-reported outcome), RT (radiotherapy)

      Keywords

      Introduction

      The assessment of treatment-related symptoms and severity is subject to the perception and interpretation of healthcare professionals such as physicians and study nurses [
      • Trotti A.
      • Colevas A.D.
      • Setser A.
      • Basch E.
      Patient-reported outcomes and the evolution of adverse event reporting in oncology.
      ]. Patient-reported outcomes (PROs) have been suggested to improve communication between patients and healthcare professionals and to facilitate early detection of adverse events [
      • Trotti A.
      • Colevas A.D.
      • Setser A.
      • Basch E.
      Patient-reported outcomes and the evolution of adverse event reporting in oncology.
      ,
      • Basch E.
      The missing voice of patients in drug-safety reporting.
      ]. Thus, inclusion of the patient perspective has received increased attention in recent years [
      • Basch E.
      Beyond the FDA PRO guidance: steps toward integrating meaningful patient-reported outcomes into regulatory trials and US drug labels.
      ]. Treatment-related symptoms have been shown to affect patients' physical and psychosocial domains of quality of life [
      • Bacon C.G.
      • Giovannucci E.
      • Testa M.
      • Glass T.A.
      • Kawachi I.
      The association of treatment-related symptoms with quality-of-life outcomes for localized prostate carcinoma patients.
      ]. However, patients and healthcare professionals have been shown to have low to moderate agreement in symptom assessment [
      • Atkinson T.M.
      • Ryan S.J.
      • Bennett A.V.
      • Stover A.M.
      • Saracino R.M.
      • Rogak L.J.
      • et al.
      The association between clinician-based common terminology criteria for adverse events (CTCAE) and patient-reported outcomes (PRO): a systematic review.
      ]. Furthermore, patients were found to often grade their symptoms with a higher severity than healthcare professionals [
      • Basch E.
      • Iasonos A.
      • McDonough T.
      • Barz A.
      • Culkin A.
      • Kris M.G.
      • et al.
      Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study.
      ,
      • Gravis G.
      • Marino P.
      • Joly F.
      • Oudard S.
      • Priou F.
      • Esterni B.
      • et al.
      Patients' self-assessment versus investigators' evaluation in a phase III trial in non-castrate metastatic prostate cancer (GETUG-AFU 15).
      ,

      Rammant E, Ost P, Swimberghe M, Vanderstraeten B, Lumen N, Decaestecker K, Bultijnck R, De Meerleer G, Sarrazyn C, Colman R, Fonteyne V. Patient- versus physician-reported outcomes in prostate cancer patients receiving hypofractionated radiotherapy within a randomized controlled trial. Strahlenther Onkol. 2019 May;195(5):393-401. English. doi: 10.1007/s00066-018-1395-y.

      ].
      The high prevalence and declining mortality rates for prostate cancer in many countries due to screening and improved treatment highlight the relevance of reliably assessing the symptoms experienced by prostate cancer patients [

      Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries [published correction appears in CA Cancer J Clin. 2020 Jul;70(4):313]. CA Cancer J Clin. 2018;68(6):394-424. doi:10.3322/caac.21492

      ,
      • Litwin M.S.
      • Lubeck D.P.
      • Henning J.M.
      • Carroll P.R.
      Differences in urologist and patient assessments of health related quality of life in men with prostate cancer: results of the CaPSURE database.
      ]. The aim of this paper is to describe symptom assessments of prostate cancer patients who were treated with radiotherapy (RT) and healthcare professionals by determining their degree of agreement and exploring the direction of deviations in symptom severity assessments in a large international multicentre cohort study. Stratified and sensitivity analyses were conducted to identify potential influencing factors as well as to assess heterogeneity and robustness of results.

      Methods

      Study population

      Data from the prospective international multicentre observational cohort study REQUITE on radiotherapy toxicity were used for this analysis [

      Seibold P, Webb A, Aguado-Barrera ME, Azria D, Bourgier C, Brengues M, Briers E, Bultijnck R, Calvo-Crespo P, Carballo A, Choudhury A, Cicchetti A, Claßen J, Delmastro E, Dunning AM, Elliott RM, Fachal L, Farcy-Jacquet MP, Gabriele P, Garibaldi E, Gómez-Caamaño A, Gutiérrez-Enríquez S, Higginson DS, Johnson K, Lobato-Busto R, Mollà M, Müller A, Payne D, Peleteiro P, Post G, Rancati T, Rattay T, Reyes V, Rosenstein BS, De Ruysscher D, De Santis MC, Schäfer J, Schnabel T, Sperk E, Symonds RP, Stobart H, Taboada-Valladares B, Talbot CJ, Valdagni R, Vega A, Veldeman L, Ward T, Weißenberger C, West CML, Chang-Claude J; REQUITE consortium. REQUITE: A prospective multicentre cohort study of patients undergoing radiotherapy for breast, lung or prostate cancer. Radiother Oncol. 2019 Sep;138:59-67. doi: 10.1016/j.radonc.2019.04.034.

      ]. In REQUITE, 4,438 patients with non-metastatic breast, lung or prostate cancer were recruited from 26 radiation oncology departments via their healthcare professionals in seven European countries (Belgium, France, Germany, Great Britain, Italy, The Netherlands, Spain) and the USA prior to the start of their RT treatment in 2014-2016. The majority of prostate cancer patients was diagnosed in this time period. 27% of the patients had a previous prostatectomy between 1996 and 2016. In brief, adult men with prostate cancer and planned potentially curable radiotherapy according to local RT regimens were eligible. Patients received external beam radiotherapy (EBRT) and/or brachytherapy, with some patients additionally receiving hormone therapy (ADT) and/or previously undergoing prostatectomy. Overall, 1,760 prostate cancer patients with detailed RT data were included in the study. Written informed consent was obtained from all patients. Local ethics committees approved the study which is registered at www.controlled-trials.com ISRCTN98496463.

      Symptom measures

      Healthcare professionals in radiation oncology departments graded 26 gastrointestinal, urinary, and sexual dysfunction symptoms using the Common Terminology Criteria for Adverse Events (CTCAE) version 4.0 [
      • U.S. Department of Health and Human Services
      National Institutes of Health, National Cancer Institute.
      ]. CTCAE grades 0-5 correspond to absent, mild, moderate, severe, life-threatening symptoms and death due to an adverse event. Patients graded their symptoms using a standardized pelvic symptoms questionnaire with 19 items based on Late Effects of Normal Tissue (LENT) / CTCAE [
      • Farnell D.J.
      • Mandall P.
      • Anandadas C.
      • Routledge J.
      • Burns M.P.
      • Logue J.P.
      • et al.
      Development of a patient-reported questionnaire for collecting toxicity data following prostate brachytherapy.
      ]. Patients and healthcare professionals prospectively assessed pelvic symptoms and quality of life/fatigue (EORTC QLQ-C30) at baseline before start of RT, at the end of RT as well as annually thereafter until at least two years after RT [
      • Aaronson N.K.
      • Ahmedzai S.
      • Bergman B.
      • Bullinger M.
      • Cull A.
      • Duez N.J.
      • et al.
      The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology.
      ]. The present analysis compared symptom assessments at the end of RT, as the highest frequency of symptoms with the least loss to follow-up was observed at this time point. Patients were included if one or more symptom assessments were provided by either patient or healthcare professional at the end of RT.
      To compare symptom assessments, the ratings of patients and healthcare professionals were aligned. The questionnaires and their translation details can be found in Appendix A. Eleven comparable symptoms were identified: gastrointestinal symptoms including proctitis, diarrhoea, rectal bleeding and management of sphincter control; urinary symptoms including haematuria, urinary incontinence, urinary frequency, urinary urgency, urinary retention, as well as sexual dysfunction symptoms including erectile dysfunction (ED) and libido/orgasmic dysfunction. ED was analysed as a dichotomized variable (grade 0 vs. ≥1) due to the differential item phrasing.

      Statistical Analysis

      The degree of agreement in symptom assessment between patients and healthcare professionals was quantified using the chance-corrected agreement coefficient AC2 [
      • Gwet K.L.
      Computing inter-rater reliability and its variance in the presence of high agreement.
      ]. A difference in symptom assessment by one grade was considered as partial agreement. A difference of two or more grades was considered as disagreement. The magnitude of the agreement coefficients was classified using Altman’s Kappa Benchmark Scale [

      Altman DG. Practical Statistics for Medical Research. London: Chapman and Hall, 1991. p404–408

      ]. Coefficients ≤0.20 were interpreted as poor, 0.21-0.40 as fair, 0.41-0.60 as moderate, 0.61-0.80 as good, and 0.81-1.00 as very good agreement. 95% confidence intervals were calculated. In the absence of a symptom assessment by either the patient or the healthcare professional, the grading provided was included in the calculation of AC2. If neither grading was obtained, the subjects were excluded from the calculation of the respective agreement coefficient.
      Truncated linear weights were appended to account for partial agreement [
      • Gwet K.L.
      Handbook of inter-rater reliability: the definitive guide to measuring the extent of agreement among multiple raters.
      ]. Symptomatic patients were defined as those for whom the patient, the healthcare professional or both reported the presence of the respective symptom with CTCAE grade ≥1.
      Percent agreement and respective proportions of deviations by ≥1 grade are shown graphically for each symptom to explore whether patients or healthcare professionals graded symptoms with a higher severity.
      To assess potential heterogeneity, stratified analyses were conducted according to fatigue at end of RT (≥39 vs. <39 normalized EORTC QLQ-C30 scores), countries, education, hormone therapy and age (<70 vs. ≥70 years) [
      • Giesinger J.M.
      • Kuijpers W.
      • Young T.
      • Tomaszewski K.A.
      • Friend E.
      • Zabernigg A.
      • et al.
      Thresholds for clinical importance for four key domains of the EORTC QLQ-C30: physical functioning, emotional functioning, fatigue and pain.
      ].
      As a sensitivity analysis to examine the robustness of agreement, partial agreements were given less weight and no weight in the calculation of the AC2 coefficients by appending truncated radical weights and identity weights [
      • Gwet K.L.
      Handbook of inter-rater reliability: the definitive guide to measuring the extent of agreement among multiple raters.
      ].
      All statistical analyses were performed with R 4.1.1 and the package irrCAC [
      • R Core Team
      R: A language and environment for statistical computing.
      ,

      Gwet KL. irrCAC: Computing Chance-Corrected Agreement Coefficients (CAC). 2019. https://CRAN.R-project.org/package=irrCAC

      ]. Visualisations were created using the R package ggplot2 [
      • Wickham H.
      ggplot2: Elegant Graphics for Data Analysis.
      ].

      Results

      Baseline symptoms prior to RT were graded by 1,714 patients and 1,758 healthcare professionals. 1,756 patients were identified for whom at least one symptom assessment by patient or healthcare professional was provided at the end of RT. Patients were largely of European descent with a mean age of 69 years (Table 1) and had a Gleason Score of ≥7. About 50% were classified as low/intermediate risk groups. 27% of patients underwent radical prostatectomy and 69% were treated with hormone therapy. 72% of the patients received an EBRT dose per fraction of ≤2 Gy.
      Table 1Selected characteristics of included prostate cancer patients of the REQUITE cohort with symptom assessment at the end of radiotherapy (RT).
      CharacteristicN = 1 756
      n (%) and may not add up to 100% due to missings. 2 The proportion of patients with clinically important levels of fatigue was determined as proposed by Giesinger et al. [18]. 3 If provided, pathological T stage was considered, else MRI T stage or clinical T stage. 4 The higher grade of the cN and pN classification is reported. 5 The D'Amico risk classification for prostate cancer classifies patients into low (clinical T stage T1c, T2a and PSA level ≤ 10 ng/mL and Gleason score of ≤ 6), intermediate (clinical T stage T2b or PSA level 11-20 ng/mL or Gleason score of 7) and high (clinical T stage T2c or PSA level > 20 ng/mL or Gleason score of ≥ 8) risk for biochemical recurrence after surgery [30]. 6 Excluding patients receiving brachytherapy only.
      Country
      Belgium325 (18.5%)
      France251 (14.3%)
      Germany77 (4.4%)
      Italy193 (11%)
      Netherlands73 (4.2%)
      Spain284 (16.2%)
      UK509 (29%)
      USA44 (2.5%)
      Age
      Mean (SD)69 (7)
      Range42 – 88
      BMI (kg/m2)
      <25473 (26.9%)
      25-30881 (50.2%)
      >30394 (22.4%)
      Ethnicity
      European Descent1 689 (96.2%)
      Other Ethnic Background60 (3.4%)
      Smoking Status
      Current204 (11.6%)
      Former864 (49.2%)
      Never681 (38.8%)
      Fatigue (yes)2373 (21.2%)
      Tumour Category3
      T1252 (14.4%)
      T2841 (47.9%)
      T3/T4602 (34.3%)
      Lymph Node Category4
      N01 007 (57.3%)
      N1122 (6.9%)
      NX627 (35.7%)
      Gleason Score
      <7314 (17.9%)
      71 001 (57%)
      >7437 (24.9%)
      Prostate-Specific Antigen (ng/mL), pre-diagnostic biopsy
      ≤10995 (56.7%)
      >10–20402 (22.9%)
      >20293 (16.7%)
      D’Amico Risk Classification5
      Low Risk142 (8.1%)
      Intermediate Risk753 (42.9%)
      High Risk828 (47.2%)
      Prior Prostatectomy (yes) (Treatment Years 1996 – 2016)481 (27.4%)
      Hormone Therapy ADT (yes)1 218 (69.4%)
      Radiotherapy (Treatment Years 2014 – 2016)
      EBRT type
      3D conformal radiotherapy288 (16.4%)
      IMRT231 (13.2%)
      VMAT1158 (65.9%)
      Duration of EBRT (days)6
      Mean (SD)48 (11)
      Range18 – 100
      EBRT without brachytherapy (N = 1 574, 89.6%)
      EBRT regimens
      1.6 Gy/d; 59.2 Gy total dose2 (<1%)
      1.8 – 2.0 Gy/d; 66 – 72 Gy total dose322 (20.5%)
      1.8 – 2.0 Gy/d; 74 – 81 Gy total dose860 (54.6%)
      2.1 – 2.5 Gy/d; 63 – 77 Gy total dose124 (7.9%)
      >2.5 – 3.4 Gy/d; 51.2 – 69.25 Gy total dose261 (16.6%)
      EBRT with brachytherapy (N = 103, 5.9%)
      EBRT regimens
      1.8 – 2.0 Gy fraction dose; 45 – 50.4 Gy24 (23.3%)
      2.45 – 2.54 Gy fraction dose; 36.8 – 38.1 Gy79 (76.7%)
      Brachytherapy type
      HDR84 (81.6%)
      LDR19 (18.4%)
      HDR dose (Iridium, Cobalt-60; median, range)15 Gy (12.6 – 21.0)
      LDR dose (mostly palladium; median, range)99.5 Gy (85.0 – 108.0)
      Brachytherapy alone (N = 79, 4.5%)
      Brachytherapy type
      HDR8 (10.1%)
      LDR71 (89.9%)
      HDR dose (Iridium; median, range)19 Gy (19.0 – 25.2)
      LDR dose (mostly iodine; median, range)145 Gy (124.0 – 160.0)
      D: day. EBRT: External beam radiotherapy. Gy: Gray. HDR: High dose rate. IMRT: Intensity-modulated radiation therapy. LDR: Low dose rate. VMAT: Volumetric Arc Therapy.
      1 n (%) and may not add up to 100% due to missings. 2 The proportion of patients with clinically important levels of fatigue was determined as proposed by Giesinger et al.
      • Giesinger J.M.
      • Kuijpers W.
      • Young T.
      • Tomaszewski K.A.
      • Friend E.
      • Zabernigg A.
      • et al.
      Thresholds for clinical importance for four key domains of the EORTC QLQ-C30: physical functioning, emotional functioning, fatigue and pain.
      . 3 If provided, pathological T stage was considered, else MRI T stage or clinical T stage. 4 The higher grade of the cN and pN classification is reported. 5 The D'Amico risk classification for prostate cancer classifies patients into low (clinical T stage T1c, T2a and PSA level ≤ 10 ng/mL and Gleason score of ≤ 6), intermediate (clinical T stage T2b or PSA level 11-20 ng/mL or Gleason score of 7) and high (clinical T stage T2c or PSA level > 20 ng/mL or Gleason score of ≥ 8) risk for biochemical recurrence after surgery
      • D'Amico A.V.
      • Whittington R.
      • Malkowicz S.B.
      • Schultz D.
      • Blank K.
      • Broderick G.A.
      • et al.
      Biochemical outcome after radical prostatectomy, external beam radiation therapy, or interstitial radiation therapy for clinically localized prostate cancer.
      . 6 Excluding patients receiving brachytherapy only.
      All symptoms were predominantly graded by both patients and healthcare professionals (Table 2, Figure 1). The proportion of missing assessments by patients is uniformly distributed across symptoms (5.9%–8.8%, Figure 1). However, fewer healthcare professionals than patients provided assessment of sexual dysfunction.
      Table 2Translated symptom assessments of N = 1,756 included prostate cancer patients and healthcare professionals of the REQUITE cohort at the end of radiotherapy (RT).
      CTCAEGastrointestinal
      Proctitis (AC2: 0.19)Diarrhoea (AC2: 0.79)Management of Sphincter Control (AC2: 0.88)Rectal Bleeding (AC2: 0.94)
      PROPROPROPRO
      0123Missing0123Missing0123Missing0123Missing
      03415086636190929108672791250265541311013597813132
      124853011814144165123723112112224596125
      28251357817164669237205271
      3020502144100000
      Missing0001030000200001000
      CTCAEUrinary
      Urinary Frequency (AC2: 0.37)Urinary Urgency (AC2: -0.06)Urinary Retention* (AC2: 0.44)Urinary Incontinence (AC2: 0.73)
      PROPROPROPRO
      0123Missing0123Missing0123Missing0123Missing
      0273131193621321337366695343254928932541162984
      195144816253154524375070711060107721419
      202315314141031010271201181158169
      301200
      Missing01400230210001200
      UrinarySexual Dysfunction
      Haematuria* (AC2: 0.98)Erectile DysfunctionErectile Dysfunction (dichotomized) (AC2: 0.77)Libido/Orgasmic Dysfunction (AC2: 0.49)
      PROPROPROPRO
      CTCAE0123Missing0123Missing0123Missing0123Missing
      01529153151118817191188819138881733
      13315148054726421351024851824659071
      212004736145405515211741
      383783
      Missing2000621807386218738106905358
      * Grade 2 and Grade 3 were grouped due to the item phrasings.
      Figure thumbnail gr1
      Figure 1Absolute frequency of provided and missing symptom assessments of 1,756 prostate cancer patients and healthcare professionals of the REQUITE cohort at end of radiotherapy.
      AC2 coefficients for the agreement between the overall patient population and healthcare professionals showed very good (haematuria, rectal bleeding, management of sphincter control) and good (diarrhoea, ED, urinary incontinence) agreement for six of the eleven symptoms (Figure 2). Libido/orgasmic dysfunction and urinary retention showed moderate, urinary frequency fair agreement. Poor degrees of agreement were observed for proctitis and urinary urgency.
      Figure thumbnail gr2
      Figure 2AC2 coefficients for the agreement in symptom assessment between 1,756 included prostate cancer patients and healthcare professionals of the REQUITE cohort at the end of radiotherapy. The number of symptomatic patients, defined as those for whom either patient, healthcare professional or both assessed a grade ≥ 1 for the respective symptom, is given in brackets for each symptom. Erectile dysfunction was included dichotomously due to differential item phrasing. AC2: Gwet's AC2 agreement coefficient; 95% CI: 95% confidence interval
      Agreement declined for eight of eleven symptoms when the analysis was restricted to the symptomatic patient subgroup and remained similar for libido/orgasmic dysfunction, urinary frequency and ED. Differences of AC2 agreement coefficients for the overall patient population and symptomatic patients varied in magnitude. Less prevalent symptoms, such as haematuria, rectal bleeding, and management of sphincter control, showed considerably lower agreement coefficients.
      Appending truncated or identity weights resulted in equal or lower agreement for all symptoms (Appendix B).
      We conducted stratified analyses according to fatigue, country of treatment, hormonal treatment, education and age. Patients with fatigue showed lower agreement with healthcare professionals for all symptoms compared (Figure 3a). The largest differences in agreement coefficients between subgroups were observed for proctitis and urinary retention. More easily observable symptoms that rarely occur, such as haematuria and rectal bleeding, showed very good agreement with little between-country variation (Figure 3b). More subjective symptoms such as proctitis showed substantial differences between countries with agreement coefficients ranging from poor to good. For seven of the eleven symptoms compared, Spain showed the highest agreement among countries, whereas the lowest coefficients were observed for Germany and the United Kingdom. For the stratified analysis according to hormone therapy there was generally lower agreement in symptom assessment for patients who received hormone therapy than those who did not (Appendix C). Higher agreement was observed for patients with lower education (Appendix D). Differences in agreement coefficients by age groups were small and did not show a consistent pattern (Appendix E).
      Figure thumbnail gr3
      Figure 3AC2 coefficients for the agreement in symptom assessment between 1,756 prostate cancer patients and healthcare professionals of the REQUITE cohort at the end of radiotherapy stratified (a) by fatigue and (b) by hormone therapy. (A) 373 patients were classified as experiencing clinically important levels of fatigue (≥39 normalized scores), whereas 1260 patients were categorized as non-fatigue (<39)
      [
      • Giesinger J.M.
      • Kuijpers W.
      • Young T.
      • Tomaszewski K.A.
      • Friend E.
      • Zabernigg A.
      • et al.
      Thresholds for clinical importance for four key domains of the EORTC QLQ-C30: physical functioning, emotional functioning, fatigue and pain.
      ]
      . AC2: Gwet's AC2 agreement coefficient; 95% CI: 95% confidence interval
      Figure 4a/b shows the percent agreement between patients and healthcare professionals and the relative frequency of deviations by one or more grades.
      Figure thumbnail gr4
      Figure 4Percent agreement and deviations in symptom assessment between (a) 1,756 included prostate cancer patients and healthcare professionals and (b) between included symptomatic prostate cancer patients and healthcare professionals of the REQUITE cohort at the end of radiotherapy. For (A) the total number of patients considered is given in brackets. Higher symptom ratings by patients are shown left of the (green) percent agreement, higher symptom ratings by healthcare professionals are shown right thereof. Erectile dysfunction was included dichotomously due to item phrasing.
      In the overall patient population, patients and healthcare professionals mostly agreed on symptom assessment. However, patients usually graded their symptoms more severely than healthcare professionals, except for sexual dysfunction and haematuria. Urinary frequency, urinary urgency and proctitis, which were among the five most frequently reported symptoms, showed the lowest percent agreement.
      In the subpopulation of symptomatic patients, there was substantially less percent agreement compared to the overall patient population (Figure 4b). Symptomatic patients graded most symptoms more severely than healthcare professionals, except for sexual dysfunction and haematuria. Particularly low percent agreement was observed for proctitis and management of sphincter control. The proportion of higher patient ratings by more than one grade increased for all symptoms.

      Discussion

      The aim of this analysis was to determine the degree of agreement and the direction of deviations in symptom assessment between prostate cancer patients and healthcare professionals. The observed agreement between the overall patient population and healthcare professionals was very good and good for six of eleven symptoms. Symptomatic patients and patients with fatigue tended to agree less with healthcare professional assessment. Patients graded symptoms generally more severely than healthcare professionals, with substantial proportions of deviations of >1 grades observed for symptomatic patients.
      In the overall patient population, the highest agreement was observed for symptoms that are presumably more perceptible to healthcare professionals, such as haematuria, rectal bleeding, and the management of sphincter control. Consistently, the lowest agreement was found for subjective, less observable symptoms such as proctitis and urinary urgency which is consistent with findings for other cancer types [
      • Basch E.
      • Iasonos A.
      • McDonough T.
      • Barz A.
      • Culkin A.
      • Kris M.G.
      • et al.
      Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study.
      ,
      • Chandwani K.D.
      • Zhao F.
      • Morrow G.R.
      • Deshields T.L.
      • Minasian L.M.
      • Manola J.
      • et al.
      Lack of Patient-Clinician Concordance in Cancer Patients: Its Relation With Patient Variables.
      ]. However, the symptoms with the highest concordance have the least number of symptomatic patients. A high proportion of asymptomatic patients may 'inflate' the determined degree of agreement [
      • Atkinson T.M.
      • Rogak L.J.
      • Heon N.
      • Ryan S.J.
      • Shaw M.
      • Stark L.P.
      • et al.
      Exploring differences in adverse symptom event grading thresholds between clinicians and patients in the clinical trial setting.
      ].
      To address this, agreement in the symptomatic patient subgroup was assessed. Symptomatic patients and healthcare professionals usually showed lower agreement except for urinary frequency and ED. AC2 coefficients were considerably lower for less prevalent symptoms compared to the overall patient population. Haematuria showed very good agreement in the overall patient population, but only fair agreement in symptomatic patients. Agreement for rectal bleeding and management of sphincter control declined from very good to moderate. However, also more prevalent symptoms, such as urinary retention with 967 symptomatic patients, showed a substantial decrease from moderate to poor concordance.
      To the best of our knowledge, for the first time, stratified analyses examined agreement by fatigue, country as well as further patient and treatment characteristics. Stratified analysis according to hormone therapy showed generally lower agreement in symptom assessment for patients who received hormone therapy than those who did not (Appendix C). Further stratification by type of hormone therapy did not suggest a consistent trend. However, the analysed time point at the end of RT may not be ideal for this analysis, as the hormone therapy may not have been received for a sufficient length of time. Higher agreement was furthermore observed for patients with lower education, although these results may also reflect differences in patient populations and treatments between countries (Appendix D). Differences in agreement coefficients by age groups were small and did not show a consistent pattern (Appendix E).
      In line with previous studies, patients tended to grade their symptoms more severely than healthcare professionals [

      Rammant E, Ost P, Swimberghe M, Vanderstraeten B, Lumen N, Decaestecker K, Bultijnck R, De Meerleer G, Sarrazyn C, Colman R, Fonteyne V. Patient- versus physician-reported outcomes in prostate cancer patients receiving hypofractionated radiotherapy within a randomized controlled trial. Strahlenther Onkol. 2019 May;195(5):393-401. English. doi: 10.1007/s00066-018-1395-y.

      ]. In the overall patient population, these deviations were largest for proctitis, urinary urgency and urinary retention. These discrepancies may be due to perceived and perceptible in contrast to experienced symptom severity. Patients also graded urinary frequency and urinary incontinence more severely, which is compatible with the substantial disagreement regarding urinary leakage reported by [
      • McCammon K.A.
      • Kolm P.
      • Main B.
      • Schellhammer P.F.
      Comparative quality-of-life analysis after radical prostatectomy or external beam radiation for localized prostate cancer.
      ]. Urinary, bowel and sexual symptoms have been shown to affect the psychosocial domains of quality of life in prostate cancer patients [
      • Bacon C.G.
      • Giovannucci E.
      • Testa M.
      • Glass T.A.
      • Kawachi I.
      The association of treatment-related symptoms with quality-of-life outcomes for localized prostate carcinoma patients.
      ]. Thus, healthcare professionals may also have underestimated the associated psychological burden. A high proportion of symptomatic patients were found to rate their symptom severity more than 1 grade higher than healthcare professionals. This is particularly striking, as a two-point difference in CTCAE grading is sufficient to adjust treatment [
      • Atkinson T.M.
      • Li Y.
      • Coffey C.W.
      • Sit L.
      • Shaw M.
      • Lavene D.
      • et al.
      Reliability of adverse symptom event reporting by clinicians.
      ].
      For a non-negligible proportion of patients (14%), healthcare professional assessment of sexual dysfunction was lacking (while PROs were available). Possible explanations include that healthcare professionals did not address sexual dysfunction due to a perceived sensitivity or considered them to be of lower priority e.g. because of older age or marital status. Patients may also have been more hesitant to provide information about sexual dysfunctions during the consultation. It can be hypothesized that differences in gender or age of healthcare professionals and patients may impede the dialogue on sexual dysfunction.
      In case both patient and healthcare professional assessments were provided, the agreement for ED was good which is consistent with [
      • Litwin M.S.
      • Lubeck D.P.
      • Henning J.M.
      • Carroll P.R.
      Differences in urologist and patient assessments of health related quality of life in men with prostate cancer: results of the CaPSURE database.
      ,
      • Sonn G.A.
      • Sadetsky N.
      • Presti J.C.
      • Litwin M.S.
      Differing perceptions of quality of life in patients with prostate cancer and their doctors.
      ]. Lack of assessment by healthcare professionals underlines the importance of symptom assessment by patients themselves. Without the collection of PROs, information on sexual dysfunction burden would be available for a lower proportion of patients. As sexual dysfunction is common after prostate cancer treatment, this is particularly important [
      • Chung E.
      • Brock G.
      Sexual rehabilitation and cancer survivorship: a state of art review of current literature and management strategies in male sexual dysfunction among prostate cancer survivors.
      ]. Even though patients’ responses are considered the most reliable source, they may be subject to certain biases if, for example, they do not want to disappoint the treating physician by reporting severe symptoms [
      • Sonn G.A.
      • Sadetsky N.
      • Presti J.C.
      • Litwin M.S.
      Differing perceptions of quality of life in patients with prostate cancer and their doctors.
      ].
      Agreement with healthcare professionals was lower for patients with fatigue. Symptom assessment might be improved by routinely consulting PROs of affected patients. Furthermore, agreement tended to be stronger for patients who did not receive hormone therapy, except for sexual dysfunction. Hormone therapy may facilitate the reporting of sexual dysfunction symptoms by patients, as these have already been raised as expected symptoms for androgen deprivation therapy. As an apparent contributing factor to sexual dysfunction, it might encourage communication between patients and healthcare professionals about these symptoms.
      In the sensitivity analysis, lower classifications of agreement strength were found for unweighted coefficients. Thus, without accounting for partial agreement, concordance between patients and healthcare professionals may have been underestimated.
      The work presented has some limitations. As patients and healthcare professionals used different symptom assessment tools, the conversion of PROs into CTCAE items is a source of potential bias [
      • Gravis G.
      • Marino P.
      • Joly F.
      • Oudard S.
      • Priou F.
      • Esterni B.
      • et al.
      Patients' self-assessment versus investigators' evaluation in a phase III trial in non-castrate metastatic prostate cancer (GETUG-AFU 15).
      ,
      • Schuurhuizen C.S.E.W.
      • Marino P.
      • Braamse A.M.J.
      • Buffart L.M.
      • Joly F.
      • Fizazi K.
      • et al.
      Impact of Patient- and Clinician-Reported Cumulative Toxicity on Quality of Life in Patients With Metastatic Castration-Naïve Prostate Cancer.
      ]. Furthermore, the PRO questionnaire was initially validated in a cohort of patients with brachytherapy [
      • Farnell D.J.
      • Mandall P.
      • Anandadas C.
      • Routledge J.
      • Burns M.P.
      • Logue J.P.
      • et al.
      Development of a patient-reported questionnaire for collecting toxicity data following prostate brachytherapy.
      ]. As symptoms were compared in a population where all patients received RT, the resulting bias is considered negligible. Ideally, the data collection instruments provided to patients and healthcare professionals would be identical without requiring additional translations. The primary analyses did not control for potential confounders. We conducted stratified analyses though. Generalisability, also to other ethnic backgrounds, may be limited since not all eligible patients participated in the study. However, overall the study population is a good representation of real world data.
      Among the strengths of this analysis are the large number of patient and healthcare professional assessments analysed, making it one of the largest studies on agreement in prostate cancer patients, as well as extensive standardised data collected prospectively on patient characteristics and treatment received. Furthermore, the AC2 agreement coefficient is considered a paradox-resistant alternative to the commonly used Cohen’s k [
      • Basch E.
      • Iasonos A.
      • McDonough T.
      • Barz A.
      • Culkin A.
      • Kris M.G.
      • et al.
      Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study.
      ,

      Rammant E, Ost P, Swimberghe M, Vanderstraeten B, Lumen N, Decaestecker K, Bultijnck R, De Meerleer G, Sarrazyn C, Colman R, Fonteyne V. Patient- versus physician-reported outcomes in prostate cancer patients receiving hypofractionated radiotherapy within a randomized controlled trial. Strahlenther Onkol. 2019 May;195(5):393-401. English. doi: 10.1007/s00066-018-1395-y.

      ,
      • Gwet K.L.
      Computing inter-rater reliability and its variance in the presence of high agreement.
      ,
      • Fromme E.K.
      • Eilers K.M.
      • Mori M.
      • Hsieh Y.C.
      • Beer T.M.
      How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30.
      ]. In addition, partial agreement between patients and healthcare professionals were included as well as stratified and subgroup analyses to assess heterogeneity and robustness of findings, allowing for a finer depiction of their agreement.
      In conclusion, agreement was better for observable than subjective symptoms, with patients usually grading symptoms more severely. PROs provide valuable insights into the experience of patients whose symptoms tended to be often underestimated and should complement symptom assessment by healthcare professionals, in particular in patients with fatigue and/or undergoing hormone therapy. PROs might be routinely integrated into clinical practice by providing paper or online questionnaires to the patients prior to their follow-up visits and discussing salient items with their healthcare professionals. This may facilitate the identification of undetected or underestimated symptom burden, enable early treatment adjustment and improve communication with patients. In addition, missing values in toxicity data collection might be reduced by substituting assessments of high agreement symptoms such as haematuria and rectal bleeding with available PRO or Healthcare Professional assessments, respectively.

      Data Availability Statement

      The data that support the findings of this study are available from the corresponding author upon reasonable request (fees may apply).

      Acknowledgements

      We sincerely thank all patients that participated in the REQUITE study and all REQUITE staff involved at the following sites: Belgium: Ghent University Hospital; KU Leuven. France: ICM Montpellier, CHU Nîmes (Department of Radiation Oncology, CHU Nîmes, Nîmes, France). Germany: Zentrum für Strahlentherapie Freiburg (Dr. Petra Stegmaier); Städtisches Klinikum Karlsruhe (Dr. Bernhard Neu); ViDia Christliche Kliniken Karlsruhe (Prof. Johannes Claßen); Klinikum der Stadt Ludwigshafen GmbH (PD Dr. Thomas Schnabel); Universitätsklinikum Mannheim: Anette Kipke and Christiane Zimmermann. The researchers at DKFZ also thank Anusha Müller, Irmgard Helmbold, Thomas Heger, Sabine Behrens, Nicholas Schreck, Axel Benner. Petra Seibold is supported by ERA PerMed 2018 funding (BMBF #01KU1912) and BfS funding (#3619S42261). Italy: Fondazione IRCCS Istituto Nazionale dei Tumori, Milano; Candiolo Cancer Institute – FPO, IRCCS. Tiziana Rancati was partially funded by Fondazione Italo Monzino. The Netherlands: Sylvie Canisius at Maastro Clinics, Maastricht. Spain: Santiago: Complexo Hospitalario Universitario de Santiago. Ana Vega is supported by Spanish Instituto de Salud Carlos III (ISCIII) funding, an initiative of the Spanish Ministry of Economy and Innovation partially supported by AvaEuropean Regional Development FEDER Funds (INT20/00071, PI19/01424, INT15/00070, INT16/00154, INT17/00133, INT20/00071, PI19/01424, PI16/00046, PI13/02030, PI10/00164), and through the Autonomous Government of Galicia (Consolidation and structuring program: IN607B); UK: University Hospitals of Leicester NHS Trust; Manchester: Jacki Routledge at Christie NHS Foundation Trust, Manchester. Catharine West, Ananya Choudhury and Zoe Lingard are supported by NIHR Manchester Biomedical Research Centre and Catharine West is supported by Cancer Research UK (C1094/A18504, C147/A25254). USA: Mount Sinai Hospital, New York.
      Funding
      REQUITE was funded from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement #601826. The follow-up extension of the REQUITE patients is supported by ERA-NET ERA PerMed 2018 funding (BMBF, DS-CAT, ICSIII, FRRB, ANR) and BfS funding (#3619S42261).
      Conflict of Interest
      D. Azria: none related to the current manuscript. Outside the current manuscript: involved in the creation of the start-up NovaGray in 2015. D. De Ruysscher: none related to the current manuscript. Outside the current manuscript: advisory board of Astra Zeneca, Bristol-Myers-Squibb, Roche/Genentech, Merck/Pfizer, Celgene, Noxxon, Mologen and has received investigator initiated grants from Bristol-Myers-Squibb, Boehringer Ingelheim and Astra-Zeneca. E. Sperk: none related to the current manuscript. Outside the current manuscript: Lecture honoraria Zeiss Meditec Ag, travel support Zeiss Meditec AG.
      The other authors declare that they do not have any conflict of interest.
      Author contributions
      PS, JCC, IH: study design. PH: data analysis. PH, PS: wrote original draft. PS, PH, JCC, IH: data interpretation. CW, CT: chief investigators of the REQUITE study. MEAB, BA, DA, RB, JCC, AC, MPFJ, VF, AGC, ML, TR, BR, DDR, ES, RPS, CT, RV, AV, LV: patient enrolment and data collection. TW, EB: prostate cancer patient advocates of the REQUITE study. JCC, PS, ZL, TH, AW: study and data management. All authors reviewed and approved the final manuscript.

      References

        • Trotti A.
        • Colevas A.D.
        • Setser A.
        • Basch E.
        Patient-reported outcomes and the evolution of adverse event reporting in oncology.
        J Clin Oncol. 2007; 25: 5121-5127https://doi.org/10.1200/JCO.2007.12.4784
        • Basch E.
        The missing voice of patients in drug-safety reporting.
        N Engl J Med. 2010; 362: 865-869https://doi.org/10.1056/NEJMp0911494
        • Basch E.
        Beyond the FDA PRO guidance: steps toward integrating meaningful patient-reported outcomes into regulatory trials and US drug labels.
        Value Health. 2012 May; 15: 401-403https://doi.org/10.1016/j.jval.2012.03.1385
        • Bacon C.G.
        • Giovannucci E.
        • Testa M.
        • Glass T.A.
        • Kawachi I.
        The association of treatment-related symptoms with quality-of-life outcomes for localized prostate carcinoma patients.
        Cancer. 2002; 94: 862-871https://doi.org/10.1002/cncr.10248
        • Atkinson T.M.
        • Ryan S.J.
        • Bennett A.V.
        • Stover A.M.
        • Saracino R.M.
        • Rogak L.J.
        • et al.
        The association between clinician-based common terminology criteria for adverse events (CTCAE) and patient-reported outcomes (PRO): a systematic review.
        Support Care Cancer. 2016 Aug; 24: 3669-3676https://doi.org/10.1007/s00520-016-3297-9
        • Basch E.
        • Iasonos A.
        • McDonough T.
        • Barz A.
        • Culkin A.
        • Kris M.G.
        • et al.
        Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study.
        Lancet Oncol. 2006 Nov; 7: 903-909https://doi.org/10.1016/S1470-2045(06)70910-X
        • Gravis G.
        • Marino P.
        • Joly F.
        • Oudard S.
        • Priou F.
        • Esterni B.
        • et al.
        Patients' self-assessment versus investigators' evaluation in a phase III trial in non-castrate metastatic prostate cancer (GETUG-AFU 15).
        Eur J Cancer. 2014 Mar; 50: 953-962https://doi.org/10.1016/j.ejca.2013.11.034
      1. Rammant E, Ost P, Swimberghe M, Vanderstraeten B, Lumen N, Decaestecker K, Bultijnck R, De Meerleer G, Sarrazyn C, Colman R, Fonteyne V. Patient- versus physician-reported outcomes in prostate cancer patients receiving hypofractionated radiotherapy within a randomized controlled trial. Strahlenther Onkol. 2019 May;195(5):393-401. English. doi: 10.1007/s00066-018-1395-y.

      2. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries [published correction appears in CA Cancer J Clin. 2020 Jul;70(4):313]. CA Cancer J Clin. 2018;68(6):394-424. doi:10.3322/caac.21492

        • Litwin M.S.
        • Lubeck D.P.
        • Henning J.M.
        • Carroll P.R.
        Differences in urologist and patient assessments of health related quality of life in men with prostate cancer: results of the CaPSURE database.
        J Urol. 1998; 159: 1988-1992
      3. Seibold P, Webb A, Aguado-Barrera ME, Azria D, Bourgier C, Brengues M, Briers E, Bultijnck R, Calvo-Crespo P, Carballo A, Choudhury A, Cicchetti A, Claßen J, Delmastro E, Dunning AM, Elliott RM, Fachal L, Farcy-Jacquet MP, Gabriele P, Garibaldi E, Gómez-Caamaño A, Gutiérrez-Enríquez S, Higginson DS, Johnson K, Lobato-Busto R, Mollà M, Müller A, Payne D, Peleteiro P, Post G, Rancati T, Rattay T, Reyes V, Rosenstein BS, De Ruysscher D, De Santis MC, Schäfer J, Schnabel T, Sperk E, Symonds RP, Stobart H, Taboada-Valladares B, Talbot CJ, Valdagni R, Vega A, Veldeman L, Ward T, Weißenberger C, West CML, Chang-Claude J; REQUITE consortium. REQUITE: A prospective multicentre cohort study of patients undergoing radiotherapy for breast, lung or prostate cancer. Radiother Oncol. 2019 Sep;138:59-67. doi: 10.1016/j.radonc.2019.04.034.

        • U.S. Department of Health and Human Services
        National Institutes of Health, National Cancer Institute.
        Common Terminology Criteria for Adverse Events (CTCAE) Version. 2009; 4
        • Farnell D.J.
        • Mandall P.
        • Anandadas C.
        • Routledge J.
        • Burns M.P.
        • Logue J.P.
        • et al.
        Development of a patient-reported questionnaire for collecting toxicity data following prostate brachytherapy.
        Radiother Oncol. 2010 Oct; 97: 136-142https://doi.org/10.1016/j.radonc.2010.05.011
        • Aaronson N.K.
        • Ahmedzai S.
        • Bergman B.
        • Bullinger M.
        • Cull A.
        • Duez N.J.
        • et al.
        The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology.
        J Natl Cancer Inst. 1993 Mar 3; 85: 365-376https://doi.org/10.1093/jnci/85.5.365
        • Gwet K.L.
        Computing inter-rater reliability and its variance in the presence of high agreement.
        Br J Math Stat Psychol. 2008; 61: 29-48https://doi.org/10.1348/000711006X126600
      4. Altman DG. Practical Statistics for Medical Research. London: Chapman and Hall, 1991. p404–408

        • Gwet K.L.
        Handbook of inter-rater reliability: the definitive guide to measuring the extent of agreement among multiple raters.
        4th ed. Advanced Analytics Press, Gaithersburg2014
        • Giesinger J.M.
        • Kuijpers W.
        • Young T.
        • Tomaszewski K.A.
        • Friend E.
        • Zabernigg A.
        • et al.
        Thresholds for clinical importance for four key domains of the EORTC QLQ-C30: physical functioning, emotional functioning, fatigue and pain.
        Health Qual Life Outcomes. 2016 Jun; 7: 87https://doi.org/10.1186/s12955-016-0489-4
        • R Core Team
        R: A language and environment for statistical computing.
        R Foundation for Statistical Computing, Vienna, Austria2021
      5. Gwet KL. irrCAC: Computing Chance-Corrected Agreement Coefficients (CAC). 2019. https://CRAN.R-project.org/package=irrCAC

        • Wickham H.
        ggplot2: Elegant Graphics for Data Analysis.
        Springer-Verlag, New York2016
        • Chandwani K.D.
        • Zhao F.
        • Morrow G.R.
        • Deshields T.L.
        • Minasian L.M.
        • Manola J.
        • et al.
        Lack of Patient-Clinician Concordance in Cancer Patients: Its Relation With Patient Variables.
        J Pain Symptom Manage. 2017 Jun; 53: 988-998https://doi.org/10.1016/j.jpainsymman.2016.12.347
        • Atkinson T.M.
        • Rogak L.J.
        • Heon N.
        • Ryan S.J.
        • Shaw M.
        • Stark L.P.
        • et al.
        Exploring differences in adverse symptom event grading thresholds between clinicians and patients in the clinical trial setting.
        J Cancer Res Clin Oncol. 2017 Apr; 143: 735-743https://doi.org/10.1007/s00432-016-2335-9
        • McCammon K.A.
        • Kolm P.
        • Main B.
        • Schellhammer P.F.
        Comparative quality-of-life analysis after radical prostatectomy or external beam radiation for localized prostate cancer.
        Urology. 1999; 54: 509-516https://doi.org/10.1016/s0090-4295(99)00163-6
        • Atkinson T.M.
        • Li Y.
        • Coffey C.W.
        • Sit L.
        • Shaw M.
        • Lavene D.
        • et al.
        Reliability of adverse symptom event reporting by clinicians.
        Qual Life Res. 2012 Sep; 21: 1159-1164https://doi.org/10.1007/s11136-011-0031-4
        • Sonn G.A.
        • Sadetsky N.
        • Presti J.C.
        • Litwin M.S.
        Differing perceptions of quality of life in patients with prostate cancer and their doctors.
        J Urol. 2009; 182: 2296-2302https://doi.org/10.1016/j.juro.2009.07.027
        • Chung E.
        • Brock G.
        Sexual rehabilitation and cancer survivorship: a state of art review of current literature and management strategies in male sexual dysfunction among prostate cancer survivors.
        J Sex Med. 2013; 10: 102-111https://doi.org/10.1111/j.1743-6109.2012.03005.x
        • Schuurhuizen C.S.E.W.
        • Marino P.
        • Braamse A.M.J.
        • Buffart L.M.
        • Joly F.
        • Fizazi K.
        • et al.
        Impact of Patient- and Clinician-Reported Cumulative Toxicity on Quality of Life in Patients With Metastatic Castration-Naïve Prostate Cancer.
        J Natl Compr Canc Netw. 2018 Dec; 16: 1481-1488https://doi.org/10.6004/jnccn.2018.7069
        • Fromme E.K.
        • Eilers K.M.
        • Mori M.
        • Hsieh Y.C.
        • Beer T.M.
        How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30.
        J Clin Oncol. 2004; 22: 3485-3490https://doi.org/10.1200/JCO.2004.03.025
        • D'Amico A.V.
        • Whittington R.
        • Malkowicz S.B.
        • Schultz D.
        • Blank K.
        • Broderick G.A.
        • et al.
        Biochemical outcome after radical prostatectomy, external beam radiation therapy, or interstitial radiation therapy for clinically localized prostate cancer.
        JAMA. 1998 Sep 16; 280: 969-974https://doi.org/10.1001/jama.280.11.969