Brain metastases from NSCLC treated with stereotactic radiotherapy: prediction mismatch between two different radiomic platforms

Published:November 23, 2022DOI:


      • Using different platforms for radiomic extraction affects models’ performance.
      • Variables’ relevance is inconsistent among platforms.
      • MRI features are correlated to radiosurgery response in brain metastases from NSCLC.
      • Higher number of radiomic features does not necessarily imply better performance.


      Background and purpose

      Radiomics enables the mining of quantitative features from medical images. The influence of the radiomic feature extraction software on the final performance of models is still a poorly understood topic. This study aimed to investigate the ability of radiomic features extracted by two different radiomic platforms to predict clinical outcomes in patients treated with radiosurgery for brain metastases from non-small cell lung cancer. We developed models integrating pre-treatment magnetic resonance imaging (MRI)-derived radiomic features and clinical data.

      Materials and methods

      Pre-radiotherapy gadolinium enhanced axial T1-weighted MRI scans were used. MRI images were re-sampled, intensity-shifted, and histogram-matched before radiomic extraction by means of two different platforms (PyRadiomics and SOPHiA Radiomics). We adopted LASSO Cox regression models for multivariable analyses by creating radiomic, clinical, and combined models using three survival clinical endpoints (local control, distant progression, and overall survival). The statistical analysis was repeated 50 times with different random seeds and the median concordance index was used as performance metric of the models.


      We analysed 276 metastases from 148 patients. The use of the two platforms resulted in differences in both the quality and the number of extractable features. That led to mismatches in terms of end-to-end performance, statistical significance of radiomic scores, and clinical covariates found significant in combined models.


      This study shed new light on how extracting radiomic features from the same images using two different platforms could yield several discrepancies. That may lead to acute consequences on drawing conclusions, comparing results across the literature, and translating radiomics into clinical practice.

      Graphical abstract



      ALK (Anaplastic lymphoma kinase), BED (Biologically Effective Dose), BM (Brain Metastasis), C-index (Concordance index), CR (Complete Response), CT (Computed Tomography), DP (Distant Progression), EGFR (Epidermal growth factor receptor), EQD2 (Equivalent dose in 2Gy fractions), GLCM (Gray Level Co-Occurrence Matrix), GLDZM (Gray Level Distance Zone Matrix), GLRLM (Gray Level Run Length Matrix), GLSZM (Gray Level Size Zone Matrix), HR (Hazard Ratio), IBSI (Imaging Biomarker Standardization Initiative), IEO (Istituto Europeo di Oncologia (European Institute of Oncology) IRCCS, Milan, Italy), KM (Kaplan-Meier), KPS (Karnofsky Performance Status), LASSO (Least Absolute Shrinkage and Selection Operator), LC (Local Control), LoG (Laplacian of Gaussian), MRI (Magnetic Resonance Imaging), NGLDM (Neighbourhood Gray Level Dependence Matrix), NGTDM (Neighbouring Gray Tone Difference Matrix), NSCLC (Non-Small Cell Lung Cancer), OS (Overall Survival), PD (Progression Disease), PR (Partial Response), PyR (PyRadiomics), RS (Radiomic Score), RT (Radiotherapy), RTSS (Radiation Therapy Structure Sets), SD (Stable Disease), SR (SOPHiA Radiomics), SRS (Stereotactic Radiosurgery), T1-w (T1-weighted)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Radiotherapy and Oncology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Lambin P.
        • Rios-Velazquez E.
        • Leijenaar R.
        • Carvalho S.
        • van Stiphout R.
        • Granton P.
        Radiomics: Extracting more information from medical images using advanced feature analysis.
        Eur J Cancer. 2012; 48: 441-446
        • Rinaldi L.
        • De Angelis S.P.
        • Raimondi S.
        • Rizzo S.
        • Fanciullo C.
        • Rampinelli C.
        • et al.
        Reproducibility of radiomic features in CT images of NSCLC patients: an integrative analysis on the impact of acquisition and reconstruction parameters.
        European Radiol Exp. 2022; 6: 1-13
        • Midya A.
        • Chakraborty J.
        • Gönen M.
        • Do R.
        • Simpson A.
        Influence of CT acquisition and reconstruction parameters on radiomic feature reproducibility.
        J Med Imaging (Bellingham). 2018; 5: 11-20
        • Ger R.
        • Meier J.
        • Pahlka R.
        Effects of alterations in positron emission tomography imaging parameters on radiomics features.
        PLoS One. 2019; 14: e0221877
        • Wong J.
        • Baine M.
        • Wisnoskie S.
        Effects of interobserver and interdisciplinary segmentation variabilities on CT-based radiomics for pancreatic cancer.
        Sci Rep. 2021; 11
        • Pavic M.
        • Bogowicz M.
        • Würms X.
        • Glatz S.
        • Finazzi T.
        • Riesterer O.
        • et al.
        Influence of inter-observer delineation variability on radiomics stability in different tumor sites.
        Acta Oncol. 2018; 57: 1070-1074
        • Kocher M.
        • Ruge M.
        • Galldiks N.
        • Lohmann P.
        Applications of radiomics and machine learning for radiotherapy of malignant brain tumors.
        Strahlenther Onkol. 2020; 196: 856-867
        • Delgadillo R.
        • Ford J.
        • Abramowitz M.
        • Dal Pra A.
        • Pollack A.
        • Stoyanova R.
        The role of radiomics in prostate cancer radiotherapy.
        Strahlenther Onkol. 2020; 196: 900-912
        • Nardone V.
        • Boldrini L.
        • Grassi R.
        • Franceschini D.
        • Morelli I.
        • Becherini C.
        • et al.
        Radiomics in the setting of neoadjuvant radiotherapy: A new approach for tailored treatment.
        Cancers (Basel). 2021; 13
        • Giannitto C.
        • Marvaso G.
        • Botta F.
        • Raimondi S.
        • Alterio D.
        • Ciardo D.
        Association of quantitative MRI-based radiomic features with prognostic factors and recurrence rate in oropharyngeal squamous cell carcinoma.
        Neoplasma. 2020; 267 (Epub 2020 Aug 13. PMID: 32787435): 1437-1446
        • Gugliandolo S.
        • Pepa M.
        • Isaksson L.
        • Marvaso G.
        • Raimondi S.
        • Botta F.
        • et al.
        MRI-based radiomics signature for localized prostate cancer: a new clinical tool for cancer aggressiveness prediction? Sub-study of prospective phase II trial on ultra-hypofractionated radiotherapy (AIRC IG-13218).
        Eur Radiol. 2021; 231 (Epub 2020 Aug 27. PMID: 32852590): 716-728
        • Peng L.
        • Parekh V.
        • Huang P.
        • Lin D.
        • Sheikh K.
        • Baker B.
        Distinguishing true progression from radionecrosis after stereotactic radiation therapy for brain metastases with machine learning and radiomics.
        Int J Radiat Oncol Biol Phys. 2018; 102: 1236-1243
        • Zhang Z.
        • Yang J.
        • Ho A.
        • Jiang W.
        • Logan J.
        • Wang X.
        A predictive model for distinguishing radiation necrosis from tumour progression after gamma knife radiosurgery based on radiomic features from MR images.
        Eur Radiol. 2018; 28: 2255-2263
        • Ortiz-Ramon R.
        • Larroza A.
        • Arana E.
        • Moratal D.
        A radiomics evaluation of 2D and 3D MRI texture features to classify brain metastases from lung cancer and melanoma.
        Conf Proc IEEE Eng Med Biol Soc. 2017; : 493-496
        • Kniep H.
        • Madesta F.
        • Schneider T.
        • Hanning U.
        • Schonfeld M.
        • Schon G.
        Radiomics of brain MRI: utility in prediction of metastatic tumor type.
        Radiology. 2019; 290: 479-487
        • Qian Z.
        • Li Y.
        • Wang Y.
        • Li L.
        • Li R.
        • Wang K.
        Differentiation of glioblastoma from solitary brain metastases using radiomic machine-learning classifiers.
        Cancer Lett. 2019; 451: 128-135
        • Cha Y.
        • Jang W.
        • Kim M.
        • Yoo H.
        • Paik E.
        • Jeong H.
        Prediction of response to stereotactic radiosurgery for brain metastases using convolutional neural networks.
        Anticancer Res. 2018; 38: 5437-5445
        • Della Seta M.
        • Collettini F.
        • Chapiro J.
        • Angelidis A.
        • Engeling F.
        • Hamm B.
        • Kaul D.
        A 3D quantitative imaging biomarker in pre-treatment MRI predicts overall survival after stereotactic radiation therapy of patients with a singular brain metastasis.
        Acta Radiol. 2019; 60: 1496-1503
        • Huang C.
        • Lee C.
        • Yang H.
        Radiomics as prognostic factor in brain metastases treated with Gamma Knife radiosurgery.
        J Neurooncol. 2020; 146: 439-449
        • Zhang J.
        Computer tomography radiomics-based nomogram in the survival prediction for brain metastases from non-small cell lung cancer underwent whole brain radiotherapy.
        Front Oncol. 2021;
        • Zwanenburg A.
        • Vallières M.
        • Abdalah M.
        • Aerts H.J.W.
        The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping.
        Radiology. 2020; 295: 328-338
        • Foy J.R.
        • Li H.K.R.
        • Giger M.
        • AL-Hallaq H.
        • Armato S.
        Variation in algorithm implementation across radiomics software.
        J Medical Imag (Bellingham, Wash.). 2018; 5
        • Bogowicz M.
        • Leijenaar R.
        • Tanadini-Lang S.
        Post-radiochemotherapy PET radiomics in head and neck cancer – The influence of radiomics implementation on the reproducibility of local control tumor models.
        Radiother Oncol. 2017; 125: 385-391
        • Fornacon-Wood I.
        • Mistry H.
        • Ackermann C.
        Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform.
        Eur Radiol. 2020; 30: 6241-6250
        • Chang Y.
        • Lafata K.
        • Wang C.
        • Duan X.
        • Geng R.
        • Yang Z.
        • Yin F.F.
        Digital phantoms for characterizing inconsistencies among radiomics extraction toolboxes.
        Biomed Phys Eng Express. 2020; 6
        • Van Griethuysen J.
        • Fedorov A.
        • Parmar C.
        • Hosny A.
        • Aucoin N.
        • Narayan V.
        • et al.
        Computational radiomics system to decode the radiographic phenotype.
        Cancer Res. 2017; 77: 104-107
      1. S. GENETICS, «SOPHiA Radiomics v2.1». Technopole Izarbel - 231 Allée Fauste d’Elhuyar, 64210 Bidart, France Brevetto

        • Kaplan E.L.
        • Meier P.
        Nonparametric Estimation from Incomplete Observations.
        J Am Stat Assoc. 1958; 53: 457-481
        • Harrell F.
        • Califf R.
        • Pryor D.
        • Lee K.
        • Rosati R.
        Evaluating the yield of medical tests.
        JAMA. 1982; 247: 2543-2546
        • Ward J.H.
        Hierarchical Grouping to Optimize an Objective Function.
        J Am Stat Assoc. 1963; 58: 236-244
        • Tibshirani R.
        Regression Shrinkage and Selection via the Lasso.
        J Roy Stat Soc. 1996; 58: 267-288
        • Cox D.R.
        Regression models and life tables (with discussion).
        J Roy Stat Soc. 1972; 34: 187-220
        • Depeursinge A.
        • Andrearczyk V.
        • Whybra P.
        • van Griethuysen J.
        • Müller H.
        • Schaer R.
        Standardised convolutional filtering for radiomics.
        arXiv [csCV]. 2020;
        • Kothari G.
        • Korte J.
        • Lehrer E.J.
        • Zaorsky N.G.
        • Lazarakis S.
        • Kron T.
        • et al.
        A systematic review and meta-analysis of the prognostic value of radiomics based models in non-small cell lung cancer treated with curative radiotherapy.
        Radiother Oncol. 2021; 155: 188-203
        • Bettinelli A.
        • Marturano F.
        • Avanzo M.
        • Loi E.
        • Menghi E.
        • Mezzenga E.
        • et al.
        A novel benchmarking approach to assess the agreement among radiomic tools.
        Radiology. 2022; : 533-541