Advertisement

Tracking tumor biology with radiomics: A systematic review utilizing a radiomics quality score

  • Author Footnotes
    1 Both authors contributed equally.
    Sebastian Sanduleanu
    Correspondence
    Corresponding author at: Department of Radiation Oncology, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, Universiteitssingel 40, 6229 ER Maastricht, The Netherlands.
    Footnotes
    1 Both authors contributed equally.
    Affiliations
    Department of Radiation Oncology, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands

    The-D-Lab: Decision Support for Precision Medicine, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands
    Search for articles by this author
  • Author Footnotes
    1 Both authors contributed equally.
    Henry C. Woodruff
    Footnotes
    1 Both authors contributed equally.
    Affiliations
    Department of Radiation Oncology, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands

    The-D-Lab: Decision Support for Precision Medicine, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands

    MAASTRO Clinic, Department of Radiation Oncology, Maastricht, The Netherlands
    Search for articles by this author
  • Evelyn E.C. de Jong
    Affiliations
    Department of Radiation Oncology, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands

    The-D-Lab: Decision Support for Precision Medicine, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands
    Search for articles by this author
  • Janna E. van Timmeren
    Affiliations
    Department of Radiation Oncology, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands

    The-D-Lab: Decision Support for Precision Medicine, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands
    Search for articles by this author
  • Arthur Jochems
    Affiliations
    Department of Radiation Oncology, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands

    The-D-Lab: Decision Support for Precision Medicine, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands
    Search for articles by this author
  • Ludwig Dubois
    Affiliations
    Department of Radiation Oncology, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands

    Department of Radiotherapy, The M-Lab Group, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands
    Search for articles by this author
  • Philippe Lambin
    Affiliations
    Department of Radiation Oncology, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands

    The-D-Lab: Decision Support for Precision Medicine, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands

    Department of Radiotherapy, The M-Lab Group, GROW – School for Oncology and Developmental Biology, Maastricht University Medical Centre+, The Netherlands
    Search for articles by this author
  • Author Footnotes
    1 Both authors contributed equally.
Open AccessPublished:May 18, 2018DOI:https://doi.org/10.1016/j.radonc.2018.03.033

      Abstract

      Introduction: In this review we describe recent developments in the field of radiomics along with current relevant literature linking it to tumor biology. We furthermore explore the methodologic quality of these studies with our in-house radiomics quality scoring (RQS) tool. Finally, we offer our vision on necessary future steps for the development of stable radiomic features and their links to tumor biology. Methods: Two authors (S.S. and H.W.) independently performed a thorough systematic literature search and outcome extraction to identify relevant studies published in MEDLINE/PubMed (National Center for Biotechnology Information, NCBI), EMBASE (Ovid) and Web of Science (WoS). Two authors (S.S, H.W) separately and two authors (J.v.T and E.d.J) concordantly scored the articles for their methodology and analyses according to the previously published radiomics quality score (RQS). Results: In summary, a total of 655 records were identified till 25-09-2017 based on the previously specified search terms, from which n = 236 in MEDLINE/PubMed, n = 215 in EMBASE and n = 204 from Web of Science. After determining full article availability and reading the available articles, a total of n = 41 studies were included in the systematic review. The RQS scoring resulted in some discrepancies between the reviewers, e.g. reviewer H.W scored 4 studies ≥50%, reviewer S.S scored 3 studies ≥50% while reviewers J.v.T and E.d.J scored 1 study ≥50%. Up to nine studies were given a quality score of 0%. The majority of studies were scored below 50%. Discussion: In this study, we performed a systematic literature search linking radiomics to tumor biology. All but two studies (n = 39) revealed that radiomic features derived from ultrasound, CT, PET and/or MR are significantly associated with one or several specific tumor biologic substrates, from somatic mutation status to tumor histopathologic grading and metabolism. Considerable inter-observer differences were found with regard to RQS scoring, while important questions were raised concerning the interpretability of the outcome of such scores.

      Keywords

      Medical images are estimated to account for as much as 90 percent of total stored medical data, with collections of over 30 billion medical images currently available in the United States private sector domain for advanced artificial intelligence pattern analysis []. Medical images such as computed tomography (CT), magnetic resonance imaging (MRI) and positron emission tomography (PET) are obtained for nearly all oncology patients undergoing therapy. Currently, most patients will undergo multiple imaging sessions during the course of their treatment. These digital data are increasingly available in the standard DICOM data format, allowing for easy and consistent analysis.
      Over the past decade the endeavor to transform digital standard of care medical images to mineable high-dimensional data by extracting mathematically quantitative features has been at the forefront of imaging research. Radiomic studies may help to characterize tumor biology in vivo by correlating imaging features with ground truth pathology substrates.
      Although medical images are commonly used by radiologists to describe a pathologic substrate, features extracted in the radiomic process are not part of radiologists’ lexica since they are generally not visible to the naked eye. The hypothesis is that these features provide added relevant tumor-biology related genomic, cellular and metabolic information [
      • Lee G.
      • Lee H.Y.
      • Ko E.S.
      • et al.
      Radiomics and imaging genomics in precision medicine.
      ,
      • Lee G.
      • Lee H.Y.
      • Park H.
      • et al.
      Radiomics and its emerging role in lung cancer research, imaging biomarkers and clinical management: state of the art.
      ]. Features derived from radiomic analysis (e.g., intensity, shape, texture, wavelet, etc.) can provide information that is complementary to other relevant clinical information and are therefore intended to augment currently available clinical decision support systems (CDSS) [
      • Lambin P.
      • van Stiphout R.G.
      • Starmans M.H.
      • et al.
      Predicting outcomes in radiation oncology–multifactorial decision support systems.
      ,
      • Lambin P.
      • Roelofs E.
      • Reymen B.
      • et al.
      Rapid Learning health care in oncology’ – an approach towards decision support systems enabling customised radiotherapy’.
      ,
      • Lambin P.
      • Zindler J.
      • Vanneste B.
      • et al.
      Modern clinical research: How rapid learning health care and cohort multiple randomised clinical trials complement traditional evidence based medicine.
      ,
      • Lambin P.
      • Zindler J.
      • Vanneste B.G.
      • et al.
      Decision support systems for personalized and participative radiation oncology.
      ]. The final goal is to improve personalized medical decision-making (see Fig. 1): establishing reliable models that can stratify therapy outcomes and can be used for assessing patients’ personal trade-off between the risks and benefits of different treatment options [
      • Gani C.
      • Bonomo P.
      • Zwirner K.
      • et al.
      Organ preservation in rectal cancer – challenges and future strategies.
      ,
      • Skripcak T.
      • Belka C.
      • Bosch W.
      • et al.
      Creating a data exchange strategy for radiotherapy research: towards federated databases and anonymised public datasets.
      ,
      • O’Connor J.P.B.
      Radiomics: rethinking the role of clinical imaging.
      ].
      Figure thumbnail gr1
      Fig. 1From top to bottom. (a) Intra-tumor heterogeneity, the rapid evolution of tumors over time and in response to therapies, and the potential sampling bias involved in a single tumor biopsy (current model) limits the efficient identification and qualification of biomarkers for clinical use. Radiomics can alleviate these problems through ‘virtual biopsies’ at multiple sites and time points. (b) Radiomics potentially encapsulates information from the anatomic to the genomic level.
      In this review recent developments in the field of radiomics are described, along with relevant literature linking it to tumor biology. Metrics for internal/external validity and completeness of the radiomic workflow have been calculated for these studies with our in-house developed radiomics quality score (RQS) tool [
      • Lambin P.
      • Leijenaar R.T.H.
      • Deist T.M.
      • et al.
      Radiomics: the bridge between medical imaging and personalized medicine.
      ]. Introducing this tool highlights the methodology upon which inference was drawn with regard to the link between radiomics and biology. General comments and recommendations for the improvement of this tool are presented. Finally, we offer our vision on necessary future steps for the development of stable and robust radiomic biomarkers and their link to tumor biology.

      Materials and methods

      Systematic search strategy

      The research question for this systematic review is described as: “What are the known studies linking radiomics to tumor biology and what is the methodological quality of these studies?”
      We conducted a comprehensive literature search to identify relevant studies published in MEDLINE/PubMed (National Center for Biotechnology Information, NCBI), EMBASE (Ovid) and Web of Science (WoS) until 25-09-2017. We used a search string containing free-text and/or Medical Subject Headings (MeSH) search of 3 key search terms: ‘biology’, ‘neoplasms’ and ‘radiomics’ while search terms such as ‘imaging features’ were not used as our purpose was to review manuscripts using more recent, high throughput technology. Details on the search method are shown in the Supplementary Appendix.

      Study selection

      We included articles satisfying the following inclusion criteria: (1) articles were full-text and written in English, (2) the study population consisted of humans or animals with acquired tissue samples or biologic biomarkers, and (3) radiomic analysis was performed on mammography, ultrasound, CT, PET or MR images. Duplicate findings were discarded to ensure that no data overlap occurred. Further selection was performed by applying the following exclusion criteria: (1) studies not linking fundamental biologic substrate (e.g., blood biomarkers) to radiomics, and (2) case-reports, (systematic) reviews, and expert opinion papers.

      Data extraction

      Two authors (S.S. and H.W.) independently performed a thorough systematic literature search and outcome extraction. From the included articles, data concerning study characteristics was extracted and tabulated (author, publication year, study population, radiomic feature extraction, main results and conclusion) together with measurement characteristics (i.e., method for quantifying biological mechanism). Furthermore, most relevant statistical results (e.g., p-values, AUC, c-index) were extracted and summarized. p-Values <0.05 were considered indicative of statistical significance.
      S.S., H.W. separately and E.d.J. and J.v.T. concordantly scored the articles for their methodology according to the radiomics quality score (RQS) [
      • Lambin P.
      • Leijenaar R.T.H.
      • Deist T.M.
      • et al.
      Radiomics: the bridge between medical imaging and personalized medicine.
      ], consisting of sixteen key components (e.g., robust segmentation, use of standardized imaging protocols, use of validation dataset); each component was assigned a number of points corresponding to its importance as shown in Table 1. Agreement between the reviewers was assessed by means of a modified Fleiss kappa statistic (linear weights, 1000 iterations for the percentile Bootstrap and for the Monte Carlo test, alpha = 0.05) in R-studio for Windows (R version 1.0.136, Boston). Although the RQS serves as a guideline to evaluate the radiomic workflow presented in a given publication for internal consistency, clinical relevance and applicability, and reproducibility, it neither completely reveals the methodologic quality nor does it reflect the overall quality or importance of the research and as such it should not replace the reader’s scientific appraisal. In the context of this review it is used as an example for a protocol that can be applied when trying to assess the reliability of correlations, classifiers and predictors built using radiomics as their basis.
      Table 1Radiomics quality score as described in earlier work
      • Lambin P.
      • Leijenaar R.T.H.
      • Deist T.M.
      • et al.
      Radiomics: the bridge between medical imaging and personalized medicine.
      . A radiomic study can achieve a total of 36 points maximum. Higher scores provide an indication of higher quality research.
      CriteriaPoints
      1Image protocol quality – well-documented image protocols (e.g., contrast, slice thickness, energy, etc.) and/or usage of public image protocols allow reproducibility/replicability+1 (if protocols are well-documented)

      +1 (if public protocol is used)
      2Multiple segmentations – possible actions are: segmentation by different physicians/algorithms/software, perturbing segmentations by (random) noise, segmentation at different breathing cycles. Analyze feature robustness to segmentation variabilities+1
      3Phantom study on all scanners – detect inter-scanner differences and vendor-dependent features. Analyze feature robustness to these sources of variability+1
      4Imaging at multiple time points – collect individuals’ images at additional time points. Analyze feature robustness to temporal variabilities (e.g., organ movement, organ expansion/shrinkage).+1
      5Feature reduction or adjustment for multiple testing – decreases the risk of overfitting. Overfitting is inevitable if the number of features exceeds the number of samples. Consider feature robustness when selecting features−3 (if neither measure is implemented)

      +3 (if either measure is implemented)
      6Multivariable analysis with non radiomic features (e.g., EGFR mutation) – is expected to provide a more holistic model. Permits correlating/inferencing between radiomics and non radiomics features+1
      7Detect and discuss biological correlates – demonstration of phenotypic differences (possibly associated with underlying gene–protein expression patterns) deepens understanding of radiomics and biology+1
      8Cut-off analyses – determine risk groups by either the median, a previously published cut-off or report a continuous risk variable. Reduces the risk of reporting overly optimistic results+1
      9Discrimination statistics – report discrimination statistics (e.g., C-statistic, ROC curve, AUC) and their statistical significance (e.g., p-values, confidence intervals). One can also apply resampling method (e.g., bootstrapping, cross-validation)+1 (if a discrimination statistic and its statistical significance are reported)

      +1 (if also an resampling method technique is applied)
      10Calibration statistics – report calibration statistics (e.g., Calibration-in-the-large/slope, calibration plots) and their statistical significance (e.g., p-values, confidence intervals). One can also apply resampling method (e.g., bootstrapping, cross-validation)+1 (if a calibration statistic and its statistical significance are reported)

      +1 (if also an resampling method technique is applied)
      11Prospective study registered in a trial database – provides the highest level of evidence supporting the clinical validity and usefulness of the radiomics biomarker+7 (for prospective validation of a radiomics signature in an appropriate trial)
      12Validation – the validation is performed without retraining and without adaptation of the cut-off value, provides crucial information with regard to credible clinical performance−5 (if validation is missing)

      +2 (if validation is based on a dataset from the same institute)

      +3 (if validation is based on a dataset from another institute)

      +4 (if validation is based on two datasets from two distinct institutes)

      +4 (if the study validates a previously published signature)

      +5 (if validation is based on three or more datasets from distinct institutes)

      *Datasets should be of comparable size and should have at least 10 events per model feature.
      13Comparison to ‘gold standard’ – assess the extent to which the model agrees with/is superior to the current ‘gold standard’ method (e.g., TNM-staging for survival prediction). This comparison shows the added value of radiomics+2
      14Potential clinical utility – report on the current and potential application of the model in a clinical setting (e.g., decision curve analysis)+2
      15Cost-effectiveness analysis – report on the cost-effectiveness of the clinical application (e.g., quality adjusted life years generated)+1
      16Open science and data – make code and data publicly available. Open science facilitates knowledge transfer and reproducibility of the study+1 (if scans are open source)

      +1 (if region of interest segmentations are open source)

      +1 (if code is open source)

      +1 (if radiomics features are calculated on a set of representative ROIs and the calculated features + representative ROIs are open source)
      Total points (36 = 100%)
      Abbreviations: AUC: area under the curve, EGFR: epidermal growth factor receptor.

      Results

      In summary (see Fig. 2), a total of 655 records were identified until 25-09-2017 based on the previously specified search terms, from which n = 236 in MEDLINE/PubMed, n = 215 in EMBASE and n = 204 from Web of Science. After imposing language-restrictions and removing duplicate findings, 450 articles remained. Further screening by reading the abstracts and applying the outlined inclusion criteria resulted in 78 potentially eligible studies. After determining full article (open source as well as request print) availability and reading the available articles taking into account the exclusion criteria set out for this systematic review, a total of 41 studies were included, while 37 articles were eliminated. A tabulated view of the principal results, conclusion and RQS score per component and rater for these studies is provided in Supplementary Table 1 and Fig. 3a–c. A brief summary of the mean RQS scores between reviewers J.v.T. and E.d.J., H.W. and S.S. and the biological correlates, and if mentioned the microbiologic technique, is presented in Table 2. The inter-observer agreement with upper/lower level confidence interval is presented per RQS component in Fig. 4.
      Figure thumbnail gr3a
      Fig. 3(a–c) Heat map of scoring per individual RQS component for raters (a) J.v.T. and E.d.J, (b) H.W (c) S.S.
      Figure thumbnail gr3b
      Fig. 3(a–c) Heat map of scoring per individual RQS component for raters (a) J.v.T. and E.d.J, (b) H.W (c) S.S.
      Figure thumbnail gr3c
      Fig. 3(a–c) Heat map of scoring per individual RQS component for raters (a) J.v.T. and E.d.J, (b) H.W (c) S.S.
      Table 2Summary of systematic review with RQS-scoring.
      Study (Ref.)ModalityDiseaseBiological correlate(Mean) RQS score
      Guo et al.
      • Guo W.T.
      • Li H.
      • Zhu Y.T.
      • et al.
      Prediction of clinical phenotypes in invasive breast carcinomas from the integration of radiomics and genomics data.
      MRBCA genomic dataset of 144 genomic features for 70 genes, including 70 gene expression features (normalized read counts of RNA-seq data), 70 copy number (Affymetrix Genome-Wide Human SNP Array 6.0) features, and 4 methylation features (Infinium HumanMethylation450 BeadChip)18.5%
      Grossmann et al.
      • Grossmann P.
      • Stringfield O.
      • El-Hachem N.
      • et al.
      Defining the biological basis of radiomic phenotypes in lung cancer.
      CTLCGene expression of 21,766 unique genes (custom Rosetta/Merck

      Affymetrix 2.0 microarray chipset). Immunohistochemical staining for CD3, a T-cell co-receptor and ReIA (p65)
      38.9%
      Aerts et al.
      • Aerts H.J.W.L.
      • Velazquez E.R.
      • Leijenaar R.T.H.
      • et al.
      Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach.
      CTNSCLC

      HN
      Gene expression: affymetrix chips with the custom chipset HuRSTA_2a520709 for 21,766 genes55.6%
      Vargas et al.
      • Vargas H.A.
      • Veeraraghavan H.
      • Micco M.
      • et al.
      A novel representation of inter-site tumour heterogeneity from pre-treatment computed tomography textures classifies ovarian cancers by clinical outcome.
      CTHGSOCAmplification of 19q12 involving cyclin E1 gene (CCNE1)17.6%
      Velazquez et al.
      • Velazquez E.R.
      • Parmar C.
      • Liu Y.
      • et al.
      Somatic mutations drive distinct imaging phenotypes in lung cancer.
      CTLACEGFR and KRAS mutation status50.0%
      Li et al.
      • Li H.
      • Zhu Y.
      • Burnside E.S.
      • et al.
      MR imaging radiomics signatures for predicting the risk of breast cancer recurrence as given by research versions of mammaprint, oncotype DX, and PAM50 gene assays.
      MRBCCore biopsy assessment of expression of ER, PR, HER2. MammaPrint (70 gene), Oncotype DX (21 gene), and PAM50 (50 gene) assays23.1%
      Yu et al.
      • Yu J.
      • Shi Z.
      • Ji C.
      • et al.
      Anatomical location differences between mutated and wild-type isocitrate dehydrogenase 1 in low-grade gliomas.
      MRLGGParaffin slide PCR assessment of Isocitrate Dehydrogenase 1 (IDH1) status1.9%
      Grossmann et al.
      • Grossmann P.
      • Gutman D.A.
      • Dunn W.D.
      • et al.
      Imaging-genomics reveals driving pathways of MRI derived volumetric tumor phenotype features in Glioblastoma.
      MRGBMGene expression (mRNA): 583 gene sets containing at least 15 and at most 500 genes11.1%
      Lee et al.
      • Lee J.
      • Narang S.
      • Martinez J.J.
      • et al.
      Associating spatial diversity features of radiologically defined tumor habitats with epidermal growth factor receptor driver status and 12-month survival in glioblastoma: methods and preliminary investigation.
      MRGBMGene expression: EGFR mutation and copy number status10.2%
      Yu et al.
      • Yu J.
      • Shi Z.
      • Lian Y.
      • et al.
      Noninvasive IDH1 mutation estimation based on a quantitative radiomics approach for grade II glioma.
      MRLGGParaffin slide PCR assessment of Isocitrate Dehydrogenase 1 (IDH1) status37.9%
      Yip et al.
      • Yip S.S.F.
      • Kim J.
      • Coroner T.P.
      • et al.
      Associations between somatic mutations and metabolic imaging phenotypes in non-small cell lung cancer.
      FDG-PETNSCLCTissue samples of primary tumors through biopsy or surgical resection. Somatic mutations were tested using a PCR–based method (EGFR, KRAS etc.) or PROFILE Oncomap (mass spectrometry genotyping technique analyzing > than 470 unique mutations in 41 oncogenes)15.8%
      Gevaert et al.
      • Gevaert O.
      • Echegaray S.
      • Khuong A.
      • et al.
      Predictive radiogenomics modeling of EGFR mutation status in lung cancer.
      CTLCTumor histopathologic subtype. Mutation testing was done for both EGFR and KRAS using multiplex PCR20.3%
      Yoon et al.
      • Yoon H.J.
      • Sohn I.
      • Cho J.H.
      • et al.
      Decoding tumor phenotypes for ALK, ROS1, and RET fusions in lung adenocarcinoma using a radiomics approach.
      FDG-PET

      CT
      LACTumor histopathologic subtype. Molecular analysis: genomic DNA or RNA extracted from lung tumors using standard protocols (RNeasyMini Kit and QiAamp DNAMini Kit, Qiagen, Hilden, Germany). ALK, ROS1, and RET fusion assay using nCounterTM gene expression assays23.3%
      Emaminejad et al.
      • Emaminejad N.
      • Qian W.
      • Guan Y.B.
      • et al.
      Fusion of quantitative image and genomic biomarkers to improve prognosis assessment of early stage lung cancer patients.
      CTNSCLCTwo genomic biomarkers, ERCC1 and RRM1, were evaluated using the selected tumor specimen and a standard IHC-based analytic method22.2%
      Panth et al.
      • Panth K.M.
      • Leijenaar R.T.H.
      • Carvalho S.
      • et al.
      Is there a causal relationship between genetic changes and radiomics-based image features? An in vivo preclinical experiment with doxycycline inducible GADD34 tumor cells.
      CTCRCGADD34 gene expression15.7%
      Aerts et al.
      • Aerts H.J.
      • Grossmann P.
      • Tan Y.
      • et al.
      Defining a radiomic response phenotype: a pilot study using targeted therapy in NSCLC.
      CTNSCLCGenomic DNA from snap frozen tumor resection specimens analyzed for most common EGFR-sensitizing mutations (exons 19 and 21) with PCR-based methods. EGFR wild-type (WT) tumors were also tested for KRAS mutations39.8%
      Ghosh et al.
      • Ghosh P.
      • Tamboli P.
      • Vikram R.
      • et al.
      Imaging-genomic pipeline for identifying gene mutations using three-dimensional intra-tumor heterogeneity features.
      CTccRCCBAP1 mutation status18.5%
      Hanania et al.
      • Hanania A.N.
      • Bantis L.E.
      • Feng Z.D.
      • et al.
      Quantitative imaging to evaluate malignant potential of IPMNs.
      CTIPMNQuantitative analysis of resected specimens of the pancreatic cysts and pancreas parenchyma to differentiate high grade from low grade lesions14.8%
      Suo et al.
      • Suo S.T.
      • Cheng J.J.
      • Cao M.Q.
      • et al.
      Assessment of heterogeneity difference between edge and core by using texture analysis: differentiation of malignant from inflammatory pulmonary nodules and masses.
      CTLCSurgical resection specimens, transbronchial lung biopsy, CT-guided percutaneous biopsy or clinical examination and therapy to differentiate between benign and malignant pulmonary nodules/masses8.3%
      Bae et al.
      • Bae J.M.
      • Jeong J.Y.
      • Lee H.Y.
      • et al.
      Pathologic stratification of operable lung adenocarcinoma using radiomics features extracted from dual energy ct images.
      CTLACHistopathologic tumor grade assessment based on specimen from complete resection35.2%
      Pena et al.
      • Pena E.
      • Ojiaku M.
      • Inacio et al., J.R.
      Can CT and MR shape and textural features differentiate benign versus malignant pleural lesions?.
      CT

      MR
      mLAC

      mPM
      Histopathologic analysis malignancy12.9%
      Ginsburg et al.
      • Ginsburg S.B.
      • Algohary A.
      • Pahwa S.
      • et al.
      Radiomic features for prostate cancer detection on MRI differ between the transition and peripheral zones: preliminary findings from a multi-institutional study.
      MRPCHistopathologic Gleason tumor grading29.6%
      Bickelhaupt et al.
      • Bickelhaupt S.
      • Paech D.
      • Kickingereder P.
      • et al.
      Prediction of malignancy by a radiomic signature from contrast agent-free diffusion MRI in suspicious breast lesions found on screening mammography.
      MRBCTumor histopathologic grade25.9%
      Permuth et al.
      • Permuth J.B.
      • Choi J.
      • Balarunathan Y.
      • et al.
      Combining radiomic features with a miRNA classifier may improve prediction of malignant pathology for pancreatic intraductal papillary mucinous neoplasms.
      CTIPMNTumor histopathologic grade. 800 miRNAs from archived plasma using Nanostring’s nCounter digital technology12.0%
      Zhang et al.
      • Zhang X.
      • Xu X.
      • Tian Q.
      • et al.
      Radiomics assessment of bladder cancer grade using texture features from diffusion-weighted imaging.
      MRBCTumor histopathologic grade.17.6%
      Guo et al.
      • Guo Y.
      • Hu Y.
      • Qiao M.
      • et al.
      Radiomics analysis on ultrasound for prediction of biologic behavior in breast invasive ductal carcinoma.
      UltrasoundBCTumor histopathologic grade (Nottingham). Tumor staining with hematoxylin-eosin and examined in formalin-fixed, paraffin-embedded material. Expression of ER, PR, HER2, and Ki-67 was assessed by IHC analysis17.6%
      Song et al.
      • Song S.H.
      • Park H.
      • Lee H.
      • et al.
      Imaging phenotyping using radiomics to predict micropapillary pattern within lung adenocarcinoma.
      CTLACTumor histologic subtype.33. 3%
      Patil et al.
      • Patil R.
      • Mahadevaiah G.
      • Dekker A.
      An approach toward automatic classification of tumor histopathology of non-small cell lung cancer based on radiomic features.
      CTNSCLCTumor histopathologic subtype.0%
      Parmar et al.
      • Parmar C.
      • Leijenaar R.T.
      • Grossmann P.
      • et al.
      Radiomic feature clusters and prognostic signatures specific for lung and head & neck cancer.
      CTNSCLC

      HNSCC
      Tumor histopathologic subtype. HPV status.38.9%
      Lin et al.
      • Lin Y.C.
      • Lin G.
      • Hong J.H.
      • et al.
      Diffusion radiomics analysis of intratumoral heterogeneity in a murine prostate cancer model following radiotherapy: pixelwise correlation with histology.
      MRPCH&E staining of snap-frozen tumor sections in liquid nitrogen9.3%
      Zhang et al.
      • Zhang Q.
      • Xiao Y.
      • Suo J.
      • et al.
      Sonoelastomics for breast tumor classification: a radiomics approach with clustering-based feature selection on sonoelastography.
      UltrasoundBCCore biopsy or fine-needle aspiration cytology for histopathologic assessment32.4%
      Marin et al.
      • Marin Z.
      • Batchelder K.A.
      • Toner B.C.
      • et al.
      Mammographic evidence of microenvironment changes in tumorous breasts.
      MammographyBCTumor microenvironment5.6%
      Coquery et al.
      • Coquery N.
      • Francois O.
      • Lemasson B.
      • et al.
      Microvascular MRI and unsupervised clustering yields histology-resembling images in two rat models of glioma.
      MRLGGImmunohistochemical analysis of coronal cryosections. Hematoxylin–erythrosine staining performed to characterize tumor cell density. Tumor hypoxia was determined using the Hypoxyprobe-1 kit (pimonidazole detection)29.6%
      Choi et al.
      • Choi E.R.
      • Lee H.Y.
      • Jeong J.Y.
      • et al.
      Quantitative image variables reflect the intratumoral pathologic heterogeneity of lung adenocarcinoma.
      CTLACTumor histopathologic subtype37.0%
      Wu et al.
      • Wu W.M.
      • Parmar C.
      • Grossmann P.
      • et al.
      Exploratory study to identify radiomics classifiers for lung cancer histology.
      CTLCTumor histopathologic subtype38.9%
      Li et al.
      • Li H.
      • Zhu Y.
      • Burnside E.S.
      • et al.
      Quantitative MRI radiomics in the prediction of molecular classifications of breast cancer subtypes in the TCGA/TCIA data set.
      MRBCImmunohistochemical molecular classification/subtyping (ER, PR, EGFR2, luminal A/B, Her2) based on tumor biopsy15.7%
      Gnep et al.
      • Gnep K.
      • Fargeas A.
      • Gutierrez-Carvajal R.E.
      • et al.
      Haralick textural features on T2-weighted MRI are associated with biochemical recurrence following radiotherapy for peripheral zone prostate cancer.
      MRPCPSA level analysis22.2%
      Wang et al.
      • Wang J.
      • Kato F.
      • Oyama-Manabe N.
      • et al.
      Identifying triple-negative breast cancer using background parenchymal enhancement heterogeneity on dynamic contrast-enhanced MRI: a pilot radiomics study.
      MRBCExpression of ER, PR, HER2 and Ki67 by immunohistochemical analysis of tumor specimens20.4%
      Yin et al.
      • Yin Q.
      • Hung S.C.
      • Wang L.
      • et al.
      Associations between tumor vascularity, vascular endothelial growth factor expression and PET/MRI radiomic signatures in primary clear-cell-renal-cell-carcinoma: proof-of-concept study.
      FDG-PET

      MR
      ccRCCTumor samples portioned for snap freezing in liquid nitrogen/ formalin fixed and paraffin embedded. Definiens Tissue Studio software used to measure microvascular density (MVD) in CD31 stained slides21.3%
      Ha et al.
      • Ha S.
      • Park S.
      • Bang et al., J.
      Characterize locally advanced breast cancer: histopathologic characteristics, response to neoadjuvant chemotherapy, and prognosis.
      FDG-PET

      CT
      LABCTumor immunohistochemical parameters, including Ki67, ER, PgR, and HER2 were assessed. Metabolic radiomics was assessed from pretreatment 18F-FDG PET/CT scans15.7%
      Lopez et al.
      • Lopez C.J.
      • Nagornaya N.
      • Parra N.A.
      • et al.
      Association of radiomics and metabolic tumor volumes in radiation treatment of glioblastoma multiforme.
      MRGBMMagnetic resonance spectroscopy based volumetric maps of N-acetyl aspartate (NAA) and choline (Cho) metabolic tumor volumes (MTV)8.3%
      NSCLC-non small cell lung cancer; HN-head and neck cancer; BC-breast cancer (LA = locally advanced); HGSOC-high grade serous ovarian cancer; LGG-low grade glioma; PC-prostate cancer; LAC-lung adenocarcinoma (m = metastatic); GBM-glioblastoma multiforme; PM-pleural mesothelioma (m = malignant); ccRCC-clear cell renal cell carcinoma; IPMN-intraductal papillary mucinous neoplasms; LC-lung cancer; HNSCC-head and neck squamous cell carcinoma; BC-bladder cancer; CRC-colorectal carcinoma.
      Although many correlations between radiomics and specimen biology are weak, all but one study [
      • Guo W.T.
      • Li H.
      • Zhu Y.T.
      • et al.
      Prediction of clinical phenotypes in invasive breast carcinomas from the integration of radiomics and genomics data.
      ] reported that at least one radiomic model was significantly associated with one or several specific biologic mechanisms. In total 38 studies were on clinical cancer patients and three on preclinical data. Occasionally multiple links between radiomics and biology were addressed, e.g., histopathological grading and gene expression. For instance, Grossmann et al. identified and independently validated thirteen radiomic-pathway modules with coherent expression patterns, eleven of which were significantly associated with overall survival, stage, or histology [
      • Grossmann P.
      • Stringfield O.
      • El-Hachem N.
      • et al.
      Defining the biological basis of radiomic phenotypes in lung cancer.
      ]. By basing these clinical associations exclusively on radiomic features, they could demonstrate that the associated molecular pathways robustly matched radiomic-based hypotheses. For example Laplace of Gaussian intensity standard deviation, wavelet intensity variance, and wavelet texture entropy were prognostic and associated with staging. These features were highly enriched (pathway analysis) for immune system, p53, and cell-cycle regulation pathways, biological processes that are widely recognized to play key roles in lung cancer.
      A total of 17 studies established a correlation between cancer imaging features and a certain gene expression or gene mutation, a process called radiogenomics [
      • Guo W.T.
      • Li H.
      • Zhu Y.T.
      • et al.
      Prediction of clinical phenotypes in invasive breast carcinomas from the integration of radiomics and genomics data.
      ,
      • Grossmann P.
      • Stringfield O.
      • El-Hachem N.
      • et al.
      Defining the biological basis of radiomic phenotypes in lung cancer.
      ,
      • Aerts H.J.W.L.
      • Velazquez E.R.
      • Leijenaar R.T.H.
      • et al.
      Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach.
      ,
      • Vargas H.A.
      • Veeraraghavan H.
      • Micco M.
      • et al.
      A novel representation of inter-site tumour heterogeneity from pre-treatment computed tomography textures classifies ovarian cancers by clinical outcome.
      ,
      • Velazquez E.R.
      • Parmar C.
      • Liu Y.
      • et al.
      Somatic mutations drive distinct imaging phenotypes in lung cancer.
      ,
      • Li H.
      • Zhu Y.
      • Burnside E.S.
      • et al.
      MR imaging radiomics signatures for predicting the risk of breast cancer recurrence as given by research versions of mammaprint, oncotype DX, and PAM50 gene assays.
      ,
      • Yu J.
      • Shi Z.
      • Ji C.
      • et al.
      Anatomical location differences between mutated and wild-type isocitrate dehydrogenase 1 in low-grade gliomas.
      ,
      • Grossmann P.
      • Gutman D.A.
      • Dunn W.D.
      • et al.
      Imaging-genomics reveals driving pathways of MRI derived volumetric tumor phenotype features in Glioblastoma.
      ,
      • Lee J.
      • Narang S.
      • Martinez J.J.
      • et al.
      Associating spatial diversity features of radiologically defined tumor habitats with epidermal growth factor receptor driver status and 12-month survival in glioblastoma: methods and preliminary investigation.
      ,
      • Yu J.
      • Shi Z.
      • Lian Y.
      • et al.
      Noninvasive IDH1 mutation estimation based on a quantitative radiomics approach for grade II glioma.
      ,
      • Yip S.S.F.
      • Kim J.
      • Coroner T.P.
      • et al.
      Associations between somatic mutations and metabolic imaging phenotypes in non-small cell lung cancer.
      ,
      • Gevaert O.
      • Echegaray S.
      • Khuong A.
      • et al.
      Predictive radiogenomics modeling of EGFR mutation status in lung cancer.
      ,
      • Yoon H.J.
      • Sohn I.
      • Cho J.H.
      • et al.
      Decoding tumor phenotypes for ALK, ROS1, and RET fusions in lung adenocarcinoma using a radiomics approach.
      ,
      • Emaminejad N.
      • Qian W.
      • Guan Y.B.
      • et al.
      Fusion of quantitative image and genomic biomarkers to improve prognosis assessment of early stage lung cancer patients.
      ,
      • Panth K.M.
      • Leijenaar R.T.H.
      • Carvalho S.
      • et al.
      Is there a causal relationship between genetic changes and radiomics-based image features? An in vivo preclinical experiment with doxycycline inducible GADD34 tumor cells.
      ,
      • Aerts H.J.
      • Grossmann P.
      • Tan Y.
      • et al.
      Defining a radiomic response phenotype: a pilot study using targeted therapy in NSCLC.
      ,
      • Ghosh P.
      • Tamboli P.
      • Vikram R.
      • et al.
      Imaging-genomic pipeline for identifying gene mutations using three-dimensional intra-tumor heterogeneity features.
      ]. Importantly, a causal relationship rather than a correlation was demonstrated between genetic changes and image features, reported by Panth et al. [
      • Panth K.M.
      • Leijenaar R.T.H.
      • Carvalho S.
      • et al.
      Is there a causal relationship between genetic changes and radiomics-based image features? An in vivo preclinical experiment with doxycycline inducible GADD34 tumor cells.
      ]. In this preclinical study gene induction with doxycycline (dox+) and irradiation translated into significant changes in radiomic features in GADD34 mouse xenografts (between the dox+ and irradiation groups 8 and 4 of these features remained consistent for 40 and 80kVp, respectively).
      A total of nine studies associated radiomic features with tumor histopathological grading/malignancy prediction [
      • Hanania A.N.
      • Bantis L.E.
      • Feng Z.D.
      • et al.
      Quantitative imaging to evaluate malignant potential of IPMNs.
      ,
      • Suo S.T.
      • Cheng J.J.
      • Cao M.Q.
      • et al.
      Assessment of heterogeneity difference between edge and core by using texture analysis: differentiation of malignant from inflammatory pulmonary nodules and masses.
      ,
      • Bae J.M.
      • Jeong J.Y.
      • Lee H.Y.
      • et al.
      Pathologic stratification of operable lung adenocarcinoma using radiomics features extracted from dual energy ct images.
      ,
      • Pena E.
      • Ojiaku M.
      • Inacio et al., J.R.
      Can CT and MR shape and textural features differentiate benign versus malignant pleural lesions?.
      ,
      • Ginsburg S.B.
      • Algohary A.
      • Pahwa S.
      • et al.
      Radiomic features for prostate cancer detection on MRI differ between the transition and peripheral zones: preliminary findings from a multi-institutional study.
      ,
      • Bickelhaupt S.
      • Paech D.
      • Kickingereder P.
      • et al.
      Prediction of malignancy by a radiomic signature from contrast agent-free diffusion MRI in suspicious breast lesions found on screening mammography.
      ,
      • Permuth J.B.
      • Choi J.
      • Balarunathan Y.
      • et al.
      Combining radiomic features with a miRNA classifier may improve prediction of malignant pathology for pancreatic intraductal papillary mucinous neoplasms.
      ,
      • Zhang X.
      • Xu X.
      • Tian Q.
      • et al.
      Radiomics assessment of bladder cancer grade using texture features from diffusion-weighted imaging.
      ,
      • Guo Y.
      • Hu Y.
      • Qiao M.
      • et al.
      Radiomics analysis on ultrasound for prediction of biologic behavior in breast invasive ductal carcinoma.
      ]. Another 15 studies explored the correlation between imaging features and a specific histological substrate, e.g., hormone receptor status or microvascular density (MVD) [
      • Guo Y.
      • Hu Y.
      • Qiao M.
      • et al.
      Radiomics analysis on ultrasound for prediction of biologic behavior in breast invasive ductal carcinoma.
      ,
      • Song S.H.
      • Park H.
      • Lee H.
      • et al.
      Imaging phenotyping using radiomics to predict micropapillary pattern within lung adenocarcinoma.
      ,
      • Patil R.
      • Mahadevaiah G.
      • Dekker A.
      An approach toward automatic classification of tumor histopathology of non-small cell lung cancer based on radiomic features.
      ,
      • Parmar C.
      • Leijenaar R.T.
      • Grossmann P.
      • et al.
      Radiomic feature clusters and prognostic signatures specific for lung and head & neck cancer.
      ,
      • Lin Y.C.
      • Lin G.
      • Hong J.H.
      • et al.
      Diffusion radiomics analysis of intratumoral heterogeneity in a murine prostate cancer model following radiotherapy: pixelwise correlation with histology.
      ,
      • Zhang Q.
      • Xiao Y.
      • Suo J.
      • et al.
      Sonoelastomics for breast tumor classification: a radiomics approach with clustering-based feature selection on sonoelastography.
      ,
      • Marin Z.
      • Batchelder K.A.
      • Toner B.C.
      • et al.
      Mammographic evidence of microenvironment changes in tumorous breasts.
      ,
      • Coquery N.
      • Francois O.
      • Lemasson B.
      • et al.
      Microvascular MRI and unsupervised clustering yields histology-resembling images in two rat models of glioma.
      ,
      • Choi E.R.
      • Lee H.Y.
      • Jeong J.Y.
      • et al.
      Quantitative image variables reflect the intratumoral pathologic heterogeneity of lung adenocarcinoma.
      ,
      • Wu W.M.
      • Parmar C.
      • Grossmann P.
      • et al.
      Exploratory study to identify radiomics classifiers for lung cancer histology.
      ,
      • Li H.
      • Zhu Y.
      • Burnside E.S.
      • et al.
      Quantitative MRI radiomics in the prediction of molecular classifications of breast cancer subtypes in the TCGA/TCIA data set.
      ,
      • Gnep K.
      • Fargeas A.
      • Gutierrez-Carvajal R.E.
      • et al.
      Haralick textural features on T2-weighted MRI are associated with biochemical recurrence following radiotherapy for peripheral zone prostate cancer.
      ,
      • Wang J.
      • Kato F.
      • Oyama-Manabe N.
      • et al.
      Identifying triple-negative breast cancer using background parenchymal enhancement heterogeneity on dynamic contrast-enhanced MRI: a pilot radiomics study.
      ,
      • Yin Q.
      • Hung S.C.
      • Wang L.
      • et al.
      Associations between tumor vascularity, vascular endothelial growth factor expression and PET/MRI radiomic signatures in primary clear-cell-renal-cell-carcinoma: proof-of-concept study.
      ,
      • Ha S.
      • Park S.
      • Bang et al., J.
      Characterize locally advanced breast cancer: histopathologic characteristics, response to neoadjuvant chemotherapy, and prognosis.
      ]. Finally, three studies found correlations between tumor metabolism and radiomic features [
      • Yip S.S.F.
      • Kim J.
      • Coroner T.P.
      • et al.
      Associations between somatic mutations and metabolic imaging phenotypes in non-small cell lung cancer.
      ,
      • Ha S.
      • Park S.
      • Bang et al., J.
      Characterize locally advanced breast cancer: histopathologic characteristics, response to neoadjuvant chemotherapy, and prognosis.
      ,
      • Lopez C.J.
      • Nagornaya N.
      • Parra N.A.
      • et al.
      Association of radiomics and metabolic tumor volumes in radiation treatment of glioblastoma multiforme.
      ].
      The best performing classifier achieved an AUC of 0.96 in differentiating between high and low grade intraductal papillary mucinous neoplasms (IPMNs) (this performance was based on internal cross-validation, rather than on an independent validation cohort) [
      • Hanania A.N.
      • Bantis L.E.
      • Feng Z.D.
      • et al.
      Quantitative imaging to evaluate malignant potential of IPMNs.
      ]. The poorest performing classifier, ‘tumor diameter’, achieved an AUC of 0.56 in the prediction of EGFR mutation status [
      • Aerts H.J.
      • Grossmann P.
      • Tan Y.
      • et al.
      Defining a radiomic response phenotype: a pilot study using targeted therapy in NSCLC.
      ].
      The RQS scoring resulted in some discrepancies between the reviewers, e.g. H.W. scored 4 studies ≥50%, S.S. scored 3 studies ≥50% while J.v.T. and E.d.J. scored 1 study ≥50%. Up to nine studies were given a quality score of 0%. The majority of studies were scored below 50% due to lacking robust segmentation, external validation and cut-off analyses/reporting. Nearly all studies were retrospective and lacking a calibration plot. Striking was that a lack of a prospective study design registered in a trial database accounted for as high as 7/36 (nearly 20%) of the total RQS score. Not a single study investigated cost-effectiveness of a potential clinically implemented radiomic signature.

      Discussion

      In this study, we performed a systematic literature review of studies addressing the interplay between radiomic features and tumor biology. In total 39 out of 41 studies reveal that radiomic features derived from mammography, ultrasound, CT, PET and/or MRI are significantly associated with one or several specific tumor biologic substrates, from somatic mutation status to tumor histopathological grading and tumor metabolism.
      To assist with benchmarking the radiomic workflow for reporting, internal consistency, reproducibility and clinical applicability we utilized the RQS score [
      • Lambin P.
      • Leijenaar R.T.H.
      • Deist T.M.
      • et al.
      Radiomics: the bridge between medical imaging and personalized medicine.
      ], which expands upon initiatives such as the recent established consensus ‘Transparent Reporting of a multivariable prediction model for individual Prognosis or Diagnosis’ (TRIPOD) statement, albeit in a radiomics context. Although the RQS score makes no claims about the overall significance or impact of a study, it may guide the reader in their appraisal of the workflow. The RQS score mentions the use of a biological correlate in a specific study, but does not quantify nor puts in perspective the link between tumor biologic substrate and radiomics.
      A good example here of the contrast between workflow quality and impact is a study performed by Grossmann et al. [
      • Grossmann P.
      • Stringfield O.
      • El-Hachem N.
      • et al.
      Defining the biological basis of radiomic phenotypes in lung cancer.
      ] which was assigned less than half of the RQS points (mean = 38.9%) e.g., due to not adequately reporting the image protocol and not addressing the cut-off values used for the different biologic endpoints while on the other hand exhibiting a sound methodology (e.g., feature reduction, external validation) and performing a comprehensive genetic profiling of the tumors on a relatively large study population (n = 351).
      Overall, we found only minor discrepancies between lower RQS score studies and the claims about clinical significance made by these studies. A total of 30 out of 41 papers had an average RQS score of <30%, while only 3/30 of these studies claimed to be non-exploratory but of significant impact on clinical outcome and/or decision-making. The other 2 studies had an average score ≥50 and both of these studies [
      • Aerts H.J.W.L.
      • Velazquez E.R.
      • Leijenaar R.T.H.
      • et al.
      Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach.
      ,
      • Velazquez E.R.
      • Parmar C.
      • Liu Y.
      • et al.
      Somatic mutations drive distinct imaging phenotypes in lung cancer.
      ] addressed a potential clinical relevance for the reported imaging biomarker. Furthermore, we found some differences in the inter-rater interpretation of the different RQS score components, justifying the need for more robust and easily interpretable methodological scoring systems for radiomic workflows and reporting. The greatest difference was found for RQS-score component ‘potential clinical applicability’, for instance the study by Zhang et al. [
      • Zhang Q.
      • Xiao Y.
      • Suo J.
      • et al.
      Sonoelastomics for breast tumor classification: a radiomics approach with clustering-based feature selection on sonoelastography.
      ] receiving two points by two scorers (J.v.T./E.d.J. and H.W.) and 0 points by one scorer (S.S.). This difference is related to the high interpretability of the statements made in the conclusion section of these studies combined with a lack of clear definition within the RQS score. Future versions of the RQS score should aim at making a clearer distinction between the criteria for allocation of points, especially in ambiguous components such as ‘potential clinical applicability’.
      Throughout the entire radiomic workflow major limitations may influence the conclusion of the study [
      • O’Connor J.P.B.
      Radiomics: rethinking the role of clinical imaging.
      ,
      • Lambin P.
      • Leijenaar R.T.H.
      • Deist T.M.
      • et al.
      Radiomics: the bridge between medical imaging and personalized medicine.
      ,
      • Yip S.S.F.
      • Aerts H.J.W.L.
      Applications and limitations of radiomics.
      ]: Radiomic studies systematically underreport study designs and (semi-) automatic and manual delineations of large amounts of images are time consuming and subject to error. Moreover, selection and pre-processing of images prior to feature extraction are very heterogeneous, and the terminology used for the derived image features lack a standard lexicon, factors which might significantly influence the conclusion. Additionally, modeling methodology and the reporting of outcomes in a clinical relevant fashion is lacking in a large amount of studies.
      The two major types of features extracted in the radiomic process are semantic and agnostic [
      • Gillies R.J.
      • Kinahan P.E.
      • Hricak H.
      Radiomics: images are more than pictures they are data.
      ]. Some recent studies have focused on semantic features, which are qualitative radiologic features, e.g., presence of cavitation, location of the lesion, presence of pleural effusion and shape of the lesion [
      • Yip S.S.F.
      • Kim J.
      • Coroner T.P.
      • et al.
      Associations between somatic mutations and metabolic imaging phenotypes in non-small cell lung cancer.
      ,
      • Lopez C.J.
      • Nagornaya N.
      • Parra N.A.
      • et al.
      Association of radiomics and metabolic tumor volumes in radiation treatment of glioblastoma multiforme.
      ,
      • Hasegawa M.
      • Sakai F.
      • Ishikawa R.
      • et al.
      CT features of epidermal growth factor receptor-mutated adenocarcinoma of the lung: comparison with non-mutated adenocarcinoma.
      ,
      • Rizzo S.
      • Petrella F.
      • Buscarino V.
      • et al.
      CT radiogenomic characterization of EGFR, K-RAS, and ALK mutations in non-small cell lung cancer.
      ,
      • Fave X.
      • Zhang L.
      • Yang J.
      • et al.
      Delta-radiomics features for the prediction of patient outcomes in non-small cell lung cancer.
      ]. These studies found significant differences in the semantic features between mutated tumors and non-mutated tumors. EGFR mutated tumors were for example associated with air bronchogram and pleural retraction, while KRAS mutated tumors were associated with nodules in non-tumor lobes and ALK rearrangements were associated with pleural effusion [
      • Rizzo S.
      • Petrella F.
      • Buscarino V.
      • et al.
      CT radiogenomic characterization of EGFR, K-RAS, and ALK mutations in non-small cell lung cancer.
      ]. Both types of radiomic features convey pertinent information that is complementary to each other. Radiologist-scored semantic annotation is a time consuming and costly process and regards non-mineable, qualitative data. There is a stride to capture semantic data by means of deep learning computer methods to achieve higher inter-reader agreement, faster throughput and lower variance [
      • Gillies R.J.
      • Kinahan P.E.
      • Hricak H.
      Radiomics: images are more than pictures they are data.
      ]. In contrast, agnostic radiomic features can be swiftly extracted with minimal operator input and make up the bulk of highly mineable quantitative data.
      A key hypothesis driving radiomics research is the potential to enable a spatiotemporal and quantitative measurement of both intra- and intertumoral heterogeneity based on medical images. The studies presented in this systematic review lay a foundation for such research by exploring the correlation of imaging biomarkers and the underlying biology. The majority of the studies presented fall within the field of radiogenomics; genomic heterogeneity within tumors and across metastatic tumor sites in the same patient is currently thought to be a major cause of treatment failure and the emergence of (targeted) therapy resistance [
      • van Timmeren J.E.
      • Leijenaar R.T.H.
      • van Elmpt W.
      • et al.
      Survival prediction of non-small cell lung cancer patients using radiomics analyses on cone-beam CT images.
      ]. Phylogenetic reconstruction has revealed branched evolutionary tumor growth, wherein nearly 70% of all somatic mutations are not detectable across every tumor region [
      • Gerlinger M.
      • Rowan A.J.
      • Horswell et al., S.
      Intratumor heterogeneity and branched evolution revealed by multiregion sequencing.
      ]. This may lead to underestimation of the tumor genomic landscape portrayed from single tumor-biopsy samples. Aside from (epi)genetic heterogeneity a complex and highly heterogeneous synergistic interplay exists between tumor cellular (proteomic, metabolic, morphological) phenotype and the microenvironment surrounding the tumor bed.
      Several studies investigated quantitative CT/PET-derived features associated with genomic and proteomic tumor expression [
      • Guo W.T.
      • Li H.
      • Zhu Y.T.
      • et al.
      Prediction of clinical phenotypes in invasive breast carcinomas from the integration of radiomics and genomics data.
      ,
      • Grossmann P.
      • Stringfield O.
      • El-Hachem N.
      • et al.
      Defining the biological basis of radiomic phenotypes in lung cancer.
      ,
      • Aerts H.J.W.L.
      • Velazquez E.R.
      • Leijenaar R.T.H.
      • et al.
      Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach.
      ,
      • Vargas H.A.
      • Veeraraghavan H.
      • Micco M.
      • et al.
      A novel representation of inter-site tumour heterogeneity from pre-treatment computed tomography textures classifies ovarian cancers by clinical outcome.
      ,
      • Velazquez E.R.
      • Parmar C.
      • Liu Y.
      • et al.
      Somatic mutations drive distinct imaging phenotypes in lung cancer.
      ,
      • Li H.
      • Zhu Y.
      • Burnside E.S.
      • et al.
      MR imaging radiomics signatures for predicting the risk of breast cancer recurrence as given by research versions of mammaprint, oncotype DX, and PAM50 gene assays.
      ,
      • Yu J.
      • Shi Z.
      • Ji C.
      • et al.
      Anatomical location differences between mutated and wild-type isocitrate dehydrogenase 1 in low-grade gliomas.
      ,
      • Grossmann P.
      • Gutman D.A.
      • Dunn W.D.
      • et al.
      Imaging-genomics reveals driving pathways of MRI derived volumetric tumor phenotype features in Glioblastoma.
      ,
      • Lee J.
      • Narang S.
      • Martinez J.J.
      • et al.
      Associating spatial diversity features of radiologically defined tumor habitats with epidermal growth factor receptor driver status and 12-month survival in glioblastoma: methods and preliminary investigation.
      ,
      • Yu J.
      • Shi Z.
      • Lian Y.
      • et al.
      Noninvasive IDH1 mutation estimation based on a quantitative radiomics approach for grade II glioma.
      ,
      • Yip S.S.F.
      • Kim J.
      • Coroner T.P.
      • et al.
      Associations between somatic mutations and metabolic imaging phenotypes in non-small cell lung cancer.
      ,
      • Gevaert O.
      • Echegaray S.
      • Khuong A.
      • et al.
      Predictive radiogenomics modeling of EGFR mutation status in lung cancer.
      ,
      • Yoon H.J.
      • Sohn I.
      • Cho J.H.
      • et al.
      Decoding tumor phenotypes for ALK, ROS1, and RET fusions in lung adenocarcinoma using a radiomics approach.
      ,
      • Emaminejad N.
      • Qian W.
      • Guan Y.B.
      • et al.
      Fusion of quantitative image and genomic biomarkers to improve prognosis assessment of early stage lung cancer patients.
      ,
      • Panth K.M.
      • Leijenaar R.T.H.
      • Carvalho S.
      • et al.
      Is there a causal relationship between genetic changes and radiomics-based image features? An in vivo preclinical experiment with doxycycline inducible GADD34 tumor cells.
      ,
      • Aerts H.J.
      • Grossmann P.
      • Tan Y.
      • et al.
      Defining a radiomic response phenotype: a pilot study using targeted therapy in NSCLC.
      ,
      • Ghosh P.
      • Tamboli P.
      • Vikram R.
      • et al.
      Imaging-genomic pipeline for identifying gene mutations using three-dimensional intra-tumor heterogeneity features.
      ,
      • Permuth J.B.
      • Choi J.
      • Balarunathan Y.
      • et al.
      Combining radiomic features with a miRNA classifier may improve prediction of malignant pathology for pancreatic intraductal papillary mucinous neoplasms.
      ,
      • Guo Y.
      • Hu Y.
      • Qiao M.
      • et al.
      Radiomics analysis on ultrasound for prediction of biologic behavior in breast invasive ductal carcinoma.
      ,
      • Li H.
      • Zhu Y.
      • Burnside E.S.
      • et al.
      Quantitative MRI radiomics in the prediction of molecular classifications of breast cancer subtypes in the TCGA/TCIA data set.
      ,
      • Wang J.
      • Kato F.
      • Oyama-Manabe N.
      • et al.
      Identifying triple-negative breast cancer using background parenchymal enhancement heterogeneity on dynamic contrast-enhanced MRI: a pilot radiomics study.
      ,
      • Ha S.
      • Park S.
      • Bang et al., J.
      Characterize locally advanced breast cancer: histopathologic characteristics, response to neoadjuvant chemotherapy, and prognosis.
      ]. One study (Yip et al.) found that EGFR mutated tumors shows different metabolic tumor phenotypes on PET than KRAS mutated tumor [
      • Yip S.S.F.
      • Kim J.
      • Coroner T.P.
      • et al.
      Associations between somatic mutations and metabolic imaging phenotypes in non-small cell lung cancer.
      ]. Another study (Aerts et al.) found that baseline radiomic features were able to predict mutation status and were also able to predict tyrosine kinase inhibitor (TKI) treatment response [
      • Aerts H.J.
      • Grossmann P.
      • Tan Y.
      • et al.
      Defining a radiomic response phenotype: a pilot study using targeted therapy in NSCLC.
      ].
      The supplementary value of radiomic to current genomic biomarkers is well illustrated in a study (Emaminejad et al.) where a quantitative image feature-based classifier yielded significantly higher discriminatory power than a genomic biomarker-based (AUC of 0.78 ± 0.06 and 0.68 ± 0.07 respectively) classifier in predicting early stage lung cancer recurrence risk [
      • Emaminejad N.
      • Qian W.
      • Guan Y.B.
      • et al.
      Fusion of quantitative image and genomic biomarkers to improve prognosis assessment of early stage lung cancer patients.
      ]. Fusion of prediction scores generated by the two classifiers further improved prediction performance to 0.84 ± 0.05, indicating that there is non-overlapping value added by the radiomic signature.
      As for the temporal heterogeneity, radiomics offers the possibility of longitudinal use in treatment monitoring and optimization and in active surveillance. Although such applications of radiomics (also called delta-, or Δ-radiomics) have yet to be explored in depth [
      • Mak R.H.
      • Digumarthy S.R.
      • Muzikansky A.
      • et al.
      Role of 18F-fluorodeoxyglucose positron emission tomography in predicting epidermal growth factor receptor mutations in non-small cell lung cancer.
      ], they may prove to be clinically relevant. Another new development is the so-called ‘4D radiomics’ approach, which can be used to quantitatively monitor treatment over time and perform (early) response assessment. In window-of-opportunity trials, radiomic features extracted from images before (and after) receiving an experimental drug and prior to the start of conventional treatment can potentially serve as predictive biomarker for clinical endpoints such as progression-free survival and (early) tumor response [
      • Glimelius B.
      • Lahn M.
      Window-of-opportunity trials to evaluate clinical activity of new molecular entities in oncology.
      ,
      • Larue R.T.H.M.
      • Van De Voorde L.
      • Berbée M.
      • et al.
      A phase 1 “window-of-opportunity” trial testing evofosfamide (TH-302), a tumour-selective hypoxia-activated cytotoxic prodrug, with preoperative chemoradiotherapy in oesophageal adenocarcinoma patients.
      ]. A recent study shows the predictive value of radiomic features extracted from weekly CT images for NSCLC patient outcomes [
      • Mak R.H.
      • Digumarthy S.R.
      • Muzikansky A.
      • et al.
      Role of 18F-fluorodeoxyglucose positron emission tomography in predicting epidermal growth factor receptor mutations in non-small cell lung cancer.
      ]. Cone-beam CT (CBCT) images might also be a good candidate for 4D radiomics, since these are generally acquired at multiple time points during treatment [
      • van Timmeren J.E.
      • Leijenaar R.T.H.
      • van Elmpt W.
      • et al.
      Feature selection methodology for longitudinal cone-beam CT radiomics.
      ,
      • Turner N.C.
      • Reis-Filho J.S.
      Genetic heterogeneity and cancer drug resistance.
      ].
      The link and interplay of metabolic, genomic and histologic information with clinical and imaging parameters is crucial for the establishment of effective personalized and reliable treatment strategies, especially in limited tissue settings. Hanania et al. serves as an example here by evaluating the malignant potential of intraductal papillary mucinous neoplasms (IPMN’s) in order to stratify patients for surgical resection [
      • Hanania A.N.
      • Bantis L.E.
      • Feng Z.D.
      • et al.
      Quantitative imaging to evaluate malignant potential of IPMNs.
      ]. Currently the consensus-based Fukuoka imaging criteria are the main approach to pre-operatively identify malignant IPMNs. With these criteria benign lesions are incorrectly labeled for surgery 1/3rd of the time at high volume and experienced centers, resulting in surgical overtreatment. The most predictive radiomic marker (a gray-level co-occurrence matrix (GLCM) feature) on the other hand differentiated low grade (LG) and high grade (HG) lesions better than the Fukuoka criteria (21% incorrectly labeled for surgery) with an AUC of 0.82 at a sensitivity of 85% and specificity of 68%. Even when adequate cytology and/or surgical biopsy specimens are available, radiomics offers important advantages for the assessment of tumor biology. It is estimated that the error rate of cancer histopathology can be as high as 23% and that most clinically relevant solid tumors are highly heterogeneous over time at the phenotypic, physiologic, and genomic levels [
      • Gillies R.J.
      • Kinahan P.E.
      • Hricak H.
      Radiomics: images are more than pictures they are data.
      ].
      With the advent of high-resolution imaging and the use of specific molecular imaging markers the future holds potential for the more precise characterization of tumor biology. In the field of radiotherapy the biological phenomenon of tumor hypoxia, a common feature in many human and animal solid tumors that can be visualized by means of novel molecular imaging techniques (e.g., the hypoxia PET-tracers FMISO, FAZA and HX4), has led to recent enthusiasm for hypoxic radio-sensitizers and hypoxia-selective cytotoxins as an adjunct therapy before tumor irradiation [
      • Hong B.J.
      • Kim J.
      • Jeong H.
      • et al.
      Tumor hypoxia and reoxygenation: the yin and yang for radiotherapy.
      ]. Radiomics could be a potential candidate to capture the temporal changes of such biological processes over various tumor habitats.
      An interesting development for adequate ex vivo human and in vivo animal tissue biology characterization by means of CT-based radiomics would be the introduction of micro-CT systems [

      Gregor T, Kochová P, Eberlová L, et al., Correlating micro-CT imaging with quantitative histology. injury and skeletal biomechanics. 2012; 173–191.

      ]. These systems are based on a rotating system of X-ray tube and detectors, and have a construction similar to that of human CT except that their dimensions are adapted to small animals. Compared with standard human CT devices, which offer an instrumental resolution limit of approximately 0.4 mm, the recent development of X-ray microtomography (micro-CT) has introduced resolutions similar to that of routine histology (spatial resolution of in vivo micro-CT ranges from 100 to 30 µm) [

      Gregor T, Kochová P, Eberlová L, et al., Correlating micro-CT imaging with quantitative histology. injury and skeletal biomechanics. 2012; 173–191.

      ]. Although the clinical application is limited to ex vivo human tissue and whole-body imaging of small animals due to the high radiation exposure, these techniques alone, or in combination with nuclear medicine techniques (micro-SPECT/CT and micro-PET/CT) may provide more detailed anatomical imaging information (which in turn might capture voxel-size dependent imaging features more precisely compared to clinical scanners) together with valuable metabolic information.
      Another promising development is the current data-sharing efforts set in place, such as the NCI-backed Cancer Imaging Archive (TCIA) [

      The Cancer Imaging Archive (TCIA): http://www.cancerimagingarchive.net.

      ]. Open access to de-identified medical images, histological, clinical and genomic data from cancer centers across the world may aid the further development and validation of robust biology-correlated radiomic models.
      A major limitation in this systematic review is the omission of analysis of publication bias, e.g., by means of a Funnel plot [
      • Egger M.
      • Davey Smith G.
      • Schneider M.
      • et al.
      Bias in meta-analysis detected by a simple, graphical test.
      ]. Withholding negative results from publication is especially a concern in the radiomics field, where the application of more sophisticated machine learning and deep learning networks on vaster amounts of clinical as well as imaging data has led to more robust models that nearly always produce a satisfactory model. Additionally, there are several limitations to the RQS score. Several components are not always applicable to every study, e.g., clinical utility for pre-clinical studies or ‘comparison to gold standard’ for outcomes in which no gold standard exists for the assessment of the outcome (e.g., tumor heterogeneity). Other components are not described in detail in the scoring table (e.g. what should specifically be mentioned on clinical utility) in the scoring table AND/OR are relatively ambiguous, for example in the case that only a subgroup of features (special diversity, texture etc.) has been extracted (in strict sense this is not feature selection). Furthermore the interpretability of the RQS score is left to a large extent to the reviewer, as it is not meant to report on the overall quality of a manuscript or the research it presents, but rather to guide the reader on a radiomic workflow appraisal. These findings indicate that the RQS tool will require incremental improvements in order to become a widely accepted assessment tool for the methodology of radiomic studies. Some of these improvements include a clearer definition of the RQS components including the allocation of scores, the measurement of tool validity (criterion, content, construct), and reliability.
      Our vision for the future of radiomics is intrepid, acknowledging that fundamental questions still need to be addressed before clinical applicability of radiomic imaging biomarkers will be achieved.
      With the advent of pre-clinical inducible knockout system studies, whereby a specific target gene of interest can be inactivated at a given time point, researchers will hopefully be able to draw more robust conclusions in the so-far mainly correlative relationship of radiomic features with tumor biology [
      • Panth K.M.
      • Leijenaar R.T.H.
      • Carvalho S.
      • et al.
      Is there a causal relationship between genetic changes and radiomics-based image features? An in vivo preclinical experiment with doxycycline inducible GADD34 tumor cells.
      ]. To find imaging biomarkers with stronger causal rather than correlative link with biological mechanisms would in our vision require not only radiomic studies on a vast amount of ultra-high resolution images to present high discrimination statistics in multiple validation sets, but also benchmarking and standardization of image pre-processing and feature extraction and data-sharing techniques that exploit matching ontologies.

      Conflict of interest

      Authors declare there are no conflicts of interest.

      Acknowledgements

      Authors acknowledge financial support from ERC advanced grant (ERC-ADG-2015, n° 694812 – Hypoximmuno) and the QuIC-ConCePT project, which is partly funded by EFPI A companies and the Innovative Medicine Initiative Joint Undertaking (IMI JU) under Grant Agreement No. 115151 . This research is also supported by the Dutch Technology Foundation STW (grant n° 10696 DuCAT & n° P14-19 Radiomics STRaTegy), which is the applied science division of NWO, and the Technology Programme of the Ministry of Economic Affairs. Authors also acknowledge financial support from the EU 7th framework program (ARTFORCE – n° 257144 , REQUITE – n° 601826 ), SME Phase 2 (RAIL – n° 673780 ), EUROSTARS (DART, DECIDE, COMPACT), the European Program H2020-2015-17 ( BD2Decide – PHC30-689715 , ImmunoSABR – n° 733008 and PREDICT – ITN – n° 766276 ), Interreg V-A Euregio Meuse-Rhine (“Euradiomics”), Kankeronderzoekfonds Limburg from the Health Foundation Limburg and the Dutch Cancer Society .

      Appendix A. Supplementary data

      References

      1. Forbes (2015): IBM acquires merge health to supplement Watson healthcare. https://www.forbes.com/forbes/welcome/?toURL=https://www.forbes.com/sites/greatspeculations/2015/08/12/ibm-acquires-merge-health-to-supplement-watson-healthcare/&refURL=https://www.google.nl/&referrer=https://www.google.nl/

        • Lee G.
        • Lee H.Y.
        • Ko E.S.
        • et al.
        Radiomics and imaging genomics in precision medicine.
        Precision and Future Medicine. 2017; 1: 10-31
        • Lee G.
        • Lee H.Y.
        • Park H.
        • et al.
        Radiomics and its emerging role in lung cancer research, imaging biomarkers and clinical management: state of the art.
        Eur J Radiol. 2017; 86: 297-307
        • Lambin P.
        • van Stiphout R.G.
        • Starmans M.H.
        • et al.
        Predicting outcomes in radiation oncology–multifactorial decision support systems.
        Nat Rev Clin Oncol. 2013; 10: 27-40
        • Lambin P.
        • Roelofs E.
        • Reymen B.
        • et al.
        Rapid Learning health care in oncology’ – an approach towards decision support systems enabling customised radiotherapy’.
        Radiother Oncol. 2013; 109: 159-164
        • Lambin P.
        • Zindler J.
        • Vanneste B.
        • et al.
        Modern clinical research: How rapid learning health care and cohort multiple randomised clinical trials complement traditional evidence based medicine.
        Acta Oncol. 2015; 23: 1-12
        • Lambin P.
        • Zindler J.
        • Vanneste B.G.
        • et al.
        Decision support systems for personalized and participative radiation oncology.
        Adv Drug Deliv Rev. 2017; 109: 131-153
        • Gani C.
        • Bonomo P.
        • Zwirner K.
        • et al.
        Organ preservation in rectal cancer – challenges and future strategies.
        Clin Translat Radiat Oncol. 2017; 3: 9-15
        • Skripcak T.
        • Belka C.
        • Bosch W.
        • et al.
        Creating a data exchange strategy for radiotherapy research: towards federated databases and anonymised public datasets.
        Radiother Oncol. 2014; 113: 303-309
        • O’Connor J.P.B.
        Radiomics: rethinking the role of clinical imaging.
        eLife. 2017; : 6
        • Lambin P.
        • Leijenaar R.T.H.
        • Deist T.M.
        • et al.
        Radiomics: the bridge between medical imaging and personalized medicine.
        Nat Rev Clin Oncol. 2017; 14: 749-762
        • Guo W.T.
        • Li H.
        • Zhu Y.T.
        • et al.
        Prediction of clinical phenotypes in invasive breast carcinomas from the integration of radiomics and genomics data.
        J Med Imaging. 2015; 2: 041007
        • Grossmann P.
        • Stringfield O.
        • El-Hachem N.
        • et al.
        Defining the biological basis of radiomic phenotypes in lung cancer.
        eLife. 2017; : 6
        • Aerts H.J.W.L.
        • Velazquez E.R.
        • Leijenaar R.T.H.
        • et al.
        Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach.
        Nat Commun. 2014; 5: 4006
        • Vargas H.A.
        • Veeraraghavan H.
        • Micco M.
        • et al.
        A novel representation of inter-site tumour heterogeneity from pre-treatment computed tomography textures classifies ovarian cancers by clinical outcome.
        Eur Radiol. 2017; 27: 3991-4001
        • Velazquez E.R.
        • Parmar C.
        • Liu Y.
        • et al.
        Somatic mutations drive distinct imaging phenotypes in lung cancer.
        Cancer Res. 2017; 77
        • Li H.
        • Zhu Y.
        • Burnside E.S.
        • et al.
        MR imaging radiomics signatures for predicting the risk of breast cancer recurrence as given by research versions of mammaprint, oncotype DX, and PAM50 gene assays.
        Radiology. 2016; 281: 382-391
        • Yu J.
        • Shi Z.
        • Ji C.
        • et al.
        Anatomical location differences between mutated and wild-type isocitrate dehydrogenase 1 in low-grade gliomas.
        Int J Neurosci. 2017; 127: 873-880
        • Grossmann P.
        • Gutman D.A.
        • Dunn W.D.
        • et al.
        Imaging-genomics reveals driving pathways of MRI derived volumetric tumor phenotype features in Glioblastoma.
        BMC Cancer. 2016; 16: 611
        • Lee J.
        • Narang S.
        • Martinez J.J.
        • et al.
        Associating spatial diversity features of radiologically defined tumor habitats with epidermal growth factor receptor driver status and 12-month survival in glioblastoma: methods and preliminary investigation.
        J Med Imaging. 2015; 2: 041006
        • Yu J.
        • Shi Z.
        • Lian Y.
        • et al.
        Noninvasive IDH1 mutation estimation based on a quantitative radiomics approach for grade II glioma.
        Eur Radiol. 2017; 27: 3509-3522
        • Yip S.S.F.
        • Kim J.
        • Coroner T.P.
        • et al.
        Associations between somatic mutations and metabolic imaging phenotypes in non-small cell lung cancer.
        J Nucl Med. 2017; 58: 569-576
        • Gevaert O.
        • Echegaray S.
        • Khuong A.
        • et al.
        Predictive radiogenomics modeling of EGFR mutation status in lung cancer.
        Sci Rep UK. 2017; 7: 41674
        • Yoon H.J.
        • Sohn I.
        • Cho J.H.
        • et al.
        Decoding tumor phenotypes for ALK, ROS1, and RET fusions in lung adenocarcinoma using a radiomics approach.
        Medicine. 2015; 94: e1753
        • Emaminejad N.
        • Qian W.
        • Guan Y.B.
        • et al.
        Fusion of quantitative image and genomic biomarkers to improve prognosis assessment of early stage lung cancer patients.
        IEEE Trans Bio-Med Eng. 2016; 63: 1034-1043
        • Panth K.M.
        • Leijenaar R.T.H.
        • Carvalho S.
        • et al.
        Is there a causal relationship between genetic changes and radiomics-based image features? An in vivo preclinical experiment with doxycycline inducible GADD34 tumor cells.
        Radiother Oncol. 2015; 116: 462-466
        • Aerts H.J.
        • Grossmann P.
        • Tan Y.
        • et al.
        Defining a radiomic response phenotype: a pilot study using targeted therapy in NSCLC.
        Sci Rep. 2016; 6: 33860
        • Ghosh P.
        • Tamboli P.
        • Vikram R.
        • et al.
        Imaging-genomic pipeline for identifying gene mutations using three-dimensional intra-tumor heterogeneity features.
        J Med Imaging. 2015; 2: 041009
        • Hanania A.N.
        • Bantis L.E.
        • Feng Z.D.
        • et al.
        Quantitative imaging to evaluate malignant potential of IPMNs.
        Oncotarget. 2016; 7: 85776-85784
        • Suo S.T.
        • Cheng J.J.
        • Cao M.Q.
        • et al.
        Assessment of heterogeneity difference between edge and core by using texture analysis: differentiation of malignant from inflammatory pulmonary nodules and masses.
        Acad Radiol. 2016; 23: 1115-1122
        • Bae J.M.
        • Jeong J.Y.
        • Lee H.Y.
        • et al.
        Pathologic stratification of operable lung adenocarcinoma using radiomics features extracted from dual energy ct images.
        Oncotarget. 2017; 8: 523-535
        • Pena E.
        • Ojiaku M.
        • Inacio et al., J.R.
        Can CT and MR shape and textural features differentiate benign versus malignant pleural lesions?.
        Acad Radiol. 2017; (S1076-6332(17)30133-2): 20
        • Ginsburg S.B.
        • Algohary A.
        • Pahwa S.
        • et al.
        Radiomic features for prostate cancer detection on MRI differ between the transition and peripheral zones: preliminary findings from a multi-institutional study.
        J Magn Reson Imaging. 2017; 46: 184-193
        • Bickelhaupt S.
        • Paech D.
        • Kickingereder P.
        • et al.
        Prediction of malignancy by a radiomic signature from contrast agent-free diffusion MRI in suspicious breast lesions found on screening mammography.
        J Magn Reson Imaging. 2017; 46: 604-616
        • Permuth J.B.
        • Choi J.
        • Balarunathan Y.
        • et al.
        Combining radiomic features with a miRNA classifier may improve prediction of malignant pathology for pancreatic intraductal papillary mucinous neoplasms.
        Oncotarget. 2016; 7: 85785-85797
        • Zhang X.
        • Xu X.
        • Tian Q.
        • et al.
        Radiomics assessment of bladder cancer grade using texture features from diffusion-weighted imaging.
        J Magn Reson Imaging. 2017; 15
        • Guo Y.
        • Hu Y.
        • Qiao M.
        • et al.
        Radiomics analysis on ultrasound for prediction of biologic behavior in breast invasive ductal carcinoma.
        Clin Breast Cancer. 2017; : 1-7
        • Song S.H.
        • Park H.
        • Lee H.
        • et al.
        Imaging phenotyping using radiomics to predict micropapillary pattern within lung adenocarcinoma.
        J Thorac Oncol. 2016; 12: 624-632
        • Patil R.
        • Mahadevaiah G.
        • Dekker A.
        An approach toward automatic classification of tumor histopathology of non-small cell lung cancer based on radiomic features.
        Tomography. 2016; 2: 374-377
        • Parmar C.
        • Leijenaar R.T.
        • Grossmann P.
        • et al.
        Radiomic feature clusters and prognostic signatures specific for lung and head & neck cancer.
        Sci Rep. 2015; 5: 11044
        • Lin Y.C.
        • Lin G.
        • Hong J.H.
        • et al.
        Diffusion radiomics analysis of intratumoral heterogeneity in a murine prostate cancer model following radiotherapy: pixelwise correlation with histology.
        J Magn Reson Imaging. 2017; 27: 3509-3522
        • Zhang Q.
        • Xiao Y.
        • Suo J.
        • et al.
        Sonoelastomics for breast tumor classification: a radiomics approach with clustering-based feature selection on sonoelastography.
        Ultrasound Med Biol. 2017; 43: 1058-1069
        • Marin Z.
        • Batchelder K.A.
        • Toner B.C.
        • et al.
        Mammographic evidence of microenvironment changes in tumorous breasts.
        Med Phys. 2017; 44: 1324-1336
        • Coquery N.
        • Francois O.
        • Lemasson B.
        • et al.
        Microvascular MRI and unsupervised clustering yields histology-resembling images in two rat models of glioma.
        J Cereb Blood Flow Met. 2014; 34: 1354-1362
        • Choi E.R.
        • Lee H.Y.
        • Jeong J.Y.
        • et al.
        Quantitative image variables reflect the intratumoral pathologic heterogeneity of lung adenocarcinoma.
        Oncotarget. 2016; 7: 67302-67313
        • Wu W.M.
        • Parmar C.
        • Grossmann P.
        • et al.
        Exploratory study to identify radiomics classifiers for lung cancer histology.
        Front Oncol. 2016; 6: 71
        • Li H.
        • Zhu Y.
        • Burnside E.S.
        • et al.
        Quantitative MRI radiomics in the prediction of molecular classifications of breast cancer subtypes in the TCGA/TCIA data set.
        NPJ Breast Cancer. 2016; 2: 16012
        • Gnep K.
        • Fargeas A.
        • Gutierrez-Carvajal R.E.
        • et al.
        Haralick textural features on T2-weighted MRI are associated with biochemical recurrence following radiotherapy for peripheral zone prostate cancer.
        J Magn Reson Imaging. 2017; 45: 103-117
        • Wang J.
        • Kato F.
        • Oyama-Manabe N.
        • et al.
        Identifying triple-negative breast cancer using background parenchymal enhancement heterogeneity on dynamic contrast-enhanced MRI: a pilot radiomics study.
        PLoS One. 2015; 10: e0143308
        • Yin Q.
        • Hung S.C.
        • Wang L.
        • et al.
        Associations between tumor vascularity, vascular endothelial growth factor expression and PET/MRI radiomic signatures in primary clear-cell-renal-cell-carcinoma: proof-of-concept study.
        Sci Rep. 2017; 7: 43356
        • Ha S.
        • Park S.
        • Bang et al., J.
        Characterize locally advanced breast cancer: histopathologic characteristics, response to neoadjuvant chemotherapy, and prognosis.
        Sci Rep. 2017; 7: 1556
        • Lopez C.J.
        • Nagornaya N.
        • Parra N.A.
        • et al.
        Association of radiomics and metabolic tumor volumes in radiation treatment of glioblastoma multiforme.
        Int J Radiat Oncol. 2017; 97: 586-595
        • Yip S.S.F.
        • Aerts H.J.W.L.
        Applications and limitations of radiomics.
        Phys Med Biol. 2016; 61: 150-166
        • Gillies R.J.
        • Kinahan P.E.
        • Hricak H.
        Radiomics: images are more than pictures they are data.
        Radiology. 2016; 278: 563-577
        • Hasegawa M.
        • Sakai F.
        • Ishikawa R.
        • et al.
        CT features of epidermal growth factor receptor-mutated adenocarcinoma of the lung: comparison with non-mutated adenocarcinoma.
        J Thorac Oncol. 2016; 11: 819-826
        • Rizzo S.
        • Petrella F.
        • Buscarino V.
        • et al.
        CT radiogenomic characterization of EGFR, K-RAS, and ALK mutations in non-small cell lung cancer.
        Eur Radiol. 2015; 26: 32-42
        • Fave X.
        • Zhang L.
        • Yang J.
        • et al.
        Delta-radiomics features for the prediction of patient outcomes in non-small cell lung cancer.
        Sci Rep. 2017; 7: 588
        • van Timmeren J.E.
        • Leijenaar R.T.H.
        • van Elmpt W.
        • et al.
        Survival prediction of non-small cell lung cancer patients using radiomics analyses on cone-beam CT images.
        Radiother Oncol. 2017; 123: 363-369
        • Gerlinger M.
        • Rowan A.J.
        • Horswell et al., S.
        Intratumor heterogeneity and branched evolution revealed by multiregion sequencing.
        NEJM. 2012; 366: 883-891
        • Mak R.H.
        • Digumarthy S.R.
        • Muzikansky A.
        • et al.
        Role of 18F-fluorodeoxyglucose positron emission tomography in predicting epidermal growth factor receptor mutations in non-small cell lung cancer.
        Oncologist. 2011; 16: 319-326
        • Glimelius B.
        • Lahn M.
        Window-of-opportunity trials to evaluate clinical activity of new molecular entities in oncology.
        Ann Oncol. 2011; 22: 1717-1725
        • Larue R.T.H.M.
        • Van De Voorde L.
        • Berbée M.
        • et al.
        A phase 1 “window-of-opportunity” trial testing evofosfamide (TH-302), a tumour-selective hypoxia-activated cytotoxic prodrug, with preoperative chemoradiotherapy in oesophageal adenocarcinoma patients.
        BMC Cancer. 2016; 16: 644
        • van Timmeren J.E.
        • Leijenaar R.T.H.
        • van Elmpt W.
        • et al.
        Feature selection methodology for longitudinal cone-beam CT radiomics.
        Acta Oncol. 2017; 1–7
        • Turner N.C.
        • Reis-Filho J.S.
        Genetic heterogeneity and cancer drug resistance.
        Lancet Oncol. 2012; 13: e178-e185
        • Hong B.J.
        • Kim J.
        • Jeong H.
        • et al.
        Tumor hypoxia and reoxygenation: the yin and yang for radiotherapy.
        Radiat Oncol J. 2016; 34: 239-249
      2. Gregor T, Kochová P, Eberlová L, et al., Correlating micro-CT imaging with quantitative histology. injury and skeletal biomechanics. 2012; 173–191.

      3. The Cancer Imaging Archive (TCIA): http://www.cancerimagingarchive.net.

        • Egger M.
        • Davey Smith G.
        • Schneider M.
        • et al.
        Bias in meta-analysis detected by a simple, graphical test.
        BMJ. 1997; 315: 629-634