Tools for consensus analysis of experts’ contours for radiotherapy structure definitions

Abstract 

Background and purpose

To demonstrate and examine the ability of a newly developed software tool to estimate and analyze consensus contours from manually created contours by expert radiation oncologists.

Material and methods

Several statistical methods and a graphical user interface were developed. For evaluation purposes, we used three breast cancer CT scans from the RTOG Breast Cancer Atlas Project. Specific structures were contoured before and after the experts’ consensus panel meeting. Differences in the contours were evaluated qualitatively and quantitatively by the consensus software tool. Estimates of consensus contours were analyzed for the different structures and Dice-similarity and Dice-Jaccard coefficients were used for comparative evaluation.

Results

Based on kappa statistics, highest levels of agreement were seen in the left-breast, lumpectomy, and heart. Significant improvements between pre- and post-consensus contours were seen in delineation of the chestwall and breasts while significant variations were noticed in the supraclavicular and internal mammary nodes. Dice calculations for all pre-consensus STAPLE estimations and final consensus panel structures reached 0.80 or greater for the heart, left/right-breast, case-A lumpectomy, and chestwall.

Conclusions

Using the consensus software tool incorporating STAPLE estimates provided the ability to create contours similar to the ones generated by expert physicians.

Keywords: Structure definition, Radiotherapy treatment planning, Experts’ consensus, Statistical modeling, Software tools