CLEAR item#9

“Sample size calculation. Describe how the sample size or power was determined before or after the study (e.g., sample size/power calculation, based on availability).” [1] (from the article by Kocak et al.; licensed under CC BY 4.0)

Reporting examples for CLEAR item#9

Example#1. “In the estimation of sample size for radiomics model construction, at least 48 cases (12 macrotrabecular-massive [MTM] cases and 36 non-MTM cases) were required in each dataset according to the following input and assumption: power, 0.9; two-sided significance level, 0.05; alternative hypothesis of the area under the curve (AUC), 0.800 compared with the null hypothesis of the AUC, 0.500, and an allocation ratio of sample sizes in the negative and positive groups of 3” [2] (from the article by Feng et al.; licensed under CC BY 4.0)

Example#2. “To evaluate the sample size for the validation group, we performed a power analysis based on the LNM rates of the training dataset and the validation dataset.” [3] (from the article by An et al.; licensed under CC BY 2.0)

Example#3. “A sample size of the development cohort was calculated using Riley and colleagues’ approach. We hypothesized an incidence of significant clinical deterioration within 30 days at 20%; among mild COVID-19, 16 parameters included in the clinical prediction models and an expected Harrell’s c-index of 0.78 (Nagelkerke’s R2 of 0.25). The resulting sample size was at least 826 patients. For the external validation, we aimed to recruit at least one hundred clinical deterioration events for each validation cohort, as recommended by Vergouwe.” [4] (from the article by Zysman et al.; licensed under CC BY 2.0)

Explanation and elaboration of CLEAR item#9

Item#9 highlights the accurate description of the sample size and how the optimal sample size was determined. Radiomics studies should describe if a priori power analysis for sample size calculation was performed to determine the minimal sample to yield sufficient power for the radiomics model construction [5]. Radiomics studies with an insufficient number of cases due to the small samples, especially if divided into training and validation cohorts, can lead to unreliable model performance and lack of generalization due to overfitting due to the high number of extracted features [6]. This can be particularly problematic in radiomics studies analyzing rare diseases or conditions. A minimum number of 10–15 samples per feature in the final radiomic model has been suggested for radiomics studies, based on previous evidence on logistic regression-based modelling [7].

References

  1. Kocak B, Baessler B, Bakas S, et al (2023) CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII. Insights Imaging 14:75. https://doi.org/10.1186/s13244-023-01415-8
  2. Feng Z, Li H, Liu Q, et al (2023) CT Radiomics to Predict Macrotrabecular-Massive Subtype and Immune Status in Hepatocellular Carcinoma. Radiology 307:e221291. https://doi.org/10.1148/radiol.221291
  3. An C, Park YW, Ahn SS, et al (2021) Radiomics machine learning study with a small sample size: Single random training-test set split may lead to unreliable results. PLOS ONE 16:e0256152. https://doi.org/10.1371/journal.pone.0256152
  4. Zysman M, Asselineau J, Saut O, et al (2023) Development and external validation of a prediction model for the transition from mild to moderate or severe form of COVID-19. Eur Radiol 33:9262–9274. https://doi.org/10.1007/s00330-023-09759-x
  5. Moskowitz CS, Welch ML, Jacobs MA, et al (2022) Radiomic Analysis: Study Design, Statistical Analysis, and Other Bias Mitigation Strategies. Radiology 304:265–273. https://doi.org/10.1148/radiol.211597
  6. Xu L, Yang P, Liang W, et al (2019) A radiomics approach based on support vector machine using MR images for preoperative lymph node status evaluation in intrahepatic cholangiocarcinoma. Theranostics 9:5374–5385. https://doi.org/10.7150/thno.34149
  7. Shur JD, Doran SJ, Kumar S, et al (2021) Radiomics in Oncology: A Practical Guide. Radiogr Rev Publ Radiol Soc N Am Inc 41:1717–1732. https://doi.org/10.1148/rg.2021210037

Back Next