SCALED Choices for Clinical Outcome Prediction

Machine learning (ML) has demonstrated a remarkable ability to identify complex patterns in healthcare data, yet high predictive performance does not necessarily translate into clinically useful decision support. Clinical prediction models must balance the competing demands of patient care and efficient resource allocation while remaining interpretable and robust to real-world data conditions.

In this work, we compare key ML design choices against literature-based baseline methodologies using two prediction tasks derived from the MIMIC-IV database: emergency department (ED) disposition prediction and intensive care unit (ICU) readmission prediction. We examine the effects of data preprocessing, feature engineering, class imbalance strategies, and decision threshold selection within a shared XGBoost modeling framework.

Our results demonstrate that model performance and prediction behavior across the ED disposition and ICU readmission prediction tasks are highly sensitive to ML design choices, with substantial tradeoffs observed between precision and recall. In particular, decision threshold tuning provides flexible operating points that can be adapted to different clinical and operational priorities by balancing false positives and false negatives. These findings highlight that clinically useful and realistic ML models should balance false positives and false negatives rather than optimizing individual performance metrics alone, ultimately supporting both patient care and hospital resource allocation. More broadly, our findings suggest that ML design choices can significantly influence model behavior and should be carefully evaluated when developing and deploying clinical prediction systems.