Leakage-Aware, Fusion-Enhanced Machine Learning Pipeline for Diabetes Screening: Integrating Resampling Comparison, Model Fusion, Conformal Prediction, and Calibration Diagnostics

Abstract

Diabetes mellitus is a globally prevalent chronic metabolic disorder whose timely identification is critical to preventing irreversible end-organ damage. Although machine learning (ML) approaches have shown considerable promise for automated diabetes screening, the field is hampered by methodological deficiencies—notably data leakage during preprocessing and oversampler selection performed outside cross-validation folds—that collectively produce overoptimistic and clinically unreliable performance estimates. We present a rigorously leakage-aware, multi-stage ML pipeline evaluated on a large-scale publicly available synthetic clinical dataset comprising 100,000 records and 15 features (prevalence 8.5%). The pipeline explicitly addresses two categories of leakage: (i) preprocessing leakage, by fitting all transformers exclusively on training data; and (ii) resampling leakage, by performing oversampler comparison entirely within cross-validation folds. Other leakage types (temporal, label, feature) are acknowledged as scope limitations. The pipeline encompasses: (i) stratified three-way splitting into training, calibration, and held-out test partitions; (ii) an in-fold cross-validated comparison of seven resampling strategies against a no-resampling class-weighted baseline, with SMOTETomek selected automatically on cross-validated F1; (iii) training of four base classifiers (Logistic Regression, Random Forest, HistGradientBoosting, LightGBM); (iv) six model-fusion strategies; (v) isotonic probability calibration; (vi) decision curve analysis; and (vii) Mondrian class-conditional split-conformal prediction. Among all models, LightGBM, HistGradientBoosting, and the logarithmic pooling fusion ensemble (Fus-LogPool) achieved statistically equivalent top performance: PR-AUC of 0.8807 (Fus-LogPool), 0.8801 (LightGBM), and 0.8798 (HistGB), with pairwise McNemar tests uniformly non-significant (p > 0.5), indicating that a simple well-tuned single gradient-boosted tree is as effective as fusion on this dataset. Mondrian conformal prediction attained 90.39% empirical coverage (target 90%), with 98.82% singleton predictions and singleton accuracy of 91.46%. Decision curve analysis confirmed net clinical benefit over treat-all and treat-none strategies across clinically relevant risk thresholds (5%–20%). SHAP global explanations identified HbA1c level, blood glucose level, BMI, and age as the dominant risk determinants. Importantly, all findings derive from a synthetic or semi-synthetic dataset without a verified clinical source, and must be validated on prospective real-world cohorts before any clinical application. The fully reproducible pipeline is released as an open-source Kaggle notebook.