Tag: calibration

A scikit-learn NPS ordinal classifier with SMOTE, probability calibration, utility-based thresholding, and PSI drift checks. The parts that make it useful to the retention team, not just accurate on a dashboard.

Outcome: Shipped a calibrated multiclass NPS model with a utility-driven operating threshold and a PSI-based drift loop, giving the retention team a per-customer detractor probability they can act on and a rule for when to retrain.

How imbalanced classifiers can keep a strong AUC while producing probabilities that break thresholds, alerts, and cost-sensitive decisions in production.

Outcome: Defined a production calibration gate that logs Brier score, ECE, reliability diagrams, cost-sensitive thresholds, run metadata, and promotion criteria for imbalanced classifiers.

How to add coverage-guaranteed prediction sets, temperature scaling calibration, and risk-coverage curves to a classifier using MAPIE — the pieces that make uncertainty quantification operationally useful rather than decorative.

Outcome: Added coverage-guaranteed prediction sets and operational abstention gates to a classification pipeline, cutting acted-upon error rate without retraining the model.

A practical playbook for turning classifier scores into reliable probabilities that can support ranking, thresholds, SLAs, and cost-sensitive decisions.

Outcome: Defined a calibration workflow that separates ranking from probability quality, uses scikit-learn calibration correctly, and carries thresholds and monitoring into production.

A production-friendly scikit-learn pattern for mixed tabular data, class imbalance, calibrated probabilities, threshold selection, and model persistence.

Outcome: Defined an end-to-end scikit-learn classification pipeline that keeps preprocessing, imbalance handling, probability calibration, evaluation, thresholding, and production artifacts aligned.