Iterative Robust Satisficing: Minimizing Performance Degradation Under Distribution Shift

Abstract

Modern neural networks often achieve high accuracy on their training distribution but degrade sharply under distribution shifts. We address this problem through *Robust Satisficing* (RS), an optimization objective that seeks parameters which attain a target level of in-distribution performance while minimizing *fragility*, defined as the rate at which performance deteriorates as the data distribution departs from training. We develop a gradient-based algorithm, *Iterative Robust Satisficing* (IRS), that directly optimizes this criterion. Across a range of synthetic and real-world distribution shifts, including long-tailed image classification, group shifts induced by spurious correlations, and natural shifts in tabular regression, IRS consistently improves performance on minority and worst-case groups without sacrificing overall accuracy. Notably, IRS achieves these robustness gains with a per-step computational cost similar to standard stochastic gradient descent and requires only a single forward and backward pass per update. Together, these results suggest that minimizing fragility provides a practical and effective alternative to existing robust training methods for learning models that remain reliable under distribution shift.