Open Problem: Data Selection for Regression Tasks

Abstract

This note proposes a set of open problems concerning data selection in regression tasks. The central question is: given a natural learning rule 𝒜 and a selection budget n, how well can 𝒜 perform when trained on n examples selected from a larger dataset? We present concrete instances of this question in basic regression settings, including mean estimation and linear regression.