Representation Costs in Data Science: Foundations and the Quasi-Banach Spaces of Deep Neural Networks (opens in new tab)

We develop a general framework for analyzing representation costs of parametric data-fitting methods through their parameter-space regularizers. From this abstract perspective, we define representation costs for arbitrary parametric models and reveal their induced (native) function spaces. This unifies recent function-space views of data-fitting methods. We also prove that many natural results hold in this abstract setting, including represent...