Title:Learning with Statistical Equality Constraints
Abstract:As machine learning applications grow increasingly ubiquitous and complex, they face an increasing set of requirements beyond accuracy. The prevalent approach to handle this challenge is to aggregate a weighted combination of requirement violation penalties into the training objective. To be effective, this approach requires careful tuning of these hyperparameters (weights), involving trial-and-error and cross-validation, which becomes ineffective even for a moderate number of requirements. These issues are exacerbated when the requirements involve parities or equalities, as is the case in fairness and boundary v…
Title:Learning with Statistical Equality Constraints
Abstract:As machine learning applications grow increasingly ubiquitous and complex, they face an increasing set of requirements beyond accuracy. The prevalent approach to handle this challenge is to aggregate a weighted combination of requirement violation penalties into the training objective. To be effective, this approach requires careful tuning of these hyperparameters (weights), involving trial-and-error and cross-validation, which becomes ineffective even for a moderate number of requirements. These issues are exacerbated when the requirements involve parities or equalities, as is the case in fairness and boundary value problems. An alternative technique uses constrained optimization to formulate these learning problems. Yet, existing approximation and generalization guarantees do not apply to problems involving equality constraints. In this work, we derive a generalization theory for equality-constrained statistical learning problems, showing that their solutions can be approximated using samples and rich parametrizations. Using these results, we propose a practical algorithm based on solving a sequence of unconstrained, empirical learning problems. We showcase its effectiveness and the new formulations enabled by equality constraints in fair learning, interpolating classifiers, and boundary value problems.
| Comments: | to be published in the 39th Annual Conference on Neural Information Processing Systems |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2511.14320 [cs.LG] |
| (or arXiv:2511.14320v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2511.14320 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aneesh Barthakur [view email] [v1] Tue, 18 Nov 2025 10:24:13 UTC (4,546 KB)