Abstract:Curvature influences generalization, robustness, and how reliably neural networks respond to small input perturbations. Existing sharpness metrics are typically defined in parameter space (e.g., Hessian eigenvalues) and can be expensive, sensitive to reparameterization, and difficult to interpret in functional terms. We introduce a scalar curvature measure defined directly in input space: the curvature rate {\lambda}, given by the exponential growth rate of higher-order input derivatives. Empirically, {\lambda} is estimated as the slope of log ||D^n f|| versus n for small n. This growth-rate perspective unifies classical analytic quantities: for analytic fun…
Abstract:Curvature influences generalization, robustness, and how reliably neural networks respond to small input perturbations. Existing sharpness metrics are typically defined in parameter space (e.g., Hessian eigenvalues) and can be expensive, sensitive to reparameterization, and difficult to interpret in functional terms. We introduce a scalar curvature measure defined directly in input space: the curvature rate {\lambda}, given by the exponential growth rate of higher-order input derivatives. Empirically, {\lambda} is estimated as the slope of log ||D^n f|| versus n for small n. This growth-rate perspective unifies classical analytic quantities: for analytic functions, {\lambda} corresponds to the inverse radius of convergence, and for bandlimited signals, it reflects the spectral cutoff. The same principle extends to neural networks, where {\lambda} tracks the emergence of high-frequency structure in the decision boundary. Experiments on analytic functions and neural networks (Two Moons and MNIST) show that {\lambda} evolves predictably during training and can be directly shaped using a simple derivative-based regularizer, Curvature Rate Regularization (CRR). Compared to Sharpness-Aware Minimization (SAM), CRR achieves similar accuracy while yielding flatter input-space geometry and improved confidence calibration. By grounding curvature in differentiation dynamics, {\lambda} provides a compact, interpretable, and parameterization-invariant descriptor of functional smoothness in learned models.
| Comments: | 14 pages |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2511.01438 [cs.LG] |
| (or arXiv:2511.01438v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2511.01438 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jacob Poschl [view email] [v1] Mon, 3 Nov 2025 10:46:03 UTC (132 KB)