⛰️ Gradient Descent - pfh · Scour

Gradient Descent For Logistic and Linear Regression 🎪Convex Optimization

·5d

Convergence rates for gradient descent in the training of overparameterized artificial neural networks with piecewise affine activation 🎪Convex Optimization

LoRA and Weight Decay (2023) 🎪Convex Optimization

irhum.github.io·2d·Hacker News

Streamlined optical training of large-scale modern deep learning architectures with direct feedback alignment 🗺️UMAP

Backpropagation 🎪Convex Optimization

pugsley.bearblog.dev·3d

Convergent Abstraction Hypothesis 🔢Embeddings

lesswrong.com·6d

MANGO: Meta-Adaptive Network Gradient Optimization for Online Continual Learning 🎪Convex Optimization

Training Neural Networks with Optimal Double-Bayesian Learning 📊Empirical Bayes

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad 🎪Convex Optimization

Fast Spawn\&Prune (FS\&P): Global convergence of stochastic conic particle gradient descent via birth/death process 🎪Convex Optimization

High-dimensional Limit of SGD for Diagonal Linear Networks 🗺️Manifold Learning

Replacement Learning: Training Neural Networks with Fewer Parameters 🗺️Manifold Learning

Feature Learning in Linear-Width Two-Layer Networks: Two vs. One Step of Gradient Descent 📊Empirical Bayes

LionMuon: Alternating Spectral and Sign Descent for Efficient Training 🗺️Manifold Learning

Optimal Asymptotic Rates for (Stochastic) Gradient Descent under the Local PL-Condition: A Geometric Approach 🎪Convex Optimization

Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional data 📊Empirical Bayes

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training 🗺️UMAP

Rethinking Neural Network Learning Rates: A Stackelberg Perspective 📊Empirical Bayes

Optimized projection-free algorithms for online learning: construction and worst-case analysis 🎪Convex Optimization

Turning Stale Gradients into Stable Gradients: Coherent Coordinate Descent with Implicit Landscape Smoothing for Lightweight Zeroth-Order Optimization 🎪Convex Optimization

Log in to enable infinite scrolling