论文标题
LALR：Lipschitz自适应学习率和神经网络中自适应学习率的理论和实验验证
LALR: Theoretical and Experimental validation of Lipschitz Adaptive Learning Rate in Regression and Neural Networks
论文作者
论文摘要
我们为平均绝对错误损失函数和分位数损失函数的自适应学习率政策提出了一个理论框架，并评估其对回归任务的有效性。该框架基于Lipschitz的连续性理论，特别是利用学习率与Lipschitz损失函数常数之间的关系。基于实验，我们发现自适应学习率政策与恒定的学习率政策相比，最多可以更快地收敛。
We propose a theoretical framework for an adaptive learning rate policy for the Mean Absolute Error loss function and Quantile loss function and evaluate its effectiveness for regression tasks. The framework is based on the theory of Lipschitz continuity, specifically utilizing the relationship between learning rate and Lipschitz constant of the loss function. Based on experimentation, we have found that the adaptive learning rate policy enables up to 20x faster convergence compared to a constant learning rate policy.
