Regularization tools for training large feed-forward neural networks using automatic differentiation*
作者:
Jerry Erikssont,
Mårten Gulliksson,
Per Lindström,
Per-åke Wedin,
期刊:
Optimization Methods and Software
(Taylor Available online 1998)
卷期:
Volume 10,
issue 1
页码: 49-69
ISSN:1055-6788
年代: 1998
DOI:10.1080/10556789808805701
出版商: Gordon and Breach Science Publishers
关键词: Neural network training;;Tikhonov regularization;;automatic differentiation;;large-scale problems
数据来源: Taylor
摘要:
We describe regularization tools for training large-scale artificial feed-forward neural networks. We propose algorithms that explicitly use a sequence of Tikhonov regularized nonlinear least squares problems. For large-scale problems, methods using new special purpose automatic differentiation are used in a conjugate gradient method for computing a truncated Gauss—Newton search direction. The algorithms developed utilize the structure of the problem in different ways and perform much better than a Polak-Ribiere based method. All algorithms are tested using benchmark problems and guidelines by Lutz Prechelt in the Probenl package. All software is written in Matlab and gathered in a toolbox.
点击下载:
PDF (769KB)
返 回