Evaluation of nonlinear optimization methods for the learning algorithm of artificial neural networks
作者:
Hideyuki Takagi,
Shigeo Sakaue,
Hayato Togawa,
期刊:
Systems and Computers in Japan
(WILEY Available online 1992)
卷期:
Volume 23,
issue 1
页码: 101-111
ISSN:0882-1666
年代: 1992
DOI:10.1002/scj.4690230109
出版商: Wiley Subscription Services, Inc., A Wiley Company
数据来源: WILEY
摘要:
AbstractThis paper describes the implementation of nonlinear optimization methods into the learning of neural networks (NN) and the speed efficiency of four proposed improvements into the backpropagation algorithm. The problems of the backpropagation learning method are pointed out first, and the efficiency of implementing a nonlinear optimization method as a solution to this problems is described.Two nonlinear optimization methods are selected after inspecting several nonlinear methods from the viewpoint of NN learning to avoid the problem of the backpropagation algorithm. These are the linear search method by Davies, Swann, and Campey (DSC), and the conjugate gradient method by Fletcher and Reeves.The NN learning algorithms with these standard methods being implemented are formulated. Moreover, the following four improvements of the nonlinear optimization methods are proposed to shorten the NN learning time: (a) fast forward calculation in linear search by consuming a larger amount of memories; (b) avoiding the trap to local minimum point in an early stage of linear search; (c) applying a linear search method suitable for parallel processing; and (d) switching the gradient direction using the conjugate gradient method. The evaluation results have shown that all methods described here are effective in shortening the learning time.
点击下载:
PDF
(980KB)
返 回