首页   按字顺浏览 期刊浏览 卷期浏览 Analysis of an approximate gradient projection method with applications to the backprop...
Analysis of an approximate gradient projection method with applications to the backpropagation algorithm*

 

作者: Luo Zhi-Quan,   Tseng Paul,  

 

期刊: Optimization Methods and Software  (Taylor Available online 1994)
卷期: Volume 4, issue 2  

页码: 85-101

 

ISSN:1055-6788

 

年代: 1994

 

DOI:10.1080/10556789408805580

 

出版商: Gordon and Breach Science Publishers

 

关键词: Neural networks, Backpropagation, Gradient projection

 

数据来源: Taylor

 

摘要:

We analyze the convergence of an approximate gradient projection method for minimizing the sum of continuously differentiable functions over a nonempty closed convex set. In this method, the functions are aggregated and, at each iteration, a succession of gradient steps, one for each of the aggregate functions, is applied and the result is projected onto the convex set. We show that if the gradients of the functions are bounded and Lipschitz continuous over a certain level set and the stepsizes are chosen to be proportional to a certain residual squared or to be square summable, then every cluster point of the iterates is a stationary point. We apply these results to the backpropagation algorithm to obtain new deterministic convergence results for this algorithm. We also discuss the issues of parallel implementation and give a simple criterion for choosing the aggregation.

 

点击下载:  PDF (570KB)



返 回