摘要:

We analyze the convergence of an approximate gradient projection method for minimizing the sum of continuously differentiable functions over a nonempty closed convex set. In this method, the functions are aggregated and, at each iteration, a succession of gradient steps, one for each of the aggregate functions, is applied and the result is projected onto the convex set. We show that if the gradients of the functions are bounded and Lipschitz continuous over a certain level set and the stepsizes are chosen to be proportional to a certain residual squared or to be square summable, then every cluster point of the iterates is a stationary point. We apply these results to the backpropagation algorithm to obtain new deterministic convergence results for this algorithm. We also discuss the issues of parallel implementation and give a simple criterion for choosing the aggregation.

ISSN:1055-6788    

DOI:10.1080/10556789408805580

出版商:Gordon and Breach Science Publishers    

年代:1994

数据来源: Taylor

摘要:

A general convergence theorem is proposed for a family of serial and parallel nonmonotone unconstrained minimization methods with perturbations. A principal application of the theorem is to establish convergence of backpropagation (BP), the classical algorithm for training artificial neural networks. Under certain natural assumptions, such as divergence of the sum of the learning rates and convergence of the sum of their squares, it is shown that every accumulation point of the BP iterates is a stationary point of the error function associated with the given set of training examples. The results presented cover serial and parallel BP, as well as modified BP with a momentum term.

ISSN:1055-6788    

DOI:10.1080/10556789408805581

出版商:Gordon and Breach Science Publishers    

年代:1994

数据来源: Taylor

摘要:

We study here convergence properties of serial and parallel backpropagation algorithm for training of neural nets, as well as its modification with momentum term. It is shown that these algorithms can be put into the general framework of the stochastic gradient methods. This permits to consider from the same positions both stochastic and deterministic rules for the selection of components (training examples) of the error function to minimize at each iteration. We obtained weaker conditions on the stepsize for deterministic case and provide quite general synchronization rule for parallel version.

ISSN:1055-6788    

DOI:10.1080/10556789408805582

出版商:Gordon and Breach Science Publishers    

年代:1994

数据来源: Taylor

摘要:

In this paper the problem of neural network training is formulated as the unconstrained minimization of a sum of differentiate error terms on the output space. For problems of this form we consider solution algorithms of the backpropagation-type, where the gradient evaluation is split into different steps, and we state sufficient convergence conditions that exploit the special structure of the objective function. Then we define a globally convergent algorithm that uses the knowledge of the overall error function for the computation of the learning rates. Potential advantages and possible shortcomings of this approach, in comparison with alternative approaches are discussed.

ISSN:1055-6788    

DOI:10.1080/10556789408805583

出版商:Gordon and Breach Science Publishers    

年代:1994

数据来源: Taylor

摘要:

Neural network algorithms have been shown to provide good solutions for a variety of non-linear optimization problems, ranging from classification to function approximation in high dimension space. These algorithms are capable of “learning” a target function from a set of “training examples”without strong assumption about the function. In this paper we show an example of applying neural networks to time series analysis and prediction. Backpropagation algorithm is used to train layered, feed-forward networks to model a complex, non-linear time series. A general state space formulation is adopted to analyze the problem and aCascaded Methodis used to predict multiple steps into the future. A fast parallel implementation of Backpropagation on the Connection Machine allowed us to do extensiveexploratory data analysisto search for good neural net predictive models on large data sets.

ISSN:1055-6788    

DOI:10.1080/10556789408805584

出版商:Gordon and Breach Science Publishers    

年代:1994

数据来源: Taylor

ISSN:1055-6788    

DOI:10.1080/10556789408805579

出版商:Gordon and Breach Science Publishers    

年代:1994

数据来源: Taylor