摘要:

This paper reviews research on combining artificial neural nets, and provides an overview of, and an introduction to, the papers contained in this special issue, and its companion (Connection Science, 9, 1). Two main approaches, ensemble-based, and modular, are identified and considered. An ensemble, or committee, is made up of a set of nets, each of which is a general function approximator. The members of the ensemble are combined in order to obtain better generalization performance than would be achieved by any of the individual nets. The main issues considered here under the heading of ensemble-based approaches are a how to combine the outputs of the ensemble members, b how to create candidate ensemble members and c which methods lead to the most effective ensembles? Under the heading of modular approaches, we begin by considering a divide-and-conquer approach by which a function is automatically decomposed into a number of subfunctions which are treated by specialist modules. Other modular approaches are also identified and considered, for while the divide-and-conquer approach is designed to improve performance, the term modularity can be given a wider interpretation. The broadly defined topic of modularity includes the explicit decomposition of a task based on the designer's understanding, and the exploitation of specialist modules in order to accomplish tasks which could not be performed by a monolithic net.

ISSN:0954-0091    

DOI:10.1080/095400996116785

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

Collinearity or linear dependency among a number of estimators may pose a serious problem when combining these estimators. The corresponding outputs of a number of neural networks NNs , which are trained to approximate the same quantity or quantities , may be highly correlated. Thus, the estimation of the optimal weights for combining such networks may be subjected to the harmful effects of collinearity, which results in a final model with inferior generalizations ability compared with the individual networks. In this paper, we investigate the harmful effects of collinearity on the estimation of the optimal weights for combining a number on NNs. We discuss an approach for selecting the component networks in order to improve the generalization ability of the combined model. Our experimental results demonstrate significant improvements in the generalization ability of a combined model as a result of the proper selection of the component networks. The approximation accuracy of the combined model is compared with two common alternatives: the apparent best network or the simple average of the corresponding outputs of the networks.

ISSN:0954-0091    

DOI:10.1080/095400996116794

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

A neural network NN ensemble is a very successful technique where the outputs of a set of separately trained NNs are combined to form one unified prediction. An effective ensemble should consist of a set of networks that are not only highly correct, but ones that make their errors on different parts of the input space as well; however, most existing techniques only indirectly address the problem of creating such a set. We present an algorithm called ADDEMUP that uses genetic algorithms to search explicitly for a highly diverse set of accurate trained networks. ADDEMUP works by first creating an initial population, then uses genetic operators to create new networks continually, keeping the set of networks that are highly accurate while disagreeing with each other as much as possible. Experiments on four real-world domains show that ADDEMUP is able to generate a set of trained networks that is more accurate than several existing ensemble approaches. Experiments also show ADDEMUP is able to incorporate prior knowledge effectively, if available, to improve the quality of its ensemble.

ISSN:0954-0091    

DOI:10.1080/095400996116802

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

Bootstrap samples with noise are shown to be an effective smoothness and capacity control technique for training feedforward networks and for other statistical methods such as generalized additive models. It is shown that noisy bootstrap performs best in conjunction with weight-decay regularization and ensemble averaging. The two-spiral problem, a highly non-linear, noise-free data, is used to demonstrate these findings. The combination of noisy bootstrap and ensemble averaging is also shown useful for generalized additive modelling, and is also demonstrated on the well-known Cleveland heart data.

ISSN:0954-0091    

DOI:10.1080/095400996116811

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

We describe a decorrelation network training method for improving the quality of regression learning in 'ensemble' neural networks NNs that are composed of linear combinations of individual NNs. In this method, individual networks are trained by backpropogation not only to reproduce a desired output, but also to have their errors linearly decorrelated with the other networks. Outputs from the individual networks are then linearly combined to produce the output of the ensemble network. We demonstrate the performances of decorrelated network training on learning the 'three-parity' logic function, a noisy sine function and a one-dimensional non-linear function, and compare the results with the ensemble networks composed of independently trained individual networks without decorrelation training . Empirical results show than when individual networks are forced to be decorrelated with one another the resulting ensemble NNs have lower mean squared errors than the ensemble networks having independently trained individual networks. This method is particularly applicable when there is insufficient data to train each individual network on disjoint subsets of training patterns.

ISSN:0954-0091    

DOI:10.1080/095400996116820

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

Using an ensemble of classifiers, instead of a single classifier, can lead to improved generalization. The gains obtained by combining, however, are often affected more by the selection of what is presented to the combiner than by the actual combining method that is chosen. In this paper, we focus on data selection and classifier training methods, in order to 'prepare' classifiers for combining. We review a combining framework for classification problems that quantifies the need for reducing the correlation among individual classifiers. Then, we discuss several methods that make the classifiers in an ensemble more complementary. Experimental results are provided to illustrate the benefits and pitfalls of reducing the correlation among classifiers, especially when the training data are in limited supply.

ISSN:0954-0091    

DOI:10.1080/095400996116839

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

Algorithms for reducing variance in neural network prediction using committee and resampling techniques bootstrap and cross-validation are presented. Their effectiveness is tested on data sets with different levels of noise and on medical diagnosis data sets. The methods are most effective when the noise level in the data is high or the size of the learning set is small, which consequently produces high variance. The algorithms will not be of much help in cases where the error of prediction is mainly due to bias. An increase in bias is observed due to smaller effective learning size in the bootstrap and crossvalidation committee. The impact of increased bias on the performance ranges from negligible to completely undermining the benefit of reducing the variance.

ISSN:0954-0091    

DOI:10.1080/095400996116848

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

Several methods of aggregating outcomes of classifiers are applied to artificial and real magnetic resonance MR spectra. Logistic regression, linear combination of classifiers, fuzzy integration, stacked generalization and some other fusion methods, as well as different ways of estimating necessary parameters, are considered. The results indicate that the fusion of classifiers improves the performance of the individual classifiers. On real MR spectra, which are characterized by the paucity of experimental data and low signal-tonoise ratio, the results vary. Some methods perform well on some data sets and poorly on others. Strategies are recommended to gain from classifier aggregation.

ISSN:0954-0091    

DOI:10.1080/095400996116857

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor