摘要:

Connectionist networks that have learned one task can be reused on related tasks in a process that is called 'transfer'. This paper surveys recent work on transfer, and includes an overview of the articles in this volume. A number of distinctions between kinds of transfer are identified, and future directions for research are explored. The study of transfer has a long history in cognitive science. Discoveries about transfer in human cognition can inform applied efforts. Advances in applications can also inform cognitive studies.

ISSN:0954-0091    

DOI:10.1080/095400996116866

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

The purpose of this paper is to review the cognitive literature regarding transfer in order to provide a context for the consideration of transfer in neural networks. We consider transfer under the three general headings of analogy, skill transfer and metaphor. The emphasis of the research in each of these areas is quite different and the literatures are largely distinct. Important common themes emerge, however, relating to the role of similarity, the importance of 'surface content' and the nature of the representations that are used. We will draw out these common themes and note ways of facilitating transfer. We also briefly note possible implications for the study of transfer in neural networks.

ISSN:0954-0091    

DOI:10.1080/095400996116875

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

We introduce a new approach to the training of classifiers for performance on multiple tasks. The proposed hybrid training method leads to improved generalization via a better low-dimensional representation of the problem space. The quality of the representation is assessed by embedding it in a two-dimensional space using multi-dimensional scaling, allowing a direct visualization of the results. The performance of the approach is demonstrated on a highly non-linear image classification task.

ISSN:0954-0091    

DOI:10.1080/095400996116884

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

A category learning experiment involving human participants compared the difficulties of four types of shift learning. Initial learning was of an exclusive-or (XOR) structure on two of three stimulus dimensions. One shift type was a reversal, a second shift was to a single previously relevant dimension, a third shift was to a single previously irrelevant dimension, and a fourth shift was to an XOR on one previously relevant dimension and one previously irrelevant dimension. Results showed that reversal shift was easiest, followed, in order, by shift to a single previously relevant dimension, shift to a single previously irrelevant dimension, and a shift to a new XOR. An extended version of the ALCOVE model, called AMBRY, qualitatively fits the data. The model incorporates two essential principles. First, internal category representations that can be quickly remapped to overt responses are important for accounting for the ease of reversal shift. Second, perseverating dimensional attention is important for accounting for the ease of shifting to a previously relevant dimension as opposed to a previously irrelevant dimension. It is suggested that any model of these effects will need to implement both of these principles.

ISSN:0954-0091    

DOI:10.1080/095400996116893

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

The Osgood surface for transfer in human associative learning is introduced (Osgood, 1949). It describes the relationship between stimulus and response similarities and transfer of learning. In this paradigm, first a list A is learned, then a list B, followed by retesting on list A. Simulation results indicate that three-layer networks with backpropagation do not only show 'catastrophic interference' but also 'hypertransfer'. Two-layer networks do not suffer from this. Hypertransfer is explained with reference to hidden-layer representations formed during learning. Since it cannot account for this very general trait of human behavior, backpropagation's role as a tool for models of human memory must be watched very carefully.

ISSN:0954-0091    

DOI:10.1080/095400996116901

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

In this paper we explore the topic of the consolidation of information in neural network learning. One problem in particular has limited the ability of a broad range of neural networks to perform ongoing learning and consolidation. This is 'catastrophic forgetting', the tendency for new information, when it is learned, to disrupt old information. We will review and slightly extend the rehearsal and pseudorehearsal solutions to the catastrophic forgetting problem presented in Robins (1995). The main focus of this paper is to then relate these mechanisms to the consolidation processes which have been proposed in the psychological literature regarding sleep. We suggest that the catastrophic forgetting problem in artificial neural networks (ANNs) is a problem that has actually occurred in the evolution of the mammalian brain, and that the pseudorehearsal solution to the problem in ANNs is functionally equivalent to the sleep consolidation solution adopted by the brain. Finally, we review related work by McClelland et al. (1995) and propose a tentative model of learning and sleep that emphasizes consolidation mechanisms and the role of the hippocampus.

ISSN:0954-0091    

DOI:10.1080/095400996116910

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor

摘要:

With a distinction made between two forms of task knowledge transfer, 'representational' and 'functional', eta MTL, a modified version of the multiple task learning (MTL) method of functional (parallel) transfer, is introduced. The eta MTL method employs a separate learning rate, etak, for each task output node k. etak varies as a function of a measure of relatedness, Rk, between the kth task and the primary task of interest. Results of experiments demonstrate the ability of eta MTL to dynamically select the most related source task(s) for the functional transfer of prior domain knowledge. The eta MTL method of learning is nearly equivalent to standard MTL when all parallel tasks are sufficiently related to the primary task, and is similar to single task learning when none of the parallel tasks are related to the primary task.

ISSN:0954-0091    

DOI:10.1080/095400996116929

出版商:Taylor & Francis Group    

年代:1996

数据来源: Taylor