AbstractLearning is one of the most important problems in the speech recognition of unspecified speakers. The authors have already proposed a set of phoneme templates and non‐supervised learning. By that method, a satisfactory result was obtained for the recognition of five Japanese vowels and ten numerals. The learning is essentially a selection of templates, and there was no positive adaptation in the sense that the template matched to each speaker is constructed. To improve further the performance of the learning it is necessary to construct the template which pertinently represents the features of the speaker. The already‐reported scheme is pointed out as unsatisfactory in the adaptation from such a viewpoint. The authors then investigated a new nonsupervised learning algorithm where the traditional learning algorithm is improved to enhance the adaptability, and the phoneme templates are constructed automatically by the nonsupervised learning with a batch processing by storing speech data. This paper discusses the two nonsupervised learning algorithms and the word recognition system containing those algorit