The paper proposes a neural network technique to classify numerals using Zernike moments that are invariant to rotation only. In order to make them invariant to scale and shift, we introduce modified Zernike moments based on regular moments. Owing to the large number of Zernike moments used, it is computationally more efficient to select a subset of them that can discriminate as well as the original set. The subset is determined using stepwise discriminant analysis. The performance of a subset is examined through its comparison to the original set. The results are shown of using such a scheme to classify scaled, rotated, and shifted binary images and images that have been perturbed with random noise. In addition to the neural network approach, the Fisher's classifier is also used, which is a parametric classifier. A comparative study of their performances shows that the neural network approach produces better classification accuracy than the Fisher's classifier. When a suitable subset of Zernike moments is used, the classifiers perform well, just like the original set. The performance of the classifiers is also examined. The computational time is greatly reduced when a suitable subset of Zernike moments is used.