Although the calculated reliability of a test based on a small sample of subjects is an estimate of the population reliability and, hence, is subject to sampling fluctuation, little application has been made of statistical inference techniques to the reliability coefficient. The need for such techniques was recognized as early as the 1940s, and a few attempts have been made to introduce analysis of variance (anova) procedures and to relate the sampling distribution of reliability coefficient estimates to the well‐knownFdistribution (e.g. Jackson&Ferguson, 1941; Ebel, 1951). It was, however, Kristof (1963) who first presented a rather complete sampling theory of reliability estimates and a method to apply it. Feldt (1965) derived similar results based on ananovamodel and applied it to a binary item test case. The above sampling theories were based on normality assumptions regarding the true and error score distributions. It is, however, conceivable that real data will not always meet the rigorous assumption of normality, and yet very little is known about the effects of non‐normality on the sampling distribution of reliability estimates. The present paper contains the results of an investigation of the effects of non‐normality under a mixed modelanova. Both analytical and computer simulation methods are