Sample size requirements that achieve a prespecified expected width or lower limit for a confidence interval about the intraclass kappa statistic are supplied for the case of two raters and a dichotomous outcome measure. These results should be useful in the planning stages of an interobserver agreement study in which the focus is on estimation rather than hypothesis‐testing. In order to quantify the loss of precision obtained when choosing a dichotomous trait for this purpose, corresponding results are also provided for the case of a continuous outcome measure