Summary.This study described criterion‐referenced tests of electrocardiograph reading skill and basic knowledge in cardiology, assessed their dependability and validity, and explored their impact on overall certification rates. Data indicated that the standard‐setting processes and the two criterion‐referenced tests produced dependable results both separately and together. Scores of each had the expected relationships with quality of residency training and experience with the examination. Moreover, these sections of the examination identified a small subset of examinees who failed the criterion‐referenced sections but passed the norm‐referenced examinations. Taken as a whole these results replicate the findings of an earlier study in terms of a criterion‐referenced test of electrocardiograph reading skill and extended them to a criterion‐referenced test of basic knowledge