Abstract:Knowledge engineers often find that experts’ estimates of uncertainty change from one day to the next and therefore often seem unreliable. This article explores the unreliability of probability estimates. Forty undergraduates answered 31 questions concerning the probability of daily events at two separate times. Three answer modalities were available: probability, chances and percent of the time. This paper finds that reliability decreases when answer modalities change from one time to the next. It is hypothesized that problem‐solving approaches changed along with the modalities. The subjects represented a wide range in reliability, with the most reliable subjects changing answer modalities less than the unreliable subjects. These results suggest that knowledge engineers should endeavor to have experts express their uncertainty estimates using consistent modalit