Radiologists make the decision to report or dismiss a possible cancer based not only on the finding itself, but also in the comparison with selected areas of the background. We examined the effects of fixating, for the first time, the location where the radiologist either reported the presence of a malignant mass or visually inspected the mass but did not report it, and the effects of pairing radiologists to read the same cases. Four experienced mammographers participated in this experiment. They read a set of 20 cases twice. Eye-position tracking was used to monitor the visual search behaviour of the observers. Spatial frequency analysis was used to determine the characteristics of the areas of the background fixated by the observers. Radiologists had more fixations in the cases where they agreed how to manage the lesion than when they disagreed. Correlation between the areas of the background sampled by the radiologists and an “average” representation of the background increased after the observers fixated for the first time a malignant mass that they reported. Fixating, for the first time, a location where the radiologist reports a malignant mass or a location containing a cancer that the radiologist visually inspects but decides not to report, has a significant effect on any further sampling of the background. Furthermore, care should be taken when pairing radiologists, because some observers showed such a similar visual search behaviour that not much would be gained by having them read the same cases.