Optical second‐harmonic generation (SHG) data are presented on the cleaning of Si(001) surfaces covered with thin (20–40 Å) oxide layers. The removal of an oxide layer by annealing, results in the formation of a 2×1 reconstructed surface and a concurrent increase in second‐harmonic intensity of nearly two orders of magnitude. A comparable signal increase is observed in the transition from an unreconstructed passivated Si(001) surface, produced by etching, to a clean 2×1 reconstructed surface. It is concluded that SHG is very sensitive to the presence of surface states on the 2×1 reconstructed Si(001) surface.