Ultrathin Co films were epitaxially grown on a Cu(001) surface with different initial roughness created by Ar ion sputtering. The roughness of the Cu substrate and the Co film were characterized by high resolution low energy electron diffraction. The measured angular profiles were compared with a diffraction theory for rough surfaces and the roughness parameters were extracted quantitatively. Magnetic hysteresis loops of these characterized films were measured by surface magneto‐optic Kerr effect. The hysteresis loop shape and loop area can be related to the nanoscale roughness in the Co films. For the roughest film with interface width ≊1.2t, wheretis the single atomic step height, the magnetization is reduced several fold compared with that of smooth films with interface width ≊0.5t. Also, the coercivity in the roughest film is the highest and there exists a wide range of nucleation centers and coercive fields for magnetic domain reversals. These are related to the high step density in the rough substrate as the pinning centers. The hysteresis loop changes its shape and area under a sinusoidal external magnetic field as functions of frequency and field amplitude for all films. For the smooth films in the low frequency and low field regimes the loop area shows a 2/3 power law scaling behavior. The 2/3 value of scaling exponents are consistent with the prediction of a dynamic mean field theory with a double‐well energy barrier. For a film with the same interface width but different step density and lateral correlation length the scaling exponents deviate from 2/3 value drastically.