The evolution of large scale vortices as coherent structures in a free shear layer and the corresponding energetics in Fourier space are investigated numerically. The results confirm that vortex pairing as a result of selective energy transfer directly to the subharmonic is a rare event when other instability modes are present with random phases. For this naturally perturbed shear flow, vortices are observed to coalesce locally and intermittently in a manner that leaves isolated vortices between paired states. The spatial and temporal intermittency results in a gradual energy transfer from the linearly unstable mode to larger length scales.