Previous experimental studies indicate that the steady two‐dimensional flow in a lid‐driven cavity becomes unstable and goes through a sequence of transitions before becoming turbulent. In this study, an analysis of this instability is undertaken. The two‐dimensional base flow is computed numerically over a range of Reynolds numbers and is perturbed with three‐dimensional disturbances. The partial differential equations governing the evolution of these perturbations are then obtained using linear stability analysis and normal mode analysis. Using a finite difference discretization, a generalized eigenvalue problem is formulated from these equations whose solution gives the dispersion relation between complex growth rate and wave number. An eigenvalue solver using simultaneous iteration is employed to identify the dominant eigenvalue which is indicative of the growth rate of these perturbations and the associated eigenfunction which characterizes the secondary state. This paper presents stability curves to identify the critical Reynolds number and the critical wavelength of the neutral mode and discusses the mechanism of instability through energy calculations. This paper finds that the loss of stability of the base flow is due to a long wavelength mode at a critical Reynolds number (Re) of 594. The mechanism is analyzed through a novel application of the Reynolds–Orr equations and shown to be due to a Goertler type instability. The stability curves are relatively flat indicating that this state will be challenged by many shorter wavelength modes at a slightly higher Reynolds number. In fact, a second competing mode with a wavelength close to the cavity width was found to be unstable at Re=730. The present results of the reconstructed flow based on these eigenfunctions at the neutral state, show striking similarities to the experimental observations.