This study deals with the instabilities that arise in the flow generated in a rotating tank by the evolution of a two-layer density stratified fluid. Numerical investigations have been performed by direct simulation of the Navier-Stokes equations for axisymmetric and fully three-dimensional flows. In the former case results have shown the attainment, in a very short time, of an equilibrium position and the formation of an anticyclonic structure in the upper light layer and a cyclonic one in the lower layer, consistently with the observation of Griffiths and Linden. In the long term, however, the Ekman layer at the bottom damps out the cyclone and a steady state with only an anticyclone in the upper layer is reached. In three-dimensions the flow is unstable to azimuthal disturbances and the steady state is no longer achieved. In particular a ring of cyclonic vorticity, surrounding the anticyclone, by the combined effects of baroclinic and barotropic processes, breaks, entrains vorticity from the anticyclone and eventually forms vortex pairs. As observed by Griffiths and Linden the azimuthal wave number(n*)of the instability depends on the Richardson number(Ri)and the ratio between the depth of the light fluid and the total depth(&dgr;). However, since several modes, in addition to the most unstable, are amplified an initial perturbation whose energy is not equidistributed among the modes can lead to an instability with wave number different from the expectedn*. Finally, the analysis of the equation for the energy of the instability has shown that the instability is initially driven by baroclinic effects, even for low values of&dgr;. The barotropic source, in contrast, sets in only in the large-amplitude phase of the instability and its effect is larger when&dgr;is small. ©1997 American Institute of Physics.