We have studied the surface morphology and growth mechanism of metalorganic vapor phase epitaxy grown strained‐layer InxGa1−xAs (x=0.2 or 0.5) on GaAs with atomic force microscopy. Morphological instability of monolayer steps was observed on a 10 nm thick strained‐layer In0.2Ga0.8As. Three‐dimensional (3D) growth was observed forx=0.5 when grown at 650 °C. By lowering the growth temperature to 600 °C, the growth mode is 2‐D for 5 nm films (x=0.5). Monolayer steps and 2D islands can be seen. Increasing the layer thickness to 7.5 nm at 600 °C caused the growth of 3D islands and the generation of misfit dislocations. ©1995 American Institute of Physics.