A model for strain relaxation in linear compositionally graded epitaxial layers is derived. Equilibrium dynamics for misfit dislocation generation is used in each compositional segment of the graded layers through the film thickness. A local relaxation thicknesses in the compositionally graded layers is calculated as a function of film thickness. The calculated critical thicknesses show good correspondence with the depth of dislocation‐free regions which are observed at the top of graded layers with different grading rates. From analysis of the model, we explain the origin of dislocation structures and minimized threading dislocation density in the compositionally graded layers. The compositional grading produces a large equilibrium dislocation spacing compared to that of highly mismatched conventional stepwise growths, and thus enables misfit dislocations to glide freely without dislocation–dislocation interactions. At successive local relaxation thicknesses, lateral dislocations pile up horizontally by continuous dislocation generation and expansion. The analysis with calculated results coincides with dislocation structures observed by transmission electron microscopy in InGaAs graded layers on GaAs.