Measurements of the hole density in carbon‐doped GaAs and AlxGa1−xAs as a function of the annealing temperature are presented. It is shown that after sample annealing at low temperatures (T<550 °C), the hole concentration increases in all samples doped ≥1×1019cm−3with a simultaneous decrease in the hole mobility. However, sample annealing at higher temperatures (T≳600 °C) results in a reduction of the hole concentration in all samples doped with carbon at concentrations higher than ≊5×1019cm−3. The reduction in hole concentration is also accompanied by an increase in lattice parameter of the carbon‐doped epilayer. The observed changes in the electrical and microstructural properties are explained in terms of two different mechanisms: (1) the passivation of carbon acceptors by the incorporation of hydrogen during growth, and (2) the change in the lattice site location of carbon atoms upon annealing. Direct determination of the lattice site location of carbon in samples doped ≥5×1019cm−3showed that the fraction of interstitial carbon after annealing atT≳600 °C is at least 70% higher than in the as‐grown samples.