We have examined in detail the formation of surface relief structures in azobenzene polyesters formed by polarization holography with orthogonally circularly polarized laser beams. We show that it is possible to separate the contribution to the diffraction efficiency into an anisotropic part and a surface relief part by examining the polarization content of the first order diffracted beam. By studying the dynamics of the growth of the grating, we show that the gratings due to anisotropy and surface relief appear at the same time. Atomic force microscopic investigations of the film after irradiation reveal a strongly polarization dependent surface relief pattern. ©1997 American Institute of Physics.