A new phase matching technique is described which is based on phase mismatch balancing in crystal or waveguide segments that are short compared to their respective coherence lengths. The technique is demonstrated experimentally in a mixed bulk crystal/waveguide structure in KTiOPO4, resulting in a second‐harmonic generation conversion efficiency of 15%/W/cm2at 1.064 &mgr;m, close to the theoretical maximum. We show that this technique can significantly broaden processing latitude for fabricating practical nonlinear optical waveguide devices.