Guided‐wave acousto‐optic diffraction has been demonstrated on a silicon substrate for the first time. The surface acoustic wave is launched from a ZnO/Tab2O5/SiO2/Si multilayered structure, while a Ta2O5/SiO2/Si optical waveguide is used for the acousto‐optic interaction. The measured diffraction efficiency is 3.3% per milliwatt of acoustic power. An order‐better performance is possible with improved transducer geometry. The silicon‐based guided‐wave acousto‐optic technology is preferred for eventual integration of both optical and electronic components for integrated optical applications such as rf spectrum analyzer and fiber‐optical terminals. Experiments with a similar structure on a glass substrate is also reported.