For a control system consisting of a first-order plant and a relay in a closed loop, and with a gaussian white noise disturbance added to the plant input, the Fokker-Planck equation is applied to find the first and second probability density functions of the plant output. Closed-form expressions are then obtained for the autocorrelations, cross-correlations, spectral densities and cross-spectral densities of all the system signals. The results may be used to test the assumptions made in approximate methods of analysis of stochastic bang-bang control systems.