A molecular beam mass spectrometer system has been designed and constructed for the specific purpose of measuring the gaseous composition of the vapor environment during chemical vapor deposition of diamond. By the intrinsic nature of mass analysis, this type of design is adaptable to a broad range of other applications that rely either on thermal‐ or plasma‐induced chemical kinetics. When gas is sampled at a relatively high process pressure (∼2700 Pa for our case), supersonic gas expansion at the sampling orifice can cause the detected signals to have a complicated dependence on the operating conditions. A comprehensive discussion is given on the effect of gas expansion on mass discrimination and signal scaling with sampling pressure and temperature, and how these obstacles can be overcome. This paper demonstrates that radical species can be detected with a sensitivity better than 10 ppm by the use of threshold ionization. A detailed procedure is described whereby one can achieve quantitative analysis of the detected species with an accuracy of ±20%. This paper ends with an example on the detection of H, H2, CH3, CH4, and C2H2during diamond growth.