AbstractSpectral efficiency for land mobile radio channels is primarily limited by adjacent channel interference. Conventional digital modulation schemes do not achieve current specifications for adjacent channel interference at high data rates. The adjacent channel interference can be mitigated by the intelligent choice of transmitter components such as the modulation scheme and the transmit filter. We present a trellis coded modulation (TCM) scheme that can provide high data rates and at the same time reduce adjacent channel interference. We present methods to choose an appropriate TCM scheme under constraints of Euclidean distance and peak‐to‐average power. We also present methods to optimize transmit filters to minimize adjacent channel interference under various constraints. Examples are provided to illustrate the gains that can be obtai