A finite time control problem for Markov processes with arbitrary state and action space is studied. The value of a given strategy is defined by the expected total reward up to a given point of time where the rewards are defined on the one hand by a reward rate and on the other hand by a valuation of the jumps of the controlled process. A necessary and sufficient condition for the existence of an optimal strategy is given. The structure of the studied processes enables us to interpret a class of optimization problems in queueing theory as special case of the general model.