In this paper, we deal with the problem of designing non-overshooting feedback control systems when the input is a step. We also consider the more general problem of designing controllers to track a step optimally with some predetermined amount of allowable overshoot. These problems are cast as infinite linear programming problems. We show that both these problems always have a solution (under some standard assumptions). It is then shown that the counterpart of this problem, that is, the problem of designing non-undershooting feedback control systems, need not have a solution in general. But it is proved that one can always design a feedback controller to achieve as little percentage undershoot as desired. Finally, it is shown that the problem of designing controllers to simultaneously satisfy bounds on the overshoot and undershoot always has a solution if some undershoot is allowed for. The duals to the infinite linear programming problems are formulated and the absense of a duality gap is established for each of the problems using the concept of quasi-relative interiors. Most of the paper deals with discrete-time single-input single-output systems through extensions of the techniques to a class of multi-input multi-output systems are also discussed.