AbstractStructured finite action‐finite state space discounted Markovian decision problems are analyzed. Any problem of a general class is shown to be equivalent to a “separated” problem with decomposable problem structure. A modified policy iteration approach is developed for this decomposable reformulation. Both analytic and computer evaluations of the decomposition algorithm's effectiveness are pres