This paper deals with the state-space constrained optimal control problems with control variables appearing linearly by the concept of decomposition. To solve this continuous optimal control problem, we first discretize the time and replace the system of differential equations by difference equations. For this resulting discrete optimal control problem, fixing the value of state variables reduces the given problem to a finite number of independent linear programming problems which are parameterized by the value of state variables. From this point of view, after para. meterizing by the value of state variables, we outer-linearize the resulting itifimal valuo functions in the minimond and apply the relaxation strategy to the new constraints arising as a consequence of outer-linearization. An algorithm is proposed which requires baek-and-forth iteration between a master problem and a finite number of linear programming subproblems. Finite convergence of this algorithm follows directly from the finite number of constraints of the master problem.