This paper proposes a hierarchical recursive algorithm for non-linear optimal control systems through combining the improved new predication method of large-scale system theory with the single-term Walsh series approach. Piecewise constant and discrete-time solutions with minimal mean-square error are obtained, and the proposed method has merits in both computation and efficiency. The numerical examples show that the proposed method is very effective even for systems with lower dimensions.