The purpose of this paper is to obtain some upper and lower bounds for the optimal performance of discrete-time stochastic optimal control problems with incomplete state observation and/or with unknown constant parameters. These bounds are useful for evaluation of various suboptimal policies. For a discrete-state case, Åström (l965) has shown two theorems which give these bounds. Our methods are extensions of his in several ways. Utilizing the result of our methods, an evaluation of the state information obtained from the observation signal is also developed.