The general recursive parameter estimation problem of identifying unknown parameters subject to sequential observation is studied from the information theoretic viewpoint. The entropy theorem for the recursive estimation problem is first presented to give the upper and lower bounds of the reduction of the processed error entropy under sequential estimation. This result is then developed to yield the conditional entropy theorem; a lower bound on the conditional error entropy given past measurements is obtained. As an application of the conditional entropy theorem, recursive state estimation problems such as filtering, smoothing and prediction are investigated.