Consider a natural exponential family parameterized byθ. It is well known that the standard conjugate prior onθis characterized by a condition of posterior linearity for the expectation of the model mean parameterμ. Often, however, this family is not parameterized in terms ofθbut rather in terms of a more usual parameter, such at the meanμ. The main question we address is: Under what conditions does astandard conjugate prior on μinduce a linear posterior expectation onμitself? We prove that essentially this happens iff the exponential family has quadratic variance function. A consequence of this result is that the standard conjugate onμcoincides with the prior onμinduced by the standard conjugate onθiff the variance function is quadratic. The rest of the article covers more specific issues related to conjugate priors for exponential families. In particular, we analyze the monotonicity of the expected posterior variance forμwith respect to the sample size and the hyperparameter “prior sample size” that appears in the conjugate distribution. Finally, we consider a situation in which a class of priors onθ, say Γ, is specified by some moment conditions. We revisit and extend previous results relating conjugate priors to Γ-least favorable distributions and Γ-minimax estimators.