The paper presents approaches far a number of classical and elementary topics in probability and optimization in estimation theory. The author hopes that the approaches can offer useful alternatives (such as simpler or more complete derivations) to the classical approaches. Most topics are illustrated by one or more typical examples. In the area of estimation theory we present a definition of MLE, which resolves the well-known problems on the boundary of the parameter space, and an elementary derivation of the normal equations in least squares estimation.