It is claimed that the reasons for using matrices of derivatives, in appropriate situations, are as compelling as those for using matrices. This paper provides basic material for such use. Different types of matrix derivatives are defined and illustrated. Simple and easy techniques are then derived and are shown to be applicable to a considerable collection of matrix functions. Applications are made to such problems as establishing matrix integrals from scalar ones, determining maximum likelihood estimates for complex likelihood functions, optimizing matrix functions when there are matrices of side conditions, and evaluating the Jacobians of certain classes of transformations. The emphasis is on simplicity of derivation and on breadth of application.