A correlation matrix between three variables has to satisfy certain conditions. Such a matrix essentially contains three numbers and thus can be represented by a point in three dimensions. The set of all possible correlation matrices yields a convex solid body with an uncommon shape. All its cross sections perpendicular to the axes are ellipses. At the same time, its surface contains the vertices and edges of a regular tetrahedron. Another unusual shape is obtained for banded correlation matrices between four variables.