A class of penalty methods for solving a nonlinear optimization problem is considered, which – starting at an arbitrary point – produce trial solutions belonging to the interior of the feasible region after a finite number of steps and which in the convex case are Hadamard-stable algorithms needing no compactness conditions for convergence, if the Slater condition is satisfied. Otherwise these methods work as exterior penalty methods.