Abstract: The upper bound finite element method converts the problem of finding a kinematic admissible velocity field into a mathematical programming one, which can overcome the difficulty of artificially constructing a kinematic velocity field, thus, it has a broad prospect in applications to complex problems. The formulation of the upper bound finite element method based on nonlinear programming can avoid linearization of yield functions, as a result, it greatly reduces the optimization variables and saves a great deal of memory space. However, this leads to a nonlinear programming model that is quite complex. By introducing a nonlinear upper bound programming model, the steps for its optimization using feasible arc interior point algorithm are discussed. Firstly, the BFGS formula is taken as the updating rules for Hessian of yield functions to avoid the ill-conditioning problem in computation. Secondly, by constructing a feasible arc, the shortcoming of a too short step when the current iteration point reaches the nonlinear constraint boundary is overcome. Finally, the Wolfe's line search technique is used for step-length search which enhances the line search efficiency. Example analysis by MATLAB programming shows that the proposed method is highly efficient, numerically stable and accurate enough for engineering practice, thus, it is applicable to most soil stability problems.