Abstract: Based on the limit equilibrium theory and the slice analysis method for the stability of soil-nailing structure, according to the assumption of parabolic slip surface for the relationship between the composite soil-nailing support structure and the stability factor of safety, using the geometric relationship and the integration method, a searching model for the most dangerous slip surface is established, and a formula for stability factor of safety is derived. The function relations between the geometrical controlling parameters of sliding surface and the stability factor of safety are deduced. The integral stability of a foundation pit with prestressed anchor composite soil nailing structure in Lanzhou is calculated by using the genetic algorithms. Finally, based on the circular sliding failure model, the stability factor of safety is compared and analyzed. The results show that the proposed method can avoid the optimization design into the local minimum value. Therefore, it is proved that this method has the calculation efficiency and certain accuracy.