Abstract: The reduced-scaled physical model test under the conventional gravity of 1g is an important and common method for analyzing failure of embankment dams due to overtopping. However, the similarity of the overtopping failure process between the prototype embankment dam and the scaled model is a lasting technical problem to be resolved. Based on the incipient limit state analysis of homogeneous clay dams under the action of overtopping flow, the idea of adjusting the compaction degree of the model dam is proposed to reduce its erosion resistance so that the similarity optimization on the overtopping failure process of the homogeneous clay dam can be achieved. The shear strength and cohesion of clay decrease nonlinearly with the decrease of compaction degree. Taking the dam break of an actual embankment dam as the prototype, similar hydraulic flume tests on break of homogeneous clay dams due to overtopping are conducted with different compaction degrees and geometric scales. The results show that the breaching process of the model dam with the same compactness as the prototype is completely different from that of the prototype, and the difference between the model and the prototype will become more significant with the decrease of the geometric scale. Appropriately reducing the compaction degree of the model clay dam can significantly improve the similarity between the model and the prototype in breach processes such as breach morphology and time.