Physical vulnerability is a challenging and fundamental issue in landslide risk assessment. Previous studies mostly focus on generalized vulnerability assessment from landslides or other types of slope failures, such as debris flow and rockfall, while the long-term damage induced by slow-moving landslides is usually ignored. In this study, a method was proposed to construct physical vulnerability curves for masonry buildings by taking the Manjiapo landslide as an example. The landslide's force acting on the buildings' foundation is calculated by applying the landslide residual-thrust calculation method. Considering four rainfall scenarios, the buildings' physical responses to the thrust are simulated in terms of potential inclination by using Timoshenko's deep-beam theory. By assuming the landslide safety factor to be landslide intensity and inclination ratio to be vulnerability, a physical vulnerability curve is fitted and the relative function is constructed by applying a Weibull distribution function. To investigate the effects of buildings' parameters that influence vulnerabilities, the length, width, height, and foundation depth and Young's modulus of the foundation are analysed. The validation results on the case building show that the physical vulnerability function can give a good result in accordance with the investigation in the field. The results demonstrate that the building length, width, and foundation depth are the three most critical factors that affect the physical vulnerability value. Also, the result shows that the higher the ratio of length to width of the building, the more serious the damage to the building. Similarly, the shallower the foundation depth is, the more serious the damage will be. We hope that the established physical vulnerability curves can serve as tools for the quantitative risk assessment of slow-moving landslides.