Rural households in the Mahafaly region of semi-arid SW-Madagascar strongly depend on the exploitation of natural resources for their basic needs and income regeneration. An overuse of such resources threatens the natural environment and people’s livelihood. Our study focuses on the diversity and use of wild yams and medicinal plants. We hypothesized that knowledge on the use of these resources highly depends on farmers’ socio-economic household characteristics. To test this hypothesis, an ethnobotanical survey was conducted based on semi-structured interviews recording socio-economic base data and information on local knowledge of medicinal and wild yam species. This was followed by field inventories compiling plant material for botanical identification. Six species of wild yam and a total of 214 medicinal plants from 68 families and 163 genera were identified. Cluster and discriminant analysis yielded two groups of households with different wealth status characterized by differences in livestock numbers, off-farm activities, agricultural land and harvests. A generalized linear model highlighted that economic factors significantly affect the collection of wild yams, whereas the use of medicinal plants depends to a higher degree on socio-cultural factors. Wild yams play an important role in local food security in the Mahafaly region, especially for poor farmers, and medicinal plants are a primary source of health care for the majority of local people. Our results indicate the influence of socio-economic household characteristics on the use of forest products and its intensity, which should be considered in future management plans for local and regional forest conservation.