Losses at every stage in the food system influence the extent to which nutritional requirements of a growing global population can be sustainably met. Inefficiencies and losses in agricultural production and consumer behaviour all play a role. This paper aims to understand better the magnitude of different losses and to provide insights into how these influence overall food system efficiency. We take a systems view from primary production of agricultural biomass through to human food requirements and consumption. Quantities and losses over ten stages are calculated and compared in terms of dry mass, wet mass, protein and energy. The comparison reveals significant differences between these measurements, and the potential for wet mass figures used in previous studies to be misleading. The results suggest that due to cumulative losses, the proportion of global agricultural dry biomass consumed as food is just 6% (9.0% for energy and 7.6% for protein), and 24.8% of harvest biomass (31.9% for energy and 27.8% for protein). The highest rates of loss are associated with livestock production, although the largest absolute losses of biomass occur prior to harvest. Losses of harvested crops were also found to be substantial, with 44.0% of crop dry matter (36.9% of energy and 50.1% of protein) lost prior to human consumption. If human over-consumption, defined as food consumption in excess of nutritional requirements, is included as an additional inefficiency, 48.4% of harvested crops were found to be lost (53.2% of energy and 42.3% of protein). Over-eating was found to be at least as large a contributor to food system losses as consumer food waste. The findings suggest that influencing consumer behaviour, e.g. to eat less animal products, or to reduce per capita consumption closer to nutrient requirements, offer substantial potential to improve food security for the rising global population in a sustainable manner.