Oxygen uptake ($\dot{\mathrm{V}}\mathrm{O}$${}_2$) is an important metric in any exercise test including walking and running. It can be measured using portable spirometers or metabolic analyzers. Those devices are, however, not suitable for constant use by consumers due to their costs, difficulty of operation and their intervening in the physical integrity of their users. Therefore, it is important to develop approaches for the indirect estimation of $\dot{\mathrm{V}}\mathrm{O}$${}_2$-based measurements of motion parameters, heart rate data and application-specific measurements from consumer-grade sensors. Typically, these approaches are based on linear regression models or neural networks. This study investigates how motion data contribute to $\dot{\mathrm{V}}\mathrm{O}$${}_2$ estimation accuracy during unconstrained running and walking. The results suggest that a long short term memory (LSTM) neural network can predict oxygen consumption with an accuracy of 2.49 mL/min/kg (95% limits of agreement) based only on speed, speed change, cadence and vertical oscillation measurements from an inertial navigation system combined with a Global Positioning System (INS/GPS) device developed by our group, worn on the torso. Combining motion data and heart rate data can significantly improve the $\dot{\mathrm{V}}\mathrm{O}$${}_2$ estimation resulting in approximately 1.7–1.9 times smaller prediction errors than using only motion or heart rate data.