Building energy consumption prediction has a significant effect on energy control, design optimization, retrofit evaluation, energy price guidance, and prevention and control of COVID-19 in buildings, providing a guarantee for energy efficiency and carbon neutrality. This study reviews 116 research papers on data-driven building energy prediction from the perspective of data and machine learning algorithms and discusses feasible techniques for prediction across time scales, building levels, and energy consumption types in the context of the factors affecting data-driven building energy prediction. The review results revealed that the outdoor dry-bulb temperature is a vital factor affecting building energy consumption. In data-driven building energy consumption prediction, data preprocessing enables prediction across time scales, energy consumption feature extraction enables prediction across energy consumption types, and hyperparameter optimization enables prediction across time scales and building layers.