Accurate load forecasting is of vital importance for improving the energy utilization efficiency and economic profitability of intelligent buildings. However, load forecasting is restricted in the popularization and application of conventional load forecasting techniques due to the great difficulty in obtaining numerical weather prediction data at the hourly level and the requirement to conduct predictions on multiple time scales. Under the condition of lacking meteorological forecast data, this paper proposes to utilize a temporal convolutional network (TCN) to extract the coupled spatial features among multivariate loads. The reconstructed features are then input into the long short-term memory (LSTM) neural network to achieve the extraction of load time features. Subsequently, the self-attention mechanism is employed to strengthen the model’s ability to extract feature information. Finally, load forecasting is carried out through a fully connected network, and a multi-time scale prediction model for building multivariate loads based on TCN–LSTM–self-attention is constructed. Taking a hospital building as an example, this paper predicts the cooling, heating, and electrical loads of the hospital for the next 1 h, 1 day, and 1 week. The experimental results show that on multiple time scales, the TCN–LSTM–self-attention prediction model proposed in this paper is more accurate than the LSTM, CNN-LSTM, and TCN-LSTM models. Especially in the task of predicting cooling, heating, and electrical loads on a 1-week scale, the model proposed in this paper achieves improvements of 16.58%, 6.77%, and 3.87%, respectively, in the RMSE indicator compared with the TCN-LSTM model.