Intermittency in the grid creates operational issues for power system operators (PSO). One such intermittent parameter is load. Accurate prediction of the load is the key to proper planning of the power system. This paper uses regression analyses for short-term load forecasting (STLF). Assumed load data are first analyzed and outliers are identified and treated. The cleaned data are fed to regression methods involving Linear Regression, Decision Trees (DT), Support Vector Machine (SVM), Ensemble, Gaussian Process Regression (GPR), and Neural Networks. The best method is identified based on statistical analyses using parameters such as Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Square Error (MSE), $R^2$, and Prediction Speed. The best method is further optimized with the objective of reducing MSE by tuning hyperparameters using Bayesian Optimization, Grid Search, and Random Search. The algorithms are implemented in Python and Matlab Platforms. It is observed that the best methods obtained for regression analysis and hyperparameter tuning for an assumed data set are Decision Trees and Grid Search, respectively. It is also observed that, due to hyperparameter tuning, the MSE is reduced by 12.98%.