: Maintaining a balance between scheduled and actual demand is critical for grid stability and economic efficiency in deregulated electricity markets, particularly under regulatory frameworks such as India’s Deviation Settlement Mechanism (DSM) that imposes nonlinear and frequency-sensitive penalties on deviations. Although artificial intelligence models have significantly enhanced short-term load forecasting (STLF) accuracy, most existing approaches focus on minimizing statistical error metrics like Mean Absolute Error (MAE) and Root Mean Square Error (RMSE), without explicitly addressing economic penalty exposure, resulting in a gap between predictive accuracy and regulatory cost optimization. This research proposes a DSM-aware analytical framework for short-horizon (six-block) demand estimation based on four months of half-hourly operational data which has 5,472 number of time blocks. A structured preprocessing pipeline and domain-driven feature engineering methodology are used to develop 40 features capturing temporal dependencies, cyclic behavior, volatility, and seasonal patterns. Four benchmark models—LSTM, GRU, XGBoost, and Transformer—are trained using a sliding window technique and evaluated with statistical metrics and a DSM cost formulation based on deviation-weighted penalty rates. Results demonstrate that XGBoost obtains the highest statistical accuracy (MAE ≈ 110 MW, RMSE ≈ 192 MW), while LSTM and GRU performs moderate (MAE ≈ 224–249 MW), and Transformer exhibits comparable MAE (~248 MW) but larger RMSE (~347 MW), indicating greater variance. Economic evaluation shows that Transformer achieves the lowest DSM cost (₹6.65 million), outperforming LSTM (₹7.06 million), XGBoost (₹7.19 million), and GRU (₹7.38 million). Residual analysis shows that while XGBoost reduces average error, it produces occasional high-magnitude deviations, whereas Transformer suppresses extreme deviation spikes effectively that influence DSM penalties. These findings emphasize that reducing statistical error does not ensure economic optimality, emphasizing the necessity of tail-risk-aware forecasting and encouraging the development of cost-sensitive, DSM-integrated predictive models for real-world grid operations.