With ongoing advancements in natural language processing (NLP) and deep learning methods, the demand for computational and memory resources has considerably increased, which signifies the determination of efficient and compact models in resource-constrained environments. A comprehensive overview of the most recent advancements in pruning and quantization methods for deep neural networks is provided in this paper. Numerous cutting-edge techniques that harness the complementary advantages of pruning and quantization have been analyzed, highlighting their effectiveness in reducing model size, enhancing computational efficiency, and minimizing memory usage. These techniques include Quantization and Sparsity Aware Fine Tuning, Compression Learning by In-Parallel Pruning-Quantization, GroupReduce, Quantization-Pruned Attention, Structured Pruning, Normalized Linear Quantization (Prune and NLQ), Quantization and Pruning for Sentiment Analysis, an Automatic mixed-precision Quantization approach for BERT compression (AQ-BERT), Mixed Precision Quantization, Unstructured Pruning, and Quantization, and Magnitude Pruning. The datasets utilized, models employed, and outcomes achieved are taken into account within this research. The utilization of pruning and quantization techniques across diverse deep-learning tasks, NLP, and sentiment analysis are also discussed. Moreover, issues such as compatibility with hardware configurations, optimization complexities, accuracy degradation, and other constraints have been analysed. Several challenges and limitations of weight or unit pruning that are utilized for optimizing memory and quantization techniques to enhance precision are explored. The in-depth analysis of these state-of-the-art techniques and experiments provides a broad understanding. Furthermore, strategies to effectively reduce the computational and memory demands of neural networks without compromising their performance are also analysed.

The diesel generators sets (DGs) and battery storage systems (BSS) are the essential energy sources in a modern high-rise buildings. In this paper DG, BSS and Photovoltaic system (PV) has been considered to minimize the grid power injection using a centralized Energy Management System (EMS). Machine Learning (ML) techniques are used to predict the performance of various regression models by comparing grid power and load curves. It includes Artificial Neural Network (ANN), Wide Neural Network (WNN), Linear Regression (LR), Linear Regression Interaction (LR-I), Linear Regression Stepwise (LR-S), Regression Fine Tree (RF-T), Regression Coarse Tree (RC-T) and Gaussian Process Regression (GPR) based techniques. The Demand Side Management (DSM) techniques such as peak shaving and valley filling is integrated with ML technique in a Hybrid energy source (HS) system.The comparative analysis of results depicts the effective reshaping of the grid profile without scheduling or disconnecting the loads. Matlab simulation software is used to validate the results.

Enhancement of the dispatching capacity and efficiency of grid management requires knowledge of photovoltaic power generation beforehand. Intrinsically, photovoltaic power generation is highly volatile and irregular, which impedes its prediction accuracy. In order to handle these constraints, this paper proposes deep learning-based approaches and a pre-processing algorithm. The proposed scheme employs Pearson’s Correlation Coefficient to find the similarity between atmospheric variables and PV power generation. Based on high PCC values, top atmospheric variables, and PV power generated time series data are passed through the Empirical Mode Decomposition (EMD) to simplify the complex data streams into Intrinsic Mode Functions (IMFs). Further, to streamline the prediction process, the proposed correlation-based signal synthesis (CBSS) algorithm finds combinations of these IMFs, which have a high correlation value between atmospheric variables and PV power data. Deep learning models of algorithms Long Short Term Memory (LSTM) and Nonlinear Autoregressive Network with Exogenous Inputs (NARX) network with the configurations of three networks, a single network, and the direct approach employed for the prediction of IMFs combinations. LSTM network was analyzed under the Adaptive moment estimation (ADAM), Stochastic Gradient Descent with Momentum (SGDM), and Root Mean Square Propagation (RMSP) optimization. Extensive experimentation was evaluated using atmospheric data collected at the Climate, Energy, and Water Research Institute (CEWRI), NARC, Islamabad, Pakistan. RMSE, MAE, MAPE, and R ² performance measures show promising prediction results for the LSTM under the configuration of three networks and ADAM optimization.

Deep neural networks (DNN) are in high demand because of their widespread applications in natural language processing, image processing, and a lot of other domains. However, due to their computational expense, over-parameterization, and large memory requirements, DNN applications often require the use of substantial model resources. This strict requirement of latency and limited memory availability are hurdles in the device deployment of these technologies. Therefore, a common idea could be to mitigate the DNN-based models’ size without any performance degradation using different compression techniques. During the last few years, a great deal of progress has been made in the field of Natural Language Processing (NLP) using deep learning approaches. The objective of this survey paper is to offer a thorough overview of the various pruning methods applied in the context of NLP. In this paper, we review several recent pruning-based schemes used for converting standard networks into their compact and accelerated versions. Traditionally, pruning is a technique for improving latency, reducing model size, and computational complexity which is a viable approach to deal with the above-mentioned challenges. In general, these techniques are divided into two main categories: structural and unstructured pruning methods. Structural pruning methods are further classified into filter, channel, layer, block, and movement pruning. Whereas, neuron, magnitude-based, and iterative pruning lie in the category of unstructured pruning. For each method, we discuss the related metrics and benchmarks. Then recent work on each method is discussed in detail, which provides insightful analysis of the performance, related applications, and pros and cons. Then, a comparative analysis is provided to analyze the differences among approaches. Finally, the paper concludes with possible future directions and some technical challenges.

Automatic Target Recognition (ATR) using Correlation Pattern Recognition (CPR) in IoT-based applications encounters limitations like limited memory and inadequate computational resources. One reason is the required quantity of reference templates for each target/object to cover all features of a target/object. To mitigate the issue of reference templates per target/object without accuracy degradation, this paper proposes energy compaction-based CPR autoencoder-training . Additionally, a newly proposed performance metric known as Peak Energy Gain (PEG) estimates the quality of the correlation plane and the feature compression capability CPR methods. The proposed, composite filtering strategy, Eigen Maximum Average Correlation Height (EMACH), and Extended Eigen Maximum Average Correlation Height ( E ² MACH) are vigorously validated using publicly available biometric and object image databases. By training a single reference template, the proposed training method achieves 97.97% mean accuracy with the second-best approach of E ² MACH that attains 53.04% mean accuracy on the Pose Estimation Database. For bio-metric fingerprint verification, the mean Equal Error Rate (EER) of the proposed approach and the composite strategy is 3% and 29.69%, respectively on the FVC2002DB1A database. Similarly, the mean EER of the proposed approach and the composite strategy is 10.55% and 26.32%, respectively on the FVC2006IA database. For FEI faces dataset, the proposed method achieves 1.41% mean EER, and the composite filtering approach achieves 21.43% mean EER. On the University of Tehran Iris database, the proposed autoencoder-based methodology obtains 19.07%, and 18.07% mean EER on the left and right side iris instances, respectively. The comparative results for each dataset demonstrate superiority of AE-based method over the state-of-the-art CPR methods.

Prior knowledge of wind energy production can significantly contribute to the enhancement of the dispatching capacity and grid management efficiency. Innately, wind speed has high unpredictable nature; this hinders the predictor from achieving good accuracy. To address this limitation, the proposed method in this paper provides the pre-processing step before deep learning-based prediction of wind speed. The pre-processing step involves thirteen nature-inspired algorithms that minimize the proposed objective function to obtain IMF combinations of atmospheric variables. The objective function ensures the selection of IMF (Intrinsic Mode Function) combinations of wind speed with a high correlation with the IMF combinations of other atmospheric variables. Each nature-inspired algorithm provides the outputs, which are further refined by the proposed OAIMF (Optimal Search IMF algorithm), which ensures the selection of fewer IMF combinations of wind speed. A case study of the Climate, Energy, and Water Research Institute (CEWRI), NARC, Islamabad, Pakistan, is considered for experimental analysis. RMSE, MAE, MAPE, and R ² performance measures are used for experimental verification. The proposed technique achieved an RMSE of 2.73 on the LSTM network and an RMSE of 3.86 on the GRU network. In the direct method, the RMSE value of 19.78 is obtained on the LSTM network, and the RMSE of 18.89 is achieved on the GRU network. Similarly, the proposed approach outperformed the other state-of-the-art pre-processing prediction techniques based on RMSE, MAE, MAPE, and R ² measures.