21 September 2021 in Transactions of the Association for Computational Linguistics
Pre-trained Transformer-based models have achieved state-of-the-art performance for various Natural Language Processing (NLP) tasks. However, these models often have billions of parameters, and thus are too resource- hungry and computation-intensive to suit low- capability devices or applications with strict latency requirements. One potential remedy for this is model compression, which has attracted considerable research attention. Here, we summarize the research in compressing Transformers, focusing on the especially popular BERT model. In particular, we survey the state of the art in compression for BERT, we clarify the current best practices for compressing large-scale Transformer models, and we provide insights into the workings of various methods. Our categorization and analysis also shed light on promising future research directions for achieving lightweight, accurate, and generic NLP models.
6 November 2019 in IEEE Journal on Selected Areas in Communications
Due to recent cyber attacks on various cyberphysical systems (CPSes), traditional isolation based security schemes in the critical systems are insufficient to deal with the smart adversaries in CPSes with advanced information and communication technologies (ICTs). In this paper, we develop real-time assessment and mitigation of an attack’s impact as a system’s built-in mechanisms. We study a general class of attacks, which we call time delay attack, that delays the transmissions of control data packets in the CPS control loops. Based on a joint stability-safety criterion, we propose the attack impact assessment consisting of (i) a machine learning (ML) based safety classification, and (ii) a tandem stability-safety classification that exploits a basic relationship between stability and safety, namely that an unstable system must be unsafe whereas a stable system may not be safe. In this assessment approach, the ML addresses a state explosion problem in the safety classification, whereas the tandem structure reduces false negatives in detecting unsafety arising from imperfect ML. We apply our approach to assess the impact of the attack on power grid automatic generation control, and accordingly develop a two-tiered mitigation that tunes the control gain automatically to restore safety where necessary and shed load only if the tuning is insufficient. We also apply our attack impact assessment approach to a thermal power plant control system consisting of two PID control loops. A mitigation approach by tuning the PID controller is also proposed. Extensive simulations based on a 37-bus system model and a thermal power plant control system are conducted to evaluate the effectiveness of our assessment and mitigation approaches.
11 February 2021 in IEEE Transactions on Smart Grid
Control and communication technologies are key building blocks of cyber-physical systems (CPSes) that can improve the efficiency of the physical processes. However, they also make a CPS vulnerable to cyberattacks that can cause disruptions or even severe damage. This article focuses on one particular type of CPS cyberattack, namely the time delay attack (TDA), which exploits vulnerabilities in the communication channels to cause potentially serious harm to the system. Much work proposed for TDA detection is tested offline only and under strong assumptions. In order to construct a practical solution to deal with real-world scenarios, we propose a deep learning-based method to detect and characterize TDA. Specifically, we design a hierarchical long short-term memory model to process raw data streams from relevant CPS sensors online and continually monitor embedded signals in the data to detect and characterize the attack. Moreover, various strategies of interpreting the outputs of the model are proposed, which allow the user to tune the performance based on different objectives. We evaluate our model on two representative types of CPS, namely power plant control system (PPCS) and automatic generation control (AGC).Code and dataset can be found at: https://github.com/prakharg24/tda For TDA detection, our solution achieves an accuracy of 92% in PPCS, compared with 81% by random forests (RFs) and 72% by k-nearest neighbours (kNNs). For AGC, our solution achieves 98% accuracy, compared with 74% by RFs and 71% by kNNs. It also reduces the mean absolute error in the delay value characterization from about six to two seconds in the PPCS, and from about three seconds to half a second in the AGC, with about 3x to 4x shorter reaction latency in both systems.
29 July 2019 in ACM Transactions on Sensor Networks
Separation of Control and Data Planes (SCDP) is a desirable paradigm for low-power multi-hop wireless sensor networks requiring high network performance and manageability. Existing SCDP networks generally adopt an in-band control plane scheme in that the control-plane messages are delivered by their data-plane networks. The physical coupling of the two planes may lead to undesirable consequences. Recently, multi-radio platforms (e.g., TI CC1350 and OpenMote B) are increasingly available, which make the physical separation of the control and data planes possible. To advance the network architecture design, we propose to leverage on the long-range communication capability of the Low-Power Wide-Area Network (LPWAN) radios to form one-hop out-of-band control planes. LoRaWAN, an open, inexpensive, and ISM band based LPWAN radio, is chosen to prototype our out-of-band control plane called LoRaCP. Several characteristics of LoRaWAN such as downlink-uplink asymmetry and primitive ALOHA media access control need to be dealt with to achieve high reliability and efficiency. To address these challenges, a TDMA-based multi-channel transmission control is designed, which features an urgent channel and negative acknowledgment. On a testbed of 16 nodes, LoRaCP is applied to physically separate the control-plane network of the Collection Tree Protocol (CTP) from its Zigbee-based data-plane network. Extensive experiments show that LoRaCP increases CTP’s packet delivery ratio from 65% to 80% in the presence of external interference, while consuming a per-node average radio power of 2.97mW only.
5 February 2020 in IEEE Transactions on Network Science and Engineering
In the smart grid, the high dynamics of electrical loads introduce uncertainty to the system. Reporting demand information by customers in solving the economic dispatch control (EDC) problem is a promising approach, but it raises important privacy concerns. One solution is to add random noise to aggregate queries of demand reports can provide differential privacy (DP) for individual customers. However, the noisy query information can adversely degrade the EDC's optimality. In this paper, we first analyze the cost in demand reporting in terms of how DP-induced noise will increase the total generation cost and show that the noise amounts are intricately coupled among customers in determining the total cost. To deal with the coupling, we apply the principle of Shapley value to attribute fair shares of the total cost and study the properties of this cost-sharing approach. To achieve efficient sharing, we propose heuristic algorithms to approximate the Shapley value. Moreover, we study the important network effects of the per-customer DP cost. Accordingly, we design a pricing scheme for the customers that we prove to be truthful. Trace-driven simulations based on a 5-bus power system model validate our analysis and illustrate the performance of the proposed cost-sharing algorithms.