26 January 2022 in IEEE Access
Machine learning algorithms represent the intelligence that controls many information systems and applications around us. As such, they are targeted by attackers to impact their decisions. Text created by machine learning algorithms has many types of applications, some of which can be considered malicious especially if there is an intention to present machine-generated text as human-generated. In this paper, we surveyed major subjects in adversarial machine learning for text processing applications. Unlike adversarial machine learning in images, text problems and applications are heterogeneous. Thus, each problem can have its own challenges. We focused on some of the evolving research areas such as: malicious versus genuine text generation metrics, defense against adversarial attacks, and text generation models and algorithms. Our study showed that as applications of text generation will continue to grow in the near future, the type and nature of attacks on those applications and their machine learning algorithms will continue to grow as well. Literature survey indicated an increasing trend in using pre-trained models in machine learning. Word/sentence embedding models and transformers are examples of those pre-trained models. Adversarial models may utilize same or similar pre-trained models as well. In another trend related to text generation models, literature showed effort to develop universal text perturbations to be used in both black-and white-box attack settings. Literature showed also using conditional GANs to create latent representation for writing types. This usage will allow for a seamless lexical and grammatical transition between various writing styles. In text generation metrics, research trends showed developing successful automated or semi-automated assessment metrics that may include human judgement. Literature showed also research trends of designing and developing new memory models that increase performance and memory utilization efficiency without validating real-time constraints. Many research efforts evaluate different defense model approaches and algorithms. Researchers evaluated different types of targeted attacks, and methods to distinguish human versus machine generated text.
1 June 2024 in Cluster Computing
The breakthroughs in Machine learning and deep neural networks have revolutionized the handling of critical practical challenges, achieving state-of-the-art performance in various computer vision tasks. Notably, the application of deep neural networks in optical character recognition (OCR) has significantly enhanced the performance of OCR systems, making them a pivotal preprocessing component in text analysis pipelines for crucial applications such as license plate recognition (LPR) systems, where the efficiency of OCR is paramount. However, despite the advancements, the integration of deep neural networks in OCR introduces inherent security vulnerabilities, particularly susceptibility to adversarial examples. Adversarial examples in LPR systems are crafted by introducing perturbations to original license plate images, which can effectively compromise the integrity of the license plate recognition process, leading to erroneous license plate number identification. Given that the primary goal of OCR in this context is to accurately recognize license plate numbers, even a single misinterpreted character can significantly impact the overall performance of the LPR system. The vulnerability of LPR systems to adversarial attacks underscores the urgent need to address the security weaknesses inherited from deep neural networks. In response to these challenges, the exploration of alternative defense mechanisms, such as image denoising and in-painting, presents a compelling approach to bolstering the resilience of LPR systems against adversarial attacks. By prioritizing practical implementation and integration of image denoising and inpainting techniques align with the operational requirements of real-world LPR systems. These methods can be seamlessly integrated into existing pipelines, offering pragmatic and accessible means of enhancing security without imposing significant computational overhead. By embracing a multi-faceted approach that combines the strengths of traditional image processing techniques, the research endeavors to develop comprehensive and versatile defense strategies tailored to the specific vulnerabilities and requirements of LPR systems. This holistic approach aims to fortify LPR systems against adversarial threats, thereby fostering increased trust and reliability in the deployment of OCR and LPR technologies across various domains and applications.
18 May 2022 in IEEE Transactions on Engineering Management
Despite the promising success of federated learning in various application areas, its inherent vulnerability to adversarial attacks hinders its applicability in security-critical areas. This calls for developing viable defense measures against such attacks. A prerequisite for this development, however, is the understanding of what creates, promotes, and aggravates this vulnerability. To date, developing this understanding remains an outstanding gap in the literature. Accordingly, this paper presents an attempt at developing such an understanding. Primarily, this is achieved from two main perspectives. The first perspective concerns addressing the factors, elements, and parameters contributing to the vulnerability of federated learning models to adversarial attacks, their degrees of severity, and combined effects. This includes addressing diverse operating conditions, attack types and scenarios, and collaborations between attacking agents. The second perspective regards analyzing the appearance of the adversarial property of a model in how it updates its coefficients and exploiting this for defense purposes. These analyses are conducted through extensive experiments on image and text classification tasks. Simulation results reveal the importance of specific parameters and factors on the severity of this vulnerability. Besides, the proposed defense strategy is shown able to provide promising performances.
27 September 2023 in arXiv
Federated learning enables multiple clients to collaboratively contribute to the learning of a global model orchestrated by a central server. This learning scheme promotes clients' data privacy and requires reduced communication overheads. In an application like network traffic classification, this helps hide the network vulnerabilities and weakness points. However, federated learning is susceptible to backdoor attacks, in which adversaries inject manipulated model updates into the global model. These updates inject a salient functionality in the global model that can be launched with specific input patterns. Nonetheless, the vulnerability of network traffic classification models based on federated learning to these attacks remains unexplored. In this paper, we propose GABAttack, a novel genetic algorithm-based backdoor attack against federated learning for network traffic classification. GABAttack utilizes a genetic algorithm to optimize the values and locations of backdoor trigger patterns, ensuring a better fit with the input and the model. This input-tailored dynamic attack is promising for improved attack evasiveness while being effective. Extensive experiments conducted over real-world network datasets validate the success of the proposed GABAttack in various situations while maintaining almost invisible activity. This research serves as an alarming call for network security experts and practitioners to develop robust defense measures against such attacks.