In conventional federated learning, each device is restricted to train a network model of a same structure. This greatly hinders the application of federated learning in edge devices and IoT scenarios where the data and devices are quite heterogeneous because of their different hardware equipment and communication networks. At the same time, most of the existing studies about federated learning of heterogeneous models are limited to horizontal heterogeneity which share a highly homogeneous vertical structure. Little work has been done on vertical heterogeneity such as models with different number of functional layers or different connection methods within them, not to mention the integrated heterogeneity scenarios. In DraftFed, a novel draft-based approach is proposed to implement personalized federated learning for integrated heterogeneous models. Unlike traditional federated learning in which the parameters/gradients are exchanged, DraftFed uses drafts as key knowledge to guide mutual learning of models, which makes it suitable for model structure personalization application scenarios..

Cyber Threat Intelligence (CTI) plays a crucial role in cybersecurity. However, traditional information extraction has low accuracy due to the specialization of CTI and the concealment of relations. To improve the performance of CTI relation extraction in the knowledge graph, we propose a relation extraction architecture called Adversarial Training for Cyber Threat Intelligence Relation Extraction (AT4CTIRE). Additionally, we developed a large-scale cybersecurity dataset for CTI analysis and evaluation called Cyber Threat Intelligence Analysis (CTIA). Inspired by Generative Adversarial Networks, we integrate contextual semantics to refine our study. Firstly, we use some wrong triples with incorrect relations to train the generator and produce high-quality generated triples as adversarial samples. Secondly, the discriminator used actual and generated samples as training data. Integrating the discriminator and the context-embedding module facilitates a deeper understanding of contextual CTI within threat triples. Finally, training a discriminator identified the relation between the threat entities. Experimentally, we set two CTI datasets and only one baseline that we could find to test the effect in the cybersecurity domain. We also took general knowledge graph completion tests. The results demonstrate that AT4CTIRE outperforms existing methods with improved extraction accuracy and a remarkable expedited training convergence rate.

Feng Xie, Xin Song, Xiang Zeng, Xuechen Zhao, Lei Tian, Bin Zhou, Yusong Tan. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.