We explore the application of Large Language Models (LLMs) (specifically GPT-3.5-turbo) to extract specifications and automating understanding of networking protocols from Internet Request for Comments (RFC) documents. LLMs have proven successful in specialized domains like medical and legal text understanding, and this work investigates their potential in automatically comprehending RFCs. We develop Artifact Miner, a tool to extract diagram artifacts from RFCs. We then couple extracted artifacts with natural language text to extract protocol automata using GPT-turbo 3.5 (chatGPT) and present our zero-shot and few-shot extraction results. We call this framework for FSM extraction 'PROSPER: Protocol Specification Miner'. We compare PROSPER with existing state-of-the-art techniques for protocol FSM state and transition extraction. Our experiments indicate that employing artifacts along with text for extraction can lead to lower false positives and better accuracy for both extracted states and transitions. Finally, we discuss efficient prompt engineering techniques, the errors we encountered, and pitfalls of using LLMs for knowledge extraction from specialized domains such as RFC documents.
The evolution from 5G to 6G cellular networks signifies a crucial advancement towards enhanced robustness and automation driven by the promise of ubiquitous Artificial Intelligence (AI) to overhaul network operations, commonly referred to as AIOps. However, 6G network operators also need to deal with evolving threats at the edge to ensure data integrity and availability. We introduce 6G-XSEC, the first framework that seeks to automatically monitor, analyze, and explain anomalies and threats at the cellular network edge. Our framework enhances the emerging Open Radio Access Network (O-RAN) control plane with run-time analytic capabilities and explainability. A distinguishing aspect of our framework is the use of expert referencing, a coupling of lightweight unsupervised deep learning-based anomaly detection with large language models (LLMs) to first detect, analyze, and subsequently explain complicated real-world cellular threats and anomalies at run-time, based on enhanced security telemetry from the O-RAN data plane. We build a prototype 6G-XSEC framework and evaluate it against 5 end-to-end cellular attacks from the literature, achieving 100% detection rate with our best model. We also propose effective LLM prompt templates for attack analysis and present qualitative results from 5 popular LLMs.
Online medical forums have become a predominant platform for answering health-related information needs of consumers. However, with a significant rise in the number of queries and the limited availability of experts, it is necessary to automatically classify medical queries based on a consumer's intention, so that these questions may be directed to the right set of medical experts. Here, we develop a novel medical knowledge-aware BERT-based model (MedBERT) that explicitly gives more weightage to medical concept-bearing words, and utilize domain-specific side information obtained from a popular medical knowledge base. We also contribute a multi-label dataset for the Medical Forum Question Classification (MFQC) task. MedBERT achieves state-of-the-art performance on two benchmark datasets and performs very well in low resource settings.