Since the launch of ChatGPT, a powerful AI Chatbot developed by OpenAI, large language models (LLMs) have made significant advancements in both academia and industry, bringing about a fundamental engineering paradigm shift in many areas. While LLMs are powerful, it is also crucial to best use their power where “prompt” plays a core role. However, the booming LLMs themselves, including excellent APIs like ChatGPT, have several inherent limitations: (1) temporal lag of training data, and (2) the lack of physical capabilities to perform external actions. Recently, we have observed the trend of utilizing prompt-based tools to better utilize the power of LLMs for downstream tasks, but a lack of systematic literature and standardized terminology, partly due to the rapid evolution of this field. Therefore, in this work, we survey related prompting tools and promote the concept of the “Prompting Framework” (PF), i.e. the framework for managing, simplifying, and facilitating interaction with LLMs. We define the lifecycle of the PF as a hierarchical structure, from bottom to top, namely: Data Level, Base Level, Execute Level, and Service Level. We also systematically depict the overall landscape of the emerging PF field and discuss potential future research and challenges. To continuously track the developments in this area, we maintain a repository at https://github.com/lxx0628/Prompting-Framework-Survey , which can be a useful resource sharing platform for both academic and industry in this field.

In industrial control systems, the generation and verification of Programmable Logic Controller (PLC) code are crucial for ensuring operational efficiency and safety. While Large Language Models (LLMs) have made strides in automated code generation, they fall short in providing correctness guarantees and specialized support for PLC programming (which has its own programming language and clear logical structures). To address these challenges, this paper introduces Agents4PLC, a novel framework that not only automates PLC code generation but also introduces code-level verification and repair built upon an LLM-based multi-agent system, which together is capable of directly producing operational PLC code without any human interaction. To comprehensively evaluate our framework, we first establish a new benchmark specially designed for the critical area of verifiable PLC code generation, which includes hundreds of natural language requirements, human-written and verified formal specifications, and finally reference PLC code. Then, we carefully designed a multi-agent workflow combining a set of expert agents responsible for different code generation tasks including planning, coding, validation and debugging towards generating correct PLC code. For each agent, we also incorporate optimization strategies such as Retrieval-Augmented Generation (RAG), advanced prompt engineering techniques, and Chain-of-Thought strategies which are shown to be effective to enhance the ability of these expert ‘agents’. Evaluation against the benchmark demonstrates that Agents4PLC significantly outperforms existing methods, achieving superior results across a series of increasingly rigorous evaluation metrics. This research highlights the potential of LLM agent-based code generation in real-world industrial control systems and the importance of code-level verification in generating correct code with formal guarantees.

Zero-shot fault diagnosis can identify unseen faults by predicting attributes. However, existing methods ignore the multi-grained characteristics of attributes, namely the varying levels of detail in describing fault categories. We recognize the following considerations for the first time: (1) attributes show typical multi-grained characteristics, which could be expressed in a coarse-to-fine-grained hierarchical structure; (2) multi-grained attributes play different roles in fault diagnosis, where coarse-grained attributes indicate the rough range of faults, while fine-grained attributes facilitate the precise identification of fault types. In this paper, a fuzzy hierarchical zero-shot learning method is proposed to solve these issues. First, the attributes are divided into different layers according to the coarse-to-fine granularity via expert knowledge rather than being treated equally. Then, a knowledge transfer strategy is designed to transfer the knowledge from coarse-grained attributes to fine-grained ones, which can improve attribute prediction accuracy. Finally, a fuzzy inference strategy is developed to distinguish the effect of attributes with different granularity on fault inference. This strategy can identify the faults stepwise in a coarse-to-fine-grained order. The effectiveness of the proposed method is verified by a real thermal power plant process.

Specific emitter identification (SEI) is significant in military communication scenarios, cognitive radio and self-organized networks. However, these methods only consider the feature of signals or the feature after signal transformation. In other words, the time-domain correlation of each feature and relationships between features are seldom taken into account. A novel method is therefore proposed, which includes a transformation to convert the specific emitter signal into a graph tensor and a model named time domain graph tensor attention network (TDGTAN) to encode graph tensors for SEI. Specifically, the model includes two main parts. The first part is intra graph propagation, which uses the relationship between different sampling points through message propagation in each graph. The other part is inter propagation, which propagates cross layer messages between different graphs at the same sampling point, so as to realize the use of the relationship between different features. Extensive experiments are conducted on real-world dataset, and the result shows that the proposed approach acquires higher accuracy and intriguing anti-interference performance. In addition, the proposed model also has higher parameter utilization and calculation efficiency.

Generative large language models (LLMs) have revolutionized natural language processing with their transformative and emergent capabilities. However, recent evidence indicates that LLMs can produce harmful content that violates social norms, raising significant concerns regarding the safety and ethical ramifications of deploying these advanced models. Thus, it is both critical and imperative to perform a rigorous and comprehensive safety evaluation of LLMs before deployment. Despite this need, owing to the extensiveness of LLM generation space, it still lacks a unified and standardized risk taxonomy to systematically reflect the LLM content safety, as well as automated safety assessment techniques to explore the potential risks efficiently. To bridge the striking gap, we propose S-Eval, a novel LLM-based automated Safety Evaluation framework with a newly defined comprehensive risk taxonomy. S-Eval incorporates two key components, i.e., an expert testing LLM M _t and a novel safety critique LLM M _c . The expert testing LLM M _t is responsible for automatically generating test cases in accordance with the proposed risk management (including 8 risk dimensions and a total of 102 subdivided risks). The safety critique LLM M _c can provide quantitative and explainable safety evaluations for better risk awareness of LLMs. In contrast to prior works, S-Eval differs in significant ways: (i) efficient – we construct a multi-dimensional and open-ended benchmark comprising 220,000 test cases across 102 risks utilizing M _t and conduct safety evaluations for 21 influential LLMs via M _c on our benchmark. The entire process is fully automated and requires no human involvement. (ii) effective – extensive validations show S-Eval facilitates a more thorough assessment and better perception of potential LLM risks, and M _c not only accurately quantifies the risks of LLMs but also provides explainable and in-depth insights into their safety, surpassing comparable models such as LLaMA-Guard-2. (iii) adaptive – S-Eval can be flexibly configured and adapted to the rapid evolution of LLMs and accompanying new safety threats, test generation methods and safety critique methods thanks to the LLM-based architecture. We further study the impact of hyper-parameters and language environments on model safety, which may lead to promising directions for future research. S-Eval has been deployed in our industrial partner for the automated safety evaluation of multiple LLMs serving millions of users, demonstrating its effectiveness in real-world scenarios.

With the development of wireless communication, specific emitter identification (SEI) is important for the management and security of instrumentation and smart devices. Given that the signal differences between different devices of the same type are caused by hardware damage and are mainly concentrated in the high frequencies, the high-frequency component of the signal is reconstructed by Fourier transform, attention mechanism, and inverse Fourier transform in this paper. The reconstructed high-frequency component of the signal is then fed into a recurrent neural network (RNN) to extract features from the time dimension. The frequency attention module and the time attention module are connected serially, which on the one hand allows the overall network to pay attention to both the frequency and time characteristics without increasing the amount of data, and on the other hand ensures that the results of the frequency attention must facilitate the subsequent RNN for feature extraction. The parameter sizes of many SEI methods are measured. The results show that the model proposed in this paper has the highest parameter efficiency and low storage costs. The results on a real-world dataset show that the proposed model has the highest accuracy. These advantages have essential significance for deploying the model in practical applications.