The coordination of distributed energy resources (DERs) within virtual power plants (VPPs) is expected to generate significant economic benefits and enhance the operational stability of modern power systems. However, achieving massive coordination of heterogeneous and uncertain DERs remains a challenge in current research. To address this issue, this article proposes a novel bi-level optimization approach based on mean-field reinforcement learning (MFRL) to enable the coordination of massive DERs in VPPs. The problem is decomposed into multiple subproblems: the upper-level subproblem models power dispatch among integrated energy systems (IESs) in response to coordinated demand, while a series of lower-level subproblems determine the operational schemes of DERs within individual IESs. Considering the large decision space, an MFRL algorithm with fast Shapley credit allocation is developed to efficiently solve the upper-level optimization. Meanwhile, the lower-level subproblems are formulated as small-scale mixed-integer linear programming (MILP) problems, addressing the difficulties caused by IES heterogeneity in applying mean-field approximation. Simulation results show that the proposed approach significantly improves convergence speed and reduces the global cost of VPP operation, especially in massive-scale scenarios. In test scenarios ranging from 10 to 500 agents, the proposed bi-level optimization approach improves the objective by 4.8%–26.6%, compared to the advanced baseline method.

Effective management of multi-intersection traffic signal control (MTSC) is vital for intelligent transportation systems. Multi-agent reinforcement learning (MARL) has shown promise in achieving MTSC. However, existing MARL-based MTSC algorithms have primarily focused on capturing the spatial relationship between multi-intersection traffic signals but have overlooking the importance of the temporally stable traffic pattern . This pattern refers to the fixed positions and relatively stable traffic flow between intersections over short periods in real-world MTSC scenarios, which indicates that the learned spatial relationships between traffic signals should co-evolve over time. To this end, we propose a novel algorithm called Coevolutionary Multi-Agent Reinforcement Learning (CoevoMARL). CoevoMARL employs a graph neural network to capture the complex spatial interaction network among traffic signals. Furthermore, we propose a relationship-driven progressive LSTM (RDP-LSTM) that dynamically evolves the learned spatial interaction network over time by leveraging insights from the temporally stable traffic pattern . To accelerate convergence, we also propose the mutual information reward optimization (MIRO) technique, which strengthens the correlation between policy learning and high-performance samples by using a mutual information-based intrinsic reward. Experimental results on both synthetic and realistic datasets demonstrate the superiority of CoevoMARL over existing MTSC algorithms, providing valuable insights into incorporating the temporally stable traffic pattern.

Interpretable Policy Distillation aims to imitate a Deep Reinforcement Learning (DRL) policy into a self-explainable model. However, the distilled policy usually does not generalize well to complex tasks. To investigate this phenomenon, we examine the experience pools of DRL tasks and find that these interactive experience distributions are heavy-tailed. However, this critical issue is largely ignored by existing approaches and thus, they do not fully unitize the less frequent but very critical experience points. To address this issue, we propose to characterize decision boundaries via the minimum experience retention to deal with the heavy-tailed experience distributions. Our method identifies critical experience points that are close to the model’s decision boundaries and such experience points are more critical because they portray the prerequisite of a model to take an action. As a result, our method distills the DRL policy to a self-explainable structure without neural structure and ambiguous intermediate parameters. Through experiments on six games, we show that our method outperforms the state-of-the-art baselines in cumulative rewards, stability, and faithfulness.

Legal text generation and legal question-and-answer tasks impose stringent requirements on factual accuracy, evidence traceability and normative consistency. Conventional dynamic retrieval-augmented generation methods are difficult to directly adapt to legal task demands such as legal provision citation, terminology standardization and case evidence organization. Following the DRAGIN paradigm, this paper proposes LARIN (Lightweight Adapted Retrieval-Augmented Inference Network for Legal Issues), an adaptive retrieval-augmented reasoning framework tailored for legal scenarios. While maintaining the mainstream dynamic retrieval workflow, LARIN makes targeted optimizations in three key modules: retrieval triggering, query construction and evidence fusion. Specifically, LINDA identifies retrieval trigger points by comprehensively considering uncertainty, attention influence and semantic importance; JUDGE rewrites queries for legal terms and statutory expressions; MERF conducts relevance ranking, redundancy elimination and evidence fusion for legal provisions and case materials. In the CAIL2018 Chinese legal judgment prediction task, LARIN achieves a charge prediction exact-match accuracy of 0.3316, a micro-F1 score of 0.3837, and a precision of 0.3951, while the sentencing exact-match accuracy reaches 0.1575. Meanwhile, its average retrieval frequency is 1.81 and average token consumption stands at 373.15. Experimental results on CAIL2018 demonstrate that legal-oriented adaptation based on DRAGIN improves retrieval-augmented reasoning in this Chinese legal judgment prediction setting while maintaining low retrieval overhead. Cross-dataset robustness on additional legal QA or judgment benchmarks remains to be further validated.