The emergence of energy storage system (ESS) enables the service provider to profit from the price difference between purchasing electric energy from utility companies and selling it to customers through battery operations while more frequent charging/discharging behaviors cause battery degradation. However, the accurate estimation of the ESS degradation cost is one of the main obstacles for ESS participating in energy management. This paper considers a smart grid scenario, consisting of an accurate lithium-ion battery degradation (ALBD) model, renewable energy generators and bilateral energy flow from/to the utility grid. We develop an online energy-scheduling approach aiming to maximize the operating profit of the system from energy trading and guarantee the long-term battery usage. We formulate the trade-off between energy trading profit and battery degradation cost as a joint optimization problem, and further transform it into a Markov Decision Process (MDP) problem. To solve the uncertainty of electricity price and demand, we apply a gated recurrent unit to forecast the next day’s price and demand. Then, we develop a deep Q-network to learn an optimized energy-scheduling strategy. The numerical simulations show that our proposed approach can improve the operating profit from 6.04% to 20.54% compared with the three baselines.

Vision-and-Language Navigation (VLN) is a challenging task that requires a robot to navigate in photo-realistic environments with human natural language promptings. Recent studies aim to handle this task by constructing the semantic spatial map representation of the environment, and then leveraging the strong ability of reasoning in large language models for generalizing code for guiding the robot navigation. However, these methods face limitations in instance-level and attribute-level navigation tasks as they cannot distinguish different instances of the same object. To address this challenge, we propose a new method, namely, Instance-aware Visual Language Map (IVLMap), to empower the robot with instance-level and attribute-level semantic mapping, where it is autonomously constructed by fusing the RGBD video data collected from the robot agent with special-designed natural language map indexing in the bird’s-in-eye view. Such indexing is instance-level and attribute-level. In particular, when integrated with a large language model, IVLMap demonstrates the capability to i) transform natural language into navigation targets with instance and attribute information, enabling precise localization, and ii) accomplish zero-shot end-to-end navigation tasks based on natural language commands. Extensive navigation experiments are conducted. Simulation results illustrate that our method can achieve an average improvement of 14.4% in navigation accuracy. Code and demo are released at https://ivlmap.github.io/.

The development of artificial intelligence (AI) game agents that use deep reinforcement learning (DRL) algorithms to process visual information for decision-making has emerged as a key research focus in both academia and industry. However, previous game agents have struggled to execute multiple commands simultaneously in a single decision, failing to accurately replicate the complex control patterns that characterize human gameplay. In this paper, we utilize the ViZDoom environment as the DRL research platform and transform the agent–environment interactions into a Partially Observable Markov Decision Process (POMDP). We introduce an advanced multi-agent deep reinforcement learning (DRL) framework, specifically a Multi-Agent Proximal Policy Optimization (MA-PPO), designed to optimize target acquisition while operating within defined ammunition and time constraints. In MA-PPO, each agent handles distinct parallel tasks with custom reward functions for performance evaluation. The agents make independent decisions while simultaneously executing multiple commands to mimic human-like gameplay behavior. Our evaluation compares MA-PPO against other DRL algorithms, showing a 30.67% performance improvement over the baseline algorithm.

Given the uncertainty from renewable production, local loads and battery operating states in microgrid, it is vital to develop an efficient energy management scheme to improve system economics and enhance grid reliability. In this paper, we consider a renewable integrated microgrid scenario including an energy storage system (ESS), bidirectional energy flow from/to conventional power grid and ESS health estimation. We develop a novel demand response-based scheme for microgrid energy management with a long short-term memory (LSTM) network and reinforcement learning (RL), aiming to improve the system operating profit from energy-trading and reduce the battery health cost from energy-scheduling. Specifically, to overcome the uncertainty from future, we utilize LSTM to forecast the unknown demand and electricity price. To obtain the desired ESS control scheme, we apply RL to learn an optimal energy-scheduling strategy. To improve the critical performance of the RL paradigm, we propose a random greedy strategy to encourage exploration. Numerical results show that our proposed algorithm outperforms the baselines by improve the system operating profit by 8.27% and 17.31% while ensuring ESS operating safety. By integrating energy efficiency with sustainable energy management practices, our scheme contributes to long-term environmental and economic resilience.

Indoor scene recognition is a crucial component in vision-and-language navigation (VLN), which involves guiding an agent to navigate through unseen, photo-realistic environments using natural language instructions. However, existing researches have not yet endowed VLN agents with robust indoor scene recognition capabilities, which are essential for enhancing the agent’s spatial perception and environmental understanding. In this paper, we introduce the Room-to-Room Scene Recognition (R2R-SR) dataset, designed to enhance the spatial perception capabilities of agents in VLN. The R2R-SR dataset includes 7,283 images categorized into 10 room classes, sourced from the Matterport3D (MP3D) simulator. To alleviate the imbalance in training data distribution across room categories, we propose a panoramic environment generation technique using Llava and Stable Diffusion for data Augmentation (LSDA). Our proposed LSDA-based model achieves a Top-1 accuracy of 75.9%, outperforming the best competitors in the convolutional neural network (CNN) and vision transformer (ViT) domains. The qualitative results highlight the model’s proficiency in identifying distinct features across various room categories, showcasing its effectiveness in indoor scene recognition tasks.