Cheongwoong Kang, Jaesik Choi. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.

This paper deals with the concept of multi-robot task allocation, referring to the assignment of multiple robots to tasks such that an objective function is maximized. The performance of existing meta-heuristic methods worsens as the number of robots or tasks increases. To tackle this problem, a novel Markov decision process formulation for multi-robot task allocation is presented for reinforcement learning. The proposed formulation sequentially allocates robots to tasks to minimize the total time taken to complete them. Additionally, we propose a deep reinforcement learning method to find the best allocation schedule for each problem. Our method adopts the cross-attention mechanism to compute the preference of robots to tasks. The experimental results show that the proposed method finds better solutions than meta-heuristic methods, especially when solving large-scale allocation problems.

Recently, the use of quadrotors has increased in numerous applications, such as agriculture, rescue, transportation, inspection, and localization. Time-optimal quadrotor waypoint tracking is defined as controlling quadrotors to follow the given waypoints as quickly as possible. Although PID control is widely used for quadrotor control, it is not adaptable to environmental changes, such as various trajectories and dynamic external disturbances. In this work, we discover that adjusting PID control frequencies is necessary for adapting to environmental changes by showing that the optimal control frequencies can be different for different environments. Therefore, we suggest a method to schedule the PID position and attitude control frequencies for time-optimal quadrotor waypoint tracking. The method includes (1) a Control Frequency Agent (CFA) that finds the best control frequencies in various environments, (2) a Quadrotor Future Predictor (QFP) that predicts the next state of a quadrotor, and (3) combining the CFA and QFP for time-optimal quadrotor waypoint tracking under unknown external disturbances. The experimental results prove the effectiveness of the proposed method by showing that it reduces the travel time of a quadrotor for waypoint tracking.

Reinforcement Learning (RL) has enabled autonomous agents to achieve superhuman performance in diverse domains, including games, navigation, and robotic control. Despite these successes, RL agents often struggle with overfitting to specific training environments, which can hinder their adaptability to new or rare scenarios. Domain randomization is a widely used technique designed to enhance policy robustness by training models in a variety of randomized contexts. In traditional domain randomization, however, models tend to prioritize learning from more common domains, often neglecting rare ones. To address this imbalance, we propose Balanced Domain Randomization (BDR) that balances the learning focus based on the rarity of contexts. We assess context rarity in the embedding space using statistical analysis of context vectors and adjust the loss weights for each transition based on this rarity. This ensures that the agent dedicates adequate attention to rare domains. Our experimental results show that BDR efficiently enhances worst-case performance, significantly improving the robustness of RL-based robotic controllers across diverse conditions. This study provides a robust framework for RL agents, reducing risks in uncommon scenarios and ensuring reliable performance in varied environments.

Large language models (LLMs) often make factually incorrect responses despite their success in various applications. In this paper, we hypothesize that relying heavily on simple co-occurrence statistics of the pre-training corpora is one of the main factors that cause factual errors. Our results reveal that LLMs are vulnerable to the co-occurrence bias, defined as preferring frequently co-occurred words over the correct answer. Consequently, LLMs struggle to recall facts whose subject and object rarely co-occur in the pre-training dataset although they are seen during finetuning. We show that co-occurrence bias remains despite scaling up model sizes or finetuning. Therefore, we suggest finetuning on a debiased dataset to mitigate the bias by filtering out biased samples whose subject-object co-occurrence count is high. Although debiased finetuning allows LLMs to memorize rare facts in the training set, it is not effective in recalling rare facts unseen during finetuning. Further research in mitigation will help build reliable language models by preventing potential errors. The code is available at \url{https://github.com/CheongWoong/impact_of_cooccurrence}.

An environment is a complex space in which numerous agents and objects interact. In this environment, rational agents act to maximize utility, and their actions are learned through reinforcement learning algorithms using deep neural networks. Since most environments are non-stationary, a single agent cannot solely achieve maximum utility through its observations and inferences using a network. However, when multiple agents can communicate with each other, a single agent can access richer information about the state of the environment simply by receiving messages from other agents. Nonetheless, the inferred information from other agents is generated by a neural network, which can be vulnerable or incorporate the personal perspective of the agents processing the information. For example, even when presented with a single observation, two agents can encode significantly different messages due to variations in their parameter modules. Therefore, it is crucial to consider the subjectivity of messages in multi-agent reinforcement learning (MARL). Building on the assumption that the encoded information is inherently subjective and reflects the agent that encodes it, we address the challenge of objective and subjective information in MARL. One limitation is that the information processed by a neural network is often challenging to interpret. In the initial phases of addressing this subjectivity concern, we focus on a relatively minor challenge: determining whether subjective information surpasses objective information in utility. Consequently, we compare the usefulness of messages exchanged between agents in two forms: raw observation and the output of a neural network.