This paper presents an approach for solving the crowd navigation problem in an unknown and dynamic environment based on deep reinforcement learning. In our approach, we first make four leader agents learn how to reach their goals and avoid collisions with static and dynamic obstacles in an unknown environment by use of proximal policy optimization combined with Long short-term memory and a collision prediction algorithm. In the second stage, we make each leader agent arrive at a specific goal several times and record its trajectory as the guiding path so that the members in its group know how to reach their goals. We adopt the Reciprocal Velocity Obstacle algorithm to make agents not collide with others. Finally, we simulate the scenario of four groups moving towards their goals simultaneously using the Unity 3D engine. The experimental results demonstrate self-learning ability of a crowd who can reach their goals successfully in an unknown and dynamic environment.

There have been some biomechanics based control systems that have achieved better realistic virtual human motion. Yet their abilities to adapt the changing environments are weaker than the traditional control systems with characters driven by proportional derivative actuators directly. In our method, we build a hierarchical neuromuscular virtual human motion control system that consists of a low-level spine reflex layer and a high-level policy control layer. The spine reflex layer uses a feedback net to map sensory information to excitations which stimulate muscles to generate joint torques. The policy control layer includes a deep neural network which provides a learned action policy to spine reflex layer for achieving terrain-adaptive motion skills. The particle swarm optimization algorithm is used to optimize the gain factors of the feedback net for finding out a basic policy to make the virtual human walk on the flat terrain autonomously. The proximal policy optimization algorithm is employed to train the deep neural network in policy control layer for learning how to modulate the actions to adapt the changing terrain. The simulation results in Matlab show that virtual human can walk smoothly and better adapt to the given terrain changes. It demonstrates that our control system improves the terrain-adaptive walking skill of the neuromuscular virtual human.

Generally, most approaches using methods such as cropping, rotating, and flipping achieve more data to train models for improving the accuracy of detection and segmentation. However, due to the difficulties of labeling such data especially semantic segmentation data, those traditional data augmentation methodologies cannot help a lot when the training set is really limited. In this paper, a model named OFA-Net (One For All Network) is proposed to combine object detection and semantic segmentation tasks. Meanwhile, using a strategy called “1-N Alternation” to train the OFA-Net model, which can make a fusion of features from detection and segmentation data. The results show that object detection data can be recruited to better the segmentation accuracy performance, and furthermore, segmentation data assist a lot to enhance the confidence of predictions for object detection. Finally, the OFA-Net model is trained without traditional data augmentation methodologies and tested on the KITTI test server. The model works well on the KITTI Road Segmentation challenge and can do a good job on the object detection task.