Focusing on the task of point-to-point navigation for an autonomous driving vehicle, we propose a novel deep learning model trained with end-to-end and multi-task learning manners to perform both perception and control tasks simultaneously. The model is used to drive the ego vehicle safely by following a sequence of routes defined by the global planner. The perception part of the model is used to encode high-dimensional observation data provided by an RGBD camera while performing semantic segmentation, semantic depth cloud (SDC) mapping, and traffic light state and stop sign prediction. Then, the control part decodes the encoded features along with additional information provided by GPS and speedometer to predict waypoints that come with a latent feature space. Furthermore, two agents are employed to process these outputs and make a control policy that determines the level of steering, throttle, and brake as the final action. The model is evaluated on CARLA simulator with various scenarios made of normal-adversarial situations and different weathers to mimic real-world conditions. In addition, we do a comparative study with some recent models to justify the performance in multiple aspects of driving. Moreover, we also conduct an ablation study on SDC mapping and multi-agent to understand their roles and behavior. As a result, our model achieves the highest driving score even with fewer parameters and computation load. To support future studies, we share our codes at https://github.com/oskarnatan/end-to-end-driving.
11 February 2022 in IEEE Transactions on Intelligent Transportation Systems
We present a novel compact deep multi-task learning model to handle various autonomous driving perception tasks in one forward pass. The model performs multiple views of semantic segmentation, depth estimation, light detection and ranging (LiDAR) segmentation, and bird's eye view projection simultaneously without being supported by other models. We also provide an adaptive loss weighting algorithm to tackle the imbalanced learning issue that occurred due to plenty of given tasks. Through data pre-processing and intermediate sensor fusion techniques, the model can process and combine multiple input modalities retrieved from RGB cameras, dynamic vision sensors (DVS), and LiDAR placed at several positions on the ego vehicle. Therefore, a better understanding of a dynamically changing environment can be achieved. Based on the ablation study, the model variant trained with our proposed method achieves a better performance. Furthermore, a comparative study is also conducted to clarify its performance and effectiveness against the combination of some recent models. As a result, our model maintains better performance even with much fewer parameters. Hence, the model can inference faster with less GPU memory utilization. Moreover, the result tends to be consistent in 3 different CARLA simulation datasets and 1 real-world nuScenes-lidarseg dataset. To support future research, we share codes and other files publicly at https://github.com/oskarnatan/compact-perception.
In this work, we introduce DeepIPC, a novel end-to-end model tailored for autonomous driving, which seamlessly integrates perception and control tasks. Unlike traditional models that handle these tasks separately, DeepIPC innovatively combines a perception module, which processes RGBD images for semantic segmentation and generates bird’s eye view (BEV) mappings, with a controller module that utilizes these insights along with GNSS and angular speed measurements to accurately predict navigational waypoints. This integration allows DeepIPC to efficiently translate complex environmental data into actionable driving commands. Our comprehensive evaluation demonstrates DeepIPC’s superior performance in terms of drivability and multi-task efficiency across diverse real-world scenarios, setting a new benchmark for end-to-end autonomous driving systems with a leaner model architecture. The experimental results underscore DeepIPC’s potential to significantly enhance autonomous vehicular navigation, promising a step forward in the development of autonomous driving technologies. For further insights and replication, we will make our code and datasets available at https://github.com/oskarnatan/DeepIPC.
Maintaining the water quality of a pond is one of the main issues on aquaculture management. Water quality represents the condition of a pond based on several water parameters such as dissolved oxygen (DO), temperature, pH, and salinity. All of these parameters need to be strictly supervised since it affects the life-sustainability of cultivated organisms. However, DO is said to be the main parameter since it affects the growth and survival rate of the shrimp. Therefore, a water quality control and monitoring system is needed to maintain water parameters at acceptable value. The system is developed on a mini-PC and microcontroller which are integrated with several sensors and actuator forming an embedded system. Then, this system is used to collect water quality data that is consisting of several water parameters and control the DO as the main parameter. In accordance with the stability needs against the sensitive environment, a fuzzy logic-based controller is developed to maintain the DO rate in the water. This system is also equipped with SIM800 module to notice the farmer by SMS, built-in wifi module for web-based data logging, and improved with Android-based graphical user interface (GUI) to perform user-friendly monitoring. From the experiment results, a fuzzy controller that is attached to the system can control the DO at the acceptable value of 6 ppm. The controller is said to have high robustness since its deviation for long-time use is only 0.12 ppm. Another test shows that the controller is able to overcome the given disturbance and easily adapt when the DO’s set point is changed.  Finally, the system is able to collect and store the data into cloud storage periodically and show the data on a website.