15 October 2022 in Applied Intelligence
13 August 2024 in IEEE Sensors Journal
In this paper, we propose a method for recognizing kick motions using a multiple-input multiple-output (MIMO) frequency-modulated continuous wave (FMCW) radar system combined with deep learning techniques. Smart trunk opener (STO) systems that provides users with hands-free trunk operation have been gaining attention. To address the limitations of prevalent STO systems, which rely on capacitive or ultrasonic sensors, we propose employing a 60 GHz MIMO FMCW radar system. Our design of a 60 GHz MIMO FMCW radar system can detect kick motions and estimate their range, velocity, and angle, using a signal processing chain that includes a two-dimensional fast Fourier transform and multiple antenna elements. Additionally, using the acquired information on the range, velocity, and angle of the kick motion, we propose a deep learning-based model for recognizing specific kick motions to operate the STO system. This model is designed to take sequences of range, velocity, and angle as its input, unlike the conventional motion recognition methods that treats radar data as an image. We analyze the performances of various deep learning-based models using a dataset of various kick motions obtained through real-life measurements. Results showed that the one-dimensional convolutional neural network achieved more than 97% in accuracy, F1 score, recall and precision. Furthermore, when compared with the models soly based on velocity information, overall performances decreased to around 66%, proving the effectiveness of the MIMO FMCW radar-based kick motion recognition.
9 April 2025 in IEEE Sensors Journal
This article proposes a method for integrating the camera and radar data using nonlinear homography transformations. The lens distortion effects of a camera cause differences in object positions to have a nonlinear relationship with pixel differences in the captured image. Therefore, accurate sensor integration is impossible when using linear homography transformations to represent camera track information on the radar coordinate plane or vice versa. Thus, this article proposes a method to integrate the camera-estimated track with the radar track by applying lateral and longitudinal corrections. For lateral corrections, the reference point of the camera track is adjusted from the bottom center of the bounding box to a position that reflects the angle between the vanishing point and the target. For longitudinal corrections, a nonlinear homography transformation is used to address the accuracy degradation caused by lens distortion. Using experimental data, we demonstrate that track matching accuracy significantly improved from an initial root mean square error of 0.92–0.45 m after applying two separate correction steps in each direction, highlighting the effectiveness of the correction process in enhancing sensor accuracy.