Although vehicle location-based services are prevalent outdoors, we are back into darkness in many GPS blocked environments, such as tunnels, indoor parking garages, and multilevel flyovers. Existing smartphone-based solutions usually adopt inertial dead reckoning to infer the trajectory, but low-quality inertial sensors in phones are plagued by heavy noises, causing unbounded localization errors through double integrations for movements. In this article, we propose VeTorch, a smartphone inertial odometry that devises an inertial sequence learning framework to track vehicles in real time when GPS signal is not available. Specifically, we transform the inertial dynamics from the phone to the vehicle regardless of the arbitrary phone's placement in the car and explore a temporal convolutional network to learn the vehicle's moving dependencies directly from the inertial data. To tackle the heterogeneous smartphone properties and driving habits, we propose a federated learning-based active model training mechanism to produce customized models for individual smartphones, without incurring user privacy issues. We implement a highly efficient prototype and conduct extensive experiments on two large-scale real-world traffic data sets collected by a modern ride-hailing platform. Our results outperform the state-of-the-art vehicular inertial dead-reckoning solutions on both accuracy and efficiency.

User authentication on smartphones is the key to many applications, which must satisfy both security and convenience. We propose a novel user authentication system EchoPrint , which leverages acoustics and vision for secure and convenient user authentication, without requiring any special hardware. EchoPrint actively emits almost inaudible acoustic signals from the earpiece speaker to “illuminate” the user's face and authenticates the user by the unique features extracted from the echoes bouncing off the 3D facial contour. To combat changes in phone-holding poses thus echoes, a convolutional neural network (CNN) is trained to extract reliable acoustic features, which are further combined with visual facial features extracted from state-of-the-art face recognition deep models to feed a binary support vector machine (SVM) classifier for final authentication. Because the echo features depend on 3D facial geometries, EchoPrint is not easily spoofed by images or videos like 2D visual face recognition systems. It needs only commodity hardware, thus avoiding the extra costs of special sensors in solutions like FaceID. Experiments with 62 volunteers and non-human objects such as images, photos, and sculptures show that EchoPrint achieves 93.75 percent balanced accuracy and 93.50 percent F-score, while the average precision is 98.05 percent using acoustic features and basic facial landmarks. The precision is further improved to 99.96 percent with sophisticated visual features.

Estimating monocular depth and ego-motion via unsupervised learning has emerged as a promising approach in autonomous driving, mobile robots, and augmented reality (AR)/VR applications. It avoids intensive efforts to collect a large amount of ground truth and further improves the scene construction density and long-term tracking accuracy in simultaneous localization and mapping (SLAM) systems. However, existing approaches are susceptible to illumination variations and blurry pictures due to fast movements in real-world driving scenarios. In this article, we propose a novel unsupervised learning framework to fuse the complementary strength of visual and inertial measurements for monocular depth estimation. It learns both forward and backward inertial sequences at multiple subspaces to produce environment-independent and scale-consistent motion features and selectively weights inertial and visual modalities to adapt to various scenes and motion states. In addition, we explore a novel virtual stereo model to adopt such depth estimates in the monocular SLAM system, thus improving the system efficiency and accuracy. Extensive experiments on the KITTI, EuRoC, and TUM datasets have shown our effectiveness in terms of monocular depth estimation, SLAM initialization efficiency, and pose estimation accuracy compared with the state-of-the-art.

Estimating the origin-destination travel time is a fundamental problem in many location-based services for vehicles, e.g., ride-hailing, vehicle dispatching, and route planning. Recent work has made significant progress to accuracy, but they largely rely on GPS trajectories which are too coarse to model many personalized driving behaviors, e.g., differentiating novice and veteran drivers. In this paper, we propose Customized Travel Time Estimation (CTTE) that fuses GPS trajectories, smartphone inertial data, and road network within a deep recurrent neural network. It constructs a road link traffic database with topology representation, speed statistics, and query distribution. It also calibrates inertial readings, estimates the arbitrary phone’s pose in car, and detects multiple aggressive driving events (e.g., bump judders, sharp turns, sharp slopes, frequent lane shifts, overspeeds, and sudden brakes). Finally, we demonstrate our solution on two typical transportation problems, i.e., predicting traffic speed at holistic level and estimating customized travel time at personal level, within a multi-task learning structure. Experiments on two large-scale real-world traffic datasets from DiDi platform show our effectiveness compared with the state-of-the-art.

Visual-based unsupervised learning [1] – [3] has emerged as a promising approach in estimating monocular depth and ego-motion, avoiding intensive efforts on collecting and labeling the ground truth. However, they are still restrained by the brightness constancy assumption among video sequences, especially susceptible with frequent illumination variations or nearby textureless surroundings in indoor environments. In this article, we selectively combine the complementary strength of visual and inertial measurements, i.e., videos extract static and distinct features while inertial readings depict scale-consistent and environment-agnostic movements, and propose a novel unsupervised learning framework to predict both monocular depth and ego-motion trajectory simultaneously. This challenging task is solved by learning both forward and backward inertial sequences to eliminate inevitable noises, and reweighting visual and inertial features via gated neural networks in various environments or with user-specific moving dynamics. In addition, we also employ structure cues to produce scene depths from a single image and explore structure consistency constraints to calibrate the depth estimates in indoor buildings. Experiments on the outdoor KITTI data set and our dedicated indoor prototype reveal that our approach consistently outperforms the state of the art on both depth and ego-motion estimates. To the best of our knowledge, this is the first work to fuse visual and inertial data without any supervision signals for monocular depth and ego-motion estimation, and our solution remain effective and robust even in textureless indoor scenarios.

Most existing works in Industrial Internet of Things (IIoT) anomaly detection either depend on computationally intensive models that exceed the capabilities of multiaccess edge computing (MEC) servers, or lightweight models that lack robustness, making them unadaptable in IIoT infrastructures. To address these challenges, we propose THREADS , a hierarchical anomaly detection framework tailored for IIoT applications. The Instance thread utilizes an efficient variational auto encoder to produce instant feedback and offloads most of the workload to MECs. On the other hand, the Shadow thread employs an attention-enhanced transformer discriminator to examine low-confidence results in the cloud. Experimental results on five large-scale datasets show THREADS achieves an average F1-Score of 0.8537 in the hierarchical mode where most of the workloads are handled by MECs, and the random access memory and CPU usage is reduced by up to 29% and 88%, respectively. Meanwhile, THREADS achieves an F1-Score of 0.8563 in a cloud-based mode, consistently outperforming state-of-the-art approaches.