In this paper, we study the problem of jointly estimating the optical flow and scene flow from synchronized 2D and 3D data. Previous methods either employ a complex pipeline that splits the joint task into independent stages, or fuse 2D and 3D information in an “early-fusion” or “late-fusion” manner. Such one-size-fits-all approaches suffer from a dilemma of failing to fully utilize the characteristic of each modality or to maximize the inter-modality complementarity. To address the problem, we propose a novel end-to-end framework, which consists of 2D and 3D branches with multiple bidirectional fusion connections between them in specific layers. Different from previous work, we apply a point-based 3D branch to extract the LiDAR features, as it preserves the geometric structure of point clouds. To fuse dense image features and sparse point features, we propose a learnable operator named bidirectional camera-LiDAR fusion module (Bi-CLFM). We instantiate two types of the bidirectional fusion pipeline, one based on the pyramidal coarse-to-fine architecture (dubbed CamLiPWC), and the other one based on the recurrent all-pairs field transforms (dubbed CamLiRAFT). On FlyingThings3D, both CamLiPWC and CamLiRAFT surpass all existing methods and achieve up to a 47.9% reduction in 3D end-point-error from the best published result. Our best-performing model, CamLiRAFT, achieves an error of 4.26% on the KITTI Scene Flow benchmark, ranking 1st among all submissions with much fewer parameters. Besides, our methods have strong generalization performance and the ability to handle non-rigid motion.

RFID tags have been widely deployed to report valuable information about tagged objects or surrounding environment. To collect such information, the key is to avoid the tag-to-tag collision in the open wireless channel. Polling, as a widely used anti-collision protocol, provides a request-response way to interrogate tags. The basic polling however needs to broadcast the tedious tag ID (96 bits) to query a tag, which is time-consuming. For example, collecting only 1-bit information (e.g., battery status) but with 96-bit overhead is a great limitation. This paper studies how to design efficient polling protocols to collect tag information quickly. The basic idea is to minimize the length of the polling vector as well as to avoid useless communication. We first propose an efficient Hash polling protocol (HPP) that uses hash indices rather than tag IDs as the polling vector to query each tag. The length of the polling vector is dropped from 96 bits to no more than 16 bits (the number of tags is less than 100,000). We then propose a tree-based polling protocol (TPP) that avoids redundant transmission in HPP. By constructing a binary polling tree, TPP transmits only different postfix of the neighbor polling vectors; the same prefix is reserved without any retransmission. The result is that the length of the polling vector reduces to only 3.4 bits. Finally, we propose an incremental polling protocol (IPP) that updates the polling vector based on the difference in value between the current polling vector and the previous one. By sorting the indices and dynamically updating them, IPP drops the polling vector to 1.6 bits long, 60 times less than 96-bit IDs. Extensive simulation results show that our best protocol IPP outperforms the state-of-the-art information collection protocol.

Currently, the logistic industry has introduced 3D reconstruction to monitor the package placement in the warehouse. Previous 3D reconstruction solutions mainly utilize computer vision or sensor-based methods, which are restricted to the line-of-sight or the battery life. Therefore, we propose a passive RFID-based solution, called RF-3DScan, to perform 3D reconstruction on tagged packages, including the package orientation and the package stacking. The basic idea is that a moving antenna can obtain RF-signals from the tags attached on packages with the 1D linear mobile scanning. Through extracting phase differences to build angle profiles for each tag, RF-3DScan derives their relative positions, further determine the package orientation and the coarse-grained package stacking. By simply performing the 2D scanning, RF-3DScan can provide fine-grained package stacking determination. We implement a prototype system of RF-3DScan and evaluate its performance in real settings. Our experiment results show that RF-3DScan can achieve about 92.5% identification accuracy of the bottom face, and average error about 4.08° of the rotation angle. For the package stacking, 1D scanning can achieve the comparable performance in comparison with 2D scanning.

Heavy-hitter detection is a fundamental task in network traffic measurement and security. Existing work faces the dilemma of suffering dynamic and imbalanced traffic characteristics or lowering the detection efficiency and flexibility. In this paper, we propose a flexible sketch called SwitchSketch that embraces dynamic and skewed traffic for efficient and accurate heavy-hitter detection. The key idea of SwitchSketch is allowing the sketch to dynamically switch among different modes and take full use of each bit of the memory. We present an encoding-based switching scheme together with a flexible bucket structure to jointly achieve this goal by using a combination of design features, including variable-length cells, shrunk counters, embedded metadata, and switchable modes. We further implement SwitchSketch on the NetFPGA-1G-CML board. Experimental results based on real Internet traces show that SwitchSketch achieves a high Fβ-Score of threshold-t detection (consistently higher than 0.938) and over 99% precision rate of top-k detection under a tight memory size (e.g., 100KB). Besides, it outperforms the state-of-the-art by reducing the ARE by 30.77%\sim99.96%. All related implementations are open-sourced.

Performing accurate sensing in diverse environments is a challenging issue in wireless sensing technologies. Existing solutions usually require collecting a large number of samples to train a classifier for every environment, or further assume similar sample distribution between different environments such that a model trained in one environment can be transferred to another. In this paper, we propose RF-Siamese, an RFID-based gesture sensing approach that achieves comparable accuracy to existing solutions but requires only a few samples in each eivironment. RF-Siamese leverages Siamese networks to distinguish different gestures with only a small number of samples and is enhanced by several novel designs to achieve high accuracy in diverse environments. First, the network structure and parameters (e.g., loss function and distance metric) are carefully designed to be suitable for RFID gesture recognition. Second, a permutation-based dataset generation strategy is proposed to make full use of the collected samples to enhance the recognition accuracy. Third, a template matching method is proposed to extend the Siamese network to classify multiple gestures. Extensive experiments on commercial RFID devices demonstrate that RF-Siamese achieves a high accuracy of 0.93 with only one sample of each gesture when recognizing 18 different gestures, while state-of-the-art approaches based on transfer learning and meta learning achieve an accuracy of only 0.59 and 0.70, respectively.