Traditional frame-based cameras, foundational in computer vision, face challenges like motion blur, restricted dynamic range, and limited temporal resolution, which limit their use in various scenarios. Neuromorphic dynamic vision sensors (DVSs), inspired by biological vision systems, offer a significant shift with exceptional temporal resolution and pixel-level detection of light intensity changes, generating a continuous stream of events that encode time, location, and polarity. DVSs surpass traditional cameras with a dynamic range of 140 dB (compared to 60 dB), resilience to uneven lighting, minimal latency, ultra-high temporal resolution equivalent to over 100,000 frames per second, and low power consumption of just 1mW. These capabilities mark DVSs as a transformative advancement in sensing technology. However, DVSs demand novel processing techniques due to their unique data output. This survey revisits both overlooked and newly available research, categorizing established and emerging techniques. It explores a range of applications, from autonomous systems, vehicle navigation, object detection, and depth estimation to gesture recognition. In addition, the survey introduces newer applications like super-resolution, rolling shutter correction, robotic manipulation, data compression, and space situational awareness. The paper also tackles key challenges, such as DVS calibration, frame reconstruction, and simulation. To support future advancements, it provides a comprehensive overview of simulation platforms and the latest datasets, aiming to inspire further research and promote wider adoption of DVS technology in rapidly evolving fields.

We describe a fast and efficient camera pose refinement and Structure from Motion (SfM) method for sequential aerial imagery with applications to georegistration and 3-D reconstruction. Inputs to the system are 2-D images combined with initial noisy camera metadata measurements, available from on-board sensors (e.g., camera, global positioning system, and inertial measurement unit). Georegistration is required to stabilize the ground-plane motion to separate camera-induced motion from object motion to support vehicle tracking in aerial imagery. In the proposed approach, we recover accurate camera pose and (sparse) 3-D structure using bundle adjustment for sequential imagery (BA4S) and then stabilize the video from the moving platform by analytically solving for the image-plane-to-ground-plane homography transformation. Using this approach, we avoid relying upon image-to-image registration, which requires estimating feature correspondences (i.e., matching) followed by warping between images (in a 2-D space) that is an error prone process for complex scenes with parallax, appearance, and illumination changes. Both our SfM (BA4S) and our analytical ground-plane georegistration method avoid the use of iterative consensus combinatorial methods like RANdom SAmple Consensus which is a core part of many published approaches. BA4S is very efficient for long sequential imagery and is more than 130 times faster than VisualSfM, 35 times faster than MavMap, and about 274 times faster than Pix4D. Various experimental results demonstrate the efficiency and robustness of the proposed pipeline for the refinement of camera parameters in sequential aerial imagery and georegistration.

We introduce a novel discrete cosine transform-based feature (DCTF) descriptor designed for both robustly matching features in aerial video and tracking features across wide-baseline oblique views in aerial wide area motion imagery (WAMI). Our DCTF descriptor preserves local structure more compactly in the frequency domain by utilizing the mathematical properties of the discrete cosine transform (DCT) and outperforms widely used the spatial-domain feature extraction methods, such as speeded up robust features (SURF) and scale-invariant feature transform (SIFT). The DCTF descriptor can be used in combination with other feature detectors, such as SURF and features from accelerated segment test (FAST), for which we provide experimental results. The performance of DCTF for image matching and feature tracking is evaluated on two city-scale aerial WAMI data sets (ABQ-215 and LA-351) and a synthetic aerial drone video data set digital imaging and remote sensing image generation (Rochester Institute of Technology (RIT)-DIRSIG). DCTF is a compact 120-D descriptor that is less than half the dimensionality of state-of-the-art deep learning-based approaches, such as SuperPoint, LF-Net, and DeepCompare, which requires no learning and is domain-independent. Despite its small size, the DCTF descriptor surprisingly produces the highest image matching accuracies ( F ₁ = 0.76 and ABQ-215), the longest maximum and average feature track lengths, and the lowest tracking error (0.3 pixel, LA-351) compared with both handcrafted and deep learning features.

Premise: Aerial imagery from small unmanned aerial vehicle systems is a promising approach for high‐throughput phenotyping and precision agriculture. A key requirement for both applications is to create a field‐scale mosaic of the aerial imagery sequence so that the same features are in registration, a very challenging problem for crop imagery.Methods: We have developed an improved mosaicking pipeline, Video Mosaicking and summariZation (VMZ), which uses a novel two‐dimensional mosaicking algorithm that minimizes errors in estimating the transformations between successive frames during registration. The VMZ pipeline uses only the imagery, rather than relying on vehicle telemetry, ground control points, or global positioning system data, to estimate the frame‐to‐frame homographies. It exploits the spatiotemporal ordering of the image frames to reduce the computational complexity of finding corresponding features between frames using feature descriptors. We compared the performance of VMZ to a standard two‐dimensional mosaicking algorithm (AutoStitch) by mosaicking imagery of two maize (Zea mays) research nurseries freely flown with a variety of trajectories.Results: The VMZ pipeline produces superior mosaics faster. Using the speeded up robust features (SURF) descriptor, VMZ produces the highest‐quality mosaics.Discussion: Our results demonstrate the value of VMZ for the future automated extraction of plant phenotypes and dynamic scouting for crop management.

In this paper we propose a probabilistic model to parameterize human interactive behaviour from human motion. To Support the model taxonomy, we use Laban Movement Analysis (LMA), proposed by Rudolph Laban [11], to characterize human non-verbal communication. In interpersonal communication, body motion carries a lot of meaningful information, useful to analyse group dynamic behaviors in a wide range of social scenarios (e.g. behaviour analysis of human interpersonal activities and surveillance system). Taking the advantage of interpretation of social signals defined by Alex Pentland [19], and the descriptive body movement analysis proposed by Laban, we identified characteristics allowing both works to complement each other. To explore in group dynamics, we attempt to show the existent connections between Pentland's descriptions for Interpersonal Behaviours (IBs), and LMA parameters for human body part motions. Those relations are the keys to characterize the interpersonal communication. Given the uncertainty of the phenomenon, Bayesian's methodology is applied. The results present LMA parameters as reliable indicators for IBs, allowing us to generalize the model.

A novel approach for three-dimensional (3D) volumetric reconstruction of an object inside a scene is proposed. A camera network is used to observe the scene. Each camera within the network is rigidly coupled with an Inertial Sensor (IS). A virtual camera is defined for each IS–camera couple using the concept of infinite homography, by fusion of inertial and visual information. Using the inertial data and without planar ground assumption, a set of virtual horizontal planes are defined. The intersections of these inertial-based virtual planes with the object are registered using the concept of planar homography. Moreover a method to estimate the translation vectors among virtual cameras is proposed, which just needs the relative heights of two 3D points in the scene with respect to one of the cameras and their correspondences on the image planes. Different experimental results for the proposed 3D reconstruction method are provided on two different types of scenarios. In the first type, a single IS–camera couple is used and placed in different locations around the object. In the second type, the 3D reconstruction of a walking person (dynamic case) is performed where a set of installed cameras in a smart-room is used for the data acquisition. Moreover, a set of experiments are simulated to analyse the accuracy of the translation estimation method. The experimental results show the feasibility and effectiveness of the proposed framework for the purpose of multi-layer data registration and volumetric reconstruction.