This article presents the first keyword spotting (KWS) IC that uses a ring-oscillator-based time-domain processing technique for its analog feature extractor (FEx). Its extensive usage of time-encoding schemes allows the analog audio signal to be processed in a fully time-domain manner except for the voltage-to-time conversion stage of the analog front end. Benefiting from fundamental building blocks based on digital logic gates, it offers better technology scalability compared to conventional voltage-domain designs. Fabricated in a 65-nm CMOS process, the prototyped KWS IC occupies 2.03 mm2 and dissipates 23- $\mu$ W power consumption, including analog FEx and digital neural network classifier. The 16-channel time-domain FEx achieves a 54.89-dB dynamic range for 16-ms frame shift size while consuming 9.3 $\mu$ W. The measurement result verifies that the proposed IC performs a 12-class KWS task on the Google Speech Command dataset (GSCD) with $>$ 86% accuracy and 12.4-ms latency.
Dynamic vision sensors (DVS) represent a promising new technology, offering low power consumption, sparse output, high temporal resolution, and wide dynamic range. These features make DVS attractive for new research areas including scientific and space-based applications; however, more precise understanding of how sensor input maps to output under real-world constraints is needed. Often, metrics used to characterize DVS report baseline performance by measuring observable limits but fail to characterize the physical processes at the root of those limits. To address this limitation, we describe step-by-step procedures to measure three important performance parameters: (1) temporal contrast threshold, (2) cutoff frequency, and (3) refractory period. Each procedure draws inspiration from previous work, but links measurements sequentially to infer physical phenomena at the root of measured behavior. Results are reported over a range of brightness levels and user-defined biases. The threshold measurement technique is validated with test-pixel node voltages, and a first-order low-pass approximation of photoreceptor response is shown to predict event cutoff temporal frequency to within 9% accuracy. The proposed method generates lab-measured parameters compatible with the event camera simulator v2e, allowing more accurate generation of synthetic datasets for innovative applications.
Forthcoming infrared event-based sensors will have utility in the space-based surveillance domain where they could potentially perform traditional sensing and tracking functions with significantly enhanced temporal resolution and reduced downstream datalink demands and power consumption. As a first step toward extending event-based sensing technology into the mid-wave infrared, a DC simulation of the conventional event-based sensor unit cell's photoreceptor circuit is performed, and the results are compared with measurements of a printed circuit board implementation of the same circuit to assess what design freedom is available to interface the photoreceptor with a mid-wave infrared photodetector. Detailed analysis of the circuit and measurements provides insight into which fundamental properties of the transistors drive the photoreceptor's dynamic range and demonstrates several characteristics that are relevant to mid-wave infrared sensing. These characteristics include the capability to produce a stable, low voltage bias on the infrared photodetector to maximize sensitivity, and that operating the circuit below room temperature increases the photoreceptor's dynamic range. Measurements show that even the simplest implementation of the photoreceptor circuit exhibits a dynamic range of logarithmic compression of 150 dB at room temperature. However, the dynamic range is ultimately limited by the mid-wave infrared photodetector's dark current activation energy, which is significantly lower than the threshold voltage (energy) of the photoreceptor's feedback transistor and, thus, there is an incentive for the photodetector's dark current to be diffusion-limited.
We present the first study into the long-term effects of radiation on an Event-based Vision Sensor (EVS) using real-world data from orbit. Falcon Neuro is an experimental, first-of-its-kind payload attached to the exterior of the International Space Station (ISS) operating two DAVIS 240C Event-based Vision Sensors. This study considers data gathered by Falcon Neuro between January 2022 and September 2024 over a wide range of scenes from Earth-facing and space-facing sensors. Falcon Neuro contains the first working EVS system in orbit. While EVS radiation degradation has been studied on the ground, this is the first study of degradation for EVS cameras of any kind in a real, uncontrolled environment. EVS pixel circuits are unique, analog, and far more complex than CMOS or CCD cameras. By utilizing distinct and unique features in the data created by the different pixel circuits in the camera, we show that degradation effects over the life of the mission caused by radiation or other sources have been minimal, with only one of the 18 measures displaying a convincing deterioration trend. Ultimately, we demonstrate that DAVIS 240C Event-based Vision Sensors have a high aptitude for surviving long-term space flight.
Dynamic Vision Sensor (DVS) event camera models are important tools for predicting camera response, optimizing biases, and generating realistic simulated datasets. Existing DVS models have been useful, but have not demonstrated high realism for challenging HDR scenes combined with adequate computational efficiency for array-level scene simulation. This paper reports progress towards a physically realistic and computationally efficient DVS model based on large-signal differential equations derived from circuit analysis, with parameters fitted from pixel measurements and circuit simulation. These are combined with an efficient stochastic event generation mechanism based on first-passage-time theory, allowing accurate noise generation with timesteps greater than 1000x longer than previous methods
Event-based vision sensors (EVS) represent a relatively new class of vision sensor that achieves key advantages over conventional frame-based cameras; however, characterizing and describing their performance with practical metrics for consumers continues to be a challenge because methods and metrics used to characterize conventional cameras are not compatible with their unique mode of operation. Key performance parameters for EVS include contrast thresholds, pixel bandwidth (related to response latency), refractory period, and noise. Accurately measuring any one of these is a nontrivial task, and many of these parameters change as a function of absolute illumination. Several researchers have laid the groundwork for EVS characterization methods; however, the community still lacks a standard testing protocol. Furthermore, the accuracy of reported metrics greatly depends on the method, testing environment, and interpretation. Until now, no study has attempted to explicitly validate EVS test methodologies or quantify their ability to produce accurate estimates of key parameters. In response, this study employs a realistic pixel-level simulation to compare various testing methods used to characterize EVS and quantify their accuracy. In addition, best practices are identified. Two independent measurement techniques are shown to reliably estimate contrast thresholds to within 2% relative error, although one is more compatible with low-light measurements. Among the pixel bandwidth measurement methods evaluated, we find that one conventional method of plotting pixel response as a function of stimulus frequency is unable to reliably predict low-pass filter (LPF) corner frequency, and we propose a new method that estimates frequency response to ≤10% error up to 5 kHz when the refractory period is 10 μs. We also provide insight into a previous method for measuring refractory period, and show that it can estimate refractory periods as low as 20 μs to within 10% error with pixel bandwidth ≥5 kHz. Finally, we include a discussion on noise characterization, and propose development of a more complete metric which accounts for threshold, bandwidth, and refractory period.