In order to eliminate the noise on the pulse interval of high-speed spike-based image sensor, a pulse interval compensation (PIC) algorithm is proposed. Based on the analysis of the principle of sensor noise generation, a pulse interval error model is constructed and the compensation algorithm is developed. First, raw pulse data is obtained from the sensor system, and then the coefficient matrix is used to compensate the non-uniform pulse interval caused by the dark current. Then, the guided filter separates high-frequency noise from scenes details and the pulse interval compensation matrix is obtained in the corresponding pulse interval. Finally, the pulse interval compensation is performed on the raw pulse data for correction. In static scene experiments, the proposed method is compared with the existing state-of-the-art methods and shows a clear advantage in denoising in both low texture and rich texture scenes. In the experiment to film a turntable at a rotation speed of 2000 r/min, 3000 r/min, 6000 r/min, the average roughness decreases from 0.2830 to 0.2034, 0.2937 to 0.2139 and 0.3252 to 0.2501 respectively. These experiments proved that the proposed method can effectively eliminate the noise of the spike-based image sensor.
In this paper, the non-ideal factors, which include spatial noise and temporal noise, are analyzed and suppressed in the high-speed spike-based image sensor, which combines the high-speed scanning sequential format with the method that uses the interspike time interval to indicate the scene information. In this imager, spatial noise contains device mismatch, which results in photo response non-uniformity (PRNU) and the non-uniformity of dark current. By multiplying the measured coefficient matrix the photo response non-uniformity is suppressed, and the non-uniformity of dark current is suppressed by correcting the interspike time interval based on the time interval of dark current. The temporal noise is composed of the shot noise and thermal noise. This kind of noise can be eliminated when using the spike frequency to restore the image. The experimental results show that, based on the spike frequency method, the standard deviation of the image decreases from 18.4792 to 0.5683 in the uniform bright light by using the calibration algorithm. While in the relatively uniform dark condition, the standard deviation decreases from 1.5812 to 0.4516. Based on interspike time interval method, because of time mismatch and temporal noise, the standard deviation of the image changes from 27.4252 to 27.4977 in the uniform bright light by using the calibration algorithm. While in the uniform dark condition, the standard deviation decreases from 2.361 to 0.3678.
27 October 2020 in IEEE Transactions on Circuits and Systems for Video Technology
Active image sensors can only record scenes at low frame rates. The dynamic vision sensor (DVS) can capture high-speed scenes under challenging lighting conditions. However, DVS discards absolute light intensity information. The dynamic and active pixel vision image sensor (DAVIS) can output events and low-frame-rate grayscale images simultaneously. This paper proposes a high frame rate video reconstruction and deblurring algorithm based on events and low-speed blurred image sequences from DAVIS. First, we visualized events as a sequence of event images and performed image denoising process. Then, we used the extended 3D partial recursive search (E-3DPRS) method to generate high-performance motion vector field (MVF) between adjacent denoising event images with low computational complexity. MVF reflects the object motion trajectory with high time resolution. Furthermore, MVF is used as a priori information to restore potentially sharp images, which alleviates the highly ill-posed deblurring process. At last, we reconstructed a high frame rate sharp video based on a sharp image sequence and MVFs. Extensive experiments show that our proposed algorithm outperforms the existing state-of-the-art methods and achieves better average feature similarity and structural similarity.
A 128-stage CMOS time delay integration (TDI) image sensor with on-chip digital accumulator was presented in this paper. By using two sets of sampling capacitors and an optimized timing, the on-chip column parallel cyclic analog-todigital converter (ADC) in the sensor conducts the correlated double sampling, digital program gain amplifying, and A/D conversion all altogether in one ADC's conversion time simultaneously. An on-chip digital accumulator suitable for this sensor is also proposed. A prototype 1024 × 128 CMOS TDI image sensor is fabricated in the 0.18-μm CMOS technology, and the measurement results proof that the pixels and readout circuits in the sensor work properly. With a line rate of 3875 lines/s, the sensor costs a power consumption of 290 mW and a silicon area of 18.2 mm × 18.9 mm. The estimated maximum sensitivity of the fabricated sensor is 2010 V/lux· sec. This paper is suitable for applications in low-illumination, high scanning speed, and remote sensing systems.
A method of biomimetic visual perception and image reconstruction based on pulse sequence of events is proposed, and the proposed method facilitates a behavioral model of vision image sensing. Based on the model, the relationship between light condition and quality of reconstructed videos is analyzed. In this sensing method, pulse sequence is defined as synchronous pulse sequence. The sensing method adopts a synchronous pulse sequence readout and asynchronous pixel level sensing mechanism. Light intensity can be calculated by pulse sequences and image or video can be further reconstructed. As the photocurrent increases, the real-time performance of reconstructed image increases, but the depth of the reconstructed image decreases. Experiments are performed on sensing models with two types of pulse sequence generating methods. Method 1 adopts 1-bit data for each pixel, and Method 2 adopts 1-bit data accompanied by 3-bit time label which enables a lower frame rate. In the experiment with static images, the reconstructed images show better image depth under lower photocurrent. The MSE (mean square error) of the reconstructed image with 450fA maximum photocurrent on Method 1 is 0.0737, whereas the MSE in the same condition on Method 2 is 0.0083. In the experiment with high-speed videos of 20000fps, the reconstructed videos with various photocurrents are evaluated by MSE analysis. When maximum photocurrent is between 6750fA and 8250fA as well as between 11250fA and 16500fA, the reconstructed high-speed videos have good MSE results, where video real-time and image depth are balanced.
24 September 2020 in IEEE Journal of the Electron Devices Society
The output features of pixels in CMOS image sensors (CISs) are influenced by different exposure conditions. This paper presents an analytical model to describe the output characteristics of the exposure process in pinned photodiode (PPD) CMOS image sensors with the accumulation of three charge sources: photogenerated charge, p-n junction-generated charge, and emission charge. In the proposed model, the difference between the time-based and light intensity-based photo response process is illustrated. The model also reveals the relationship between photodiode potential and light intensity at different exposure values. At the low exposure values, the PD potential increases with light intensity, and a contrary trend is observed at the high exposure values. This concludes that a larger linear output range can be obtained in high light intensity conditions. Furthermore, the improved model provides development analysis in terms of light intensity influence for long exposure time noise. The models were verified with technology computer-aided design simulation and the test devices were fabricated using a 0.18-μm CIS process. The model demonstrates good consistency with simulation and measured results.