In this paper, a prototype complementary metal-oxide-semiconductor (CMOS) image sensor with a 2.8-μm backside-illuminated (BSI) pixel with a lateral overflow integration capacitor (LOFIC) architecture is presented. The pixel was capable of a high conversion gain readout with 160 μV/e− for low light signals while a large full-well capacity of 120 ke− was obtained for high light signals. The combination of LOFIC and the BSI technology allowed for high optical performance without degradation caused by extra devices for the LOFIC structure. The sensor realized a 70% peak quantum efficiency with a normal (no anti-reflection coating) cover glass and a 91% angular response at ±20° incident light. This 2.8-μm pixel is potentially capable of higher than 100 dB dynamic range imaging in a pure single exposure operation.
A lateral overflow integration capacitor (LOFIC) complementary metal oxide semiconductor (CMOS) image sensor can realize high-dynamic-range (HDR) imaging with combination of a low-conversion-gain (LCG) signal for large maximum signal electrons and a high-conversion-gain (HCG) signal for electron-referred noise floor. However, LOFIC-CMOS image sensor requires a two-channel read-out chain for LCG and HCG signals whose polarities are inverted. In order to provide an area-efficient LOFIC-CMOS image sensor, a one-channel read-out chain that can process both HCG and LCG signals is presented in this paper. An up/down double-sampling circuit composed of an inverting amplifier for HCG signals and a non-inverting attenuator for LCG signals can reduce the area of the read-out chain by half compared to the conventional two-channel read-out chain. A test chip is fabricated in a 0.18 m CMOS process with a metal–insulator–metal (MIM) capacitor, achieving a readout noise of 130 for the HCG signal and 1.19 V for the LCG input window. The performance is equivalent to 103 dB of the dynamic range with our previous LOFIC pixel in which HCG and LCG conversion gains are, respectively, 160 and 10 .
In order to improve image recognition technologies in an IoT environment, we propose a data reduction scheme for a feature-extractable CMOS image sensor and present simulation results for object recognition using feature data. We evaluated the accuracy of the simulated feature data in object recognition based on YOLOX trained with a feature dataset. According to our simulation results, the obtained object recognition accuracy was with the large-scale COCO dataset, even though the amount of data was reduced by compared to conventional RGB color images. When the dataset was replaced with the RAISE RAW image dataset for more accurate simulation, the object recognition accuracy improved to . Furthermore, the feature-extractable CMOS image sensor can switch its operation mode between RGB color image mode and feature data mode. When the trigger for switching from feature data mode to RGB color image mode was set to the detection of a large-sized person, the feature data achieved an accuracy of with the COCO dataset.
For the coming trillion sensors universe, image recognition based on deep learning is becoming increasingly important. However, since feature extraction is taken place prior to classification in a neural network, data captured by conventional image sensors are redundant. Our research group has proposed an image sensor that can extract lightweight features, and confirmed that the data size of the feature data is reduced by 97.7% compared to the conventional RGB color image with recognition accuracy decreased by 13.0%. In this paper, a pixel device for the feature extractable image sensor is presented, which can output both RGB color image and the feature that is grayscale horizontal edge.
In order to realize image information security starting from the data source, challenge–response (CR) device authentication, based on a Physically Unclonable Function (PUF) with a 2 Mpixel CMOS image sensor (CIS), is studied, in which variation of the transistor in the pixel array is utilized. As each CR pair can be used only once to make the CIS PUF resistant to the modeling attack, CR authentication with CIS can be carried out 4050 times, with basic post-processing to generate the PUF ID. If a larger number of authentications is required, advanced post-processing using Lehmer encoding can be utilized to carry out authentication times. According to the PUF performance evaluation, the authentication error rate is less than ppm. Furthermore, the area overhead of the CIS chip for the basic and advanced post-processing is only and , respectively, based on a Verilog HDL model circuit design.
A backdoor attack is a type of attack method that induces deep neural network (DNN) misclassification. The adversary who aims to trigger the backdoor attack inputs the image with a specific pattern (the adversarial mark) into the DNN model (backdoor model). In general, the adversary mark is created on the physical object input to an image by capturing a photo. With this conventional method, the success of the backdoor attack is not stable because the size and position change depending on the shooting environment. So far, we have proposed a method of creating an adversarial mark for triggering backdoor attacks by means of a fault injection attack on the mobile industry processor interface (MIPI), which is the image sensor interface. We propose the image tampering model, with which the adversarial mark can be generated in the actual fault injection to create the adversarial mark pattern. Then, the backdoor model was trained with poison data images, which the proposed simulation model created. We conducted a backdoor attack experiment using a backdoor model trained on a dataset containing 5% poison data. The clean data accuracy in normal operation was 91%; nevertheless, the attack success rate with fault injection was 83%.