The frame rate of the digital high-speed video camera was 2000 frames per second (fps) in 1989, and has been exponentially increasing. A simulation study showed that a silicon image sensor made with a 130 nm process technology can achieve about 10¹⁰ fps. The frame rate seems to approach the upper bound. Rayleigh proposed an expression on the theoretical spatial resolution limit when the resolution of lenses approached the limit. In this paper, the temporal resolution limit of silicon image sensors was theoretically analyzed. It is revealed that the limit is mainly governed by mixing of charges with different travel times caused by the distribution of penetration depth of light. The derived expression of the limit is extremely simple, yet accurate. For example, the limit for green light of 550 nm incident to silicon image sensors at 300 K is 11.1 picoseconds. Therefore, the theoretical highest frame rate is 90.1 Gfps (about 10¹¹ fps)

Light in flight was captured by a single shot of a newly developed backside-illuminated multi-collection-gate image sensor at a frame interval of 10 ns without high-speed gating devices such as a streak camera or post data processes. This paper reports the achievement and further evolution of the image sensor toward the theoretical temporal resolution limit of 11.1 ps derived by the authors. The theoretical analysis revealed the conditions to minimize the temporal resolution. Simulations show that the image sensor designed following the specified conditions and fabricated by existing technology will achieve a frame interval of 50 ps. The sensor, 200 times faster than our latest sensor will innovate advanced analytical apparatuses using time-of-flight or lifetime measurements, such as imaging TOF-MS, FLIM, pulse neutron tomography, PET, LIDAR, and more, beyond these known applications.

The paper summarizes the evolution of the Backside-Illuminated Multi-Collection-Gate (BSI MCG) image sensors from the proposed fundamental structure to the development of a practical ultimate-high-speed silicon image sensor. A test chip of the BSI MCG image sensor achieves the temporal resolution of 10 ns. The authors have derived the expression of the temporal resolution limit of photoelectron conversion layers. For silicon image sensors, the limit is 11.1 ps. By considering the theoretical derivation, a high-speed image sensor designed can achieve the frame rate close to the theoretical limit. However, some of the conditions conflict with performance indices other than the frame rate, such as sensitivity and crosstalk. After adjusting these trade-offs, a simple pixel model of the image sensor is designed and evaluated by simulations. The results reveal that the sensor can achieve a temporal resolution of 50 ps with the existing technology.

The paper presents an ultra-high-speed image sensor for motion pictures of reproducible events emitting very weak light. The sensor is backside-illuminated. Each pixel is equipped with the multiple collection gates (MCG) at the center of the front side. Each collection gate is connected to an in-pixel large memory unit, which can accumulate image signals captured by repetitive imaging. The combination of the backside illumination, image signal accumulation, and slow readout from the in-pixel signal storage after an image capturing operation offers a very high sensitivity. Pipeline signal transfer from the MCG to the in-pixel memory units enables the sensor to achieve a large frame count and a very high frame rate at the same time. A test sensor was fabricated with a pixel count of 32 × 32 pixels. Each pixel is equipped with four collection gates, each connected to a memory unit with 305 elements; thus, with a total frame count of 1220 (305 × 4) frames. The test camera achieved 25 Mfps, while the sensor was designed to operate at 50 Mfps.

The theoretical temporal resolution limit $t_T$ of a silicon photodiode (Si PD) is 11.1 ps. We call “super temporal resolution” the temporal resolution that is shorter than that limit. To achieve this resolution, Germanium is selected as a candidate material for the photodiode (Ge PD) for visible light since the absorption coefficient of Ge for the wavelength is several tens of times higher than that of Si, allowing a very thin PD. On the other hand, the saturation drift velocity of electrons in Ge is about 2/3 of that in Si. The ratio suggests an ultra-short propagation time of electrons in the Ge PD. However, the diffusion coefficient of electrons in Ge is four times higher than that of Si. Therefore, Monte Carlo simulations were applied to analyze the temporal resolution of the Ge PD. The estimated theoretical temporal resolution limit is 0.26 ps, while the practical limit is 1.41 ps. To achieve a super temporal resolution better than 11.1 ps, the driver circuit must operate at least 100 GHz. It is thus proposed to develop, at first, a short-wavelength infrared (SWIR) ultra-high-speed image sensor with a thicker and wider Ge PD, and then gradually decrease the size along with the progress of the driver circuits.