The rapid development of neural networks has led to tremendous applications in image segmentation, speech recognition, and medical image diagnosis, etc. Among various hardware implementations of neural networks, silicon photonics is considered one of the most promising approaches due to its CMOS compatibility, accessible integration platforms, mature fabrication techniques, and abundant optical components. In addition, neuromorphic computing based on silicon photonics can provide massively parallel processing and high-speed operations with low power consumption, thus enabling further exploration of neural networks. Here, we focused on the development of neuromorphic computing based on silicon photonics, introducing this field from the perspective of electronic–photonic co-design and presenting the architecture and algorithm theory. Finally, we discussed the prospects and challenges of neuromorphic silicon photonics.

Event cameras draw inspiration from biological systems, boasting low latency and high dynamic range while consuming minimal power. The most current approach to processing Event Cloud often involves converting it into frame-based representations, which neglects the sparsity of events, loses fine-grained temporal information, and increases the computational burden. In contrast, Point Cloud is a popular representation for processing 3-dimensional data and serves as an alternative method to exploit local and global spatial features. Nevertheless, previous point-based methods show an unsatisfactory performance compared to the frame-based method in dealing with spatio-temporal event streams. In order to bridge the gap, we propose EventMamba, an efficient and effective framework based on Point Cloud representation by rethinking the distinction between Event Cloud and Point Cloud, emphasizing vital temporal information. The Event Cloud is subsequently fed into a hierarchical structure with staged modules to process both implicit and explicit temporal features. Specifically, we redesign the global extractor to enhance explicit temporal extraction among a long sequence of events with temporal aggregation and State Space Model (SSM) based Mamba. Our model consumes minimal computational resources in the experiments and still exhibits SOTA point-based performance on six different scales of action recognition datasets. It even outperformed all frame-based methods on both Camera Pose Relocalization (CPR) and eye-tracking regression tasks.

Optical neural networks (ONN) based on micro-ring resonators (MRR) have emerged as a promising alternative to significantly accelerating the massive matrix-vector multiplication (MVM) operations in artificial intelligence (AI) applications. However, the limited scale of MRR arrays presents a challenge for AI acceleration. The disparity between the small MRR arrays and the large weight matrices in AI necessitates extensive MRR writings, including reprogramming and calibration, resulting in considerable latency and energy overheads. To address this problem, we propose a novel design methodology to lessen the need for frequent weight reloading. Specifically, we propose a reuse and blend (R&B) architecture to support efficient layer-wise and block-wise weight sharing, which allows weights to be reused several times between layers/ blocks. Experiment results demonstrate the R&B system can maintain comparable accuracy with 69% energy savings and 57% latency improvement. These results highlight the promise of the R&B to enable efficient deployment of advanced deep learning models on photonic accelerators.