An end-to-end deep neural network, ResU-net, is developed for quantitative photoacoustic imaging. A residual learning framework is used to facilitate optimization and to gain better accuracy from considerably increased network depth. The contracting and expanding paths enable ResU-net to extract comprehensive context information from multispectral initial pressure images and, subsequently, to infer a quantitative image of chromophore concentration or oxygen saturation (${\mathrm{sO}}_2$). According to our numerical experiments, the estimations of ${\mathrm{sO}}_2$ and indocyanine green concentration are accurate and robust against variations in both optical property and object geometry. An extremely short reconstruction time of 22 ms is achieved.

Recent advances in artificial intelligence have enhanced the abilities of mobile robots in dealing with complex and dynamic scenarios. However, to enable computationally intensive algorithms to be executed locally in multitask robots with low latency and high efficiency, innovations in computing hardware are required. Here, we report TianjicX, a neuromorphic computing hardware that can support true concurrent execution of multiple cross-computing-paradigm neural network (NN) models with various coordination manners for robotics. With spatiotemporal elasticity, TianjicX can support adaptive allocation of computing resources and scheduling of execution time for each task. Key to this approach is a high-level model, “Rivulet,” which bridges the gap between robotic-level requirements and hardware implementations. It abstracts the execution of NN tasks through distribution of static data and streaming of dynamic data to form the basic activity context, adopts time and space slices to achieve elastic resource allocation for each activity, and performs configurable hybrid synchronous-asynchronous grouping. Thereby, Rivulet is capable of supporting independent and interactive execution. Building on Rivulet with hardware design for realizing spatiotemporal elasticity, a 28-nanometer TianjicX neuromorphic chip with event-driven, high parallelism, low latency, and low power was developed. Using a single TianjicX chip and a specially developed compiler stack, we built a multi-intelligent-tasking mobile robot, Tianjicat, to perform a cat-and-mouse game. Multiple tasks, including sound recognition and tracking, object recognition, obstacle avoidance, and decision-making, can be concurrently executed. Compared with NVIDIA Jetson TX2, latency is substantially reduced by 79.09 times, and dynamic power is reduced by 50.66%.

Photoacoustic imaging relies on diffused photons for optical contrast and diffracted ultrasound for high resolution. As a tomographic imaging modality, often an inverse problem of acoustic diffraction needs to be solved to reconstruct a photoacoustic image. The inverse problem is complicated by the fact that the acoustic properties, including the speed of sound distribution, in the image field of view are unknown. During reconstruction, subtle changes of the speed of sound in the acoustic ray path may accumulate and give rise to noticeable blurring in the image. Thus, in addition to the ultrasound detection bandwidth, inaccurate acoustic modeling, especially the unawareness of the speed of sound, defines the image resolution and influences image quantification. Here, we proposed a method termed feature coupling to jointly reconstruct the speed of sound distribution and a photoacoustic image with improved sharpness, at no additional hardware cost. Simulations, phantom studies, and in vivo experiments demonstrated the effectiveness and reliability of our method.

An optical fiber based Fabry-Pérot interferometer whose resonant wavelength can be dynamically tuned was designed and realized for photoacoustic mesoscopy. The optical path length (OPL) of the Fabry-Pérot cavity can be modulated by a photothermal heating process, which was achieved by adjusting the power of a 650 nm heating laser. The optical heating process can effectively change the thickness and refractive index of the polymer spacer of the sensor cavity. The robustness of the sensor can be greatly improved by proper packaging. The interferometer was interrogated by a relatively cheap wavelength-fixed 1550 nm laser for broadband and sensitive ultrasound detection, eliminating the requirement for an expensive tunable interrogation laser. The sensing module was then integrated into a photoacoustic mesoscopic imaging system. Two phantom imaging experiments and an ex vivo imaging experiment demonstrated the capability of such a miniature sensor. The interferometer has an acoustic detection bandwidth of up to 30 MHz and a noise equivalent pressure of 40 mPa/Hz^1/2 (i.e., 220 Pa over the full detection bandwidth). The new tuning mechanism and the batch-production compatibility of the sensor holds promises for commercialization and parallelized detection.

For many optical imaging modalities, image qualities are inevitably degraded by wavefront distortions caused by varying light speed. In optical microscopy and astronomy, adaptive optics (AO) has long been applied to compensate for such unwanted aberrations. Photoacoustic computed tomography (PACT), despite relying on the ultrasonic wave for image formation, suffers from the acoustic version of the same problem. However, this problem has traditionally been regarded as an inverse problem of jointly reconstructing both the initial pressure and the sound speed distributions. In this work, we proposed a method similar to indirect wavefront sensing in AO. We argued that wavefront distortions can be extracted and corrected by a frequency domain analysis of local images. In addition to an adaptively reconstructed aberration-free image, the speed of sound map can be subsequently estimated. We demonstrated the method by in silico, phantom, and in vivo experiments.

Artificial neural networks (ANNs) have gained considerable momentum in the past decade. Although at first the main task of the ANN paradigm was to tune the connection weights in fixed-architecture networks, there has recently been growing interest in evolving network architectures toward the goal of creating artificial general intelligence. Lagging behind this trend, current ANN hardware struggles for a balance between flexibility and efficiency but cannot achieve both. Here, we report on a novel approach for the on-demand generation of complex networks within a single memristor where multiple virtual nodes are created by time multiplexing and the non-trivial topological features, such as small-worldness, are generated by exploiting device dynamics with intrinsic cycle-to-cycle variability. When used for reservoir computing, memristive complex networks can achieve a noticeable increase in memory capacity a and respectable performance boost compared to conventional reservoirs trivially implemented as fully connected networks. This work expands the functionality of memristors for ANN computing.