The use of intra-operative imaging system as an intervention solution to provide more accurate localization of complicated structures has become a necessity during the neurosurgery. However, due to the limitations of conventional imaging systems, high-quality real-time intra-operative imaging remains as a challenging problem. Meanwhile, photoacoustic imaging has appeared so promising to provide images of crucial structures such as blood vessels and microvasculature of tumors. To achieve high-quality photoacoustic images of vessels regarding the artifacts caused by the incomplete data, we proposed an approach based on the combination of time-reversal (TR) and deep learning methods. The proposed method applies a TR method in the first layer of the network which is followed by the convolutional neural network with weights adjusted to a set of simulated training data for the other layers to estimate artifact-free photoacoustic images. It was evaluated using a generated synthetic database of vessels. The mean of signal to noise ratio (SNR), peak SNR, structural similarity index, and edge preservation index for the test data were reached 14.6 dB, 35.3 dB, 0.97 and 0.90, respectively. As our results proved, by using the lower number of detectors and consequently the lower data acquisition time, our approach outperforms the TR algorithm in all criteria in a computational time compatible with clinical use.

In the last two decades, photoacoustic imaging (PAI) as a non-invasive hybrid imaging modality has been used in a wide range of preclinical and clinical applications, such as functional brain mapping,^1,2 molecular imaging,^3,4 cancer diagnosis and staging,^{5 – 8} tissue vasculature imaging,^{9 – 12} guiding interventional procedures,^13,14 and dental health.^15,16 PAI detects the optical absorption contrast in tissue through the conversion of light to heat and thermoelastic effect, leading to the generation of acoustic waves.^{17 – 19} When the tissue is illuminated by short light pulses, the endogenous chromophores, such as hemoglobin, generate a photoacoustic (PA) signal due to their optical absorption.^20,21 In this procedure, the light energy is transformed into acoustic waves, and the efficacy of this conversion is often affected by the presence of noise arising from the surrounding background.^22,23 Therefore, the PA signal is often mixed by background noise, including thermal-acoustic noise in the medium as well as the transducer and electronic noises.²⁴ White Gaussian noise is one of the most common models for these types of randomly distributed thermal and electronic noise.²⁵ However, there is another component of noise arising from the light attenuation phenomena, caused by scattering characteristics of the tissue.²⁶ Furthermore, other types of noise disturb the PA signals, such as the fixed-pattern noise caused by electromagnetic interference. The combination of these different types of noise in the PA signal leads to a low signal-to-noise ratio (SNR) and results in a poor quality reconstructed PA image.^{27 – 30}

The gradient vector flow (GVF) algorithm has been used extensively as an efficient method for medical image segmentation. This algorithm suffers from poor robustness against noise as well as lack of convergence in small scale details and concavities. As a cure to this problem, in this paper the idea of multi scale is applied to the traditional GVF algorithm for segmentation of brain tumors in MRI images. Using this idea, the active contour is evolved with respect to scaled edge maps in a multi scale manner. The edge detection performance of the modified GVF algorithm is further enhanced by applying a threshold-based edge detector to improve the edge map. The Bspline snake is selected for representation of the active contour, due to its ability to capture corners and its local control. The results showed an improvement of 30% in the accuracy of tumor segmentation against traditional GVF and 10 % as compared to Bspline GVF in the presence of noise, besides the repeatability of the algorithm in contrast to traditional GVF. The clinical evaluation also proved the accuracy and sensitivity of the proposed method as 92.8% and 95.4%, respectively.

Electromagnetic-based navigation bronchoscopy requires accurate and robust estimation of the bronchoscope position inside the bronchial tree. However, respiratory motion, coughing, patient movement, and airway deformation inflicted by bronchoscope significantly hinder the accuracy of intraoperative bronchoscopic localization. In this study, a real-time and automatic registration procedure was proposed to superimpose the current location of the bronchoscope to corresponding locations on a centerline extracted from bronchial computed tomography (CT) images. A centerline-guided Gaussian mixture model (CG-GMM) was introduced to register a bronchoscope’s position concerning extracted centerlines. A GMM was fitted to bronchoscope positions where the orientation likelihood was chosen to assign the membership probabilities of the mixture model, which led to preserving the global and local structures. The problem was formulated and solved under the expectation maximization framework, where the feature correspondence and spatial transformation are estimated iteratively. Validation was performed on a dynamic phantom with four different respiratory motions and four human real bronchoscopy (RB) datasets. Results of the experiments conducted on the bronchial phantom showed that the average positional tracking error using the proposed approach was equal to 1.98 ± 0.98 mm that was reduced in comparison with independent electromagnetic tracking (EMT), iterative closest point (ICP), and coherent point drift (CPD) methods by 64%, 58%, and 53%, respectively. In the patient assessment part of the study, the average positional tracking error was 4.73 ± 4.76 mm and compared to ICP, and CPD methods showed 31.4% improvement of successfully tracked frames. Our approach introduces a novel method for real-time respiratory motion compensation that provides reliable guidance during bronchoscopic interventions and, thus could increase the diagnostic yield of transbronchial biopsy.

During neurosurgical procedures, the neuro-navigation system’s accuracy is affected by the brain shift phenomenon. One popular strategy is to compensate for brain shift using intraoperative ultrasound (iUS) registration with pre-operative magnetic resonance (MR) scans. This requires a satisfactory multimodal image registration method, which is challenging due to the low image quality of ultrasound and the unpredictable nature of brain deformation during surgery. In this paper, we propose an automatic unsupervised end-to-end MR–iUS registration approach named the Dual Discriminator Bayesian Generative Adversarial Network (D²BGAN). The proposed network consists of two discriminators and a generator optimized by a Bayesian loss function to improve the functionality of the generator, and we add a mutual information loss function to the discriminator for similarity measurements. Extensive validation was performed on the RESECT and BITE datasets, where the mean target registration error (mTRE) of MR–iUS registration using D²BGAN was determined to be 0.75 ± 0.3 mm. The D²BGAN illustrated a clear advantage by achieving an 85% improvement in the mTRE over the initial error. Moreover, the results confirmed that the proposed Bayesian loss function, rather than the typical loss function, improved the accuracy of MR–iUS registration by 23%. The improvement in registration accuracy was further enhanced by the preservation of the intensity and anatomical information of the input images.

Brain shift is an important obstacle to the application of image guidance during neurosurgical interventions. There has been a growing interest in intra-operative imaging to update the image-guided surgery systems. However, due to the innate limitations of the current imaging modalities, accurate brain shift compensation continues to be a challenging task. In this study, the application of intra-operative photoacoustic imaging and registration of the intra-operative photoacoustic with pre-operative MR images are proposed to compensate for brain deformation. Finding a satisfactory registration method is challenging due to the unpredictable nature of brain deformation. In this study, the co-sparse analysis model is proposed for photoacoustic-MR image registration, which can capture the interdependency of the two modalities. The proposed algorithm works based on the minimization of mapping transform via a pair of analysis operators that are learned by the alternating direction method of multipliers. The method was evaluated using an experimental phantom and ex vivo data obtained from a mouse brain. The results of the phantom data show about 63% improvement in target registration error in comparison with the commonly used normalized mutual information method. The results proved that intra-operative photoacoustic images could become a promising tool when the brain shift invalidates pre-operative MRI.