During surgery, a surgeon has to remove pathological lesions, replace abnormal tissue, or reconnect or repair damaged tissue structures, all of this while healthy tissue areas have to be maintained and organs at risk have to stay completely untreated and healthy. In order to accomplish this objective, to remove pathological while saving healthy tissue, the surgeon has to differentiate between healthy tissue areas and abnormal or damaged tissue. This manual differentiation process is very complex as the visual occurrence of living tissue, whether healthy or abnormal, often shows no significant optical differences under white light illumination. Furthermore, pathological structures mainly develop between or in the direct proximity to important structures and have most fine extensions. These facts make it hard to distinguish pathological from healthy tissue and, as a consequence, the structure exposition process is of high risk and a significant degree of difficulty, because damage to healthy structures can cause a temporary or permanent loss of functionality of the affected region.

Flap loss through limited perfusion remains a major complication in reconstructive surgery. Continuous monitoring of perfusion will facilitate early detection of insufficient perfusion. Remote or imaging photoplethysmography (rPPG/iPPG) as a non-contact, non-ionizing, and non-invasive monitoring technique provides objective and reproducible information on physiological parameters. The aim of this study is to establish rPPG for intra- and postoperative monitoring of flap perfusion in patients undergoing reconstruction with free fasciocutaneous flaps (FFCF). We developed a monitoring algorithm for flap perfusion, which was evaluated in 15 patients. For 14 patients, ischemia of the FFCF in the forearm and successful reperfusion of the implanted FFCF was quantified based on the local signal. One FFCF showed no perfusion after reperfusion and devitalized in the course. Intraoperative monitoring of perfusion with rPPG provides objective and reproducible results. Therefore, rPPG is a promising technology for standard flap perfusion monitoring on low costs without the need for additional monitoring devices.

We develop a stereo-multispectral endoscopic prototype in which a filter-wheel is used for surgical guidance to remove cholesteatoma tissue in the middle ear. Cholesteatoma is a destructive proliferating tissue. The only treatment for this disease is surgery. Removal is a very demanding task, even for experienced surgeons. It is very difficult to distinguish between bone and cholesteatoma. In addition, it can even reoccur if not all tissue particles of the cholesteatoma are removed, which leads to undesirable follow-up operations. Therefore, we propose an image-based method that combines multispectral tissue classification and 3D reconstruction to identify all parts of the removed tissue and determine their metric dimensions intraoperatively. The designed multispectral filter-wheel 3D-endoscope prototype can switch between narrow-band spectral and broad-band white illumination, which is technically evaluated in terms of optical system properties. Further, it is tested and evaluated on three patients. The wavelengths 400 nm and 420 nm are identified as most suitable for the differentiation task. The stereoscopic image acquisition allows accurate 3D surface reconstruction of the enhanced image information. The first results are promising, as the cholesteatoma can be easily highlighted, correctly identified, and visualized as a true-to-scale 3D model showing the patient-specific anatomy.

The optical properties of human tissues have an effect on many diagnostic and therapeutic applications in laser medicine, but they are also important for routine medical diagnostics. Light can penetrate and then escape tissue, which increases the interest in understanding the scattering and absorption properties of human tissues, as it is the basis for medical diagnostics and therapy.¹ Specifying the optical properties of human tissue is the initial step in interpreting measurements correctly. These properties can then be used to predict the light distribution and deposition in tissue for many applications, e.g., intraoperative tissue analysis.

One of the largest problems for the transplantation medicine is the lack of donor grafts.¹ Compared to 2010, the rate of postmortem organ donations in Germany has decreased by 29% in 2019, which corresponds to 1210 organs. Despite an improvement with respect to 2017, the need for organs in 2019 (9271 organs required) far exceeds the number of available donor organs (3767 organs transplanted).² This high demand for donor organs shows in particular the need to increase the number of successfully transplanted organs. Optimal use of the few donor organs is urgently required not only with regard to the individual person to be treated but also from a social and political point of view. The most frequently transplanted organs are kidney, liver, heart, and lung. However, in Germany, the discrepancy between supply and demand in kidney transplantation is alarming. A total of 2291 kidneys has been available for transplantation in 2018, compared to 7526 open transplant recipients at the end of the year.³ Apart from this shortage, the rejection rate for available kidney transplants has been 14.8% in 2019.² Due to the lack of an accurate predictive organ evaluation strategy, certain organs were rejected although they would still have been suitable for transplantation.^{4 – 7} The decision to accept or reject an organ for transplantation depends primarily on the expertise of the transplant team rather than on an objective measurement. In addition in the years 2014 to 2019, intra- or postoperative complications of kidney transplantations requiring a second kidney transplantation have been between 15.3% and 20.4%.^2,3,8 Therefore, an optical evaluation tool to monitor organ quality before transplantation would be of great interest.

We present an interactive and multimodal-based augmented reality system for computer-assisted surgery in the context of ear, nose and throat (ENT) treatment. The proposed processing pipeline uses fully digital stereoscopic imaging devices, which support multispectral and white light imaging to generate high resolution image data, and consists of five modules. Input/output data handling, a hybrid multimodal image analysis and a bi-directional interactive augmented reality (AR) and mixed reality (MR) interface for local and remote surgical assistance are of high relevance for the complete framework. The hybrid multimodal 3D scene analysis module uses different wavelengths to classify tissue structures and combines this spectral data with metric 3D information. Additionally, we propose a zoom-independent intraoperative tool for virtual ossicular prosthesis insertion (e.g. stapedectomy) guaranteeing very high metric accuracy in sub-millimeter range (1/10 mm). A bi-directional interactive AR/MR communication module guarantees low latency, while consisting surgical information and avoiding informational overload. Display agnostic AR/MR visualization can show our analyzed data synchronized inside the digital binocular, the 3D display or any connected head-mounted-display (HMD). In addition, the analyzed data can be enriched with annotations by involving external clinical experts using AR/MR and furthermore an accurate registration of preoperative data. The benefits of such a collaborative surgical system are manifold and will lead to a highly improved patient outcome through an easier tissue classification and reduced surgery risk.