Recent advancements in Large Language Models (LLMs) and Large Multi-modal Models (LMMs) have shown potential in various medical applications, such as Intelligent Medical Diagnosis. Although impressive results have been achieved, we find that existing benchmarks do not reflect the complexity of real medical reports and specialized in-depth reasoning capabilities. In this work, we establish a comprehensive benchmark in the field of medical specialization and introduced RJUA-MedDQA, which contains 2000 real-world Chinese medical report images poses several challenges: comprehensively interpreting imgage content across a wide variety of challenging layouts, possessing the numerical reasoning ability to identify abnormal indicators and demonstrating robust clinical reasoning ability to provide the statement of disease diagnosis, status and advice based on a collection of medical contexts. We carefully design the data generation pipeline and proposed the Efficient Structural Restoration Annotation (ESRA) Method, aimed at restoring textual and tabular content in medical report images. This method substantially enhances annotation efficiency, doubling the productivity of each annotator, and yields a 26.8% improvement in accuracy. We conduct extensive evaluations, including few-shot assessments of 5 LMMs which are capable of solving Chinese medical QA tasks. To further investigate the limitations and potential of current LMMs, we conduct comparative experiments on a set of strong LLMs by using image-text generated by ESRA method. We report the performance of baselines and offer several observations: (1) The overall performance of existing LMMs is still limited; however LMMs more robust to low-quality and diverse-structured images compared to LLMs. (3) Reasoning across context and image content present significant challenges. We hope this benchmark helps the community make progress on these challenging tasks in multi-modal medical document understanding and facilitate its application in healthcare. Our dataset will be publicly available for noncommercial use at https://github.com/Alipay-Med/medDQA_benchmark.git

When the evolution of discrete time quantum walk is carried out for particles, the ramble state is prone to error due to the influence of system noise. A multiparticle quantum walk error correction algorithm based on the two-lattice Bose–Hubbard model is proposed in this study. First, two point Bose–Hubbard models are constructed according to the local Euclidean generator, and it is proved that the two elements in the model can be replaced arbitrarily. Second, the relationship between the transition intensity and entanglement degree of the particles in the model is obtained by using the Bethe hypothesis method. Third, the position of the quantum lattice is coded and the quantum state exchange gate is constructed. Finally, the state replacement of quantum walk on the lattice point is carried out by switching the walker to the lattice point of quantum error correction code, and the replacement is carried out again. The entanglement of quantum particles in the double-lattice Bose–Hubbard model is simulated numerically. When the ratio of the interaction between particles and the transition intensity of particles is close to 0, the entanglement operation of quantum particles in the model can be realized by using this algorithm. According to the properties of the Bose–Hubbard model, quantum walking error correction can be realized after particle entanglement. This study introduces the popular restnet network as a training model, which increases the decoding speed of the error correction circuit by about 33%. More importantly, the lower threshold limit of the convolutional neural network (CNN) decoder is increased from 0.0058 under the traditional minimum weight perfect matching (MWPM) to 0.0085, which realizes the stable progress of quantum walk with high fault tolerance rate.