1 April 2024 in Proceedings of the VLDB Endowment
Discovering tables from poorly maintained data lakes is a significant challenge in data management. Two key tasks are identifying joinable and unionable tables, crucial for data integration, analysis, and machine learning. However, there's a lack of a comprehensive benchmark for evaluating existing methods. To address this, we introduce LakeBench, a large-scale table discovery benchmark. It evaluates effectiveness, efficiency, and scalability of table join & union search methods. With over 16 million real tables, LakeBench is 1,600X larger than existing datasets and 100X larger in storage size. It includes synthesized and real queries with ground truth, totaling more than 10 thousand queries - 10X more than used in any existing evaluation. We spent over 7,500 human hours labeling these queries and constructing diverse query categories for thorough evaluation. Our benchmark thoroughly evaluates state-of-the-art table discovery methods, providing insights into their performance and highlighting research opportunities.
29 May 2026 in The Journal of Physical Chemistry A
Advanced photon sources deliver X-ray with unprecedented brilliance and coherence, facilitating sophisticated multimodal characterization of diverse materials while simultaneously generating data volumes that far exceed the capacity of traditional, expert-driven analytical workflows. This growing gap between data acquisition and interpretation has become a critical bottleneck for timely scientific discovery, which creates a critical need for high-throughput and automated data analysis tools. Recent advances in artificial intelligence (AI) offer a transformative potential to address this challenge, enabling the direct mapping of complex experimental observations into scientific insights. However, existing AI-driven tools are remain largely fragmented and task-specific, limiting their adoption of modern multimodal experiments at the facility scale. Here we present IPSBrain, a unified AI-powered platform that serves as Intelligent Photon Source Brain for automated analysis of multimodal synchrotron experimental data. The platform integrated a series of novel, self-developed machine-learning-based models that enable automated data analysis from multiple experimental techniques, including diffraction, scattering, X-ray absorption, and tomograph. The platform is accessible through an intuitive web interface, which significantly lowers the technical barrier to advanced scientific data interpretation. By enabling end-to-end intelligent workflows from data acquisition to quantitative physical insight, IPSBrain supports high-throughput, reproducible, and scalable analysis aligned with the operational demands of modern synchrotron radiation sources. IPSBrain not only demonstrates a practical and extensible framework for AI-driven synchrotron data analysis but also establishes a transferable paradigm for intelligent data analysis of other large-scale research infrastructures, such as neutron sources and free-electron lasers.
23 December 2025 in ACM Transactions on Information Systems
A data lake maintains large amounts of heterogeneous data with different data schemas and query interfaces. Efficiently querying and analyzing the heterogeneous data enables users to gain more complete insights. In this article, we study a novel problem of distributed keyword search across heterogeneous data sources. Traditional distributed search algorithms generally require the predefined crossing edges connecting relevant data instances for communication between different sources, which is unpractical for the data lake due to the schema heterogeneity. To effectively perform keyword search over the data lake, we first introduce canonical graphs and then develop a best-first search algorithm called UnifySea, which explores the answers across different sources based on the unified identification of related instances. To further improve the query efficiency, we propose a novel incremental keyword search algorithm called DistSea, which just need to identify the promising relevant data between different sources. DistSea incrementally calculates the optimal answers based on locally partial evaluation. Equipped with several efficient pruning rules, DistSea reduces unpromising tree calculation across different sources. Experimental evaluations on six real-world benchmarks demonstrate the effectiveness, efficiency, and scalability of the proposed algorithms.