Liao Qu

Liao Qu

Auto-generated profile, based on publicly available information

Liao Qu

Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, USA

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 February 2024 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
RGRuohao Guo
Ruohao Guo
XYXianghua Ying
Xianghua Ying
YQYanyu Qi
Yanyu Qi
LQLiao Qu
Liao Qu
Recent years have witnessed a growing interest in co-object segmentation and multi-modal salient object detection. Many efforts are devoted to segmenting co-existed objects among a group of images or detecting salient objects from different modalities. Albeit the appreciable performance achieved on respective benchmarks, each of these methods is limited to a specific task and cannot be generalized to other tasks. In this paper, we develop a Uni fied TR ansformer-based framework, namely UniTR , aiming at tackling the above tasks individually with a unified architecture. Specifically, a transformer module (CoFormer) is introduced to learn the consistency of relevant objects or complementarity from different modalities. To generate high-quality segmentation maps, we adopt a dual-stream decoding paradigm that allows the extracted consistent or complementary information to better guide mask prediction. Moreover, a feature fusion module (ZoomFormer) is designed to enhance backbone features and capture multi-granularity and multi-semantic information. Extensive experiments show that our UniTR performs well on 17 benchmarks , and surpasses existing state-of-the-art approaches.

28 October 2024
Verified
Conference Paper
Open access
RGRuohao Guo
Ruohao Guo
LQLiao Qu
Liao Qu
DNDantong Niu
Dantong Niu
YQYanyu Qi
Yanyu Qi
YWYue Wenzhen
Yue Wenzhen
XYXianghua Ying
Xianghua Ying
Audio-visual semantic segmentation (AVSS) aims to segment and classify sounding objects in videos with acoustic cues. However, most approaches operate on the close-set assumption and only identify pre-defined categories from training data, lacking the generalization ability to detect novel categories in practical applications. In this paper, we introduce a new task: open-vocabulary audio-visual semantic segmentation, extending AVSS task to open-world scenarios beyond the annotated label space. This is a more challenging task that requires recognizing all categories, even those that have never been seen nor heard during training. Moreover, we propose the first open-vocabulary AVSS framework, OV-AVSS, which mainly consists of two parts: 1) a universal sound source localization module to perform audio-visual fusion and locate all potential sounding objects and 2) an open-vocabulary classification module to predict categories with the help of the prior knowledge from large-scale pre-trained vision-language models. To properly evaluate the open-vocabulary AVSS, we split zero-shot training and testing subsets based on the AVSBench-semantic benchmark, namely AVSBench-OV. Extensive experiments demonstrate the strong segmentation and zero-shot generalization ability of our model on all categories. On the AVSBench-OV dataset, OV-AVSS achieves 55.43% mIoU on base categories and 29.14% mIoU on novel categories, exceeding the state-of-the-art zero-shot method by 41.88%/20.61% and open-vocabulary method by 10.2%/11.6%. The code is available at https://github.com/ruohaoguo/ovavss.

12 May 2025 in Expert Systems with Applications
Verified
Journal Article
Closed access
RGRuohao Guo
Ruohao Guo
WYWenzhen Yue
Wenzhen Yue
LQLiao Qu
Liao Qu
YQYanyu Qi
Yanyu Qi
DNDantong Niu
Dantong Niu
XYXianghua Ying
Xianghua Ying

28 October 2024
Verified
Conference Paper
Open access
RGRuohao Guo
Ruohao Guo
DNDantong Niu
Dantong Niu
LQLiao Qu
Liao Qu
YQYanyu Qi
Yanyu Qi
JSJi Shi
Ji Shi
XYXianghua Ying
Xianghua Ying
Panoramic audio-visual saliency detection is to segment the most attention-attractive regions in 360° panoramic videos with sound. To meticulously delineate the detected salient regions and effectively model human attention shift, we extend this task to more fine-grained instance scenarios: identifying salient object instances and inferring their saliency ranks. In this paper, we propose the first instance-level framework that can simultaneously be applied to segmentation and ranking of multiple salient objects in panoramic videos. Specifically, it consists of a distortion-aware pixel decoder to overcome panoramic distortions, a sequential audio-visual fusion module to integrate audio-visual information, and a spatio-temporal object decoder to separate individual instances and predict their saliency scores. Moreover, owing to the absence of such annotations, we create the ground-truth saliency ranks for the PAVS10K benchmark. Extensive experiments demonstrate that our model is capable of achieving state-of-the-art performance on the PAVS10K for both saliency detection and ranking tasks. The code is available at https://github.com/ruohaoguo/pavsodr.

20 October 2024
Verified
Book Chapter
Closed access
PDF file
YQYanyu Qi
Yanyu Qi
RGRuohao Guo
Ruohao Guo
ZLZhenbo Li
Zhenbo Li
DNDantong Niu
Dantong Niu
LQLiao Qu
Liao Qu