Song Bai

Song Bai

Auto-generated profile, based on publicly available information

Song Bai

Bytedance, Beijing, China

Research Focus & Metrics

Academic Career Length

37Years (1990-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 February 2021 in Pattern Recognition
Verified
Journal Article
Closed access
SBSong Bai
Song Bai
FZFeihu Zhang
Feihu Zhang
PTPhilip H.S. Torr
Philip H.S. Torr

16 October 2018 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
PTPeng Tang
Peng Tang
Xinggang WangXinggang Wang
Xinggang Wang
SBSong Bai
Song Bai
WSWei Shen
Wei Shen
XBXiang Bai
Xiang Bai
AYAlan Loddon Yuille
Alan Loddon Yuille
Weakly Supervised Object Detection (WSOD), using only image-level annotations to train object detectors, is of growing importance in object recognition. In this paper, we propose a novel deep network for WSOD. Unlike previous networks that transfer the object detection problem to an image classification problem using Multiple Instance Learning (MIL), our strategy generates proposal clusters to learn refined instance classifiers by an iterative process. The proposals in the same cluster are spatially adjacent and associated with the same object. This prevents the network from concentrating too much on parts of objects instead of whole objects. We first show that instances can be assigned object or background labels directly based on proposal clusters for instance classifier refinement, and then show that treating each cluster as a small new bag yields fewer ambiguities than the directly assigning label method. The iterative instance classifier refinement is implemented online using multiple streams in convolutional neural networks, where the first is an MIL network and the others are for instance classifier refinement supervised by the preceding one. Experiments are conducted on the PASCAL VOC, ImageNet detection, and MS-COCO benchmarks for WSOD. Results show that our method outperforms the previous state of the art significantly.

10 August 2022 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
XLXiaolong Liu
Xiaolong Liu
QWQimeng Wang
Qimeng Wang
YHYao Hu
Yao Hu
XTXu Tang
Xu Tang
SZShiwei Zhang
Shiwei Zhang
XBXiang Bai
Xiang Bai
Temporal action detection (TAD) aims to determine the semantic label and the temporal interval of every action instance in an untrimmed video. It is a fundamental and challenging task in video understanding. Previous methods tackle this task with complicated pipelines. They often need to train multiple networks and involve hand-designed operations, such as non-maximal suppression and anchor generation, which limit the flexibility and prevent end-to-end learning. In this paper, we propose an end-to-end Transformer-based method for TAD, termed TadTR. Given a small set of learnable embeddings called action queries, TadTR adaptively extracts temporal context information from the video for each query and directly predicts action instances with the context. To adapt Transformer to TAD, we propose three improvements to enhance its locality awareness. The core is a temporal deformable attention module that selectively attends to a sparse set of key snippets in a video. A segment refinement mechanism and an actionness regression head are designed to refine the boundaries and confidence of the predicted instances, respectively. With such a simple pipeline, TadTR requires lower computation cost than previous detectors, while preserving remarkable performance. As a self-contained detector, it achieves state-of-the-art performance on THUMOS14 (56.7% mAP) and HACS Segments (32.09% mAP). Combined with an extra action classifier, it obtains 36.75% mAP on ActivityNet-1.3. Code is available at https://github.com/xlliu7/TadTR .

18 June 2022 in International Journal of Computer Vision
Verified
Journal Article
Open access
PDF file
JQJiyang Qi
Jiyang Qi
YGYan Gao
Yan Gao
YHYao Hu
Yao Hu
Xinggang WangXinggang Wang
Xinggang Wang
XLXiaoyu Liu
Xiaoyu Liu
SBSong Bai
Song Bai
Can our video understanding systems perceive objects when a heavy occlusion exists in a scene? To answer this question, we collect a large-scale dataset called OVIS for occluded video instance segmentation, that is, to simultaneously detect, segment, and track instances in occluded scenes. OVIS consists of 296k high-quality instance masks from 25 semantic categories, where object occlusions usually occur. While our human vision systems can understand those occluded instances by contextual reasoning and association, our experiments suggest that current video understanding systems cannot. On the OVIS dataset, the highest AP achieved by state-of-the-art algorithms is only 16.3, which reveals that we are still at a nascent stage for understanding objects, instances, and videos in a real-world scenario. We also present a simple plug-and-play module that performs temporal feature calibration to complement missing object cues caused by occlusion. Built upon MaskTrack R-CNN and SipMask, we obtain a remarkable AP improvement on the OVIS dataset. The OVIS dataset and project code are available at http://songbai.site/ovis.

20 October 2022
Verified
Book Chapter
Closed access
PDF file
JWJunfeng Wu
Junfeng Wu
QLQihao Liu
Qihao Liu
YJYi Jiang
Yi Jiang
SBSong Bai
Song Bai
AYAlan Yuille
Alan Yuille
XBXiang Bai
Xiang Bai

16 October 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
SBSong Bai
Song Bai
YLYingwei Li
Yingwei Li
YZYuyin Zhou
Yuyin Zhou
QLQizhu Li
Qizhu Li
PTPhilip H.S. Torr
Philip H.S. Torr
Person re-identification (re-ID) has attracted much attention recently due to its great importance in video surveillance. In general, distance metrics used to identify two person images are expected to be robust under various appearance changes. However, our work observes the extreme vulnerability of existing distance metrics to adversarial examples, generated by simply adding human-imperceptible perturbations to person images. Hence, the security danger is dramatically increased when deploying commercial re-ID systems in video surveillance. Although adversarial examples have been extensively applied for classification analysis, it is rarely studied in metric analysis like person re-identification. The most likely reason is the natural gap between the training and testing of re-ID networks, that is, the predictions of a re-ID network cannot be directly used during testing without an effective metric. In this work, we bridge the gap by proposing Adversarial Metric Attack, a parallel methodology to adversarial classification attacks. Comprehensive experiments clearly reveal the adversarial effects in re-ID systems. Meanwhile, we also present an early attempt of training a metric-preserving network, thereby defending the metric against adversarial attacks. At last, by benchmarking various adversarial settings, we expect that our work can facilitate the development of adversarial attack and defense in metric-based applications.