Shao-Ping Lu

Shao-Ping Lu

Auto-generated profile, based on publicly available information

Shao-Ping Lu

Nankai University

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 January 2021 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
ZZZhao Zhang
Zhao Zhang
ZLZheng Lin
Zheng Lin
JXJun Xu
Jun Xu
WJWen-Da Jin
Wen-Da Jin
SLShao-Ping Lu
Shao-Ping Lu
DFDeng-Ping Fan
Deng-Ping Fan
RGB-D salient object detection (SOD) aims to segment the most attractive objects in a pair of cross-modal RGB and depth images. Currently, most existing RGB-D SOD methods focus on the foreground region when utilizing the depth images. However, the background also provides important information in traditional SOD methods for promising performance. To better explore salient information in both foreground and background regions, this paper proposes a Bilateral Attention Network (BiANet) for the RGB-D SOD task. Specifically, we introduce a Bilateral Attention Module (BAM) with a complementary attention mechanism: foreground-first (FF) attention and background-first (BF) attention. The FF attention focuses on the foreground region with a gradual refinement style, while the BF one recovers potentially useful salient information in the background region. Benefited from the proposed BAM module, our BiANet can capture more meaningful foreground and background cues, and shift more attention to refining the uncertain details between foreground and background regions. Additionally, we extend our BAM by leveraging the multi-scale techniques for better SOD performance. Extensive experiments on six benchmark datasets demonstrate that our BiANet outperforms other state-of-the-art RGB-D SOD methods in terms of objective metrics and subjective visual comparison. Our BiANet can run up to 80 fps on 224×224 RGB-D images, with an NVIDIA GeForce RTX 2080Ti GPU. Comprehensive ablation studies also validate our contributions.

30 November 2018 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
MWMiao Wang
Miao Wang
GYGuo-Ye Yang
Guo-Ye Yang
JLJin-Kun Lin
Jin-Kun Lin
SZSong-Hai Zhang
Song-Hai Zhang
ASAriel Shamir
Ariel Shamir
SHShi-Min Hu
Shi-Min Hu
Video stabilization techniques are essential for most hand-held captured videos due to high-frequency shakes. Several 2D-, 2.5D-, and 3D-based stabilization techniques have been presented previously, but to the best of our knowledge, no solutions based on deep neural networks had been proposed to date. The main reason for this omission is shortage in training data as well as the challenge of modeling the problem using neural networks. In this paper, we present a video stabilization technique using a convolutional neural network. Previous works usually propose an off-line algorithm that smoothes a holistic camera path based on feature matching. Instead, we focus on low-latency, real-time camera path smoothing that does not explicitly represent the camera path and does not use future frames. Our neural network model, called StabNet, learns a set of mesh-grid transformations progressively for each input frame from the previous set of stabilized camera frames and creates stable corresponding latent camera paths implicitly. To train the network, we collect a dataset of synchronized steady and unsteady video pairs via a specially designed hand-held hardware. Experimental results show that our proposed online method performs comparatively to the traditional off-line video stabilization methods without using future frames while running about 10 times faster. More importantly, our proposed StabNet is able to handle low-quality videos, such as night-scene videos, watermarked videos, blurry videos, and noisy videos, where the existing methods fail in feature extraction or matching.

15 June 2022 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
JXJiehang Xie
Jiehang Xie
XCXuanbai Chen
Xuanbai Chen
TZTianyi Zhang
Tianyi Zhang
YZYixuan Zhang
Yixuan Zhang
SLShao-Ping Lu
Shao-Ping Lu
YYYulu Yang
Yulu Yang
Narrative videos usually illustrate the main content through multiple narrative information such as audios, video frames and subtitles. Existing video summarization approaches rarely consider the multiple dimensional narrative inputs, or ignore the impact of shots artistic assembly when directly applied to narrative videos. This paper introduces a multimodal-based and aesthetic-guided narrative video summarization method. Our method leverages multimodal information including visual content, subtitles and audio information through our specified key shots selection, subtitle summarization, and highlight extraction components. Furthermore, under the guidance of cinematographic aesthetic, we design a novel shots assembly module to ensure the shot content completeness and then assemble the selected shots into a desired summary. Besides, our method also provides the flexible specification for shots selection, to achieve which it automatically selects semantically related shots according to the user-designed text. By conducting a large number of quantitative experimental evaluations and user studies, we demonstrate that our method effectively preserves important narrative information of the original video, and it is capable of rapidly producing high-quality and aesthetic-guided narrative video summaries.

5 February 2018 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
BCBeerend Ceulemans
Beerend Ceulemans
SLShao-Ping Lu
Shao-Ping Lu
GLGauthier Lafruit
Gauthier Lafruit
Adrian MunteanuAdrian Munteanu
Adrian Munteanu
In many advanced multimedia systems, multiview content can offer more immersion compared to classical stereoscopy. The feeling of immersiveness is increased substantially by offering motion-parallax, as well as stereopsis. This drives both the so-called free-navigation and super-multiview technologies. However, it is currently still challenging to acquire, store, process, and transmit this type of content. This paper presents a novel multiview-interpolation framework for wide-baseline camera arrays. The proposed method comprises several novel components, including point cloud-based filtering, improved de-ghosting, multireference color blending, and depth-aware MRF-based disocclusion in painting. The method offers robustness against depth errors caused by quantization and smoothing across object boundaries. Furthermore, the available input color and depth are maximally exploited while preventing propagation of unreliable information to virtual viewpoints. The experimental results show that the proposed method outperforms the state-of-the-art View Synthesis Reference Software (VSRS 4.1) both in objective terms as well as subjectively, based on a visual assessment on a high-end light-field three-dimensional display.

25 July 2018 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
MWMiao Wang
Miao Wang
ASAriel Shamir
Ariel Shamir
JLJin-Kun Lin
Jin-Kun Lin
GYGuo-Wei Yang
Guo-Wei Yang
SLShao-Ping Lu
Shao-Ping Lu
SHShi-Min Hu
Shi-Min Hu
Selfie photography from the hand-held camera is becoming a popular media type. Although being convenient and flexible, it suffers from low camera motion stability, small field of view, and limited background content. These limitations can annoy users, especially, when touring a place of interest and taking selfie videos. In this paper, we present a novel method to create what we call a BiggerSelfie that deals with these shortcomings. Using a video of the environment that has partial content overlap with the selfie video, we stitch plausible frames selected from the environment video to the original selfie frames and stabilize the composed video content with a portrait-preserving constraint. Using the proposed method, one can easily obtain a stable selfie video with expanded background content by merely capturing some background shots. We show various results and several evaluations to demonstrate the applicability of our method.

12 April 2018 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
RZRui Zhong
Rui Zhong
Ionut SchiopuIonut Schiopu
Ionut Schiopu
BCBruno Cornelis
Bruno Cornelis
SLShao-Ping Lu
Shao-Ping Lu
JYJunsong Yuan
Junsong Yuan
Adrian MunteanuAdrian Munteanu
Adrian Munteanu
In this paper, a novel approach to encode lenslet (LL) images is proposed. The method departs from traditional block-based coding structures and employs a hexagonal-shaped pixel cluster, called macro-pixel, as an elementary coding unit. A novel prediction mode based on dictionary learning is proposed, whereby macro-pixels are represented by a sparse linear combination of atoms from a generic dictionary. Additionally, an optimized linear prediction mode and a directional prediction mode specifically designed for macro-pixels are proposed. Rate-distortion optimization is utilized to select the best intra prediction mode for each macro-pixel. Experimental results on the light field image data set show that the proposed coding system outperforms HEVC and the state-of-the-art in LL image coding with an average peak signal to noise ratio gain of 3.33 and 1.41 dB, respectively, and with rate savings of 67.13% and 34.30%, respectively.