Yao Wang

Yao Wang

Auto-generated profile, based on publicly available information

Yao Wang

New York University Tandon School of Engineering, Electrical and Computer Engineering Department, Brooklyn, NY, USA

Research Focus & Metrics

Academic Career Length

20Years (2007-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 February 2021 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
TCTong Chen
Tong Chen
HLHaojie Liu
Haojie Liu
ZMZhan Ma
Zhan Ma
QSQiu Shen
Qiu Shen
XCXun Cao
Xun Cao
YWYao Wang
Yao Wang
This article proposes an end-to-end learnt lossy image compression approach, which is built on top of the deep nerual network (DNN)-based variational auto-encoder (VAE) structure with Non-Local Attention optimization and Improved Context modeling (NLAIC). Our NLAIC 1) embeds non-local network operations as non-linear transforms in both main and hyper coders for deriving respective latent features and hyperpriors by exploiting both local and global correlations, 2) applies attention mechanism to generate implicit masks that are used to weigh the features for adaptive bit allocation, and 3) implements the improved conditional entropy modeling of latent features using joint 3D convolutional neural network (CNN)-based autoregressive contexts and hyperpriors. Towards the practical application, additional enhancements are also introduced to speed up the computational processing (e.g., parallel 3D CNN-based context prediction), decrease the memory consumption (e.g., sparse non-local processing) and reduce the implementation complexity (e.g., a unified model for variable rates without re-training). The proposed model outperforms existing learnt and conventional (e.g., BPG, JPEG2000, JPEG) image compression methods, on both Kodak and Tecnick datasets with the state-of-the-art compression efficiency, for both PSNR and MS-SSIM quality measurements. We have made all materials publicly accessible at https://njuvision.github.io/NIC for reproducible research.

8 April 2024 in Nature Machine Intelligence
Verified
Journal Article
Open access
PDF file
XCXupeng Chen
Xupeng Chen
RWRan Wang
Ran Wang
AKAmirhossein Khalilian-Gourtani
Amirhossein Khalilian-Gourtani
LYLeyao Yu
Leyao Yu
PDPatricia Dugan
Patricia Dugan
AFAdeen Flinker
Adeen Flinker
Decoding human speech from neural signals is essential for brain–computer interface (BCI) technologies that aim to restore speech in populations with neurological deficits. However, it remains a highly challenging task, compounded by the scarce availability of neural signals with corresponding speech, data complexity and high dimensionality. Here we present a novel deep learning-based neural speech decoding framework that includes an ECoG decoder that translates electrocorticographic (ECoG) signals from the cortex into interpretable speech parameters and a novel differentiable speech synthesizer that maps speech parameters to spectrograms. We have developed a companion speech-to-speech auto-encoder consisting of a speech encoder and the same speech synthesizer to generate reference speech parameters to facilitate the ECoG decoder training. This framework generates natural-sounding speech and is highly reproducible across a cohort of 48 participants. Our experimental results show that our models can decode speech with high correlation, even when limited to only causal operations, which is necessary for adoption by real-time neural prostheses. Finally, we successfully decode speech in participants with either left or right hemisphere coverage, which could lead to speech prostheses in patients with deficits resulting from left hemisphere damage.

12 February 2014 in BMC Cancer
Verified
Journal Article
Open access
PDF file
HLHua yu Lu
Hua yu Lu
JLJingjing Lu
Jingjing Lu
XLXiaohong Li
Xiaohong Li
HZHui Zhu
Hui Zhu
XFXiangjun Fan
Xiangjun Fan
ZWZhiwei Wang
Zhiwei Wang
Background: Pancreatic cancer is one of the most aggressive cancers, and the aggressiveness of pancreatic cancer is in part due to its intrinsic and extrinsic drug resistance characteristics, which are also associated with the acquisition of epithelial-to-mesenchymal transition (EMT). Increasing evidence suggests that EMT-type cells share many biological characteristics with cancer stem-like cells. And miR-200 has been identified as a powerful regulator of EMT. Methods: Cancer Stem Cells (CSCs) of human pancreatic cancer cell line PANC-1 were processed for CD24, CD44 and ESA multi-colorstaining, and sorted out on a BD FACS Aria II machine. RT-qPCR was performed using the miScript PCR Kit to assay the expression of miR-200 family. In order to find the role of miR-200a in the process of EMT, miR-200a mimic was transfected to CSCs. Results: Pancreatic cancer cells with EMT phenotype displayed stem-like cell features characterized by the expression of cell surface markers CD24, CD44 and epithelial-specific antigen (ESA), which was associated with decreased expression of miR-200a. Moreover, overexpression of miR-200a was resulted in down-regulation of N-cadherin, ZEB1 and vimentin, but up-regulation of E-cadherin. In addition, miR-200a overexpression inhibited cell migration and invasion in CSCs. Conclusion: In our study, we found that miR-200a played an important role in linking the characteristics of cancer stem-like cells with EMT-like cell signatures in pancreatic cancer. Selective elimination of cancer stem-like cells by reversing the EMT phenotype to mesenchymal-to-epithelial transition (MET) phenotype using novel agents would be useful for prevention and/or treatment of pancreatic cancer.

3 November 2020 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
HLHaojie Liu
Haojie Liu
MLMing Lu
Ming Lu
ZMZhan Ma
Zhan Ma
FWFan Wang
Fan Wang
ZXZhihuang Xie
Zhihuang Xie
YWYao Wang
Yao Wang
Over the past two decades, traditional block-based video coding has made remarkable progress and spawned a series of well-known standards such as MPEG-4, H.264/AVC and H.265/HEVC. On the other hand, deep neural networks (DNNs) have shown their powerful capacity for visual content understanding, feature extraction and compact representation. Some previous works have explored the learnt video coding algorithms in an end-to-end manner, which show the great potential compared with traditional methods. In this paper, we propose an end-to-end deep neural video coding framework (NVC), which uses variational autoencoders (VAEs) with joint spatial and temporal prior aggregation (PA) to exploit the correlations in intra-frame pixels, inter-frame motions and inter-frame compensation residuals, respectively. Novel features of NVC include: 1) To estimate and compensate motion over a large range of magnitudes, we propose an unsupervised multiscale motion compensation network (MS-MCN) together with a pyramid decoder in the VAE for coding motion features that generates multiscale flow fields, 2) we design a novel adaptive spatiotemporal context model for efficient entropy coding for motion information, 3) we adopt nonlocal attention modules (NLAM) at the bottlenecks of the VAEs for implicit adaptive feature extraction and activation, leveraging its high transformation capacity and unequal weighting with joint global and local information, and 4) we introduce multi-module optimization and a multi-frame training strategy to minimize the temporal error propagation among P-frames. NVC is evaluated for the low-delay causal settings and compared with H.265/HEVC, H.264/AVC and the other learnt video compression methods following the common test conditions, demonstrating consistent gains across all popular test sequences for both PSNR and MS-SSIM distortion metrics.

24 January 2019 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
SMShervin Minaee
Shervin Minaee
YWYao Wang
Yao Wang
Signal decomposition is a classical problem in signal processing, which aims to separate an observed signal into two or more components, each with its own property. Usually, each component is described by its own subspace or dictionary. Extensive research has been done for the case where the components are additive, but in real-world applications, the components are often non-additive. For example, an image may consist of a foreground object overlaid on a background, where each pixel either belongs to the foreground or the background. In such a situation, to separate signal components, we need to find a binary mask which shows the location of each component. Therefore, it requires solving a binary optimization problem. Since most of the binary optimization problems are intractable, we relax this problem to the approximated continuous problem and solve it by alternating optimization technique. We show the application of the proposed algorithm for three applications: separation of text from a background in images, separation of moving objects from a background undergoing global camera motion in videos, and separation of sinusoidal and spike components in 1-D signals. We demonstrate in each case that considering the non-additive nature of the problem can lead to a significant improvement.

12 February 2019 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Verified
Journal Article
Closed access
LSLiyang Sun
Liyang Sun
FDFanyi Duanmu
Fanyi Duanmu
YLYong Liu
Yong Liu
YWYao Wang
Yao Wang
YYYinghua Ye
Yinghua Ye
DDDavid Dai
David Dai
360° video on-demand streaming is a key component of the emerging Virtual Reality (VR) and Augmented Reality (AR) applications. In such applications, sending the entire 360° video demands extremely high network bandwidth that may not be affordable by today’s networks. On the other hand, sending only the predicted user’s field of view (FoV) is not viable as it is hard to achieve perfect FoV prediction in on-demand streaming, where it is better to prefetch the video multiple seconds ahead, to absorb the network bandwidth fluctuation. This paper proposes a two-tier solution, where the base tier delivers the entire 360° span at a lower quality with a long prefetching buffer, and the enhancement tier delivers the predicted FoV at a higher quality using a short buffer. The base tier provides robustness to both network bandwidth variations and FoV prediction errors. The enhancement tier improves the video quality if it is delivered in time and FoV prediction is accurate. We study the optimal rate allocation between the two tiers and buffer provisioning for the enhancement tier to achieve the optimal trade-off between video quality and streaming robustness. We also design periodic and adaptive optimization frameworks to adapt to the bandwidth variations and FoV prediction errors in realtime. Through simulations driven by real LTE and WiGig network bandwidth traces and user FoV traces, we demonstrate that the proposed two-tier systems can achieve a high-level of Quality-of-Experience (QoE) in the face of network bandwidth and user FoV dynamics.