Junhong Zhao

Dr. Junhong Zhao

orcid

Research Focus & Metrics

Academic Career Length

15Years (2012-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 August 2021 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
JZJunhong Zhao
Junhong Zhao
ACAndrew Chalmers
Andrew Chalmers
TRTaehyun Rhee
Taehyun Rhee
High dynamic range (HDR) panoramic environment maps are widely used to illuminate virtual objects to blend with real-world scenes. However, in common applications for augmented and mixed-reality (AR/MR), capturing 360° surroundings to obtain an HDR environment map is often not possible using consumer-level devices. We present a novel light estimation method to predict 360° HDR environment maps from a single photograph with a limited field-of-view (FOV). We introduce the Dynamic Lighting network (DLNet), a convolutional neural network that dynamically generates the convolution filters based on the input photograph sample to adaptively learn the lighting cues within each photograph. We propose novel Spherical Multi-Scale Dynamic (SMD) convolutional modules to dynamically generate sample-specific kernels for decoding features in the spherical domain to predict 360° environment maps. Using DLNet and data augmentations with respect to FOV, an exposure multiplier, and color temperature, our model shows the capability of estimating lighting under diverse input variations. Compared with prior work that fixes the network filters once trained, our method maintains lighting consistency across different exposure multipliers and color temperature, and maintains robust light estimation accuracy as FOV increases. The surrounding lighting information estimated by our method ensures coherent illumination of 3D objects blended with the input photograph, enabling high fidelity augmented and mixed reality supporting a wide range of environmental lighting conditions and device sensors.

12 June 2020 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
ACAndrew Chalmers
Andrew Chalmers
JZJunhong Zhao
Junhong Zhao
DMDaniel Medeiros
Daniel Medeiros
TRTaehyun Rhee
Taehyun Rhee
Corresponding lighting and reflectance between real and virtual objects is important for spatial presence in augmented and mixed reality (AR and MR) applications. We present a method to reconstruct real-world environmental lighting, encoded as a reflection map (RM), from a conventional photograph. To achieve this, we propose a stacked convolutional neural network (SCNN) that predicts high dynamic range (HDR) 360° RMs with varying roughness from a limited field of view, low dynamic range photograph. The SCNN is progressively trained from high to low roughness to predict RMs at varying roughness levels, where each roughness level corresponds to a virtual object's roughness (from diffuse to glossy) for rendering. The predicted RM provides high-fidelity rendering of virtual objects to match with the background photograph. We illustrate the use of our method with indoor and outdoor scenes trained on separate indoor/outdoor SCNNs showing plausible rendering and composition of virtual objects in AR/MR. We show that our method has improved quality over previous methods with a comparative user study and error metrics.

1 July 2023
Verified
Conference Paper
Closed access
YCYutong Chen
Yutong Chen
JZJunhong Zhao
Junhong Zhao
WZWei-Qiang Zhang
Wei-Qiang Zhang
It is in high demand to generate facial animation with high realism, but it remains a challenging task. Existing approaches of speech-driven facial animation can produce satisfactory mouth movement and lip synchronization, but show weakness in dramatic emotional expressions and flexibility in emotion control. This paper presents a novel deep learning-based approach for expressive facial animation generation from speech that can exhibit wide-spectrum facial expressions with controllable emotion type and intensity. We propose an emotion controller module to learn the relationship between the emotion variations (e.g., types and intensity) and the corresponding facial expression parameters. It enables emotion-controllable facial animation, where the target expression can be continuously adjusted as desired. The qualitative and quantitative evaluations show that the animation generated by our method is rich in facial emotional expressiveness while retaining accurate lip movement, outperforming other state-of-the-art methods.

30 May 2023 in Journal of Computer Science and Technology
Verified
Journal Article
Closed access
PDF file
FZFanglue Zhang
Fanglue Zhang
JZJunhong Zhao
Junhong Zhao
YZYun Zhang
Yun Zhang
SZStefanie Zollmann
Stefanie Zollmann

21 October 2022
Verified
Book Chapter
Closed access
PDF file
JWJoshua Weir
Joshua Weir
JZJunhong Zhao
Junhong Zhao
ACAndrew Chalmers
Andrew Chalmers
TRTaehyun Rhee
Taehyun Rhee

11 December 2022
Verified
Conference Paper
Closed access
YNYuting Nie
Yuting Nie
JZJunhong Zhao
Junhong Zhao
WZWei-Qiang Zhang
Wei-Qiang Zhang
JBJinfeng Bai
Jinfeng Bai
Language identification is the task of automatically determining the identity of a language conveyed by a spoken segment. It has a profound impact on the multilingual interoperability of an intelligent speech system. Despite language identification attaining high accuracy on medium or long utterances (>3 s), the performance on short utterances (<=1s) is still far from satisfactory. We propose a BERT-based language identification system (BERT-LID) to improve language identification performance, especially on short-duration speech segments. We extend the original BERT model by taking the phonetic posteriorgrams (PPG) derived from the front-end phone recognizer as input. Then we deployed the optimal deep classifier followed by it for language identification. Our BERT-LID model can improve the baseline accuracy by about 6.5% on long-segment identification and 19.9% on short-segment identification, demonstrating our BERT-LID’s effectiveness to language identification.