Xinyu Gong

Xinyu Gong

Auto-generated profile, based on publicly available information

Xinyu Gong

Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 January 2021 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
YJYifan Jiang
Yifan Jiang
XGXinyu Gong
Xinyu Gong
DLDing Liu
Ding Liu
YCYu Cheng
Yu Cheng
CFChen Fang
Chen Fang
ZWZhangyang (Atlas) Wang
Zhangyang (Atlas) Wang
Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data? As one such example, this paper explores the low-light image enhancement problem, where in practice it is extremely challenging to simultaneously take a low-light and a normal-light photo of the same visual scene. We propose a highly effective unsupervised generative adversarial network, dubbed EnlightenGAN, that can be trained without low/normal-light image pairs, yet proves to generalize very well on various real-world test images. Instead of supervising the learning using ground truth data, we propose to regularize the unpaired training using the information extracted from the input itself, and benchmark a series of innovations for the low-light image enhancement problem, including a global-local discriminator structure, a self-regularized perceptual loss fusion, and the attention mechanism. Through extensive experiments, our proposed approach outperforms recent methods under a variety of metrics in terms of visual quality and subjective user study. Thanks to the great flexibility brought by unpaired training, EnlightenGAN is demonstrated to be easily adaptable to enhancing real-world images from various domains. Our codes and pre-trained models are available at: https://github.com/VITA-Group/EnlightenGAN.

31 October 2022
Verified
Book Chapter
Closed access
PDF file
ZFZhiwen Fan
Zhiwen Fan
YJYifan Jiang
Yifan Jiang
PWPeihao Wang
Peihao Wang
XGXinyu Gong
Xinyu Gong
DXDejia Xu
Dejia Xu
ZWZhangyang (Atlas) Wang
Zhangyang (Atlas) Wang

1 December 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Open access
WCWuyang Chen
Wuyang Chen
XGXinyu Gong
Xinyu Gong
JWJunru Wu
Junru Wu
YWYunchao Wei
Yunchao Wei
HSHumphrey Shi
Humphrey Shi
ZWZhangyang (Atlas) Wang
Zhangyang (Atlas) Wang
This work targets designing a principled and unified training-free framework for Neural Architecture Search (NAS), with high performance, low cost, and in-depth interpretation. NAS has been explosively studied to automate the discovery of top-performer neural networks, but suffers from heavy resource consumption and often incurs search bias due to truncated training or approximations. Recent NAS works Mellor et al. 2021, Chen et al. 2021, Abdelfattah et al. 2021 start to explore indicators that can predict a network's performance without training. However, they either leveraged limited properties of deep networks, or the benefits of their training-free indicators were not applied to more extensive search methods. By rigorous correlation analysis, we present a unified framework to understand and accelerate NAS, by disentangling “ TEG ” characteristics of searched networks – T rainability, E xpressivity, G eneralization – all assessed in a training-free manner. The TEG indicators could be scaled up and integrated with various NAS search methods, including both supernet and single-path NAS approaches. Extensive studies validate the effective and efficient guidance from our TEG-NAS framework, leading to both improved search accuracy and over 56% reduction in search time cost. Moreover, we visualize search trajectories on three landscapes of “TEG” characteristics, observing that a good local minimum is easier to find on NAS-Bench-201 given its simple topology, whereas balancing “TEG” characteristics is much harder on the DARTS space due to its complex landscape geometry.

6 May 2022 in CAAI Transactions on Intelligence Technology
Verified
Journal Article
Open access
MSMingwei Sun
Mingwei Sun
XGXinyu Gong
Xinyu Gong
HNHaitao Nie
Haitao Nie
MIMuhammad Minhas Iqbal
Muhammad Minhas Iqbal
BXBin Xie
Bin Xie
Evaluation of calligraphic copy is the core of Chinese calligraphy appreciation and inheritance. However, previous aesthetic evaluation studies often focussed on photos and paintings, with few attempts on Chinese calligraphy. To solve this problem, a Siamese regression aesthetic fusion method is proposed, named SRAFE, for Chinese calligraphy based on the combination of calligraphy aesthetics and deep learning. First, a dataset termed Evaluated Chinese Calligraphy Copies (E3C) is constructed for aesthetic evaluation. Second, 12 hand‐crafted aesthetic features based on the shape, structure, and stroke of calligraphy are designed. Then, the Siamese regression network (SRN) is designed to extract the deep aesthetic representation of calligraphy. Finally, the SRAFE method is built by fusing the deep aesthetic features with the hand‐crafted aesthetic features. Experimental results show that scores given by SRAFE are similar to the aesthetic evaluation label of E3C, proving the effectiveness of the authors’ method.

24 January 2021 in arXiv
Verified
Preprint
Open access
PDF file
YJYifan Jiang
Yifan Jiang
XGXinyu Gong
Xinyu Gong
DLDing Liu
Ding Liu
YCYu Cheng
Yu Cheng
CFChen Fang
Chen Fang
ZWZhangyang (Atlas) Wang
Zhangyang (Atlas) Wang
Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data? As one such example, this paper explores the low-light image enhancement problem, where in practice it is extremely challenging to simultaneously take a low-light and a normal-light photo of the same visual scene. We propose a highly effective unsupervised generative adversarial network, dubbed EnlightenGAN, that can be trained without low/normal-light image pairs, yet proves to generalize very well on various real-world test images. Instead of supervising the learning using ground truth data, we propose to regularize the unpaired training using the information extracted from the input itself, and benchmark a series of innovations for the low-light image enhancement problem, including a global-local discriminator structure, a self-regularized perceptual loss fusion, and attention mechanism. Through extensive experiments, our proposed approach outperforms recent methods under a variety of metrics in terms of visual quality and subjective user study. Thanks to the great flexibility brought by unpaired training, EnlightenGAN is demonstrated to be easily adaptable to enhancing real-world images from various domains. The code is available at \url{https://github.com/yueruchen/EnlightenGAN}

16 March 2021 in arXiv
Verified
Preprint
Open access
PDF file
WCWuyang Chen
Wuyang Chen
XGXinyu Gong
Xinyu Gong
ZWZhangyang Wang
Zhangyang Wang
Neural Architecture Search (NAS) has been explosively studied to automate the discovery of top-performer neural networks. Current works require heavy training of supernet or intensive architecture evaluations, thus suffering from heavy resource consumption and often incurring search bias due to truncated training or approximations. Can we select the best neural architectures without involving any training and eliminate a drastic portion of the search cost? We provide an affirmative answer, by proposing a novel framework called training-free neural architecture search (TE-NAS). TE-NAS ranks architectures by analyzing the spectrum of the neural tangent kernel (NTK) and the number of linear regions in the input space. Both are motivated by recent theory advances in deep networks and can be computed without any training and any label. We show that: (1) these two measurements imply the trainability and expressivity of a neural network; (2) they strongly correlate with the network's test accuracy. Further on, we design a pruning-based NAS mechanism to achieve a more flexible and superior trade-off between the trainability and expressivity during the search. In NAS-Bench-201 and DARTS search spaces, TE-NAS completes high-quality search but only costs 0.5 and 4 GPU hours with one 1080Ti on CIFAR-10 and ImageNet, respectively. We hope our work inspires more attempts in bridging the theoretical findings of deep networks and practical impacts in real NAS applications. Code is available at: https://github.com/VITA-Group/TENAS.