Humphrey Shi

Humphrey Shi

Auto-generated profile, based on publicly available information

Humphrey Shi

Georgia Institute of Technology, Atlanta, GA, USA

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 July 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
ZHZilong Huang
Zilong Huang
Xinggang WangXinggang Wang
Xinggang Wang
YWYunchao Wei
Yunchao Wei
LHLichao Huang
Lichao Huang
HSHumphrey Shi
Humphrey Shi
THThomas S. Huang
Thomas S. Huang
Contextual information is vital in visual understanding problems, such as semantic segmentation and object detection. We propose a criss-cross network (CCNet) for obtaining full-image contextual information in a very effective and efficient way. Concretely, for each pixel, a novel criss-cross attention module harvests the contextual information of all the pixels on its criss-cross path. By taking a further recurrent operation, each pixel can finally capture the full-image dependencies. Besides, a category consistent loss is proposed to enforce the criss-cross attention module to produce more discriminative features. Overall, CCNet is with the following merits: 1) GPU memory friendly. Compared with the non-local block, the proposed recurrent criss-cross attention module requires $11\times$ less GPU memory usage. 2) High computational efficiency. The recurrent criss-cross attention significantly reduces FLOPs by about 85 percent of the non-local block. 3) The state-of-the-art performance. We conduct extensive experiments on semantic segmentation benchmarks including Cityscapes, ADE20K, human parsing benchmark LIP, instance segmentation benchmark COCO, video segmentation benchmark CamVid. In particular, our CCNet achieves the mIoU scores of 81.9, 45.76 and 55.47 percent on the Cityscapes test set, the ADE20K validation set and the LIP validation set respectively, which are the new state-of-the-art results. The source codes are available at https://github.com/speedinghzl/CCNethttps://github.com/speedinghzl/CCNet .

8 August 2023 in International Journal of Computer Vision
Verified
Journal Article
Open access
PDF file
YMYiqun Mei
Yiqun Mei
YFYuchen Fan
Yuchen Fan
YZYulun Zhang
Yulun Zhang
JYJIAHUI YU
JIAHUI YU
YZYuqian Zhou
Yuqian Zhou
HSHumphrey Shi
Humphrey Shi
Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales. However, recent advanced deep convolutional neural network-based methods for image restoration do not take full advantage of self-similarities by relying on self-attention neural modules that only process information at the same scale. To solve this problem, we present a novel Pyramid Attention module for image restoration, which captures long-range feature correspondences from a multi-scale feature pyramid. Inspired by the fact that corruptions, such as noise or compression artifacts, drop drastically at coarser image scales, our attention module is designed to be able to borrow clean signals from their “clean” correspondences at the coarser levels. The proposed pyramid attention module is a generic building block that can be flexibly integrated into various neural architectures. Its effectiveness is validated through extensive experiments on multiple image restoration tasks: image denoising, demosaicing, compression artifact reduction, and super resolution. Without any bells and whistles, our PANet (pyramid attention module with simple network backbones) can produce state-of-the-art results with superior accuracy and visual quality. Our code is available at https://github.com/SHI-Labs/Pyramid-Attention-Networks

16 June 2024
Verified
Conference Paper
Closed access
XXXingqian Xu
Xingqian Xu
JGJiayi Guo
Jiayi Guo
ZWZhangyang (Atlas) Wang
Zhangyang (Atlas) Wang
GHGao Huang
Gao Huang
IEIrfan Essa
Irfan Essa
HSHumphrey Shi
Humphrey Shi
Text-to-image (T2I) research has grown explosively in the past year, owing to the large-scale pre-trained diffusion models and many emerging personalization and editing approaches. Yet, one pain point persists: the text prompt engineering, and searching high-quality text prompts for customized results is more art than science. Moreover, as commonly argued: “an image is worth a thousand words” - the attempt to describe a desired image with texts often ends up being ambiguous and cannot comprehensively cover delicate visual details, hence necessitating more additional controls from the visual domain. In this paper, we take a bold step forward: taking “Text” out of a pretrained T2I diffusion model, to reduce the burdensome prompt engineering efforts for users. Our proposed frame-work, Prompt-Free Diffusion, relies on only visual inputs to generate new images: it takes a reference image as “context”, an optional image structural conditioning, and an initial noise, with absolutely no text prompt. The core architecture behind the scene is Semantic Context Encoder (SeeCoder), substituting the commonly used CLIP-based or LLM-based text encoder. The reusability of SeeCoder also makes it a convenient drop-in component: one can also pre-train a SeeCoder in one T2I model and reuse it for another. Through extensive experiments, Prompt-Free Diffusion is experimentally found to (i) outperform prior exemplar-based image synthesis approaches; (ii) perform on par with state-of-the-art T2I models using prompts following the best practice; and (iii) be naturally extensible to other downstream applications such as anime figure generation and virtual try-on, with promising quality. Our code and models will be open-sourced.

29 September 2020
Verified
Book Chapter
Closed access
PDF file
HYHanchao Yu
Hanchao Yu
XCXiao Chen
Xiao Chen
HSHumphrey Shi
Humphrey Shi
TCTerrence Chen
Terrence Chen
THThomas S. Huang
Thomas S. Huang
SSShanhui Sun
Shanhui Sun

25 October 2024
Verified
Book Chapter
Closed access
PDF file
SJSiyu Jiao
Siyu Jiao
HZHongguang Zhu
Hongguang Zhu
JHJiannan Huang
Jiannan Huang
YZYao Zhao
Yao Zhao
YWYunchao Wei
Yunchao Wei
HSHumphrey Shi
Humphrey Shi

1 December 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Open access
WCWuyang Chen
Wuyang Chen
XGXinyu Gong
Xinyu Gong
JWJunru Wu
Junru Wu
YWYunchao Wei
Yunchao Wei
HSHumphrey Shi
Humphrey Shi
ZWZhangyang (Atlas) Wang
Zhangyang (Atlas) Wang
This work targets designing a principled and unified training-free framework for Neural Architecture Search (NAS), with high performance, low cost, and in-depth interpretation. NAS has been explosively studied to automate the discovery of top-performer neural networks, but suffers from heavy resource consumption and often incurs search bias due to truncated training or approximations. Recent NAS works Mellor et al. 2021, Chen et al. 2021, Abdelfattah et al. 2021 start to explore indicators that can predict a network's performance without training. However, they either leveraged limited properties of deep networks, or the benefits of their training-free indicators were not applied to more extensive search methods. By rigorous correlation analysis, we present a unified framework to understand and accelerate NAS, by disentangling “ TEG ” characteristics of searched networks – T rainability, E xpressivity, G eneralization – all assessed in a training-free manner. The TEG indicators could be scaled up and integrated with various NAS search methods, including both supernet and single-path NAS approaches. Extensive studies validate the effective and efficient guidance from our TEG-NAS framework, leading to both improved search accuracy and over 56% reduction in search time cost. Moreover, we visualize search trajectories on three landscapes of “TEG” characteristics, observing that a good local minimum is easier to find on NAS-Bench-201 given its simple topology, whereas balancing “TEG” characteristics is much harder on the DARTS space due to its complex landscape geometry.