Sean Fanello

Sean Fanello

Auto-generated profile, based on publicly available information

Sean Fanello

Google Inc., United States of America

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 May 2020 in Computer Graphics Forum
Verified
Journal Article
Closed access
PDF file
ATA. Tewari
A. Tewari
OFO. Fried
O. Fried
JTJ. Thies
J. Thies
VSV. Sitzmann
V. Sitzmann
SLS. Lombardi
S. Lombardi
MZM. Zollhöfer
M. Zollhöfer

16 March 2022 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
XDXiaoming Deng
Xiaoming Deng
DZDexin Zuo
Dexin Zuo
YZYinda Zhang
Yinda Zhang
ZCZhaopeng Cui
Zhaopeng Cui
JCJian Cheng
Jian Cheng
HWHongan Wang
Hongan Wang
3D hand pose estimation is a challenging problem in computer vision due to the high degrees-of-freedom of hand articulated motion space and large viewpoint variation. As a consequence, similar poses observed from multiple views can be dramatically different. In order to deal with this issue, view-independent features are required to achieve state-of-the-art performance. In this paper, we investigate the impact of view-independent features on 3D hand pose estimation from a single depth image, and propose a novel recurrent neural network for 3D hand pose estimation, in which a cascaded 3D pose-guided alignment strategy is designed for view-independent feature extraction and a recurrent hand pose module is designed for modeling the dependencies among sequential aligned features for 3D hand pose estimation. In particular, our cascaded pose-guided 3D alignments are performed in 3D space in a coarse-to-fine fashion. First, hand joints are predicted and globally transformed into a canonical reference frame; Second, the palm of the hand is detected and aligned; Third, local transformations are applied to the fingers to refine the final predictions. The proposed recurrent hand pose module for aligned 3D representation can extract recurrent pose-aware features and iteratively refines the estimated hand pose. Our recurrent module could be utilized for both single-view estimation and sequence-based estimation with 3D hand pose tracking. Experiments show that our method improves the state-of-the-art by a large margin on popular benchmarks with the simple yet efficient alignment and network architectures.

20 January 2023 in arXiv
Verified
Preprint
Open access
PDF file
VTVladimir Tankovich
Vladimir Tankovich
CHChristian Häne
Christian Häne
YZYinda Zhang
Yinda Zhang
AKAdarsh Kowdle
Adarsh Kowdle
SFSean Fanello
Sean Fanello
SBSofien Bouaziz
Sofien Bouaziz
This paper presents HITNet, a novel neural network architecture for real-time stereo matching. Contrary to many recent neural network approaches that operate on a full cost volume and rely on 3D convolutions, our approach does not explicitly build a volume and instead relies on a fast multi-resolution initialization step, differentiable 2D geometric propagation and warping mechanisms to infer disparity hypotheses. To achieve a high level of accuracy, our network not only geometrically reasons about disparities but also infers slanted plane hypotheses allowing to more accurately perform geometric warping and upsampling operations. Our architecture is inherently multi-resolution allowing the propagation of information across different levels. Multiple experiments prove the effectiveness of the proposed approach at a fraction of the computation required by state-of-the-art methods. At the time of writing, HITNet ranks 1st-3rd on all the metrics published on the ETH3D website for two view stereo, ranks 1st on most of the metrics among all the end-to-end learning approaches on Middlebury-v3, ranks 1st on the popular KITTI 2012 and 2015 benchmarks among the published methods faster than 100ms.

28 November 2023
Verified
Conference Paper
Open access
YZYiqin Zhao
Yiqin Zhao
RPRohit Pandey
Rohit Pandey
YZYinda Zhang
Yinda Zhang
RDRuofei Du
Ruofei Du
FTFeitong Tan
Feitong Tan
SFSean Fanello
Sean Fanello
Mobile cameras have revolutionized content creation, allowing casual users to capture professional-looking photos. However, capturing the perfect moment can still be challenging, making post-capture editing desirable. In this work, we introduce ExShot, a mobile-oriented expression editing system that delivers high-quality, fast, and interactive editing experiences. Unlike existing methods that rely on learning expression priors, we leverage mobile photo sequences to extract expression information on demand. This design insight enables ExShot to address challenges related to diverse expressions, facial details, environment entanglement, and interactive editing. At the core lies ExprNet, a lightweight deep learning model that extracts and refines expression features. To train our model, we captured portrait images with diverse expressions, incorporating pre-processing and lighting augmentation techniques to ensure data quality. Our comprehensive evaluation results demonstrate that ExShot outperforms other editing approaches by up to 29.02% in PSNR. Ablation studies validate the effectiveness of our design choices, and user studies with 28 participants confirm the strong desire for expression editing and the superior synthesis quality of ExShot, while also identifying areas for further investigation.