Shohei Nobuhara

Shohei Nobuhara

Auto-generated profile, based on publicly available information

Shohei Nobuhara

Department of Intelligence Science and Technology, Graduate School of Informatics, Kyoto University, Kyoto, Japan

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 December 2017 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
HJHanbyul Joo
Hanbyul Joo
TSTomas Simon
Tomas Simon
XLXulong Li
Xulong Li
HLHao Liu
Hao Liu
LTLei Tan
Lei Tan
YSYaser Sheikh
Yaser Sheikh
We present an approach to capture the 3D motion of a group of people engaged in a social interaction, where inter-occlusions are frequent and functional. The Panoptic Studio is a system organized around the thesis that social interactions should be measured through the integration of perceptual analyses over a large variety of viewpoints. We present a modularized system designed around this principle, consisting of integrated structural, hardware, and software innovations. The system takes, as input, 480 synchronized video streams of multiple people engaged in social activities, and produces, as output, the labeled time-varying 3D structure of anatomical landmarks on individuals in the space. Our algorithm is designed to fuse the "weak" perceptual processes in the large number of views by progressively generating skeletal proposals from low-level appearance cues, and a framework for temporal refinement is also presented by associating body parts to reconstructed dense 3D trajectory stream. Our system and method are the first in reconstructing full body motion of more than five people engaged in social interactions without using markers. We also empirically demonstrate the impact of the number of views in achieving this goal.

1 December 2018 in Solar Energy
Verified
Journal Article
Closed access
JZJinsong Zhang
Jinsong Zhang
RVRodrigo Verschae
Rodrigo Verschae
SNShohei Nobuhara
Shohei Nobuhara
JLJean-François Lalonde
Jean-François Lalonde

20 July 2022 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
SLSang-Eun Lee
Sang-Eun Lee
KSKeisuke Shibata
Keisuke Shibata
SNSoma Nonaka
Soma Nonaka
SNShohei Nobuhara
Shohei Nobuhara
KNKo Nishino
Ko Nishino
We propose a novel camera calibration method for a room-scale multi-view imaging system. Our key idea is to leverage our articulated body movements as a calibration target. We show that a freely moving person provides trajectories of a set of oriented points ( e.g ., neck joint with spine direction) from which we can estimate the locations and poses of all cameras observing them. The method only requires the cameras to be synced and that 2D human poses are estimated in each view sequence. By elevating these 2D poses to 3D which directly provides a set of oriented 3D joints, we compute the extrinsic parameters of all cameras with a linear algorithm. We also show that this enables self-supervision of the 3D joint estimator for refinement, and the iteration of the two leads to accurate camera extrinsics and 3D pose estimates up to scale. Extensive experiments using synthetic and real data demonstrate the effectiveness and flexibility of the method. The method will serve as a useful tool to expand the utility of multi-view vision systems as it eliminates the need for cumbersome on-site calibration procedures.

22 November 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
ZCZhe Chen
Zhe Chen
SNShohei Nobuhara
Shohei Nobuhara
KNKo Nishino
Ko Nishino
We introduce a novel neural network-based BRDF model and a Bayesian framework for object inverse rendering, i.e., joint estimation of reflectance and natural illumination from a single image of an object of known geometry. The BRDF is expressed with an invertible neural network, namely, normalizing flow, which provides the expressive power of a high-dimensional representation, computational simplicity of a compact analytical model, and physical plausibility of a real-world BRDF. We extract the latent space of real-world reflectance by conditioning this model, which directly results in a strong reflectance prior. We refer to this model as the invertible neural BRDF model (iBRDF). We also devise a deep illumination prior by leveraging the structural bias of deep neural networks. By integrating this novel BRDF model and reflectance and illumination priors in a MAP estimation formulation, we show that this joint estimation can be computed efficiently with stochastic gradient descent. We experimentally validate the accuracy of the invertible neural BRDF model on a large number of measured data and demonstrate its use in object inverse rendering on a number of synthetic and real images. The results show new ways in which deep neural networks can help solve challenging radiometric inverse problems.

1 December 2016 in Methods in Oceanography
Verified
Journal Article
Closed access
RKRyo Kawahara
Ryo Kawahara
SNShohei Nobuhara
Shohei Nobuhara
TMTakashi Matsuyama
Takashi Matsuyama

26 April 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
MKMeng-Yu Jennifer Kuo
Meng-Yu Jennifer Kuo
RKRyo Kawahara
Ryo Kawahara
SNShohei Nobuhara
Shohei Nobuhara
KNKo Nishino
Ko Nishino
We introduce a novel 3D sensing method for recovering a consistent, dense 3D shape of a dynamic, non-rigid object in water. The method reconstructs a complete (or fuller) 3D surface of the target object in a canonical frame (e.g., rest shape) as it freely deforms and moves between frames by estimating underwater 3D scene flow and using it to integrate per-frame depth estimates recovered from two near-infrared observations. The reconstructed shape is refined in the course of this global non-rigid shape recovery by leveraging both geometric and radiometric constraints. We implement our method with a single camera and a light source without the orthographic assumption on either by deriving a practical calibration method that estimates the point source position with respect to the camera. Our reconstruction method also accounts for scattering by water. We prototype a video-rate imaging system and show 3D shape reconstruction results on a number of real-world static, deformable, and dynamic objects and creatures in real-world water. The results demonstrate the effectiveness of the method in recovering complete shapes of complex, non-rigid objects in water, which opens new avenues of application for underwater 3D sensing in the sub-meter range.