Takayuki Shinohara

Mr. Takayuki Shinohara

Tokyo Institute of Technology, Tokyo, Japan
orcid

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
24 June 2020 in Sensors
Verified
Journal Article
Open access
PDF file
TSTakayuki Shinohara
Takayuki Shinohara
HXHaoyi Xiu
Haoyi Xiu
MMMasashi Matsuoka
Masashi Matsuoka
In the computer vision field, many 3D deep learning models that directly manage 3D point clouds (proposed after PointNet) have been published. Moreover, deep learning-based-techniques have demonstrated state-of-the-art performance for supervised learning tasks on 3D point cloud data, such as classification and segmentation tasks for open datasets in competitions. Furthermore, many researchers have attempted to apply these deep learning-based techniques to 3D point clouds observed by aerial laser scanners (ALSs). However, most of these studies were developed for 3D point clouds without radiometric information. In this paper, we investigate the possibility of using a deep learning method to solve the semantic segmentation task of airborne full-waveform light detection and ranging (lidar) data that consists of geometric information and radiometric waveform data. Thus, we propose a data-driven semantic segmentation model called the full-waveform network (FWNet), which handles the waveform of full-waveform lidar data without any conversion process, such as projection onto a 2D grid or calculating handcrafted features. Our FWNet is based on a PointNet-based architecture, which can extract the local and global features of each input waveform data, along with its corresponding geographical coordinates. Subsequently, the classifier consists of 1D convolutional operational layers, which predict the class vector corresponding to the input waveform from the extracted local and global features. Our trained FWNet achieved higher scores in its recall, precision, and F1 score for unseen test data—higher scores than those of previously proposed methods in full-waveform lidar data analysis domain. Specifically, our FWNet achieved a mean recall of 0.73, a mean precision of 0.81, and a mean F1 score of 0.76. We further performed an ablation study, that is assessing the effectiveness of our proposed method, of the above-mentioned metric. Moreover, we investigated the effectiveness of our PointNet based local and global feature extraction method using the visualization of the feature vector. In this way, we have shown that our network for local and global feature extraction allows training with semantic segmentation without requiring expert knowledge on full-waveform lidar data or translation into 2D images or voxels.

11 December 2020 in Remote Sensing
Verified
Journal Article
Open access
PDF file
HXHaoyi Xiu
Haoyi Xiu
TSTakayuki Shinohara
Takayuki Shinohara
MMMasashi Matsuoka
Masashi Matsuoka
MIMunenari Inoguchi
Munenari Inoguchi
KKKen Kawabe
Ken Kawabe
KHKei Horie
Kei Horie
Collapsed buildings should be detected with the highest priority during earthquake emergency response, due to the associated fatality rates. Although deep learning-based damage detection using vertical aerial images can achieve high performance, as depth information cannot be obtained, it is difficult to detect collapsed buildings when their roofs are not heavily damaged. Airborne LiDAR can efficiently obtain the 3D geometries of buildings (in the form of point clouds) and thus has greater potential to detect various collapsed buildings. However, there have been few previous studies on deep learning-based damage detection using point cloud data, due to a lack of large-scale datasets. Therefore, in this paper, we aim to develop a dataset tailored to point cloud-based building damage detection, in order to investigate the potential of point cloud data in collapsed building detection. Two types of building data are created: building roof and building patch, which contains the building and its surroundings. Comprehensive experiments are conducted under various data availability scenarios (pre–post-building patch, post-building roof, and post-building patch) with varying reference data. The pre–post scenario tries to detect damage using pre-event and post-event data, whereas post-building patch and roof only use post-event data. Damage detection is implemented using both basic and modern 3D point cloud-based deep learning algorithms. To adapt a single-input network, which can only accept one building’s data for a prediction, to the pre–post (double-input) scenario, a general extension framework is proposed. Moreover, a simple visual explanation method is proposed, in order to conduct sensitivity analyses for validating the reliability of model decisions under the post-only scenario. Finally, the generalization ability of the proposed approach is tested using buildings with different architectural styles acquired by a distinct sensor. The results show that point cloud-based methods can achieve high accuracy and are robust under training data reduction. The sensitivity analysis reveals that the trained models are able to locate roof deformations precisely, but have difficulty recognizing global damage, such as that relating to the roof inclination. Additionally, it is revealed that the model decisions are overly dependent on debris-like objects when surroundings information is available, which leads to misclassifications. By training on the developed dataset, the model can achieve moderate accuracy on another dataset with different architectural styles without additional training.

1 December 2019
Verified
Conference Paper
Closed access
HXHaoyi Xiu
Haoyi Xiu
TSTakayuki Shinohara
Takayuki Shinohara
MMMasashi Matsuoka
Masashi Matsuoka
Semantic segmentation of 3D point cloud data is a crucial step for various applications such as autonomous driving. Conventional methods for semantic segmentation of point cloud heavily rely on numerous hand-crafted features often derived from the 3D covariance matrix, which are typically time-consuming. In this work, a dynamic scale graph convolutional neural network is proposed to perform semantic segmentation on 3D point cloud without relying on extensive exploitation of hand-crafted features. Using only spatial coordinates, backscattered intensity and spectral information derived from aerial images, our proposed method aims at modeling multi-scale local structural information by combining dynamic scale sampling and multi-scale neighbor graphs. Sampling and neighbor graph construction are both implemented on the fly such that no preprocessing or data augmentation is needed before training. Evaluated using the ISPRS 3D Semantic Labeling Contest, our approach has achieved the performance on par with the current state-of-the-art in terms of average f1-score without hand-crafted features and less computational overhead.

2 November 2021 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Verified
Journal Article
Open access
TSTakayuki Shinohara
Takayuki Shinohara
HXHaoyi Xiu
Haoyi Xiu
MMMasashi Matsuoka
Masashi Matsuoka
Since 2017, many deep learning methods for 3D point clouds observed by airborne LiDAR (airborne 3D point clouds) have been proposed. Moreover, not only a deep learning method for airborne 3D point clouds but also a deep learning method for points and their waveforms observed by full-waveform LiDAR (airborne FW data) was proposed. We need to achieve highly accurate land cover classification by using airborne FW data, but open data often only have airborne 3D point clouds available. Therefore, to improve the performance of land cover classification when using airborne 3D point clouds published as open data, it is important to restore waveforms from airborne 3D point clouds. In this paper, we propose a deep learning model to translate an airborne 3D point cloud to airborne FW data (called a point-to-waveform translation model, point2wave) using a conditional generative adversarial net (cGAN). Our point2wave is a cGAN pipeline consisting of a generator that translates the waveform corresponding to each point from the input airborne 3D Point Cloud and discriminators that calculate the distance between the translated waveform and the ground truth waveform. Using a set of point clouds and waveforms dataset, we have experimented to translate points into the waveforms by point2wave. Experimental results showed that point2wave could translate waveforms from the airborne 3D point cloud and the translated fake waveforms achieved nearly the same land cover classification performance as the real waveforms.

1 December 2019
Verified
Conference Paper
Closed access
TSTakayuki Shinohara
Takayuki Shinohara
HXHaoyi Xiu
Haoyi Xiu
MMMasashi Matsuoka
Masashi Matsuoka
Deep learning methods that directly handle three-dimensional point clouds, e.g., PointNet, have recently been proposed. Moreover, deep learning-based-techniques have demonstrated excellent performance for supervised learning tasks on point clouds such as classification and segmentation for certain open datasets. In this study, the possibility of using a deep learning method, which is an unsupervised representation learning method to extract features from airborne raw full-waveform data, is investigated. Thus, a novel end-to-end autoencoder network called FWNetAE is proposed to address representation learning challenges on raw full-waveform data. On the encoder side, a PointNet-based network extracts the latent vector of raw full-waveform data. Subsequently, a fully connected network-based decoder deforms the latent vector into the input full-waveform light detection and ranging (LiDAR) data, thus achieving fewer reconstruction errors.

3 November 2020
Verified
Conference Paper
Open access
TSTakayuki Shinohara
Takayuki Shinohara
HXHaoyi Xiu
Haoyi Xiu
MMMasashi Matsuoka
Masashi Matsuoka
During the last few years, in the field of computer vision, sophisticated deep learning methods have been developed to accomplish semantic segmentation tasks of 3D point cloud data. Additionally, many researchers have extended the applicability of these methods, such as PointNet or PointNet++, beyond semantic segmentation tasks of indoor scene data to large-scale outdoor scene data observed using airborne laser scanning systems equipped with light detection and ranging (LiDAR) technology. Most extant studies have only investigated geometric information (x, y, and z or longitude, latitude, and height) and have omitted rich radiometric information. Therefore, we aim to extend the applicability of deep learning-based model from the geometric data into radiometric data acquired with airborne full-waveform LiDAR without converting the waveform into 2D images or 3D voxels. We simultaneously train two models: a local module for local feature extraction and a global module for acquiring wide receptive fields for the waveform. Furthermore, our proposed model is based on waveform-aware convolutional techniques. We evaluate the effectiveness of the proposed method using benchmark large-scale outdoor scene data. By integrating the two outputs from the local module and the global module, our proposed model had achieved higher mean recall value 0.92 than previous methods and higher F1 scores for all six classes than the other 3D Deep Learning method. Therefore, our proposed network consisting of the local and global module successfully resolves the semantic segmentation task of full-waveform LiDAR data without requiring expert knowledge.