This paper concerns the problem of facial landmark detection. We provide a unique new analysis of the features produced at intermediate layers of a convolutional neural network (CNN) trained to regress facial landmark coordinates. This analysis shows that while being processed by the CNN, face images can be partitioned in an unsupervised manner into subsets containing faces in similar poses (i.e., 3D views) and facial properties (e.g., presence or absence of eye-wear). Based on this finding, we describe a novel CNN architecture, specialized to regress the facial landmark coordinates of faces in specific poses and appearances. To address the shortage of training data, particularly in extreme profile poses, we additionally present data augmentation techniques designed to provide sufficient training examples for each of these specialized sub-networks. The proposed Tweaked CNN (TCNN) architecture is shown to outperform existing landmark detection methods in an extensive battery of tests on the AFW, ALFW, and 300W benchmarks. Finally, to promote reproducibility of our results, we make code and trained models publicly available through our project webpage.

We propose a method designed to push the frontiers of unconstrained face recognition in the wild with an emphasis on extreme out-of-plane pose variations. Existing methods either expect a single model to learn pose invariance by training on massive amounts of data or else normalize images by aligning faces to a single frontal pose. Contrary to these, our method is designed to explicitly tackle pose variations. Our proposed Pose-Aware Models (PAM) process a face image using several pose-specific, deep convolutional neural networks (CNN). 3D rendering is used to synthesize multiple face poses from input images to both train these models and to provide additional robustness to pose variations at test time. Our paper presents an extensive analysis on the IJB-A benchmark, evaluating the effects that landmark detection accuracy, CNN layer selection, and pose model selection all have on the performance of the recognition pipeline. It further provides comparative evaluations on the IARPA Janus Benchmarks A (IJB-A) and the PIPA dataset. These tests show that our approach outperforms existing methods, even surprisingly matching the accuracy of methods that were specifically fine-tuned to the target dataset. Parts of this work previously appeared in [1] and [2].

The greenhouse industry achieves stable agricultural production worldwide. Various information and communication technology techniques to model and control the environment have been applied as data from environmental sensors and actuators in greenhouses are monitored in real time. The current study designed data-based, deep learning models for evapotranspiration (ET) and humidity in tomato greenhouses. Using time-series data and applying long short-term memory (LSTM) modeling, an ET prediction model was developed and validated in comparison with the Stanghellini model. Training with 20-day and testing with 3-day data resulted in RMSEs of 0.00317 and 0.00356 kgm⁻² s⁻¹, respectively. The standard error of prediction indicated errors of 5.76 and 6.45% in training and testing, respectively. Variables were used to produce a feature map using a two-dimensional convolution layer which was transferred to a subsequent layer and finally connected with the LSTM structure for modeling. The RMSE in humidity prediction using the test dataset was 2.87, indicating a performance better than conventional RNN-LSTM models. Irrigation plans and humidity control may be more accurately conducted in greenhouse cultivation using this model.

Soft sensors are attracting significant attention in human–machine interaction due to their high flexibility and adaptability. However, estimating motion state from these sensors is difficult due to their nonlinearity and noise. In this paper, we propose a deep learning network for a smart glove system to predict the moving state of a piezoelectric soft sensor. We implemented the network using Long-Short Term Memory (LSTM) units and demonstrated its performance in a real-time system based on two experiments. The sensor’s moving state was estimated and the joint angles were calculated. Since we use moving state in the sensor offset calculation and the offset value is used to estimate the angle value, the accurate moving state estimation results in good performance for angle value estimation. The proposed network performed better than the conventional heuristic method in estimating the moving state. It was also confirmed that the calculated values successfully mimic the joint angles measured using a leap motion controller.

This paper presents a calibration method for a robot Head-Eye system which can be used in a humanoid robot vision system. This method estimates the transformation between the robot coordinate and the camera coordinate. The calibration procedure uses visual measurements and kinematic information as the inputs of nonlinear optimization. For this calibration method, an objective function for the optimization, that uses the Minimum Variance method is defined. The procedure of this method is very simple and intuitive. Besides, the result of this method is considerably precise and robust against the noisy environment. The performance is compared with earlier approaches and the results of simulation and actual experiment are followed.