The most significant barrier to success in human activity recognition is extracting and selecting the right features. In traditional methods, the features are chosen by humans, which requires the user to have expert knowledge or to do a large amount of empirical study. Newly developed deep learning technology can automatically extract and select features. Among the various deep learning methods, convolutional neural networks (CNNs) have the advantages of local dependency and scale invariance and are suitable for temporal data such as accelerometer (ACC) signals. In this paper, we propose an efficient human activity recognition method, namely Iss2Image (Inertial sensor signal to Image), a novel encoding technique for transforming an inertial sensor signal into an image with minimum distortion and a CNN model for image-based activity classification. Iss2Image converts real number values from the X, Y, and Z axes into three color channels to precisely infer correlations among successive sensor signal values in three different dimensions. We experimentally evaluated our method using several well-known datasets and our own dataset collected from a smartphone and smartwatch. The proposed method shows higher accuracy than other state-of-the-art approaches on the tested datasets.

Personalized emotion recognition provides an individual training model for each target user in order to mitigate the accuracy problem when using general training models collected from multiple users. Existing personalized speech emotion recognition research has a cold-start problem that requires a large amount of emotionally-balanced data samples from the target user when creating the personalized training model. Such research is difficult to apply in real environments due to the difficulty of collecting numerous target user speech data with emotionally-balanced label samples. Therefore, we propose the Robust Personalized Emotion Recognition Framework with the Adaptive Data Boosting Algorithm to solve the cold-start problem. The proposed framework incrementally provides a customized training model for the target user by reinforcing the dataset by combining the acquired target user speech with speech from other users, followed by applying SMOTE (Synthetic Minority Over-sampling Technique)-based data augmentation. The proposed method proved to be adaptive across a small number of target user datasets and emotionally-imbalanced data environments through iterative experiments using the IEMOCAP (Interactive Emotional Dyadic Motion Capture) database.

The objective of this research is the rapid reconstruction of ancient buildings of historical importance using a single image. The key idea of our approach is to reduce the infinite solutions that might otherwise arise when recovering a 3D geometry from 2D photographs. The main outcome of our research shows that the proposed methodology can be used to reconstruct ancient monuments for use as proxies for digital effects in applications such as tourism, games, and entertainment, which do not require very accurate modeling. In this article, we consider the reconstruction of ancient Mughal architecture including the Taj Mahal. We propose a modeling pipeline that makes an easy reconstruction possible using a single photograph taken from a single view, without the need to create complex point clouds from multiple images or the use of laser scanners. First, an initial model is automatically reconstructed using locally fitted planar primitives along with their boundary polygons and the adjacency relation among parts of the polygons. This approach is faster and more accurate than creating a model from scratch because the initial reconstruction phase provides a set of structural information together with the adjacency relation, which makes it possible to estimate the approximate depth of the entire structural monument. Next, we use manual extrapolation and editing techniques with modeling software to assemble and adjust different 3D components of the model. Thus, this research opens up the opportunity for the present generation to experience remote sites of architectural and cultural importance through virtual worlds and real-time mobile applications. Variations of a recreated 3D monument to represent an amalgam of various cultures are targeted for future work.

We introduce a unified remote console with an augmented reality (AR) interface to interact with appliances in a home network environment. Computer vision techniques are used in order to implement the AR interface. With the introduced remote console, users can control multiple appliances with a single remote console in intuitive ways such as gestures. It can also be used to manage multimedia files between appliances. For example, a user can move video clips from a camcorder to a TV set without any physical action of connecting appliances, if she uses the remote console. We used an open-standard middleware, UPnP, for our implementation.

The well-known “fat finger” issue limits the interaction performance of trajectory-based finger gestures. To alleviate this issue, this work focuses on the possibility of using additional continuous auditory feedback to assist trajectory-based finger gestures. First, the experiment validated that, with the visual feedback only, the bare fingertip led to more errors in drawing of intersectional points, endpoints of closed gestures, and gestural length and shape variability compared to when the finger-attached pen was used. Then, we designed different types of auditory feedback (discrete beep, static, gradual) to provide additional information on the spatial relationship between finger-contact point and the endpoints or intersections of predefined gestures. An experiment that evaluates the effects of individual or combination of designed auditory feedback on trajectory-based finger gestures was conducted. These results show a few differences between them. However, a combination of gradual (amplitude and frequency) continuous sound and beep reached the highest drawing accuracy for trajectory-based finger gestures, which is similar to that of a finger-attached pen. This research offers insights and implications for the future design of continuous auditory feedback on small touchscreens.