Augmented Reality (AR) has proposed several types of interaction techniques such as 3D interactions, natural interactions, tangible interactions, spatial awareness interactions and multimodal interactions. Usually, interaction technique in AR involve unimodal interaction technique that only allows user to interact with AR content by using one modality such as gesture, speech, click, etc. Meanwhile, the combination of more than one modality is called multimodal. Multimodal can contribute to human and computer interaction more efficient and will enhance better user experience. This is because, there are a lot of issues have been found when user use unimodal interaction technique in AR environment such as fat fingers. Recent research has shown that multimodal interface (MMI) has been explored in AR environment and has been applied in various domain. This paper presents an empirical study of some of the key aspects and issues in multimodal interaction augmented reality, touching on the interaction technique and system framework. We reviewed the question of what are the interaction techniques that have been used to perform a multimodal interaction in AR environment and what are the integrated components applied in multimodal interaction AR frameworks. These two questions were used to be analysed in order to find the trends in multimodal field as a main contribution of this paper. We found that gesture, speech and touch are frequently used to manipulate virtual object. Most of the integrated component in MMI AR framework discussed only on the concept of the framework components or the information centred design between the components. Finally, we conclude this paper by providing ideas for future work involving this field.

These recent years have witnessed the importance of indoor localization and tracking as people are spending more time indoors, which facilitates determining the location of an object. Indoor localization enables accurate and reliable location-based services and navigation within buildings, where GPS signals are often weak or unavailable. With the rapid progress of smartphones and their growing usage, smartphone-based positioning systems are applied in multiple applications. The smartphone is embedded with an inertial measurement unit (IMU) that consists of various sensors to determine the walking pattern of the user and form a pedestrian dead reckoning (PDR) algorithm for indoor navigation. As such, this study reviewed the literature on indoor localization based on smartphones. Articles published from 2015 to 2022 were retrieved from four databases: Science Direct, Web of Science (WOS), IEEE Xplore, and Scopus. In total, 109 articles were reviewed from the 4186 identified based on inclusion and exclusion criteria. This study unveiled the technology and methods utilized to develop indoor localization systems. Analyses on sample size, walking patterns, phone poses, and sensor types reported in previous studies are disclosed in this study. Next, academic challenges, motivations, and recommendations for future research endeavors are discussed. Essentially, this systematic literature review (SLR) highlights the present research overview. The gaps identified from the SLR may assist future researchers in planning their research work to bridge those gaps.

Given the wide range of sensor applications, pedestrian activity recognition research using smartphone sensors has gained significant attention. Recognizing activities can yield valuable insights into a person’s actions and the context of the activities. This study proposed a bidirectional long short-term memory based on the feature fusion model with a convolutional neural network (BiLSTM-BFF with CNN) to integrate time and frequency domain features and CNN. The fused feature vector was used as input in the BiLSTM network. The BiLSTM-BFF with CNN model recognized 14 types of pedestrian activity. New pedestrian activity datasets were collected from smartphone sensors used by different types of people (men, women, children, people with limps, pregnant women) and activities (walking, fast walking, elevator up and down, step escalator up and down, walking with step escalator up and down, flat escalator up and down, walking with flat escalator up and down, upstairs and downstairs). The efficiency of the proposed BiLSTM-BFF with the CNN model was validated by conducting experiments using this new dataset. The proposed method demonstrated 95.35% accuracy in recognizing pedestrian activities. The results highlighted the superior accuracy of the proposed method compared to other methods.

Conventionally, the interaction between a user and augmented reality (AR) application is limited. Mostly, it allows virtual information browsing on the AR marker or basic manipulation such as moving and resizing of the 3D model. This study presents a rich interaction that allows users to mimic physical action such as shaking and pouring in chemistry experiment using card-based or box-based marker design. The user is required to follow the instruction to select the correct AR marker that represents apparatus or solutions and performs the physical action to conduct an experiment. Usability’s result showed the respondents were satisfied with the application.