Augmented Reality (AR) has proposed several types of interaction techniques such as 3D interactions, natural interactions, tangible interactions, spatial awareness interactions and multimodal interactions. Usually, interaction technique in AR involve unimodal interaction technique that only allows user to interact with AR content by using one modality such as gesture, speech, click, etc. Meanwhile, the combination of more than one modality is called multimodal. Multimodal can contribute to human and computer interaction more efficient and will enhance better user experience. This is because, there are a lot of issues have been found when user use unimodal interaction technique in AR environment such as fat fingers. Recent research has shown that multimodal interface (MMI) has been explored in AR environment and has been applied in various domain. This paper presents an empirical study of some of the key aspects and issues in multimodal interaction augmented reality, touching on the interaction technique and system framework. We reviewed the question of what are the interaction techniques that have been used to perform a multimodal interaction in AR environment and what are the integrated components applied in multimodal interaction AR frameworks. These two questions were used to be analysed in order to find the trends in multimodal field as a main contribution of this paper. We found that gesture, speech and touch are frequently used to manipulate virtual object. Most of the integrated component in MMI AR framework discussed only on the concept of the framework components or the information centred design between the components. Finally, we conclude this paper by providing ideas for future work involving this field.

In many academic fields, predicting student academic success using data mining techniques has long been a major research issue. Monitoring students in higher education institutions (HEIs) and having the ability to predict student performance is important to improve academic quality. The objective of the study is to (1) identify features that form clusters that have holistic characteristics and (2) develop and validate a prediction model for each of the clusters to predict student performance holistically. For this study, both classification and clustering methods will be used using Support Vector Machine (SVM) and K-means clustering. Three clusters were identified using K-means clustering. Based on the learning program outcome feature, there are primarily three types of students: low, average, and high performance. The prediction model with the new labels obtained from the clusters also gained higher accuracy when compared to the student dataset with labels using their semester grade.

Augmented reality is one of the technologies that will impact teaching and learning. When augmented reality is applied in an educational setting, it can provide a superior learning environment because educators can add specific virtual information to enrich the existing learning materials or to create new learning materials that integrate the environment. Therefore, discussions on augmented reality applications for learning can be vastly found in the literature. Each application utilizes the resources available by using approaches appropriate for the learning outcome in question. The key problem with achieving learning outcome in analysing group one elements in a periodic table, is that some students have difficulties in visualizing an atom and what more to relate them with reactivity. The objective of this paper is to design and develop a learning application that supports science, technology, engineering and mathematics (STEM) education based on guided discovery learning to assist the students through questions based on exposure to various evidences via augmented reality. This application incorporates the use of 3D model of atoms and video of real experiments. To achieve the objective, action research is used to solve existing problem in a selected school. The application was evaluated by 25 students aged 16 years in a secondary school in Malaysia and has proven to be effective in improving the students’ marks in tests conducted at the end of the application usage. The students can get many benefits from using the augmented reality based application because it supports Education 4.0 like flexibility in teaching and learning where learning is freed for the limitation of time, place and pace of study.

The aim of this paper is to evaluate user satisfaction based on System Usability Scale (SUS) questionnaire on Augmented Reality (AR) application for productive vocabulary using speech recognition. There is still lack of research focus on user satisfaction towards the use of AR-based app using speech recognition for vocabulary learning in early education. The first objective of this paper is to develop an AR application for children using speech recognition to enhance productive vocabulary learning that integrates visual script (orthography) and audio (phonology). The second objective is to evaluate and identify the user satisfaction in enhancing productive vocabulary methods by combining AR technology and speech recognition to the parents and teachers by doing a mixed method testing such as questionnaire and interview. To achieve this, an interview session was conducted with the experts and SUS questionnaire was given to the teachers and parents of the student to evaluate user satisfaction. The results show that the research hypotheses of this study were successfully achieved. It revealed that the teachers and students were satisfied with the application based on SUS score (SUS score > 68). Since the total SUS score is 80.3, which is above average, it shows that they were satisfied with the overall application. Apart from that, the mean of the usability study based on questionnaire also shows that the users have found that this application is usable in terms of learnability study and that this application is easy to use.

The power of mobile devices has been harnessed as a platform for augmented reality (AR) that has embarked on opportunities to combine more inputs called multimodal inputs. There has been little research into the multimodal inputs that combine emotion, speech and markers in a mobile AR learning environment. This study aims to propose a framework for a mobile AR learning system that incorporates the combination of multimodal inputs, namely emotion, image-based marker and speech in order to determine how such a combination can enhance the learning experience. The proposed framework integrates the multimodal inputs based on the decision tree and develops into a four-phase learning system based on Kolb’s experiential learning model. To evaluate this learning system, 38 students were selected and divided into two groups for an experiment on vocabulary at a primary school. Quantitative findings showed better results concerning learning effectiveness, mental load, engagement, competency and challenge when the three multimodal inputs, speech, marker and emotion, were combined. Therefore, the proposed multimodal framework can be used as a guideline to develop a multimodal based AR application for learning environment by integrating multimodal inputs and Kolb’s experiential learning model.