The use of mobile devices, especially smartphones, has become popular in recent years. There is an increasing need for cross-device interaction techniques that seamlessly integrate mobile devices and large display devices together. This paper develops a novel cross-device cursor position system that maps a mobile device’s movement on a flat surface to a cursor’s movement on a large display. The system allows a user to directly manipulate objects on a large display device through a mobile device and supports seamless cross-device data sharing without physical distance restrictions. To achieve this, we utilize sound localization to initialize the mobile device position as the starting location of a cursor on the large screen. Then, the mobile device’s movement is detected through an accelerometer and is accordingly translated to the cursor’s movement on the large display using machine learning models. In total, 63 features and 10 classifiers were employed to construct the machine learning models for movement detection. The evaluation results have demonstrated that three classifiers, in particular, gradient boosting, linear discriminant analysis (LDA), and naïve Bayes, are suitable for detecting the movement of a mobile device.
Over the past four decades, the prophecy from computer pundits and prognosticators pointed to the looming arrival of the paperless office era. However, forty years later, physical paper documents are still playing a significant role due to the ease of use, superior readability, and availability. The drawbacks of paper sheets are that they are hard to modify and retrieve, have limited space, and are environmentally unfriendly. Augmenting paper documents with digital information from mobile devices extends the two-dimensional space of physical paper documents. Various camera-based recognition and detection devices have been proposed to augment paper documents with digital information. However, there are still some limitations that exist in these systems. This paper presents a novel, low cost, spatially aware, mobile system called Ultrasonic PhoneLens. The Ultrasonic PhoneLens adopts two-dimensional dynamic image presentation and ultrasonic sound positioning techniques. It consists of two ultrasonic sound sensors, one Arduino mini-controller board, and one android mobile device. Based on the location of the mobile device over the physical paper, Ultrasonic PhoneLens can retrieve pre-saved digital information from a mobile database for the object (such as a text, a paragraph, or an image) in a paper document. An empirical study was conducted to evaluate the system performance. The results indicate that our system has a better performance in tasks such as browsing multivalent documents and sharing digital information than the Wiimote PhoneLens system.