The rapid growth of AR platforms, combined with the rising predictive power of intelligent systems, will fundamentally change interactive computing. Interaction will increasingly happen on the go, causing I/O to become constrained, ultimately leading to reliance on user intent prediction for aid. In this pictorial, we argue that to support the development of such systems, new predictive UI toolkits are required. We place the reader in the shoes of an App designer and outline the challenges that will be faced. We then describe a new predictive toolkit, leveraging Fuzzy Widgets, or “Fidgets” as the main UI building block. Fidgets extend Responsive Design into the realm of intelligent systems, to adapt not only to spatial constraints, but to system predictions as well. We then describe a working implementation of a predictive music application, built using our described framework, showcasing its benefits and range of adaptive abilities.
Advances in artificial intelligence open the possibility of predicting and recommending sequences of GUI commands to a user. An interesting question raised by this capability is how to present such recommendations to the user – as a sequential set of individual command recommendations, or as one aggregated recommendation consisting of multiple commands. In this paper we propose an interface for aggregated command recommendation and conduct controlled studies to compare sequential versus aggregated command recommendation across a range of simulated utility conditions. Our results indicate that aggregated command recommendation can improve overall task performance over sequential recommendation, and that this benefit comes from enabling users to rapidly recognize and use high-utility aggregated recommendations. The aggregated command recommendation approach also reduced deliberation time when evaluating and correcting imperfect sets of recommended commands.