The lack of information provided by line arts makes user guidedcolorization a challenging task for computer vision. Recent contributions from the deep learning community based on Generative Adversarial Network (GAN) have shown incredible results compared to previous techniques. These methods employ user input color hints as a way to condition the network. The current state of the art has shown the ability to generalize and generate realistic and precise colorization by introducing a custom dataset and a new model with its training pipeline. Nevertheless, their approach relies on randomly sampled pixels as color hints for training. Thus, in this contribution, we introduce a stroke simulation based approach for hint generation, making the model more robust to messy inputs. We also propose a new cleaner dataset, and explore the use of a double generator GAN to improve visual fidelity.
4 July 2022 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Ubiquitous touch sensing surfaces are largely influenced by touchscreens' look and feel and fail to express the physical richness of existing surrounding materials. We introduce Sensurfaces, a plug-and-play electronic module that allows to rapidly experiment with touch-sensitive surfaces while preserving the original appearance of materials. Sensurfaces is composed of plug-and-play modules that can be connected together to expand the size and number of materials composing a sensitive surface. The combination of Sensurfaces modules allows the creation of small or large multi-material sensitive surfaces that can detect multi-touch but also body proximity, pose, pass, or even human steps. In this paper, we present the design and implementation of Sensurfaces. We propose a design space describing the factors of Sensurfaces interfaces. Then, through a series of technical evaluations, we demonstrate the capabilities of our system. Finally, we report on two workshops validating the usability of our system.
13 July 2016 in International Journal of Wireless Information Networks
Handwriting is a complex motor activity. Neurodevelopmental disorders, such as dysgraphia can lead to severe handwriting difficulties. These dysfunctions could be predicted by measuring the pen’s motion, the pressure of the tip and the grip force while handwriting. Various hardware platforms have been proposed to sense pen movement. However, the grip force sensing is more challenging and often requires high-end sensing devices. In this paper, we propose a low-cost pen grip sensing tool to detect pressures and positions of fingers on the pen. It consists of a flexible, mutual-capacitive sensor rolled around the pen and connected to an inexpensive microcontroller. We provide initial tests of our grip sensing tool on multiple single-touch conditions. The results show a homogeneity over the flexible sensor and a linearity of the response with several curvatures. Such novel device open new opportunities for neurodevelopemental disorders research and new possibilities for clinical purposes.
In a world of rich, complex, and demanding audio environments, intelligent systems can mediate our interaction with the sounds around us - both to enable meaningful, aesthetic experiences and to transition work from humans to computational agents. Drawing from several years of our research, we suggest that the design of such systems must be driven by a deep understanding of auditory cognition. In this article, we discuss two concrete approaches we take toward cognition-informed interface design - one that begins with sounds themselves to form explicit, contextualized, cognitive models, built on the foundations of large-data parsing infrastructure; and one that begins with the individual, built from intuition surrounding the influence of the cognitive state on perception. We point toward an unexplored and compelling future at their intersection.
Personal assistants are becoming more pervasive in our envi-ronments but still do not provide natural interactions. Their lack of realism in term of expressiveness and their lack of visual feedback can create frustrating experiences and make users lose patience. In this sense, we propose an end-to-end trainable neural architecture for text-driven 3D mouth animations. Previous works showed such architectures provide better realism and could open the door for integrated affective Human Computer Interface (HCI). Our study shows that such visual feedback improves users’ comfort for 78%of the candidates significantly while slightly improving their time perception.