Sketching is a natural and intuitive communication tool used for expressing concepts or ideas which are difficult to communicate through text or speech alone. Sketching is therefore used for a variety of purposes, from the expression of ideas on two-dimensional (2D) physical media, to object creation, manipulation, or deformation in three-dimensional (3D) immersive environments. This variety in sketching activities brings about a range of technologies which, while having similar scope, namely that of recording and interpreting the sketch gesture to effect some interaction, adopt different interpretation approaches according to the environment in which the sketch is drawn. In fields such as product design, sketches are drawn at various stages of the design process, and therefore, designers would benefit from sketch interpretation technologies which support these differing interactions. However, research typically focuses on one aspect of sketch interpretation and modeling such that literature on available technologies is fragmented and dispersed. In this paper, we bring together the relevant literature describing technologies which can support the product design industry, namely technologies which support the interpretation of sketches drawn on 2D media, sketch-based search interactions, as well as sketch gestures drawn in 3D media. This paper, therefore, gives a holistic view of the algorithmic support that can be provided in the design process. In so doing, we highlight the research gaps and future research directions required to provide full sketch-based interaction support.

This work is funded by the University of Malta, under the research grant SCERP02-03.Vectorization algorithms described in the literature assume that the drawings being vectorized are either binary images or have a clear white background. Sketches of artistic objects however also contain shadows which help the artist to portray intent, particularly in potentially ambiguous sketches. Such sketches are difficult to binarise since the shading strokes make these sketches non bimodal. For this reason, we describe a circle-based vectorization algorithm that uses signatures obtained from sample points on the line strokes to identify and vectorize the line strokes in the sketch. We show that the proposed algorithm performs as well as other vectorization techniques described in the literature, despite the shadows present in the sketch.peer-reviewe

Retrieving text embedded within images is a challenging task in real-world settings. Multiple problems such as low-resolution and the orientation of the text can hinder the extraction of information. These problems are common in environments such as Tor Darknet and Child Sexual Abuse images, where text extraction is crucial in the prevention of illegal activities. In this work, we evaluate eight text recognizers and, to increase the performance of text transcription, we combine these recognizers with rectification networks and super-resolution algorithms. We test our approach on four state-of-the-art and two custom datasets (TOICO-1K and Child Sexual Abuse (CSA)-text, based on text retrieved from Tor Darknet and Child Sexual Exploitation Material, respectively). We obtained a 0.3170 score of correctly recognized words in the TOICO-1K dataset when we combined Deep Convolutional Neural Networks (CNN) and rectification-based recognizers. For the CSA-text dataset, applying resolution enhancements achieved a final score of 0.6960. The highest performance increase was achieved on the ICDAR 2015 dataset, with an improvement of 4.83% when combining the MORAN recognizer and the Residual Dense resolution approach. We conclude that rectification outperforms super-resolution when applied separately, while their combination achieves the best average improvements in the chosen datasets.

In this paper we introduce a path extraction algorithm for multi-stroke scribbled paths by making use of path-centred concentric sampling circles. Circle and line geometry is then exploited to efficiently obtain piece-wise linear models of the multi-stroke segments in the drawing. Parzen window estimation is used to obtain the probability distribution of the grey-level profile of the sampling circles to determine the intersecting angle of the sampling circle with the stroke segments and hence determine the line model parameters. The results obtained show that the algorithm identifies the line models accurately while reducing considerably the computational time required to obtain the line models.