We implement high-quality sketch colorization using two-stage conditional generative adversarial network (GAN) training based on different intermediate features. The intermediate features used in autonomous colorization are the grayscale parsing and interval pixel-level color parsing. The autonomous colorization based on grayscale parsing feature can learn the spacial topology of pixels in the first stage to guide the colorization in the second stage. The autonomous colorization based on pixel-level color parsing feature can learn the color information of few feature points in the first stage to guide the colorization of all pixels in the second stage. Additionally, we use the intermediate feature of sampling points as constraint and achieve the color reconstruction using Laplacian mesh editing as a special second stage. Furthermore, the interactive colorization uses the superpixel color parsing as the intermediate feature. Specifically, we use the simple linear iterative cluster (SLIC) to obtain a palette that maintains the edges in the first stage to guide the colorization in the second stage. As for evaluation metrics, we propose a colorcoded local binary pattern (CCLBP) score based on color distances from the first-order 8 pixels to the central pixel, to measure the degrees of color blurring and mess. We also propose a light-sensitivity (LS) score based on the reversed grayscale map, to measure the degrees of auto painting and overfitting of the color hint. According to the L1 distances between the original and generated color images based on these scores, compared with state-of-the-art methods including one stage approaches such as pix2pix and PaintsChainer and two-stage approaches such as Style2Paints and DeepColor, our model can achieve the highest-quality autonomous colorization. Moreover, compared with pix2pix, PaintsChainer and Style2Paints with color hints, according to the proposed objective evaluation as well as the user visual study, our model can achieve the highest-quality interactive colorization as well.

Images are an important carrier for emotional expression. Human can understand emotions in image easily and quickly, whereas it is a very challenging task for machines to extract accurate emotions. In this study, we propose a novel spatial and channel-wise attention-based emotion prediction model, SCEP, to assist computers in recognizing the emotions of images more accurately. SCEP integrates both spatial attention and channel-wise weight mechanisms into a classical convolutional neural network (CNN) layer structure to predict image emotions, on the grounds that the spatial attention mechanism can enhance the contrast between salient regions and potentially irrelevant regions, and that the channel-wise weight mechanism can emphasize informative features while suppressing less useful features. The SCEP model outputs emotion values in a continuous 2-D valence and arousal space, so that more emotions can be expressed than by simply discretely classifying emotions. To validate the effectiveness of our model, we use an existing image dataset with a widespread emotion distribution for testing. Extensive experiments show that when compared to base models (i.e. VGG and ResNet) without spatial attention or channel-wise mechanisms, SCEP can improve the accuracy of emotion prediction (evaluated by concordance correlation coefficient) by ~ 3%-5% in the arousal domain, and by ~ 3-6% in the valence domain. Therefore, we conclude that using SCEP can bring higher accuracy in emotion prediction.

This article investigated the timbre of musical sound and visual color; comprehensively used experimental psychology, audio information processing and other experimental methods and technical means; and conducted research on the relationship between timbre attributes and visual color. First, we designed and implemented a subjective perception experiment based on audiovisual cross-modal timbre-color correlation. Through the statistical analysis of the experimental data, we obtained a complete timbre-color cross-modal correlation dataset. Second, through the visual processing and correlation analysis of the timbre-color correlation data, it is proven that there is a certain correlation between the timbre dimension and the color dimension. On this basis, we used three algorithms to construct a timbre-color correlation model, namely, multiple linear regression, BP neural network and SVR, and we verified the accuracy of the three models. The timbre-color correlation dataset constructed in this paper can provide basic data support for audiovisual cross-modal research. The timbre-color correlation model constructed in this paper can provide a theoretical basis for cross-modal audiovisual applications. In addition, the timbre-color cross-modality research method in this paper can provide new research ideas for audio-visual cross-modality research.

The effect of different regions of contrast on visual comfort through a subjective evaluation experiment with stereoscopic images generated by a computer is investigated. The image sequences varied across the factors of disparity magnitude, disparity type, region contrast and in the luminance relationship between the main part of the image and the background. Experimental results indicated that the region contrast would significantly affect the feeling of visual comfort under different combinations of disparity magnitude and disparity type. When the disparity magnitude of the main body was too large that may cause obvious convergence–accommodation conflict, decreasing the region contrast would effectively reduce the discomfort.

A multi-swarm-evolutionary structure based on the parasitic relationship in the biosphere is proposed in this paper and, according to the conception, the Para-PSO-ABC algorithm (ParaPA), combined with merits of the modified particle swarm optimization (MPSO) and artificial bee colony algorithm (ABC), is conducted with the multimodal routing strategy to enhance the safety and the cost issue for the mobile robot path planning problem. The evolution is divided into three stages, where the first is the independent evolutionary stage, with the same evolution strategies for each swarm. The second is the fusion stage, in which individuals are evolved hierarchically in the parasitism structure. Finally, in the interaction stage, a multi-swarm-elite strategy is used to filter the information through a predefined cross function among swarms. Meanwhile, the segment obstacle-avoiding strategy is proposed to accelerate the searching speed with two fitness functions. The best path is selected according to the performance on the safety and consumption issues. The introduced algorithm is examined with different obstacle allocations and simulated in the real routing environment compared with some typical algorithms. The results verify the productiveness of the parasitism-relation-based structure and the stage-based evolution strategy in path planning.

Control system for stage lighting is mainly composed of lighting console, dimmer and auxiliary equipment. At present, the functional design of lighting control console is mostly based on MCU and lighting dimmer, which uses SCR (silicon controlled rectifier) to control lighting. There are lots of defects in advanced technique, information storage, expansibility and real-time performance. Also the dimmer based on SCR could engender a lot of harmonics. So the power system and equipment will be influenced. In this paper, lighting console based on DMX512 protocol using ARM chip and new sine wave lighting dimmer based on AC chopping control with IGBT are put forward. The experimental results show that this system has perfect function, strong expansibility, easily upgrade and meets the needs of the stage lighting control system. Our work has some theoretical significance and application value for the development of stage lighting control system.