This paper presents a method to infer skin lesion segmentation based on multiple deep convolutional neural network (DCNN) models by employing fully connected conditional random fields (CRFs). This method is on the strength of the synergism between ensemble learning which is responsible for introducing diversity from multiple DCNN models and CRFs inference which is in charge of probabilistic inference based on random fields over dermoscopy images. Contrasting to single DCNN models, the proposed method can gain better segmentation by comprehensively utilizing the advances and performance preferences of multiple different DCNN models. In comparison with simple ensemble schemes, it can effectively and precisely refine the fuzzy lesion boundary by utilizing the information in test images to maximize label agreement between similar pixels. Further, an engineering bonus is the feasibility of parallelization for the heavy operation, predicting on multiple DCNN models. In experiments, we tested the effectiveness and robustness of the proposed method on the mainstream datasets ISIC 2017 and PH2, and the results were competitive with the state-of-art methods. we also confirmed that the proposed method can capture the local information in fuzzy dermoscopy images being able to find more accurate lesion borders with a good boost on Boundary Recall (BR) metric. Moreover, since the hyper-parameters in CRFs are explainable, it is possible to adjust them manually to reach better results case by case, being attractive in practice. This work is of value on integration between the deep learning technologies and probabilistic inference in resolving lesion segmentation, and has great potential to be applied in similar tasks.

This paper presents LesserDNN, a model that uses a set of floating-point values {-1.0, -0.5, -0.25, -0.125, -0.0625, 0.0625, 0.125, 0.25, 0.5, 1.0} as quantized weights, and a new learning algorithm for the proposed model. In previous studies on deep neural networks (DNNs) with quantized weights, because DNNs employ the gradient descent method as their learning algorithm, quantized weights were applied only during the inference stage. Due to differentiability properties, quantized weights cannot be used when the gradient descent method is applied during training. To address this issue, we devised an algorithm based on simulated annealing. Since simulated annealing has no differentiability requirements, LesserDNN can utilize quantized weights during training. With the use of quantized weights and this simulated annealing-based algorithm, the learning process becomes a combinatorial problem. The proposed algorithm was applied to train networks on MNIST handwriting datasets. The tested models were trained with the simulated annealing-based algorithm and quantized weights, achieving the same level of accuracy as gradient descent-based comparison methods. Thus, LesserDNN has a simple design and small implementation scale because backpropagation is not applied. Moreover, this model achieves a high accuracy.

Despite being heavily used in the training of deep neural networks (DNNs), multipliers are resource-intensive and insufficient in many different scenarios. Previous discoveries have revealed the superiority when activation functions, such as the sigmoid, are calculated by shift-and-add operations, although they fail to remove multiplications in training altogether. In this paper, we propose an innovative approach that can convert all multiplications in the forward and backward inferences of DNNs into shift-and-add operations. Because the model parameters and backpropagated errors of a large DNN model are typically clustered around zero, these values can be approximated by their sine values. Multiplications between the weights and error signals are transferred to multiplications of their sine values, which are replaceable with simpler operations with the help of the product to sum formula. In addition, a rectified sine activation function is utilized for further converting layer inputs into sine values. In this way, the original multiplication-intensive operations can be computed through simple add-and-shift operations. This trigonometric approximation method provides an efficient training and inference alternative for devices with insufficient hardware multipliers. Experimental results demonstrate that this method is able to obtain a performance close to that of classical training algorithms. The approach we propose sheds new light on future hardware customization research for machine learning.