Given normal training samples, anomaly detection in videos can be regarded as a challenging problem of identifying unexpected events. The state-of-the-art approaches generally resort to the autoencoder model by using a single encoder to capture the motion and content patterns jointly. Nevertheless, due to the lack of accurate labels of normal and abnormal samples, how to detect anomalies is decided by the subjective understanding of models. It infers that different models will prefer to mine different patterns according to the characteristics of models. We call this problem as a pattern bias problem. To alleviate this problem, a novel Multi-Encoder Single-Decoder network, termed as MESDnet, is proposed in the spirit of encoding motion and content cues individually with multiple encoders. MESDnet is of end-to-end learning ability and real-time running speed. Particularly, the differences between adjacent frames and the raw frames are used as the motion and content sources, respectively. Then, a decoder takes charge of detecting anomalies in the way of observing reconstructing error towards the video frames by using the multi-stream encoded motion and content features simultaneously. The experiments on the CUHK Avenue dataset, the UCSD Pedestrian dataset, and the ShanghaiTech Campus dataset verify the effectiveness of MESDnet.
26 February 2021 in IEEE Transactions on Geoscience and Remote Sensing
Fast and accurate plant counting tools affect revolution in modern agriculture. Agricultural practitioners, however, expect the output of the tools to be not only accurate but also explainable. Such explainability often refers to the ability to infer which instance is counted. One intuitive way is to generate a bounding box for each instance. Nevertheless, compared with counting by detection, plant counts can be inferred more directly in the local count framework, while one thing reproaching this paradigm is its poor explainability of output visualization. In particular, we find that the poor explainability becomes a bottleneck limiting the counting performance. To address this, we explore the idea of guided upsampling and background suppression where a novel upsampling operator is proposed to allow count redistribution, and segmentation decoders with different fusion strategies are investigated to suppress background, respectively. By integrating them into our previous counting model TasselNetV2, we introduce TasselNetV3 series: TasselNetV3-Lite and TasselNetV3-Seg. We validate the TasselNetV3 series on three public plant counting data sets and a new unmanned aircraft vehicle (UAV)-based data set, covering maize tassels counting, wheat ears counting, and rice plants counting. Extensive results show that guided upsampling and background suppression not only improve counting performance but also enable explainable visualization. Aside from state-of-the-art performance, we have several interesting observations: 1) a limited-receptive-field counter in most cases outperforms a large-receptive-field one; 2) it is sufficient to generate empirical segmentation masks from dotted annotations; 3) middle fusion is a good choice to integrate foreground–background a priori knowledge; and 4) decoupling the learning of counting and segmentation matters.
Rice density is closely related to yield estimation, growth diagnosis, cultivated area statistics, and management and damage evaluation. Currently, rice density estimation heavily relies on manual sampling and counting, which is inefficient and inaccurate. With the prevalence of digital imagery, computer vision (CV) technology emerges as a promising alternative to automate this task. However, challenges of an in-field environment, such as illumination, scale, and appearance variations, render gaps for deploying CV methods. To fill these gaps towards accurate rice density estimation, we propose a deep learning-based approach called the Scale-Fusion Counting Classification Network (SFC2Net) that integrates several state-of-the-art computer vision ideas. In particular, SFC2Net addresses appearance and illumination changes by employing a multicolumn pretrained network and multilayer feature fusion to enhance feature representation. To ameliorate sample imbalance engendered by scale, SFC2Net follows a recent blockwise classification idea. We validate SFC2Net on a new rice plant counting (RPC) dataset collected from two field sites in China from 2010 to 2013. Experimental results show that SFC2Net achieves highly accurate counting performance on the RPC dataset with a mean absolute error (MAE) of 25.51, a root mean square error (MSE) of 38.06, a relative MAE of 3.82%, and a of 0.98, which exhibits a relative improvement of 48.2% w.r.t. MAE over the conventional counting approach CSRNet. Further, SFC2Net provides high-throughput processing capability, with 16.7 frames per second on images. Our results suggest that manual rice counting can be safely replaced by SFC2Net at early growth stages. Code and models are available online at https://git.io/sfc2net.
This paper presents a thorough evaluation of twenty-one state-of-the-art widely-used crop segmentation algorithms, motived by their significance in vision tasks for further analysis. An ideal crop segmentation algorithm can effectively extract crop information, thus providing an important precondition for the application of intelligent agriculture analytics. In order to enable researchers in this field to fully understand various crop segmentation methods, this paper proposes a new classification strategy of object segmentation by dividing the algorithms into pixel-based and region-based approaches at first, and then systematically evaluating various crop segmentation methods with a unified data benchmark and four common metrics. A new dataset which incorporates crop variety, environment condition and observation distance into consideration is constructed for demonstrating the experiments and comparisons. The effectiveness and robustness of these algorithms were evaluated by three sets of comparative experiments. Based on the quantitative results, we summarize the advantages and disadvantages of the evaluated algorithms from the segmentation performances with four metric indicators. Furthermore, the discussion and evaluation results will provide great support for precision agriculture analysis.
27 November 2021 in Journal of Ambient Intelligence and Humanized Computing
<!-- *** Custom HTML *** --> Development of computer vision technologies has been widely used to increase the level of agricultural intelligence. Crop counting, an application of image counting, plays a fundamental role in agricultural information automation. However, the complex cotton field environment is likely lead to incorrect detection of the target position or fragmentation of the segmentation results, resulting in a decrease in counting accuracy. Despite this, computer vision technologies have shown great potential to effectively solve this task. To solve the problem of multimode cotton boll counting in a complicated environment, an in-field cotton boll counting algorithm based on density classification is proposed. First, the algorithm encodes the global context information with a density level classification estimator. Then, the input images are converted into high-dimensional feature maps by density map estimator with a multicolumn structure. Finally, through the feature fusion neural network, the classification information is combined with high-dimensional feature maps to generate a high-quality density map, and then the cotton bolls are counted. In particular, we collected and labeled a cotton boll counting dataset with 758 high-resolution images for experiment and comparison, which can be divided by different environmental conditions and observations sites. In addition, the relationship of cotton yield against the cotton bolls counting can be assessed by adopting our proposed algorithm. Experimental results demonstrate that the proposed algorithm achieves a lower counting error and better effectiveness and robustness than other comparative algorithms.