Most animal species on Earth are insects, and recent reports suggest that their abundance is in drastic decline. Although these reports come from a wide range of insect taxa and regions, the evidence to assess the extent of the phenomenon is sparse. Insect populations are challenging to study, and most monitoring methods are labor intensive and inefficient. Advances in computer vision and deep learning provide potential new solutions to this global challenge. Cameras and other sensors can effectively, continuously, and noninvasively perform entomological observations throughout diurnal and seasonal cycles. The physical appearance of specimens can also be captured by automated imaging in the laboratory. When trained on these data, deep learning models can provide estimates of insect abundance, biomass, and diversity. Further, deep learning models can quantify variation in phenotypic traits, behavior, and interactions. Here, we connect recent developments in deep learning and computer vision to the urgent demand for more cost-efficient monitoring of insects and other invertebrates. We present examples of sensor-based monitoring of insects. We show how deep learning tools can be applied to exceptionally large datasets to derive ecological information and discuss the challenges that lie ahead for the implementation of such solutions in entomology. We identify four focal areas, which will facilitate this transformation: 1) validation of image-based taxonomic identification; 2) generation of sufficient training data; 3) development of public, curated reference databases; and 4) solutions to integrate deep learning and molecular tools.

High‐resolution monitoring is fundamental to understand ecosystems dynamics in an era of global change and biodiversity declines. While real‐time and automated monitoring of abiotic components has been possible for some time, monitoring biotic components—for example, individual behaviours and traits, and species abundance and distribution—is far more challenging. Recent technological advancements offer potential solutions to achieve this through: (i) increasingly affordable high‐throughput recording hardware, which can collect rich multidimensional data, and (ii) increasingly accessible artificial intelligence approaches, which can extract ecological knowledge from large datasets. However, automating the monitoring of facets of ecological communities via such technologies has primarily been achieved at low spatiotemporal resolutions within limited steps of the monitoring workflow. Here, we review existing technologies for data recording and processing that enable automated monitoring of ecological communities. We then present novel frameworks that combine such technologies, forming fully automated pipelines to detect, track, classify and count multiple species, and record behavioural and morphological traits, at resolutions which have previously been impossible to achieve. Based on these rapidly developing technologies, we illustrate a solution to one of the greatest challenges in ecology: the ability to rapidly generate high‐resolution, multidimensional and standardised data across complex ecologies.

Insects are declining in abundance and diversity, but their population trends remain uncertain as insects are difficult to monitor. Manual methods require substantial time investment in trapping and subsequent species identification. Camera trapping can alleviate some of the manual fieldwork, but the large quantities of image data are challenging to analyse. By embedding the image analyses into the recording process using computer vision techniques, it is possible to focus efforts on the most ecologically relevant image data. Here, we present an intelligent camera system, capable of detecting, tracking, and identifying individual insects in situ. We constructed the system from commercial off‐the‐shelf components and used deep learning open source software to perform species detection and classification. We present the Insect Classification and Tracking algorithm (ICT) that performs real‐time classification and tracking at 0.33 frames per second. The system can upload summary data on the identity and movement track of insects to a server via the internet on a daily basis. We tested our system during the summer 2020 and detected 2994 insect tracks across 98 days. We achieved an average precision of 89% for correctly classified insect tracks of eight different species. This result was based on 504 manually verified tracks observed in videos during 10 days with varying insect activities. Using the track data, we could estimate the mean residence time for individual flower visiting insects within the field of view of the camera, and we were able to show a substantial variation in residence time among insect taxa. For honeybees, which were most abundant, residence time also varied through the season in relation to the plant species in bloom. Our proposed automated system showed promising results in non‐destructive and real‐time monitoring of insects and provides novel information about phenology, abundance, foraging behaviour, and movement ecology of flower visiting insects.

Insect monitoring methods are typically very time-consuming and involve substantial investment in species identification following manual trapping in the field. Insect traps are often only serviced weekly, resulting in low temporal resolution of the monitoring data, which hampers the ecological interpretation. This paper presents a portable computer vision system capable of attracting and detecting live insects. More specifically, the paper proposes detection and classification of species by recording images of live individuals attracted to a light trap. An Automated Moth Trap (AMT) with multiple light sources and a camera was designed to attract and monitor live insects during twilight and night hours. A computer vision algorithm referred to as Moth Classification and Counting (MCC), based on deep learning analysis of the captured images, tracked and counted the number of insects and identified moth species. Observations over 48 nights resulted in the capture of more than 250,000 images with an average of 5675 images per night. A customized convolutional neural network was trained on 2000 labeled images of live moths represented by eight different classes, achieving a high validation $F_1$-score of 0.93. The algorithm measured an average classification and tracking $F_1$-score of 0.71 and a tracking detection rate of 0.79. Overall, the proposed computer vision system and algorithm showed promising results as a low-cost solution for non-destructive and automatic monitoring of moths.

Reported insect declines have dramatically increased the global demand for standardized insect monitoring data. Image-based monitoring can generate such data cost-efficiently and non-invasively. However, extracting ecological data from images is more challenging for insects than for vertebrates because of their small size and great diversity. Deep learning facilitates fast and accurate insect detection and identification, but the lack of training data for coveted deep learning models is a major obstacle for their application. We present a large annotated image dataset of functionally important insect taxa. The primary dataset consists of 29,960 annotated insects representing nine taxa including bees, hoverflies, butterflies and beetles across more than two million images recorded with ten time-lapse cameras mounted over flowers during the summer of 2019. The insect image dataset was extracted using an iterative approach: First, a preliminary detection model identified candidate insects. Second, candidate insects were manually screened by users of an online citizen science platform. Finally, all annotations were quality checked by experts. We used the dataset to train and compare the performance of selected You Only Look Once (YOLO) deep learning algorithms. We show that these models detect and classify small insects in complex scenes with unprecedented accuracy. The best performing YOLOv5 model consistently identifies nine dominant insect species that play important roles in pollination and pest control across Europe. The model reached an average precision of 92.7% and recall of 93.8% in detection and classification across species. Importantly, when presented with uncommon or unclear insects not seen during training, our model detects 80% of individuals and usually interprets them as closely related species. This is a useful property to (1) detect rare insects for which training data are absent, and (2) generate new training data to correctly identify those insects in future. Our camera system, dataset and deep learning framework show promising results in non-destructive monitoring of insects. Furthermore, resulting data are useful to quantify phenology, abundance, and foraging behaviour of flower-visiting insects. Above all, this dataset represents a critical first benchmark for future development and evaluation of deep learning models for insect detection and identification.