This paper presents results on the detection and identification mango fruits from colour images of trees. We evaluate the behaviour and the performances of the Faster R-CNN network to determine whether it is robust enough to "detect and classify" fruits under particularly heterogeneous conditions in terms of plant cultivars, plantation scheme, and visual information acquisition contexts. The network is trained to distinguish the 'Kent', 'Keitt', and "Boucodiekhal" mango cultivars from 3,000 representative labelled fruit annotations. The validation set composed of about 7,000 annotations was then tested with a confidence threshold of 0.7 and a Non-Maximal-Suppression threshold of 0.25. With a F1-score of 0.90, the Faster R-CNN is well suitable to the simple fruit detection in tiles of 500x500 pixels. We then combine a multi-tiling approach with a Jaccard matrix to merge the different parts of objects detected several times, and thus report the detections made at the tile scale to the native 6,000x4,000 pixel size images. Nonetheless with a F1-score of 0.56, the cultivar identification Faster R-CNN network presents some limitations for simultaneously detecting the mango fruits and identifying their respective cultivars. Despite the proven errors in fruit detection, the cultivar identification rates of the detected mango fruits are in the order of 80%. The ideal solution could combine a Mask R-CNN for the image pre-segmentation of trees and a double-stream Faster R-CNN for detecting the mango fruits and identifying their respective cultivar to provide predictions more relevant to users' expectations.
Grassland represents more than half of the agricultural land. Numerous metrics (biomass, functional trait, species composition) can be used to describe grassland vegetation and its multiple functions. The measures of these metrics are generally destructive and laborious. Indirect measurements using optical tools are a possible alternative. Some tools have high spatial resolutions (digital camera), and others have high spectral resolutions (Near Infrared Spectrometry NIRS). A plenoptic camera is a multifocal camera that produces clear images at different depths in an image. The objective of this study was to test the interest of combining plenoptic images and NIRS data to characterize different descriptors of two Mediterranean legumes mixtures. On these mixtures, we measured biomass, species biomass, and functional trait diversity. NIRS and plenoptic images were acquired just before the field measurements. The plenoptic images were analyzed using Trainable Weka Segmentation ImageJ to evaluate the percentage of each species in the image. We calculated the average and standard deviation of the different colors (red, green, blue reflectance) in the image. We assessed the percentage of explanation of outputs of the images and NIRS analyses using variance partition and partial least squares. The biomass Trifolium michelianum and Vicia sativa were predicted with more than 50% variability explained. For the other descriptors, the variability explained was lower but nevertheless significant. The percentage variance explained was nevertheless quite low, and further work is required to produce a useable tool, but this work already demonstrates the interest in combining image analysis and NIRS.