Almost all the world’s food is grown in open fields, where plant phenotypes can be very different from those observed in greenhouses. Geneticists and agronomists studying food crops routinely detect, measure, and classify a wide variety of phenotypes in fields that contain many visually distinct types of a single crop. Augmenting humans in these tasks by automatically interpreting images raises some important and nontrivial challenges for research in computer vision. Nonetheless, the rewards for overcoming these obstacles could be exceptionally high for today’s 7 billion people, let alone the 9.6 billion projected by 2050 (United Nations Department of Economic and Social Affairs, Population Division, World Population Prospects: The 2012 Revision). To stimulate dialog between researchers in computer vision and those in genetics and agronomy, we offer our views on three computational challenges that are central to many phenotyping tasks. These are disambiguating one plant from another; assigning an individual plant’s organs to it; and identifying field phenotypes from those shown in archival images. We illustrate these challenges with annotated photographs of maize highlighting the regions of interest. We also describe some of the experimental, logistical, and photographic constraints on image collection and processing. While collecting the data sets needed for algorithmic experiments requires sustained collaboration and funding, the images we show and have posted should allow one to consider the problems, think of possible approaches, and decide on the next steps.

Everyone needs experimental data to understand biology. Exactly how and from what the data were obtained determines an experiment's results, specifies how it can be reproduced, and conditions our analyses and interpretations. These details of materials, methods, and analyses are the experiment's provenance. Today, as it has been for hundreds of years, experimental provenance is recorded in some form of laboratory notebook. But as data migrate from the experimentalist's mind and notebook to publication, the lab server, the archival database, or the cloud, this essential information now vanishes. Like interpretation, our ability to reproduce results depends on knowing how they were produced by others. Shorn of their immediate context, the methodological ideas and information that were perfectly transparent to the experimentalist (or computationalist!) become opportunities for error-prone reconstruction by others, even within the same group [1-3]. That reconstruction requires (repeated) private communications, rereading notebook entries, polling one's own or a group's collective memory, and looking at the specimens. None of those methods are reliable, and all are tedious. As big data become a reality, it will be ever more imperative to encapsulate experimental provenance with the data. But how do we get that information out of the brains and notebooks in the first place? This is a problem of information capture, not data formats; of laboratory practice, not resource discovery; and of the million flowers of experimental creativity, not ontology building. Of course, ontologies, interoperable grid resources, and efficient search are important and appealing-but absent experimental provenance, they are biologically moot. The obvious solution might seem to be standards for post hoc data annotation by biologists. Indeed, several previous efforts have defined sets of "minimal" metadata about particular types of high-throughput experiments, beginning with the minimal information needed for microarray experiments (the MIAME criteria) [4]. However, experience shows three fundamental problems with this approach. First, despite vigorous encouragement from computational biologists, most deposited datasets lack such annotation [1]. Second, the universe of experiments performed, let alone possible, far exceeds the stamina of even the most earnest committees to promulgate definitions and criteria. As always, science outruns nomenclature. The third fundamental problem is even more basic than effort or invention. Many experimentally inclined biologists are too ill equipped and too busy to produce electronic provenance metadata in almost any form. Large experimental consortia and high-throughput facilities often do develop in-house provenance systems, and these and commercial ones are available (for example, see [5,6]). But smaller groups also generate essential data, often with a wide variety of experiments that don't fit existing packages, standards, or ontologies. Their provenance information tends to be fragmented, buried in a mix of paper and electronic records, and dependent on the

Premise: Aerial imagery from small unmanned aerial vehicle systems is a promising approach for high‐throughput phenotyping and precision agriculture. A key requirement for both applications is to create a field‐scale mosaic of the aerial imagery sequence so that the same features are in registration, a very challenging problem for crop imagery.Methods: We have developed an improved mosaicking pipeline, Video Mosaicking and summariZation (VMZ), which uses a novel two‐dimensional mosaicking algorithm that minimizes errors in estimating the transformations between successive frames during registration. The VMZ pipeline uses only the imagery, rather than relying on vehicle telemetry, ground control points, or global positioning system data, to estimate the frame‐to‐frame homographies. It exploits the spatiotemporal ordering of the image frames to reduce the computational complexity of finding corresponding features between frames using feature descriptors. We compared the performance of VMZ to a standard two‐dimensional mosaicking algorithm (AutoStitch) by mosaicking imagery of two maize (Zea mays) research nurseries freely flown with a variety of trajectories.Results: The VMZ pipeline produces superior mosaics faster. Using the speeded up robust features (SURF) descriptor, VMZ produces the highest‐quality mosaics.Discussion: Our results demonstrate the value of VMZ for the future automated extraction of plant phenotypes and dynamic scouting for crop management.

Unoccupied aerial vehicles ( UAV s) are increasingly used for high throughput phenotyping in quantitative genetics and breeding trials. In principle, freely flown vehicles would permit real‐time flexibility in identifying and monitoring regions and plants of interest. Mosaicking multiple images provides a high resolution global image, and consumer‐grade UAV s offer low cost, ease of flying, and excellent red green blue ( RGB ) cameras. However, the vehicles' inaccurate telemetry complicates estimating the homographies between pairs of frames during mosaicking, and accumulated errors distort later portions of the mosaic. Crop fields are particularly challenging because their regular planting pattern and very similar plants eliminate the distinctive features that guide mosaicking. To meet these challenges for a wider range of investigators, we propose DroneZaic , an end‐to‐end pipeline that dynamically samples video frames, automates camera and gimbal calibration, estimates homographies, and generates mini‐mosaics. Together, these techniques significantly reduce errors in the output mosaics. Our unsupervised deep learning model component is trained on a comprehensive video dataset comprising different flight trajectories, maize lines, growth stages, and synthetic illumination data augmentation, which involves systematically altering lighting conditions and adding noise to enhance model generalizability. DroneZaic and its refined CorNetv3 are more accurate, achieving a 13.1% improvement in absolute percentage error, 14.11 times faster than affine scale‐invariant feature transform (ASIFT), and more robust than our earlier CorNet and CorNetv2 . We demonstrate DroneZaic 's effectiveness and generalizability in computing accurate mosaics of imagery captured by freely flown UAV s.