Event-based sensing, i.e. the asynchronous detection of luminance changes, promises low-energy, high dynamic range, and sparse sensing. This stands in contrast to whole image frame-wise acquisition by standard cameras. Here, we systematically investigate the implications of event-based sensing in the context of visual motion, or flow, estimation. Starting from a common theoretical foundation, we discuss different principal approaches for optical flow detection ranging from gradient-based methods over plane-fitting to filter based methods and identify strengths and weaknesses of each class. Gradient-based methods for local motion integration are shown to suffer from the sparse encoding in address-event representations (AER). Approaches exploiting the local plane like structure of the event cloud, on the other hand, are shown to be well suited. Within this class, filter based approaches are shown to define a proper detection scheme which can also deal with the problem of representing multiple motions at a single location (motion transparency). A novel biologically inspired efficient motion detector is proposed, analyzed and experimentally validated. Furthermore, a stage of surround normalization is incorporated. Together with the filtering this defines a canonical circuit for motion feature detection. The theoretical analysis shows that such an integrated circuit reduces motion ambiguity in addition to decorrelating the representation of motion related activations.

Physiological and psychophysical studies have demonstrated the importance of colinearity in visual processing. Motivated by these empirical findings we present a novel computational model of recurrent long-range processing in the primary visual cortex. Unlike other models we restrict the long-range interaction to cells of parallel orientation with colinear aligned receptive fields. We also employ a recurrent interaction using modulatory feedback, in accordance with empirical findings. Self-normalizing shunting equations guarantee the saturation of activities after a few recurrent cycles. The primary computational goal of the model is to evaluate local, often noisy orientation measurements within a more global context and to selectively enhance coherent activity by excitatory, modulating feedback. All model simulations were done with the same set of parameters. We show that the model qualitatively reproduces empirical data of response facilitation and suppression for a single bar element depending on the local surround outside the classical receptive field (M. K. Kapadia, M. Ito, C. D. Gilbert, & G. Westheimer, 1995). Next we evaluate the model performance for the processing of artificial and natural images. We quantitatively evaluate the model using two measures of contour saliency and orientation significance. We show that both measures monotonically increase during the recurrent interaction and saturate after a small number of recurrent cycles. The model clarifies how basic tasks of early vision can be accomplished within a single, biologically plausible architecture.

The processing of a visual stimulus can be subdivided into a number of stages. Upon stimulus presentation there is an early phase of feedforward processing where the visual information is propagated from lower to higher visual areas for the extraction of basic and complex stimulus features. This is followed by a later phase where horizontal connections within areas and feedback connections from higher areas back to lower areas come into play. In this later phase, image elements that are behaviorally relevant are grouped by Gestalt grouping rules and are labeled in the cortex with enhanced neuronal activity (object-based attention in psychology). Recent neurophysiological studies revealed that reward-based learning influences these recurrent grouping processes, but it is not well understood how rewards train recurrent circuits for perceptual organization. This paper examines the mechanisms for reward-based learning of new grouping rules. We derive a learning rule that can explain how rewards influence the information flow through feedforward, horizontal and feedback connections. We illustrate the efficiency with two tasks that have been used to study the neuronal correlates of perceptual organization in early visual cortex. The first task is called contour-integration and demands the integration of collinear contour elements into an elongated curve. We show how reward-based learning causes an enhancement of the representation of the to-be-grouped elements at early levels of a recurrent neural network, just as is observed in the visual cortex of monkeys. The second task is curve-tracing where the aim is to determine the endpoint of an elongated curve composed of connected image elements. If trained with the new learning rule, neural networks learn to propagate enhanced activity over the curve, in accordance with neurophysiological data. We close the paper with a number of model predictions that can be tested in future neurophysiological and computational studies. Our experience with the visual world allows us to group image elements that belong to the same perceptual object and to segregate them from other objects and the background. If subjects learn to group contour elements, this experience influences neuronal activity in early visual cortical areas, including the primary visual cortex (V1). Learning presumably depends on alterations in the pattern of connections within and between areas of the visual cortex. However, the processes that control changes in connectivity are not well understood. Here we present the first computational model that can train a neural network to integrate collinear contour elements into elongated curves and to trace a curve through the visual field. The new learning algorithm trains fully recurrent neural networks, provided the connectivity causes the networks to reach a stable state. The model reproduces the behavioral performance of monkeys trained in these tasks and explains the patterns of neuronal activity in the visual cortex that emerge during learning, which is remarkable because the only feedback for the model is a reward for successful trials. We discuss a number of the model predictions that can be tested in future neuroscientific work.

Junctions provide important cues in various perceptual tasks, such as the determination of occlusion relationships for figure-ground separation, transparency perception, and object recognition, among others. In computer vision, junctions are used in a number of tasks, like point matching for image tracking or correspondence analysis. We propose a biologically motivated approach to junction representation in which junctions are implicitly characterized by high activity for multiple orientations within a cortical hypercolumn. A local measure of circular variance is suggested to extract junction points from this distributed representation. Initial orientation measurements are often fragmented and noisy. A coherent contour representation can be generated by a model of V1 utilizing mechanisms of collinear long-range integration and recurrent interaction. In the model, local oriented contrast estimates that are consistent within a more global context are enhanced while inconsistent activities are suppressed. In a series of computational experiments, we compare junction detection based on the new recurrent model with a feedforward model of complex cells. We show that localization accuracy and positive correctness in the detection of generic junction configurations such as L- and T-junctions is improved by the recurrent long-range interaction. Further, receiver operating characteristics analysis is used to evaluate the detection performance on both synthetic and camera images, showing the superior performance of the new approach. Overall, we propose that nonlocal interactions implemented by known mechanisms within V1 play an important role in detecting higher-order features such as corners and junctions.