James J. DiCarlo

James J. DiCarlo

Auto-generated profile, based on publicly available information

James J. DiCarlo

Massachusetts Institute of Technology

Research Focus & Metrics

Academic Career Length

24Years (2003-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 May 2014 in Proceedings of the National Academy of Sciences
Verified
Journal Article
Closed access
PDF file
DYDaniel L. K. Yamins
Daniel L. K. Yamins
HHHa Hong
Ha Hong
CCCharles F. Cadieu
Charles F. Cadieu
ESEthan A. Solomon
Ethan A. Solomon
DSDarren Seibert
Darren Seibert
JDJames J. DiCarlo
James J. DiCarlo
Significance: Humans and monkeys easily recognize objects in scenes. This ability is known to be supported by a network of hierarchically interconnected brain areas. However, understanding neurons in higher levels of this hierarchy has long remained a major challenge in visual systems neuroscience. We use computational techniques to identify a neural network model that matches human performance on challenging object categorization tasks. Although not explicitly constrained to match neural data, this model turns out to be highly predictive of neural responses in both the V4 and inferior temporal cortex, the top two layers of the ventral visual hierarchy. In addition to yielding greatly improved models of visual cortex, these results suggest that a process of biological performance optimization directly shaped neural mechanisms.

1 February 2012 in Neuron
Verified
Journal Article
Closed access
JDJames J. DiCarlo
James J. DiCarlo
DZDavide Zoccolan
Davide Zoccolan
NRNicole C. Rust
Nicole C. Rust

1 August 2007 in Trends in Cognitive Sciences
Verified
Journal Article
Closed access
JDJames J. DiCarlo
James J. DiCarlo
DCDavid D. Cox
David D. Cox

29 April 2019 in Nature Neuroscience
Verified
Journal Article
Open access
PDF file
KKKohitij Kar
Kohitij Kar
JKJonas Kubilius
Jonas Kubilius
KSKailyn Schmidt
Kailyn Schmidt
EIElias B. Issa
Elias B. Issa
JDJames J. DiCarlo
James J. DiCarlo
Non-recurrent deep convolutional neural networks (CNNs) are currently the best at modeling core object recognition, a behavior that is supported by the densely recurrent primate ventral stream, culminating in the inferior temporal (IT) cortex. If recurrence is critical to this behavior, then primates should outperform feedforward-only deep CNNs for images that require additional recurrent processing beyond the feedforward IT response. Here we first used behavioral methods to discover hundreds of these ‘challenge’ images. Second, using large-scale electrophysiology, we observed that behaviorally sufficient object identity solutions emerged ~30 ms later in the IT cortex for challenge images compared with primate performance-matched ‘control’ images. Third, these behaviorally critical late-phase IT response patterns were poorly predicted by feedforward deep CNN activations. Notably, very-deep CNNs and shallower recurrent CNNs better predicted these late IT responses, suggesting that there is a functional equivalence between additional nonlinear transformations and recurrence. Beyond arguing that recurrent circuits are critical for rapid object identification, our results provide strong constraints for future recurrent model development.

3 May 2019 in Science
Verified
Journal Article
Closed access
PDF file
PBPouya Bashivan
Pouya Bashivan
KKKohitij Kar
Kohitij Kar
JDJames J. DiCarlo
James J. DiCarlo
Predicting behavior of visual neurons: To what extent are predictive deep learning models of neural responses useful for generating experimental hypotheses? Bashivan et al. took an artificial neural network built to model the behavior of the target visual system and used it to construct images predicted to either broadly activate large populations of neurons or selectively activate one population while keeping the others unchanged. They then analyzed the effectiveness of these images in producing the desired effects in the macaque visual cortex. The manipulations showed very strong effects and achieved considerable and highly selective influence over the neuronal populations. Using novel and non-naturalistic images, the neural network was shown to reproduce the overall behavior of the animals' neural responses. Science , this issue p. eaav9436

13 July 2018 in The Journal of Neuroscience
Verified
Journal Article
Closed access
PDF file
RRRishi Rajalingham
Rishi Rajalingham
EIElias B. Issa
Elias B. Issa
PBPouya Bashivan
Pouya Bashivan
KKKohitij Kar
Kohitij Kar
KSKailyn Schmidt
Kailyn Schmidt
JDJames J. DiCarlo
James J. DiCarlo
Primates, including humans, can typically recognize objects in visual images at a glance despite naturally occurring identity-preserving image transformations (e.g., changes in viewpoint). A primary neuroscience goal is to uncover neuron-level mechanistic models that quantitatively explain this behavior by predicting primate performance for each and every image. Here, we applied this stringent behavioral prediction test to the leading mechanistic models of primate vision (specifically, deep, convolutional, artificial neural networks; ANNs) by directly comparing their behavioral signatures against those of humans and rhesus macaque monkeys. Using high-throughput data collection systems for human and monkey psychophysics, we collected more than one million behavioral trials from 1472 anonymous humans and five male macaque monkeys for 2400 images over 276 binary object discrimination tasks. Consistent with previous work, we observed that state-of-the-art deep, feedforward convolutional ANNs trained for visual categorization (termed DCNNIC models) accurately predicted primate patterns of object-level confusion. However, when we examined behavioral performance for individual images within each object discrimination task, we found that all tested DCNNIC models were significantly nonpredictive of primate performance and that this prediction failure was not accounted for by simple image attributes nor rescued by simple model modifications. These results show that current DCNNIC models cannot account for the image-level behavioral patterns of primates and that new ANN models are needed to more precisely capture the neural mechanisms underlying primate object vision. To this end, large-scale, high-resolution primate behavioral benchmarks such as those obtained here could serve as direct guides for discovering such models. SIGNIFICANCE STATEMENT Recently, specific feedforward deep convolutional artificial neural networks (ANNs) models have dramatically advanced our quantitative understanding of the neural mechanisms underlying primate core object recognition. In this work, we tested the limits of those ANNs by systematically comparing the behavioral responses of these models with the behavioral responses of humans and monkeys at the resolution of individual images. Using these high-resolution metrics, we found that all tested ANN models significantly diverged from primate behavior. Going forward, these high-resolution, large-scale primate behavioral benchmarks could serve as direct guides for discovering better ANN models of the primate visual system.