Jacob S. Prince

Jacob S. Prince

Auto-generated profile, based on publicly available information

Jacob S. Prince

Harvard University

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 November 2022 in eLife
Verified
Journal Article
Open access
JPJacob S Prince
Jacob S Prince
ICIan Charest
Ian Charest
JKJan Kurzawski
Jan Kurzawski
JPJohn A Pyles
John A Pyles
MTMichael J Tarr
Michael J Tarr
KKKendrick Kay
Kendrick Kay
Advances in artificial intelligence have inspired a paradigm shift in human neuroscience, yielding large-scale functional magnetic resonance imaging (fMRI) datasets that provide high-resolution brain responses to thousands of naturalistic visual stimuli. Because such experiments necessarily involve brief stimulus durations and few repetitions of each stimulus, achieving sufficient signal-to-noise ratio can be a major challenge. We address this challenge by introducing GLMsingle, a scalable, user-friendly toolbox available in MATLAB and Python that enables accurate estimation of single-trial fMRI responses (glmsingle.org). Requiring only fMRI time-series data and a design matrix as inputs, GLMsingle integrates three techniques for improving the accuracy of trial-wise general linear model (GLM) beta estimates. First, for each voxel, a custom hemodynamic response function (HRF) is identified from a library of candidate functions. Second, cross-validation is used to derive a set of noise regressors from voxels unrelated to the experiment. Third, to improve the stability of beta estimates for closely spaced trials, betas are regularized on a voxel-wise basis using ridge regression. Applying GLMsingle to the Natural Scenes Dataset and BOLD5000, we find that GLMsingle substantially improves the reliability of beta estimates across visually-responsive cortex in all subjects. Comparable improvements in reliability are also observed in a smaller-scale auditory dataset from the StudyForrest experiment. These improvements translate into tangible benefits for higher-level analyses relevant to systems and cognitive neuroscience. We demonstrate that GLMsingle: (i) helps decorrelate response estimates between trials nearby in time; (ii) enhances representational similarity between subjects within and across datasets; and (iii) boosts one-versus-many decoding of visual stimuli. GLMsingle is a publicly available tool that can significantly improve the quality of past, present, and future neuroimaging datasets sampling brain activity across many experimental conditions.

29 November 2022 in Nature Communications
Verified
Journal Article
Open access
PDF file
SKSharif I. Kronemer
Sharif I. Kronemer
MAMark Aksen
Mark Aksen
JDJulia Z. Ding
Julia Z. Ding
JRJun Hwan Ryu
Jun Hwan Ryu
QXQilong Xin
Qilong Xin
HBHal Blumenfeld
Hal Blumenfeld
The full neural circuits of conscious perception remain unknown. Using a visual perception task, we directly recorded a subcortical thalamic awareness potential (TAP). We also developed a unique paradigm to classify perceived versus not perceived stimuli using eye measurements to remove confounding signals related to reporting on conscious experiences. Using fMRI, we discovered three major brain networks driving conscious visual perception independent of report: first, increases in signal detection regions in visual, fusiform cortex, and frontal eye fields; and in arousal/salience networks involving midbrain, thalamus, nucleus accumbens, anterior cingulate, and anterior insula; second, increases in frontoparietal attention and executive control networks and in the cerebellum; finally, decreases in the default mode network. These results were largely maintained after excluding eye movement-based fMRI changes. Our findings provide evidence that the neurophysiology of consciousness is complex even without overt report, involving multiple cortical and subcortical networks overlapping in space and time.

15 February 2023 in Communications Biology
Verified
Journal Article
Open access
PDF file
NJNidhi Jain
Nidhi Jain
AWAria Wang
Aria Wang
MHMargaret M. Henderson
Margaret M. Henderson
RLRuogu Lin
Ruogu Lin
JPJacob S. Prince
Jacob S. Prince
LWLeila Wehbe
Leila Wehbe
Visual cortex contains regions of selectivity for domains of ecological importance. Food is an evolutionarily critical category whose visual heterogeneity may make the identification of selectivity more challenging. We investigate neural responsiveness to food using natural images combined with large-scale human fMRI. Leveraging the improved sensitivity of modern designs and statistical analyses, we identify two food-selective regions in the ventral visual cortex. Our results are robust across 8 subjects from the Natural Scenes Dataset (NSD), multiple independent image sets and multiple analysis methods. We then test our findings of food selectivity in an fMRI “localizer” using grayscale food images. These independent results confirm the existence of food selectivity in ventral visual cortex and help illuminate why earlier studies may have failed to do so. Our identification of food-selective regions stands alongside prior findings of functional selectivity and adds to our understanding of the organization of knowledge within the human visual system.

27 September 2024 in Science Advances
Verified
Journal Article
Open access
JPJacob S. Prince
Jacob S. Prince
GAGeorge A. Alvarez
George A. Alvarez
TKTalia Konkle
Talia Konkle
Modular and distributed coding theories of category selectivity along the human ventral visual stream have long existed in tension. Here, we present a reconciling framework—contrastive coding—based on a series of analyses relating category selectivity within biological and artificial neural networks. We discover that, in models trained with contrastive self-supervised objectives over a rich natural image diet, category-selective tuning naturally emerges for faces, bodies, scenes, and words. Further, lesions of these model units lead to selective, dissociable recognition deficits, highlighting their distinct functional roles in information processing. Finally, these pre-identified units can predict neural responses in all corresponding face-, scene-, body-, and word-selective regions of human visual cortex, under a highly constrained sparse positive encoding procedure. The success of this single model indicates that brain-like functional specialization can emerge without category-specific learning pressures, as the system learns to untangle rich image content. Contrastive coding, therefore, provides a unifying account of object category emergence and representation in the human brain.

29 March 2022 in bioRxiv
Verified
Preprint
Open access
PDF file
CCColin Conwell
Colin Conwell
JPJacob S. Prince
Jacob S. Prince
KKKendrick Kay
Kendrick Kay
GAGeorge A. Alvarez
George A. Alvarez
TKTalia Konkle
Talia Konkle
The rapid development and open-source release of highly performant computer vision models offers new potential for examining how different inductive biases impact representation learning and emergent alignment with the high-level human ventral visual system. Here, we assess a diverse set of 224 models, curated to enable controlled comparison of different model properties, testing their brain predictivity using large-scale functional magnetic resonance imaging data. We find that models with qualitatively different architectures (e.g. CNNs versus Transformers) and markedly different task objectives (e.g. purely visual contrastive learning versus vision-language alignment) achieve near equivalent degrees of brain predictivity, when other factors are held constant. Instead, variation across model visual training diets yields the largest, most consistent effect on emergent brain predictivity. Overarching model properties commonly suspected to increase brain predictivity (e.g. greater effective dimensionality; learnable parameter count) were not robust indicators across this more extensive survey. We highlight that standard model-to-brain linear re-weighting methods may be too flexible, as most performant models have very similar brain-predictivity scores, despite significant variation in their underlying representations. Broadly, our findings point to the importance of visual diet, challenge common assumptions about the methods used to link models to brains, and more concretely outline future directions for leveraging the full diversity of existing open-source models as tools to probe the common computational principles underlying biological and artificial visual systems.

1 September 2023 in Science Advances
Verified
Journal Article
Open access
KVKasper Vinken
Kasper Vinken
JPJacob S. Prince
Jacob S. Prince
TKTalia Konkle
Talia Konkle
MLMargaret S. Livingstone
Margaret S. Livingstone
Face cells are neurons that respond more to faces than to non-face objects. They are found in clusters in the inferotemporal cortex, thought to process faces specifically, and, hence, studied using faces almost exclusively. Analyzing neural responses in and around macaque face patches to hundreds of objects, we found graded response profiles for non-face objects that predicted the degree of face selectivity and provided information on face-cell tuning beyond that from actual faces. This relationship between non-face and face responses was not predicted by color and simple shape properties but by information encoded in deep neural networks trained on general objects rather than face classification. These findings contradict the long-standing assumption that face versus non-face selectivity emerges from face-specific features and challenge the practice of focusing on only the most effective stimulus. They provide evidence instead that category-selective neurons are best understood by their tuning directions in a domain-general object space.