Samuel Schulter

Samuel Schulter

Auto-generated profile, based on publicly available information

Samuel Schulter

NEC Labs America

Research Focus & Metrics

Academic Career Length

12Years (2015-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 November 2022
Verified
Book Chapter
Closed access
PDF file
SZShiyu Zhao
Shiyu Zhao
ZZZhixing Zhang
Zhixing Zhang
SSSamuel Schulter
Samuel Schulter
LZLong Zhao
Long Zhao
BKB.G Vijay Kumar
B.G Vijay Kumar
DMDimitris N. Metaxas
Dimitris N. Metaxas

7 October 2018
Verified
Book Chapter
Closed access
PDF file
SSSamuel Schulter
Samuel Schulter
MZMenghua Zhai
Menghua Zhai
NJNathan Jacobs
Nathan Jacobs
MCManmohan Chandraker
Manmohan Chandraker

1 June 2023
Verified
Conference Paper
Open access
ZKZaid Khan
Zaid Khan
BKBg Vijay Kumar
Bg Vijay Kumar
SSSamuel Schulter
Samuel Schulter
XYXiang Yu
Xiang Yu
YFYun Fu
Yun Fu
MCManmohan Chandraker
Manmohan Chandraker
Finetuning a large vision language model (VLM) on a target dataset after large scale pretraining is a dominant paradigm in visual question answering (VQA). Datasets for specialized tasks such as knowledge-based VQA or VQA in non natural-image domains are orders of magnitude smaller than those for general-purpose VQA. While collecting additional labels for specialized tasks or domains can be challenging, unlabeled images are often available. We introduce SelTDA (Self-Taught Data Augmentation), a strategy for finetuning large VLMs on small-scale VQA datasets. SelTDA uses the VLM and target dataset to build a teacher model that can generate question-answer pseudolabels directly conditioned on an image alone, allowing us to pseudolabel unlabeled images. SelTDA then finetunes the initial VLM on the original dataset augmented with freshly pseudolabeled images. We describe a series of experiments showing that our self-taught data augmentation increases robustness to adversarially searched questions, counterfactual examples and rephrasings, improves domain generalization, and results in greater retention of numerical reasoning skills. The proposed strategy requires no additional annotations or architectural modifications, and is compatible with any modern encoder-decoder multimodal transformer. Code available at https://github.com/codezakh/SelTDA.

1 June 2020
Verified
Conference Paper
Closed access
ZJZiyu Jiang
Ziyu Jiang
BLBuyu Liu
Buyu Liu
SSSamuel Schulter
Samuel Schulter
ZWZhangyang (Atlas) Wang
Zhangyang (Atlas) Wang
MCManmohan Chandraker
Manmohan Chandraker
We address the challenging task of occlusion-aware indoor 3D scene understanding. We represent scenes by a set of planes, where each one is defined by its normal, offset and two masks outlining (i) the extent of the visible part and (ii) the full region that consists of both visible and occluded parts of the plane. We infer these planes from a single input image with a novel neural network architecture. It consists of a two-branch category-specific module that aims to predict layout and objects of the scene separately so that different types of planes can be handled better. We also introduce a novel loss function based on plane warping that can leverage multiple views at training time for improved occlusion-aware reasoning. In order to train and evaluate our occlusion-reasoning model, we use the ScanNet dataset and propose (i) a strategy to automatically extract ground truth for both visible and hidden regions and (ii) a new evaluation metric that specifically focuses on the prediction in hidden regions. We empirically demonstrate that our proposed approach can achieve higher accuracy for occlusion reasoning compared to competitive baselines on the ScanNet dataset, e.g. 42.65% relative improvement on hidden regions.

29 October 2022
Verified
Book Chapter
Closed access
PDF file
ZKZaid Khan
Zaid Khan
BKB. G. Vijay Kumar
B. G. Vijay Kumar
XYXiang Yu
Xiang Yu
SSSamuel Schulter
Samuel Schulter
MCManmohan Chandraker
Manmohan Chandraker
YFYun Fu
Yun Fu

1 September 2015
Verified
Conference Paper
Closed access
MZMatthias Zeppelzauer
Matthias Zeppelzauer
GPGeorg Poier
Georg Poier
MSMarkus Seidl
Markus Seidl
CRChristian Reinbacher
Christian Reinbacher
CBChristian Breiteneder
Christian Breiteneder
SSSamuel Schulter
Samuel Schulter
Petroglyphs (rock engravings) are important artifacts for the documentation and analysis of early human life. Recent improvements in 3D scanning and 3D reconstruction enable the accurate 3D reconstruction of petroglyphs from rock surfaces at sub-millimeter resolution. To enable the indexing, matching, and recognition of petroglyphs in petroglyph databases, the shapes must first be segmented from the reconstructed rock surface. The absence of robust 3D segmentation methods for petroglyphs leaves a gap in the digital processing workflow. In this paper, we present a semi-automatic method for petroglyph segmentation for high-resolution 3D surface reconstructions. A comprehensive evaluation shows that our method is able to robustly segment petroglyphs with high accuracy and that the incorporation of 3D information is crucial to solve the segmentation problem. The presented method represents a major step towards the completion of a full 3D digital processing workflow of petroglyphs.