Shir Amir

Shir Amir

Auto-generated profile, based on publicly available information

Shir Amir

Weizmann Institute of Science, Israel

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 February 2023
Verified
Book Chapter
Closed access
PDF file
SAShir Amir
Shir Amir
YGYossi Gandelsman
Yossi Gandelsman
SBShai Bagon
Shai Bagon
TDTali Dekel
Tali Dekel

30 November 2023 in ACM Transactions on Graphics
Verified
Journal Article
Open access
NTNarek Tumanyan
Narek Tumanyan
OBOmer Bar-Tal
Omer Bar-Tal
SAShir Amir
Shir Amir
SBShai Bagon
Shai Bagon
TDTali Dekel
Tali Dekel
We present a method for semantically transferring the visual appearance of one natural image to another. Specifically, our goal is to generate an image in which objects in a source structure image are “painted” with the visual appearance of their semantically related objects in a target appearance image. To integrate semantic information into our framework, our key idea is to leverage a pre-trained and fixed Vision Transformer (ViT) model. Specifically, we derive novel disentangled representations of structure and appearance extracted from deep ViT features. We then establish an objective function that splices the desired structure and appearance representations, interweaving them together in the space of ViT features. Based on our objective function, we propose two frameworks of semantic appearance transfer – “Splice”, which works by training a generator on a single and arbitrary pair of structure-appearance images, and “SpliceNet”, a feed-forward real-time appearance transfer model trained on a dataset of images from a specific domain . Our frameworks do not involve adversarial training, nor do they require any additional input information such as semantic segmentation or correspondences. We demonstrate high-resolution results on a variety of in-the-wild image pairs, under significant variations in the number of objects, pose, and appearance. Code and supplementary material are available in our project page: splice-vit.github.io.

20 November 2023 in arXiv
Verified
Preprint
Open access
PDF file
NTNarek Tumanyan
Narek Tumanyan
OBOmer Bar-Tal
Omer Bar-Tal
SAShir Amir
Shir Amir
SBShai Bagon
Shai Bagon
TDTali Dekel
Tali Dekel
We present a method for semantically transferring the visual appearance of one natural image to another. Specifically, our goal is to generate an image in which objects in a source structure image are "painted" with the visual appearance of their semantically related objects in a target appearance image. To integrate semantic information into our framework, our key idea is to leverage a pre-trained and fixed Vision Transformer (ViT) model. Specifically, we derive novel disentangled representations of structure and appearance extracted from deep ViT features. We then establish an objective function that splices the desired structure and appearance representations, interweaving them together in the space of ViT features. Based on our objective function, we propose two frameworks of semantic appearance transfer -- "Splice", which works by training a generator on a single and arbitrary pair of structure-appearance images, and "SpliceNet", a feed-forward real-time appearance transfer model trained on a dataset of images from a specific domain. Our frameworks do not involve adversarial training, nor do they require any additional input information such as semantic segmentation or correspondences. We demonstrate high-resolution results on a variety of in-the-wild image pairs, under significant variations in the number of objects, pose, and appearance. Code and supplementary material are available in our project page: splice-vit.github.io.