Synaptic vesicle glycoprotein 2A (SV2A) in human brains is an important biomarker of synaptic loss associated with several disorders, including Alzheimer's disease (AD). However, SV2A tracers, such as [11C]UCB-J, are less available in clinical practice due to constraints that include cost and the need for an onsite cyclotron. As a result, there is interest in generating synthetic SV2A PET images using a more commonly used tracer, or another widely available image modality. We therefore investigated the feasibility of using a deep learning approach to generate synthetic [11C]UCB-J SV2A PET images from T1-weighted (T1w) Magnetic Resonance Imaging (MRI) and/or [18F]FDG PET imaging. In our study, we used paired data from 54 participants for training and evaluation, including 22 healthy controls (HC) and 32 AD patients. We implemented two architectures, 3D UNet and 3D Multi-stage ResUNet (MS ResUNet) for translating either MRI or [18F]FDG PET or both into [11C]UCB-J PET images. Our experiments show that, through progressive learning 3D MS ResUNet achieved less bias compared to a single-stage design. Adopting residual connection, cross-stage feature fusion and supervised attention strategies to our model contributes to better gradient flow, mitigates information loss and enables selective focus, leading to better generation results. Additionally, combining MRI T1w and [18F]FDG PET images provides the best prediction performance, whereas having [18F]FDG PET alone produces better synthetic images than using MRI alone. We also evaluated our model’s performance for 14 typical brain regions. The MS ResUNet achieved less than 5% bias in all regions. Furthermore, the differences between AD and HC in certain regions (e.g., hippocampus and entorhinal) are still significant in generated images (p=0.000). In conclusion, the 3D MS ResUNet was able to generate desirable SV2A PET images that are visually consistent with true images in terms of contrast and tracer distribution, quantitatively with low bias (-0.26% ± 5.56%) and high similarity (R2=0.975), for both AD and HC. Compared with the 3D vanilla UNet (FDG: −2.39% ± 5.55%, MRI: −5.03% ± 13.24%, or both: −1.89% ± 6.04%), the modified design produced more robust SV2A images. The synthetic images can potentially provide additional pathological insights into AD and other neurological disorders without increasing radiation exposure and cost.