Synthetic aperture radar (SAR) provides an all-weather and all-time imaging platform, which is more reliable than electro-optical (EO) remote sensing imagery under extreme weather/lighting conditions. While many large-scale EO-based remote sensing datasets have been released for computer vision tasks, there are few publicly available SAR image datasets due to the high costs associated with acquisition and labeling. Recent works have applied deep learning methods for image translation between SAR and EO. However, the effectiveness of those techniques on high-resolution images has been hindered by a common limitation. Non-linear geometric distortions, induced by different imaging principles of optical and radar sensors, have caused insufficient pixel-wise correspondence between an EO-SAR patch pair. Such a phenomenon is not prominent in low-resolution EO-SAR datasets, e.g., SEN1-2, one of the most frequently used datasets, and thus has been seldom discussed. To address this issue, a new dataset SN6-SAROPT with sub-meter resolution is introduced, and a novel image translation algorithm designed to tackle geometric distortions adaptively is proposed in this paper. Extensive experiments have been conducted to evaluate the proposed algorithm, and the results have validated its superiority over other methods for both SAR to EO (S2E) and EO to SAR (E2S) tasks, especially for urban areas in high-resolution images.