Due to their unique structural design, portal cranes have been extensively utilized in bulk cargo and container terminals. The bearing fault of their drive motors is a critical issue that significantly impacts their operational efficiency. Moreover, the problem of imbalanced fault samples has a more pronounced influence on the application of novel fault diagnosis methods. To address this, the paper presents a new method called bidirectional gated recurrent domain adversarial transfer learning (BRDATL), specifically designed for imbalanced samples from portal cranes’ drive motor bearings. Initially, a bidirectional gated recurrent unit (Bi-GRU) is used as a feature extractor within the network to comprehensively extract features from both source and target domains. Building on this, a new Correlation Maximum Mean Discrepancy (CAMMD) method, integrating both Correlation Alignment (CORAL) and Maximum Mean Discrepancy (MMD), is proposed to guide the feature generator in providing domain-invariant features. Considering the real-time data characteristics of portal crane drive motor bearings, we adjusted the CWRU and XJTU-SY bearing datasets and conducted comparative experiments. The experimental results show that the accuracy of the proposed method is up to 99.5%, which is obviously higher than other methods. The presented fault diagnosis model provides a practical and theoretical framework for diagnosing faults in portal cranes’ field operation environments.