Gearboxes are vital in complex mechanical systems such as automobiles and wind turbines, where accurate fault diagnosis is crucial for safety and reliability. However, real-world diagnostic scenarios often involve challenges such as data imbalance, which limit the effectiveness of traditional supervised learning methods. To address this issue, this paper proposes an unsupervised subdomain adaptation framework that incorporates a joint feature extraction module and a margin-aware adaptive weighting strategy. The joint feature extraction module is constructed by integrating a bidirectional long short-term memory network with a multi-head self-attention mechanism to capture both temporal dependencies and global contextual features from fault signals while mitigating the effects of negative transfer. The margin-aware strategy dynamically adjusts sample importance according to class margins to enhance recognition of minority faults. To further improve generalization, a subdomain alignment module is employed to reduce the distributional discrepancy between the source and target domains. Extensive experiments conducted on multiple real-world gearbox fault datasets demonstrate that the proposed method consistently outperforms existing approaches under imbalanced conditions.