Camera localization aims to estimate the six-degree-of-freedom camera pose from a single RGB image and is essential for robotics, autonomous navigation, and augmented reality. Despite recent progress in learning-based localization, robust pose estimation remains challenging under large viewpoint changes, occlusions, repetitive textures, and complex scene layouts. Existing methods mainly rely on local feature matching or independent feature regression, while the structural dependencies among multi-scale visual representations are not fully explored. To address this issue, we propose Uncertainty-aware Bidirectional Graph Learning (UBGL) for RGB-based camera localization. UBGL extracts hierarchical visual features and converts them into graph-structured representations, where nodes describe scene information at different semantic levels. A bidirectional graph learning module is introduced to exchange information between low-level geometric cues and high-level contextual features, enabling joint modeling of local correspondences and global scene structure. In addition, an uncertainty-aware relation modeling strategy estimates the reliability of graph connections and helps reduce the influence of unstable feature interactions during graph reasoning. The refined graph representations are then projected into dense scene-coordinate maps, and the final camera pose is recovered using a differentiable pose solver. Experiments on the 7Scenes and Cambridge Landmarks datasets show that UBGL achieves competitive localization accuracy, especially improving rotation estimation on the 7Scenes benchmark while maintaining comparable performance on Cambridge Landmarks. Ablation studies further demonstrate the effectiveness of bidirectional graph interaction and uncertainty-aware relation modeling.