Developing reliable railway fault detection systems is crucial for ensuring both safety and operational efficiency. Various artificial intelligence frameworks, especially deep learning models, have shown significant potential in enhancing fault detection within railway infrastructure. This study explores the application of deep learning models for railway fault detection, focusing on both transfer learning architectures and a novel classification framework. Transfer learning was utilized with architectures such as ResNet50V2, Xception, VGG16, MobileNet, and InceptionV3, which were fine-tuned to classify railway track images into defective and non-defective categories. Additionally, the state-of-the-art YOLOv11 model was adapted for the same classification task, leveraging advanced data augmentation techniques to achieve high accuracy. Among the transfer learning models, VGG16 demonstrated the best performance with a test accuracy of 89.18%. However, YOLOv11 surpassed all models, achieving a test accuracy of 92.64% while maintaining significantly lower computational demands. These findings underscore the versatility of deep learning models and highlight the potential of YOLOv11 as an efficient and accurate solution for railway fault classification tasks.