Eye diseases, a significant global health concern, require timely detection to prevent vision loss. The alarming prevalence of eye diseases necessitates immediate action through early diagnosis, making it urgent to develop an automatic detection system. Many researchers have been working to develop such systems. Yet, existing solutions still face difficulties in achieving high-performance accuracy due to challenges like lacking feature effectiveness, high computational demands, and incomplete disease coverage. To overcome these challenges, we proposed a novel eye-disease detection system leveraging multi-stage deep learning technologies. In the study, we employed a preprocessing approach to ensure the system’s robustness against rotation and translation, enhancing its effectiveness across varied conditions. Then, we employed a lightweight three-stage deep learning approach for extracting effective features and specific advantages. In the procedure, Stage 1 focuses on extracting fine-grained features using deep learning layers where the layers can automatically learn and identify complex patterns associated with various eye diseases, improving feature effectiveness and overall system accuracy. Then, we employed stage 2, which is constructed with two branches, each composed of convolutional blocks and identity blocks; this stage extracts hierarchical features by concatenating the outputs of the two branches. This hierarchical approach captures both low-level and high-level features, enhancing the extracted features’ richness and robustness and leading to better classification performance. We concatenated the two branch features that fed into the classification module, producing a probabilistic eye disease presence map. By converting hierarchical features into precise disease predictions, this stage ensures accurate probabilistic outputs, aiding better decision-making and diagnosis. We evaluated the proposed model with OCT2017, Dataset-101, and Retinal OCT C8 datasets, demonstrating an accuracy improvement of up to 1% over existing state-of-the-art models in both multi-class and binary classification tasks. The lightweight design and reduced computational requirements of the model highlight its applicability for real-world deployment, particularly in resource-constrained environments. This computer-aided detection system offers a meaningful advancement in the field of automatic eye disease detection by providing a more accurate and efficient tool that can be deployed widely.