Electricity production from photovoltaic (PV) systems has accelerated in the last few decades. Numerous environmental factors, particularly the buildup of dust on PV panels have resulted in a significant loss in PV energy output. To detect the dust and thus reduce power loss, several techniques are being researched, including thermal imaging, image processing, sensors, cameras with IoT, machine learning, and deep learning. In this study, a new dataset of images of dusty and clean panels is introduced and applied to the current state-of-the-art (SOTA) classification algorithms. Afterward, a new convolutional neural network (CNN) architecture, SolNet, is proposed that deals specifically with the detection of solar panel dust accumulation. The performance and results of the proposed SolNet and other SOTA algorithms are compared to validate its efficiency and outcomes where SolNet shows a higher accuracy level of 98.2%. Hence, both the dataset and SolNet can be used as benchmarks for future research endeavors. Furthermore, the classes of the dataset can also be expanded for multiclass classification. At the same time, the SolNet model can be fine-tuned by tweaking the hyperparameters for further improvements.