Automating industrial processes, particularly quality inspection, is a key objective in manufacturing. While welding tasks are frequently automated, inspection processes remain largely manual. Advances in computer vision and AI, especially ViTs, now enable more effective defect detection and classification, offering opportunities to automate these workflows. This study evaluates ViTs for identifying defects in aluminum welding using the Aluminum 5083 TIG dataset. The analysis spans binary classification (detecting defects) and multiclass categorization (Good Weld, Burn Through, Contamination, Lack of Fusion, Misalignment, and Lack of Penetration). ViTs achieved 98% to 99% accuracy across both tasks, significantly outperforming prior models such as dense and CNNs, which struggled to surpass 80% accuracy in binary and 70% in multiclass tasks. These results, achieved with datasets of 2400 to 8000 images, highlight ViTs’ efficiency even with limited data. The findings underline the potential of ViTs to enhance manufacturing inspection processes by enabling faster, more reliable, and cost-effective automated solutions, reducing reliance on manual inspection methods.