This paper demonstrates that the instrumented indentation test (IIT), together with a trained artificial neural network (ANN), has the capability to characterize the mechanical properties of the local parts of a welded steel structure such as a weld nugget or heat affected zone. Aside from force-indentation depth curves generated from the IIT, the profile of the indented surface deformed after the indentation test also has a strong correlation with the materials’ plastic behavior. The profile of the indented surface was used as the training dataset to design an ANN to determine the material parameters of the welded zones. The deformation of the indented surface in three dimensions shown in images were analyzed with the computer vision algorithms and the obtained data were employed to train the ANN for the characterization of the mechanical properties. Moreover, this method was applied to the images taken with a simple light microscope from the surface of a specimen. Therefore, it is possible to quantify the mechanical properties of the automotive steels with the four independent methods: (1) force-indentation depth curve; (2) profile of the indented surface; (3) analyzing of the 3D-measurement image; and (4) evaluation of the images taken by a simple light microscope. The results show that there is a very good agreement between the material parameters obtained from the trained ANN and the experimental uniaxial tensile test. The results present that the mechanical properties of an unknown steel can be determined by only analyzing the images taken from its surface after pushing a simple indenter into its surface.