The spatial characteristics of cracks are significant indicators to assess and evaluate the health of existing buildings and infrastructures. However, the current manual crack description method is time consuming and labor consuming. To improve the efficiency of crack inspection, advanced computer vision‐based techniques have been utilized to detect cracks automatically at image level and grid‐cell level. But existing crack detections are of (high specificity) low generality and inefficient, in terms that conventional approaches are unable to identify and measure diverse cracks concurrently at pixel level. Therefore, this research implements a novel deep learning technique named fully convolutional network (FCN) to address this problem. First, FCN is trained by feeding multiple types of cracks to semantically identify and segment pixel‐wise cracks at different scales. Then, the predicted crack segmentations are represented by single‐pixel width skeletons to quantitatively measure the morphological features of cracks, providing valuable crack indicators for assessment in practice, such as crack topology, crack length, max width, and mean width. To validate the prediction, the predicted segmentations are compared with recent advanced method for crack recognition and ground truth. For crack segmentation, the accuracy, precision, recall, and F1 score are 97.96%, 81.73%, 78.97%, and 79.95%, respectively. For crack length, the relative measurement error varies from −48.03% to 177.79%, meanwhile that ranges from −13.27% to 24.01% for crack width. The results show that FCN is feasible and sufficient for crack identification and measurement. Although the accuracy is not as high as CrackNet because of three types of errors, the prediction has been increased to pixel level and the training time has been dramatically decreased to several per cents of previous methods due to the novel end‐to‐end structure of FCN, which combines typical convolutional neural networks and deconvolutional layers.