Through-focus scanning optical microscopy (TSOM) is one of the recommended measurement methods in semiconductor manufacturing industry in recent years because of its rapid and nondestructive properties. As a computational imaging method, TSOM takes full advantage of the information from defocused images rather than only concentrating on focused images. In order to improve the accuracy of TSOM in nanoscale dimensional measurement, this paper proposes a two-input deep-learning TSOM method based on Convolutional Neural Network (CNN). The TSOM image and the focused image are taken as the two inputs of the network. The TSOM image is processed by three columns convolutional channels and the focused image is processed by a single convolution channel for feature extraction. Then, the features extracted from the two kinds of images are merged and mapped to the measuring parameters for output. Our method makes effective use of the image information collected by TSOM system, for which the measurement process is fast and convenient with high accuracy. The MSE of the method can reach 5.18 nm² in the measurement of gold lines with a linewidth range of 247–1010 nm and the measuring accuracy is much higher than other deep-learning TSOM methods.