The mechanisms of lettuce growth in plant factories under artificial light (PFALs) are well known, whereby the crop is generally used as a model in horticultural science. Deep learning has also been tested several times using PFALs. Despite their numerous advantages, the performance of deep learning models is commonly evaluated based only on their accuracy. Therefore, the objective of this study was to train deep neural networks and analyze the deeper abstraction of the trained models. In total, 443 images of three lettuce cultivars were used for model training, and several deep learning algorithms were compared using multivariate linear regression. Except for linear regression, all models showed adequate accuracies for the given task, and the convolutional neural network (ConvNet) model showed the highest accuracy. Based on color mapping and the distribution of the two-dimensional t-distributed stochastic neighbor embedding (t-SNE) results, ConvNet effectively perceived the differences among the lettuce cultivars under analysis. The extension of the target domain knowledge with complex models and sufficient data, similar to ConvNet with multitask learning, is possible. Therefore, deep learning algorithms should be investigated from the perspective of feature extraction.