Background: Identifying cervical lymph node metastasis (LNM) in primary thyroid cancer preoperatively using ultrasound is challenging. Therefore, a non-invasive method is needed to assess LNM accurately. Purpose: To address this need, we developed the Primary Thyroid Cancer Lymph Node Metastasis Assessment System (PTC-MAS), a transfer learning-based and B-mode ultrasound images-based automatic assessment system for assessing LNM in primary thyroid cancer. Methods: The system has two parts: YOLO Thyroid Nodule Recognition System (YOLOS) for obtaining regions of interest (ROIs) of nodules, and LMM assessment system for building the LNM assessment system using transfer learning and majority voting with extracted ROIs as input. We retained the relative size features of nodules to improve the system’s performance. Results: We evaluated three transfer learning-based neural networks (DenseNet, ResNet, and GoogLeNet) and majority voting, which had the area under the curves (AUCs) of 0.802, 0.837, 0.823, and 0.858, respectively. Method III preserved relative size features and achieved higher AUCs than Method II, which fixed nodule size. YOLOS achieved high precision and sensitivity on a test set, indicating its potential for ROIs extraction. Conclusions: Our proposed PTC-MAS system effectively assesses primary thyroid cancer LNM based on preserving nodule relative size features. It has potential for guiding treatment modalities and avoiding inaccurate ultrasound results due to tracheal interference.