Sheep face recognition models deployed on edge devices require a good trade-off between model size and accuracy, but the existing recognition models cannot do so. To solve the above problems, this paper combines Mobilenetv2 with Vision Transformer to propose a balanced sheep face recognition model called MobileViTFace. MobileViTFace enhances the model’s ability to extract fine-grained features and suppress the interference of background information through Transformer to distinguish different sheep faces more effectively. Thus, it can distinguish different sheep faces more effectively. The recognition accuracy of 96.94% is obtained on a self-built dataset containing 5490 sheep face photos of 105 sheep, which is a 9.79% improvement compared with MobilenetV2, with only a small increase in Params (the number of parameters) and FLOPs (floating-point operations). Compared to models such as Swin-small, which currently performs SOTA, Params and FLOPs are reduced by nearly ten times, whereas recognition accuracy is only 0.64% lower. Deploying MobileViTFace on the Jetson Nano-based edge computing platform, real-time and accurate recognition results are obtained, which has implications for practical production.