Rip currents pose significant threats to coastal safety and to autonomous marine vehicles operating nearshore. However, deploying high-precision monitoring models on resource-limited platforms remains challenging. This paper proposes a comprehensive framework that combines a novel architecture and compression methodology. First, we introduce RipSegNet, integrating a re-parameterized backbone network to efficiently extract multi-scale features from amorphous, low-contrast rip currents; a content-aware upsampling neck structure to preserve diffuse boundaries; and a lightweight shared convolutional segmenter to improve parameter efficiency. Second, we propose a collaborative two-stage compression pipeline: the first stage employs a task-aware, dimension-compensated macro-architecture pruning strategy; the second stage applies a knowledge distillation scheme guided by soft labels with a cosine annealing weight schedule. The final model, RipSegNet-Lite, achieves a Mask mAP of 0.919 with just 4.0 GFLOPs, reaching near lossless compression with over 58% compression. Qualitative analysis using heat maps and result masks, along with statistical significance evaluations confirm the model’s advantages in improving accuracy while reducing costs. This study provides an efficient, scalable solution for rip current monitoring, enabling reliable real-time instance segmentation on resource-limited platforms and enhancing coastal public safety and environmental awareness in autonomous marine systems.