Both infectious hematopoietic necrosis virus (IHNV) and infectious pancreatic necrosis virus (IPNV) are the causative agents of acute and highly contagious diseases of juvenile salmonids, resulting in severe economic losses to these cold-water fish globally. There is an urgent need to explore antiviral agents against IHNV and IPNV due to the lack of commercially available vaccines and antiviral drugs. More importantly, the co-infection of IHNV and IPNV is prevalent in nature, which not only aggravates extensive damage to the salmonids but also poses challenges to its prevention and control. The antiviral effects of a crude polysaccharide derived from seaweed (CSP) on IHNV and IPNV were evaluated in this study separately. Furthermore, the underlying antiviral mechanisms of CSP to IHNV and IPNV were analyzed, respectively. The results showed that CSP possessed excellent safety and good ability to inhibit IHNV, IPNV, and their co-infection. CSP preferred to act at the early stage of viral infection. The antiviral mechanism of CSP on IHNV is possibly involved in preventing viral attachment and release, while in IPNV, it is involved in suppressing viral attachment, entry, and release. Taken together, the results of this study shed new light on developing novel agents against viral infection in salmonid fish.