Alternaria is a ubiquitous fungal genus in many ecosystems, consisting of species and strains that can be saprophytic, endophytic, or pathogenic to plants or animals, including humans. Alternaria species can produce a variety of secondary metabolites (SMs), especially low molecular weight toxins. Based on the characteristics of host plant susceptibility or resistance to the toxin, Alternaria phytotoxins are classified into host-selective toxins (HSTs) and non-host-selective toxins (NHSTs). These Alternaria toxins exhibit a variety of biological activities such as phytotoxic, cytotoxic, and antimicrobial properties. Generally, HSTs are toxic to host plants and can cause severe economic losses. Some NHSTs such as alternariol, altenariol methyl-ether, and altertoxins also show high cytotoxic and mutagenic activities in the exposed human or other vertebrate species. Thus, Alternaria toxins are meaningful for drug and pesticide development. For example, AAL-toxin, maculosin, tentoxin, and tenuazonic acid have potential to be developed as bioherbicides due to their excellent herbicidal activity. Like altersolanol A, bostrycin, and brefeldin A, they exhibit anticancer activity, and ATX V shows high activity to inhibit the HIV-1 virus. This review focuses on the classification, chemical structure, occurrence, bioactivity, and biosynthesis of the major Alternaria phytotoxins, including 30 HSTs and 50 NHSTs discovered to date.