Air pollution over the Seto Inland Sea (SIS) is among the most severe of any region in Japan and is considered to be affected by both long-range and local pollution. To unravel the long-term trends of aerosol pollution over this region, in this study, measurements from the moderate resolution imaging spectroradiometer instrument onboard the Terra satellite were analyzed over two decades, from 2001 to 2020. Fine-mode aerosol optical depth (AOD_f) was calculated to estimate the amount of aerosol produced by anthropogenic emissions. The results showed that the AOD_f over the SIS increased from 2001 to 2004, had a flat trend from 2005 to 2009, and decreased from 2010 to 2020. To clarify the impact of long-range transport from the Asian continent to the SIS, the AOD_f over the Yellow Sea was also investigated and was found to increase and level off during the 2000s, after which it decreased, especially after 2014. This decrease can be attributed to emission regulations in China. The above analysis suggests that the aerosol pollution status in the SIS during the late 2010s was similar to that during the early 2000s. Over the SIS, the lowest AOD_f value was found in 2020, with the values in January–March and June–July approximately 30% and 30%–60% lower than the average values during the same periods in 2018–2019, respectively. The reduction found in January–March could be related to the decline in the long-range transport with restrictions on human activity due to the COVID-19 pandemic. Meanwhile, the reduction during June–July could be related to the decline of local emission sources. Considering the large SO₂ decline in 2020, regulations on SO₂ emitted from ships that started from 1 January 2020 are one possible factor for the improvement of aerosol pollution over the SIS in 2020.