In the complex and variable marine environment, the navigation and localization of autonomous underwater vehicles (AUVs) are very important and challenging. When the conventional Kalman filter (KF) is applied to the cooperative localization of leader–follower AUVs, the outliers in the sensor observations will have a substantial adverse effect on the localization accuracy of the AUVs. Meanwhile, inaccurate noise covariance matrices may result in significant estimation errors. In this paper, we proposed an improved Sage–Husa adaptive extended Kalman filter (improved SHAEKF) for the cooperative localization of multi-AUVs. Firstly, the measurement anomalies were evaluated by calculating the Chi-square test statistics based on the innovation. The detection threshold was determined according to the confidence level of the Chi-square test, and the Chi-square test statistics exceeding the threshold were regarded as measurement abnormalities. When measurement anomalies occurred, the Sage–Husa adaptive extended Kalman filter algorithm was improved by suboptimal maximum a posterior estimation using weighted exponential fading memory, and the measurement noise covariance matrix was adjusted online. The numerical simulation of leader–follower multi-AUV cooperative localization verified the effectiveness of the improved SHAEKF and demonstrated that the average root mean square and the average standard deviation of the localization errors based on the improved SHAEKF were significantly reduced in the case of the presence of measurement abnormalities.