Multi-objective Bayesian optimization (MOBO) is a powerful framework for optimizing black-box functions with conflicting objectives and costly evaluations. A key component in MOBO is the probabilistic surrogate model used to guide the search for the optimum, which in practice is most often implemented using independent Gaussian processes (GPs), one per objective. A less common but more powerful approach is to use multi-output GPs to exploit correlations between objectives. However, with this model, independent observation noise across objectives is typically assumed, which ignores part of the correlation structure. In this work, we focus on multi-output GPs that can also capture correlations in the observational noise, and study how the presence and structure of noise affect the joint predictive distribution, the NEHVI acquisition function, and ultimately the optimization performance. Through theoretical analysis and extensive synthetic and real-world experiments, we show that multi-output GPs can improve optimization efficiency when evaluations are noisy. Importantly, however, when the evaluations are noiseless, we observe, both theoretically and empirically, only small gains by using multi-output GPs, and cheaper independent GPs are preferred. Furthermore, our synthetic experiments show that explicitly modeling noise correlations provides additional advantages when the objectives themselves are negatively correlated. On the other hand, in real-world experiments, where model bias may be relevant, we observe that modeling noise correlations has little impact on the optimization results. These findings provide practical insights into when and how to use multi-output GP models in the context of MOBO.