Background: Immune checkpoint inhibitors (ICIs) have shown promising anti-tumor activities and are widely used for the treatment of advanced cancers. However, they may lead to immune-related adverse events (irAEs) and some of them, such as hypophysitis, can be life-threatening. Here, early diagnosis is critical. Methods: We retrospectively analyzed 40 melanoma patients who developed hypophysitis during ICI treatment with either ipilimumab and/or anti-PD1 therapy and compared them to 40 control patients who did not develop hypophysitis during the ICI treatment, matched for age, gender, type of immunotherapy, and stage. Clinical data and blood values such as LDH, CRP, TSH, T3, T4, and absolute immune cell counts were retrieved from the medical records. Patient characteristics, laboratory values, progression-free survival, and overall survival were compared between the two groups. Results: Patients with ir-hypophysitis had a median age of 59 years, and most of them were male. Clinically, frequent symptoms were fatigue, headache, dizziness, and gastrointestinal symptoms such as nausea or abdominal pain. The onset of ir-hypophysitis differed much between ipilimumab- (median 8 weeks) and anti-PD1 (median 40 weeks)-induced hypophysitis (p < 0.001). At baseline, besides a slightly increased CRP level (p = 0.06), no differences were observed in patients who later developed hypophysitis compared to the control. After treatment started, hypophysitis patients showed a constant and significant decline in T4 levels from the start of therapy until diagnosis (p < 0.05), independent of the ICI treatment regime. However, a decline in T3 and TSH was only noted in patients with ipilimumab-induced ir-hypophysitis. Furthermore, serum sodium levels declined rapidly at the diagnosis of hypophysitis (p < 0.001). In addition, there was a constant increase in the absolute counts of eosinophils and lymphocytes from baseline in hypophysitis patients (p < 0.05). Conclusion: Ir-hypophysitis reveals different clinical pictures and onset times depending on the ICI regime used. Whereas a drop in T4 levels was indicative of developing hypophysitis independent of the ICI regime, TSH levels only declined in patients under ipilimumab-based ICI regimes. To best monitor our patients, it is important to recognize these differences.