Targeting the immune checkpoint receptors cytotoxic T lymphocyte antigen 4 (CTLA-4), programmed cell death protein 1 (PD-1), or programmed cell death 1 ligand 1 (PD-L1) represents a very attractive treatment modality for tumor patients. The administration of antibodies against these receptors can promote efficient antitumor effects and can induce objective clinical responses in about 20–40% patients with various tumor types, accompanied by improved survival. Based on their therapeutic efficiency, several antibodies have been approved for the treatment of tumor patients. However, many patients do not respond to checkpoint inhibitor therapy. Therefore, the identification of biomarkers is required to guide patient selection for this treatment modality. Here, we summarize recent studies investigating the PD-L1 expression or mutational load of tumor tissues as well as the frequency and phenotype of immune cells in tumor patients prior to and during CTLA-4 or PD-1/PD-L1 inhibitor treatment.