Secondary lymphoid organs (SLOs) provide the confined microenvironment required for stromal cells to interact with immune cells to initiate adaptive immune responses resulting in B cell differentiation. Here, we studied three patients from two families with functional hyposplenism, absence of tonsils, and complete lymph node aplasia, leading to recurrent bacterial and viral infections. We identified biallelic loss-of-function mutations in LTBR, encoding the lymphotoxin beta receptor (LTβR), primarily expressed on stromal cells. Patients with LTβR deficiency had hypogammaglobulinemia, diminished memory B cells, regulatory and follicular T helper cells, and dysregulated expression of several tumor necrosis factor family members. B cell differentiation in an ex vivo coculture system was intact, implying that the observed B cell defects were not intrinsic in nature and instead resulted from LTβR-dependent stromal cell interaction signaling critical for SLO formation. Collectively, we define a human inborn error of immunity caused primarily by a stromal defect affecting the development and function of SLOs.