Cells, as well as the nuclei inside them, experience significant mechanical stress in diverse biological processes, including contraction, migration, and adhesion. The structural stability of nuclei must therefore be maintained in order to protect genome integrity. Despite extensive knowledge on nuclear architecture and components, however, the underlying physical and molecular mechanisms remain largely unknown. We address this by subjecting isolated human cell nuclei to microneedle-based quantitative micromanipulation with a series of biochemical perturbations of the chromatin. We find that the mechanical rigidity of nuclei depends on the continuity of the nucleosomal fiber and interactions between nucleosomes. Disrupting these chromatin features by varying cation concentration, acetylating histone tails, or digesting linker DNA results in loss of nuclear rigidity. In contrast, the levels of key chromatin assembly factors, including cohesin, condensin II, and CTCF, and a major nuclear envelope protein, lamin, are unaffected. Together with in situ evidence using living cells and a simple mechanical model, our findings reveal a chromatin-based regulation of the nuclear mechanical response and provide insight into the significance of local and global chromatin structures, such as those associated with interdigitated or melted nucleosomal fibers.