Significance: The observation that cells sense and adapt to the physical stiffness of their environment is consistent across many different cell types and has profound implications in the final biological phenotype. In a diseased state or transformed cell, the substrate rigidity sensing and adaptation machinery is perturbed or outright abolished, which leads to progression of the disease, thus fully understanding this process is vital. Here, we found that the organization of the actin cytoskeleton in fibroblasts was governed by active and passive cross-linkers and the relative cell-to-substrate elasticity, and we present a phenomenological model detailing this. As cell rigidity sensing is a robust mechanism observed in various contexts, a generalized physics-based understanding such as presented here will yield universal insights.