In mammals, interactions between sequences within topologically associating domains enable control of gene expression across large genomic distances. Yet it is unknown how frequently such contacts occur, how long they last and how they depend on the dynamics of chromosome folding and loop extrusion activity of cohesin. By imaging chromosomal locations at high spatial and temporal resolution in living cells, we show that interactions within topologically associating domains are transient and occur frequently during the course of a cell cycle. Interactions become more frequent and longer in the presence of convergent CTCF sites, resulting in suppression of variability in chromosome folding across time. Supported by physical models of chromosome dynamics, our data suggest that CTCF-anchored loops last around 10 min. Our results show that long-range transcriptional regulation might rely on transient physical proximity, and that cohesin and CTCF stabilize highly dynamic chromosome structures, facilitating selected subsets of chromosomal interactions.