Over the last few years, the directed self-assembly of block copolymers by surface patterns has transitioned from academic curiosity to viable contender for commercial fabrication of next-generation nanocircuits by lithography. Recently, it has become apparent that kinetics, and not only thermodynamics, plays a key role for the ability of a polymeric material to self-assemble into a perfect, defect-free ordered state. Perfection, in this context, implies not more than one defect, with characteristic dimensions on the order of 5 nm, over a sample area as large as 100 cm². In this work, we identify the key pathways and the corresponding free energy barriers for eliminating defects, and we demonstrate that an extraordinarily large thermodynamic driving force is not necessarily sufficient for their removal. By adopting a concerted computational and experimental approach, we explain the molecular origins of these barriers and how they depend on material characteristics, and we propose strategies designed to overcome them. The validity of our conclusions for industrially relevant patterning processes is established by relying on instruments and assembly lines that are only available at state-of-the-art fabrication facilities, and, through this confluence of fundamental and applied research, we are able to discern the evolution of morphology at the smallest relevant length scales—a handful of nanometers—and present a view of defect annihilation in directed self-assembly at an unprecedented level of detail. Significance A molecular model is used to calculate the free energy of formation of ordered and disordered copolymer morphologies. We rely on advanced methodologies to identify the minimum free energy pathways that connect such states of the material. Our predictions for defect formation and annealing are compared with experimental observations. Our results provide a detailed molecular view of isolated block copolymer defects, which measure approximately 5 nm and represent isolated events in large areas. They are true “needles in the hay stack” that can only be studied by concerted molecular simulations and dedicated access to production-level fabrication tools. We show that defect annealing is an activated process, where defects are eliminated by operating near the order−disorder transition.