Low-temperature embrittlement in the heat-affected zone (HAZ) of heavy-wall X80 weld joints is a primary challenge for arctic oil & gas exploitation. In this paper, the influence of intercritically reheated coarse-grained HAZ (ICCGHAZ) on the low-temperature toughness of the weld joint in a 22 mm thick X80 spiral submerged arc welded pipe was studied through instrumented Charpy V-notch impact test at −80~20 °C and corresponding fracture surface characterization. The results indicated that the influence of ICCGHAZ on the overall toughness of the weld joint is related to temperature. At temperatures below −45 °C, individual and tiny martensite-austenite (MA) constituent debonding can trigger cleavage fracture–which was proved to be nucleation-controlled–and the probability of embrittlement of the ICCGHAZ increases. At temperatures higher than −45 °C, only relatively large or closely distributed MA constituent in ICCGHAZ satisfies the conditions to trigger propagation-controlled cleavage fractures, and the influence of ICCGHAZ on the overall toughness is not remarkable.