While metal organic frameworks (MOFs) have been extensively explored as a platform for developing porous metal oxides, another intriguing direction is to use MOFs as precursors to prepare carbonaceous materials. By simple one-step carbonization, MOFs can be turned into promising hierarchical carbon materials. Such a technique can be also used to convert MOF-based composites to carbon-based composites with task-specific functionality different from that of the precursors. In this study, this strategy is adopted to prepare a magnetic cobalt–graphene (MCG) nanocomposite by carbonizing a self-assembly of a cobalt-based MOF, ZIF-67, and graphene oxide (GO). The preparation of MCG represents a simple alternative route to synthesize magnetic graphene materials and graphene-supported cobalt materials. By combining cobalt and reduced graphene oxide (RGO), the as-prepared MCG can be an effective catalyst to activate peroxymonosulfate (PMS) in the advanced oxidation process. Thus, the activation capability of MCG is evaluated by decolorizing acid yellow (AY) dye in water. MCG exhibited an enhanced catalytic activity to activate PMS compared to carbonized ZIF-67 because RGO also activated PMS and improved electron transport ability. The kinetics of the decolorization of AY (10 mg L⁻¹) was 0.0119 min⁻¹ with PMS = 200 mg L⁻¹ and MCG = 500 mg L⁻¹. The activation energy of the decolorization using PMS activated by MCG was found to be 12 kJ mol⁻¹. Factors influencing the PMS activation were also investigated including temperature, pH, UV, ultrasonication and inhibitors. To evaluate the long-term catalytic activity of MCG, a 50-cycle decolorization test was performed and the regeneration efficiency remained at 97.6% over 50 cycles, showing its stable and effective catalytic activity. These features make MCG a promising catalyst to activate PMS.