O-GlcNAc hydrolase (OGA) removes O-linked N-acetylglucosamine (O-GlcNAc) from a myriad of nucleocytoplasmic proteins. Through co-expression and assembly of OGA fragments, we determined the three-dimensional structure of human OGA, revealing an unusual helix-exchanged dimer that lays a structural foundation for an improved understanding of substrate recognition and regulation of OGA. Structures of OGA in complex with a series of inhibitors define a precise blueprint for the design of inhibitors that have clinical value.