The aim of the present study was to elucidate the anti-diabetic effects of the crude polysaccharide and rhamnose-enriched polysaccharide derived from G. lithophila on streptozotocin (STZ)-induced diabetic Wistar rats. Treatment with crude polysaccharide and rhamnose-enriched polysaccharide showed increases in body weight and pancreatic insulin levels and a decrease in blood glucose levels compared with control diabetic rats. The blood concentrations of total cholesterol (TC), triglycerides (TGs), low-density lipoprotein (LDL) and very-low-density lipoprotein (VLDL) decreased, and high-density lipoprotein (HDL) increased both in the crude polysaccharide- and rhamnose-enriched polysaccharide-treated rats. Superoxide dismutase (SOD) and glutathione peroxidase (GPx) levels increased, and malondialdehyde (MDA) levels decreased in the livers, kidneys and pancreases of crude polysaccharide- and rhamnose-enriched polysaccharide-treated rats. Immunohistological examination further confirmed that restoration of the normal cellular size of the islets of Langerhans and the rebirth of β-cells were found to be greater in the body region than in the head and tail regions of the pancreas. The crude polysaccharide- and rhamnose-enriched polysaccharide-treated diabetic rats showed normal blood glucose levels and insulin production, and reversed cholesterol levels and enzymatic actions. Therefore, rhamnose-enriched polysaccharide from G. lithophila acts as a potent anti-diabetic agent to treat diabetes and can lead to the development of an alternative medicine for diabetes in the future.