AZ31 magnesium alloy sheets were prepared by low-speed extrusion at different temperatures, i.e., 350 °C, 400 °C, and 450 °C. The microstructure evolution and mechanical properties of extruded AZ31 magnesium alloy sheets were studied. Results indicate that the low-speed extrusion obviously improved the microstructure of magnesium alloys. As the extrusion temperature decreased, the grain size for the produced AZ31 magnesium alloy sheets decreased, and the (0001) basal texture intensity of the extruded sheets increased. The yield strength and tensile strength of the extruded sheets greatly increased as the extrusion temperature decreased. The AZ31 magnesium alloy sheet prepared by low-speed extrusion at 350 °C exhibited the finest grain size and the best mechanical properties. The average grain size, yield strength, tensile strength, and elongation of the extruded sheet prepared by low-speed extrusion at 350 °C were ~2.7 μm, ~226 MPa, ~353 MPa, and ~16.7%, respectively. These properties indicate the excellent mechanical properties of the extruded sheets prepared by low-speed extrusion. The grain refinement effect and mechanical properties of the extruded sheets produced in this work were obviously superior to those of magnesium alloys prepared using traditional extrusion or rolling methods reported in other related studies.