Monitoring grassland vegetation growth is of vital importance to scientific grazing and grassland management. People expect to be able to use a portable device, like a mobile phone, to monitor grassland vegetation growth at any time. In this paper, we propose a handheld grassland vegetation monitoring system to achieve the goal of monitoring grassland vegetation growth. The system includes two parts: the hardware unit is a hand-held multispectral imaging tool named ASQ-Discover based on a smartphone, which has six bands (wavelengths)—including three visible bands (450 nm, 550 nm, 650 nm), a red-edge band (750 nm), and two near-infrared bands (850 nm, 960 nm). The imagery data of each band has a size of 5120 × 3840 pixels with 8-bit depth. The software unit improves image quality through vignetting removal, radiometric calibration, and misalignment correction and estimates and analyzes spectral traits of grassland vegetation (Fresh Grass Ratio (FGR), NDVI, NDRE, BNDVI, GNDVI, OSAVI and TGI) that are indicators of vegetation growth in grassland. We introduce the hardware and software unit in detail, and we also experiment in five pastures located in Haiyan County, Qinghai Province. Our experimental results show that the handheld grassland vegetation growth monitoring system has the potential to revolutionize the grassland monitoring that operators can conduct when using a hand-held tool to achieve the tasks of grassland vegetation growth monitoring.