A highly selective and sensitive method for Cd(II) detection was developed based on aptamer and gold nanoparticles (AuNPs) combined with a colorimetric smartphone readout. The experimental conditions such as reaction time of polydiene dimethyl ammonium chloride (PDDA) and AuNPs, PDDA dose, time of aptamer and PDDA incubation, and aptamer concentration were optimized. Under the optimized conditions, the color and red(R) value of the solution was concentration-dependent on Cd(II). The proposed method exhibited a linear range of 1–400 ng/mL (r² = 0.9794) with a limit of detection (LOD) of 1 ng/mL. This method had been successfully applied to test and quantify Cd(II) in water and rice samples, and the results were in full agreement with those from the atomic absorption spectrometer. Therefore, low-cost colorimetry demonstrated its potential for practical application in visual or quantitative detection with a smartphone. This approach can be readily applied to other analytes.