Nickel oxide (NiO) is a wide band gap semiconductor material that is used as an electrochromic layer or an ion storage layer in electrochromic devices. In this work, the effect of annealing temperature on sol-gel NiO films was investigated. Fourier transform infrared spectroscopy (FTIR) showed that the formation of NiO via decomposition of the precursor nickel acetate occurred at about 300 °C. Meanwhile, an increase in roughness was observed by Atomic force microscope (AFM), and precipitation of a large number of crystallites was observed at 500 °C. X-ray Diffraction (XRD) showed that the NiO film obtained at such a temperature showed a degree of crystallinity. The film crystallinity and crystallite size also increased with increasing annealing temperature. An ultraviolet spectrophotometer was used to investigate the optical band gap of the colored NiO films, and it was found that the band gap increased from 3.65 eV to 3.74 eV with the increase in annealing temperature. An electrochromic test further showed that optical modulation density and coloring efficiency decreased with the increase in crystallite size. The electrochromic reaction of the nickel oxide film is more likely to occur at the crystal interface and is closely related to the change of the optical band gap. An NiO film with smaller crystallite size is more conducive to ion implantation and the films treated at 300 °C exhibit optimum electrochromic behavior.