The thermal conductivity of thermal insulation materials directly affects the building energy consumption. The types and constituents of thermal insulation materials in thermal insulation boards are the key to determining the insulation performance. By optimizing the material constituents and ratios, this paper proposes an improved graphite composite insulation board (GCIB), which has lower thermal conductivity and good fire resistance. Through theoretical derivation, it is found that the limit range of the thermal conductivity of the new GCIB is 0.042–0.064 W/(m · K). Combined with the results of theoretical value analysis, and according to the ratios of material components, the random distribution function of each material component is constructed, and the numerical model of GCIB is established. Through numerical analysis, the range of thermal conductivity of the new composite insulation board is 0.046–0.050 W/(m · K). Finally, we establish an experimental model of the new GCIB. Through the model test of six GCIBs, the thermal conductivity of the new GCIB is obtained as 0.046 W/(m · K), which is in good agreement with the results of theoretical analysis and numerical simulation. Through theoretical analysis, numerical simulation and a sample test, this paper verifies the better thermal insulation performance of the improved GCIB, providing theoretical and numerical simulation methods for the new GCIB, as well as a theoretical reference for the promotion and application of the GCIB.