Remote sensing estimation based on the dimidiate pixel model (DPM) using vegetation indices (VIs) is a common approach for mapping fractional vegetation cover (FVC). The major drawback of DPM is that it does not consider real endmember conditions and multiple scattering between soil and vegetation. An analysis of FVC uncertainties caused by these model deficiencies is still lacking. Here, we first calculated the FVC theoretical uncertainty caused by reflectance uncertainties based on the law of prapagation of uncertainty (LPU). Then, we tested the performance of DPM using six VIs over 3-D forest scenes. We simulated both Aqua-MODIS and Landsat-OLI surface reflectance (SR) at their corresponding spatial resolutions and spectral response functions (SRFs) using a well-validated 3-D radiative transfer (RT) model which helps to separate the model and input uncertainties. We found that ratio vegetation index (RVI)- and enhanced vegetation index (EVI)-based models were most affected by sensors, followed by the normalized difference vegetation index (NDVI)-, enhanced vegetation index 2 (EVI2)-, renormalized difference vegetation index (RDVI)-, and difference vegetation index (DVI)-based models. Without considering SR uncertainties, the DVI-based model performed best (FVC absolute difference < 0.1); however, the commonly used NDVI model reached a maximum difference of 0.35. At the same time, input uncertainty increased the uncertainty of FVC retrieval. We noticed that the increase of solar zenith angle (SZA) resulted in a clear increase of retrieved FVC under the uniform distribution, which can be explained by the increased shadow proportion. Besides, model accuracy was dominated by the purity of soil (vegetation) endmember in low (high) vegetation cover area. This study provides a reference for the selection of the optimal VI for FVC retrieval based on the DPM.