The performance of multiple-input single-output (MISO) transmission is highly dependent on the accuracy of the channel state information (CSI) at the base station (BS), which necessitates precise CSI estimation and reliable feedback from the user equipment. However, the overhead of the CSI feedback occupies substantial uplink bandwidth resources. To alleviate the overhead, this paper proposes a novel task-based quantizer for uplink MISO visible light communication (VLC) systems. In particular, a hybrid radio frequency (RF)/VLC system is considered, where VLC links are mainly used for large-volume downlink transmissions and RF links are used for uplink CSI feedback. Since the RF bandwidth resources are limited, the CSI is quantified to reduce the uplink resource requirements, which, however, inevitably causes CSI estimation errors at the BS. To guarantee the CSI estimation accuracy while minimizing the RF resource cost, a task-based quantization scheme for channel estimation (TQ-CE) is proposed. In the TQ-CE, both the quantized codebook and the post-processing matrix are optimized to minimize the mean square error (MSE) of the channel estimation. Taking the minimum MSE as the target task, the TQ-CE leverages vector quantization (VQ) to generate a codebook, which is designed to reduce the feedback overhead without compromising the precision of the channel estimation. Then, an optimal closed-form solution of the post-processing matrix is derived based on the minimum mean square error (MMSE) criterion. The simulation results demonstrate that the proposed TQ-CE achieved $0.25\mathrm{Mbit}/\mathrm{s}$ and $0.62\mathrm{Mbit}/\mathrm{s}$ higher data rates compared with the conventional scalar quantization-based channel estimation (SQ-CE) schemes and vector quantization-based channel estimation (VQ-CE) schemes, respectively. Moreover, in terms of the feedback overhead, compared with the 18-bit SQ-CE, the 4-bit TQ-CE achieved a 22.2% reduction in uplink bits.