To investigate the influence of polymer matrices on the tensile and impact properties of long fiber-reinforced thermoplastic (LFT) composites, composites of long basalt fiber-reinforced thermoplastic were developed in this work. Two types of polyethylene, namely 8008 and 100S, and two types of polyethylene, namely C4220 and K8303, were chosen as the matrices. The fiber volume fractions were set as 2.8%, 5.9%, 8.1%, and 10.6%. The melt flow index (MFI), crystallinity, tensile properties, impact strength, and fracture morphology of the neat polymers and the corresponding composites were tested. The composites of 8008 showed the highest tensile strength since neat 8008 showed a much higher MFI value and crystallinity. The composites of 8008 and K8303 showed a much higher tensile modulus since the neat thermoplastic showed a higher tensile modulus than the other two composites. The polymer toughness was the factor that determined whether the polymer could be toughened by fibers. Moreover, the interfacial shear strength was calculated and compared with the matrix shear strength, based on which fracture modes of the LFT were predicted. Effective methods were proposed for further improvement of the mechanical properties. The results of this paper were essential for attaining the anticipated properties when designing LFT composites.