This paper presents a novel silicon-based piezoresistive pressure sensor composed of a silicon layer with sensing elements and a glass cover for hermetic packaging. Unlike conventional designs, this study employs numerical simulation to analyze the influence of varying roughness levels of the sensitive membrane on the sensor’s output response. Simulation results demonstrate that pressure sensors with smoother sensitive membranes exhibit superior performance in terms of sensitivity (5.07 mV/V/MPa), linearity (0.67% FS), hysteresis (0.88% FS), and repeatability (0.75% FS). Furthermore, an optimized process for controlling membrane roughness was achieved by adjusting the concentration of the etchant solution. Experimental results reveal that a membrane roughness of 35.37 nm was attained under conditions of 80 °C and 25 wt% TMAH. Additionally, the fabrication process of this piezoresistive pressure sensor was significantly simplified and cost-effective due to the adoption of a backside wet etching technique. The fabricated sensor demonstrates excellent performance metrics, including a sensitivity of 5.07 mV/V/MPa, a full-scale (FS) output of 101.42 mV, a hysteresis of 0.88% FS, a repeatability of 0.75% FS, and a nonlinearity of 0.67% FS. These results indicate that the proposed sensor is a promising tool for precise pressure measurement applications, offering both high performance and cost efficiency. This study not only advances the understanding of the impact of membrane roughness on sensor performance but also provides a practical and scalable fabrication approach for piezoresistive pressure sensors.