The shallow, soft subsea formations, characterized by low strength and poor stability, lead to complex interactions between the screw motor drilling tool and the wellbore wall during directional drilling, complicating the accurate evaluation of the tool’s deflection capability. To address this issue, this paper proposes an integrated mechanical analysis method combining three-dimensional finite element analysis and transient dynamic analysis. By establishing a finite element model using 12-DOF (degree-of-freedom) spatial rigid-frame Euler–Bernoulli beam elements, coupled with well trajectory coordinate transformation and Rayleigh damping matrix, a precise description of drill string dynamic behavior is achieved. Furthermore, the introduction of pipe–soil dynamics and the p-y curve method improves the calculation of contact reaction forces between drilling tools and formation. Case studies demonstrate that increasing the tool face rotation angle intensifies lateral forces at the bit and stabilizer, with the predicted maximum dogleg severity within the first 10 m ahead of the bit progressively increasing. When the tool face rotation angle exceeds 2.5°, the maximum dogleg severity reaches 17.938°/30 m. With a gradual increase in the drilling pressure, the maximum bending stress on the drilling tool, maximum lateral cutting force, and stabilizer lateral forces progressively decrease, while vertical cutting forces and bit lateral forces gradually increase. However, the predicted maximum dogleg severity increases within the first 10 m ahead of the bit remain relatively moderate, suggesting the necessity for the multi-objective optimization of drilling pressure and related parameters prior to actual operations.