Autonomous vehicles (AVs) will operate alongside human-driven vehicles for an extended transition period, during which aggressive human driving may become a major source of risk. This study proposes an integrated safety-control framework that combines real-world-data-driven behavior modeling with deep reinforcement learning to design longitudinal AV control strategies for mixed traffic. Aggressive, general, and defensive driving patterns are calibrated from the CitySim dataset, and dynamic aggressiveness is incorporated into an improved car-following model. A proximal policy optimization algorithm with a Kullback–Leibler penalty is then used to learn multi-objective strategies balancing safety, efficiency, comfort, and fuel economy in freeway and signalized-intersection scenarios. The results show that the behavior-aware strategies exhibit different strengths across traffic environments. On the freeway, the defensive-threshold strategy maintains a larger time headway, reduces positive acceleration, and lowers system-level fuel consumption, whereas the default, aggressive, and general strategies preserve higher traffic efficiency. At the intersection, signal control narrows the differences among strategies and limits the influence of longitudinal threshold settings on most evaluated indicators. These findings provide a quantitative basis for selecting behavior-aware control thresholds and designing robust AV strategies for mixed-autonomy traffic containing aggressive human drivers.