Ship collision is the most common type of accident in the Northern Baltic Sea, posing a risk to the safety of maritime transportation. Near miss detection from automatic identification system (AIS) data provides insight into maritime transportation safety. Collision risk always triggers a ship to maneuver for safe passing. Some frenetic rudder actions occur at the last moment before ship collision. However, the relationship between ship behavior and collision risk is not fully clarified. Therefore, this work proposes a novel method to improve near miss detection by analyzing ship behavior characteristic during the encounter process. The impact from the ship attributes (including ship size, type, and maneuverability), perceived risk of a navigator, traffic complexity, and traffic rule are considered to obtain insights into the ship behavior. The risk severity of the detected near miss is further quantified into four levels. This proposed method is then applied to traffic data from the Northern Baltic Sea. The promising results of near miss detection and the model validity test suggest that this work contributes to the development of preventive measures in maritime management to enhance to navigational safety, such as setting a precautionary area in the hotspot areas. Several advantages and limitations of the presented method for near miss detection are discussed.