Field studies on traffic noise-induced annoyance have predominantly used estimated outside noise levels. We intended to complement existing knowledge with exposure–response relationships that are based on precise indoor noise measurements. Acoustic recordings inside the bedrooms of nightly road traffic and annoyance ratings in the following morning were obtained from 40 suburban residents (mean age 29.1 years ± 11.7; 26 females). We derived exposure–response functions for the probability to be “annoyed at least a little” (%LA). Further analyses compared data from the current study with those from two earlier studies on railway and aircraft noise. Annoyance increased with the number of traffic events and the equivalent sound pressure level. The inclusion of non-acoustical factors (such as assessment of road transport) improved the prediction considerably. When comparing the different traffic noise sources, %LA was higher for road than for air traffic at a given L_Aeq,night, but higher for road and railway than for air traffic at a given number of noise events. Acoustical as well as non-acoustical factors impact short-term annoyance induced by road, railway, and air traffic. Annoyance varies across noise sources, which may be due to differences in acoustical characteristics or in the temporal noise distribution throughout the night.