We examined the night-to-night associations of evening use of alcohol, caffeine, and nicotine with actigraphically estimated sleep duration, sleep efficiency, and wake after sleep onset (WASO) among a large cohort of African American adults. Participants in the Jackson Heart Sleep Study underwent wrist actigraphy for an average of 6.7 nights and completed concurrent daily sleep diary assessments to record any consumption of alcohol, caffeine, and nicotine within 4 hours of bedtime. Linear mixed-effect models were fit and adjusted for age, sex, educational attainment, body mass index, depression, anxiety, stress, and having work/school the next day. Eligible participants (n = 785) were an average of 63.7 years (SD: 10.6), and were predominantly female (67.9%). There were 5164 days of concurrent actigraphy and sleep diary data. Evening alcohol use was associated with that night’s lower sleep efficiency (−0.98% [95% CI: −1.67% to −0.29%], p = 0.005), but not with WASO or sleep duration. Evening nicotine use was associated with that night’s lower sleep efficiency [1.74% (95% CI: −2.79 to −0.68), p = 0.001] and 6.09 minutes higher WASO ([95% CI: 0.82 to 11.35], p = 0.02), but was not associated with sleep duration. Evening caffeine use was not associated with any of the sleep parameters. Nicotine and alcohol use within 4 hours of bedtime were associated with increased sleep fragmentation in the associated night, even after controlling for multiple potential confounders. These findings support the importance of sleep health recommendations that promote the restriction of evening alcohol and nicotine use to improve sleep continuity.