The stress response in the brain is not fully understood, although stress is one of the risk factors for developing mental disorders. On the other hand, the stimulation of the olfactory system can influence stress levels, and a certain smell has been empirically known to have a stress-suppressing effect, indeed. In this review, we first outline what stress is and previous studies on stress-responsive biomarkers (stress markers) in the brain. Subsequently, we confirm the olfactory system and review previous studies on the relationship between smell and stress response by species, such as humans, rats, and mice. Numerous studies demonstrated the stress-suppressing effects of aroma. There are also investigations showing the effects of odor that induce stress in experimental animals. In addition, we introduce recent studies on the effects of aroma of coffee beans and essential oils, such as lavender, cypress, α-pinene, and thyme linalool on the behavior and the expression of stress marker candidates in the brain. The transfer of volatile components into the brain is also discussed while using the results of thyme linalool as an example. These studies may provide a good opportunity to connect chemical research at the molecular level with neuropharmacological approaches in the future.