Dense passage retrieval has recently boosted the performance of involved systems such as question answering or search engine. On this problem, prior works trained a dense retriever by learning to rank, i.e. ranking relevant/positive passages higher than irrelevant/negative ones. In this paper, we propose a Stratified Ranking approach for Dense Passage Retrieval (SR-DPR), which performs three-way ranking instead of the typical two-way positive-negative ranking. SR-DPR is concerned with three relevance levels, which are positive, hard negative and random/easy negative. We train SR-DPR model by minimizing a contrastive negative log-likelihood (NLL) loss involved with these three relevance levels, which is a finer-grained version of the N-pair loss [16]. To efficiently implement SR-DPR, we designed three data pipelines, each pipeline is used to learn the contrast between two out of three relevance levels. SR-DPR outperforms the strong baseline DPR [5] by 0.6-1.5% retrieval accuracy on Natural questions [6] dataset and 3-6% of that on Zalo Legal Text Retrieval dataset. SR-DPR also gives competitive results compared with current state-of-the-art methods without requiring complicated training regime or intensive hardware resources. The idea of stratified ranking of SR-DPR is not restricted to the scope of dense passage retrieval but can be applied in any contrastive learning problem. We conducted detailed ablation studies to give insights into SR-DPR’s behavior.