Cache side channel attacks, as a type of cryptanalysis, seriously threaten the security of the cryptosystem. These attacks continuously monitor the memory addresses associated with the victim’s secret information, which cause frequent memory access on these addresses. This paper proposes CacheHawkeye, which uses the frequent memory access characteristic of the attacker to detect attacks. CacheHawkeye monitors memory events by CPU hardware performance counters. We proved the effectiveness of CacheHawkeye on Flush+Reload and Flush+Flush attacks. In addition, we evaluated the accuracy of CacheHawkeye under different system loads. Experiments demonstrate that CacheHawkeye not only has good accuracy but can also adapt to various system loads.