Group keys by their access frequency. You maintain a map from each frequency to an ordered collection of keys. The ordering within each group handles recency.
When you access a key, remove it from its current frequency group and insert it into the next one. This "promotion" is if you use a linked list or ordered dictionary per group.
You also track minFreq, the lowest active frequency. When you evict, grab the oldest entry from the minFreq group. If that group empties, the next insertion resets minFreq to , so it stays correct.
Three hash maps work together: one maps keys to values, another maps keys to frequencies, and the third maps each frequency to its ordered key set.