Well, I guess I can now answer my own question!
Found that I can use indinispan as the cache and went with that. With Infinispan, you can set up expiry time and lifespan for cached entries. So that solves this problem. The default cache is just a bounded map with 1000 max entries.