Our process was pretty straight
forward. We (ok, I) picked memcache for three reasons:
1) Automatic expiration of records means I don't have to do my own
garbage collection.
2) AWS offered ElastiCache which was built off of Memcache and
could allow us to move to Couchbase if needed (it's not)
3) Our Ops folk suggested Memcache because they've had the least
number of problems dealing with it.
Since I started working on things, a few other things have
happened:
1) AWS now offers a managed Redis service
2) AWS became a bit more reliable in a subsequent release.
(Haven't done a lot of rigorous checks there, since most of the
problems come about because Redis REALLY doesn't like it when
memory gets full.)
3) Some of the storage requirements changed.
Ah, the joys of initial implementations of a service. This is why
abstraction is a dear, sweet friend of mine.
Honestly, the thing I'm having the hardest time trying to work out
is the optimal way that records should be stored and accessed in
the various data stores. As anyone with a link to
http://www.eecs.berkeley.edu/~rcs/research/interactive_latency.html
can tell you, pulling stuff off a wire is one of the worst things
you can do. So, the question is "Do you pull lots of data in one
block, or lots of blocks of little data?" Either way, there's
going to be a factor of inefficiency. (e.g. someone who has a lot
of CHIDs with only a one or two very active CHIDs would have a
pretty good sized block of extra data pumped back and forth in the
case of a single record organized by UAID, vs. someone with a lot
of very active CHIDs having to do lots of little updates.)
I'm not really going to try to address it right now, since it's
premature optimization without actual use metrics, and there be
both dragons and hairy yaks there, but I tend to like having
options on hand for when things start to go wonky.
I'll be interested to see what y'all pick, since your constraints
and benefits aren't going to be the same.
On 2013/9/25 5:16 AM, Daniel Bevenius wrote: