[Design of JBossCache] - Re: memcached client/server
by genman
ConsistentHashing allows you to replicate to as many servers as you want. Imagine a circle with dots representing "hashed server addresses". Then, pick a point, go clockwise/counterclockwise on the circle and find two or more unique instances.
| package org.jboss.cache.util;
|
| import java.util.Collection;
| import java.util.Collections;
| import java.util.HashSet;
| import java.util.Set;
| import java.util.SortedMap;
| import java.util.TreeMap;
|
| import org.jgroups.Address;
|
| /**
| * Consistent hashing algorithm; see http://weblogs.java.net/blog/tomwhite/archive/2007/11/consistent_hash.html
| *
| * @param <T> type of nodes to use; consider {@link Address}
| */
| public class ConsistentHash<T> {
|
| /**
| * Default (but overrideable) hash function.
| */
| public static class HashFunction {
|
| /**
| * Implementation from ConcurrentHashMap.
| * @param o object to cache
| * @return hopefully a well distributed hash code result
| */
| public int hash(Object o) {
| int i = o.hashCode();
| i += ~(i << 9);
| i ^= i >>> 14;
| i += i << 4;
| i ^= i >>> 10;
| return i;
| }
|
| }
|
| private final HashFunction hashFunction;
| private final int numberOfReplicas;
| private final SortedMap<Integer, T> circle = new TreeMap<Integer, T>();
|
| public ConsistentHash(HashFunction hashFunction, int numberOfReplicas,
| Collection<T> nodes) {
| this.hashFunction = hashFunction;
| this.numberOfReplicas = numberOfReplicas;
|
| for (T node : nodes) {
| add(node);
| }
| }
|
| /**
| * Constructs a new ConsistentHash with default options.
| */
| public ConsistentHash() {
| this(new HashFunction(), 5, Collections.<T>emptyList());
| }
|
| /**
| * Adds a node.
| */
| public void add(T node) {
| for (int i = 0; i < numberOfReplicas; i++) {
| circle.put(hashFunction.hash(node.toString() + i), node);
| }
| }
|
| /**
| * Removes a node.
| */
| public void remove(T node) {
| for (int i = 0; i < numberOfReplicas; i++) {
| circle.remove(hashFunction.hash(node.toString() + i));
| }
| }
|
| /**
| * Returns N nodes for a key.
| */
| public Set<T> get(Object key, int count) {
| if (count <= 0)
| throw new IllegalArgumentException("count");
| if (key == null)
| throw new NullPointerException("key");
| if (circle.isEmpty()) {
| return Collections.emptySet();
| }
| // TODO bound the "count" to the number of instances
| Set<T> nodes = new HashSet<T>(count);
| int hash = hashFunction.hash(key);
| for (T node : circle.tailMap(hash).values()) {
| nodes.add(node);
| if (nodes.size() == count)
| return nodes;
| }
| for (T node : circle.values()) {
| nodes.add(node);
| if (nodes.size() == count)
| return nodes;
| }
| return nodes;
| }
|
| /**
| * Returns a debug <code>String</code>.
| */
| @Override
| public String toString()
| {
| return super.toString() +
| " numberOfReplicas=" + this.numberOfReplicas +
| " circle=" + this.circle +
| "";
| }
|
| }
|
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4177513#4177513
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4177513
17 years, 6 months
[Design of JBossCache] - Re: memcached client/server
by manik.surtani@jboss.com
My issue with the metadata approach is that it can lead to unnecessary traffic, and while this traffic is small, could be a scalability problem.
I very much prefer the consistent hashing approach if we can make it work for us. Using two hash functions to map an Fqn to two different servers is one approach, but this is hard when you want > 1 backup copy.
Regarding memcached in relation to JBC, I actually see JBC as a memcached server rather than a client. E.g., Bela's memcached server code could act as a "front end" to JBC, and instead of putting stuff in a CHM, it could put stuff into a JBC instance. This buys us:
1. The ability to use different types of clients talking to JBC (any memcached client library should work)
2. The ability to be colocated in-VM as well, for a more p2p-like setup
To make this work we would need clients to be aware of the consistent hashing algo, so it could direct requests to the relevant server (as an optimisation), and for clients that don't do this (as well as colocated clients) the JBC instance would need a subclass of the ClusteredCacheLoader that has the consistent hashing algo. We then automatically have cache instances acting as L1 caches if they aren't the "host" cache for a particular fqn, and automatically would have L1 cache invalidation (assuming the cache uses INVALIDATION. REPLICATION would be pointless anyway in such a setup).
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4177433#4177433
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4177433
17 years, 6 months
[Design of Messaging on JBoss (Messaging/JBoss)] - Re: Notifications over JBM core connections
by jmesnil
"timfox" wrote :
| What if there is no jmx server running? All this stuff should work with no JMX server too.
|
It does work even when there is no JMX MBeanServer (the ManagementService keeps a registry of managed resources using ObjectNames as their keys)
"timfox" wrote :
| I think it might be better to introduce a new packet on the wireformat ManagementMessage and send via that, then there's no need for special routing in the post office - it can be intercepted at the session level
|
"timfox" wrote :
| "jmesnil wrote : * the management service creates a NotificationListener for the given ObjectName
| |
|
| This seems JMX dependent to me
|
Same as above, I use a NotificationBroadcaster to keep track of notifications but it does not depend on any MBeanServer
"timfox" wrote :
| Are notifications supposed to be persisent (i.e. do they survive after the session that created them has died?)
|
| If not, then the listener could just be the session, and you could use the standard session send functionality to send the notification
I do not intend for the notifications to be persistent.
It makes sense to move the notification handling to the session.
But it still got the same issue: ServerSession.send() expect a ServerMessage that I need to create.
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4177415#4177415
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4177415
17 years, 6 months