Comments on Tom White: Consistent Hashing

Hi. Do you have any clue of how to create an algor...

2008-01-03T20:37:00.000Z

Hi. Do you have any clue of how to create an algorithm which tracks the history of joins/leves of members and delivers the same node for the same key if it previously looked it up. Perhaps I'm explaining this in bad terms but something like a (in memory or persistent) database in cojunction with a consistent hash.

perhaps:
public Address getAddress(key)
{
if(lookedUpMap.containsKey(key))
{
return (Address)lookedUpMap.get(key)
}
else
{
Address a = get(key);
lookedUpMap.put(key, a);
return a;
}
}

Then it is up to the client to check if the previously stored node is reachable at this very moment.

An extension to this cache would be to return a Set which size is equal to the nr of replicas of each key/value.

If a value is stored on two or more nodes then the possibilty for that at least one of the servers is up increases a lot. I'm building a distributed storage solution and the data cannnot be lost just because the cache "recalculates" the hash :)

Ahhh enough already I will implement something like this :)

Cool! I'm as we speak creating a distributed cachi...

2007-12-25T09:37:00.000Z

Cool! I'm as we speak creating a distributed caching and searching system which uses JGroups for membership. The biggest problem I faced was this exact thing. What to do on the member-joined/leaved events and for the system to be able to know at all times to which node to send what command :)

The caching system is strictly following the Map (and SortedMap) interface and a bunch of implementations have been implemented. LFU, LRU, MRU, Diskbased B+Tree (jdbm), ehcache wrapper, memcached java client wrapper, hibernate support...
I like the Map interface since it is quite clean..

The impl I'm working on now is a cache/persister which uses HDFS as persistance layer. See how that turns out. The line between a cache and a persistence engine is fine.

And of course all caches must be searchable = My own indexer/searcher + Lucene free text index/search, ohh and all must be able to work in a distributed environment.. fuck it is a big task.

morrita - Glad you enjoyed the post and thanks for...

2007-12-02T22:26:00.000Z

morrita - Glad you enjoyed the post and thanks for the translation! Tom

good article!i've made Japanese translation for yo...

2007-12-01T06:06:00.000Z

good article!
i've made Japanese translation for your article. which is available at
http://www.hyuki.com/yukiwiki/wiki.cgi?ConsistentHashing .
if you have any trouble, please let me know.
thank you for your work.