[jbosscache-dev] JBCACHE-388 Provide modified data incallbacks

Sat Nov 4 23:13:44 EST 2006

How about a variation on Ben's idea of providing an enum to specify the
type of change?  Sample API follows.

/**
 * Called before and after a {@link Node} is modified. Data passed in
the
 * <code>modData</code> param can be used to determine what was
modified.
 * 
 * When called with <code>pre == true</code>, <code>modData</code> is
the 
 * initial state of the {@link Node} before modification. 
 * 
 * When called with <code>pre == false</code>, the contents of
<code>modData</code>
 * depend on the value of <code>modTtype</code>:
 * <ul>
 * <li>PUT_KEY_VALUE: Map contains the single key/value pair that
was added.</li>
 * <li>REMOVE_KEY_VALUE: Map contains the single key/value pair
that was removed.</li>
 * <li>PUT_MAP: Map contains the new state of the {@link Node}
following modification. 
 * This map includes modified key/value pairs as well as any that were
not affected.</li>
 * </ul>
 * 
 * Implementations interested in seeing the difference in the node data
in the PUT_MAP case
 * can cache the <code>modData</code> map passed when <code>pre ==
true</code>, and then
 * when the <code>pre == false</code> callback is received, pass the
cached map and the new 
 * <code>modData</code> to {@link
org.jboss.cache.util.Util.diffNodeData(Map pre, Map post)}.
 *
 * @param fqn Fqn of the node being modified
 * @param pre <code>true</code> if the modification is about to be
applied,
 * <code>false</code> if it has already been applied
 * @param isLocal <code>true</code> if the modification originated
locally, 
 * <code>false</code> if it was replicated from another
node
 * @param modType PUT_KEY_VALUE, REMOVE_KEY_VALUE or PUT_MAP
 * @param modData Unmodifiable {@link Map}; will not be
<code>null</code>. See
 * description above.
 */
public void nodeModified(Fqn fqn, boolean pre, boolean isLocal,
ModificationType modType, Map modData);

public class Util {

  public static MapModifications diffNodeData(Map pre, Map post) {
     // do the diff here
  }

  public static class MapModifications {
     public Map addedEntries;
     public Map removedEntries;
     public Map modifiedEntries;
  }
}

Advantage here is JBC goes to little effort to support this, so overhead
is low if the listener isn't interested.  CacheListener impls that are
interested in modifications, but where the app never calls put(Fqn, Map)
also have very little overhead.  I believe Jerry's DistributedState is
such a case.  The big overhead of doing the diff is only incurred by
CacheListener impls that care, and we provide much of the code to do it
for them.

- Brian

Jerry Gauthier wrote:
> This issue is relevant for DistributedState in AS 5.0 where
> the implementation was rewritten to use TreeCache for the
> underlying replicated store.  There's currently a mismatch
> between the events published by DistributedState and those offered by
> TreeCache. Specifically, DistributedState identifies the data key and
> new value for modified categories (which are TreeCache
> nodes); it also identifies keys that are removed.  Currently
> it's not possible to obtain this information from TreeCache
> unless the subscriber responds to the "pre"
> and "post" modification events by retrieving the modified
> node's Map before/after and then performing a diff operation.
> 
> I've looked at the new CacheListener interface and it looks
> like it helps a little since it provides the Map as part of
> the event.  But as Ben pointed out, it's still necessary for
> the subscriber to diff the maps to determine what was
> changed.  So this doesn't appear to help significantly for
> DistributedState as the hard part is performing the diff, not
> retrieving the map. 
> 
> Jerry
> 
> 
>>>> "Brian Stansberry" <brian.stansberry at jboss.com> 10/30/06 11:33 AM
>>>> 
> How common is the use case where the listener wants to know
> the differences?  I'd be concerned about the overhead of
> creating objects to encapsulate the diff every time there is
> a change -- just so those objects can be passed to a
> CacheListener that ignores the data.
> 
> OTOH, even if the current approach of just handing out the
> new data map is used, that should at least be a
> Collections.unmodifiableMap wrapping the internal data
> structure, so there's overhead there as well.
> Probably
> a wash, except in the put(Fqn, Map) case, where doing the diff is
> more tedious. 
> 
> I guess I have no strong opinion on this...
> 
> jbosscache-dev-bounces at lists.jboss.org wrote:
>> Hmm, which approach do people prefer?
>> 
>> And how would a nodeModified event from an operation like put(Fqn,
>> Map) be represented, since it may represent both data added as well
>> as data modified?  I don't think this should result in multiple
>> callbacks. 
>> 
>> Cheers,
>> 
>> 
>>> Yes, I did realize that. But implementation wise, it should be
>>> straightforward since Cache knows exactly the identifier, right? We
>>> just need to add an "identifier" enum in the nodeModified callback.
>>> Downside is one more parameter but upside is faster processing such
>>> that we don't hog the calling thread.
>>> 
>>> -----Original Message-----
>>> From: Manik Surtani [mailto:msurtani at redhat.com]
>>> Sent: Monday, October 30, 2006 11:15 PM
>>> To: Ben Wang
>>> Cc: jbosscache-dev at lists.jboss.org
>>> Subject: Re: [jbosscache-dev] JBCACHE-388 Provide modified data in
>>> callbacks 
>>> 
>>> Well, nodeModified covers 3 scenarios:
>>> 
>>> 1) Data added to map
>>> 2) Data removed from map
>>> 3) Existing map data changed
>>> 
>>> if we just passed in deltas, we'd also need to pass in keys to
>>> describe the operation.  I.e., a nodeVisited event could pass in a
>>> key-value pair, but we'd also need to pass in an identifier to
>>> describe the modification.  Which makes it pretty tedious.
>  This is
>>> why I opted for just passing a snapshot of the state of data before
>>> and after the mod. 
>>> 
>>> 
>>> --
>>> Manik Surtani
>>> 
>>> Lead, JBoss Cache
>>> JBoss, a division of Red Hat
>>> 
>>> Email: msurtani at redhat.com
>>> Telephone: +44 7786 702 706
>>> MSN: manik at surtani.org
>>> Yahoo/AIM/Skype: maniksurtani
>>> 
>>> 
>>> 
>>> On 29 Oct 2006, at 16:20, Ben Wang wrote:
>>> 
>>>> Yes, are are right. Silly me, I was looking at the pre only.
>>>> 
>>>> Still, my question is that will it make more sense during post-
>>>> nodeModified notification to deliver only the data modified (not
>>>> the new data map). If I am only interested what portion of my data
>>>> has been modified (and I belive this is a quite common use case),
>>>> then I would need to keep an old data map and diff it with the new
>>>> data map. Doable but it would be a performance killer for sure.
>>>> 
>>>> Thanks,
>>>> 
>>>> -Ben
>>>> 
>>>> -----Original Message-----
>>>> From: Manik Surtani [mailto:msurtani at redhat.com]
>>>> Sent: Friday, October 27, 2006 11:14 PM
>>>> To: Ben Wang
>>>> Cc: jbosscache-dev at lists.jboss.org
>>>> Subject: Re: [jbosscache-dev] JBCACHE-388 Provide modified data in
>>>> callbacks 
>>>> 
>>>> Is this when pre is true or false?
>>>> 
>>>> If pre is true, the data will be the old data.  If pre is false,
>>>> it will be the new data. -- Manik Surtani
>>>> 
>>>> Lead, JBoss Cache
>>>> JBoss, a division of Red Hat
>>>> 
>>>> Email: msurtani at redhat.com
>>>> Telephone: +44 7786 702 706
>>>> MSN: manik at surtani.org
>>>> Yahoo/AIM/Skype: maniksurtani
>>>> 
>>>> 
>>>> 
>>>> On 27 Oct 2006, at 13:47, Ben Wang wrote:
>>>> 
>>>>> Manik,
>>>>> 
>>>>> Can you clarify the semantics for the modified callbacks?
>>>>> Specifically, I am looking at the test case:
>>>>> CacheListenerTest.testRemoveData()
>>>>> calling
>>>>>       cache.remove(fqn, "key2");
>>>>> Still has the nodeModified() coming with the original data map.
>>>>> Is this the expected behavior? 
>>>>> 
>>>>> I am hoping to see a semantics that bundles exactly the modified
>>>>> data. 
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> -Ben
> 
> _______________________________________________
> jbosscache-dev mailing list
> jbosscache-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jbosscache-dev