OK, found the culprit ...
It is this "weird" rule -- why not remove if the value class is the same?!
private boolean shouldRemove(Object value, Object previousValue) {
return !(value == null || previousValue == null) &&
!value.getClass().equals(previousValue.getClass());
}
in
---
private void processPutKeyValueCommand(final PutKeyValueCommand command, final
InvocationContext ctx, final Object previousValue, TransactionContext transactionContext)
{
final boolean usingSkipIndexCleanupFlag = usingSkipIndexCleanup(command);
//whatever the new type, we might still need to cleanup for the previous value (and
schedule removal first!)
Object value = extractValue(command.getValue());
if (!usingSkipIndexCleanupFlag && updateKnownTypesIfNeeded(previousValue)
&& shouldRemove(value, previousValue)) {
if (shouldModifyIndexes(command, ctx)) {
transactionContext = transactionContext == null ? makeTransactionalEventContext()
: transactionContext;
removeFromIndexes(previousValue, extractValue(command.getKey()),
transactionContext);
---
-Ales
On 03 Apr 2015, at 12:18, Ales Justin <ales.justin(a)gmail.com>
wrote:
I think the problem is in our SearchWorkCreator, and how HS doesn't provide old value
when doing update (while doing delete).
*
https://github.com/capedwarf/capedwarf-blue/blob/master/datastore/src/mai...
<
https://github.com/capedwarf/capedwarf-blue/blob/master/datastore/src/mai...
We "explode" original value with multiple "similar" values -- due to
how projections and indexing works in GAE.
But since we don't get old value, we cannot fully re-construct previous
"similar" values.
This used to work.
What changed in HS?
Or how to get old value to do proper delete?
e.g.
Grabbing the old value from cache (is it really still there? the old one that is ...),
and creating proper DELETE manually, where I can then change UPDATE to ADD.
-Ales
> On 02 Apr 2015, at 21:00, Ales Justin <ales.justin(a)gmail.com
<mailto:ales.justin@gmail.com>> wrote:
>
> When debugging, I see this -- it get "bbb" twice (old and new, where I
would actually expect stale "ccc"):
>
> results = {java.util.ArrayList@17380} size = 3
> 0 = {java.lang.Object[3]@17384}
> 0 = {com.google.appengine.api.datastore.Key@17395} "test(1)"
> 1 = {java.lang.String@17396} "#Thu Apr 02 20:53:33 CEST
2015\nprop=STRING\n"
> 2 = {java.lang.String@17397} "040:aaa"
> 1 = {java.lang.Object[3]@17385}
> 0 = {com.google.appengine.api.datastore.Key@17391} "test(1)"
> 1 = {java.lang.String@17392} "#Thu Apr 02 20:51:24 CEST
2015\nprop=STRING\n"
> 2 = {java.lang.String@17393} "040:bbb"
> 2 = {java.lang.Object[3]@17386}
> 0 = {com.google.appengine.api.datastore.Key@17387} "test(1)"
> 1 = {java.lang.String@17388} "#Thu Apr 02 20:53:33 CEST
2015\nprop=STRING\n"
> 2 = {java.lang.String@17389} "040:bbb"
>
>
> But still no idea on why dup "bbb" is there ...
>
> -Ales
>
>> On 01 Apr 2015, at 11:59, Ales Justin <ales.justin(a)gmail.com
<mailto:ales.justin@gmail.com>> wrote:
>>
>> Did entity updating changed in HS5 / Lucene4?
>> (aka Document updating)
>>
>> Since this used to work for us (with HS4, Lucene3):
>>
>> ---
>> @Test
>> public void testProjectionQueriesHandleEntityModificationProperly() throws
Exception {
>> Entity e = createEntity("test", 1)
>> .withProperty("prop", Arrays.asList("aaa",
"bbb", "ccc"))
>> .store();
>>
>> Query query = new Query("test")
>> .addProjection(new PropertyProjection("prop", String.class))
>> .addSort("prop");
>>
>> assertEquals(3, service.prepare(query).asList(withDefaults()).size());
>>
>> e = createEntity(e.getKey())
>> .withProperty("prop", Arrays.asList("aaa",
"bbb"))
>> .store();
>>
>> assertEquals(2, service.prepare(query).asList(withDefaults()).size());
>>
>> service.delete(e.getKey());
>>
>> assertEquals(0, service.prepare(query).asList(withDefaults()).size());
>> }
>>
>> ---
>>
>> Where we add each list/collection value under same field:
>>
>> if (propertyValue instanceof Collection) {
>> Collection collection = (Collection) propertyValue;
>> for (Object element : collection) {
>> if (PropertyUtils.isIndexedProperty(element)) {
>> final Bridge inner = BridgeUtils.matchBridge(element);
>> luceneOptions.addFieldToDocument(propertyName,
inner.objectToString(element), document);
>> }
>> }
>>
>> ---
>>
>> As it looks like the update still keeps the old values around:
>>
>> java.lang.AssertionError: expected:<2> but was:<3>
>> at org.junit.Assert.fail(Assert.java:88)
>> at org.junit.Assert.failNotEquals(Assert.java:743)
>> at org.junit.Assert.assertEquals(Assert.java:118)
>> at org.junit.Assert.assertEquals(Assert.java:555)
>> at org.junit.Assert.assertEquals(Assert.java:542)
>> at
org.jboss.test.capedwarf.datastore.test.QueryOptimizationsTest.testProjectionQueriesHandleEntityModificationProperly(QueryOptimizationsTest.java:510)
>>
>> ---
>>
>> Any way to force / fix this?
>>
>> -Ales
>>
>