May 2017 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-3795) QueryInterceptor incorrectly relies on the return value of a RemoveCommand

by Radim Vansa (JIRA)

[ https://issues.jboss.org/browse/ISPN-3795?page=com.atlassian.jira.plugin.... ] Radim Vansa commented on ISPN-3795: ----------------------------------- [~gustavonalle] +1 If we'll index only on the primary owner, the place between EWI and xDI is sufficient (primary can always read the value), however if we want to keep ALL for replicated case, we need to have it after xDI so that the value is loaded from remote node if it's not present currently. However, in some situations the value is not loaded from persistence/remote node. While some flags can be expected to be dangerous {{IGNORE_RETURN_VALUE}} is documented to be a safe option at all times, and we should keep it that way. So, should we somehow enforce loading (that enforce flag would be set above EWI) even if the operation itself doesn't require it? > QueryInterceptor incorrectly relies on the return value of a RemoveCommand > -------------------------------------------------------------------------- > > Key: ISPN-3795 > URL: https://issues.jboss.org/browse/ISPN-3795 > Project: Infinispan > Issue Type: Bug > Components: Embedded Querying > Affects Versions: 6.0.0.Final > Reporter: Dan Berindei > Assignee: Gustavo Fernandes > > QueryInterceptor uses the return value from RemoveCommand/ReplaceCommand to remove the value from the index. > But both RemoveCommand and ReplaceCommand have a variant with an expected value parameter, and this variant return a boolean value instead of the removed/replaced value. In that case, the previous value won't be removed from the index. > QueryInterceptor should probably use the previous value from the context entries to update the index instead. -- This message was sent by Atlassian JIRA (v7.2.3#72005)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7802) Use chunked reads/writes in TcpTransport

by Radim Vansa (JIRA)

[ https://issues.jboss.org/browse/ISPN-7802?page=com.atlassian.jira.plugin.... ] Radim Vansa commented on ISPN-7802: ----------------------------------- [~dan.berindei] Because NIO (some code in sun.*) allocates a direct buffer with the same size as the provided array and stores this in a thread-local, never releasing that. http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6210541 > Use chunked reads/writes in TcpTransport > ---------------------------------------- > > Key: ISPN-7802 > URL: https://issues.jboss.org/browse/ISPN-7802 > Project: Infinispan > Issue Type: Bug > Components: Remote Protocols > Affects Versions: 9.0.0.Final > Reporter: Radim Vansa > Assignee: Tristan Tarrant > > The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. > EDIT: As Tristan pointed out, the {{BufferedInputStream}} does not grow in our use case, but if {{TcpTransport}} passes a big {{byte[]}} buffer, the implementation works as read through/write through. Therefore we need to chunk all calls. -- This message was sent by Atlassian JIRA (v7.2.3#72005)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3795) QueryInterceptor incorrectly relies on the return value of a RemoveCommand

by Gustavo Fernandes (JIRA)

[ https://issues.jboss.org/browse/ISPN-3795?page=com.atlassian.jira.plugin.... ] Gustavo Fernandes edited comment on ISPN-3795 at 5/15/17 4:48 AM: ------------------------------------------------------------------ [~rvansa] Twofold: * ClusteredQueries, where each node indexes "LOCAL" entries, in its own non-shared local index. During query time the ClusteredQuery API allows to broadcast a query to all nodes, aggregating the results to return to the user. * Shared Index: most widely used, index is shared by all nodes and Index.LOCAL allows to index the entry only once. For both cases above, though, we don't necessarily need Index.LOCAL; we can use Index.PRIMARY_OWNER that the effect will be the same, actually more efficiently: we don't need to fetch values remotely from non-owner. So I think we could get rid of Index.LOCAL, map it internally to Index.PRIMARY_OWNER and move the QI after the EWI that the context would always be available was (Author: gustavonalle): [~rvansa] Twofold: * ClusteredQueries, where each node indexes "LOCAL" entries, in its own non-shared local index. During query time the ClusteredQuery API allows to broadcast a query to all nodes, aggregating the results to return to the user. * Shared Index: most widely used, index is shared by all nodes and Index.LOCAL allows to index the entry only once. For both cases above, though, we don't necessarily need Index.LOCAL; we can use Index.PRIMARY_OWNER that the effect will be the same, actually more efficiently: we don't need to fetch values remotely from non-owner. So I think we could get rid of Index.LOCAL, and put the QI after the EWI that the context would always be available > QueryInterceptor incorrectly relies on the return value of a RemoveCommand > -------------------------------------------------------------------------- > > Key: ISPN-3795 > URL: https://issues.jboss.org/browse/ISPN-3795 > Project: Infinispan > Issue Type: Bug > Components: Embedded Querying > Affects Versions: 6.0.0.Final > Reporter: Dan Berindei > Assignee: Gustavo Fernandes > > QueryInterceptor uses the return value from RemoveCommand/ReplaceCommand to remove the value from the index. > But both RemoveCommand and ReplaceCommand have a variant with an expected value parameter, and this variant return a boolean value instead of the removed/replaced value. In that case, the previous value won't be removed from the index. > QueryInterceptor should probably use the previous value from the context entries to update the index instead. -- This message was sent by Atlassian JIRA (v7.2.3#72005)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3795) QueryInterceptor incorrectly relies on the return value of a RemoveCommand

by Gustavo Fernandes (JIRA)

[ https://issues.jboss.org/browse/ISPN-3795?page=com.atlassian.jira.plugin.... ] Gustavo Fernandes commented on ISPN-3795: ----------------------------------------- [~rvansa] Twofold: * ClusteredQueries, where each node indexes "LOCAL" entries, in its own non-shared local index. During query time the ClusteredQuery API allows to broadcast a query to all nodes, aggregating the results to return to the user. * Shared Index: most widely used, index is shared by all nodes and Index.LOCAL allows to index the entry only once. For both cases above, though, we don't necessarily need Index.LOCAL; we can use Index.PRIMARY_OWNER that the effect will be the same, actually more efficiently: we don't need to fetch values remotely from non-owner. So I think we could get rid of Index.LOCAL, and put the QI after the EWI that the context would always be available > QueryInterceptor incorrectly relies on the return value of a RemoveCommand > -------------------------------------------------------------------------- > > Key: ISPN-3795 > URL: https://issues.jboss.org/browse/ISPN-3795 > Project: Infinispan > Issue Type: Bug > Components: Embedded Querying > Affects Versions: 6.0.0.Final > Reporter: Dan Berindei > Assignee: Gustavo Fernandes > > QueryInterceptor uses the return value from RemoveCommand/ReplaceCommand to remove the value from the index. > But both RemoveCommand and ReplaceCommand have a variant with an expected value parameter, and this variant return a boolean value instead of the removed/replaced value. In that case, the previous value won't be removed from the index. > QueryInterceptor should probably use the previous value from the context entries to update the index instead. -- This message was sent by Atlassian JIRA (v7.2.3#72005)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7802) Use chunked reads/writes in TcpTransport

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-7802?page=com.atlassian.jira.plugin.... ] Dan Berindei commented on ISPN-7802: ------------------------------------ [~dereed] [~rvansa] why is writing a big byte array with a single `write()` problematic? > Use chunked reads/writes in TcpTransport > ---------------------------------------- > > Key: ISPN-7802 > URL: https://issues.jboss.org/browse/ISPN-7802 > Project: Infinispan > Issue Type: Bug > Components: Remote Protocols > Affects Versions: 9.0.0.Final > Reporter: Radim Vansa > Assignee: Tristan Tarrant > > The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. > EDIT: As Tristan pointed out, the {{BufferedInputStream}} does not grow in our use case, but if {{TcpTransport}} passes a big {{byte[]}} buffer, the implementation works as read through/write through. Therefore we need to chunk all calls. -- This message was sent by Atlassian JIRA (v7.2.3#72005)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7802) Use chunked reads/writes in TcpTransport

by Radim Vansa (JIRA)

[ https://issues.jboss.org/browse/ISPN-7802?page=com.atlassian.jira.plugin.... ] Radim Vansa updated ISPN-7802: ------------------------------ Description: The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. EDIT: As Tristan pointed out, the {{BufferedInputStream}} does not grow in our use case, but if {{TcpTransport}} passes a big {{byte[]}} buffer, the implementation works as read through/write through. Therefore we need to chunk all calls. was: The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. EDIT: As Tristan pointed out, the {{BufferedInputStream}} does not grow in our use case, but > Use chunked reads/writes in TcpTransport > ---------------------------------------- > > Key: ISPN-7802 > URL: https://issues.jboss.org/browse/ISPN-7802 > Project: Infinispan > Issue Type: Bug > Components: Remote Protocols > Affects Versions: 9.0.0.Final > Reporter: Radim Vansa > Assignee: Tristan Tarrant > > The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. > EDIT: As Tristan pointed out, the {{BufferedInputStream}} does not grow in our use case, but if {{TcpTransport}} passes a big {{byte[]}} buffer, the implementation works as read through/write through. Therefore we need to chunk all calls. -- This message was sent by Atlassian JIRA (v7.2.3#72005)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7802) Use chunked reads/writes in TcpTransport

by Radim Vansa (JIRA)

[ https://issues.jboss.org/browse/ISPN-7802?page=com.atlassian.jira.plugin.... ] Radim Vansa updated ISPN-7802: ------------------------------ Summary: Use chunked reads/writes in TcpTransport (was: Use chunked read in TcpTransport) > Use chunked reads/writes in TcpTransport > ---------------------------------------- > > Key: ISPN-7802 > URL: https://issues.jboss.org/browse/ISPN-7802 > Project: Infinispan > Issue Type: Bug > Components: Remote Protocols > Affects Versions: 9.0.0.Final > Reporter: Radim Vansa > Assignee: Tristan Tarrant > > The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. > EDIT: As Tristan pointed out, the {{BufferedInputStream}} does not grow in our use case, but -- This message was sent by Atlassian JIRA (v7.2.3#72005)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7802) Use chunked read in TcpTransport

by Radim Vansa (JIRA)

[ https://issues.jboss.org/browse/ISPN-7802?page=com.atlassian.jira.plugin.... ] Radim Vansa updated ISPN-7802: ------------------------------ Description: The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. EDIT: As Tristan pointed out, the {{BufferedInputStream}} does not grow in our use case, but was:The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. > Use chunked read in TcpTransport > -------------------------------- > > Key: ISPN-7802 > URL: https://issues.jboss.org/browse/ISPN-7802 > Project: Infinispan > Issue Type: Bug > Components: Remote Protocols > Affects Versions: 9.0.0.Final > Reporter: Radim Vansa > Assignee: Tristan Tarrant > > The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. > EDIT: As Tristan pointed out, the {{BufferedInputStream}} does not grow in our use case, but -- This message was sent by Atlassian JIRA (v7.2.3#72005)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7802) Use non-growing buffered input stream in TcpTransport

by Radim Vansa (JIRA)

[ https://issues.jboss.org/browse/ISPN-7802?page=com.atlassian.jira.plugin.... ] Radim Vansa commented on ISPN-7802: ----------------------------------- Good point, Dennis - ISPN-5740 may not be enough as a big byte[] is passed down without buffering anyway. > Use non-growing buffered input stream in TcpTransport > ----------------------------------------------------- > > Key: ISPN-7802 > URL: https://issues.jboss.org/browse/ISPN-7802 > Project: Infinispan > Issue Type: Bug > Components: Remote Protocols > Affects Versions: 9.0.0.Final > Reporter: Radim Vansa > Assignee: Tristan Tarrant > > The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. -- This message was sent by Atlassian JIRA (v7.2.3#72005)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7802) Use chunked read in TcpTransport

by Radim Vansa (JIRA)

[ https://issues.jboss.org/browse/ISPN-7802?page=com.atlassian.jira.plugin.... ] Radim Vansa updated ISPN-7802: ------------------------------ Summary: Use chunked read in TcpTransport (was: Use non-growing buffered input stream in TcpTransport) > Use chunked read in TcpTransport > -------------------------------- > > Key: ISPN-7802 > URL: https://issues.jboss.org/browse/ISPN-7802 > Project: Infinispan > Issue Type: Bug > Components: Remote Protocols > Affects Versions: 9.0.0.Final > Reporter: Radim Vansa > Assignee: Tristan Tarrant > > The buffering implementation of {{TcpTransport.socketInputStream}} needs to use fixed size input buffer, rather than growing as {{BufferedInputStream}} does. Growing buffer can lead to excessive memory consumption on heap, and more importantly, by direct memory allocated in JDK classes and pooled in thread-local caches. -- This message was sent by Atlassian JIRA (v7.2.3#72005)

8 years, 10 months

1
0
0 / 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues May 2017