[JBoss JIRA] (ISPN-8385) Document how DataContainer can store entries in different format than provided to Cache
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-8385?page=com.atlassian.jira.plugin.... ]
William Burns updated ISPN-8385:
--------------------------------
Summary: Document how DataContainer can store entries in different format than provided to Cache (was: Off-heap container entries contain addional bytes)
> Document how DataContainer can store entries in different format than provided to Cache
> ---------------------------------------------------------------------------------------
>
> Key: ISPN-8385
> URL: https://issues.jboss.org/browse/ISPN-8385
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.1.1.Final, 9.2.0.Alpha1
> Reporter: Vojtech Juranek
> Assignee: William Burns
>
> Off-heap container entries contain some additional bytes (ASCII 1,2,1,4), which probably shouldn't be there. Besides additional consumption of memory, it can cause problem as entry put into the cache with cache API is not accessible via container API (which is available to the use), i.e. something like this will fail:
> {noformat}
> cache.put(key, value);
> container = AbstractDelegatingCache.unwrapCache(cache).getAdvancedCache().getDataContainer();
> InternalCacheEntry entry = container.get(key);
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (ISPN-8385) Off-heap container entries contain addional bytes
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-8385?page=com.atlassian.jira.plugin.... ]
Work on ISPN-8385 started by William Burns.
-------------------------------------------
> Off-heap container entries contain addional bytes
> -------------------------------------------------
>
> Key: ISPN-8385
> URL: https://issues.jboss.org/browse/ISPN-8385
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.1.1.Final, 9.2.0.Alpha1
> Reporter: Vojtech Juranek
> Assignee: William Burns
>
> Off-heap container entries contain some additional bytes (ASCII 1,2,1,4), which probably shouldn't be there. Besides additional consumption of memory, it can cause problem as entry put into the cache with cache API is not accessible via container API (which is available to the use), i.e. something like this will fail:
> {noformat}
> cache.put(key, value);
> container = AbstractDelegatingCache.unwrapCache(cache).getAdvancedCache().getDataContainer();
> InternalCacheEntry entry = container.get(key);
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (ISPN-8385) Off-heap container entries contain addional bytes
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-8385?page=com.atlassian.jira.plugin.... ]
William Burns updated ISPN-8385:
--------------------------------
Status: Open (was: New)
> Off-heap container entries contain addional bytes
> -------------------------------------------------
>
> Key: ISPN-8385
> URL: https://issues.jboss.org/browse/ISPN-8385
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.1.1.Final, 9.2.0.Alpha1
> Reporter: Vojtech Juranek
> Assignee: William Burns
>
> Off-heap container entries contain some additional bytes (ASCII 1,2,1,4), which probably shouldn't be there. Besides additional consumption of memory, it can cause problem as entry put into the cache with cache API is not accessible via container API (which is available to the use), i.e. something like this will fail:
> {noformat}
> cache.put(key, value);
> container = AbstractDelegatingCache.unwrapCache(cache).getAdvancedCache().getDataContainer();
> InternalCacheEntry entry = container.get(key);
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (ISPN-8478) Reduce build memory usage
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-8478?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-8478:
------------------------------------
Status: Open (was: New)
> Reduce build memory usage
> -------------------------
>
> Key: ISPN-8478
> URL: https://issues.jboss.org/browse/ISPN-8478
> Project: Infinispan
> Issue Type: Task
> Components: Build process
> Affects Versions: 9.2.0.Alpha2
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.2.0.Beta1
>
>
> The CI Maven process runs without a max heap parameter, so it can use up to 1/4 of agent RAM. It should run with a specific limit and so should all the other JVMs it spawns: the surefire fork, Karaf containers, WildFly servers etc.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (ISPN-8478) Reduce build memory usage
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-8478?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-8478:
------------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> Reduce build memory usage
> -------------------------
>
> Key: ISPN-8478
> URL: https://issues.jboss.org/browse/ISPN-8478
> Project: Infinispan
> Issue Type: Task
> Components: Build process
> Affects Versions: 9.2.0.Alpha2
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.2.0.Beta1
>
>
> The CI Maven process runs without a max heap parameter, so it can use up to 1/4 of agent RAM. It should run with a specific limit and so should all the other JVMs it spawns: the surefire fork, Karaf containers, WildFly servers etc.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (ISPN-8385) Off-heap container entries contain addional bytes
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-8385?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-8385:
-------------------------------------
Talking with [~gustavonalle] the user can do `cache.getAdvancedCache().getKeyDataConversion().toStorage(key)` to get the underlying key as the storage requires. I will add some documentation to the `DataContainer` to say this shouldn't be used directly and the user should use the above method to convert if the underlying cache isn't stored as is.
> Off-heap container entries contain addional bytes
> -------------------------------------------------
>
> Key: ISPN-8385
> URL: https://issues.jboss.org/browse/ISPN-8385
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.1.1.Final, 9.2.0.Alpha1
> Reporter: Vojtech Juranek
> Assignee: William Burns
>
> Off-heap container entries contain some additional bytes (ASCII 1,2,1,4), which probably shouldn't be there. Besides additional consumption of memory, it can cause problem as entry put into the cache with cache API is not accessible via container API (which is available to the use), i.e. something like this will fail:
> {noformat}
> cache.put(key, value);
> container = AbstractDelegatingCache.unwrapCache(cache).getAdvancedCache().getDataContainer();
> InternalCacheEntry entry = container.get(key);
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (ISPN-8385) Off-heap container entries contain addional bytes
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-8385?page=com.atlassian.jira.plugin.... ]
William Burns edited comment on ISPN-8385 at 11/8/17 11:19 AM:
---------------------------------------------------------------
That is exactly what it is. Also to note this is only an issue with embedded off heap, server is perfectly fine since the serialization occurs on the client.
Looking at this there are 2 possible solutions:
# We could wrap the byte[] differently, thus preventing the GlobalMarshaller from occurring. Unfortunately we would then have to store an extra byte with the key and value to signify that it is a byte[] and not a serialized object. So this still has the same issue as reported. This also would then add an additional byte overhead to non byte[] entries, which when running embedded should be the most commonly used entries by far.
# If we pay attention to the upcoming cache type value and if it was _application/octet-stream_ we would know the values will always have to be byte[]. This isn't doable currently though and would only work under specific configuration settings.
# We could wrap the data container retrieved from the cache and automatically perform the encoding and wrapping for the user. This is probably the cleanest method, but I am not sure if we want to do this or not. [~gustavonalle] have an opinion? Virtually all of the components you can retrieve don't do this currently. Do we then have to go and change them all? Sounds nasty to me.
Therefore I am suggesting that this is not a bug and is just intended. If the user wants to use the data container directly they should retrieve the encoder and wrapper from the cache and do the operation themselves. We can document the data container though to highlight this to the user.
was (Author: william.burns):
That is exactly what it is. Also to note this is only an issue with embedded off heap, server is perfectly fine since the serialization occurs on the client.
Looking at this there are 2 possible solutions:
# We could wrap the byte[] differently, thus preventing the GlobalMarshaller from occurring. Unfortunately we would then have to store an extra byte with the key and value to signify that it is a byte[] and not a serialized object. So this still has the same issue as reported. This also would then add an additional byte overhead to non byte[] entries, which when running embedded should be the most commonly used entries by far.
# If we pay attention to the upcoming cache type value and if it was _application/octet-stream_ we would know the values will always have to be byte[]. This isn't doable currently though and would only work under specific configuration settings.
# We could wrap the data container retrieved from the cache and automatically perform the encoding and wrapping for the user. This is probably the cleanest method, but I am not sure if we want to do this or not. [~gustavonalle] have an opinion? Virtually all of the components you can retrieve don't do this currently. Do we then have to go and change them all? Sounds nasty to me.
Therefore I am suggesting that this is not a bug and is just intended. If the user wants to use the data container directly they should retrieve the encoder and wrapper from the cache and do the operation themselves.
> Off-heap container entries contain addional bytes
> -------------------------------------------------
>
> Key: ISPN-8385
> URL: https://issues.jboss.org/browse/ISPN-8385
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.1.1.Final, 9.2.0.Alpha1
> Reporter: Vojtech Juranek
> Assignee: William Burns
>
> Off-heap container entries contain some additional bytes (ASCII 1,2,1,4), which probably shouldn't be there. Besides additional consumption of memory, it can cause problem as entry put into the cache with cache API is not accessible via container API (which is available to the use), i.e. something like this will fail:
> {noformat}
> cache.put(key, value);
> container = AbstractDelegatingCache.unwrapCache(cache).getAdvancedCache().getDataContainer();
> InternalCacheEntry entry = container.get(key);
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (ISPN-8385) Off-heap container entries contain addional bytes
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-8385?page=com.atlassian.jira.plugin.... ]
William Burns edited comment on ISPN-8385 at 11/8/17 11:18 AM:
---------------------------------------------------------------
That is exactly what it is. Also to note this is only an issue with embedded off heap, server is perfectly fine since the serialization occurs on the client.
Looking at this there are 2 possible solutions:
# We could wrap the byte[] differently, thus preventing the GlobalMarshaller from occurring. Unfortunately we would then have to store an extra byte with the key and value to signify that it is a byte[] and not a serialized object. So this still has the same issue as reported. This also would then add an additional byte overhead to non byte[] entries, which when running embedded should be the most commonly used entries by far.
# If we pay attention to the upcoming cache type value and if it was _application/octet-stream_ we would know the values will always have to be byte[]. This isn't doable currently though and would only work under specific configuration settings.
# We could wrap the data container retrieved from the cache and automatically perform the encoding and wrapping for the user. This is probably the cleanest method, but I am not sure if we want to do this or not. [~gustavonalle] have an opinion? Virtually all of the components you can retrieve don't do this currently. Do we then have to go and change them all? Sounds nasty to me.
Therefore I am suggesting that this is not a bug and is just intended. If the user wants to use the data container directly they should retrieve the encoder and wrapper from the cache and do the operation themselves.
was (Author: william.burns):
That is exactly what it is. Also to note this is only an issue with embedded off heap, server is perfectly fine since the serialization occurs on the client.
Looking at this there are 2 possible solutions:
# We could wrap the byte[] differently, thus preventing the GlobalMarshaller from occurring. Unfortunately we would then have to store an extra byte with the key and value to signify that it is a byte[] and not a serialized object. So this still has the same issue as reported. This also would then add an additional byte overhead to non byte[] entries, which when running embedded should be the most commonly used entries by far.
# If we pay attention to the upcoming cache type value and if it was _application/octet-stream_ we would know the values will always have to be byte[]. This isn't doable currently though and would only work under specific configuration settings.
# We could wrap the data container retrieved from the cache and automatically perform the encoding and wrapping for the user. This is probably the cleanest method, but I am not sure if we want to do this or not. [~gustavonalle] have an opinion? Virtually all of the components you can retrieve don't do this currently. Do we then have to go and change them all? Sounds nasty to me.
Therefore I am saying that this
> Off-heap container entries contain addional bytes
> -------------------------------------------------
>
> Key: ISPN-8385
> URL: https://issues.jboss.org/browse/ISPN-8385
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.1.1.Final, 9.2.0.Alpha1
> Reporter: Vojtech Juranek
> Assignee: William Burns
>
> Off-heap container entries contain some additional bytes (ASCII 1,2,1,4), which probably shouldn't be there. Besides additional consumption of memory, it can cause problem as entry put into the cache with cache API is not accessible via container API (which is available to the use), i.e. something like this will fail:
> {noformat}
> cache.put(key, value);
> container = AbstractDelegatingCache.unwrapCache(cache).getAdvancedCache().getDataContainer();
> InternalCacheEntry entry = container.get(key);
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months
[JBoss JIRA] (ISPN-8385) Off-heap container entries contain addional bytes
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-8385?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-8385:
-------------------------------------
That is exactly what it is. Also to note this is only an issue with embedded off heap, server is perfectly fine since the serialization occurs on the client.
Looking at this there are 2 possible solutions:
# We could wrap the byte[] differently, thus preventing the GlobalMarshaller from occurring. Unfortunately we would then have to store an extra byte with the key and value to signify that it is a byte[] and not a serialized object. So this still has the same issue as reported. This also would then add an additional byte overhead to non byte[] entries, which when running embedded should be the most commonly used entries by far.
# If we pay attention to the upcoming cache type value and if it was _application/octet-stream_ we would know the values will always have to be byte[]. This isn't doable currently though and would only work under specific configuration settings.
# We could wrap the data container retrieved from the cache and automatically perform the encoding and wrapping for the user. This is probably the cleanest method, but I am not sure if we want to do this or not. [~gustavonalle] have an opinion? Virtually all of the components you can retrieve don't do this currently. Do we then have to go and change them all? Sounds nasty to me.
Therefore I am saying that this
> Off-heap container entries contain addional bytes
> -------------------------------------------------
>
> Key: ISPN-8385
> URL: https://issues.jboss.org/browse/ISPN-8385
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.1.1.Final, 9.2.0.Alpha1
> Reporter: Vojtech Juranek
> Assignee: William Burns
>
> Off-heap container entries contain some additional bytes (ASCII 1,2,1,4), which probably shouldn't be there. Besides additional consumption of memory, it can cause problem as entry put into the cache with cache API is not accessible via container API (which is available to the use), i.e. something like this will fail:
> {noformat}
> cache.put(key, value);
> container = AbstractDelegatingCache.unwrapCache(cache).getAdvancedCache().getDataContainer();
> InternalCacheEntry entry = container.get(key);
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 2 months