July 2018 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-7409) Replace external marshaller with user marshaller

by Ryan Emerson (JIRA)

[ https://issues.jboss.org/browse/ISPN-7409?page=com.atlassian.jira.plugin.... ] Ryan Emerson updated ISPN-7409: ------------------------------- Sprint: Sprint 9.4.0.Beta1 > Replace external marshaller with user marshaller > ------------------------------------------------ > > Key: ISPN-7409 > URL: https://issues.jboss.org/browse/ISPN-7409 > Project: Infinispan > Issue Type: Enhancement > Components: Core, Marshalling > Reporter: Galder Zamarreño > Assignee: Ryan Emerson > > ISPN-6906 brings much needed independence from JBoss Marshalling to our marshalling layer. One of the key aspects it brings it’s the separation between marshalling internal types we know about, e.g. String, byte[], CacheTopologyCommand...etc and external or unknown types, e.g. any types that extend Serializable. > Although this separation helps achieve objectives of ISPN-6906, it’s not an ideal solution: > * If user types map directly to primitive types supported by our internal marshaller, or types that depend on types that are marshalled by our internal marshaller, any changes made to the internal marshaller have a direct impact on the representation of the user types. > * In the ISPN-6906 implementation, Internal marshaller exposes its own externalizer table to the external marshaller so that it can lookup how to marshall internal types. This is done to support corner cases in Hibernate Search where some of the classes that Query module marshalls rely on the fact that some Hibernate Search classes are Serializable, but these classes reference Lucene query classes that are not for which there are externalizers defined. For query logic to work, externalizers have to be found while dealing with Serializable types are being traversed. Gustavo and Sane are aware of this issue and we should soon have externalizers for those Hibernate Search classes and avoid this issue altogether (see ISPN-7156). > So, rather than an external marshaller, a more suitable abstraction would be to have a user marshaller. The job of the user marshaller, which would be configurable, would be to marshall user types. The user marshaller should be independent of the internal marshaller, so it would not be able to piggyback on the internal externalizers. > Externalizers for user types could still be supported, by having a user externalizer table, purely used by the user marshaller. In fact, it would make sense that all externalizers that are configured in Infinispan (regardless of whether via xml, programmatic or via annotation) to be user externalizers. > User types are considered to be: > * Keys and values > * Metadata instances provided by the user via *withMetadata() calls > The benefits of the user marshaller are the following: > * Having the user marshaller configurable would make it easy to try out different marshallers when comparing how key/value types are marshalled, e.g. try out different ways to marshall Strings. > * An easier upgrade path for user types. Since user types no longer rely on internal marshaller details, Infinispan upgrades are easier to deal with. > For reference, custom commands that can be plugged via ModuleCommandFactory implementations would use the internal marshaller since these commands often marshall internal types, e.g. Address. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

5 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-8861) DIST Iteration shouldn't go remote when a shared cache store is present

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-8861?page=com.atlassian.jira.plugin.... ] William Burns updated ISPN-8861: -------------------------------- Sprint: Sprint 9.4.0.CR1 > DIST Iteration shouldn't go remote when a shared cache store is present > ----------------------------------------------------------------------- > > Key: ISPN-8861 > URL: https://issues.jboss.org/browse/ISPN-8861 > Project: Infinispan > Issue Type: Enhancement > Components: Loaders and Stores > Reporter: William Burns > Assignee: William Burns > Fix For: 9.4.0.Final > > > When a user runs a distributed iteration operation with a shared cache store, this shouldn't go remote as the local node would have all the data available to it. This is similar to the optimization when we don't go remote if the current node is an owner for all required segments. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

5 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-9332) REPL local iteration optimization cannot be used when store has write behind

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-9332?page=com.atlassian.jira.plugin.... ] William Burns updated ISPN-9332: -------------------------------- Sprint: Sprint 9.4.0.CR1 > REPL local iteration optimization cannot be used when store has write behind > ---------------------------------------------------------------------------- > > Key: ISPN-9332 > URL: https://issues.jboss.org/browse/ISPN-9332 > Project: Infinispan > Issue Type: Bug > Components: Loaders and Stores, Streams > Affects Versions: 9.3.0.Final > Reporter: William Burns > Assignee: William Burns > > When write behind is enabled, the write modification is stored on the primary owner. REPL local iteration can read from non owned data, thus causing an inconsistency. > Thus distributed streams should always go remote when not all segments are primarily owned on a given node when write behind is enabled. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

5 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7624) Change CacheLoaderInterceptor to ignore in memory for certain operations when passivation is not enabled

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-7624?page=com.atlassian.jira.plugin.... ] William Burns updated ISPN-7624: -------------------------------- Sprint: Sprint 9.4.0.CR1 > Change CacheLoaderInterceptor to ignore in memory for certain operations when passivation is not enabled > -------------------------------------------------------------------------------------------------------- > > Key: ISPN-7624 > URL: https://issues.jboss.org/browse/ISPN-7624 > Project: Infinispan > Issue Type: Bug > Components: Loaders and Stores > Reporter: William Burns > Assignee: William Burns > Fix For: 9.4.0.Final > > > CacheLoader currently reads all in memory entries and then the store for bulk operations. This can be changed to only load from the store when passivation is not used as it should contain all entries that aren't in memory. > The only issue is that if Flag.SKIP_CACHE_STORE was used on a write. In this case we could store a boolean in the interceptor so that if this Flag is ever used we always do both. Very few users use this Flag and it should improve performance a bit and entries should be seen more consistently. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

5 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-9375) Change RocksDB segmentation to use column family instead of per instance

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-9375?page=com.atlassian.jira.plugin.... ] William Burns updated ISPN-9375: -------------------------------- Sprint: Sprint 9.4.0.Beta1 > Change RocksDB segmentation to use column family instead of per instance > ------------------------------------------------------------------------ > > Key: ISPN-9375 > URL: https://issues.jboss.org/browse/ISPN-9375 > Project: Infinispan > Issue Type: Enhancement > Reporter: William Burns > Assignee: William Burns > Fix For: 9.4.0.Beta1 > > > With ISPN-8905 we added in support for segmented stores as well as a trivial method to take an existing store and replicate it for each segment. > RocksDB however has a built in support for segmentation via what they call as column families. We should try these instead which should reduce file descriptor usage and possibly increase performance. > https://github.com/facebook/rocksdb/wiki/Column-Families > https://github.com/facebook/rocksdb/blob/master/java/samples/src/main/jav... -- This message was sent by Atlassian JIRA (v7.5.0#75005)

5 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-7168) Prevent user from configuring passivation with a shared store

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-7168?page=com.atlassian.jira.plugin.... ] William Burns updated ISPN-7168: -------------------------------- Sprint: Sprint 9.4.0.Beta1 > Prevent user from configuring passivation with a shared store > ------------------------------------------------------------- > > Key: ISPN-7168 > URL: https://issues.jboss.org/browse/ISPN-7168 > Project: Infinispan > Issue Type: Bug > Affects Versions: 8.2.4.Final > Reporter: William Burns > Assignee: William Burns > Fix For: 9.4.0.Final > > > Passivation only makes sense with a non shared cache store. --The problem is that if an entry is passivated on a shared cache store only the first node will be able to activate the entry and subsequent nodes will always see null.-- > Looking closer the bigger issue is if an entry becomes passivated and then subsequently written to you will have a different value in the store as you would in memory. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

5 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-9366) Cache.size optimizations

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-9366?page=com.atlassian.jira.plugin.... ] William Burns updated ISPN-9366: -------------------------------- Sprint: Sprint 9.4.0.CR1 > Cache.size optimizations > ------------------------ > > Key: ISPN-9366 > URL: https://issues.jboss.org/browse/ISPN-9366 > Project: Infinispan > Issue Type: Enhancement > Reporter: William Burns > Assignee: William Burns > Fix For: 9.4.0.Final > > > Now that we have a segmented data container and segmented store we can add optimizations for the Cache.size operation in various configurations. > If a store is configured, passivation must be disabled for any of these optimizations related to stores to work > # Shared store can just invoke Store.size > # No store & no expired entries can invoke DataContainer.sizeIncludingExpired(IntSet) > # Non shared store can invoke Store.size(IntSet) -- This message was sent by Atlassian JIRA (v7.5.0#75005)

5 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-5037) Do not replicate value from backup owner

by Radim Vansa (JIRA)

[ https://issues.jboss.org/browse/ISPN-5037?page=com.atlassian.jira.plugin.... ] Radim Vansa commented on ISPN-5037: ----------------------------------- [~dan.berindei] Server does not use triangle; have you checked if this is still an issue there? Though it's a bit strange that we'd keep such obvious oversight for too long... > Do not replicate value from backup owner > ---------------------------------------- > > Key: ISPN-5037 > URL: https://issues.jboss.org/browse/ISPN-5037 > Project: Infinispan > Issue Type: Enhancement > Components: Core > Reporter: Radim Vansa > Assignee: Dan Berindei > Labels: 7.0 > > When a write is replicated from primary owner to backup owner, the backup owner returns the old value. This is unnecessary as the primary owner ignores this value, and introduces networking and marshalling overhead. -- This message was sent by Atlassian JIRA (v7.5.0#75005)

5 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-9261) Hot Rod Lightweight Transaction (recovery)

by Tristan Tarrant (JIRA)

[ https://issues.jboss.org/browse/ISPN-9261?page=com.atlassian.jira.plugin.... ] Tristan Tarrant updated ISPN-9261: ---------------------------------- Sprint: Sprint 9.4.0.Beta1 (was: Sprint 9.4.0.Alpha1) > Hot Rod Lightweight Transaction (recovery) > ------------------------------------------ > > Key: ISPN-9261 > URL: https://issues.jboss.org/browse/ISPN-9261 > Project: Infinispan > Issue Type: Feature Request > Components: Hot Rod, Remote Protocols, Transactions > Reporter: Pedro Ruivo > Assignee: Pedro Ruivo > Fix For: 9.4.0.Beta1 > > > Recovery support for Hot Rod Transactions -- This message was sent by Atlassian JIRA (v7.5.0#75005)

5 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-9127) Remote commands can access components before they are started

by Tristan Tarrant (JIRA)

[ https://issues.jboss.org/browse/ISPN-9127?page=com.atlassian.jira.plugin.... ] Tristan Tarrant updated ISPN-9127: ---------------------------------- Sprint: Sprint 9.4.0.Beta1 (was: Sprint 9.4.0.Alpha1) > Remote commands can access components before they are started > ------------------------------------------------------------- > > Key: ISPN-9127 > URL: https://issues.jboss.org/browse/ISPN-9127 > Project: Infinispan > Issue Type: Bug > Components: Core > Affects Versions: 9.2.2.Final, 9.3.0.Alpha1 > Reporter: Dan Berindei > Assignee: Dan Berindei > Labels: testsuite_stability > > {{PerCacheInboundInvocationHandler.handle()}} may be called before the component was started, because {{GlobalInboundInvocationHandler}} fetches it from the component registry without any checks. {{CommandsFactoryImpl.initializeReplicableCommand()}} doesn't wait for the components that it injects into remote commands to be started, either. > This started causing random test failures in {{ConcurrentStartForkChannelTest}} after ISPN-8515, which moved most initialization work from {{init()}} methods to {{start()}} methods. Because {{StateProviderImpl}} starts after {{StateTransferManagerImpl}}, it's possible for a node to receive a {{StateRequestCommand}} before {{StateProviderImpl}} has initialized: > {noformat} > 16:15:09,549 TRACE (remote-thread-Test-NodeB-p51957-t2:[org.infinispan.CONFIG]) [StateProviderImpl] Starting outbound transfer to node Test-NodeA for cache null, topology id 2, segments {0-255} > 16:15:09,551 WARN (remote-thread-Test-NodeB-p51957-t2:[]) [NonTotalOrderPerCacheInboundInvocationHandler] ISPN000071: Caught exception when handling command StateRequestCommand{cache=org.infinispan.CONFIG, origin=Test-NodeA, type=START_STATE_TRANSFER, topologyId=2, segments={0-255}} > java.lang.IllegalArgumentException: chunkSize must be greater than 0 > at org.infinispan.statetransfer.OutboundTransferTask.<init>(OutboundTransferTask.java:114) ~[classes/:?] > at org.infinispan.statetransfer.StateProviderImpl.startOutboundTransfer(StateProviderImpl.java:273) ~[classes/:?] > at org.infinispan.statetransfer.StateRequestCommand.invokeAsync(StateRequestCommand.java:101) ~[classes/:?] > at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokeCommand(BasePerCacheInboundInvocationHandler.java:94) ~[classes/:?] > {noformat} -- This message was sent by Atlassian JIRA (v7.5.0#75005)

5 years, 10 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues July 2018