[JBoss JIRA] (ISPN-9376) Enhance RemoteStore to support segmentation
by William Burns (JIRA)
William Burns created ISPN-9376:
-----------------------------------
Summary: Enhance RemoteStore to support segmentation
Key: ISPN-9376
URL: https://issues.jboss.org/browse/ISPN-9376
Project: Infinispan
Issue Type: Enhancement
Reporter: William Burns
Fix For: 9.4.0.Final
Now with the introduction of segmented store interface, we can convert the RemoteStore to support segmentation.
One thing to note is that this only works if the remote cache and the local one have the same number of segments. Otherwise data will appear missing.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (ISPN-7409) Replace external marshaller with user marshaller
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-7409?page=com.atlassian.jira.plugin.... ]
Ryan Emerson updated ISPN-7409:
-------------------------------
Status: Open (was: New)
> Replace external marshaller with user marshaller
> ------------------------------------------------
>
> Key: ISPN-7409
> URL: https://issues.jboss.org/browse/ISPN-7409
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core, Marshalling
> Reporter: Galder Zamarreño
> Assignee: Ryan Emerson
>
> ISPN-6906 brings much needed independence from JBoss Marshalling to our marshalling layer. One of the key aspects it brings it’s the separation between marshalling internal types we know about, e.g. String, byte[], CacheTopologyCommand...etc and external or unknown types, e.g. any types that extend Serializable.
> Although this separation helps achieve objectives of ISPN-6906, it’s not an ideal solution:
> * If user types map directly to primitive types supported by our internal marshaller, or types that depend on types that are marshalled by our internal marshaller, any changes made to the internal marshaller have a direct impact on the representation of the user types.
> * In the ISPN-6906 implementation, Internal marshaller exposes its own externalizer table to the external marshaller so that it can lookup how to marshall internal types. This is done to support corner cases in Hibernate Search where some of the classes that Query module marshalls rely on the fact that some Hibernate Search classes are Serializable, but these classes reference Lucene query classes that are not for which there are externalizers defined. For query logic to work, externalizers have to be found while dealing with Serializable types are being traversed. Gustavo and Sane are aware of this issue and we should soon have externalizers for those Hibernate Search classes and avoid this issue altogether (see ISPN-7156).
> So, rather than an external marshaller, a more suitable abstraction would be to have a user marshaller. The job of the user marshaller, which would be configurable, would be to marshall user types. The user marshaller should be independent of the internal marshaller, so it would not be able to piggyback on the internal externalizers.
> Externalizers for user types could still be supported, by having a user externalizer table, purely used by the user marshaller. In fact, it would make sense that all externalizers that are configured in Infinispan (regardless of whether via xml, programmatic or via annotation) to be user externalizers.
> User types are considered to be:
> * Keys and values
> * Metadata instances provided by the user via *withMetadata() calls
> The benefits of the user marshaller are the following:
> * Having the user marshaller configurable would make it easy to try out different marshallers when comparing how key/value types are marshalled, e.g. try out different ways to marshall Strings.
> * An easier upgrade path for user types. Since user types no longer rely on internal marshaller details, Infinispan upgrades are easier to deal with.
> For reference, custom commands that can be plugged via ModuleCommandFactory implementations would use the internal marshaller since these commands often marshall internal types, e.g. Address.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (ISPN-7409) Replace external marshaller with user marshaller
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-7409?page=com.atlassian.jira.plugin.... ]
Ryan Emerson updated ISPN-7409:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/6150
> Replace external marshaller with user marshaller
> ------------------------------------------------
>
> Key: ISPN-7409
> URL: https://issues.jboss.org/browse/ISPN-7409
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core, Marshalling
> Reporter: Galder Zamarreño
> Assignee: Ryan Emerson
>
> ISPN-6906 brings much needed independence from JBoss Marshalling to our marshalling layer. One of the key aspects it brings it’s the separation between marshalling internal types we know about, e.g. String, byte[], CacheTopologyCommand...etc and external or unknown types, e.g. any types that extend Serializable.
> Although this separation helps achieve objectives of ISPN-6906, it’s not an ideal solution:
> * If user types map directly to primitive types supported by our internal marshaller, or types that depend on types that are marshalled by our internal marshaller, any changes made to the internal marshaller have a direct impact on the representation of the user types.
> * In the ISPN-6906 implementation, Internal marshaller exposes its own externalizer table to the external marshaller so that it can lookup how to marshall internal types. This is done to support corner cases in Hibernate Search where some of the classes that Query module marshalls rely on the fact that some Hibernate Search classes are Serializable, but these classes reference Lucene query classes that are not for which there are externalizers defined. For query logic to work, externalizers have to be found while dealing with Serializable types are being traversed. Gustavo and Sane are aware of this issue and we should soon have externalizers for those Hibernate Search classes and avoid this issue altogether (see ISPN-7156).
> So, rather than an external marshaller, a more suitable abstraction would be to have a user marshaller. The job of the user marshaller, which would be configurable, would be to marshall user types. The user marshaller should be independent of the internal marshaller, so it would not be able to piggyback on the internal externalizers.
> Externalizers for user types could still be supported, by having a user externalizer table, purely used by the user marshaller. In fact, it would make sense that all externalizers that are configured in Infinispan (regardless of whether via xml, programmatic or via annotation) to be user externalizers.
> User types are considered to be:
> * Keys and values
> * Metadata instances provided by the user via *withMetadata() calls
> The benefits of the user marshaller are the following:
> * Having the user marshaller configurable would make it easy to try out different marshallers when comparing how key/value types are marshalled, e.g. try out different ways to marshall Strings.
> * An easier upgrade path for user types. Since user types no longer rely on internal marshaller details, Infinispan upgrades are easier to deal with.
> For reference, custom commands that can be plugged via ModuleCommandFactory implementations would use the internal marshaller since these commands often marshall internal types, e.g. Address.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (ISPN-7409) Replace external marshaller with user marshaller
by Ryan Emerson (JIRA)
[ https://issues.jboss.org/browse/ISPN-7409?page=com.atlassian.jira.plugin.... ]
Ryan Emerson reassigned ISPN-7409:
----------------------------------
Assignee: Ryan Emerson
> Replace external marshaller with user marshaller
> ------------------------------------------------
>
> Key: ISPN-7409
> URL: https://issues.jboss.org/browse/ISPN-7409
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core, Marshalling
> Reporter: Galder Zamarreño
> Assignee: Ryan Emerson
>
> ISPN-6906 brings much needed independence from JBoss Marshalling to our marshalling layer. One of the key aspects it brings it’s the separation between marshalling internal types we know about, e.g. String, byte[], CacheTopologyCommand...etc and external or unknown types, e.g. any types that extend Serializable.
> Although this separation helps achieve objectives of ISPN-6906, it’s not an ideal solution:
> * If user types map directly to primitive types supported by our internal marshaller, or types that depend on types that are marshalled by our internal marshaller, any changes made to the internal marshaller have a direct impact on the representation of the user types.
> * In the ISPN-6906 implementation, Internal marshaller exposes its own externalizer table to the external marshaller so that it can lookup how to marshall internal types. This is done to support corner cases in Hibernate Search where some of the classes that Query module marshalls rely on the fact that some Hibernate Search classes are Serializable, but these classes reference Lucene query classes that are not for which there are externalizers defined. For query logic to work, externalizers have to be found while dealing with Serializable types are being traversed. Gustavo and Sane are aware of this issue and we should soon have externalizers for those Hibernate Search classes and avoid this issue altogether (see ISPN-7156).
> So, rather than an external marshaller, a more suitable abstraction would be to have a user marshaller. The job of the user marshaller, which would be configurable, would be to marshall user types. The user marshaller should be independent of the internal marshaller, so it would not be able to piggyback on the internal externalizers.
> Externalizers for user types could still be supported, by having a user externalizer table, purely used by the user marshaller. In fact, it would make sense that all externalizers that are configured in Infinispan (regardless of whether via xml, programmatic or via annotation) to be user externalizers.
> User types are considered to be:
> * Keys and values
> * Metadata instances provided by the user via *withMetadata() calls
> The benefits of the user marshaller are the following:
> * Having the user marshaller configurable would make it easy to try out different marshallers when comparing how key/value types are marshalled, e.g. try out different ways to marshall Strings.
> * An easier upgrade path for user types. Since user types no longer rely on internal marshaller details, Infinispan upgrades are easier to deal with.
> For reference, custom commands that can be plugged via ModuleCommandFactory implementations would use the internal marshaller since these commands often marshall internal types, e.g. Address.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (ISPN-9180) Remove compat mode from Remote Query
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-9180?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-9180:
------------------------------------
Status: Pull Request Sent (was: Coding In Progress)
Git Pull Request: https://github.com/infinispan/infinispan/pull/6149
> Remove compat mode from Remote Query
> ------------------------------------
>
> Key: ISPN-9180
> URL: https://issues.jboss.org/browse/ISPN-9180
> Project: Infinispan
> Issue Type: Sub-task
> Components: Remote Querying
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
>
> The ProtobufMetadataManager is configured internally using compat mode plus the CompatibilityProtostreamMarshaller as a marshaller. This requires that HR clients use the protostream marshaller to register .proto files, since otherwise the server cannot unmarshall it.
> The ProtobufMetadataManager cache should rely instead on the MediaType sent by the HR client and convert the incoming content to java object (String) on the fly, as long as the server supports conversion between the content produced by marshaller and java object.
> Apart from that, there are multiple places in the code that registers/instantiates/configures objects based on the compat mode marshaller, and assume all content will be unmarshalled by this marshaller. It should instead rely on the MediaType information sent by the HR client and do the conversion on the fly, this would allow indexing and querying in multiple formats as long as the storage format is supported and the conversion between the input/storage is supported by the server.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months