Hi, We have an issue with protobuf metadata cache. If you run in a multi-site scenario, protobuf metadata information does not travel across sites by default. Being an internal cache, is it possible to somehow override/reconfigure it so that cross-site configuration can be added in standalone.xml?

We're currently running a periodic job that checks if the metadata is present and if not present add it. So, we have a workaround for it, but it'd be not very user friendly for end users. Thoughts?

Cheers, Galder _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Ok, we do need to find a better way to deal with this. JIRA: https://issues.jboss.org/browse/ISPN-9074 On Thu, Apr 12, 2018 at 5:56 PM Pedro Ruivo <pedro(a)infinispan.org <mailto:pedro@infinispan.org>> wrote: On 12-04-2018 15:49, Galder Zamarreno wrote: > Hi, > > We have an issue with protobuf metadata cache. > > If you run in a multi-site scenario, protobuf metadata information does > not travel across sites by default. > > Being an internal cache, is it possible to somehow override/reconfigure > it so that cross-site configuration can be added in standalone.xml? No :( since it is an internal cache, its configuration can't be changed. > > We're currently running a periodic job that checks if the metadata is > present and if not present add it. So, we have a workaround for it, but > it'd be not very user friendly for end users. > > Thoughts? Unfortunately none... it is the first time an internal cache needs to do some x-site. > > Cheers, > Galder > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

It is definitely an internal cache. Because of this, automatically backing it up to a remote site might not be such a good idea. Backups are enabled per-cache, and therefore just blindly replicating the schema cache to the other site is not a good idea. I think that we need a cache-manager-level backup setting that does the right thing. Tristan On 4/12/18 7:01 PM, Pedro Ruivo wrote: > Wouldn't be better to assume the protobuf cache doesn't fit the internal > cache use case? :) > > On 12-04-2018 17:21, Galder Zamarreno wrote: >> Ok, we do need to find a better way to deal with this. >> >> JIRA: https://issues.jboss.org/browse/ISPN-9074 >> >> On Thu, Apr 12, 2018 at 5:56 PM Pedro Ruivo <pedro(a)infinispan.org >> <mailto:pedro@infinispan.org>> wrote: >> >> >> >> On 12-04-2018 15:49, Galder Zamarreno wrote: >> > Hi, >> > >> > We have an issue with protobuf metadata cache. >> > >> > If you run in a multi-site scenario, protobuf metadata >> information does >> > not travel across sites by default. >> > >> > Being an internal cache, is it possible to somehow >> override/reconfigure >> > it so that cross-site configuration can be added in standalone.xml? >> >> No :( since it is an internal cache, its configuration can't be changed. >> >> > >> > We're currently running a periodic job that checks if the >> metadata is >> > present and if not present add it. So, we have a workaround for >> it, but >> > it'd be not very user friendly for end users. >> > >> > Thoughts? >> >> Unfortunately none... it is the first time an internal cache needs >> to do >> some x-site. >> >> > >> > Cheers, >> > Galder >> > >> > >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev(a)lists.jboss.org >> <mailto:infinispan-dev@lists.jboss.org> >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev >

I think we can certainly make it additive, especially now that we have configuration templates in place: the user supplies a base template, and the internal cache logic override with what is needed so that broken configs are less probable (but still possible). Alternatively, instead of overriding, we just check that it matches the requirements. Tristan On 4/12/18 10:10 PM, Adrian Nistor wrote: > Backing up caches with protobuf payload to a remote site will not work > if they are indexed, unless the remote site already has the schema for > the types in question, or else indexing will fail. If the cache is not > indexed it matters less. > > So the replication of protobuf metadata cache has to be arranged > somehow before any other data is replicated. Manual replication is > indeed PITA. > > I remember in the very early version of remote query the protobuf > metadata cache configuration was created programatically on startup > unless a manually defined configuration with that name was found, > already provided by the user. In that case the user's config was used. > This approach had the benefit of allowing the user to gain control if > needed. But can also lead to gloom and doom. Was that too bad to do it > again :)))? > > Adrian > > On 04/12/2018 10:27 PM, Tristan Tarrant wrote: >> It is definitely an internal cache. Because of this, automatically >> backing it up to a remote site might not be such a good idea. >> >> Backups are enabled per-cache, and therefore just blindly replicating >> the schema cache to the other site is not a good idea. >> >> I think that we need a cache-manager-level backup setting that does the >> right thing. >> >> Tristan >> >> On 4/12/18 7:01 PM, Pedro Ruivo wrote: >>> Wouldn't be better to assume the protobuf cache doesn't fit the >>> internal >>> cache use case? :) >>> >>> On 12-04-2018 17:21, Galder Zamarreno wrote: >>>> Ok, we do need to find a better way to deal with this. >>>> >>>> JIRA: https://issues.jboss.org/browse/ISPN-9074 >>>> >>>> On Thu, Apr 12, 2018 at 5:56 PM Pedro Ruivo <pedro(a)infinispan.org >>>> <mailto:pedro@infinispan.org>> wrote: >>>> >>>> >>>> >>>>       On 12-04-2018 15:49, Galder Zamarreno wrote: >>>>        > Hi, >>>>        > >>>>        > We have an issue with protobuf metadata cache. >>>>        > >>>>        > If you run in a multi-site scenario, protobuf metadata >>>>       information does >>>>        > not travel across sites by default. >>>>        > >>>>        > Being an internal cache, is it possible to somehow >>>>       override/reconfigure >>>>        > it so that cross-site configuration can be added in >>>> standalone.xml? >>>> >>>>       No :( since it is an internal cache, its configuration can't >>>> be changed. >>>> >>>>        > >>>>        > We're currently running a periodic job that checks if the >>>>       metadata is >>>>        > present and if not present add it. So, we have a >>>> workaround for >>>>       it, but >>>>        > it'd be not very user friendly for end users. >>>>        > >>>>        > Thoughts? >>>> >>>>       Unfortunately none... it is the first time an internal cache >>>> needs >>>>       to do >>>>       some x-site. >>>> >>>>        > >>>>        > Cheers, >>>>        > Galder >>>>        > >>>>        > >>>>        > _______________________________________________ >>>>        > infinispan-dev mailing list >>>>        > infinispan-dev(a)lists.jboss.org >>>>       <mailto:infinispan-dev@lists.jboss.org> >>>>        > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>        > >>>>       _______________________________________________ >>>>       infinispan-dev mailing list >>>>       infinispan-dev(a)lists.jboss.org >>>> <mailto:infinispan-dev@lists.jboss.org> >>>>       https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >