Hi guys, It's week 11 of GSoC, which means we only have about 3 weeks left until the end! Here's a quick summary of what we've been doing: * Right now I'm currently working on implementing custom object serialisers and deserialisers to give us the ability to produce whatever we want to our topics. It's been a bit of a bumpy ride so far but Matthias just did a PR <https://github.com/matzew/kafka-cdi/pull/16> which resolves some of the issues we were having with generics. We'll hopefully have this sorted by the end of the week, so we can start using it for the producers/consumers/streams that we implement. * Polina has already done AGPUSH-2159 <https://issues.jboss.org/browse/AGPUSH-2159> related to our first "custom" producer but hasn't sent a PR off yet, given the aforementioned issues. She's also currently working on getting a fancier test environment up for all of our future consumers and producers :) * Since we have a very short amount of time left after that, we'll jump right into some more implementation with AGPUSH-2165 <https://issues.jboss.org/browse/AGPUSH-2165> and AGPUSH-2166 <https://issues.jboss.org/browse/AGPUSH-2166>;, which we'll be working on in our next sprint. That's it, just wanted to update on what's going on. Thanks for the feedback and help, we always appreciate it! Dimitra _______________________________________________ aerogear-dev mailing list aerogear-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/aerogear-dev

Hi guys, here's the PR for the generic ser/des: https://github.com/ matzew/kafka-cdi/pull/17 <https://github.com/matzew/kafka-cdi/pull/17> As far as testing goes, there are a few issues with abstract classes obviously, and since most of the objects we'll be producing contain 'Variant' (which is an abstract class) we'll have to come up with a nice solution for that - we're open to ideas!

Right now it won't work "out of the box" with Polina's PR due to the List<Variant> in the PushApplication class: Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of org.jboss.aerogear.unifiedpush.api.Variant, problem: abstract types either need to be mapped to concrete types, have custom deserializer, or be instantiated with additional type information at [Source: [B@d4163ef; line: 1, column: 229] (through reference chain: org.jboss.aerogear.unifiedpush.api.PushApplication["variants"])

Two other things: * Bad things will happen if objects of incorrect type are pushed to the same topic.

This will definitely have to be addressed by enforcing some basic security (AGPUSH-2109 <https://issues.jboss.org/browse/AGPUSH-2109>)

* We'll have to make sure our logs are a little more "informative" as we can't log whole objects obviously for readability.

Right now, even with just two consumers, the log is quite overwhelming (i.e. consumer configs etc)

On Tue, Aug 8, 2017 at 9:42 PM, Dimitra Zuccarelli < dimitrazuccarelli(a)gmail.com&gt; wrote: > Hi guys, > > It's week 11 of GSoC, which means we only have about 3 weeks left until > the end! Here's a quick summary of what we've been doing: > > * Right now I'm currently working on implementing custom object > serialisers and deserialisers to give us the ability to produce whatever we > want to our topics. > It's been a bit of a bumpy ride so far but Matthias just did a PR > <https://github.com/matzew/kafka-cdi/pull/16> which resolves some of the > issues we were having with generics. We'll hopefully have this sorted by > the end of the week, so we can start using it for the > producers/consumers/streams that we implement. > > * Polina has already done AGPUSH-2159 > <https://issues.jboss.org/browse/AGPUSH-2159> related to our first > "custom" producer but hasn't sent a PR off yet, given the aforementioned > issues. She's also currently working on getting a fancier test environment > up for all of our future consumers and producers :) > > * Since we have a very short amount of time left after that, we'll jump > right into some more implementation with AGPUSH-2165 > <https://issues.jboss.org/browse/AGPUSH-2165> and AGPUSH-2166 > <https://issues.jboss.org/browse/AGPUSH-2166>;, which we'll be working on > in our next sprint. > > That's it, just wanted to update on what's going on. > Thanks for the feedback and help, we always appreciate it! > > Dimitra > _______________________________________________ aerogear-dev mailing list aerogear-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/aerogear-dev

Thanks Matthias! :) On Wed, Aug 9, 2017 at 9:46 AM, Matthias Wessendorf <matzew(a)apache.org&gt; wrote: > > > On Wed, Aug 9, 2017 at 10:35 AM, Dimitra Zuccarelli < > dimitrazuccarelli(a)gmail.com&gt; wrote: > >> Hi guys, here's the PR for the generic ser/des: https://github.com/ma >> tzew/kafka-cdi/pull/17 <https://github.com/matzew/kafka-cdi/pull/17> >> >> As far as testing goes, there are a few issues with abstract classes >> obviously, and since most of the objects we'll be producing contain >> 'Variant' (which is an abstract class) we'll have to come up with a nice >> solution for that - we're open to ideas! >> > > not sure I follow :-) but why not using one of the concrete impls of it ? > Because then we'd have to define a producer/consumer for every concrete implementation,

like SimpleKafkaProducer<String, AndroidVariant>, SimpleKafkaProducer<String, AdmVariant> etc.. I think that would be extremely inefficient

> > >> >> Right now it won't work "out of the box" with Polina's PR due to the >> List<Variant> in the PushApplication class: >> >> Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not >> construct instance of org.jboss.aerogear.unifiedpush.api.Variant, >> problem: abstract types either need to be mapped to concrete types, have >> custom deserializer, or be instantiated with additional type information >> >> at [Source: [B@d4163ef; line: 1, column: 229] (through reference >> chain: org.jboss.aerogear.unifiedpush.api.PushApplication["variants"]) >> >> >> >> > do we need to define some mapping in UPS? > What do you mean? > > >> Two other things: >> >> >> * Bad things will happen if objects of incorrect type are pushed to the >> same topic. >> > > when or how ? any code/pointers? > > >> This will definitely have to be addressed by enforcing some basic >> security (AGPUSH-2109 <https://issues.jboss.org/browse/AGPUSH-2109>) >> > > >> * We'll have to make sure our logs are a little more "informative" as we >> can't log whole objects obviously for readability. >> > +1 > > >> Right now, even with just two consumers, the log is quite overwhelming >> (i.e. consumer configs etc) >> > > we could make the kafka own logging quiet ;-) That's IMO the verbose one, > which is sometimes handy. > > But in WildFly we could limit those to WARN/ERROR > +1 It's handy for development but otherwise we should limit it I think > > >> >> On Tue, Aug 8, 2017 at 9:42 PM, Dimitra Zuccarelli < >> dimitrazuccarelli(a)gmail.com&gt; wrote: >> >>> Hi guys, >>> >>> It's week 11 of GSoC, which means we only have about 3 weeks left until >>> the end! Here's a quick summary of what we've been doing: >>> >>> * Right now I'm currently working on implementing custom object >>> serialisers and deserialisers to give us the ability to produce whatever we >>> want to our topics. >>> It's been a bit of a bumpy ride so far but Matthias just did a PR >>> <https://github.com/matzew/kafka-cdi/pull/16> which resolves some of >>> the issues we were having with generics. We'll hopefully have this sorted >>> by the end of the week, so we can start using it for the >>> producers/consumers/streams that we implement. >>> >>> * Polina has already done AGPUSH-2159 >>> <https://issues.jboss.org/browse/AGPUSH-2159> related to our first >>> "custom" producer but hasn't sent a PR off yet, given the aforementioned >>> issues. She's also currently working on getting a fancier test environment >>> up for all of our future consumers and producers :) >>> >>> * Since we have a very short amount of time left after that, we'll jump >>> right into some more implementation with AGPUSH-2165 >>> <https://issues.jboss.org/browse/AGPUSH-2165> and AGPUSH-2166 >>> <https://issues.jboss.org/browse/AGPUSH-2166>;, which we'll be working >>> on in our next sprint. >>> >>> That's it, just wanted to update on what's going on. >>> Thanks for the feedback and help, we always appreciate it! >>> >>> Dimitra >>> >> >> >> _______________________________________________ >> aerogear-dev mailing list >> aerogear-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/aerogear-dev >> > > > > -- > Matthias Wessendorf > > blog: http://matthiaswessendorf.wordpress.com/ > twitter: http://twitter.com/mwessendorf > > _______________________________________________ > aerogear-dev mailing list > aerogear-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/aerogear-dev > _______________________________________________ aerogear-dev mailing list aerogear-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/aerogear-dev