[Hawkular-dev] Event Store

[Gary Brown]

----- Original Message -----
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> On 05/08/2015 10:57 AM, Gary Brown wrote:
> > Hi
> > 
> > As discussed yesterday, we need a general event store that can
> > persist relevant events off the bus. However we didn't decide where
> > this functionality should reside, whether in an existing component
> > or in a new component.
> > 
> > Although it could be provided as a separate component, I am not
> > sure we have any use cases where it would be used independent of
> > the bus, so think it is probably best to include it as additional
> > functionality within the Bus component.
> > 
> > Agree/disagree?
> 
> With the micro-services approach in mind, I'd vote for a separate
> component for the event-store. Such component has a potential to grow
> in scope on the long term.
> 
> > The other issues are:
> > 
> > - how to determine which events should be persisted, as we may want
> > some events to trigger further processing by subscribers on the
> > bus, but not necessarily be stored for historic analysis. - for the
> > event store we will need to know the type of the event being
> > stored
> > 
> > So one possible solution to both these issues is, if the 'type' of
> > the event is provided (possibly as a header value), then it will be
> > persisted?
> 
> Could you give some examples of what "type" would be? I remember
> hearing "deployment" as one of the possible types, but would be
> interesting to think of some other cases.

Type could be anything, so a logical label such as "Log entry", "Alert", "BTMFragment", etc - just so that from an Elasticsearch perspective we could define a basic schema to identify how it should index the information for that type. Elasticsearch can dynamically create a schema based on the information it receives, but there are cases where you need to control how it processes the data (e.g. UUIDs) where you don't want it to analyse the field further.

> 
> I think there could also be a "group" or "category". This way, it
> would be possible to easily switch the persist/discard flag for a
> given set of types, depending on the current needs. For instance,
> "store all events for category 'application'" might be useful during
> an outage.
> 

Yes this may be useful.

It may be possible to simply identify certain topics that should be stored, so all events on that topic are persisted, and the topic essentially becomes the type name (or maps onto a type name). We may need further information about how the components are using the Bus to understand what is the best approach.


> > Then within Elasticsearch, we can use the tenant/persona as the
> > Index (to truly segment the data) and the event type to ensure
> > different information types are partitioned/indexed appropriately.
> 
> +1
> 
> - - Juca.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
> 
> iQEcBAEBCAAGBQJVTH5pAAoJECKM1e+fkPrXFWEH+gP8tc0TmBk4ZBwuB890Y2pk
> 2/rZ6T/BPvtypT6WtgGd4p7/kWfkFmP9C2OPAA9lb3arv/QG18MDky79vVwNJ1ZO
> 992e5otXCCb8CSJ27XFnItOM/PCBrOk/mi1K0orWm5t1ykKffu5iSN3ftzym/CQ7
> orlAfjqXN4iSfQGzHrIwkgQGbLBRkTNvtAkvd35s8p1vKBpNE1c68q+DteXgy8lq
> an17X5mOvmqnZaoJZfn7mN6eFZmCxa8YPWAICYVUFKx2yzZI37e1G4yc/ckgObw1
> AnS+SmhOO52T3wq6lqoOzhTz3Tvu/GjLYYR52HiMDzu2ru133eN1sHeEm/7jCXo=
> =suCy
> -----END PGP SIGNATURE-----
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>