Possible architectures: https://docs.google.com/drawings/d/16bVwXE2OV0M6RicihAyX97DFDY3ycEVzyRdySXKi7gI/edit
Before starting the discussion, here is a good summary of what time series data and regular data means:
|| Time series data || Regular data ||
| You normally do inserts | You do inserts, but updates more often |
| Getting some stats about what happened in the last N days | Knowledge base|
ElasticSearch: Supports both.
InfluxDB: Time series data only. No update operations
PostgreSQL: supports both (no hands-on experiment done by the team)
Prometheus: time series data only. Not viable as a storage option anyway.
Another note is that, the trying to building a knowledge base data using queries is not natural thing to do on a time series database. Aggregations would be super-complex (if even possible) and slow (due to very long time range and too many matching documents).
Possible architectures we are going to pursue:
1. As we’re going to store time series and regular data in the same storage, we can use ElasticSearch or PostgreSQL.
Grafana is able to talk to both storage options.
Very very little business logic in the service component. Acts more like a proxy.
!Screen Shot 2018-01-26 at 12.05.50.png|thumbnail!
4. Our service writes time series data to Prometheus and Prometheus writes them to a PostgreSQL.
Our service also talks to PostgreSQL directly for adding regular data.
PostgreSQL is the only option here as it is the only DB that supports time series data and regular data and Prometheus supports it.
!Screen Shot 2018-01-26 at 12.05.56.png|thumbnail!
|
|
