3:11 p.m.
Hello Everybody,
With the metrics aggregation work well under way, we have encountered the first case where
timezones are important. The task queue [1] that John wrote to be used in the aggregation
process is time sensitive. The tasks ran according to a schedule that is predefined; the
time relationship also applies to failures, retries, and cancellation. Based on the
experience we had with RHQ, not using an UTC timezone can lead to errors and loss of data
(for more details visit [2])
I propose to set the default time zone for all the services to UTC. Only allow the concept
of a timezone in the UI. This will avoid any problems due to automatic transitions between
DST and non-DST times. For example, if the user wants to schedule something 2 weeks from
now, let the UI handle the user selection. But when the user action is sent to the
backend, it should be expressed in an timezone agnostic manner (eg. milliseconds since
Unix Epoch).
The PR [1] sets the default timezone for the JODA library to be UTC for the classloader
that contains the metrics service. With the recent discussions about creating a combined
Kettle EAR or using a single classloader, that means this setting will extend to all the
services in the same classloader that use JODA. And this is just the default, if a service
really needs to set another timezone for a date object, it can be done by just specifying
the timezone.
We tried to make the change more local in the metrics code, but the approach was error
prone. Anybody could make a mistake and instantiate a date object in the current server
timezone rather than UTC. This global approach drastically limits inadvertent mistakes.
Two questions for the group:
1) Does anybody see any issues with using the UTC timezone as default at the services
layer?
2) Should we expand this setting to the JVM (not just JODA)?
[1] Task queue PR - https://github.com/hawkular/hawkular-metrics/pull/205
[2] Old RHQ thread -
https://lists.fedorahosted.org/pipermail/rhq-devel/2014-November/003709.html
Thank you,
Stefan Negrea
Software Engineer
3:44 p.m.
Le 13/05/2015 21:11, Stefan Negrea a écrit :
Hello Everybody,
With the metrics aggregation work well under way, we have encountered the first case
where timezones are important. The task queue [1] that John wrote to be used in the
aggregation process is time sensitive. The tasks ran according to a schedule that is
predefined; the time relationship also applies to failures, retries, and cancellation.
Based on the experience we had with RHQ, not using an UTC timezone can lead to errors and
loss of data (for more details visit [2])
I propose to set the default time zone for all the services to UTC. Only allow the
concept of a timezone in the UI. This will avoid any problems due to automatic transitions
between DST and non-DST times. For example, if the user wants to schedule something 2
weeks from now, let the UI handle the user selection. But when the user action is sent to
the backend, it should be expressed in an timezone agnostic manner (eg. milliseconds since
Unix Epoch).
The PR [1] sets the default timezone for the JODA library to be UTC for the classloader
that contains the metrics service. With the recent discussions about creating a combined
Kettle EAR or using a single classloader, that means this setting will extend to all the
services in the same classloader that use JODA. And this is just the default, if a service
really needs to set another timezone for a date object, it can be done by just specifying
the timezone.
We tried to make the change more local in the metrics code, but the approach was error
prone. Anybody could make a mistake and instantiate a date object in the current server
timezone rather than UTC. This global approach drastically limits inadvertent mistakes.
Two questions for the group:
1) Does anybody see any issues with using the UTC timezone as default at the services
layer?
I think it is error prone to have any joda.Date you create set in the
UTC timezone, and any util.Date you create in the server timezone.
Also I can't see where the problem to create the joda.Instant objects
properly in the scheduler utility class. Maybe I'm missing something?
2) Should we expand this setting to the JVM (not just JODA)?
-1
To make this properly the JVM must be started with user.timezone
property set to UTC. Otherwise we don't control what happens at the
server level or in libraries, because the setting is effective only when
the class is loaded.
In the end, the system would rely on external config.
Also if we do this, it means than most users will have to configure all
the loggers template, because you usually want your logs printed in the
timezone of your hosts, but the default templates will print the logs
for the JVM timezone.
I fear it's not the only annoyance which would show up.
[1] Task queue PR - https://github.com/hawkular/hawkular-metrics/pull/205
[2] Old RHQ thread -
https://lists.fedorahosted.org/pipermail/rhq-devel/2014-November/003709.html
Thank you,
Stefan Negrea
Software Engineer
_______________________________________________
hawkular-dev mailing list
hawkular-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hawkular-dev
10:04 a.m.
On May 13, 2015, at 3:44 PM, Thomas Segismont
<tsegismo(a)redhat.com> wrote:
Le 13/05/2015 21:11, Stefan Negrea a écrit :
> Hello Everybody,
>
> With the metrics aggregation work well under way, we have encountered the first case
where timezones are important. The task queue [1] that John wrote to be used in the
aggregation process is time sensitive. The tasks ran according to a schedule that is
predefined; the time relationship also applies to failures, retries, and cancellation.
Based on the experience we had with RHQ, not using an UTC timezone can lead to errors and
loss of data (for more details visit [2])
>
> I propose to set the default time zone for all the services to UTC. Only allow the
concept of a timezone in the UI. This will avoid any problems due to automatic transitions
between DST and non-DST times. For example, if the user wants to schedule something 2
weeks from now, let the UI handle the user selection. But when the user action is sent to
the backend, it should be expressed in an timezone agnostic manner (eg. milliseconds since
Unix Epoch).
>
> The PR [1] sets the default timezone for the JODA library to be UTC for the
classloader that contains the metrics service. With the recent discussions about creating
a combined Kettle EAR or using a single classloader, that means this setting will extend
to all the services in the same classloader that use JODA. And this is just the default,
if a service really needs to set another timezone for a date object, it can be done by
just specifying the timezone.
>
> We tried to make the change more local in the metrics code, but the approach was
error prone. Anybody could make a mistake and instantiate a date object in the current
server timezone rather than UTC. This global approach drastically limits inadvertent
mistakes.
>
>
> Two questions for the group:
> 1) Does anybody see any issues with using the UTC timezone as default at the services
layer?
I think it is error prone to have any joda.Date you create set in the
UTC timezone, and any util.Date you create in the server timezone.
Also I can't see where the problem to create the joda.Instant objects
properly in the scheduler utility class. Maybe I'm missing something?
The challenge is that the time zone configuration is not just limited to a single class.
Right now there are only a few places where DateTime objects are created and where the
time zone needs to be set. AS the task scheduler code grows, the number of places where we
have to worry about configuring the time zone may as well. I think that there are good
arguments being made for and against the global setting.
The one thing I ask right now though is whether we are ok with merging the branch into
master while this discussion is ongoing. If we decide that changes are needed, they can be
made after the merge. Right now other work is going to be held up until the task-queue
branch gets merged.
> 2) Should we expand this setting to the JVM (not just JODA)?
-1
To make this properly the JVM must be started with user.timezone
property set to UTC. Otherwise we don't control what happens at the
server level or in libraries, because the setting is effective only when
the class is loaded.
In the end, the system would rely on external config.
Also if we do this, it means than most users will have to configure all
the loggers template, because you usually want your logs printed in the
timezone of your hosts, but the default templates will print the logs
for the JVM timezone.
I fear it's not the only annoyance which would show up.
>
>
> [1] Task queue PR - https://github.com/hawkular/hawkular-metrics/pull/205
<https://github.com/hawkular/hawkular-metrics/pull/205>
> [2] Old RHQ thread -
https://lists.fedorahosted.org/pipermail/rhq-devel/2014-November/003709.html
<https://lists.fedorahosted.org/pipermail/rhq-devel/2014-November/003709.h...
>
>
>
> Thank you,
> Stefan Negrea
>
> Software Engineer
>
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev(a)lists.jboss.org <mailto:hawkular-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
<https://lists.jboss.org/mailman/listinfo/hawkular-dev>
>
_______________________________________________
hawkular-dev mailing list
hawkular-dev(a)lists.jboss.org <mailto:hawkular-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/hawkular-dev
<https://lists.jboss.org/mailman/listinfo/hawkular-dev>
4:21 a.m.
On 13 May 2015, at 21:44, Thomas Segismont wrote:
> 2) Should we expand this setting to the JVM (not just JODA)?
In the end, the system would rely on external config.
Don't we do that in several places already?
Where do you see the issue to put that property in
standalone.sh (.conf) ?
If we run metrics inside of e.g. OpenShift. Do we have enough
control to run "our" TZ-setting? What if all other parts of OpenShift
(incl. metrics feeds) run in Boston-time - how will we deal with that?
Also if we do this, it means than most users will have to configure
all
the loggers template, because you usually want your logs printed in
Which templates do you mean here?
For the Hawkular-services, could we have a common
time module, that does the right think when you do
java.util.Date now = HawkularTimeService.getNow();
instead of
java.util.Date now = new Date();
4:33 a.m.
Le 18/05/2015 10:21, Heiko W.Rupp a écrit :
On 13 May 2015, at 21:44, Thomas Segismont wrote:
>
>> 2) Should we expand this setting to the JVM (not just JODA)?
>
> In the end, the system would rely on external config.
Don't we do that in several places already?
I don't think so.
Where do you see the issue to put that property in
standalone.sh (.conf) ?
Because it wouldn't be configuration of the system, it would be a
requirement of the system.
If we run metrics inside of e.g. OpenShift. Do we have enough
control to run "our" TZ-setting? What if all other parts of OpenShift
(incl. metrics feeds) run in Boston-time - how will we deal with that?
> Also if we do this, it means than most users will have to configure all
> the loggers template, because you usually want your logs printed in
Which templates do you mean here?
What log4j calls "patterns" for pattern layouts.
6:07 a.m.
On 18 May 2015, at 10:33, Thomas Segismont wrote:
>> Also if we do this, it means than most users will have to
configure all
>> the loggers template, because you usually want your logs printed in
>
> Which templates do you mean here?
What log4j calls "patterns" for pattern layouts.
Yes :-)
What I meant is for the loggers of ours software.
Or of the clients like managed WildFly servers.
If the later - wouldn't they need to modify something
anyway to send it to us?
Heiko
7:46 a.m.
Le 18/05/2015 12:07, Heiko W.Rupp a écrit :
On 18 May 2015, at 10:33, Thomas Segismont wrote:
>>> >>>Also if we do this, it means than most users will have to
configure all
>>> >>>the loggers template, because you usually want your logs printed
in
>> >>
>> >>Which templates do you mean here?
> >
> >What log4j calls "patterns" for pattern layouts.
Yes:-)
What I meant is for the loggers of ours software.
Or of the clients like managed WildFly servers.
If the later - wouldn't they need to modify something
anyway to send it to us?
The clients issue is something I was concerned about too. But we had a
conversation with Stefan and as long as clients send UNIX timestamps,
everything should work fine (time since Jan,1 1970 00:00:00 UTC,
regardless of the resolution).
If we start to accept instant information from clients in the form of
dates, then we'll need to make sure they send the TZ info as well.
9:32 a.m.
On 18/05/15 07:46 AM, Thomas Segismont wrote:
Le 18/05/2015 12:07, Heiko W.Rupp a écrit :
> On 18 May 2015, at 10:33, Thomas Segismont wrote:
>>>>>>> Also if we do this, it means than most users will have to
configure all
>>>>>>> the loggers template, because you usually want your logs
printed in
>>>>> Which templates do you mean here?
>>> What log4j calls "patterns" for pattern layouts.
> Yes:-)
> What I meant is for the loggers of ours software.
> Or of the clients like managed WildFly servers.
> If the later - wouldn't they need to modify something
> anyway to send it to us?
The clients issue is something I was concerned about too. But we had a
conversation with Stefan and as long as clients send UNIX timestamps,
everything should work fine (time since Jan,1 1970 00:00:00 UTC,
regardless of the resolution).
For the timestamps, are you just looking at just seconds since the
epoch, milliseconds, nanoseconds? Is the timestamp going to just be a
simple numeric value that is passed (eg we can only ever pass
milliseconds since epoch) or something slightly more complex which
includes the resolution type?
And I assume that the machines internal clock being properly set and
periodically synchronized is outside of the scope of what Hawkular wants
to do.
3:28 a.m.
Le 19/05/2015 15:32, Matt Wringe a écrit :
On 18/05/15 07:46 AM, Thomas Segismont wrote:
> Le 18/05/2015 12:07, Heiko W.Rupp a écrit :
>> On 18 May 2015, at 10:33, Thomas Segismont wrote:
>>>>>>>> Also if we do this, it means than most users will have to
configure all
>>>>>>>> the loggers template, because you usually want your logs
printed in
>>>>>> Which templates do you mean here?
>>>> What log4j calls "patterns" for pattern layouts.
>> Yes:-)
>> What I meant is for the loggers of ours software.
>> Or of the clients like managed WildFly servers.
>> If the later - wouldn't they need to modify something
>> anyway to send it to us?
> The clients issue is something I was concerned about too. But we had a
> conversation with Stefan and as long as clients send UNIX timestamps,
> everything should work fine (time since Jan,1 1970 00:00:00 UTC,
> regardless of the resolution).
>
For the timestamps, are you just looking at just seconds since the
epoch, milliseconds, nanoseconds? Is the timestamp going to just be a
simple numeric value that is passed (eg we can only ever pass
milliseconds since epoch) or something slightly more complex which
includes the resolution type?
All Metrics endpoints expect timestamps in milliseconds since the epoch.
The only exception is the Influx endpoint, where queries support
instants with a resolution.
For example:
select value from serie1 where time < 20y
Which means:
Load values of serie1 recorded in the 20 years following the epoch
And I assume that the machines internal clock being properly set and
periodically synchronized is outside of the scope of what Hawkular wants
to do.
Right.
3303
days inactive
3310
days old
8 comments
5 participants
participants (5)
-
Heiko W.Rupp -
John Sanda -
Matt Wringe -
Stefan Negrea -
Thomas Segismont