Currently we are storing metric id as any string. There are no restrictions on how we name the metrics, and when you access a single metric via a url you just need to make sure its url encoded properly here. This works fine when dealing with a client and the Hawkular Metrics directly, but can cause problems when dealing with proxies or other intermediary servers which are not exactly url compliant. The big issue is with slashes in the metric id. Even when properly encoded this can cause problems with things like proxies not accepting them (and even certain web container don't like them). Since we need to be able to integrate over networks with these types of servers, we need to find a solution to resolve this in hawkular. Possible solutions - enforce naming of anything which can show up in a the URL to make sure it cannot include a '/' . This would include things like metric Id. This should probably be enabled by default, but we could have an option to disable this if a user needs it. - create a new endpoint which can be used to perform a POST instead of doing a GET on the metric endpoint. This would mean sending things like the metric id and query string as part of the message body in the POST instead of in the URL. Any thoughts? _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

On 18/09/15 11:32 AM, John Mazzitelli wrote: > Metric IDs from our Wildfly Agent contain "/" because those are characters that are valid for Wildfly resources and we use the Wildfly resource names as part of the metric ID to uniquely identify them. Yeah, and the cAdvisor metrics from Heapster also contain slashes, which is where I am seeing the problem right now with OpenShift proxies. Outside of the proxy, then everything works fine with having encoded slashes in the URL, which is why I don't see the issue with my own testing. But it looks like for the UI they may need to use the proxy. Its not just the OpenShift proxies that have this problem, its a common issue with some types proxies and servers as well. > > So for IDs of metrics coming from things like "/deployment=the-app.war/subsystem=ejb3" will have "/" in them. The only alternative would be for us to replace "/" with a character that is valid for URLs but invalid for Wildfly resource names. Say "$deployment=the-app.war$subsystem=ejb3" or something like that. I don't know if it really matter here how wildfly does things. Your client should be able to encode the metricid in one format when going into hawkular and then decode when they get things back from hawkular. I don't know what should be the right solution there: 1) we could enforce what is allowed in a metric id field. This means we break backwards compatibility with what we currently have, but it also means that we can open up the door to things like restricting some metrics to some internal only things (see https://issues.jboss.org/browse/HWKMETRICS-208). 2) we keep our current endpoint as is, but add a new endpoint which can do queries via a post (essentially this is just an escape route to get around the issue when dealing with systems like this). So to do the equivalence of: GET http://localhost:8080/hawkular/metrics/gauges/foo%2fbar/data?start=10&... You could do also do something like: POST http://localhost:8080/hawkular/metrics/gauges/data { "metric-id": foo/bar, "start": 10, "end": 110, "buckets":2 } 3) Band-aid solution to fix the immediate problem: update the heapster sink to encode metric ids in a format which does not include the '/' anywhere in it. This doesn't really solve the problem, but it would at least get us past the problem until we come up with a better, more permanent solution. Any thoughts? > > > ----- Original Message ----- >> Currently we are storing metric id as any string. There are no >> restrictions on how we name the metrics, and when you access a single >> metric via a url you just need to make sure its url encoded properly here. >> >> This works fine when dealing with a client and the Hawkular Metrics >> directly, but can cause problems when dealing with proxies or other >> intermediary servers which are not exactly url compliant. >> >> The big issue is with slashes in the metric id. Even when properly >> encoded this can cause problems with things like proxies not accepting >> them (and even certain web container don't like them). >> >> Since we need to be able to integrate over networks with these types of >> servers, we need to find a solution to resolve this in hawkular. >> >> Possible solutions >> >> - enforce naming of anything which can show up in a the URL to make sure >> it cannot include a '/' . This would include things like metric Id. This >> should probably be enabled by default, but we could have an option to >> disable this if a user needs it. >> >> - create a new endpoint which can be used to perform a POST instead of >> doing a GET on the metric endpoint. This would mean sending things like >> the metric id and query string as part of the message body in the POST >> instead of in the URL. >> >> Any thoughts? >> _______________________________________________ >> hawkular-dev mailing list >> hawkular-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hawkular-dev >> > _______________________________________________ > hawkular-dev mailing list > hawkular-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hawkular-dev _______________________________________________ hawkular-dev mailing list hawkular-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hawkular-dev

I don't think this is a valid reason (as the proxy can be fixed by avoiding Golang's url.Parse() command) to change our implementation and APIs.

This is not just an issue with the Kubernetes proxy, its also an issue with EAP, other web server, other apache components (or at least some older versions), and apparently a bunch of other proxies servers out there. Its a common enough problem in this space that we have already ran into in two places when dealing with OpenShift integration work. I don't think its acceptable to say that since we are compliant, its not our fault and we wont to anything to help get anyone get around this issue. We need to work in the real world and this means having to (potentially) deal with the common problems which exist out there.