[Apiman-user] API policy response data handler
Marc Savy
marc.savy at redhat.com
Fri Feb 3 06:21:52 EST 2017
I think I understand what you're asking.
IIRC:
Error = an exception occurred within the gateway (e.g. could not connect to
backend).
Failure = a policy failure occurred.
If you're wanting to look at http exception codes, just look at the
'responseCode' field in the ES Metrics.
If you're wanting a "failed/not failed" determination then perhaps just
execute `responseCode/100 == 2` in your analysis? Or is this not possible
in your use-case?
On 3 February 2017 at 10:45, Balu S <sbalu27 at gmail.com> wrote:
> Its the default one ES.
>
> # ---------------------------------------------------------------------
> # Elasticsearch Metrics Settings
> # ---------------------------------------------------------------------
>
> apiman-gateway.metrics=io.apiman.gateway.engine.es.ESMetrics
> apiman-gateway.metrics.client.type=jest
> apiman-gateway.metrics.client.protocol=${apiman.es.protocol}
> apiman-gateway.metrics.client.host=${apiman.es.host}
> apiman-gateway.metrics.client.port=${apiman.es.port}
> apiman-gateway.metrics.client.username=${apiman.es.username}
> apiman-gateway.metrics.client.password=${apiman.es.password}
> apiman-gateway.metrics.client.timeout=${apiman.es.timeout}
> apiman-gateway.metrics.client.initialize=true
>
> On Fri, Feb 3, 2017 at 11:40 AM, Marc Savy <marc.savy at redhat.com> wrote:
>
>> Which metrics implementation are you using? The data you're talking about
>> should be in there; if it's not then there's a problem.
>>
>>
>> On 3 February 2017 at 09:50, Balu S <sbalu27 at gmail.com> wrote:
>>
>>> Thanks for your inputs.
>>>
>>> Yes, I mean HTTP error codes (non-200) that are returned to client. For
>>> example, when a request is missing query parameters and the API responds
>>> with a "Bad request" (400) along with error XML. Here the error code is set
>>> at HTTP level only and Apiman metrics should consider it as bad response
>>> (in my opinion). Neither I do not see the Apiman source (HttpApiConnection)
>>> interpreting the HTTP response using the HTTP code. But with the custom
>>> policy, If I check for non-200 code and handle as failure, then metrics
>>> shows them as error.
>>>
>>> To clarify on my implementation with custom policy, I'm not trying to
>>> change the HTTP error code based on the response, rather we are unpacking
>>> the error response body and packing in different XML format as done by
>>> Apiman. Is this not a valid scenario ? I think there could other scenarios
>>> where one want to alter the response body. I agree there will be additional
>>> cost to performance and memory, but can it be not done one demand basis
>>> like how one can implement IDataPolicy to parse the response only if he
>>> needs to.
>>>
>>> Thanks
>>> Balu
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Feb 2, 2017 at 10:31 PM, Eric Wittmann <eric.wittmann at redhat.com
>>> > wrote:
>>>
>>>> The bottom line here is that you cannot return a Policy Failure (or
>>>> customize it) based on information in the response body. The response is
>>>> streamed from the back-end to the client, and at the time streaming begins,
>>>> the response code and HTTP headers have already been sent.
>>>>
>>>> It sounds to me like you're asking for a feature where you can parse
>>>> the response body *before* the policy's "apply" method is invoked. We have
>>>> such a feature for requests, but not for responses. I suspect core changes
>>>> to apiman would be required to enable that. It seems like a reasonable
>>>> request to me, as long as users of the feature understand the performance
>>>> and memory requirements of enabling it.
>>>>
>>>> -Eric
>>>>
>>>>
>>>> On Thu, Feb 2, 2017 at 1:03 PM, Marc Savy <marc.savy at redhat.com> wrote:
>>>>
>>>>> NB: This is distinct from the body you're setting which contains a
>>>>> JSON/XML payload containing the error code. It's in the HTTP protocol
>>>>> itself.
>>>>>
>>>>> On 2 February 2017 at 18:02, Marc Savy <marc.savy at redhat.com> wrote:
>>>>>
>>>>>> That sounds like metrics are going wrong, or perhaps you're
>>>>>> misinterpreting it (adding Eric).
>>>>>>
>>>>>> When you say your API returns an error, does it still return an
>>>>>> appropriate non-200 error code at the HTTP level? For instance, 500 or
>>>>>> similar? That's very important.
>>>>>>
>>>>>> There's a difference between an error and a failure - have you
>>>>>> checked both of those fields to see whether they contain the information
>>>>>> you're expecting to see.
>>>>>>
>>>>>> Certainly in my experience we *do* collect the metrics you're talking
>>>>>> about, unless I'm misunderstanding you.
>>>>>>
>>>>>> On 2 February 2017 at 17:40, Balu S <sbalu27 at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Marc,
>>>>>>>
>>>>>>> I shall explain my use case. From our API call, we return error
>>>>>>> response (in XML or JSON) for 401, 500 and so on. However, Apiman metrics
>>>>>>> seems to just consider them as good response (it is, as Apiman received the
>>>>>>> response back) and show as successful response. So I have made custom
>>>>>>> policy to intercept the response to know if it is failure and trigger the
>>>>>>> Policy Failure. This is fairly simple and straight forward as the response
>>>>>>> code will just do the purpose. But I want also to add some additional
>>>>>>> information about the failure to Policy Failure. This additional
>>>>>>> information is in the original error response which will be lost once
>>>>>>> doFailure() happens. And no we don't want to those additional information
>>>>>>> in some HTTP headers to pass around. Hence I implemented responseHandler()
>>>>>>> to handle the response buffer and like you pointed out, it seems to be too
>>>>>>> late to meddle the response.
>>>>>>>
>>>>>>> So ideally, there are possible 2 solution
>>>>>>>
>>>>>>> - Apiman metrics can interpret the response as unsuccessful for
>>>>>>> such error response from API call.
>>>>>>> - Handle the response buffer data before the write() call to
>>>>>>> response outputstream.
>>>>>>>
>>>>>>> Do you see any alternative solution?
>>>>>>>
>>>>>>> Thanks
>>>>>>> Balu
>>>>>>>
>>>>>>> On Thu, Feb 2, 2017 at 6:15 PM, Marc Savy <marc.savy at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Perhaps URLRewritingPolicy https://github.com/apiman/apim
>>>>>>>> an/blob/master/gateway/engine/policies/src/main/java/io/apim
>>>>>>>> an/gateway/engine/policies/URLRewritingPolicy.java be an
>>>>>>>> informative place to start?
>>>>>>>>
>>>>>>>> - Apiman streams data, so the client may be receiving data already
>>>>>>>> by the time you've determined you want to cancel (the connection is already
>>>>>>>> established; headers have been sent) - it's often too late to gracefully
>>>>>>>> cancel. You could try throwing an exception and seeing what happens (not
>>>>>>>> recommended practice!).
>>>>>>>>
>>>>>>>> If that doesn't work, perhaps you can explain your use-case more
>>>>>>>> clearly and explicitly so we can see what the alternatives are?
>>>>>>>>
>>>>>>>> - Policies are *static* instances, if you are assigning that buffer
>>>>>>>> to the object then it's as if you were writing "static Buffer buffer" and
>>>>>>>> different requests will all share that variable (and thus swap it out
>>>>>>>> repeatedly!).
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Marc
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2 February 2017 at 16:56, Balu S <sbalu27 at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>> I'm trying to parse the response using responseDataHandler() in
>>>>>>>>> the custom policy. In cases, if the response from API is of certain
>>>>>>>>> content, I would like the Apiman to consider as failure. But I don't find a
>>>>>>>>> way to throw policy failure from responseDataHandler(). And I cannot
>>>>>>>>> achieve this in doApply() as the ApiResponse object does not have "content"
>>>>>>>>> to parse.
>>>>>>>>>
>>>>>>>>> Also, what I found is write(chunk) in the AbstractStream is called
>>>>>>>>> after doApply, so I cannot set any attributes in it to fetch it in
>>>>>>>>> doApply() and trigger doFailure().
>>>>>>>>>
>>>>>>>>> For example, in below call, how to throw as policy failure after
>>>>>>>>> parsing the contents ? Or how can I access response content even before
>>>>>>>>> write() method.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *URLRewritingPolicy.java*
>>>>>>>>>
>>>>>>>>> @Override
>>>>>>>>> protected IReadWriteStream<ApiResponse>
>>>>>>>>> responseDataHandler(ApiResponse response,
>>>>>>>>> IPolicyContext context, URLRewritingConfig
>>>>>>>>> policyConfiguration) {
>>>>>>>>> if (policyConfiguration.isProcessResponseBody()) {
>>>>>>>>> return new URLRewritingStream(context.get
>>>>>>>>> Component(IBufferFactoryComponent.class), response,
>>>>>>>>> policyConfiguration.getFromRegex(),
>>>>>>>>> policyConfiguration.getToReplacement());
>>>>>>>>> } else {
>>>>>>>>> return null;
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> *URLRewritingStream.java*
>>>>>>>>>
>>>>>>>>> /**
>>>>>>>>> * @see io.apiman.gateway.engine.io.AbstractStream#write(io.
>>>>>>>>> apiman.gateway.engine.io.IApimanBuffer)
>>>>>>>>> */
>>>>>>>>> @Override
>>>>>>>>> public void write(IApimanBuffer chunk) {
>>>>>>>>> if (buffer == null) {
>>>>>>>>> buffer = bufferFactory.cloneBuffer(chunk);
>>>>>>>>> } else {
>>>>>>>>> buffer.append(chunk);
>>>>>>>>> }
>>>>>>>>> atEnd = false;
>>>>>>>>> processBuffer();
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>> Balu
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Apiman-user mailing list
>>>>>>>>> Apiman-user at lists.jboss.org
>>>>>>>>> https://lists.jboss.org/mailman/listinfo/apiman-user
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20170203/0aad649e/attachment-0001.html
More information about the Apiman-user
mailing list