Re: [Apiman-user] API policy response data handler

Thanks for your inputs. Yes, I mean HTTP error codes (non-200) that are returned to client. For example, when a request is missing query parameters and the API responds with a "Bad request" (400) along with error XML. Here the error code is set at HTTP level only and Apiman metrics should consider it as bad response (in my opinion). Neither I do not see the Apiman source (HttpApiConnection) interpreting the HTTP response using the HTTP code. But with the custom policy, If I check for non-200 code and handle as failure, then metrics shows them as error. To clarify on my implementation with custom policy, I'm not trying to change the HTTP error code based on the response, rather we are unpacking the error response body and packing in different XML format as done by Apiman. Is this not a valid scenario ? I think there could other scenarios where one want to alter the response body. I agree there will be additional cost to performance and memory, but can it be not done one demand basis like how one can implement IDataPolicy to parse the response only if he needs to. Thanks Balu On Thu, Feb 2, 2017 at 10:31 PM, Eric Wittmann <eric.wittmann(a)redhat.com&gt; wrote: > The bottom line here is that you cannot return a Policy Failure (or > customize it) based on information in the response body. The response is > streamed from the back-end to the client, and at the time streaming begins, > the response code and HTTP headers have already been sent. > > It sounds to me like you're asking for a feature where you can parse the > response body *before* the policy's "apply" method is invoked. We have > such a feature for requests, but not for responses. I suspect core changes > to apiman would be required to enable that. It seems like a reasonable > request to me, as long as users of the feature understand the performance > and memory requirements of enabling it. > > -Eric > > > On Thu, Feb 2, 2017 at 1:03 PM, Marc Savy <marc.savy(a)redhat.com&gt; wrote: > >> NB: This is distinct from the body you're setting which contains a >> JSON/XML payload containing the error code. It's in the HTTP protocol >> itself. >> >> On 2 February 2017 at 18:02, Marc Savy <marc.savy(a)redhat.com&gt; wrote: >> >>> That sounds like metrics are going wrong, or perhaps you're >>> misinterpreting it (adding Eric). >>> >>> When you say your API returns an error, does it still return an >>> appropriate non-200 error code at the HTTP level? For instance, 500 or >>> similar? That's very important. >>> >>> There's a difference between an error and a failure - have you checked >>> both of those fields to see whether they contain the information you're >>> expecting to see. >>> >>> Certainly in my experience we *do* collect the metrics you're talking >>> about, unless I'm misunderstanding you. >>> >>> On 2 February 2017 at 17:40, Balu S <sbalu27(a)gmail.com&gt; wrote: >>> >>>> Hi Marc, >>>> >>>> I shall explain my use case. From our API call, we return error >>>> response (in XML or JSON) for 401, 500 and so on. However, Apiman metrics >>>> seems to just consider them as good response (it is, as Apiman received the >>>> response back) and show as successful response. So I have made custom >>>> policy to intercept the response to know if it is failure and trigger the >>>> Policy Failure. This is fairly simple and straight forward as the response >>>> code will just do the purpose. But I want also to add some additional >>>> information about the failure to Policy Failure. This additional >>>> information is in the original error response which will be lost once >>>> doFailure() happens. And no we don't want to those additional information >>>> in some HTTP headers to pass around. Hence I implemented responseHandler() >>>> to handle the response buffer and like you pointed out, it seems to be too >>>> late to meddle the response. >>>> >>>> So ideally, there are possible 2 solution >>>> >>>> - Apiman metrics can interpret the response as unsuccessful for >>>> such error response from API call. >>>> - Handle the response buffer data before the write() call to >>>> response outputstream. >>>> >>>> Do you see any alternative solution? >>>> >>>> Thanks >>>> Balu >>>> >>>> On Thu, Feb 2, 2017 at 6:15 PM, Marc Savy <marc.savy(a)redhat.com&gt; >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Perhaps URLRewritingPolicy https://github.com/apiman/apim >>>>> an/blob/master/gateway/engine/policies/src/main/java/io/apim >>>>> an/gateway/engine/policies/URLRewritingPolicy.java be an informative >>>>> place to start? >>>>> >>>>> - Apiman streams data, so the client may be receiving data already by >>>>> the time you've determined you want to cancel (the connection is already >>>>> established; headers have been sent) - it's often too late to gracefully >>>>> cancel. You could try throwing an exception and seeing what happens (not >>>>> recommended practice!). >>>>> >>>>> If that doesn't work, perhaps you can explain your use-case more >>>>> clearly and explicitly so we can see what the alternatives are? >>>>> >>>>> - Policies are *static* instances, if you are assigning that buffer >>>>> to the object then it's as if you were writing "static Buffer buffer" and >>>>> different requests will all share that variable (and thus swap it out >>>>> repeatedly!). >>>>> >>>>> Regards, >>>>> Marc >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 2 February 2017 at 16:56, Balu S <sbalu27(a)gmail.com&gt; wrote: >>>>> >>>>>> Hello, >>>>>> I'm trying to parse the response using responseDataHandler() in the >>>>>> custom policy. In cases, if the response from API is of certain content, I >>>>>> would like the Apiman to consider as failure. But I don't find a way to >>>>>> throw policy failure from responseDataHandler(). And I cannot achieve this >>>>>> in doApply() as the ApiResponse object does not have "content" to parse. >>>>>> >>>>>> Also, what I found is write(chunk) in the AbstractStream is called >>>>>> after doApply, so I cannot set any attributes in it to fetch it in >>>>>> doApply() and trigger doFailure(). >>>>>> >>>>>> For example, in below call, how to throw as policy failure after >>>>>> parsing the contents ? Or how can I access response content even before >>>>>> write() method. >>>>>> >>>>>> >>>>>> *URLRewritingPolicy.java* >>>>>> >>>>>> @Override >>>>>> protected IReadWriteStream<ApiResponse> >>>>>> responseDataHandler(ApiResponse response, >>>>>> IPolicyContext context, URLRewritingConfig >>>>>> policyConfiguration) { >>>>>> if (policyConfiguration.isProcessResponseBody()) { >>>>>> return new URLRewritingStream(context.get >>>>>> Component(IBufferFactoryComponent.class), response, >>>>>> policyConfiguration.getFromRegex(), >>>>>> policyConfiguration.getToReplacement()); >>>>>> } else { >>>>>> return null; >>>>>> } >>>>>> } >>>>>> >>>>>> *URLRewritingStream.java* >>>>>> >>>>>> /** >>>>>> * @see io.apiman.gateway.engine.io.AbstractStream#write(io. >>>>>> apiman.gateway.engine.io.IApimanBuffer) >>>>>> */ >>>>>> @Override >>>>>> public void write(IApimanBuffer chunk) { >>>>>> if (buffer == null) { >>>>>> buffer = bufferFactory.cloneBuffer(chunk); >>>>>> } else { >>>>>> buffer.append(chunk); >>>>>> } >>>>>> atEnd = false; >>>>>> processBuffer(); >>>>>> } >>>>>> >>>>>> >>>>>> Best regards >>>>>> Balu >>>>>> >>>>>> _______________________________________________ >>>>>> Apiman-user mailing list >>>>>> Apiman-user(a)lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/apiman-user >>>>>> >>>>>> >>>>> >>>> >>> >> >