From Ram.Tanna at ril.com Mon Nov 6 03:34:12 2017 From: Ram.Tanna at ril.com (Ram.Tanna at ril.com) Date: Mon, 6 Nov 2017 08:34:12 +0000 Subject: [Apiman-user] Apiman 1.2.9 support for elastic search version 5.*.* Message-ID: Hi Team, Do we have support for elastic search 5+ versions ? I am using apiman 1.2.9. I have upgraded the elastic search version 5.6.3 and started getting following error. ERROR [stderr] (ESRegistryCacheInvalidator) Exception in thread "ESRegistryCacheInvalidator" java.lang.RuntimeException: com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 5 path $ 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at io.apiman.gateway.engine.es.AbstractClientFactory.initializeClient(AbstractClientFactory.java:68) 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at io.apiman.gateway.engine.es.DefaultESClientFactory.createJestClient(DefaultESClientFactory.java:121) 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at io.apiman.gateway.engine.es.DefaultESClientFactory.createClient(DefaultESClientFactory.java:68) 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at io.apiman.gateway.engine.es.AbstractESComponent.createClient(AbstractESComponent.java:61) 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at io.apiman.gateway.engine.es.AbstractESComponent.getClient(AbstractESComponent.java:51) 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at io.apiman.gateway.engine.es.PollCachingESRegistry.checkCacheVersion(PollCachingESRegistry.java:204) 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at io.apiman.gateway.engine.es.PollCachingESRegistry$6.run(PollCachingESRegistry.java:180) 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at java.lang.Thread.run(Thread.java:745) 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) Caused by: com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 5 path $ 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) at com.google.gson.JsonParser.parse(JsonParser.java:65) 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) at com.google.gson.JsonParser.parse(JsonParser.java:45) 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) at io.searchbox.action.AbstractAction.parseResponseBody(AbstractAction.java:96) 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) at io.searchbox.action.AbstractAction.createNewElasticSearchResult(AbstractAction.java:67) 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) at io.searchbox.action.GenericResultAbstractAction.createNewElasticSearchResult(GenericResultAbstractAction.java:20) 17:35:54,007 ERROR [stderr] (ESRegistryCacheInvalidator) at io.searchbox.client.http.JestHttpClient.deserializeResponse(JestHttpClient.java:146) 17:35:54,007 ERROR [stderr] (ESRegistryCacheInvalidator) at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:65) 17:35:54,008 ERROR [stderr] (ESRegistryCacheInvalidator) at io.apiman.gateway.engine.es.AbstractClientFactory.createIndex(AbstractClientFactory.java:80) 17:35:54,008 ERROR [stderr] (ESRegistryCacheInvalidator) at io.apiman.gateway.engine.es.AbstractClientFactory.initializeClient(AbstractClientFactory.java:65) 17:35:54,008 ERROR [stderr] (ESRegistryCacheInvalidator) ... 7 more 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) Caused by: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 5 path $ 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1573) 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) at com.google.gson.stream.JsonReader.checkLenient(JsonReader.java:1423) 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) at com.google.gson.stream.JsonReader.doPeek(JsonReader.java:546) 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) at com.google.gson.stream.JsonReader.peek(JsonReader.java:429) 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) at com.google.gson.JsonParser.parse(JsonParser.java:60) 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) ... 15 more Thanks and Regards, Ram Tanna "Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s). are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email. and delete this message and any attachments from your system. Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment." -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20171106/db974d1b/attachment-0001.html From marc.savy at redhat.com Mon Nov 6 13:44:39 2017 From: marc.savy at redhat.com (Marc Savy) Date: Mon, 6 Nov 2017 18:44:39 +0000 Subject: [Apiman-user] Apiman 1.2.9 support for elastic search version 5.*.* In-Reply-To: References: Message-ID: Hi Ram, My plan is: - The forthcoming release will still have ES 1.x support (final release supporting ES 1.x) - The following release will move to ES 5.x, it *should* also support 2.x with the DeleteByQuery plugin. This will be released a *few days* after to aforementioned. It will be essentially identical but move to ES 5.x support. The work to is mostly done already. Regards, Marc On 6 November 2017 at 08:34, wrote: > Hi Team, > > Do we have support for elastic search 5+ versions ? I am using apiman 1.2.9. > > I have upgraded the elastic search version 5.6.3 and started getting > following error. > > ERROR [stderr] (ESRegistryCacheInvalidator) Exception in thread > ?ESRegistryCacheInvalidator? java.lang.RuntimeException: > com.google.gson.JsonSyntaxException: > com.google.gson.stream.MalformedJsonException: Use > JsonReader.setLenient(true) to accept malformed JSON at line 1 column 5 path > $ > 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.apiman.gateway.engine.es.AbstractClientFactory.initializeClient(AbstractClientFactory.java:68) > 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.apiman.gateway.engine.es.DefaultESClientFactory.createJestClient(DefaultESClientFactory.java:121) > 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.apiman.gateway.engine.es.DefaultESClientFactory.createClient(DefaultESClientFactory.java:68) > 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.apiman.gateway.engine.es.AbstractESComponent.createClient(AbstractESComponent.java:61) > 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.apiman.gateway.engine.es.AbstractESComponent.getClient(AbstractESComponent.java:51) > 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.apiman.gateway.engine.es.PollCachingESRegistry.checkCacheVersion(PollCachingESRegistry.java:204) > 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.apiman.gateway.engine.es.PollCachingESRegistry$6.run(PollCachingESRegistry.java:180) > 17:35:54,005 ERROR [stderr] (ESRegistryCacheInvalidator) at > java.lang.Thread.run(Thread.java:745) > 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) Caused by: > com.google.gson.JsonSyntaxException: > com.google.gson.stream.MalformedJsonException: Use > JsonReader.setLenient(true) to accept malformed JSON at line 1 column 5 path > $ > 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) at > com.google.gson.JsonParser.parse(JsonParser.java:65) > 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) at > com.google.gson.JsonParser.parse(JsonParser.java:45) > 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.searchbox.action.AbstractAction.parseResponseBody(AbstractAction.java:96) > 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.searchbox.action.AbstractAction.createNewElasticSearchResult(AbstractAction.java:67) > 17:35:54,006 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.searchbox.action.GenericResultAbstractAction.createNewElasticSearchResult(GenericResultAbstractAction.java:20) > 17:35:54,007 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.searchbox.client.http.JestHttpClient.deserializeResponse(JestHttpClient.java:146) > 17:35:54,007 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:65) > 17:35:54,008 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.apiman.gateway.engine.es.AbstractClientFactory.createIndex(AbstractClientFactory.java:80) > 17:35:54,008 ERROR [stderr] (ESRegistryCacheInvalidator) at > io.apiman.gateway.engine.es.AbstractClientFactory.initializeClient(AbstractClientFactory.java:65) > 17:35:54,008 ERROR [stderr] (ESRegistryCacheInvalidator) ... 7 more > 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) Caused by: > com.google.gson.stream.MalformedJsonException: Use > JsonReader.setLenient(true) to accept malformed JSON at line 1 column 5 path > $ > 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) at > com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1573) > 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) at > com.google.gson.stream.JsonReader.checkLenient(JsonReader.java:1423) > 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) at > com.google.gson.stream.JsonReader.doPeek(JsonReader.java:546) > 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) at > com.google.gson.stream.JsonReader.peek(JsonReader.java:429) > 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) at > com.google.gson.JsonParser.parse(JsonParser.java:60) > 17:35:54,009 ERROR [stderr] (ESRegistryCacheInvalidator) ... 15 more > > Thanks and Regards, > Ram Tanna > > > "Confidentiality Warning: This message and any attachments are intended only > for the use of the intended recipient(s), are confidential and may be > privileged. If you are not the intended recipient, you are hereby notified > that any review, re-transmission, conversion to hard copy, copying, > circulation or other use of this message and any attachments is strictly > prohibited. If you are not the intended recipient, please notify the sender > immediately by return email and delete this message and any attachments from > your system. > > Virus Warning: Although the company has taken reasonable precautions to > ensure no viruses are present in this email. The company cannot accept > responsibility for any loss or damage arising from the use of this email or > attachment." > > > _______________________________________________ > Apiman-user mailing list > Apiman-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/apiman-user > From marc.savy at redhat.com Tue Nov 7 05:55:56 2017 From: marc.savy at redhat.com (Marc Savy) Date: Tue, 7 Nov 2017 10:55:56 +0000 Subject: [Apiman-user] Enhancing apiman-cli to make headless gateway configs easier to use. In-Reply-To: References: Message-ID: I thought I'd share some of the good progress that has been made in this area: The Apiman CLI tool will now: * Allow Apiman Gateway API to be driven directly. * Expose new Apiman Gateway API "list entity" functions (e.g. list orgs, APIs, Clients, Versions, etc). * List all orgs: apiman-cli gateway organization list * List all APIs in org "test": apiman-cli gateway api list --org test * Directly apply YAML declarations on the Gateway API (with status checks, etc). * Generate Headless Gateway JSON configs from the YAML declaration format. For more detail, see: https://github.com/apiman/apiman-cli/pull/28 On 25 July 2017 at 17:11, Marc Savy wrote: > Here's the PR for those who want to track progress: > > https://github.com/apiman/apiman-cli/pull/27 > > On 18 July 2017 at 23:34, Marc Savy wrote: >> >> Here's a preview of my proposed data format -- it reuses the format >> and model developed by Pete Cornish and his team; big kudos to them! >> [1]. >> >> I'm soliciting suggestions and comments, feel free to reply here (or >> reach out privately if that's not possible). >> >> Example: >> >> apiman-cli gateway apply -f /Users/msavy/tmp/sample.yaml >> >> ``` >> # sample.yaml >> --- >> system: >> gateways: >> - name: "test-gw" >> type: "REST" >> config: >> endpoint: "http://localhost:8080/apiman-gateway-api" >> username: "apimanager" >> password: "apiman123!" >> plugins: >> - name: TestPolicyFriendlyName // (1) >> groupId: "io.apiman.plugins" >> artifactId: "apiman-plugins-test-policy" >> version: "1.3.1.Final" >> org: >> name: "test" >> apis: >> - name: "example" >> version: "1.0" >> config: >> endpoint: "http://localhost:8080/services/echo" >> endpointType: "rest" >> public: true >> gateway: "test-gw" >> policies: >> - name: "CachingPolicy" // (2) >> config: >> ttl: 60 >> - plugin: TestPolicyFriendlyName // (3) >> config: >> foo: 123 >> ``` >> (1) Friendly name for plugin instead of having to refer to it by full GAV >> (2) In-built policy will be looked up to ensure it exists and resolves >> the correct FQCN. >> (3) Add a policy by plugin friendly name (else GAV) >> >> One rare edge case when multiple policies are defined in a single >> plugin. In that situation we allow disambiguation by providing the >> plugin's `id` in the `name`/`id` field: >> >> ``` >> >> policies: >> - name: PolicyOne >> plugin: PluginWithMultiplePolicies >> config: >> foo: 123 >> ``` >> >> Thoughts? Feedback? >> >> Still a bit of work to do before it's PR-ready but making some progress. >> >> Regards, >> Marc >> >> [1] With one tiny addition. >> >> On 7 July 2017 at 14:29, Marc Savy wrote: >> > We've had some people using the Apiman Gateway headless for a while >> > now, either with the new immutable registry that loads from JSON[1], >> > or simply using any existing registry via the gateway API instead of >> > using a manager. >> > >> > The main issue people encounter is that policy configuration contains >> > two fields that are difficult to work out and clumsy to encode >> > properly[1]: >> > >> > - `policyImpl` requires the plugin's URI, including the path to its >> > main class. You can work these out by looking at the plugin's source >> > code, but that's rather circuitous and it would be nicer to just >> > provide the plugin's GAV (like in the manager) and for it to be >> > resolved. >> > >> > - `policyJsonConfig`[3] needs to be escaped properly (and must valid >> > according to its schema). >> > >> > Neither of these aspects are especially user-friendly. My proposal is >> > to extend apiman-cli's functionality to allow the Apiman Gateway to be >> > configured directly via a YAML/JSON file (i.e. declaratively). >> > >> > We can therefore provide a more user-friendly interface that automates >> > the resolution of plugins; validations and escapes the policy config; >> > etc. >> > >> > A final step would be to bundle the apiman-cli tool with our distros >> > to make it easier to access. >> > >> > Any thoughts? >> > >> > Regards, >> > Marc >> > >> > [1] >> > https://apiman.gitbooks.io/apiman-installation-guide/installation-guide/vertx/download.html#_elasticsearch >> > [2] Of course, this interface was never truly designed to be used by >> > humans, so that's understandable >> > [3] Unfortunately named as it can be any arbitrary string, the policy >> > just needs to be able to decode it. For example, it could be XML. > > From marc.savy at redhat.com Tue Nov 7 06:36:50 2017 From: marc.savy at redhat.com (Marc Savy) Date: Tue, 7 Nov 2017 11:36:50 +0000 Subject: [Apiman-user] Enhancing apiman-cli to make headless gateway configs easier to use. In-Reply-To: References: Message-ID: Here are some examples. Make sure you look through all the sections to see the different approaches. Example usage session drilling into API details: $ ./apiman-cli gateway organization list test $ ./apiman-cli gateway api list --org test foo $ ./apiman-cli gateway api list --org test --api foo 1.0 3 4 5 $ ./apiman-cli gateway api list --org test --api foo --version 5 { "publicAPI" : true, "organizationId" : "test", "apiId" : "foo", "version" : "5", "endpoint" : "http://localhost:8080/services/echo", "endpointType" : "rest", "endpointContentType" : "json", "endpointProperties" : { }, "parsePayload" : false, "apiPolicies" : [ { "policyJsonConfig" : "{\"favouriteColour\":\"#ff6c54\",\"surcharge\":10}", "policyImpl" : "plugin:io.apiman.plugins:apiman-plugins-simple-example-2-policy:1.3.2-SNAPSHOT:war/io.apiman.plugins.simpleexample2.SimpleExample2Policy" } ], "maxPayloadBufferSize" : 0 } ----------- There's a tonne of power available in the YAML _declaration_ format contributed by Pete Cornish and Raleigh Pickard. It allows you to define and write your setup in a much more human friendly manner; publish to either the Apiman Manager or (as a new feature) the Apiman Gateway directly! More detail in a forthcoming blog and documentation, but... Example of using the CLI to write a declaration directly to the Gateway API: $ ./apiman-cli gateway apply -f /Users/msavy/tmp/sample.yaml INFO Loaded declaration: /Users/msavy/tmp/sample.yaml INFO Resolving plugin: TestPolicy INFO Automatically selecting policy: Test Policy INFO Publishing API Where sample.yaml is... # Simple apiman example --- system: gateways: - name: "test-gw" type: "REST" config: endpoint: "http://localhost:8080/apiman-gateway-api" username: "apimanager" password: "apiman123!" plugins: - name: TestPolicy groupId: "io.apiman.plugins" artifactId: "apiman-plugins-test-policy" version: "1.3.1.Final" org: name: "test" apis: - name: "example" version: "1.0" config: endpoint: "http://localhost:8080/services/echo" endpointType: "rest" public: true gateway: "test-gw" policies: - name: "CachingPolicy" config: ttl: 60 - plugin: TestPolicy config: foo: 123 ----- And finally, for people who want to generate the Headless JSON configs without doing all of the legwork of looking up plugin references, escaping certain fields, etc: $ ./apiman gateway generate headless -f /Users/msavy/tmp/sample.yaml -o my-headless.json INFO Loaded declaration: /Users/msavy/tmp/sample.yaml INFO Resolving plugin: TestPolicy INFO Automatically selecting policy: Test Policy $ cat my-headless.json { "apis" : [ { "publicAPI" : true, "organizationId" : "test", "apiId" : "example", "version" : "1.0", "endpoint" : "http://localhost:8080/services/echo", "endpointType" : "rest", "endpointContentType" : null, "endpointProperties" : { }, "parsePayload" : false, "apiPolicies" : [ { "policyJsonConfig" : "{\n \"ttl\" : 60\n}", "policyImpl" : "class:io.apiman.gateway.engine.policies.CachingPolicy" }, { "policyJsonConfig" : "{\n \"foo\" : 123\n}", "policyImpl" : "plugin:io.apiman.plugins:apiman-plugins-test-policy:1.3.1.Final:war/io.apiman.plugins.test_policy.TestPolicy" } ], "maxPayloadBufferSize" : 0 } ], "clients" : [ ] } This is just the barest introduction to whet people's appetites. More soon! On 7 November 2017 at 10:55, Marc Savy wrote: > I thought I'd share some of the good progress that has been made in this area: > > The Apiman CLI tool will now: > > * Allow Apiman Gateway API to be driven directly. > > * Expose new Apiman Gateway API "list entity" functions (e.g. list > orgs, APIs, Clients, Versions, etc). > * List all orgs: apiman-cli gateway organization list > * List all APIs in org "test": apiman-cli gateway api list --org test > > * Directly apply YAML declarations on the Gateway API (with status checks, etc). > > * Generate Headless Gateway JSON configs from the YAML declaration format. > > For more detail, see: > https://github.com/apiman/apiman-cli/pull/28 > > On 25 July 2017 at 17:11, Marc Savy wrote: >> Here's the PR for those who want to track progress: >> >> https://github.com/apiman/apiman-cli/pull/27 >> >> On 18 July 2017 at 23:34, Marc Savy wrote: >>> >>> Here's a preview of my proposed data format -- it reuses the format >>> and model developed by Pete Cornish and his team; big kudos to them! >>> [1]. >>> >>> I'm soliciting suggestions and comments, feel free to reply here (or >>> reach out privately if that's not possible). >>> >>> Example: >>> >>> apiman-cli gateway apply -f /Users/msavy/tmp/sample.yaml >>> >>> ``` >>> # sample.yaml >>> --- >>> system: >>> gateways: >>> - name: "test-gw" >>> type: "REST" >>> config: >>> endpoint: "http://localhost:8080/apiman-gateway-api" >>> username: "apimanager" >>> password: "apiman123!" >>> plugins: >>> - name: TestPolicyFriendlyName // (1) >>> groupId: "io.apiman.plugins" >>> artifactId: "apiman-plugins-test-policy" >>> version: "1.3.1.Final" >>> org: >>> name: "test" >>> apis: >>> - name: "example" >>> version: "1.0" >>> config: >>> endpoint: "http://localhost:8080/services/echo" >>> endpointType: "rest" >>> public: true >>> gateway: "test-gw" >>> policies: >>> - name: "CachingPolicy" // (2) >>> config: >>> ttl: 60 >>> - plugin: TestPolicyFriendlyName // (3) >>> config: >>> foo: 123 >>> ``` >>> (1) Friendly name for plugin instead of having to refer to it by full GAV >>> (2) In-built policy will be looked up to ensure it exists and resolves >>> the correct FQCN. >>> (3) Add a policy by plugin friendly name (else GAV) >>> >>> One rare edge case when multiple policies are defined in a single >>> plugin. In that situation we allow disambiguation by providing the >>> plugin's `id` in the `name`/`id` field: >>> >>> ``` >>> >>> policies: >>> - name: PolicyOne >>> plugin: PluginWithMultiplePolicies >>> config: >>> foo: 123 >>> ``` >>> >>> Thoughts? Feedback? >>> >>> Still a bit of work to do before it's PR-ready but making some progress. >>> >>> Regards, >>> Marc >>> >>> [1] With one tiny addition. >>> >>> On 7 July 2017 at 14:29, Marc Savy wrote: >>> > We've had some people using the Apiman Gateway headless for a while >>> > now, either with the new immutable registry that loads from JSON[1], >>> > or simply using any existing registry via the gateway API instead of >>> > using a manager. >>> > >>> > The main issue people encounter is that policy configuration contains >>> > two fields that are difficult to work out and clumsy to encode >>> > properly[1]: >>> > >>> > - `policyImpl` requires the plugin's URI, including the path to its >>> > main class. You can work these out by looking at the plugin's source >>> > code, but that's rather circuitous and it would be nicer to just >>> > provide the plugin's GAV (like in the manager) and for it to be >>> > resolved. >>> > >>> > - `policyJsonConfig`[3] needs to be escaped properly (and must valid >>> > according to its schema). >>> > >>> > Neither of these aspects are especially user-friendly. My proposal is >>> > to extend apiman-cli's functionality to allow the Apiman Gateway to be >>> > configured directly via a YAML/JSON file (i.e. declaratively). >>> > >>> > We can therefore provide a more user-friendly interface that automates >>> > the resolution of plugins; validations and escapes the policy config; >>> > etc. >>> > >>> > A final step would be to bundle the apiman-cli tool with our distros >>> > to make it easier to access. >>> > >>> > Any thoughts? >>> > >>> > Regards, >>> > Marc >>> > >>> > [1] >>> > https://apiman.gitbooks.io/apiman-installation-guide/installation-guide/vertx/download.html#_elasticsearch >>> > [2] Of course, this interface was never truly designed to be used by >>> > humans, so that's understandable >>> > [3] Unfortunately named as it can be any arbitrary string, the policy >>> > just needs to be able to decode it. For example, it could be XML. >> >> From marc.savy at redhat.com Mon Nov 13 14:37:55 2017 From: marc.savy at redhat.com (Marc Savy) Date: Mon, 13 Nov 2017 19:37:55 +0000 Subject: [Apiman-user] Authorization question In-Reply-To: References: Message-ID: Hey Stephen, Any progress on this? Did you manage to solve your issue? Regards, Marc On 10 August 2017 at 10:33, Marc Savy wrote: > Is your setup roughly: > > ClientApp -> Plan [Keycloak] -> API [Authz] > > Which version of Keycloak are you using? > What type of roles are you using? For example, realm. > > The way Keycloak roles are modelled has changed fairly significantly > over the last few versions of KC. We might not be handling that > correctly anymore. > > On 8 August 2017 at 11:08, Marc Savy wrote: >> Hi, >> >> If I understand your description correctly, this should work. And in >> my quick tests, it seems to work. I might not be replicating >> your setup perfectly though. >> >> For example let's imagine we have a setup such that: >> >> Client Policies [] // None >> Plan Policies [Foo, Bar] >> API Policies [Baz] >> >> This ultimately flattens to a policy chain of: >> >> Caller <-> [ Foo <-> Bar <-> Baz ] <-> API >> >> So if your setup is (N of): >> >> Plan [ Keycloak Auth ] >> API [ Authz ] >> >> This should always result in: Keycloak *then* Authz, passing roles as >> defined in config. >> >> If that isn't happening then there's a bug. I may may need to collect >> some more information from you to see whether I can replicate the >> issue. >> >> Regards, >> Marc >> >> On 5 August 2017 at 01:21, Stephen Henrie wrote: >>> >>> My goal is minimize the amount of Apiman configuration that I need to do by >>> sharing a single, common authentication Plan using the Keycloak plugin >>> across all APIs while using an API specific authorization policy for each >>> individual API. >>> >>> As such, I am trying to configure a single, global plan within Apiman that >>> can be used for ensuring authentication policy using the Keycloak plugin >>> which forwards all of my realm roles. This single plan would be assigned to >>> all of my APIs in the Org, which would allow me to only have to configure >>> the Keycloak realm information in one place. Then for each individual API, I >>> was hoping to add a single Authorization policy plugin configured with >>> endpoints and paths specific for each API. >>> >>> Something like >>> >>> Api1 ---> Keycloak Plan Abc >>> +---->Authorization Policy (123) >>> >>> Api2 ---> Keycloak Plan Abc >>> +---->Authorization Policy (456) >>> >>> >>> When I do this and call one of the API endpoints, I am getting the following >>> error: >>> >>> curl -k -H "Authorization: Bearer $T" >>> https://localhost:9443/apiman-gateway/chassi/chassi-tenant-bff/1.0/mytenants >>> >>> {"type":"Other","failureCode":10010,"responseCode":0,"message":"No roles >>> have been extracted during authentication. Make sure the authorization >>> policy comes *after* a compatible authentication policy in your >>> configuration.","headers":[]} >>> >>> It would seem that the Keycloak plugin that is configured in the Plan >>> assigned to the API is not forwarding the realm roles to the Authentication >>> policy which is also assigned to the same API. >>> >>> Is this by design? Do the authentication and authorization policies have to >>> be within the same entity (ie. Plan, Api, etc) and not passed out of a plan >>> to be used by downstream policies? If so, is there another way to configure >>> plans and policies that will allow me to accomplish my goal? >>> >>> Thanks in advance! >>> Stephen >>> >>> _______________________________________________ >>> Apiman-user mailing list >>> Apiman-user at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/apiman-user >>> From marc.savy at redhat.com Mon Nov 13 15:00:22 2017 From: marc.savy at redhat.com (Marc Savy) Date: Mon, 13 Nov 2017 20:00:22 +0000 Subject: [Apiman-user] Bind Vert.x Gateway to IP In-Reply-To: References: Message-ID: Thanks Jorge, your changes have been merged! On 5 October 2017 at 15:13, Eric Wittmann wrote: > Awesome! We'll get it merged in asap. > > On Thu, Oct 5, 2017 at 9:32 AM, Jorge Riquelme wrote: > >> Hi Eric, >> >> Thank you for the clarification. In fact, I made a change there yesterday >> to get the Gateway running, but I wasn't sure if some alternative was >> available. >> >> I created a Jira issue (https://issues.jboss.org/browse/APIMAN-1292) and >> a pull request (https://github.com/apiman/apiman/pull/627). >> >> Best regards, >> Jorge Riquelme >> >> *Jorge Riquelme Santana* >> *Co-founder & CTO*, Larix >> mobile: (+56 9) 66594454 <%28%2B56%209%29%2062481315> >> web: www.larix.cl >> >> On Thu, Oct 5, 2017 at 8:22 AM, Eric Wittmann >> wrote: >> >>> Currently I believe the vert.x gateway listens on all interfaces >>> (0.0.0.0). For example: >>> >>> https://github.com/apiman/apiman/blob/master/gateway/platfor >>> ms/vertx3/vertx3/src/main/java/io/apiman/gateway/platforms/ >>> vertx3/verticles/HttpGatewayVerticle.java#L49-L51 >>> >>> If you need a way to ONLY listen on a particular host, then I think that >>> needs to be a code change. If that's the case, submit a ticket: >>> >>> https://issues.jboss.org/projects/APIMAN/issues >>> >>> Or submit a pull request! It's a pretty simple change. :) >>> >>> -Eric >>> >>> >>> On Wed, Oct 4, 2017 at 10:24 PM, Jorge Riquelme wrote: >>> >>>> Hi, >>>> >>>> I couldn't find how to bind Vert.x Gateway to a specific IP (using the >>>> conf-es.json file, or some parameter like -b=x.x.x.x). >>>> >>>> Any help would be appreciated :) >>>> >>>> Best regards, >>>> Jorge Riquelme >>>> >>>> _______________________________________________ >>>> Apiman-user mailing list >>>> Apiman-user at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/apiman-user >>>> >>>> >>> >> > > _______________________________________________ > Apiman-user mailing list > Apiman-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/apiman-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20171113/705a15ab/attachment.html From stephen at saasindustries.com Mon Nov 13 20:36:12 2017 From: stephen at saasindustries.com (Stephen Henrie) Date: Mon, 13 Nov 2017 18:36:12 -0700 Subject: [Apiman-user] Authorization question In-Reply-To: References: Message-ID: Hi Marc, Yes and no. We decided to go a different route and not use the contracts and plans, since we don't want to have to dole out apikeys. We are just using the "public" API functionality at this point. However, in order to minimize the configuration needed for each new API entry, I went ahead and created a custom policy plugin which combines some parts of the keycloak oauth plugin and the authorization plugin. Instead of requiring all of that preconfiguration data for each API to point the plugin to a specific keycloak realm, it dynamically makes a call to Keycloak in real-time to get and cache the token's realm's public key so that the oauth token can be validated. Thanks for checking in though. Stephen On Mon, Nov 13, 2017 at 12:37 PM, Marc Savy wrote: > Hey Stephen, > > Any progress on this? Did you manage to solve your issue? > > Regards, > Marc > > On 10 August 2017 at 10:33, Marc Savy wrote: > > Is your setup roughly: > > > > ClientApp -> Plan [Keycloak] -> API [Authz] > > > > Which version of Keycloak are you using? > > What type of roles are you using? For example, realm. > > > > The way Keycloak roles are modelled has changed fairly significantly > > over the last few versions of KC. We might not be handling that > > correctly anymore. > > > > On 8 August 2017 at 11:08, Marc Savy wrote: > >> Hi, > >> > >> If I understand your description correctly, this should work. And in > >> my quick tests, it seems to work. I might not be replicating > >> your setup perfectly though. > >> > >> For example let's imagine we have a setup such that: > >> > >> Client Policies [] // None > >> Plan Policies [Foo, Bar] > >> API Policies [Baz] > >> > >> This ultimately flattens to a policy chain of: > >> > >> Caller <-> [ Foo <-> Bar <-> Baz ] <-> API > >> > >> So if your setup is (N of): > >> > >> Plan [ Keycloak Auth ] > >> API [ Authz ] > >> > >> This should always result in: Keycloak *then* Authz, passing roles as > >> defined in config. > >> > >> If that isn't happening then there's a bug. I may may need to collect > >> some more information from you to see whether I can replicate the > >> issue. > >> > >> Regards, > >> Marc > >> > >> On 5 August 2017 at 01:21, Stephen Henrie > wrote: > >>> > >>> My goal is minimize the amount of Apiman configuration that I need to > do by > >>> sharing a single, common authentication Plan using the Keycloak plugin > >>> across all APIs while using an API specific authorization policy for > each > >>> individual API. > >>> > >>> As such, I am trying to configure a single, global plan within Apiman > that > >>> can be used for ensuring authentication policy using the Keycloak > plugin > >>> which forwards all of my realm roles. This single plan would be > assigned to > >>> all of my APIs in the Org, which would allow me to only have to > configure > >>> the Keycloak realm information in one place. Then for each individual > API, I > >>> was hoping to add a single Authorization policy plugin configured with > >>> endpoints and paths specific for each API. > >>> > >>> Something like > >>> > >>> Api1 ---> Keycloak Plan Abc > >>> +---->Authorization Policy (123) > >>> > >>> Api2 ---> Keycloak Plan Abc > >>> +---->Authorization Policy (456) > >>> > >>> > >>> When I do this and call one of the API endpoints, I am getting the > following > >>> error: > >>> > >>> curl -k -H "Authorization: Bearer $T" > >>> https://localhost:9443/apiman-gateway/chassi/chassi-tenant- > bff/1.0/mytenants > >>> > >>> {"type":"Other","failureCode":10010,"responseCode":0,"message":"No > roles > >>> have been extracted during authentication. Make sure the authorization > >>> policy comes *after* a compatible authentication policy in your > >>> configuration.","headers":[]} > >>> > >>> It would seem that the Keycloak plugin that is configured in the Plan > >>> assigned to the API is not forwarding the realm roles to the > Authentication > >>> policy which is also assigned to the same API. > >>> > >>> Is this by design? Do the authentication and authorization policies > have to > >>> be within the same entity (ie. Plan, Api, etc) and not passed out of a > plan > >>> to be used by downstream policies? If so, is there another way to > configure > >>> plans and policies that will allow me to accomplish my goal? > >>> > >>> Thanks in advance! > >>> Stephen > >>> > >>> _______________________________________________ > >>> Apiman-user mailing list > >>> Apiman-user at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/apiman-user > >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20171113/6c9f444b/attachment.html From marc.savy at redhat.com Tue Nov 14 05:03:48 2017 From: marc.savy at redhat.com (Marc Savy) Date: Tue, 14 Nov 2017 10:03:48 +0000 Subject: [Apiman-user] Authorization question In-Reply-To: References: Message-ID: Hi Stephen, Instead of requiring all of that preconfiguration data for each API to > point the plugin to a specific keycloak realm, it dynamically makes a call > to Keycloak in real-time to get and cache the token's realm's public key so > that the oauth token can be validated. > Thanks for describing this. When we created the KC policy this functionality wasn't yet available on their side. I think it would be a fantastic addition to the community plugin, so I've pencilled it in. Some policies could definitely use some feature expansion and general (re)polishing; this will be a focus in the near future. I suspect the CLI tooling and/or generated headless functionality might interest you, also? I'll be sure to solicit your feedback once that goes live. Regards, Marc On Tuesday, November 14, 2017, Stephen Henrie wrote: > Hi Marc, > > Yes and no. We decided to go a different route and not use the contracts > and plans, since we don't want to have to dole out apikeys. We are just > using the "public" API functionality at this point. > > However, in order to minimize the configuration needed for each new API > entry, I went ahead and created a custom policy plugin which combines some > parts of the keycloak oauth plugin and the authorization plugin. Instead of > requiring all of that preconfiguration data for each API to point the > plugin to a specific keycloak realm, it dynamically makes a call to > Keycloak in real-time to get and cache the token's realm's public key so > that the oauth token can be validated. > > Thanks for checking in though. > > Stephen > > On Mon, Nov 13, 2017 at 12:37 PM, Marc Savy wrote: > >> Hey Stephen, >> >> Any progress on this? Did you manage to solve your issue? >> >> Regards, >> Marc >> >> On 10 August 2017 at 10:33, Marc Savy wrote: >> > Is your setup roughly: >> > >> > ClientApp -> Plan [Keycloak] -> API [Authz] >> > >> > Which version of Keycloak are you using? >> > What type of roles are you using? For example, realm. >> > >> > The way Keycloak roles are modelled has changed fairly significantly >> > over the last few versions of KC. We might not be handling that >> > correctly anymore. >> > >> > On 8 August 2017 at 11:08, Marc Savy wrote: >> >> Hi, >> >> >> >> If I understand your description correctly, this should work. And in >> >> my quick tests, it seems to work. I might not be replicating >> >> your setup perfectly though. >> >> >> >> For example let's imagine we have a setup such that: >> >> >> >> Client Policies [] // None >> >> Plan Policies [Foo, Bar] >> >> API Policies [Baz] >> >> >> >> This ultimately flattens to a policy chain of: >> >> >> >> Caller <-> [ Foo <-> Bar <-> Baz ] <-> API >> >> >> >> So if your setup is (N of): >> >> >> >> Plan [ Keycloak Auth ] >> >> API [ Authz ] >> >> >> >> This should always result in: Keycloak *then* Authz, passing roles as >> >> defined in config. >> >> >> >> If that isn't happening then there's a bug. I may may need to collect >> >> some more information from you to see whether I can replicate the >> >> issue. >> >> >> >> Regards, >> >> Marc >> >> >> >> On 5 August 2017 at 01:21, Stephen Henrie >> wrote: >> >>> >> >>> My goal is minimize the amount of Apiman configuration that I need to >> do by >> >>> sharing a single, common authentication Plan using the Keycloak plugin >> >>> across all APIs while using an API specific authorization policy for >> each >> >>> individual API. >> >>> >> >>> As such, I am trying to configure a single, global plan within >> Apiman that >> >>> can be used for ensuring authentication policy using the Keycloak >> plugin >> >>> which forwards all of my realm roles. This single plan would be >> assigned to >> >>> all of my APIs in the Org, which would allow me to only have to >> configure >> >>> the Keycloak realm information in one place. Then for each individual >> API, I >> >>> was hoping to add a single Authorization policy plugin configured with >> >>> endpoints and paths specific for each API. >> >>> >> >>> Something like >> >>> >> >>> Api1 ---> Keycloak Plan Abc >> >>> +---->Authorization Policy (123) >> >>> >> >>> Api2 ---> Keycloak Plan Abc >> >>> +---->Authorization Policy (456) >> >>> >> >>> >> >>> When I do this and call one of the API endpoints, I am getting the >> following >> >>> error: >> >>> >> >>> curl -k -H "Authorization: Bearer $T" >> >>> https://localhost:9443/apiman-gateway/chassi/chassi-tenant-b >> ff/1.0/mytenants >> >>> >> >>> {"type":"Other","failureCode":10010,"responseCode":0,"message":"No >> roles >> >>> have been extracted during authentication. Make sure the >> authorization >> >>> policy comes *after* a compatible authentication policy in your >> >>> configuration.","headers":[]} >> >>> >> >>> It would seem that the Keycloak plugin that is configured in the Plan >> >>> assigned to the API is not forwarding the realm roles to the >> Authentication >> >>> policy which is also assigned to the same API. >> >>> >> >>> Is this by design? Do the authentication and authorization policies >> have to >> >>> be within the same entity (ie. Plan, Api, etc) and not passed out of >> a plan >> >>> to be used by downstream policies? If so, is there another way to >> configure >> >>> plans and policies that will allow me to accomplish my goal? >> >>> >> >>> Thanks in advance! >> >>> Stephen >> >>> >> >>> _______________________________________________ >> >>> Apiman-user mailing list >> >>> Apiman-user at lists.jboss.org >> >>> https://lists.jboss.org/mailman/listinfo/apiman-user >> >>> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20171114/f9605adf/attachment-0001.html From marc.savy at redhat.com Tue Nov 14 11:56:46 2017 From: marc.savy at redhat.com (Marc Savy) Date: Tue, 14 Nov 2017 16:56:46 +0000 Subject: [Apiman-user] DockerHub build(s) of Apiman master commit now available Message-ID: Hi, We will now be deploying successful master builds of our all-in-one Wildfly10 image to DockerHub. To get it: docker pull apiman/on-wildfly10:master As this is based on the latest commit it should not be considered stable and is for testing, developers, advanced users, etc. At some point in the near future we hope to have high-quality Vert.x gateway container builds (via internal and external contributors). More on that soon! We've also had a lot of demand for updated OpenShift images, so it's something we're also working on. Regards, Marc