From mohamed.oukfif at devoteam.com Wed Sep 12 11:29:00 2018 From: mohamed.oukfif at devoteam.com (Mohamed Oukfif) Date: Wed, 12 Sep 2018 17:29:00 +0200 Subject: [Apiman-user] Keycloak OAuth plugin persistence bug Message-ID: Hello, We have the same issue. Do you think this constraint solves the problem? "constraint_name" UNIQUE CONSTRAINT, btree (policydef_id, template) Best regards, M.Oukfif -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20180912/423582e0/attachment.html From marc.savy at redhat.com Wed Sep 12 18:52:59 2018 From: marc.savy at redhat.com (Marc Savy) Date: Wed, 12 Sep 2018 23:52:59 +0100 Subject: [Apiman-user] Keycloak OAuth plugin persistence bug In-Reply-To: References: Message-ID: Hey Mohamed, Appreciate your message about Apiman! Could you elaborate a bit. What problem are you specifically referring to? Could you outline when the constraint is a problem for you? Regards, Marc On 12 September 2018 at 16:29, Mohamed Oukfif wrote: > Hello, > > We have the same issue. Do you think this constraint solves the problem? > > "constraint_name" UNIQUE CONSTRAINT, btree (policydef_id, template) > > Best regards, > M.Oukfif > > _______________________________________________ > Apiman-user mailing list > Apiman-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/apiman-user > From sbalu27 at gmail.com Mon Sep 17 11:03:53 2018 From: sbalu27 at gmail.com (Balu S) Date: Mon, 17 Sep 2018 17:03:53 +0200 Subject: [Apiman-user] ES version supported in Apiman 1.5.1 final. Message-ID: Hello, I am upgrading to latest Apiman 1.5.1 release from 1.3.x version. I noticed the ES is now migrated to 5.x (thanks for this change). But If I look in to ES product, the latest version is 6.x since early 2018. Is there any reason that Apiman can support only 5.x version ? I can think of one reason that embedded Jest client jar is 5.x version which makes it supported by ES 5.x version only ? Thanks Balu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20180917/8a6ba395/attachment.html From sbalu27 at gmail.com Tue Sep 18 06:38:37 2018 From: sbalu27 at gmail.com (Balu S) Date: Tue, 18 Sep 2018 12:38:37 +0200 Subject: [Apiman-user] export/import data config in apiman Message-ID: Hi Eric, Marc, I have used this export/import option in Apiman to migrate data from 1.3 to 1.5.1 version.The tools works fine and I can see the apis, policies and client apps, users are created in migrated environment. However if I notice, there are some config information stored in Elasticsearch under the index apiman_gateway. There are like 21 records for this index in Kibana. These data seems to resemble the client and api information. What are these data ? Is the exported "api manager"data different from "apiman gateway" data ? If so why are these not migrated using export/import utility ? Thanks for reading my post. Best regards Balu. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20180918/2102d246/attachment.html From sbalu27 at gmail.com Tue Sep 18 07:06:27 2018 From: sbalu27 at gmail.com (Balu S) Date: Tue, 18 Sep 2018 13:06:27 +0200 Subject: [Apiman-user] ES version supported in Apiman 1.5.1 final. In-Reply-To: References: Message-ID: update: Apparently ES 6.x does not support the data type "string" so we cannot use ES 6.x. Apiman 1.5.1 must use ES 5.x and lesser. May be we should add this note to release guide for future readers. (ESRegistryCacheInvalidator) Exception in thread "ESRegistryCacheInvalidator" java.lang.RuntimeException: java.lang.Exception: Failed to create index: 'apiman_gateway' Reason: {"root_cause": }],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [client]: No handler for type [string] declared on field [organizationId]","caused_by":{"type":"mapper_parsing_exception","reason":"No handler for type [string] declared on field [organizationId]"}} On Mon, Sep 17, 2018 at 5:03 PM Balu S wrote: > Hello, > > I am upgrading to latest Apiman 1.5.1 release from 1.3.x version. I > noticed the ES is now migrated to 5.x (thanks for this change). But If I > look in to ES product, the latest version is 6.x since early 2018. Is there > any reason that Apiman can support only 5.x version ? I can think of one > reason that embedded Jest client jar is 5.x version which makes it > supported by ES 5.x version only ? > > Thanks > Balu > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20180918/bc0a72e0/attachment.html From mohamed.oukfif at devoteam.com Thu Sep 13 03:13:35 2018 From: mohamed.oukfif at devoteam.com (Mohamed Oukfif) Date: Thu, 13 Sep 2018 09:13:35 +0200 Subject: [Apiman-user] Keycloak OAuth plugin persistence bug In-Reply-To: References: Message-ID: Hey Marc, There is the link : http://lists.jboss.org/pipermail/apiman-user/2017-June/000890.html We have a millions of records in pd_templates table. Regards, Mohamed 2018-09-13 0:52 GMT+02:00 Marc Savy : > Hey Mohamed, > > Appreciate your message about Apiman! > > Could you elaborate a bit. What problem are you specifically referring > to? Could you outline when the constraint is a problem for you? > > Regards, > Marc > > On 12 September 2018 at 16:29, Mohamed Oukfif > wrote: > > Hello, > > > > We have the same issue. Do you think this constraint solves the problem? > > > > "constraint_name" UNIQUE CONSTRAINT, btree (policydef_id, template) > > > > Best regards, > > M.Oukfif > > > > _______________________________________________ > > Apiman-user mailing list > > Apiman-user at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/apiman-user > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20180913/796d6d88/attachment-0001.html From marc.savy at redhat.com Wed Sep 19 05:04:47 2018 From: marc.savy at redhat.com (Marc Savy) Date: Wed, 19 Sep 2018 10:04:47 +0100 Subject: [Apiman-user] ES version supported in Apiman 1.5.1 final. In-Reply-To: References: Message-ID: Hi Balu, I don't think that OpenShift has moved to ES 6 yet. I'll try to see if I can make the change without breaking backwards compatibility, though. Feel free to open a bug on JIRA or GitHub. Regards, Marc On 17 September 2018 at 16:03, Balu S wrote: > Hello, > > I am upgrading to latest Apiman 1.5.1 release from 1.3.x version. I noticed > the ES is now migrated to 5.x (thanks for this change). But If I look in to > ES product, the latest version is 6.x since early 2018. Is there any reason > that Apiman can support only 5.x version ? I can think of one reason that > embedded Jest client jar is 5.x version which makes it supported by ES 5.x > version only ? > > Thanks > Balu > > _______________________________________________ > Apiman-user mailing list > Apiman-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/apiman-user > From marc.savy at redhat.com Wed Sep 19 05:07:05 2018 From: marc.savy at redhat.com (Marc Savy) Date: Wed, 19 Sep 2018 10:07:05 +0100 Subject: [Apiman-user] Keycloak OAuth plugin persistence bug In-Reply-To: References: Message-ID: Hi Mohamed, So, even with the latest version of Apiman you still see this happening? I thought we'd fixed it, but perhaps not. Regards, Marc On 13 September 2018 at 08:13, Mohamed Oukfif wrote: > Hey Marc, > > There is the link : > http://lists.jboss.org/pipermail/apiman-user/2017-June/000890.html > > We have a millions of records in pd_templates table. > > Regards, > Mohamed > > 2018-09-13 0:52 GMT+02:00 Marc Savy : >> >> Hey Mohamed, >> >> Appreciate your message about Apiman! >> >> Could you elaborate a bit. What problem are you specifically referring >> to? Could you outline when the constraint is a problem for you? >> >> Regards, >> Marc >> >> On 12 September 2018 at 16:29, Mohamed Oukfif >> wrote: >> > Hello, >> > >> > We have the same issue. Do you think this constraint solves the problem? >> > >> > "constraint_name" UNIQUE CONSTRAINT, btree (policydef_id, template) >> > >> > Best regards, >> > M.Oukfif >> > >> > _______________________________________________ >> > Apiman-user mailing list >> > Apiman-user at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/apiman-user >> > > > From sbalu27 at gmail.com Fri Sep 21 10:04:56 2018 From: sbalu27 at gmail.com (Balu S) Date: Fri, 21 Sep 2018 16:04:56 +0200 Subject: [Apiman-user] export/import data config in apiman In-Reply-To: References: Message-ID: Further to this email. The export/import tool provided in Apiman manager seems not complete. After I have imported the apimanager configuration to new environment using the tool, accessing the public API URL gives below error. It tries to check the existence of API in the Elasticsearch. But the import/export tool never imports anything to ES index. Does it mean we have to explicitly migrate the existing ES index to new ES server ? io.apiman.gateway.engine.beans.exceptions.ApiNotFoundException: API not found. at io.apiman.gateway.engine.impl.ApiRequestExecutorImpl.lambda$execute$5(ApiRequestExecutorImpl.java:340) at io.apiman.gateway.engine.impl.SecureRegistryWrapper$1.handle(SecureRegistryWrapper.java:123) at io.apiman.gateway.engine.impl.SecureRegistryWrapper$1.handle(SecureRegistryWrapper.java:112) at io.apiman.gateway.engine.es.CachingESRegistry.getApi(CachingESRegistry.java:117) On Tue, Sep 18, 2018 at 12:38 PM Balu S wrote: > Hi Eric, Marc, > > I have used this export/import option in Apiman to migrate data from 1.3 > to 1.5.1 version.The tools works fine and I can see the apis, policies and > client apps, users are created in migrated environment. > > However if I notice, there are some config information stored in > Elasticsearch under the index apiman_gateway. There are like 21 records for > this index in Kibana. These data seems to resemble the client and api > information. > > What are these data ? Is the exported "api manager"data different from > "apiman gateway" data ? If so why are these not migrated using > export/import utility ? > > Thanks for reading my post. > > Best regards > Balu. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20180921/e7e64a77/attachment.html From marc.savy at redhat.com Fri Sep 21 10:38:49 2018 From: marc.savy at redhat.com (Marc Savy) Date: Fri, 21 Sep 2018 15:38:49 +0100 Subject: [Apiman-user] export/import data config in apiman In-Reply-To: References: Message-ID: Hi Balu, The API Manager and API Gateway datastore is essentially separate -- even if it's within the same physical database often, it doesn't have to be (this provides decoupling and the gateway only has a subset of manager data in it). The import-export is only performed on the API Manager, so in some cases if you're also migrating the API Gateway then you may need to: (1) republish APIs from the API Manager (2) migrate the API Gateway datastore separately. Its schema is essentially unchanged, so I don' think it should provide you any great barrier Let me know how you get on Regards, Marc On 18 September 2018 at 11:38, Balu S wrote: > Hi Eric, Marc, > > I have used this export/import option in Apiman to migrate data from 1.3 to > 1.5.1 version.The tools works fine and I can see the apis, policies and > client apps, users are created in migrated environment. > > However if I notice, there are some config information stored in > Elasticsearch under the index apiman_gateway. There are like 21 records for > this index in Kibana. These data seems to resemble the client and api > information. > > What are these data ? Is the exported "api manager"data different from > "apiman gateway" data ? If so why are these not migrated using export/import > utility ? > > Thanks for reading my post. > > Best regards > Balu. From marc.savy at redhat.com Fri Sep 21 10:40:14 2018 From: marc.savy at redhat.com (Marc Savy) Date: Fri, 21 Sep 2018 15:40:14 +0100 Subject: [Apiman-user] export/import data config in apiman In-Reply-To: References: Message-ID: Have you tried republishing those APIs and/or migrating the api_gateway index? On 21 September 2018 at 15:04, Balu S wrote: > Further to this email. > > The export/import tool provided in Apiman manager seems not complete. After > I have imported the apimanager configuration to new environment using the > tool, accessing the public API URL gives below error. It tries to check the > existence of API in the Elasticsearch. But the import/export tool never > imports anything to ES index. > > Does it mean we have to explicitly migrate the existing ES index to new ES > server ? > > io.apiman.gateway.engine.beans.exceptions.ApiNotFoundException: API not > found. > at > io.apiman.gateway.engine.impl.ApiRequestExecutorImpl.lambda$execute$5(ApiRequestExecutorImpl.java:340) > at > io.apiman.gateway.engine.impl.SecureRegistryWrapper$1.handle(SecureRegistryWrapper.java:123) > at > io.apiman.gateway.engine.impl.SecureRegistryWrapper$1.handle(SecureRegistryWrapper.java:112) > at > io.apiman.gateway.engine.es.CachingESRegistry.getApi(CachingESRegistry.java:117) > > > On Tue, Sep 18, 2018 at 12:38 PM Balu S wrote: >> >> Hi Eric, Marc, >> >> I have used this export/import option in Apiman to migrate data from 1.3 >> to 1.5.1 version.The tools works fine and I can see the apis, policies and >> client apps, users are created in migrated environment. >> >> However if I notice, there are some config information stored in >> Elasticsearch under the index apiman_gateway. There are like 21 records for >> this index in Kibana. These data seems to resemble the client and api >> information. >> >> What are these data ? Is the exported "api manager"data different from >> "apiman gateway" data ? If so why are these not migrated using export/import >> utility ? >> >> Thanks for reading my post. >> >> Best regards >> Balu. From sbalu27 at gmail.com Mon Sep 24 11:59:33 2018 From: sbalu27 at gmail.com (Balu S) Date: Mon, 24 Sep 2018 17:59:33 +0200 Subject: [Apiman-user] export/import data config in apiman In-Reply-To: References: Message-ID: Hi Marc, Thanks for your reply. If I republish the API, then it works. But if I have 100s of API, then it seems a quite a work. I wish this can be auto registered when using export/import. FYI, I noticed an old exception (which did not occur in 1.3.1) version when loading the custom plugin. It looks like it is related to snapshot versions. I shall check further. 2018-09-24 16:16:57,376 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 82) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-h ost./apiman-gateway-api: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./apiman-gateway-api: java.lang.RuntimeException: ja va.lang.RuntimeException: java.lang.Exception: Invalid plugin file at: /data/wildfly-11.0.0.Final/standalone/data/apiman/plugins/org.test/apiman-custom-failure/0.0.1-SNAPSHO T/apimancustom-failure.war/plugin.war at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:84) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.Exception: Invalid plugin file at: /data/wildfly-11.0.0.Final/standalone/data/apiman/plugins/org.test/apiman-custom-failure/0.0.1-SNAPSHOT/apiman-custom-failure.war/plugin.war at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:241) at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:99) at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:81) ... 6 more Caused by: java.lang.RuntimeException: java.lang.Exception: Invalid plugin file at: /data/wildfly-11.0.0.Final/standalone/data/apiman/plugins/org.test/apiman-custom-failure/ 0.0.1-SNAPSHOT/apiman-custom-failure.war/plugin.war at io.apiman.gateway.platforms.war.WarEngineConfig.loadConfigClass(WarEngineConfig.java:314) at io.apiman.gateway.platforms.war.WarEngineConfig.getPolicyFailureWriterClass(WarEngineConfig.java:212) at io.apiman.gateway.platforms.war.WarGateway.loadFailureFormatter(WarGateway.java:70) at io.apiman.gateway.platforms.war.WarGateway.init(WarGateway.java:56) at io.apiman.gateway.platforms.war.listeners.WarGatewayBootstrapper.contextInitialized(WarGatewayBootstrapper.java:42) at io.undertow.servlet.core.ApplicationListeners.contextInitialized(ApplicationListeners.java:187) at io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:205) at io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:174) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508) at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:239) ... 8 more Caused by: java.lang.Exception: Invalid plugin file at: /data/wildfly-11.0.0.Final/standalone/data/apiman/plugins/org.test/apiman-custom-failure/0.0.1-SNAPSHOT/apiman-custom -failure.war/plugin.war at io.apiman.gateway.engine.impl.DefaultPluginRegistry.readPluginFile(DefaultPluginRegistry.java:309) at io.apiman.gateway.engine.impl.DefaultPluginRegistry.loadPlugin(DefaultPluginRegistry.java:246) at io.apiman.gateway.platforms.war.WarEngineConfig.loadConfigClass(WarEngineConfig.java:311) ... 23 more On Fri, Sep 21, 2018 at 4:40 PM Marc Savy wrote: > Have you tried republishing those APIs and/or migrating the api_gateway > index? > > On 21 September 2018 at 15:04, Balu S wrote: > > Further to this email. > > > > The export/import tool provided in Apiman manager seems not complete. > After > > I have imported the apimanager configuration to new environment using the > > tool, accessing the public API URL gives below error. It tries to check > the > > existence of API in the Elasticsearch. But the import/export tool never > > imports anything to ES index. > > > > Does it mean we have to explicitly migrate the existing ES index to new > ES > > server ? > > > > io.apiman.gateway.engine.beans.exceptions.ApiNotFoundException: API not > > found. > > at > > > io.apiman.gateway.engine.impl.ApiRequestExecutorImpl.lambda$execute$5(ApiRequestExecutorImpl.java:340) > > at > > > io.apiman.gateway.engine.impl.SecureRegistryWrapper$1.handle(SecureRegistryWrapper.java:123) > > at > > > io.apiman.gateway.engine.impl.SecureRegistryWrapper$1.handle(SecureRegistryWrapper.java:112) > > at > > io.apiman.gateway.engine.es > .CachingESRegistry.getApi(CachingESRegistry.java:117) > > > > > > On Tue, Sep 18, 2018 at 12:38 PM Balu S wrote: > >> > >> Hi Eric, Marc, > >> > >> I have used this export/import option in Apiman to migrate data from 1.3 > >> to 1.5.1 version.The tools works fine and I can see the apis, policies > and > >> client apps, users are created in migrated environment. > >> > >> However if I notice, there are some config information stored in > >> Elasticsearch under the index apiman_gateway. There are like 21 records > for > >> this index in Kibana. These data seems to resemble the client and api > >> information. > >> > >> What are these data ? Is the exported "api manager"data different from > >> "apiman gateway" data ? If so why are these not migrated using > export/import > >> utility ? > >> > >> Thanks for reading my post. > >> > >> Best regards > >> Balu. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20180924/2bc23696/attachment-0001.html