From sbalu27 at gmail.com Thu Mar 16 05:06:13 2017 From: sbalu27 at gmail.com (Balu S) Date: Thu, 16 Mar 2017 10:06:13 +0100 Subject: [Apiman-user] Elastic search upgrade Message-ID: Hello, I'm trying to gather metrics (using Kibana tool) from the ES instance that is being used by apiman. But what I found is the ES instance that is bundled inside the apiman is 1.7.x version which seems to be too old for current trending tools to index upon. Of course we can use old versions of those tools, for ex. Kibana 4.1.x version to index and gather metrics. But the older version are limited in functionality and I would like to use the latest if possible. Hence I have following options. 1) Use separate ElasticSearch instance with latest versions and update apiman.properties to point to this instance. And with this new version instance, I can use the latest tools. Question : Do you think the ES API 1.7.x is compatible and write indices to ES database running in latest version ? 2) Migrate the ES indices to new versions. But the below article, clearly says that the ES version must be at least 2.4.x for smooth migration using their migration tool. https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade.html 3) Upgrade bundled ES API 1.7.x to 5.2.x (latest version) or at least 2.4.x version I think it is practical to upgrade bundled ES version to 2.4.x at least (if not for latest 5.2.x) just to make it compatible for migrating indices easily. What are your thoughts? I'm happy to see any alternative options other than mentioned above. Thanks. regards Balu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20170316/f777f731/attachment.html From sbalu27 at gmail.com Fri Mar 17 09:51:11 2017 From: sbalu27 at gmail.com (Balu S) Date: Fri, 17 Mar 2017 14:51:11 +0100 Subject: [Apiman-user] Elastic search upgrade In-Reply-To: References: Message-ID: Ok, So option 3) is not straightforward. The build fails as there is bunch of API incompatibility between 1.7 and 2.4 versions. Unless one wiling to refactor the apiman-es source to use 2.4 API or may be latest 5.2. I have gone ahead with option 1 and used a separate ES instance running with 2.4.4 (cannot go higher than this version due to incompatibility with bundled ES API in apiman) version and it seems fine. Since I'm not worried about the old indices, this looks like straightforward option. But if someone want to use old indices, then you may need to dig deeper to re-index may be (?). regards Balu On Thu, Mar 16, 2017 at 10:06 AM, Balu S wrote: > Hello, > I'm trying to gather metrics (using Kibana tool) from the ES instance that > is being used by apiman. But what I found is the ES instance that is > bundled inside the apiman is 1.7.x version which seems to be too old for > current trending tools to index upon. Of course we can use old versions of > those tools, for ex. Kibana 4.1.x version to index and gather metrics. But > the older version are limited in functionality and I would like to use the > latest if possible. > > Hence I have following options. > > 1) Use separate ElasticSearch instance with latest versions and update > apiman.properties to point to this instance. And with this new version > instance, I can use the latest tools. > > Question : Do you think the ES API 1.7.x is compatible and write > indices to ES database running in latest version ? > > 2) Migrate the ES indices to new versions. But the below article, clearly > says that the ES version must be at least 2.4.x for smooth migration using > their migration tool. > > https://www.elastic.co/guide/en/elasticsearch/reference/ > current/reindex-upgrade.html > > 3) Upgrade bundled ES API 1.7.x to 5.2.x (latest version) or at least > 2.4.x version > > > I think it is practical to upgrade bundled ES version to 2.4.x at least > (if not for latest 5.2.x) just to make it compatible for migrating indices > easily. > > What are your thoughts? I'm happy to see any alternative options other > than mentioned above. Thanks. > > regards > Balu > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20170317/c4c8bd02/attachment.html From eric.wittmann at redhat.com Fri Mar 17 12:33:11 2017 From: eric.wittmann at redhat.com (Eric Wittmann) Date: Fri, 17 Mar 2017 12:33:11 -0400 Subject: [Apiman-user] Elastic search upgrade In-Reply-To: References: Message-ID: Glad that you found an acceptable workaround. If anyone on this list is interested in identifying the list of incompatibilities between ES versions (so that we can upgrade support in apiman to the latest version) then we'd be happy to make those changes. As I'm sure everyone is already aware, the full apiman solution is now in maintenance-only mode. This means that major functionality will not be added (by us). However, with some community support for something like this, I think we would be able to make time for it. Please note that Marc (msavy) continues to actively work on the Gateway, with a focus on releasing (soon) a vert.x based version. So the project isn't entirely silent! :) -Eric On Fri, Mar 17, 2017 at 9:51 AM, Balu S wrote: > Ok, So option 3) is not straightforward. The build fails as there is bunch > of API incompatibility between 1.7 and 2.4 versions. Unless one wiling to > refactor the apiman-es source to use 2.4 API or may be latest 5.2. > > I have gone ahead with option 1 and used a separate ES instance running > with 2.4.4 (cannot go higher than this version due to incompatibility with > bundled ES API in apiman) version and it seems fine. Since I'm not worried > about the old indices, this looks like straightforward option. > > But if someone want to use old indices, then you may need to dig deeper to > re-index may be (?). > > regards > Balu > > On Thu, Mar 16, 2017 at 10:06 AM, Balu S wrote: > >> Hello, >> I'm trying to gather metrics (using Kibana tool) from the ES instance >> that is being used by apiman. But what I found is the ES instance that is >> bundled inside the apiman is 1.7.x version which seems to be too old for >> current trending tools to index upon. Of course we can use old versions of >> those tools, for ex. Kibana 4.1.x version to index and gather metrics. But >> the older version are limited in functionality and I would like to use the >> latest if possible. >> >> Hence I have following options. >> >> 1) Use separate ElasticSearch instance with latest versions and update >> apiman.properties to point to this instance. And with this new version >> instance, I can use the latest tools. >> >> Question : Do you think the ES API 1.7.x is compatible and write >> indices to ES database running in latest version ? >> >> 2) Migrate the ES indices to new versions. But the below article, >> clearly says that the ES version must be at least 2.4.x for smooth >> migration using their migration tool. >> >> https://www.elastic.co/guide/en/elasticsearch/reference/curr >> ent/reindex-upgrade.html >> >> 3) Upgrade bundled ES API 1.7.x to 5.2.x (latest version) or at least >> 2.4.x version >> >> >> I think it is practical to upgrade bundled ES version to 2.4.x at least >> (if not for latest 5.2.x) just to make it compatible for migrating indices >> easily. >> >> What are your thoughts? I'm happy to see any alternative options other >> than mentioned above. Thanks. >> >> regards >> Balu >> >> > > _______________________________________________ > Apiman-user mailing list > Apiman-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/apiman-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20170317/4c8e3801/attachment-0001.html From eric.wittmann at redhat.com Tue Mar 21 14:49:29 2017 From: eric.wittmann at redhat.com (Eric Wittmann) Date: Tue, 21 Mar 2017 14:49:29 -0400 Subject: [Apiman-user] Elastic search upgrade In-Reply-To: References: Message-ID: In reality, the default embedded instance of Elasticsearch is just a separate WAR deployed to Wildfly. It's just there for convenience in the quickstart - not intended to go into production. In order to remove it, all you need to do is remove the apiman-es.war file from wildfly's deployments directory: [ewittman at host ~]$ cd ~/apiman-1.2.9.Final/wildfly-10.1.0.Final/standalone/deployments/ [ewittman at host deployments]$ rm apiman-es.war Restart Wildfly, and you're done! :) -Eric On Tue, Mar 21, 2017 at 1:51 PM, Balu S wrote: > Hi Eric, > Thanks for your response. > > One question though. I have noticed that during the server startup, even > though we specify standalone ES instance running (port configured in > apiman.properties), the default embedded instance is anyway started as well. > > 2017-03-21 18:42:21,915 INFO [org.elasticsearch.http] (ServerService > Thread Pool -- 64) [Norrin Radd] bound_address {inet[/10.12.129.105:19201]}, > publish_address {inet[/1x.xx.xxx.1xx:19201]} > > http://localhost:19201/_nodes/?pretty -- works (shows apiman specific > cluster) > > http://localhost:19200/_nodes/?pretty -- works (shows application cluster) > > So ultimately. there are two ES instances running. Although during the > actual operation It seems to use the standalone ES instance that is > mentioned in apiman.properties. But why does the need to start embedded > instance if there is already instance is running ? > > regards > Balu. > > > > > > > > On Fri, Mar 17, 2017 at 5:33 PM, Eric Wittmann > wrote: > >> Glad that you found an acceptable workaround. If anyone on this list is >> interested in identifying the list of incompatibilities between ES versions >> (so that we can upgrade support in apiman to the latest version) then we'd >> be happy to make those changes. >> >> As I'm sure everyone is already aware, the full apiman solution is now in >> maintenance-only mode. This means that major functionality will not be >> added (by us). However, with some community support for something like >> this, I think we would be able to make time for it. >> >> Please note that Marc (msavy) continues to actively work on the Gateway, >> with a focus on releasing (soon) a vert.x based version. So the project >> isn't entirely silent! :) >> >> -Eric >> >> >> On Fri, Mar 17, 2017 at 9:51 AM, Balu S wrote: >> >>> Ok, So option 3) is not straightforward. The build fails as there is >>> bunch of API incompatibility between 1.7 and 2.4 versions. Unless one >>> wiling to refactor the apiman-es source to use 2.4 API or may be latest 5.2. >>> >>> I have gone ahead with option 1 and used a separate ES instance running >>> with 2.4.4 (cannot go higher than this version due to incompatibility with >>> bundled ES API in apiman) version and it seems fine. Since I'm not worried >>> about the old indices, this looks like straightforward option. >>> >>> But if someone want to use old indices, then you may need to dig deeper >>> to re-index may be (?). >>> >>> regards >>> Balu >>> >>> On Thu, Mar 16, 2017 at 10:06 AM, Balu S wrote: >>> >>>> Hello, >>>> I'm trying to gather metrics (using Kibana tool) from the ES instance >>>> that is being used by apiman. But what I found is the ES instance that is >>>> bundled inside the apiman is 1.7.x version which seems to be too old for >>>> current trending tools to index upon. Of course we can use old versions of >>>> those tools, for ex. Kibana 4.1.x version to index and gather metrics. But >>>> the older version are limited in functionality and I would like to use the >>>> latest if possible. >>>> >>>> Hence I have following options. >>>> >>>> 1) Use separate ElasticSearch instance with latest versions and >>>> update apiman.properties to point to this instance. And with this new >>>> version instance, I can use the latest tools. >>>> >>>> Question : Do you think the ES API 1.7.x is compatible and write >>>> indices to ES database running in latest version ? >>>> >>>> 2) Migrate the ES indices to new versions. But the below article, >>>> clearly says that the ES version must be at least 2.4.x for smooth >>>> migration using their migration tool. >>>> >>>> https://www.elastic.co/guide/en/elasticsearch/reference/curr >>>> ent/reindex-upgrade.html >>>> >>>> 3) Upgrade bundled ES API 1.7.x to 5.2.x (latest version) or at least >>>> 2.4.x version >>>> >>>> >>>> I think it is practical to upgrade bundled ES version to 2.4.x at least >>>> (if not for latest 5.2.x) just to make it compatible for migrating indices >>>> easily. >>>> >>>> What are your thoughts? I'm happy to see any alternative options other >>>> than mentioned above. Thanks. >>>> >>>> regards >>>> Balu >>>> >>>> >>> >>> _______________________________________________ >>> Apiman-user mailing list >>> Apiman-user at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/apiman-user >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20170321/135fae27/attachment.html From sbalu27 at gmail.com Wed Mar 22 04:30:10 2017 From: sbalu27 at gmail.com (Balu S) Date: Wed, 22 Mar 2017 09:30:10 +0100 Subject: [Apiman-user] Elastic search upgrade In-Reply-To: References: Message-ID: perfect! Thanks Eric. On Tue, Mar 21, 2017 at 7:49 PM, Eric Wittmann wrote: > In reality, the default embedded instance of Elasticsearch is just a > separate WAR deployed to Wildfly. It's just there for convenience in the > quickstart - not intended to go into production. In order to remove it, > all you need to do is remove the apiman-es.war file from wildfly's > deployments directory: > > > [ewittman at host ~]$ cd ~/apiman-1.2.9.Final/wildfly- > 10.1.0.Final/standalone/deployments/ > [ewittman at host deployments]$ rm apiman-es.war > > > Restart Wildfly, and you're done! :) > > -Eric > > > On Tue, Mar 21, 2017 at 1:51 PM, Balu S wrote: > >> Hi Eric, >> Thanks for your response. >> >> One question though. I have noticed that during the server startup, even >> though we specify standalone ES instance running (port configured in >> apiman.properties), the default embedded instance is anyway started as well. >> >> 2017-03-21 18:42:21,915 INFO [org.elasticsearch.http] (ServerService >> Thread Pool -- 64) [Norrin Radd] bound_address {inet[/10.12.129.105:19201]}, >> publish_address {inet[/1x.xx.xxx.1xx:19201]} >> >> http://localhost:19201/_nodes/?pretty -- works (shows apiman specific >> cluster) >> >> http://localhost:19200/_nodes/?pretty -- works (shows application >> cluster) >> >> So ultimately. there are two ES instances running. Although during the >> actual operation It seems to use the standalone ES instance that is >> mentioned in apiman.properties. But why does the need to start embedded >> instance if there is already instance is running ? >> >> regards >> Balu. >> >> >> >> >> >> >> >> On Fri, Mar 17, 2017 at 5:33 PM, Eric Wittmann >> wrote: >> >>> Glad that you found an acceptable workaround. If anyone on this list is >>> interested in identifying the list of incompatibilities between ES versions >>> (so that we can upgrade support in apiman to the latest version) then we'd >>> be happy to make those changes. >>> >>> As I'm sure everyone is already aware, the full apiman solution is now >>> in maintenance-only mode. This means that major functionality will not be >>> added (by us). However, with some community support for something like >>> this, I think we would be able to make time for it. >>> >>> Please note that Marc (msavy) continues to actively work on the Gateway, >>> with a focus on releasing (soon) a vert.x based version. So the project >>> isn't entirely silent! :) >>> >>> -Eric >>> >>> >>> On Fri, Mar 17, 2017 at 9:51 AM, Balu S wrote: >>> >>>> Ok, So option 3) is not straightforward. The build fails as there is >>>> bunch of API incompatibility between 1.7 and 2.4 versions. Unless one >>>> wiling to refactor the apiman-es source to use 2.4 API or may be latest 5.2. >>>> >>>> I have gone ahead with option 1 and used a separate ES instance running >>>> with 2.4.4 (cannot go higher than this version due to incompatibility with >>>> bundled ES API in apiman) version and it seems fine. Since I'm not worried >>>> about the old indices, this looks like straightforward option. >>>> >>>> But if someone want to use old indices, then you may need to dig deeper >>>> to re-index may be (?). >>>> >>>> regards >>>> Balu >>>> >>>> On Thu, Mar 16, 2017 at 10:06 AM, Balu S wrote: >>>> >>>>> Hello, >>>>> I'm trying to gather metrics (using Kibana tool) from the ES instance >>>>> that is being used by apiman. But what I found is the ES instance that is >>>>> bundled inside the apiman is 1.7.x version which seems to be too old for >>>>> current trending tools to index upon. Of course we can use old versions of >>>>> those tools, for ex. Kibana 4.1.x version to index and gather metrics. But >>>>> the older version are limited in functionality and I would like to use the >>>>> latest if possible. >>>>> >>>>> Hence I have following options. >>>>> >>>>> 1) Use separate ElasticSearch instance with latest versions and >>>>> update apiman.properties to point to this instance. And with this new >>>>> version instance, I can use the latest tools. >>>>> >>>>> Question : Do you think the ES API 1.7.x is compatible and write >>>>> indices to ES database running in latest version ? >>>>> >>>>> 2) Migrate the ES indices to new versions. But the below article, >>>>> clearly says that the ES version must be at least 2.4.x for smooth >>>>> migration using their migration tool. >>>>> >>>>> https://www.elastic.co/guide/en/elasticsearch/reference/curr >>>>> ent/reindex-upgrade.html >>>>> >>>>> 3) Upgrade bundled ES API 1.7.x to 5.2.x (latest version) or at least >>>>> 2.4.x version >>>>> >>>>> >>>>> I think it is practical to upgrade bundled ES version to 2.4.x at >>>>> least (if not for latest 5.2.x) just to make it compatible for migrating >>>>> indices easily. >>>>> >>>>> What are your thoughts? I'm happy to see any alternative options other >>>>> than mentioned above. Thanks. >>>>> >>>>> regards >>>>> Balu >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Apiman-user mailing list >>>> Apiman-user at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/apiman-user >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20170322/e2aca9ff/attachment-0001.html From harrytpc at gmail.com Thu Mar 23 15:16:24 2017 From: harrytpc at gmail.com (Harry Trinta) Date: Thu, 23 Mar 2017 16:16:24 -0300 Subject: [Apiman-user] Edition Published APIs Message-ID: Hi, guys! When I publish a API in Apiman, I can't edit it anymore. But, sometimes, is necessary to change the API Endpoint, plan, etc, and it is not possible. Currently, I have to create another API version, or else, to delete the API and create it again with the new configuration. Is there a way to enable API edition of a published API? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20170323/e813fd6b/attachment.html From ashish.patel at futuregroup.in Fri Mar 24 00:28:55 2017 From: ashish.patel at futuregroup.in (Ashish Patel) Date: Fri, 24 Mar 2017 04:28:55 +0000 Subject: [Apiman-user] Edition Published APIs In-Reply-To: References: Message-ID: <673312ca0bd2493d9e9246a12ac639c4@futuregroup.in> Hi Harry, No offence meant, but that?s the way APIs are being designed, so that consumers are protected/intimated in advance for the changes in API contracts. Here, you can change v1.0 definition - N times before you have published to consumers. Say you have API version v1.0 published and 3 consumers already using it, now you need to change the contract parameters and these consumers have already published their applications using your API v1.0. So, ideally you should publish v1.1 along with keeping v1.0 for a given period (say a week to month) allowing consumers to move to new version v1.1 . Practically you can also retire v1.0 itself and create v1.1, however this should be done with prior communication to consumers for changing their endpoints ? seems very challenging considering different applications? different release cycles. Personal view ? anyone else has any different angle to it? Thanks & Regards, Ashish From: apiman-user-bounces at lists.jboss.org [mailto:apiman-user-bounces at lists.jboss.org] On Behalf Of Harry Trinta Sent: 24 March 2017 00:46 To: apiman-user at lists.jboss.org Subject: [Apiman-user] Edition Published APIs Hi, guys! When I publish a API in Apiman, I can't edit it anymore. But, sometimes, is necessary to change the API Endpoint, plan, etc, and it is not possible. Currently, I have to create another API version, or else, to delete the API and create it again with the new configuration. Is there a way to enable API edition of a published API? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20170324/3ba71e41/attachment.html From eric.wittmann at redhat.com Fri Mar 24 07:40:45 2017 From: eric.wittmann at redhat.com (Eric Wittmann) Date: Fri, 24 Mar 2017 07:40:45 -0400 Subject: [Apiman-user] Edition Published APIs In-Reply-To: <673312ca0bd2493d9e9246a12ac639c4@futuregroup.in> References: <673312ca0bd2493d9e9246a12ac639c4@futuregroup.in> Message-ID: Ashish is quite right about the design. The goal was to shield applications/clients from having the API contract changed out from under them AFTER they had agreed to the terms. Clearly, there are *some* changes (e.g. the API endpoint) which would not affect the Contract - so a more mature implementation of our approach would take that sort of thing into account. However, there would always be some grey area to it. We had intended to implement a feature whereby *installers* of apiman could loosen the restrictions and allow re-publishing of APIs, but we didn't get to it before the 3scale acquisition. It's actually something that wouldn't be too hard to implement if someone in the community were willing to give it a shot. Also note that "public only" APIs CAN be re-published, since by definition there are no contracts at risk. -Eric On Fri, Mar 24, 2017 at 12:28 AM, Ashish Patel wrote: > Hi Harry, > > > > No offence meant, but that?s the way APIs are being designed, so that > consumers are protected/intimated in advance for the changes in API > contracts. Here, you can change v1.0 definition - N times before you have > published to consumers. > > > > Say you have API version v1.0 published and 3 consumers already using it, > now you need to change the contract parameters and these consumers have > already published their applications using your API v1.0. So, ideally you > should publish v1.1 along with keeping v1.0 for a given period (say a week > to month) allowing consumers to move to new version v1.1 . Practically you > can also retire v1.0 itself and create v1.1, however this should be done > with prior communication to consumers for changing their endpoints ? seems > very challenging considering different applications? different release > cycles. > > > > Personal view ? anyone else has any different angle to it? > > > > Thanks & Regards, > > Ashish > > > > *From:* apiman-user-bounces at lists.jboss.org [mailto:apiman-user-bounces@ > lists.jboss.org] *On Behalf Of *Harry Trinta > *Sent:* 24 March 2017 00:46 > *To:* apiman-user at lists.jboss.org > *Subject:* [Apiman-user] Edition Published APIs > > > > Hi, guys! > > > > When I publish a API in Apiman, I can't edit it anymore. > > But, sometimes, is necessary to change the API Endpoint, plan, etc, and it > is not possible. > > Currently, I have to create another API version, or else, to delete the > API and create it again with the new configuration. > > > > Is there a way to enable API edition of a published API? > > > > Thanks > > _______________________________________________ > Apiman-user mailing list > Apiman-user at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/apiman-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/apiman-user/attachments/20170324/0cec66e3/attachment.html