[infinispan-issues] [JBoss JIRA] (ISPN-6675) Better configuration management for Kubernetes
Sebastian Łaskawiec (JIRA)
issues at jboss.org
Thu Aug 11 04:30:00 EDT 2016
[ https://issues.jboss.org/browse/ISPN-6675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277691#comment-13277691 ]
Sebastian Łaskawiec edited comment on ISPN-6675 at 8/11/16 4:29 AM:
--------------------------------------------------------------------
POC with Configmaps:
# Spin a new OpenShift cluster with Infinispan app. Use [this blog post|http://blog.infinispan.org/2016/08/running-infinispan-cluster-on-openshift.html] for reference.
# Download the latest infinispan and store it somewhere on the disk
# Modify {{$ISPN_HOME/standalone/configuration/cloud.xml}}
# Create a ConfigMap based on {{$ISPN_HOME/standalone/configuration}} directory (unfortunately you can't mount ConfigMap as a single file - you need to mount it as a volume (directory)).
{code}
$ oc create configmap cloud-xml-3 --from-file=infinispan-server-9.0.0-SNAPSHOT/standalone/configuration
configmap "cloud-xml-3" created
{code}
# Mount the directory in the pods
{code}
$ oc edit dc/infinispan-server
.. Modify parts shown below ..
apiVersion: v1
kind: DeploymentConfig
metadata:
name: infinispan-server
namespace: myproject
selfLink: /oapi/v1/namespaces/myproject/deploymentconfigs/infinispan-server
uid: 09353e5f-5f98-11e6-b7f5-54ee751d46e3
resourceVersion: '10864'
generation: 6
creationTimestamp: '2016-08-11T07:48:47Z'
labels:
app: infinispan-server
annotations:
openshift.io/generated-by: OpenShiftNewApp
spec:
strategy:
type: Rolling
rollingParams:
updatePeriodSeconds: 1
intervalSeconds: 1
timeoutSeconds: 600
maxUnavailable: 25%
maxSurge: 25%
resources:
triggers:
-
type: ConfigChange
-
type: ImageChange
imageChangeParams:
automatic: true
containerNames:
- infinispan-server
from:
kind: ImageStreamTag
namespace: myproject
name: 'infinispan-server:latest'
lastTriggeredImage: 'jboss/infinispan-server at sha256:52b4fcb1530159176ceb81ea8d9638fa69b8403c8ca5ac8aea1cdbcb645beb9a'
replicas: 1
test: false
selector:
app: infinispan-server
deploymentconfig: infinispan-server
template:
metadata:
creationTimestamp: null
labels:
app: infinispan-server
deploymentconfig: infinispan-server
annotations:
openshift.io/container.infinispan-server.image.entrypoint: '["docker-entrypoint.sh"]'
openshift.io/generated-by: OpenShiftNewApp
spec:
(3) volumes:
-
name: config-volume
configMap:
name: cloud-xml-3
containers:
-
name: infinispan-server
image: 'jboss/infinispan-server at sha256:52b4fcb1530159176ceb81ea8d9638fa69b8403c8ca5ac8aea1cdbcb645beb9a'
ports:
-
containerPort: 8888
protocol: TCP
-
containerPort: 9990
protocol: TCP
-
containerPort: 11211
protocol: TCP
-
containerPort: 11222
protocol: TCP
-
containerPort: 57600
protocol: TCP
-
containerPort: 7600
protocol: TCP
-
containerPort: 8080
protocol: TCP
-
containerPort: 8181
protocol: TCP
resources:
(2) volumeMounts:
-
name: config-volume
mountPath: /opt/jboss/infinispan-server/standalone/configuration
terminationMessagePath: /dev/termination-log
imagePullPolicy: Always
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
securityContext:
status:
latestVersion: 5
observedGeneration: 6
replicas: 1
updatedReplicas: 1
unavailableReplicas: 1
details:
message: 'caused by a config change'
causes:
-
type: ConfigChange
(1) - creates a volume based on a ConfigMap
(2) - mounts it in {{$ISPN_HOME/standalone/configuration}}
{code}
# Trigger the rolling upgrade (WARNING: This heavily relies on Kubernetes Rolling Updates so all Health/Readiness probes needs to be in place, otherwise data loss might occur)
Comments:
* This works almost exactly the same in OpenShift and Kubernetes
* Proper readiness and health check configuration is a must
* Requires restarting all pods to refresh the configuration
was (Author: sebastian.laskawiec):
POC with Configmaps:
# Spin a new OpenShift cluster with Infinispan app. Use [this blog post|http://blog.infinispan.org/2016/08/running-infinispan-cluster-on-openshift.html] for reference.
# Download the latest infinispan and store it somewhere on the disk
# Modify {{$ISPN_HOME/standalone/configuration/cloud.xml}}
# Create a ConfigMap based on {{$ISPN_HOME/standalone/configuration}} directory (unfortunately you can't mount ConfigMap as a single file - you need to mount it as a volume (directory)).
{code}
$ oc create configmap cloud-xml-3 --from-file=infinispan-server-9.0.0-SNAPSHOT/standalone/configuration
configmap "cloud-xml-3" created
{code}
# Mount the directory in the pods
{code}
$ oc edit dc/infinispan-server
.. Modify parts shown below ..
apiVersion: v1
kind: DeploymentConfig
metadata:
name: infinispan-server
namespace: myproject
selfLink: /oapi/v1/namespaces/myproject/deploymentconfigs/infinispan-server
uid: 09353e5f-5f98-11e6-b7f5-54ee751d46e3
resourceVersion: '10864'
generation: 6
creationTimestamp: '2016-08-11T07:48:47Z'
labels:
app: infinispan-server
annotations:
openshift.io/generated-by: OpenShiftNewApp
spec:
strategy:
type: Rolling
rollingParams:
updatePeriodSeconds: 1
intervalSeconds: 1
timeoutSeconds: 600
maxUnavailable: 25%
maxSurge: 25%
resources:
triggers:
-
type: ConfigChange
-
type: ImageChange
imageChangeParams:
automatic: true
containerNames:
- infinispan-server
from:
kind: ImageStreamTag
namespace: myproject
name: 'infinispan-server:latest'
lastTriggeredImage: 'jboss/infinispan-server at sha256:52b4fcb1530159176ceb81ea8d9638fa69b8403c8ca5ac8aea1cdbcb645beb9a'
replicas: 1
test: false
selector:
app: infinispan-server
deploymentconfig: infinispan-server
template:
metadata:
creationTimestamp: null
labels:
app: infinispan-server
deploymentconfig: infinispan-server
annotations:
openshift.io/container.infinispan-server.image.entrypoint: '["docker-entrypoint.sh"]'
openshift.io/generated-by: OpenShiftNewApp
spec:
(3) volumes:
-
name: config-volume
configMap:
name: cloud-xml-3
containers:
-
name: infinispan-server
image: 'jboss/infinispan-server at sha256:52b4fcb1530159176ceb81ea8d9638fa69b8403c8ca5ac8aea1cdbcb645beb9a'
ports:
-
containerPort: 8888
protocol: TCP
-
containerPort: 9990
protocol: TCP
-
containerPort: 11211
protocol: TCP
-
containerPort: 11222
protocol: TCP
-
containerPort: 57600
protocol: TCP
-
containerPort: 7600
protocol: TCP
-
containerPort: 8080
protocol: TCP
-
containerPort: 8181
protocol: TCP
resources:
(2) volumeMounts:
-
name: config-volume
mountPath: /opt/jboss/infinispan-server/standalone/configuration
terminationMessagePath: /dev/termination-log
imagePullPolicy: Always
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
securityContext:
status:
latestVersion: 5
observedGeneration: 6
replicas: 1
updatedReplicas: 1
unavailableReplicas: 1
details:
message: 'caused by a config change'
causes:
-
type: ConfigChange
(1) - creates a volume based on a ConfigMap
(2) - mounts it in {{$ISPN_HOME/standalone/configuration}}
{code}
# Trigger the rolling upgrade (WARNING: This heavily relies on Kubernetes Rolling Updates so all Health/Readiness probes needs to be in place, otherwise data loss might occur)
> Better configuration management for Kubernetes
> ----------------------------------------------
>
> Key: ISPN-6675
> URL: https://issues.jboss.org/browse/ISPN-6675
> Project: Infinispan
> Issue Type: Feature Request
> Components: Cloud Integrations
> Reporter: Sebastian Łaskawiec
> Assignee: Sebastian Łaskawiec
>
> Currently we either store the configuration in the Docker Image or use system properties to turn the things on/off. Both approaches do not scale.
> There are some other alternatives like [Kubernetes ConfigMap|http://kubernetes.io/docs/user-guide/configmap] which are worth exploring
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
More information about the infinispan-issues
mailing list