2. I add the client certs as secrets and mount them to the agent:[root@localhost master]# pwd/var/lib/origin/openshift.local.config/master[root@localhost master]# ls -ltr *etc*-rw-rw-rw-. 1 root root 1078 Sep 23 18:31 master.etcd-client.crt-rw-rw-rw-. 1 root root 1679 Sep 23 18:31 master.etcd-client.key-rw-rw-rw-. 1 root root 1675 Sep 23 18:31 etcd.server.key-rw-rw-rw-. 1 root root 2550 Sep 23 18:31 etcd.server.crt
3. Then edit the config map of the agent and add in the below, which matches the above secret mounts:oc project openshift-infraoc secrets new etcd-client-crt master.etcd-client.crtoc secrets new etcd-client-key master.etcd-client.keyoc volume rc/hawkular-openshift-agent --add --name=etcd-client-crt --type=secret --secret-name=etcd-client-crt --mount-path=/run/secrets/etcd-client-crtoc volume rc/hawkular-openshift-agent --add --name=etcd-client-key --type=secret --secret-name=etcd-client-key --mount-path=/run/secrets/etcd-client-key
4. Restart the pod to force a refresh and check the logs, which shows:oc edit configmap hawkular-openshift-agent-configurationidentity:cert_file: /run/secrets/etcd-client-crt/master.etcd-client.crtprivate_key_file: /run/secrets/etcd-client-key/master.etcd-client.key
I now have a working agent collecting from etcd.I1228 10:20:18.799687 1 prometheus_metrics_collector.go:97] DEBUG: Told to collect [2] Prometheus metrics from [https://172.17.0.8:9779/metrics]I1228 10:20:18.984615 1 metrics_storage.go:152] DEBUG: Stored datapoints for [2] metrics
openssl req -newkey rsa:2048 -nodes -keyout agent.key -out agent.csr -subj "/C=UK/ST=Yorkshire/L=Leeds/O=Home/CN=hawkular-agent"openssl x509 -signkey agent.key -in agent.csr -req -days 365 -out agent.crt
Gareth,
OK, there are a couple things here that I'm confused about. This is how I would understand things working.
If you want to connect to any https endpoint, the agent will need SOME identity (so you have to give it SOME public/private key pair - which is what the Identity section does). It doesn't have to be the key-pair of the server (in fact, under normal situations it is not - the server is identified with its own public/private key and the client with another). But the point is, if you are connecting to an https endpoint, you can't leave Identity section out of the agent config.
So when you say, "without the "Identity" configuration section set on the agent config, i'd get a TLS error" this is what I would expect. You can't leave the Identity section out when connecting via https because in that case the agent has no keys to talk TLS to the server.
What does your agent config look like when you get things to work? (I assume you do get it to work because you said without the Identity you get a TLS error, which implies you do get it to work WITH an Identity section - is this correct?) What key files are you putting in the agent Identity when you get it to work?
So I guess what I am saying is - have you tried to generate your own certificate and assigned it to your agent's Identity and then tried to connect to multiple https endpoints? Because as I mentioned earlier in another post, the agent today doesn't do server-cert verification, so it should "just work". You shouldn't need different Identities per endpoint. Once we add in verification, the endpoints you want to collect metrics from would need their server-side certs to be signed with a CA that the agent trusts (i.e. from the agent host's default root CA set) - we would then have to add the ability for the agent to be told about different CAs in case your server-side certs are, say, self-signed or signed with your own CA that isn't a trusted one found in the host's default root CA set.
Oh, and, Merry Christmas!
John Mazz
----- Original Message -----
> One of the first services i am trying to monitor is etcd. etcd in OCP is
> configured as per the below:
>
> /var/lib/origin/openshift.local.config/master/master- config.yaml
>
>
> etcdClientInfo:
> ca: ca.crt
> certFile: master.etcd-client.crt
> keyFile: master.etcd-client.key
> urls:
> - https://10.2.2.2:4001
>
> Which responds with the below cURL:
>
> curl https://10.2.2.2:4001/metrics --cacert ./ca.crt --cert
> ./master.etcd-client.crt --key ./master.etcd-client.key
>
> So without the "Identity" configuration section set on the agent config,
> i'd get a TLS error. As etcd is a core part of OCP, I don't have much
> control over the client certs and expect there might be other services
> which require the same setup using different certs that i might want to
> monitor.
>
> Hope that makes things clear, and Merry Christmas.
>
> Cheers.
>
> On Sat, Dec 24, 2016 at 3:30 PM, John Mazzitelli <mazz@redhat.com> wrote:
>
> > > Currently it seems you can only provide the agent configmap with the
> > identity
> > > field. But what i want to actually do, is provide this based on the pods
> > > config map>
> > > [chomp]
> > > Is that possible? or planned for the future?
> >
> > I was hoping this wasn't going to be needed :) But we did talk about it.
> >
> > It is not possible today because there is one major problem with what you
> > suggest that would need to be solved somehow:
> >
> > > cert_file: /var/run/secrets/client-crt/client.crt
> > > private_key_file: /var/run/secrets/client-key/client.key
> >
> > That is inside your configmap on your OpenShift project (which may or may
> > not be the same project where the agent is deployed).
> >
> > So - what file system is that actually referring to? And how does the
> > agent get access to those files?
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev@lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
>