Etcd监控

五、etcd监控

自定义抓取k8s集群外部target的三个步骤：servicemonitor + service + endpoint

【前提】
如果要抓取etcd的metrics，需要设置etcd的参数来暴露metrics：--listen-metrics-urls='http://0.0.0.0:2381'

1、创建ServiceMonitor

cat <<EOF |kubectl apply -n cattle-prometheus -f -
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    app: exporter-node
    chart: exporter-node-0.0.1
    heritage: Tiller
    io.cattle.field/appId: cluster-monitoring
    release: cluster-monitoring
    source: rancher-monitoring
  name: prometheus-operator-kube-etcd
  namespace: cattle-prometheus
spec:
  endpoints:
  - bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
    port: http-metrics
    relabelings:
    - action: replace
      regex: (.+):(.+)
      replacement: $1
      sourceLabels:
      - __address__
      targetLabel: host_ip
  namespaceSelector:
    matchNames:
    - kube-system
  selector:
    matchLabels:
      app: prometheus-operator-kube-etcd
      release: prometheus-operator
EOF

2、创建Service、Endpoints

cat <<EOF |kubectl apply -n cattle-prometheus -f -
apiVersion: v1
kind: Service
metadata:
  name: etcd-k8s
  namespace: kube-system
  labels:
    app: prometheus-operator-kube-etcd
    release: prometheus-operator
spec:
  type: ClusterIP
  ports:
  - name: http-metrics
    port: 2381
    protocol: TCP

---
apiVersion: v1
kind: Endpoints
metadata:
  name: etcd-k8s
  namespace: kube-system
  labels:
    app: prometheus-operator-kube-etcd
    release: prometheus-operator
subsets:
- addresses:
  - ip: 172.22.22.18
    nodeName: k8s-master01
  - ip: 172.22.22.19
    nodeName: k8s-master02
  - ip: 172.22.22.20
    nodeName: k8s-master03
  ports:
  - name: http-metrics
    port: 2381
    protocol: TCP
EOF

以上配置成功后，会在配置文件/etc/prometheus/config_out/prometheus.env.yaml中生成相关的scrape配置。