ljzsdut
GitHubToggle Dark/Light/Auto modeToggle Dark/Light/Auto modeToggle Dark/Light/Auto modeBack to homepage
  1. Welcome to the ljzsdut's notes
  2. /
  3. 云原生
  4. /
  5. Kubernetes
  6. /
  7. 7.2 K8s网络插件之 Network Policy
Edit page

7.2 K8s网络插件之 Network Policy

Canal：networkpolicy

networkpolicy简称netpol，类似于OS的防火墙规则，控制Pod/namespace之间的访问规则。

https://docs.projectcalico.org/v3.7/getting-started/kubernetes/installation/calico

安装canal：

**Installing Calico for policy and flannel for networking with the Kubernetes API datastore **

  1. Ensure that the Kubernetes controller manager has the following flags set: --cluster-cidr=10.244.0.0/16 and --allocate-node-cidrs=true.

    Tip: If you’re using kubeadm, you can pass --pod-network-cidr=10.244.0.0/16 to kubeadm to set the Kubernetes controller flags.

  2. If your cluster has RBAC enabled, issue the following command to configure the roles and bindings that Calico requires.

    kubectl apply -f \
https://docs.projectcalico.org/v3.2/getting-started/kubernetes/installation/hosted/canal/rbac.yaml

    Note: You can also view the manifest in your browser.

  3. Issue the following command to install Calico.

    kubectl apply -f \
https://docs.projectcalico.org/v3.2/getting-started/kubernetes/installation/hosted/canal/canal.yaml

    Note: You can also view the manifest in your browser.

  4. If you wish to enforce application layer policies and secure workload-to-workload communications with mutual TLS authentication, continue to Enabling application layer policy (optional).

创建NetworkPolicy并测试： 

#准备名称空间和相关的测试Pod：
[root@k8smaster networkpolicy]# kubectl create ns dev
namespace/dev created
[root@k8smaster networkpolicy]# kubectl create ns prod
namespace/prod created

[root@k8smaster networkpolicy]# cat pod-a.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: pod1
  labels:
    app: myapp
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
    
[root@k8smaster networkpolicy]# kubectl apply -f pod-a.yaml -n dev
pod/pod1 created
[root@k8smaster networkpolicy]# kubectl apply -f pod-a.yaml -n prod
pod/pod1 created
[root@k8smaster networkpolicy]# kubectl get pod -o wide -n dev  
NAME      READY     STATUS    RESTARTS   AGE       IP           NODE
pod1      1/1       Running   0          29s       10.244.2.3   k8snode02
[root@k8smaster networkpolicy]# kubectl get pod -o wide -n prod
NAME      READY     STATUS    RESTARTS   AGE       IP           NODE
pod1      1/1       Running   0          40s       10.244.1.3   k8snode01
[root@k8smaster networkpolicy]# curl 10.244.2.3
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@k8smaster networkpolicy]# curl 10.244.1.3
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
#以上命令创建了两个名称空间，并在两个名称空间中各创建了一个pod，通过在master上访问各个名称空间中的Pod，发现都可以访问到。

#创建NetworkPolicy：拒绝访问dev中的所有Pod    
[root@k8smaster networkpolicy]# cat ingress-deny-all.yaml 
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-ingress
spec:
  podSelector: {} #表示所有的Pod
  policyTypes:
  - Ingress     #表示启用Ingress规则，但是Ingress规则未定义任何规则，所以Ingress为拒绝所有
[root@k8smaster networkpolicy]# kubectl apply -f ingress-deny-all.yaml -n dev
networkpolicy.networking.k8s.io/deny-all-ingress created
[root@k8smaster networkpolicy]# curl 10.244.2.3
^C  #请求被阻塞了，无法访问dev中的Pod
[root@k8smaster networkpolicy]# curl 10.244.1.3
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>  #prod中的Pod可以访问

#创建NetworkPolicy：10.244.0.0/16网段中除10.244.1.2/32外的所有IP允许访问app=myapp的Pod的80和443端口。
[root@k8smaster networkpolicy]# cat allow-ipBlock-demo.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-myapp-ingress
spec:
  podSelector:
    matchLabels:
      app: myapp
  ingress:
  - from:
    - ipBlock: 
        cidr: 10.244.0.0/16
        except:
        - 10.244.1.2/32
    ports:
    - protocol: TCP
      port: 80
    - protocol: TCP
      port: 443
[root@k8smaster networkpolicy]# kubectl apply -f allow-ipBlock-demo.yaml -n dev
networkpolicy.networking.k8s.io/allow-myapp-ingress created
[root@k8smaster networkpolicy]# curl 10.244.2.3
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@k8smaster networkpolicy]# curl 10.244.1.3
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@k8smaster networkpolicy]# curl 10.244.2.3:443
curl: (7) Failed connect to 10.244.2.3:443; Connection refused #请求被linux内核拒绝了，而不是被阻塞。说明请求已经到达Pod，但是没有监听443端口，所有linux内核将请求拒绝了。
[root@k8smaster networkpolicy]# curl 10.244.2.3:6443
^C #请求被阻塞，说明请求根据就没有到达Pod。

#在dev名称空间中定义一个没有app=myapp标签的Pod
[root@k8smaster networkpolicy]# cat pod-b.yaml   
apiVersion: v1
kind: Pod
metadata:
  name: pod2
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
[root@k8smaster networkpolicy]# kubectl apply -f pod-b.yaml -n dev
pod/pod2 created
[root@k8smaster networkpolicy]# kubectl get pods -n dev -o wide --show-labels
NAME      READY     STATUS    RESTARTS   AGE       IP           NODE        LABELS
pod1      1/1       Running   0          20m       10.244.2.3   k8snode02   app=myapp
pod2      1/1       Running   0          1m        10.244.2.4   k8snode02   <none>
[root@k8smaster networkpolicy]# curl 10.244.2.4
^C  #拒绝访问
[root@k8smaster networkpolicy]# curl 10.244.2.3
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>  #允许访问

#名称空间dev中有多个NetworkPolicy，他们哪个优先生效呢？
[root@k8smaster networkpolicy]# kubectl get netpol -n dev                    
NAME                  POD-SELECTOR   AGE
allow-myapp-ingress   app=myapp      5m
deny-all-ingress      <none>         19m
[root@k8smaster networkpolicy]# curl 10.244.2.4
^C  #此时，无法访问
[root@k8smaster networkpolicy]# kubectl delete -f ingress-deny-all.yaml  -n dev
networkpolicy.networking.k8s.io "deny-all-ingress" deleted
[root@k8smaster networkpolicy]# curl 10.244.2.4
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>  #删除“拒绝所有ingress”，可以访问
[root@k8smaster networkpolicy]# kubectl apply -f ingress-deny-all.yaml  -n dev        
networkpolicy.networking.k8s.io/deny-all-ingress created
[root@k8smaster networkpolicy]# curl 10.244.2.4
^C   #创建“拒绝所有ingress”，无法访问
#总结： 同一个名称空间中有多个NetworkPolicy，其优先生效规则跟创建顺序无关。生效策略为最小最优最佳匹配规则。所以我们在使用NetworkPolicy时，可以先配置一个默认策略，然后在默认策略的基础上定义细化的策略。

NetworkPolicy示例

**几个NetworkPolicy资源配置文件示例： **

[root@k8smaster networkpolicy]# cat ingress-allow-all.yaml 
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-all-ingress
spec:
  podSelector: {} #表示所有的Pod,当然只对指定名称空间中的Pod生效
  ingress: #定义ingress规则
  - {}  #{}表示入站允许所有规则,这些规则会被应用到podSelector匹配的pod上;如果ingress字段未定义，则表示没有规则，即禁止所有traffic
  policyTypes:
  - Ingress    #表示生效规则为Ingress；Egress未写，表示不启用Egress规则,默认允许。

[root@k8smaster networkpolicy]# cat ingress-deny-all.yaml 
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-ingress
spec:
  podSelector: {} #表示所有的Pod：pod选择器，基于标签选择与Network Policy处于同一namespace下的pod，如果pod被选中，则对其应用Network Policy中定义的规则。此为可选字段，当没有此字段时，表示选中所有pod。
  policyTypes:
  - Ingress     #表示启用Ingress规则，但是Ingress规则未定义任何规则，所以Ingress为拒绝所有
  #ingress: 		#定义ingress规则

[root@k8smaster networkpolicy]# cat allow-ipBlock-demo.yaml 
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-myapp-ingress
spec:
  podSelector:
    matchLabels:
      app: myapp
  ingress:
  - from:
    - ipBlock: 
        cidr: 10.244.0.0/16
        except:
        - 10.244.1.2/32
    ports:
    - protocol: TCP
      port: 80
    - protocol: TCP
      port: 443

[root@k8smaster networkpolicy]# cat egress-deny-all.yaml 
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-egress
spec:
  podSelector: {}
  policyTypes:
  - Egress

基本原理

Network Policy是kubernetes中的一种资源类型，它从属于某个namespace。其内容从逻辑上看包含两个关键部分，一是pod选择器，基于标签选择相同namespace下的pod，将其中定义的规则作用于选中的pod。另一个就是规则了，就是网络流量进出pod的规则，其采用的是白名单模式，符合规则的通过，不符合规则的拒绝。

Network Policy对象Spec说明

首先给出示例Spec，结合示例说明Spec中的关键字段与逻辑，关于Spec完全说明参考这里。示例如下：

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - ipBlock:
        cidr: 172.17.0.0/16
        except:
        - 172.17.1.0/24
    - namespaceSelector:
        matchLabels:
          project: myproject
    - podSelector:
        matchLabels:
          role: frontend
    ports:
    - protocol: TCP
      port: 6379
  egress:
  - to:
    - ipBlock:
        cidr: 10.0.0.0/24
    ports:
    - protocol: TCP
      port: 5978

对象创建方法与其它如ReplicaSet相同。apiVersion、kind、metadata与其它类型对象含义相同，不详细描述。

.spec.PodSelector

顾名思义，它是pod选择器，基于标签选择与Network Policy处于同一namespace下的pod，如果pod被选中，则对其应用Network Policy中定义的规则。此为可选字段，当没有此字段时，表示选中所有pod。

.spec.PolicyTypes

Network Policy定义的规则可以分成两种，一种是入pod的Ingress规则，一种是出pod的Egress规则。本字段可以看作是一个开关，如果其中包含Ingress，则Ingress部分定义的规则生效，如果是Egress则Egress部分定义的规则生效，如果都包含则全部生效。当然此字段也可选，如果没有指定的话，则默认Ingress生效，如果Egress部分有定义的话，Egress才生效。怎么理解这句话，下文会提到，没有明确定义Ingress、Egress部分，它也是一种规则，默认规则而非没有规则。

.spec.ingress与.spec.egress

前者定义入pod规则，后者定义出pod规则，详细参考这里，这里只讲一下重点。上例中ingress与egress都只包含一条规则，两者都是数组，可以包含多条规则。当包含多条时，条目之间的逻辑关系是“或”，只要匹配其中一条就可以。.spec.ingress[].from 也是数组，数组成员对访问pod的外部source进行描述，符合条件的source才可以访问pod，有多种方法，如示例中的ip地址块、名称空间、pod标签等，数组中的成员也是逻辑或的关系。spec.ingress[].from.prots表示允许通过的协议及端口号。

.spec.egress.to定义的是pod想要访问的外部destination，其它与ingress相同。

默认规则

不定义规则并非没有规则，此时默认规则生效，以下展示默认规则用法。

默认禁止所有入pod流量(Default deny all ingress traffic) 

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
spec:
  podSelector: {}
  policyTypes:
  - Ingress

上例中没有定义pod选择器，表示如果namespace下的某个pod没有被任何Network Policy对象选中，则应用此对象，如果被其它Network Policy先中则不应用此对象。

policyTypes的值为Ingress，表示本例启用Ingress规则。但是本例没有定义具体的Ingress，那就应用默认规则。默认规则禁止所有入pod流量，但例外情况是如果source就是pod运行的节点，则允许通过。

默认允许所有入pod流量(Default allow all ingress traffic) 

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-all
spec:
  podSelector: {}
  ingress:
  - {}

同样没有定义pod选择器，意义与上例同。注意ingress的定义，这个是有规则的，只是规则中的条目为空，与默认规则不同，表示全部允许通过。

默认禁止所有出pod流量(Default deny all egress traffic) 

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
spec:
  podSelector: {}
  policyTypes:
  - Egress

与默认禁止所有入pod流量(Default deny all ingress traffic)同，只是流量由入变成出

默认允许所有出pod流量(Default allow all egress traffic) 

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-all
spec:
  podSelector: {}
  egress:
  - {}
  policyTypes:
  - Egress

与默认允许所有入pod流量(Default allow all ingress traffic)同，只是流量由入变成出。

默认禁止所有入出pod流量(Default deny all ingress and all egress traffic) 

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

无需详解，但请注意，pod与所运行节点之间流量不受Network Policy限制。

真实用例

下面通过一个真实示例展示Network Policy普通用法

用Deployment创建nginx pod实例并用service暴露 

$ kubectl run nginx --image=nginx --replicas=2
deployment "nginx" created
$ kubectl expose deployment nginx --port=80
service "nginx" exposed

确认创建结果 

$ kubectl get svc,pod
NAME                        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
svc/kubernetes              10.100.0.1    <none>        443/TCP    46m
svc/nginx                   10.100.0.16   <none>        80/TCP     33s
 
NAME                        READY         STATUS        RESTARTS   AGE
po/nginx-701339712-e0qfq    1/1           Running       0          35s
po/nginx-701339712-o00ef    1/1           Running       0          35s

测试nginx服务连通性 

$ kubectl run busybox --rm -ti --image=busybox /bin/sh
Waiting for pod default/busybox-472357175-y0m47 to be running, status is Pending, pod ready: false
 
Hit enter for command prompt
 
/ # wget --spider --timeout=1 nginx
Connecting to nginx (10.100.0.16:80)
/ #

通过创建Network Policy对象添加隔离性 

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: access-nginx
spec:
  podSelector:
    matchLabels:
      run: nginx
  ingress:
  - from:
    - podSelector:
        matchLabels:
          access: "true"

只允许包含access: “true"标签的pod访问nginx服务。

创建Network Policy 

$ kubectl create -f nginx-policy.yaml
networkpolicy "access-nginx" created

测试隔离性 

$ kubectl run busybox --rm -ti --image=busybox /bin/sh
Waiting for pod default/busybox-472357175-y0m47 to be running, status is Pending, pod ready: false
 
Hit enter for command prompt
 
/ # wget --spider --timeout=1 nginx
Connecting to nginx (10.100.0.16:80)
wget: download timed out
/ #

为pod添加access: “true"标签测试连通性 

$ kubectl run busybox --rm -ti --labels="access=true" --image=busybox /bin/sh
Waiting for pod default/busybox-472357175-y0m47 to be running, status is Pending, pod ready: false
 
Hit enter for command prompt
 
/ # wget --spider --timeout=1 nginx
Connecting to nginx (10.100.0.16:80)
/ #