Ingress와 네트워크 정책 예제
Ingress와 네트워크 정책 예제 (Ingress and Network Policy Example)
이 예제는 기본 HTTP Ingress 예제와 동일한 구성(Istio 프로젝트의 bookinfo 데모 마이크로서비스 앱)을 사용하면서, 그 위에 CiliumNetworkPolicy를 추가해요.
본문
이 예제는 기본 HTTP Ingress 예제와 동일한 구성(Istio 프로젝트의 bookinfo 데모 마이크로서비스 앱)을 사용한 다음, 그 위에 CiliumNetworkPolicy를 추가해요.
데모 앱 배포하기 (Deploy the Demo App)
$ kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.11/samples/bookinfo/platform/kube/bookinfo.yaml
이것은 데모 앱을 배포하는 것뿐이며, Istio 컴포넌트를 추가하는 게 아니에요. Cilium Service Mesh에서는 데모 앱 각 마이크로서비스와 함께 Envoy 사이드카가 생성되지 않는 것을 확인할 수 있어요.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
details-v1-5498c86cf5-kjzkj 1/1 Running 0 2m39s
productpage-v1-65b75f6885-ff59g 1/1 Running 0 2m39s
ratings-v1-b477cf6cf-kv7bh 1/1 Running 0 2m39s
reviews-v1-79d546878f-r5bjz 1/1 Running 0 2m39s
reviews-v2-548c57f459-pld2f 1/1 Running 0 2m39s
reviews-v3-6dd79655b9-nhrnh 1/1 Running 0 2m39s
참고
사이드카 구현이었다면 출력이 2/2 READY로 보였을 거예요. 하나는 마이크로서비스, 하나는 Envoy 사이드카 때문이에요.
첫 번째 Ingress 배포하기 (Deploy the First Ingress)
예시 Ingress 정의는 basic-ingress.yaml에서 찾을 수 있어요.
# Basic ingress for istio bookinfo demo application, which can be found in below
# https://raw.githubusercontent.com/istio/istio/release-1.11/samples/bookinfo/platform/kube/bookinfo.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: basic-ingress
namespace: default
spec:
ingressClassName: cilium
rules:
- http:
paths:
- backend:
service:
name: details
port:
number: 9080
path: /details
pathType: Prefix
- backend:
service:
name: productpage
port:
number: 9080
path: /
pathType: Prefix
$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.20.2/examples/kubernetes/servicemesh/basic-ingress.yaml
이 예제는 /details 경로에 대한 요청을 details 서비스로, /를 productpage 서비스로 라우팅해요.
서비스 목록을 보면 이 ingress에 대해 LoadBalancer 서비스가 자동으로 생성된 걸 볼 수 있어요. 클라우드 프로바이더가 외부 IP 주소를 자동으로 프로비저닝하지만, 약 30초가 걸릴 수 있어요.
# For dedicated load balancer mode
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cilium-ingress-basic-ingress LoadBalancer 10.98.169.125 10.98.169.125 80:32478/TCP 2m11s
details ClusterIP 10.102.131.226 <none> 9080/TCP 2m15s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 10m
productpage ClusterIP 10.97.231.139 <none> 9080/TCP 2m15s
ratings ClusterIP 10.108.152.42 <none> 9080/TCP 2m15s
reviews ClusterIP 10.111.145.160 <none> 9080/TCP 2m15s
# For shared load balancer mode
$ kubectl get services -n kube-system cilium-ingress
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cilium-ingress LoadBalancer 10.98.169.125 10.98.169.125 80:32690/TCP,443:31566/TCP 18m
외부 IP 주소는 Ingress에도 채워져야 해요:
$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
basic-ingress cilium * 10.98.169.125 80 97s
참고
EKS 같은 일부 프로바이더는 IP 주소 대신 완전한 도메인 이름(FQDN)을 사용해요.
Ingress가 동작하는지 확인해볼게요:
$ HTTP_INGRESS=$(kubectl get ingress basic-ingress -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ curl --fail -s http://"$HTTP_INGRESS"/details/1 | jq
{
"id": 1,
"author": "William Shakespeare",
"year": 1595,
"type": "paperback",
"pages": 200,
"publisher": "PublisherA",
"language": "English",
"ISBN-10": "1234567890",
"ISBN-13": "123-1234567890"
}
외부 잠금 정책 (External Lock-down Policy)
기본적으로 모든 외부 트래픽은 허용돼요. 외부 트래픽을 잠그기 위해 CiliumNetworkPolicy를 적용해볼게요.
apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
name: "external-lockdown"
spec:
description: "Block all the traffic originating from outside of the cluster"
endpointSelector: {}
ingress:
- fromEntities:
- cluster
$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.20.2/examples/kubernetes/servicemesh/policy/external-lockdown.yaml
이 정책을 적용하면 클러스터 외부에서 시작된 모든 요청은 403 Forbidden 응답으로 거부돼요.
$ curl --fail -v http://"$HTTP_INGRESS"/details/1
* Trying 172.18.255.194:80...
* Connected to 172.18.255.194 (172.18.255.194) port 80
> GET /details/1 HTTP/1.1
> Host: 172.18.255.194
> User-Agent: curl/8.6.0
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< content-length: 15
< content-type: text/plain
< date: Thu, 29 Feb 2024 12:59:54 GMT
< server: envoy
* The requested URL returned error: 403
* Closing connection
curl: (22) The requested URL returned error: 403
# Capture hubble flows in another terminal
$ kubectl --namespace=kube-system exec -i -t cilium-xjl4x -- hubble observe -f --identity ingress
Defaulted container "cilium-agent" out of: cilium-agent, config (init), mount-cgroup (init), apply-sysctl-overwrites (init), mount-bpf-fs (init), wait-for-node-init (init), clean-cilium-state (init), install-cni-binaries (init)
Feb 29 13:00:29.389: 172.18.0.1:53866 (ingress) -> kube-system/cilium-ingress:80 (world) http-request DROPPED (HTTP/1.1 GET http://172.18.255.194/details/1)
Feb 29 13:00:29.389: 172.18.0.1:53866 (ingress) <- kube-system/cilium-ingress:80 (world) http-response FORWARDED (HTTP/1.1 403 0ms (GET http://172.18.255.194/details/1))
클러스터 내부에서 Ingress 엔드포인트로 가는 트래픽이 여전히 허용되는지 확인해볼게요:
# The test-application.yaml contains a client pod with curl available
$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.20.2/examples/kubernetes/servicemesh/envoy/test-application.yaml
$ kubectl exec -it deployment/client -- curl -s http://$HTTP_INGRESS/details/1
{"id":1,"author":"William Shakespeare","year":1595,"type":"paperback","pages":200,"publisher":"PublisherA","language":"English","ISBN-10":"1234567890","ISBN-13":"123-1234567890"}%
또 하나 흔한 사용 사례는 특정 IP 주소 집합만 Ingress에 접근하도록 허용하는 거예요. 이는 아래 정책으로 달성할 수 있어요:
apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
name: "allow-cidr"
spec:
description: "Allow all the traffic originating from a specific CIDR"
endpointSelector:
matchExpressions:
- key: reserved:ingress
operator: Exists
ingress:
- fromCIDRSet:
# Please update the CIDR to match your environment
- cidr: 172.18.0.1/32
$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.20.2/examples/kubernetes/servicemesh/policy/allow-ingress-cidr.yaml
$ curl -s --fail http://"$HTTP_INGRESS"/details/1
{"id":1,"author":"William Shakespeare","year":1595,"type":"paperback","pages":200,"publisher":"PublisherA","language":"English","ISBN-10":"1234567890","ISBN-13":"123-1234567890"}
기본 거부(Default Deny) Ingress 정책
기본적으로 모든 트래픽을 거부하도록 CiliumClusterwideNetworkPolicy를 적용해볼게요:
---
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
name: "default-deny"
spec:
description: "Block all the traffic (except DNS) by default"
egress:
- toEndpoints:
- matchLabels:
io.kubernetes.pod.namespace: kube-system
k8s-app: kube-dns
toPorts:
- ports:
- port: '53'
protocol: UDP
rules:
dns:
- matchPattern: '*'
endpointSelector:
matchExpressions:
- key: io.kubernetes.pod.namespace
operator: NotIn
values:
- kube-system
$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.20.2/examples/kubernetes/servicemesh/policy/default-deny.yaml
이 정책을 적용하면 /details 엔드포인트에 대한 요청은 외부 및 클러스터 내 트래픽 모두에서 거부돼요.
$ curl --fail -v http://"$HTTP_INGRESS"/details/1
* Trying 172.19.255.194:80...
* Connected to 172.19.255.194 (172.19.255.194) port 80
> GET /details/1 HTTP/1.1
> Host: 172.19.255.194
> User-Agent: curl/8.6.0
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< content-length: 15
< content-type: text/plain
< date: Sun, 17 Mar 2024 13:52:38 GMT
< server: envoy
* The requested URL returned error: 403
* Closing connection
curl: (22) The requested URL returned error: 403
# Capture hubble flows in another terminal
$ kubectl --namespace=kube-system exec -i -t cilium-xjl4x -- hubble observe -f --identity ingress
Defaulted container "cilium-agent" out of: cilium-agent, config (init), mount-cgroup (init), apply-sysctl-overwrites (init), mount-bpf-fs (init), wait-for-node-init (init), clean-cilium-state (init), install-cni-binaries (init)
Mar 17 13:56:00.709: 172.19.0.1:34104 (ingress) -> default/cilium-ingress-basic-ingress:80 (world) http-request DROPPED (HTTP/1.1 GET http://172.19.255.194/details/1)
Mar 17 13:56:00.709: 172.19.0.1:34104 (ingress) <- default/cilium-ingress-basic-ingress:80 (world) http-response FORWARDED (HTTP/1.1 403 0ms (GET http://172.19.255.194/details/1))
이제 같은 엔드포인트로 가는 클러스터 내 트래픽도 거부되는지 확인해볼게요:
# The test-application.yaml contains a client pod with curl available
$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.20.2/examples/kubernetes/servicemesh/envoy/test-application.yaml
$ kubectl exec -it deployment/client -- curl -s http://$HTTP_INGRESS/details/1
Access denied
다음 단계는 /details 엔드포인트로 가는 ingress 트래픽을 허용하는 거예요:
---
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
name: allow-ingress-egress
spec:
description: "Allow all the egress traffic from reserved ingress identity to any endpoints in the cluster"
endpointSelector:
matchExpressions:
- key: reserved:ingress
operator: Exists
egress:
- toEntities:
- cluster
$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.20.2/examples/kubernetes/servicemesh/policy/allow-ingress-cluster.yaml
$ curl -s --fail http://"$HTTP_INGRESS"/details/1
{"id":1,"author":"William Shakespeare","year":1595,"type":"paperback","pages":200,"publisher":"PublisherA","language":"English","ISBN-10":"1234567890","ISBN-13":"123-1234567890"}
$ kubectl exec -it deployment/client -- curl -s http://$HTTP_INGRESS/details/1
{"id":1,"author":"William Shakespeare","year":1595,"type":"paperback","pages":200,"publisher":"PublisherA","language":"English","ISBN-10":"1234567890","ISBN-13":"123-1234567890"}
reserved:ingress를 선택하고 특정 아이덴티티로의 egress를 허용하는 NetworkPolicy도 사용할 수 있어요. 하지만 일반적으로, Cilium Ingress가 네트워킹 인프라의 일부라는 점을 고려하면 reserved:ingress 아이덴티티에서 모든 cluster 아이덴티티로의 모든 트래픽을 허용하는 것이 더 안정적일 가능성이 높아요.
더 알아보기 (Learn more)
- HTTP Ingress 예제 — HTTP Ingress 예제
- External Lock-down Policy — 외부 트래픽 잠금 정책
- Cilium Ingress — Cilium Ingress 컨트롤러