EKS-to-EKS Clustermesh 준비

EKS-to-EKS Clustermesh 준비 (EKS-to-EKS Clustermesh Preparation)

이 가이드는 clustermesh 기능 요구사항을 충족하도록 AWS EKS(Elastic Kubernetes Service) 클러스터를 설치하고 준비하는 단계별 가이드예요. 두 EKS 클러스터를 설치하고 VPC 피어링을 통해 clustermesh로 연결합니다.

출처: EKS-to-EKS Clustermesh Preparation

본문

이 가이드는 clustermesh 기능의 요구사항을 충족하도록 AWS EKS (AWS Elastic Kubernetes Service) 클러스터를 설치하고 준비하는 단계별 가이드입니다.

이 가이드에서는 두 EKS 클러스터를 설치하고 clustermesh를 통해 서로 연결합니다.

클러스터 1 설치

각 리소스 이름에 추가될 환경 변수를 만듭니다.

export NAME="$(whoami)-$RANDOM"
export AWS_REGION="eu-west-2"

VPC를 만듭니다.

Note

특정 AWS 서비스가 이 범위를 사용하므로 잠재적 문제를 방지하려면 VPC에 172.17.0.0/16 CIDR 범위를 사용하지 마세요.

Cluster_1_VPC=$(aws ec2 create-vpc \
    --cidr-block 10.0.0.0/16 \
    --tag-specifications "ResourceType=vpc,Tags=[{Key=Name,Value=Cluster_1_VPC}]" \
    --region ${AWS_REGION} \
    --query 'Vpc.{VpcId:VpcId}' \
    --output text
)

서브넷을 만듭니다.

# Create public subnets
export Cluster_1_Public_Subnet_1=$(aws ec2 create-subnet \
    --vpc-id ${Cluster_1_VPC} \
    --cidr-block 10.0.1.0/24 \
    --availability-zone ${AWS_REGION}a \
    --tag-specifications "ResourceType=subnet, Tags=[{Key=Name,Value=Cluster_1_Public_Subnet_1},{Key=kubernetes.io/role/elb,Value=1}]" \
    --query 'Subnet.{SubnetId:SubnetId}' \
    --output text
)

export Cluster_1_Public_Subnet_2=$(aws ec2 create-subnet \
    --vpc-id ${Cluster_1_VPC} \
    --cidr-block 10.0.2.0/24 \
    --availability-zone ${AWS_REGION}b \
    --tag-specifications "ResourceType=subnet, Tags=[{Key=Name,Value=Cluster_1_Public_Subnet_2},{Key=kubernetes.io/role/elb,Value=1}]" \
    --query 'Subnet.{SubnetId:SubnetId}' \
    --output text
)

# Create private subnets
export Cluster_1_Private_Subnet_1=$(aws ec2 create-subnet \
    --vpc-id ${Cluster_1_VPC} \
    --cidr-block 10.0.3.0/24 \
    --availability-zone ${AWS_REGION}a \
    --tag-specifications "ResourceType=subnet, Tags=[{Key=Name,Value=Cluster_1_Private_Subnet_1},{Key=kubernetes.io/role/internal-elb,Value=1}]" \
    --query 'Subnet.{SubnetId:SubnetId}' \
    --output text
)

export Cluster_1_Private_Subnet_2=$(aws ec2 create-subnet \
    --vpc-id ${Cluster_1_VPC} \
    --cidr-block 10.0.4.0/24 \
    --availability-zone ${AWS_REGION}b \
    --tag-specifications "ResourceType=subnet, Tags=[{Key=Name,Value=Cluster_1_Private_Subnet_2},{Key=kubernetes.io/role/internal-elb,Value=1}]" \
    --query 'Subnet.{SubnetId:SubnetId}' \
    --output text
)

인터넷 게이트웨이와 NAT를 만들고 VPC에 연결합니다.

# Create internet gateway
export Cluster_1_IGW=$(aws ec2 create-internet-gateway \
    --tag-specifications "ResourceType=internet-gateway, Tags=[{Key=Name,Value=Cluster_1_IGW}]" \
    --query 'InternetGateway.InternetGatewayId' \
    --region ${AWS_REGION} \
    --output text
)

# Attach the internet gateway to the VPC
aws ec2 attach-internet-gateway \
    --internet-gateway-id ${Cluster_1_IGW} \
    --vpc-id ${Cluster_1_VPC}

# Create NAT gateway
Cluster_1_EIP_1=$(aws ec2 allocate-address \
    --domain vpc \
     --tag-specifications "ResourceType=elastic-ip, Tags=[{Key=Name,Value=Cluster_1_EIP_1}]" \
    --query 'AllocationId' \
    --output text \
    --region ${AWS_REGION}
)

Cluster_1_EIP_2=$(aws ec2 allocate-address \
    --domain vpc \
     --tag-specifications "ResourceType=elastic-ip, Tags=[{Key=Name,Value=Cluster_1_EIP_2}]" \
    --query 'AllocationId' \
    --output text \
    --region ${AWS_REGION}
)

Cluster_1_NGW_1=$(aws ec2 create-nat-gateway \
    --subnet-id $Cluster_1_Public_Subnet_1 \
    --allocation-id ${Cluster_1_EIP_1} \
    --tag-specifications "ResourceType=natgateway, Tags=[{Key=Name,Value=Cluster_1_NGW_1}]" \
    --query 'NatGateway.{NatGatewayId:NatGatewayId}' \
    --output text
)

Cluster_1_NGW_2=$(aws ec2 create-nat-gateway \
    --subnet-id $Cluster_1_Public_Subnet_2 \
    --allocation-id ${Cluster_1_EIP_2} \
    --tag-specifications "ResourceType=natgateway, Tags=[{Key=Name,Value=Cluster_1_NGW_2}]" \
    --query 'NatGateway.{NatGatewayId:NatGatewayId}' \
    --output text
)

라우트 테이블, 라우트, 라우트 테이블 연결을 만듭니다.

# Create a public route table
export Cluster_1_Public_RT=$(aws ec2 create-route-table \
    --vpc-id ${Cluster_1_VPC} \
    --tag-specifications "ResourceType=route-table, Tags=[{Key=Name,Value=Cluster_1_Public_RT}]" \
    --query 'RouteTable.{RouteTableId:RouteTableId}' \
    --output text \
    --region ${AWS_REGION}
)

# Add a route to the internet gateway
aws ec2 create-route \
    --route-table-id ${Cluster_1_Public_RT} \
    --destination-cidr-block 0.0.0.0/0 \
    --gateway-id ${Cluster_1_IGW}

# Associate public subnets with the public route table
aws ec2 associate-route-table \
    --subnet-id ${Cluster_1_Public_Subnet_1} \
    --route-table-id ${Cluster_1_Public_RT}

aws ec2 associate-route-table \
    --subnet-id ${Cluster_1_Public_Subnet_2} \
    --route-table-id ${Cluster_1_Public_RT}

# Create private route tables
export Cluster_1_Private_RT_1=$(aws ec2 create-route-table \
    --vpc-id ${Cluster_1_VPC} \
    --tag-specifications "ResourceType=route-table, Tags=[{Key=Name,Value=Cluster_1_Private_RT_1}]" \
    --query 'RouteTable.{RouteTableId:RouteTableId}' \
    --output text \
    --region ${AWS_REGION}
)

export Cluster_1_Private_RT_2=$(aws ec2 create-route-table \
    --vpc-id ${Cluster_1_VPC} \
    --tag-specifications "ResourceType=route-table, Tags=[{Key=Name,Value=Cluster_1_Private_RT_2}]" \
    --query 'RouteTable.{RouteTableId:RouteTableId}' \
    --output text \
    --region ${AWS_REGION}
)

# Add routes to the NAT gateway
aws ec2 create-route \
    --route-table-id ${Cluster_1_Private_RT_1} \
    --destination-cidr-block 0.0.0.0/0 \
    --gateway-id ${Cluster_1_NGW_1}

aws ec2 create-route \
    --route-table-id ${Cluster_1_Private_RT_2} \
    --destination-cidr-block 0.0.0.0/0 \
    --gateway-id ${Cluster_1_NGW_2}

# Associate each private subnet with their respective private route table
aws ec2 associate-route-table \
    --subnet-id ${Cluster_1_Private_Subnet_1} \
    --route-table-id ${Cluster_1_Private_RT_1}

aws ec2 associate-route-table \
    --subnet-id ${Cluster_1_Private_Subnet_2} \
    --route-table-id ${Cluster_1_Private_RT_2}

VPC용 사용자 지정 보안 그룹을 만듭니다. EKS 클러스터와 함께 생성된 기본 보안 그룹은 컨트롤 플레인과 클러스터 내 다른 노드에서 시작되는 인그레스 트래픽만 허용합니다.

# Create a security group
export Cluster_1_SG=$(aws ec2 create-security-group \
    --group-name Cluster_1_Security_Group \
    --description "Security group for Cluster 1" \
    --vpc-id ${Cluster_1_VPC} \
    --tag-specifications "ResourceType=security-group,Tags=[{Key=Name,Value=Cluster_1_SG}]" \
    --region ${AWS_REGION} \
    --output text \
    --query 'GroupId'
)

# Add an inbound rule for all ingress traffic from the control-plane and other worker nodes within the cluster. An inbound rule for all ingress traffic from Cluster 2 will be added in the next section.
aws ec2 authorize-security-group-ingress \
    --group-id ${Cluster_1_SG} \
    --protocol all \
    --port 0 \
    --source-group ${Cluster_1_SG}\
    --region ${AWS_REGION}

이제 가상 사설 클라우드, 서브넷, NAT 게이트웨이, 인터넷 게이트웨이, 라우트 테이블이 생겼어요. CNI 없이 EKS 클러스터를 만들고 사용자 지정 VNet과 서브넷을 사용하도록 요청할 수 있습니다.

cat <<EOF >eks-cluster-1.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: ${NAME}
  region: ${AWS_REGION}
vpc:
  subnets:
    private:
      ${AWS_REGION}a:
        id: ${Cluster_1_Private_Subnet_1}
      ${AWS_REGION}b:
        id: ${Cluster_1_Private_Subnet_2}

managedNodeGroups:
- name: ng-1
    instanceType: t3.small
    securityGroups:
      attachIDs: ["${Cluster_1_SG}"]
    desiredCapacity: 2
    privateNetworking: true
    # Taint nodes so that application pods are
    # not scheduled/executed until Cilium is deployed.
    # Alternatively, see the note below.
    taints:
    - key: "node.cilium.io/agent-not-ready"
        value: "true"
        effect: "NoExecute"
EOF

eksctl create cluster -f ./eks-cluster-1.yaml

클러스터 2 설치

각 리소스 이름에 추가될 환경 변수를 만듭니다.

export NAME="$(whoami)-$RANDOM"
export AWS_REGION="eu-west-2"

VPC를 만듭니다.

Note

특정 AWS 서비스가 이 범위를 사용하므로 잠재적 문제를 방지하려면 VPC에 172.17.0.0/16 CIDR 범위를 사용하지 마세요.

Cluster_2_VPC=$(aws ec2 create-vpc \
    --cidr-block 10.1.0.0/16 \
    --tag-specifications "ResourceType=vpc,Tags=[{Key=Name,Value=Cluster_2_VPC}]" \
    --region ${AWS_REGION} \
    --query 'Vpc.{VpcId:VpcId}' \
    --output text
)

서브넷을 만듭니다.

# Create public subnets
export Cluster_2_Public_Subnet_1=$(aws ec2 create-subnet \
    --vpc-id ${Cluster_2_VPC} \
    --cidr-block 10.1.1.0/24 \
    --availability-zone ${AWS_REGION}a \
    --tag-specifications "ResourceType=subnet, Tags=[{Key=Name,Value=Cluster_2_Public_Subnet_1}]" \
    --query 'Subnet.{SubnetId:SubnetId}' \
    --output text
)

export Cluster_2_Public_Subnet_2=$(aws ec2 create-subnet \
    --vpc-id ${Cluster_2_VPC} \
    --cidr-block 10.1.2.0/24 \
    --availability-zone ${AWS_REGION}b \
    --tag-specifications "ResourceType=subnet, Tags=[{Key=Name,Value=Cluster_2_Public_Subnet_2}]" \
    --query 'Subnet.{SubnetId:SubnetId}' \
    --output text
)

# Create private subnets
export Cluster_2_Private_Subnet_1=$(aws ec2 create-subnet \
    --vpc-id ${Cluster_2_VPC} \
    --cidr-block 10.1.3.0/24 \
    --availability-zone ${AWS_REGION}a \
    --tag-specifications "ResourceType=subnet, Tags=[{Key=Name,Value=Cluster_2_Private_Subnet_1}]" \
    --query 'Subnet.{SubnetId:SubnetId}' \
    --output text
)

export Cluster_2_Private_Subnet_2=$(aws ec2 create-subnet \
    --vpc-id ${Cluster_2_VPC} \
    --cidr-block 10.1.4.0/24 \
    --availability-zone ${AWS_REGION}b \
    --tag-specifications "ResourceType=subnet, Tags=[{Key=Name,Value=Cluster_2_Private_Subnet_2}]" \
    --query 'Subnet.{SubnetId:SubnetId}' \
    --output text
)

인터넷과 NAT 게이트웨이를 만들고 VPC에 연결합니다.

# Create an internet gateway
export Cluster_2_IGW=$(aws ec2 create-internet-gateway \
    --tag-specifications "ResourceType=internet-gateway, Tags=[{Key=Name,Value=Cluster_2_IGW}]" \
    --query 'InternetGateway.InternetGatewayId' \
    --region ${AWS_REGION} \
    --output text
)

# Attach the internet gateway to the VPC
aws ec2 attach-internet-gateway \
    --internet-gateway-id ${Cluster_2_IGW} \
    --vpc-id ${Cluster_2_VPC}

# Create elastic IP addresses
Cluster_2_EIP_1=$(aws ec2 allocate-address \
    --domain vpc \
     --tag-specifications "ResourceType=elastic-ip, Tags=[{Key=Name,Value=Cluster_2_EIP_1}]" \
    --query 'AllocationId' \
    --output text \
    --region ${AWS_REGION}
)

Cluster_2_EIP_2=$(aws ec2 allocate-address \
    --domain vpc \
     --tag-specifications "ResourceType=elastic-ip, Tags=[{Key=Name,Value=Cluster_2_EIP_2}]" \
    --query 'AllocationId' \
    --output text \
    --region ${AWS_REGION}
)

# Create NAT gateways
Cluster_2_NGW_1=$(aws ec2 create-nat-gateway \
    --subnet-id ${Cluster_2_Public_Subnet_1} \
    --allocation-id ${Cluster_2_EIP_1} \
    --tag-specifications "ResourceType=natgateway, Tags=[{Key=Name,Value=Cluster_2_NGW_1}]" \
    --query 'NatGateway.{NatGatewayId:NatGatewayId}' \
    --output text
)

Cluster_2_NGW_2=$(aws ec2 create-nat-gateway \
    --subnet-id ${Cluster_2_Public_Subnet_2} \
    --allocation-id ${Cluster_2_EIP_2} \
    --tag-specifications "ResourceType=natgateway, Tags=[{Key=Name,Value=Cluster_2_NGW_2}]" \
    --query 'NatGateway.{NatGatewayId:NatGatewayId}' \
    --output text
)

라우트 테이블, 라우트, 라우트 테이블 연결을 만듭니다.

# Create a public route table
export Cluster_2_Public_RT=$(aws ec2 create-route-table \
    --vpc-id ${Cluster_2_VPC} \
    --tag-specifications "ResourceType=route-table, Tags=[{Key=Name,Value=Cluster_2_Public_RT}]" \
    --query 'RouteTable.{RouteTableId:RouteTableId}' \
    --output text \
    --region ${AWS_REGION}
)

# Add a route to the internet gateway
aws ec2 create-route \
    --route-table-id ${Cluster_2_Public_RT} \
    --destination-cidr-block 0.0.0.0/0 \
    --gateway-id ${Cluster_2_IGW}

# Associate public subnets with the public route table
aws ec2 associate-route-table \
    --subnet-id ${Cluster_2_Public_Subnet_1} \
    --route-table-id ${Cluster_2_Public_RT}

aws ec2 associate-route-table \
    --subnet-id ${Cluster_2_Public_Subnet_2} \
    --route-table-id ${Cluster_2_Public_RT}

# Create private route tables for each private subnet
export Cluster_2_Private_RT_1=$(aws ec2 create-route-table \
    --vpc-id ${Cluster_2_VPC} \
    --tag-specifications "ResourceType=route-table, Tags=[{Key=Name,Value=Cluster_2_Private_RT_1}]" \
    --query 'RouteTable.{RouteTableId:RouteTableId}' \
    --output text \
    --region ${AWS_REGION}
)

export Cluster_2_Private_RT_2=$(aws ec2 create-route-table \
    --vpc-id ${Cluster_2_VPC} \
    --tag-specifications "ResourceType=route-table, Tags=[{Key=Name,Value=Cluster_2_Private_RT_2}]" \
    --query 'RouteTable.{RouteTableId:RouteTableId}' \
    --output text \
    --region ${AWS_REGION}
)

# Add routes to the NAT gateway
aws ec2 create-route \
    --route-table-id ${Cluster_2_Private_RT_1} \
    --destination-cidr-block 0.0.0.0/0 \
    --gateway-id ${Cluster_2_NGW_1}

aws ec2 create-route \
    --route-table-id ${Cluster_2_Private_RT_2} \
    --destination-cidr-block 0.0.0.0/0 \
    --gateway-id ${Cluster_2_NGW_2}

# Associate each private subnet with their respective private route table
aws ec2 associate-route-table \
    --subnet-id ${Cluster_2_Private_Subnet_1} \
    --route-table-id ${Cluster_2_Private_RT_1}

aws ec2 associate-route-table \
    --subnet-id ${Cluster_2_Private_Subnet_2} \
    --route-table-id ${Cluster_2_Private_RT_2}

VPC용 사용자 지정 보안 그룹을 만듭니다. EKS 클러스터와 함께 생성된 기본 보안 그룹은 컨트롤 플레인과 클러스터 내 다른 노드에서 시작되는 인그레스 트래픽만 허용합니다.

# Create Security Group
export Cluster_2_SG=$(aws ec2 create-security-group \
    --group-name Cluster_2_Security_Group \
    --description "Security group for Cluster 2" \
    --tag-specifications "ResourceType=security-group,Tags=[{Key=Name,Value=Cluster_2_SG}]" \
    --vpc-id ${Cluster_2_VPC} \
    --region ${AWS_REGION} \
    --output text \
    --query 'GroupId'
)

# Add an inbound rule for all ingress traffic from the control-plane and other worker nodes within the cluster.
aws ec2 authorize-security-group-ingress \
    --group-id ${Cluster_2_SG} \
    --protocol all \
    --port 0 \
    --source-group ${Cluster_2_SG}\
    --region ${AWS_REGION}

# Add an inbound rule for all ingress traffic from Cluster 1
aws ec2 authorize-security-group-ingress \
    --group-id ${Cluster_2_SG} \
    --protocol all \
    --port 0 \
    --source-group ${Cluster_1_SG}\
    --region ${AWS_REGION}

# In Cluster 1's security group, add an inbound rule for all ingress traffic from cluster 2.
aws ec2 authorize-security-group-ingress \
    --group-id ${Cluster_1_SG} \
    --protocol all \
    --port 0 \
    --source-group ${Cluster_2_SG}\
    --region ${AWS_REGION}

이제 가상 사설 클라우드, 서브넷, NAT 게이트웨이, 인터넷 게이트웨이, 라우트 테이블이 생겼어요. CNI 없이 EKS 클러스터를 만들고 사용자 지정 VNet과 서브넷을 사용하도록 요청할 수 있습니다.

cat <<EOF >eks-cluster-2.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
name: ${NAME}
region: ${AWS_REGION}
vpc:
  subnets:
    private:
      ${AWS_REGION}a:
        id: ${Cluster_2_Private_Subnet_1}
      ${AWS_REGION}b:
        id: ${Cluster_2_Private_Subnet_2}

managedNodeGroups:
  - name: ng-2
    instanceType: t3.small
    securityGroups:
      attachIDs: [${Cluster_2_SG}]
    desiredCapacity: 2
    privateNetworking: true
    taints:
      - key: "node.cilium.io/agent-not-ready"
        value: "true"
        effect: "NoExecute"
EOF
eksctl create cluster -f ./eks-cluster-2.yaml

가상 네트워크 피어링

두 VPC 사이에 VPC 피어링을 만듭니다.

# Create VPC peering connection
export PEERING_CONNECTION_ID=$(aws ec2 create-vpc-peering-connection \
    --vpc-id ${Cluster_1_VPC} \
    --peer-vpc-id ${Cluster_2_VPC} \
    --peer-region ${AWS_REGION} \
    --output text \
    --query 'VpcPeeringConnection.VpcPeeringConnectionId'
)

# Grab the first VPC peering
export PEERING_REQUEST_ID=$(aws ec2 describe-vpc-peering-connections \
    --filters "Name=requester-vpc-info.vpc-id,Values=${Cluster_1_VPC}" \
    --query "VpcPeeringConnections[0].VpcPeeringConnectionId" \
    --output text
)

# Accept VPC peering request
aws ec2 accept-vpc-peering-connection \
    --vpc-peering-connection-id ${PEERING_REQUEST_ID} \
    --region ${AWS_REGION}

클러스터 1 VPC에서 클러스터 2 VPC로 트래픽을 전달합니다.

# Cluster 1
# Add route to Private Route Table 1
aws ec2 create-route \
    --route-table-id ${Cluster_1_Private_RT_1} \
    --destination-cidr-block 10.1.0.0/16 \
    --vpc-peering-connection-id ${PEERING_CONNECTION_ID} \
    --region ${AWS_REGION}

# Add route to Private Route Table 2
aws ec2 create-route \
    --route-table-id ${Cluster_1_Private_RT_2} \
    --destination-cidr-block 10.1.0.0/16 \
    --vpc-peering-connection-id ${PEERING_CONNECTION_ID} \
    --region ${AWS_REGION}

클러스터 2 VPC에서 클러스터 1 VPC로 트래픽을 전달합니다.

# Cluster 2
# Add route to Private Route Table 1
aws ec2 create-route \
    --route-table-id ${Cluster_2_Private_RT_1} \
    --destination-cidr-block 10.0.0.0/16 \
    --vpc-peering-connection-id ${PEERING_CONNECTION_ID} \
    --region ${AWS_REGION}

# Add route to Private Route Table 2
aws ec2 create-route \
    --route-table-id ${Cluster_2_Private_RT_2} \
    --destination-cidr-block 10.0.0.0/16 \
    --vpc-peering-connection-id ${PEERING_CONNECTION_ID} \
    --region ${AWS_REGION}

다른 클러스터의 노드는 이제 직접 통신할 수 있습니다. 모든 clustermesh 요구사항이 충족됐어요. clustermesh 활성화 지침은 Cluster Mesh 설정하기 섹션에 자세히 나와 있습니다.

더 알아보기 (Learn more)

• Cluster Mesh 설정하기
• AKS-to-AKS Clustermesh 준비
• GKE-to-GKE Clustermesh 준비