Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

멀티클라우드 Service Mesh

Kubernetes Korea User Group에서 주최한 Meetup 발표자료 입니다. (2019. 03. 07)

  • Login to see the comments

멀티클라우드 Service Mesh

  1. 1. 멀티클라우드 Service Mesh
  2. 2. 2 Contents 멀티클라우드 Istio Service Mesh Demo : Traffic Routing Multi-Cloud Service Mesh Multi-Cloud Service Mesh To-Be Model
  3. 3. 3 멀티클라우드란? § 2개 이상의 Cloud Provider가 제공하는 Public 또는 Private(On-Premise) 환경에 구성된 클라우드 Public Cloud1 Public Cloud2 Private Cloud On-Premise § 고려사항 • Workload • 적용기술 • 네트워크 § 구성 기준 • 접근성 • 확장성 • 안정성
  4. 4. 4 멀티클라우드 대표 모델 Hybrid Cloud Extend Cloud HA Cloud Bursting Cloud Workload2Workload1 Public Cloud Workload4Workload3 Private Cloud (On-Premise) Workload2Workload1 Main Cloud Workload4Workload3 Expend Cloud Workload2Workload1 Active Cloud Workload1Workload1 Standby Cloud Workload1Workload1 Workload1Workload1 Failover Workload2Workload1 Main Cloud Bursting Cloud Workload2+Workload1+ Bursting ExtendConnect
  5. 5. 5 Hybrid Cloud § Private Cloud(On-Premise)와 Public Cloud간 연결하여 Workload를 통합하는 클라우드 방식 Workload 2Workload 1 Public Cloud Workload 4Workload 3 Private Cloud (On-Premise) • 접근성 용이 (Internet 연결) • 다중 지역 배포 및 자동 확장 • On-Demand 형태의 다양한 서비스 제공 • Compliance 및 보안이 중요한 데이터 관리 • 인프라 및 H/W의 저수준 레벨까지 제어 가능 • 초기 도입 비용 지불후 Pay for use 없이 자원 제공 Connect
  6. 6. 6 Extend Cloud Workload2Workload1 Main Cloud Workload4Workload3 Expend Cloud § Main Cloud의 자원 및 Workload를 Extend Cloud를 통해 확장하는 클라우드 구성 방식 • 추가 Workload 확장 • Expend Cloud내 특화 서비스 사용가능 Extend • Workload 자원 분할 • 최적의 서비스 배포장소 선택 가능 • 이종 클라우드 상품 조합에 따른 비용 감소
  7. 7. 7 High Availability Cloud Workload2Workload1 Active Cloud Workload4Workload3 Standby Cloud Workload2Workload1 Workload4Workload3 Failover § 전체 Workload를 동일하게 구성하여 장애 발생시 서비스 무중단 혹은 빠른 복구를 제공하는 클라우드 방식 • 서비스 무중단으로 제공 가능 (Datacenter/Zone/Region 규모의 장애 발생) • 데이터는 항시 백업 혹은 복제로 RPO(복구시점목표) 최소화 • Standby Cloud 지점에 필수적인 Workload를 일부 기동하여 RTO(복구시간목표) 최소화 • Infrastructure as Code 기반 복구 자동화 (RTO 최소화)
  8. 8. 8 Bursting Cloud Workload2Workload1 Main Cloud Bursting Cloud Workload2+Workload1+ Bursting § 특정 시점에 자원 부족시 자동으로 Scale-out 하여 일시적으로 많은 Workload를 수용할 수있는 환경 제공 • 일시적인 동적 서비스 대상 Workload 자원 확장 • 여유자원 프로비저닝 및 유지 불필요 • 초과용량 기준 지정 불필요 • Infrastructure as Code 기반 자원 확장 자동화
  9. 9. 9 Service Mesh Service1 POD Container Sidecar Proxy Service2 POD Container Sidecar Proxy Service3 POD Container Sidecar Proxy § Microservice간 통신 및 트래픽을 제어하는 Service Network Layer • Microservice 간 통신이 서비스 Network 계층의 Proxy를 통해 수행 • 개별 Proxy는 서비스 내부가 아니라 서비스와 분리된 Sidecar 형태로 실행 • Sidecar Proxy들 간의 Mesh Network를 형성 • Sidecar Proxy는 연결된 Container 대신 타 서비스로의 통신 및 트래픽 제어 Service4 POD Container Sidecar Proxy Service5 POD Container Sidecar Proxy
  10. 10. 10 Istio TelemetryTraffic Management Security • mTLS (양방향 트래픽 암호화) • Auth (서비스 접근 인증) • RBAC (권한 제어) – Namespace – Service – Method • 서비스 Discovery • Load Balancing • 트래픽 라우팅 • HTTP/gRPC/TCP 트래픽 처리 • Policy 설정 • Fault Injection • Failure Handling • Monitoring (Metrics 기반) • Logging • 분산 Tracing • Service Topology 제공 • Telemetry 항목 시각화 § Service Mesh 구성 및 관리 플랫폼
  11. 11. 11 Istio 컴포넌트 § Istio Data plane : Service Mesh 구성 § Envoy : 서비스간 통신 및 트래픽 제어 Kubernetes Istio Control Plane Istio Data Plane Pilot Mixer Citadel Service1 POD Container Proxy Service2 POD Proxy Container • Istio Control plane : Service Mesh Controller 구성 • Pilot : 서비스 및 트래픽 관리, Policy 적용 • Mixer : Telemetry 연동, Policy 체크 • Citadel : 보안 관리 및 트래픽 mTLS 적용
  12. 12. 12 Service Mesh 구성용 오픈소스 § Hipster Shop Microservice Demo (https://github.com/GoogleCloudPlatform/microservices-demo)
  13. 13. 13 Service Mesh Architecture On-Premise Kubernetes Users Internet Load Balancer Istio Control Plane Istio Data Plane Pilot Mixer Citadel Recommendation Frontend LoadGenerator Advertise ProductCatalog EmailCurrency Payment Shipping Redis-Cart Cart Checkout Ingress Gateway
  14. 14. 14 Istio 트래픽 관리 정책 - 전체 Users Internet kind: Gateway meta: name: frontend-gateway spec: selector: istio: ingressgateway servers: - port: number: 80 name: http protocol: HTTP hosts: - "*" kind: ServiceEntry spec: hosts: - "metadata.google” ports: - number: 80 protocol: HTTP name: http kind: DestinationRule spec: host: productcatalogservice subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2 kind: VirtualService spec: hosts: - productcatalogservice http: - route: - destination: host: productcatalogservice subset: v1 weight: 25 - destination: host: productcatalogservice subset: v2 weight: 75 kind: VirtualService spec: hosts: - "*" gateways: - frontend-gateway http: - route: - destination: host: frontend port: number: 80 Ingress Gateway frontend Container Sidecar Proxy Productcatalogservice-v1 Container Sidecar Proxy Productcatalogservice-v2 Container Sidecar Proxy Egress Gateway Internet GCP Meta Server Istio Data Plane
  15. 15. 15 Istio 트래픽 관리 정책 - Gateway Users Internet Ingress Gateway frontend Container Sidecar Proxy Productcatalogservice-v1 Container Sidecar Proxy Productcatalogservice-v2 Container Sidecar Proxy Istio Data Plane § Ingress Gateway에 적용 § Service Mesh 외부 수신 트래픽 활성화 (기본적으로 Istio는 모든 외부 트래픽을 차단) § Edge Load Balancer 연결 § 활성화 할 프로토콜, 포트, 호스트(서비스) 설정 (HTTP 80, 전체 서비스 대상) Egress Gateway Internet GCP Meta Server apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: frontend-gateway spec: selector: istio: ingressgateway servers: - port: number: 80 name: http protocol: HTTP hosts: - "*"
  16. 16. 16 Istio 트래픽 관리 정책 – VirtualService Ingress Gateway frontend Container Sidecar Proxy Productcatalogservice-v1 Container Sidecar Proxy Productcatalogservice-v2 Container Sidecar Proxy Egress Gateway Istio Data Plane apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: productcatalogservice spec: hosts: - productcatalogservice http: - route: - destination: host: productcatalogservice subset: v1 weight: 50 - destination: host: productcatalogservice subset: v2 weight: 50 Users Internet GCP Meta Server Internet § Sidecar Proxy에 적용 § 라우팅 제어규칙 설정 § Gateway 정책과 맵핑하여 라우팅 활성화 (frontend 서비스) § 트래픽 Weight(가중치 비율) 적용 (productcatalog 서비스) apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: frontend-ingress spec: hosts: - "*" gateways: - frontend-gateway http: - route: - destination: host: frontend port: number: 80
  17. 17. 17 Istio 트래픽 관리 정책 - DestinationRule Users Internet Ingress Gateway frontend Container Sidecar Proxy Productcatalogservice-v1 Container Sidecar Proxy Productcatalogservice-v2 Container Sidecar Proxy Egress Gateway Istio Data Plane apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: productcatalogservice spec: host: productcatalogservice subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2 Internet GCP Meta Server § Sidecar Proxy에 적용 § 라우팅 발생후 적용할 정책 설정 § VirtualService 정책과 맵핑하여 라우팅 구현 § Label별 Subset(부분집합) 적용 (productcatalog v1, v2)
  18. 18. 18 Istio 트래픽 관리 정책 - ServiceEntry Users Internet Ingress Gateway frontend Container Sidecar Proxy Productcatalogservice-v1 Container Sidecar Proxy Productcatalogservice-v2 Container Sidecar Proxy Egress Gateway Internet GCP Meta Server Istio Data Plane apiVersion: networking.istio.io/v1alpha3 kind: ServiceEntry metadata: name: whitelist-egress-googleapis spec: hosts: - "169.254.169.254" # GCE metadata server - "metadata.google" # GCE metadata server - "metadata.google.internal" # GCE metadata server - "accounts.google.com" # Used to get token - "*.googleapis.com" ports: - number: 80 protocol: HTTP name: http - number: 443 protocol: HTTPS name: https § Egress Gateway에 적용 § 외부 요청을 하기위한 트래픽을 활성화 (기본적으로 Istio는 모든 외부 트래픽을 차단) § 활성화 할 프로토콜, 포트, 호스트(서비스) 설정 (GCP Meta Svr, HTTP:80, HTTPS:443)
  19. 19. 19 Multi-Cloud + Service Mesh 통합 구조 Public Cloud (IaaS – AWS, Azure, Cloud Z) Service Mesh (Istio) Private Cloud (IaaS – VMware, Openstack) On-Premise (IaaS – Bare-metal) Container Orchestration (CaaS - Kubernetes) Common tool chain (CI/CD, Provisioning, Automation, Registry, Telemetry, Policy, IAM, Security) Container Orchestration (CaaS - Kubernetes) Container Orchestration (CaaS - Kubernetes) Application1 Application2 Application3 Application4 Application 5 Application6 § Multi-Cloud 내 Service Mesh 방식으로 안정적인 서비스 통합운영 및 확장 가능한 구조
  20. 20. 20 Multi-Cloud + Service Mesh 통합을 위한 요소 1. CI/CD 기반 Code Pipeline을 통해 빠른 서비스 제공 및 확장 • Github • Jenkins • Spinnaker 2. 서비스간 통신 및 트래픽 관리를 통한 대규모의 Microservice 운영/관리 • Envoy • Istio 3. Application을 Container화 하여 Microservice 단위 Workload 일관성 유지 • Docker • Kubernetes 4. 자동화 Provisioning시스템을 활용하여 빠른 환경구성 및 Workload 자원 동적 확장 • Terraform • Packer • Ansible
  21. 21. 21 Multi-Cloud Service Mesh Architecture (Extend Cloud) On-Premise Kubernetes Users Internet Load Balancer Istio Control Plane Istio Data Plane Pilot Mixer Citadel Recommendation LoadGenerator Advertise ProductCatalog-v1 PaymentCart Ingress Gateway Cloud Z Redis-Cart Currency ShippingEmail ProductCatalog-v2 Kubernetes Istio Data Plane Internet VPN VPN Strong swan Strong swan Checkout Frontend Subnet :10.0.0.0/16, 10.1.0.0/16 172.30.0.0/16, 172.21.0.0/16 Subnet : 172.17.0.0/16 Subnet : 169.56.94.32/27
  22. 22. 22 Multi-Cloud Service Mesh 구성/설치 절차 On-Premise Cloud Z Kubernetes – Main Cluster Kubernetes – Remote Cluster 8. Istio 설치 9. 설치가 완료되면 Control Plane 컴포넌트 및 Ingress/Egress Gateway가 istio-system namespace에 생성 10. Pilot, Policy, Telemetry POD IP 복사 11. Helm으로 Istio-Remote 설치 § Setting 옵션값중 Pilot, Policy, Telemetry POD IP 추가 12. 설치가 완료되면 Sidecar-Injector 및 Citadel 컴포넌트가 istio-system namespace에 생성 13. K8s Remote Cluster의 Context config 파일 생성 14. Context config 세팅용 파일 생성 15. Namespace Sidecar Injection Label 추가 § kubectl label ns default istio- injection=enabled 16. Context config 파일 및 세팅용 파일을 K8s Main Cluster로 복사 17. Context config 세팅용 파일을 환경변수로 등록 18. K8s Remote Cluster 연결용 Secret 파일 생성 19. Namespace Sidecar Injection Label 추가 § kubectl label ns default istio- injection=enabled 4. Helm Strongswan value.yaml 설정 § K8s Remote Cluster에서 복사한 Load balancer와 맵핑된 External-IP를 설정 5. Helm으로 Strongswan 설치 6. ipsec 상태 확인 및 ping 체크 1. Helm Strongswan value.yaml 설정 2. Helm으로 Strongswan 설치 3. Strongswan의 Service 항목중 Load balancer와 맵핑된 External-IP 복사 7. ipsec 상태 확인 및 ping 체크
  23. 23. 23 Multi-Cloud Service Mesh Routing Flow 1 On-Premise Kubernetes Users Internet Load Balancer Istio Control Plane Istio Data Plane Pilot Mixer Citadel Cloud Z Kubernetes Istio Data Plane Internet VPN VPN Strong swan Strong swan productcatalogservice.default.svc.cluster.local Ingress Gateway frontend Service Proxy productCatalog-v1 10.0.61.63:3550 productCatalog-v2 172.21.192.101:3550 Service Proxy Service Proxy
  24. 24. 24 Multi-Cloud Service Mesh Routing Flow 2 On-Premise Kubernetes Load Balancer Istio Control Plane Istio Data Plane Pilot Mixer Citadel Cloud Z Kubernetes Istio Data Plane Internet VPN VPN Strong swan Strong swan Ingress Gateway frontend Service Proxy productCatalog-v1 10.0.61.63:3550 productCatalog-v2 172.21.192.101:3550 Users Internet On-Premise K8s Cluster • Service Endpoint - 10.0.61.63:3550 • Service Domain Name productcatalogservice.default.svc.cluster.local productcatalogservice.default productcatalogservice Cloud Z K8s Cluster • Service Endpoint - 172.21.192.101:3550 • Service Domain Name productcatalogservice.default.svc.cluster.local productcatalogservice.default productcatalogservice Service Proxy Service Proxy
  25. 25. 25 Multi-Cloud Service Mesh Routing Flow 3 On-Premise Kubernetes Users Internet Load Balancer Istio Control Plane Istio Data Plane Pilot Mixer Citadel Cloud Z Kubernetes Istio Data Plane Internet VPN VPN Strong swan Strong swan Ingress Gateway frontend Service Proxy productCatalog-v1 10.0.61.63:3550 productCatalog-v2 172.21.192.101:3550 productcatalogservice productcatalogservice.default productcatalogservice.default.svc.cluster.local { 10.0.61.63:3550 172.21.192.101:3550 } Service Proxy Service Proxy
  26. 26. 26 Demo : Traffic Routing On-Premise Kubernetes Users Internet Load Balancer Istio Control Plane Istio Data Plane Pilot Mixer Citadel Cloud Z Kubernetes Istio Data Plane Internet VPN VPN Strong swan Strong swan Ingress Gateway frontend Service Proxy productCatalog-v1 productCatalog-v2 Service Proxy Service Proxy apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: productcatalogservice spec: host: productcatalogservice subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2 apiVersion: extensions/v1beta1 kind: Deployment metadata: name: productcatalogservice-v1 spec: template: metadata: labels: app: productcatalogservice version: v1 spec: containers: - name: server apiVersion: extensions/v1beta1 kind: Deployment metadata: name: productcatalogservice-v2 spec: template: metadata: labels: app: productcatalogservice version: v2 spec: containers: - name: server
  27. 27. 27 Demo : Traffic Routing (100:0) On-Premise Kubernetes Users Internet Load Balancer Istio Control Plane Istio Data Plane Pilot Mixer Citadel Cloud Z Kubernetes Istio Data Plane Internet VPN VPN Strong swan Strong swan Ingress Gateway frontend Service Proxy productCatalog-v1 productCatalog-v2 Service Proxy Service Proxy apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: productcatalogservice spec: hosts: - productcatalogservice http: - route: - destination: host: productcatalogservice subset: v1
  28. 28. 28 Demo : Traffic Routing (50:50) On-Premise Kubernetes Users Internet Load Balancer Istio Control Plane Istio Data Plane Pilot Mixer Citadel Cloud Z Kubernetes Istio Data Plane Internet VPN VPN Strong swan Strong swan Ingress Gateway frontend Service Proxy productCatalog-v1 productCatalog-v2 Service Proxy Service Proxy apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: productcatalogservice spec: hosts: - productcatalogservice http: - route: - destination: host: productcatalogservice subset: v1 weight: 50 - destination: host: productcatalogservice subset: v2 weight: 50
  29. 29. 29 Demo : Traffic Routing (0:100) On-Premise Kubernetes Users Internet Load Balancer Istio Control Plane Istio Data Plane Pilot Mixer Citadel Cloud Z Kubernetes Istio Data Plane Internet VPN VPN Strong swan Strong swan Ingress Gateway frontend Service Proxy productCatalog-v1 productCatalog-v2 Service Proxy Service Proxy apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: productcatalogservice spec: hosts: - productcatalogservice http: - route: - destination: host: productcatalogservice subset: v2
  30. 30. 30 Multi-Cloud Service Mesh To-Be Model AWS Cloud Cloud Z Azure Cloud DNS or Global Server Load Balancer VPN Gateway Elastic Kubernetes Service Azure Kubernetes Service Kubernetes Kubernetes Istio Control Plane Istio Control Plane Istio Data Plane Istio Data Plane Istio Data Plane Istio Data Plane Pilot Mixer Citadel Recommendation ProductCatalogFrontend CheckoutAdvertiseEmail Ingress Gateway Load Balancer CartRedis-Cart CurrencyPayment Shipping AI Service Blockchain Bigdata,ML,DL Service Serverless VPN VPN Direct Connect Storage Gateway VPN Object StgVolumeRepositoryMonitoring TracingLogging Registry Volume RDB NoSQL CI CDServerless Github Watson Ethereum Tensorflow Load Balancer Recommendation ProductCatalogFrontend Checkout Advertise Email CartRedis-Cart Currency Payment Shipping Telemetries Registries Storages Databases Code Pipelines Ingress Gateway Egress Gateway Egress Gateway Egress Gateway Pilot Mixer Citadel VPN Egress Gateway Volume 1. Hybrid Service Mesh • AWS Cloud : Public Cloud • On-Premise : Private Cloud 2. HA Service Mesh • Hybrid Cloud : Active Cloud • Azure Cloud : Standby Cloud 3. Extend Service Mesh • Hybrid Cloud : Main Cloud • Cloud Z : Extend Cloud VolumeRDBNoSQLLDAPPolicy MgmtCluster Meta Managements Databases Gitlab On-Premise
  31. 31. Q & A
  32. 32. End Of Document 나정호 (skwjdgh1@gmail.com)

×