From HashiCorp Korea User Group Meetup 발표자: 김민규(데브시스터즈, 인프라 관리, https://github.com/synthdnb) 발표자: 김도윤(데브시스터즈, 플랫폼 API 서버 개발, https://github.com/solmonk) 발표내용: 팀의 규모가 커지면서 Secret 관리 문제가 조금씩 부각되었습니다. 예를 들면 코드에 커밋되거나, 구전으로 전해지는 Secret들, SSH Key Rotation 등의 문제를 처리하기 위해 많은 노력과 삽질이 필요했습니다. 저희 팀에서 Vault를 통해 이런 문제들을 어떻게 해결했는지 소개하려 합니다.

1. 20
Vault
DEVSISTERS ,

5. < 10

6. SSH Key

7. SSH Key
10100

8. ...

10. SSH Key
9100

11. Key Rotation

12. super_secret_password hyper_safe_passw0rd
old_ssh_key new_ssh_key

14. super_secret_password
?
hyper_safe_passw0rd

15. super_secret_password
Slack
hyper_safe_passw0rd

16. super_secret_password hyper_safe_passw0rd
B ( )
( )

17. super_secret_password hyper_safe_passw0rd ultra_secret_passw0rd

19. SSH Key
old_ssh_key new_ssh_key
100 .

20. Q: ?
OS (Ubuntu, Debian, Amazon Linux)
AWS
(SSH )

21. TF
6-7 3-4

22. 20 +

23. SSH Key
20+500+
AWS 5+ AWS VPC 10+

26. Vault SSH

27. SSH Certificate Authentication

28. ( ) Secret Store

29. ( ) Certificate Authority

30. ServerClient
Certificate Authority
.
Vault Auth

31. ServerClient
Certificate Authority
?
!
SSH Public Key

32. ServerClient
Certificate Authority
.
5 .
Signed
SSH Public Key

33. ServerClient
Certificate Authority
.
Signed
SSH Public Key

34. ServerClient
Certificate Authority
ca.pub
.
.
cf) public key
authorized_keys .

35. Server
Certificate Authority
^^
Signed
SSH Public Key

36. Server
Certificate Authority
.
T_T

37. • .

38. • .

39. • .

40. • AMI .
• !

42. • Vault
•
• .

43. FAQ
• Vault ?
•
• public key
• private key .

44. • HashiCorp :
• https://www.vaultproject.io/docs/secrets/ssh/signed-ssh-
certificates.html
• ..

45. Application
Secret

46. Application Database
db_password
data

47. ?

48. Password in Code

49. • Code Secret .
• Rotate
• Github ! ( )

50. Workarounds
AMI

52. Vault KV

53. Q. Vault ?

54. AWS
https://docs.aws.amazon.com/ko_kr/AWSEC2/latest/UserGuide/instance-identity-
documents.html

58. Vault

59. ?

60.
Tag / IAM Role/ Metadata .

62.
VAULT_TOKEN

63. vault_auth.sh
• Packer
• Vault Agent (3 )

64. FAQ
• PKCS ?
• Trust on first use
• login nonce

65. FAQ
• 100% ?
• 100%
• PASSWORD docker ENV ,
.
• Vault API

66. Vault

67. •
•
•
• ?

68. •
• Vault .
• .

69. •
•
• Vault

70. •
•
•

73. • https://www.vaultproject.io/docs/configuration/storage/
index.html
• MySQL
• DynamoDB
• CockroachDB
• Filesystem
• ....

75. Consul
• KV Store, Service Discovery, etc..
•

76. + ( )

79. Active-Standby
Consul Cluster
Active Standby

80. Active-Standby
Consul Cluster
Failed Standby

81. Active-Standby
Consul Cluster
Failed Active

82. Consul Cluster
Failed Active
ELB

83. Consul
• Service Discovery
• DNS
• (fabio, etc..)

84. ELB
• Health Check
•
•
• .

85. •
• https://www.vaultproject.io/guides/operations/vault-ha-
consul.html
• ...

88. End-to-End TLS
vault.example.com
01af324d.vault.example.com
Active Standby
vault.example.com
294ac34f.vault.example.com
ELB
TCP
Load Balancing

89. • ,
• Let's Encrypt!
• https://github.com/Neilpang/acme.sh
• Route53 DNS-01

90. Audit Logging
• logrotate s3
• syslog
• (ELK )

93. , ,
• .
•
• ex) GitHub , LDAP

94. Secret
•
• ) /secret/app/appname/rds
•
• !

95. • .
•
•
•

97. !!

100. • End-to-End TLS
• . .

101. Consul
•
•
•

102. •
• .

103. Case: Unseal
• Vault
• n m .
• 7 Unsealers ..!!

106. •
•
• Unseal
• Unseal

107. • Vault
•
• .

108. • PR ..

110. Audit Logging
• Vault
•

111. Audit Logging
• ? ->
• ->

112. • .. -> KV
• OTP .. -> TOTP Backend
• .. -> Transit Backend

113. •

114. •
• Web UI
• PROXYv1 / X-Forwarded-For support
• Secret Versioning
• CLI
• ...

115. • 1
• .
• !

116. •
• Consul
• Consul Template
• Terraform
• Packer
• Nomad

117. • Terraform
•
•

118. devsisters.com/jobs