楽天が挑むDevOps
•
36 likes•10,152 views
Systems Management Forum 2013 (http://ac.nikkeibp.co.jp/nc/smf2013s/) の基調講演資料です。楽天では2012年4月から社内でPaaSを展開しており、その基盤の上でさまざまなサービスが開発、運用されています。講演では、楽天PaaSでのDevOpsの実践方法について具体的に紹介しました。
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (14)
Similar to 楽天が挑むDevOps
Similar to 楽天が挑むDevOps (20)
More from Rakuten Group, Inc.
More from Rakuten Group, Inc. (20)
Recently uploaded
Recently uploaded (20)
楽天が挑むDevOps
- 1. 楽天が挑むDevOps July,29th 2013 YoheiSasaki PaaS Development& Operations Section Architecture CommitteeOffice DevelopmentUnit Rakuten,Inc.
- 2. About me • Yohei Sasaki • Lead of Rakuten Platform as a Service. • Technology Background: • IBM Tivoli / Ruby on Rails / Apache Hadoop / Apache CouchDB / Node.js / CloudFoundry / … • Job Background: • Technical Support Engineer / Application Developer / System Developer / System Administrator / Project Manager / … 万屋
- 3. About us • Rakuten, Inc. • Internet Service Company • Many businesses • 1000+ developers • 70+ teams • 1000+ projects インターネット企業
- 4. Go Global • Englishnization 英語公用語化
- 5. Agenda for Today • Why/What is RPaaS. • Practices in RPaaS. • Messages for DevOps people
- 7. Business Objectives • Expand business services more rapidly • Reduce the cost 早い + 安い + 旨い
- 8. Rakuten Platform as a Service • Everything should be automated. • Developer’s IN Rakuten should focus on business implementation in the latest technologies. • Platform as a Service infrastructure for Developers in Rakuten, Inc. • Like Google App Engine, Heroku, ..etc • Abstraction Layer of On-Premise/Off-Premise プライベート クラウド?
- 9. Traditional Service Building Steps • Too many human to human workflows • Too many approvals. • Too many meetings. • Too many documents. Request Servers Wait for hardware/VM creation Setup Operating System Install Software Setup monitoring Deploy website Check / Test Add to Load Balancer ... Developer Operators 複雑 ステークホルダー多い 申請書たくさん
- 10. Steps in RPaaS Developer rpaas push myapp Grab a coffee • Make it simple • Deploy • rpaas push myapp • URL mapping • rpaas map myapp xxx.rakuten.co.jp • Manage scaling • rpaas instances myapp +10 • Bind database • rpaas bind-service db1 myapp • Set Monitoring • rpaas monitor myapp –cpu 90 –mem 80 コマンド1発
- 11. Tricks – DevOps Layering Service Devs System Operators Focus on Stable Infrastructure Focus on Development Ask Response 依頼
- 12. Tricks – DevOps Layering RPaaS API Service Service DevOps RPaaS DevOps Focus on Infrastructure Features Focus on Business App Ask Response Consult Consult 自動化された標準 一緒に 考える
- 13. PaaS Reference: RPaaS Architecture IaaS Network Hardware Operating System PaaS Kernel Data Services Application Runtime Operation API SDK RPaaS API Service 一緒に 考える場所
- 14. Key Concepts • Standard Document => Programmable API • Standard Infrastructure => Infrastructure with Library
- 15. Key Concepts • Standard Document => Programmable Standard • Standard Infrastructure => Infrastructure with Library 標準化 道具
- 16. Key Concepts • Standard Document => Programmable Standard • Standard Infrastructure => Infrastructure with Library 標準化 道具 早い 旨い
- 17. Key Concepts • Standard Document => Programmable Standard • Standard Infrastructure => Infrastructure with Library 標準化 道具 早い 旨い安い
- 18. Service Team DevOps rpaas stats rpaas instances rpaas log-storage git push rpaas push rpaas map デプロイサイクルが全部自動化 デバッグ用オペレーションもAPIで提供 サーバーのことは気にしない
- 20. Change the business RPaaS DevOps Focus on Infrastructure Features System Operators Focus on Stable Infrastructure やり方を変える 安定だけではない 旨さを求める
- 21. Internal Competition • Objectives: • Provide multiple options for users PaaS IaaS Internal Customers Internal Market ユーザーを確保する ところからが仕事
- 22. Member Roles Definition • Objectives: • Clarify the responsibilities for each member. • A member has more than one role. Role In charge of: Release Manager • Conducting the release operations by cooperation with administrator. System Administrator • Handling Daily Operation. • Quick Fix on troubles. • Reporting system stats to Team Leader. System Developer • RPaaS functions from planning to operations. • Negotiation with Team Leader and Managers to decide the release line and scope. Team Leader • Coordination of system developers and related people • Evaluation for RPaaS. Manager • Budget Control • HR related things such as evaluation “兼任できる” ロールを明確にする
- 23. KPI Definition • Objectives: • Clarify the mission. • Evaluate the mission result. Role In charge of: System Administrator • # of alerts • Time to solve the alerts Release Manager • # of releases. • Time to release. System Developer • # of features newly implemented • # of solved issues Team Leader • # of launched business services ロールに目標をつけ、評価する
- 24. Release Note • Objectives: • Clarify impacts • Clarify who are customers of what you develop. リリースノートを 全員で最初に作る
- 25. Communication in Code • Objectives: • Do not use manual ops. • Anyone can do any ops 紙だけでコミュニケーションしない
- 26. Git-flow as Communication Protocol • Objectives: • Design & implementations in code repositories, not in meeting rooms. • Isolate each feature feature/* develop release/* master hotfix/* Administrator Release Manager Developer Admin/RM 独自のルールは極力作らない
- 27. Standard Release Procedure • Objectives: • Reduce manual operations. $ cd ~/rpaas-commander $ cap setup-summary $ cap setup-role -n $ cap setup-role $ cap service-in 手順書 = 動くコード
- 28. CI for infrastructure • Objectives: • Reduce the release tasks of which 80% are ‘task confirmation. 確認作業もコードで指示
- 29. Rolling Update • Objectives: • Easy to rollback • A/B test for infrastructure L7 Software LoadBalancer Virtual Machine (v1) Virtual Machine (v1) Virtual Machine (v1) Virtual Machine (v2) Virtual Machine (v2) Virtual Machine (v2) アップデートではなく新規構築
- 30. Eat our own dog food • Objectives: • Stands on customer side. • Understand pros and cons for our weapons. 自分の道具を自分で使う
- 31. Off topic?: Englishnization • Team members from 7 countries • Finding Specialists in the world • Source Code / Diagram > English?
- 33. Key Factors for DevOps • Everyone should have ‘developer role’. • Tool is a tool. • Everything should be automated.
- 34. Key Factors for DevOps • Everyone should have ‘administration role’. • Tool is a tool. • Everything should be automated.
- 35. Everyone should have ‘developer role’ • De-facto standard communication method. • Read source • Find a bug • Fix the bug • Create a new application. #!/usr/bin/env ruby puts “Hello Developers!” ソースコードいじりましょう
- 36. Tool is a tool • Chef does not solve the barrier between Devs and Ops. @@ -1,4 +1,4 @@ -package "apache" do +package "nginx" do action :install end I wanna use Nginx! Unacceptable. Pull Request Chef 使ってるかどうかは 問題ではない
- 37. Everything should be automated • Everything / Uncompromising • Automate after manual operations “手作業” を使っていいのは１ヶ月に１度 手順書が必要な作業を実施したら、 自動化するまでが仕事
- 38. Thank you!
- 39. And we are hiring! 楽天 PaaS開発エンジニア • Global Team • Emerging Technology • Trend words • Challenge Everyday