Best practices to use aws in countryside.
•Download as PPTX, PDF•
12 likes•1,457 views
April 19 AWS SUMMIT Seoul 2017 Day1 "Best practices to use AWS in countryside" Heptagon inc. / JAWS-UG Takuya Tachibana
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Best practices to use aws in countryside.
Similar to Best practices to use aws in countryside. (20)
More from Takuya Tachibana
More from Takuya Tachibana (20)
Recently uploaded
Recently uploaded (20)
Best practices to use aws in countryside.
- 1. Best practices to use AWS in countryside 일본 시골 개발자의 AWS 활용기 heptagon inc. / JAWS-UG Takuya Tachibana
- 2. Profile Takuya Tachibana (32) Live in Misawa / Aomori Leader of JAWS-UG in 2016 Heptagon inc. CEO / Founder facebook.com / biatunky Favorite AWS Service： Amazon Polly 2
- 4. Profile Takuya Tachibana (32) Live in Misawa / Aomori Leader of JAWS-UG in 2016 Heptagon inc. CEO facebook.com / biatunky Favorite AWS Service： Amazon Polly Favorite Korean Food： Samgyeopsal 4
- 5. Today’s Agenda 1 - Introduction of JAWS-UG (Japan AWS User group) 2- The use of AWS in the countryside 5
- 6. 6
- 7. AWS User Group Japan 7
- 8. Activities JAWS-UG is active in the branch units. Branches are organized by location or specialized field. Each branches plan the event for user living in these those area. We have more 200 events and meetup in a year. 8
- 9. More than 60 branches in Japan. Specialized field branches 9
- 10. 10
- 12. 12
- 13. 13
- 14. 14
- 15. 15
- 16. 16
- 17. 17
- 18. Events We have three biggest events in a year. All branches join forces to prepare. 1 - JAWS DAYS 2 - JAWS FESTA 3 - AWS Summit Exhibition Booth & Night Event 18
- 19. The biggest Event for JAWS-UG. This is taken in Tokyo . Event Cost : AWS & JAWS-UG Event planning : JAWS-UG 19
- 20. 20
- 21. Take turns holding. It does all of the work only JAWS-UG. Event Cost : JAWS-UG Event planning : JAWS-UG
- 22. JAWS-UG puts out the exhibition booth and Plans After Party of AWS Summit Tokyo. 22
- 24. Features - CDP(Cloud Design Pattern) AWS Cloud Design Patterns (CDP) are generally repeatable solutions to commonly occurring problems in cloud architecting. Clone Server PatternScale Up Pattern 24
- 27. 27
- 28. 28
- 29. Features
- 30. AWS Samurai Seiji Akatsuka is participating in AWS Summit Seoul. He contributed to the exchange of user groups across the country. Let’s talk to him :) 30
- 31. Features • LT ( Lightning Talk) 31
- 32. Features - We are Challenging!! Some members make self made Amazon Echo with Raspberry Pi. CLI branch held CLI-only conference at first time in the world.
- 33. Features - Blog “This event does not end until you write blog.” Amount of Japanese information is reason why AWS and JAWS is popular in Japan. 33
- 34. Features Web Media - JAWS-UG on ASCII -
- 35. Best practices to use AWS in countryside 일본 시골 개발자의 AWS 활용기 image:http://free-photos.gatag.net/2014/02/02/220000.html
- 36. Aomori 2.5H
- 37. Aomori
- 38. Aomori
- 41. Good place but very countryside… The farm just behind the headquarters of my company
- 42. As our company is in the countryside, we often conduct countryside projects which is small and medium . 42
- 43. What is countryside projects… - Less Less Traffic… - Small Small budget… 43
- 44. What is countryside projects… - Less Less Traffic… - Small Small budget… - Do not stop the server always !! - Need backup!!! - Need security!!!! 44
- 45. Why do we choose AWS at countryside projects? - Stability of infrastructure - Safe security - Reliable support - Scalable infrastructure - Use a variety of functions in options (Backup, Redundancy,Security…) 45
- 46. Key points 1. Use t2.family in production 2. Resources offload from EC2 3. Use of Pay-per-use services 46
- 47. Key points 1. Use t2.family in production 2. Resources offload from EC2 3. Use of Pay-per-use services 47
- 48. Feature of t2.family Cheap Cost-effectiveness is excellent . Fast（but…） During you have CPU credits it operates at a high speed . If you use all of CPU credits, very very slow… :( Amazon EC2 t2 48
- 49. How much can withstand the access ? I created environment 100 domains in t2micro instance. AmazonLinux/apache/postfix/dovecot/mysql (Optimized parameters) - WEB Wordpress:50 domains staticsite:50 domains installed Simple content. - Mail 1account / domains 100 account(domains) in total 49
- 50. Conditions of stress test Verification is successful unless reduce CPU credit. 50
- 52. Supplement In peacetime you earn credits. So It is possible to burst. You must do monitoring CPU Credits!! It is recommended stress test before use . If the credit is not enough, you should change instance size or family. In fact we provide similar environments to our customers as well.
- 53. Key points 1. Use t2.family in production 2. Resources offload from EC2 3. Use of Pay-per-use services 53
- 54. case 1) Input Cloudwatch metrics to Zabbix Issue When we monitor AWS with zabbix, we get the resource status of AWS from cloudwatch via AWS-SDK on Zabbix. Execution of AWS-SDK need high CPU power. If we are using the t2 instance, the CPU credit will decrease. a
- 55. case 1) Input Cloudwatch metrics to zabbix - Execute sdk with lambda. - Zabbix get Cloudwatch metrics via APIGateway. 55
- 56. CPU load average Switch to lambda case 1) Input Cloudwatch metrics to zabbix CPU utilization - The CPU utilization became less than half. - Lambda's usage fee is within the free tiar. - Successful CPU offload with lambda
- 57. case 2) Distribution of WEB content with spikes Issue Together with the release of new products, we have built a website on AWS. There is no big budget. But it is covered by media so it is necessary to prepare an environment that can withstand a lot of access. 57
- 58. Cloudflare We adopted CloudFlare. - Free CDN - Free custom domain and ssl - Support HTTP2 - Provide WAF(Web Application Firewall) 58
- 59. - The content of the website was mostly static content such as html, css, image andmovie. - Therefore, it responded on the CDN side without access to the origin. case 2) Distribution of WEB content with spikes Cloudflare 59
- 60. - We were able to deliver stably and speedy content even during peak times. - There was almost no charge for traffic. case 2) Distribution of WEB content with spikes Cloudflare 60
- 61. Key points 1. Use t2.family in production 2. Resources offload from EC2 3. Use of Pay-per-use services ※Pay-Per-Use service is used here in the sense that service charged only when processing. ex)Lambda,SES,S3,Spot instance 61
- 62. case 3) Statistical work with Lambda & S3 Issue - They request to automate statistical work that had been done by hand until then. - Since it is not always work, they hoped for a system that would not generate fixed costs as much as possible. 62
- 63. case 3) Statistical processing with Lambda & S3 1. Upload master data 2. Statistical processing on Lambda 3. Generate the results file to S3. 4. Notify to user with SNS 5. Download the results file
- 64. case 3) Statistical processing with Lambda & S3 - Easy operation thanks to full-managed service - Provide a complete pay-per-use - Auto scale out even if data increase 64
- 65. case 4) Notification system of AWS maintenance Issue - If you manage a large number of AWS accounts, you need a mechanism to notify events such as ec2 and RDS. - You will miss it if only email. - Since EC2 + cron generates fixed costs. 65
- 66. case 4) Notification system of AWS maintenance 66
- 67. - Invoke Lambda with Cloud Watch Events. - Lambda receive each access key/secret key from KMS. - Invoke Lambda to check the event with key. - If there is a maintenance event, it notifies slack. case 4) Notification system of AWS maintenance 67
- 68. - We have constructed a mechanism that can notify you with zero running cost. - It is possible periodic reminds, so we never forget maintenance. case 4) Notification system of AWS maintenance 68
- 69. case 5) Big Data analysis in countryside Issue - Local governments want to analyze and visualize tourism marketing to attract foreign tourists. - However, asking a large company will result in over budget. Misawa City 69
- 70. case 5) Big Data analysis in countryside - Analysis photos that tourists to Japan posted on social media by machine learning service. - Analyzed interests and trends by country from wikipedia access log and open data. - Persistent data was downloaded from Data Lake and saved offline . - Delete the AWS environment asap. 70
- 72. case 5) Big Data analysis in countryside - We completed building the infrastructure, data collection, data processing, data aggregation and analysis in only two weeks. - We could realized it with cost of several hundred dollars. (it may cost several millions if it is the way so far) - Local governments projects were implemented by local companies not large national companies.
- 73. Conclusion - There is an appropriate AWS usage in countryside. - T2.family which you can use the resources at low cost is a strong ally in countryside ! - By combining pay-per-use services, we can build inexpensive and reliable service ! 73
- 74. Conclusion - It is very important for local to solve local issues by local companies in every way. - We made it possible by using “Super Power” of AWS. - Please be gentle in the country people. 74
- 75. Thank you for your kind attention. image:http://free-photos.gatag.net/2014/02/02/220000.html 75
Editor's Notes
- Hello everyone. Please forgive me my lousy English. I’m happy to have this opportunity to speak to you today. I start my session.
- First of all, I am introducing myself. My name is Takuya Tachibana , leader of jaws-ug last year. My company Heptagon offers fully-managed services as AWS Partner. JAWS-UG has a culture that speaker introduce one’s Favorite AWS Service. Recently My favorite Service is Amazon Polly. Why Polly?
- Polly teach me pronunciations of English for this session. I introduce her.
- And My favorite korean food is Samgyeopsal.
- This is today’s agenda. first half, I introduce User Group in Japan. second half, I talk to use AWS in countyside
- So Let’s start introduction of JAWS-UG.
- First Origin of our name. As see you, jawsug come from “AWS User Group Japan”. We call JAWS-UG. JAWSUG is one of the most famous and the biggest IT community in japan.
- Usually JAWS-UG is active in the branch units. Branches is organized by location or specialized field. Core member of Each branches plan the event for user living in their area.
- There are more than sixty JAWS-UG branhes in japan. Marks on map of japan show branches by area. Branches In right side box is specialized field branches. Theare are many kind of specialized field branches like for biginer , for woman, for enterprize company , for conteier like a docker and for IoT and more. We have various branhes .
- Next I talk about logo of jawsug.
- Do you know JAWS? Is this movie famouse in korea?
- Logo of JAWSUG come from shark of movie. And this is mount FUJI the highest and the most beautifull mountain in japan. It is symbol of japan. Each branch can remake this logo.
- This is logo of JAWSUG tokyo branch. This is drawn sky tree which is the tallest tower in japan.
- This is logo of branch for women only, cloud girls branch. Pink shark is so cute and so strong like girls members!
- This is sapporo branch in hokkaido prefecture. Crab ,salmon, and Clock tower is local specialty of sapporo.
- This is oita’s logo. Oita have many hot springs. The shark put the towel on his head.
- This is security branch which formed recentry. The shark wear armor to protect himselves from attack. It is very cool!
- Next I talk about BIG EVENT of JawsUG. We held three times Natioal events in a year . All branch join forces to prepare. First jaws days. Second jaws festa. Third AWS summit tokyo.
- JAWS DAYS is the biggest event for jawsug. This year one thousand 3 hundret people participate in this event. We run eight concurrent sessions and totality of sesseions are about fifty. Jaws days are held at Tokyo in March. JAWSUG find sponsors for event and part of cost be borne by AWSjapan JAWSUG plan freely contents of event.
- KRUG friends come JAWS DAYS every year. They always make excellent presentations.
- JAWS festa is held in various city. We take turns at providing a event place. Tjis year it is held at Shikoku erea. It does all of the work for jawsfesta in the only JAWS-UG . It is very hard but very fun!
- Third big event is AWS Summit in Tokyo. JAWS-UG put out the exhibition booth and plan Party after AWS Summit Tokyo. this year this event will be held for four days from May 30.
- Next I introduce some features of JAWSUG. At the time of big events , We make event T-shirts like this. All of them are so cool.
- AWS Cloud Design Patterns (we call CDP) are generally repeatable solutions to commonly occurring problems in cloud architecting. Some japanses designed CDP. So It is popular to discuss and study in jawsug event.
- Is Mahjongg famous in korea? AWS Mahjongg is board game using CDP. This is very famous game in jaws-ug.. We make a pair like reality service. For example Combination of ELB,EC2 and RDS is winning hand of webservice .
- AWS karuta Karuta is one of the Japanese traditional card game. First,a person read Description of AWS service. And the others look for a card which is the icon of AWS service drawn about it. How many cards can you find? The winner is having the cards most. Like this way.
- Reader say It makes easy to set up, operate, and scale a relational database in the cloud. It is a fully-managed SQL database service.
- You shold find the card writen RDS icon.
- Contributer for JAWSUG was given the title of AWS Samurai. They are AWS samurai in this year.
- Today one of samurai Seiji Akatsuka come here. Seiji contributed to the exchange of user groups across the country. Let’s talk with him.
- Lightning Talk we call LT is very famous contents in japanese IT commuities. Lightning talk is a very short presentation only a few minutes. Many Japanese are not shy only when Lightningtalk.
- We love to challenge new things and new technorogy.. Some member make self made Amazon Echo with Raspberry Pi. We call it cheap Echo. Although Echo are not yet released in Japan, they have deep knowledge of echo. CLI branch is so uniq to treat only Comand line interface. And last year they held CLI-only conference at first time in the world.
- Whenever jawsug event, we say “This event does not end until you write blog.” So many participants write article about jawsug event. Amount of japanese information is one of reason AWS and JAWSUG is popular in japan.
- Ascii which is the most famous publishing company in Japan . They have made website about only jawsug. There are variety of information about jawsug of whole japan. They support and contribute to communty as media. introduction of jawsig will end here.
- Next I talk to use AWS in countyside.
- I live in Aomori Prefecture. Aomori is located in the north of Japan. It is 600 km away from Tokyo and it is very rural. It is 2.5 hours to Seoul by direct flight.
- Aomori is famous for apple, garlic, tuna and Nebuta Festival.
- Nature is very rich, there are ocean and mountain.
- In this season Cherry blossoms are beautiful. A lot of tourists come from Korea.
- Do you know him? Do you know PPAP? He is also from Aomori.
- Aomori is very good place. but very countyside.
- As our company is in the countryside, we have a lot of small and medium projects. We are professional of countyside project.
- What is the project in such countryside? Compared with large companies and services, budget is small and access to servers is less in general.
- However, stopping the server is not allowed. Of course, Customer says that "Please both backup and security are done completely". Even though the budget cheeper than bigger companys and project, quality of service is not different from these.
- So why we use AWS for small projects? I'm thinking about these reasons. As you know. AWS have Stable infrastructure, safe security, and reliable support. It usually has less access. But sometimes access increased if website is taken into media. For that, it need to be able to scale when necessary. And we can use various functions such as backup, redundancy and security as options on AWS.
- I would like to introduce key point in such countyside-projects. The first using t2 family in production environment. The second resources offload from EC2. The third is using of Pay-per-use services .
- I talk to using t2.family inproduction. Using t2 family is the most important point in countlyside.
- The t2 family can be used very cheaply. Cost performance is superior to other instance types. However, there is a limitation of CPU credit. If it is used too much, CPU resources are limited and the processing speed is very slow.
- Although we need attention to excessive use, We tried to know how we can use it. We prepared a hundred websites and mails in a t2.micro. WEB enviroment contain Wordpress 50 domain and staticsite 50 domaininstalled Simple content. Mail enviroment contain 1account per domains, total 100 account in ec2.
- We tried the test under such conditions. We have reproduced the access of a website with a small e-mail server. We prepared one hundred accesses and about 10 mails per day, more one hundred domains. And We actually put the load on it. If the CPU credit does not decrease, it will be able to operate even if 100 domains are packed into t2.micro.
- This is a graph of the Cloudwatch metrics during the stress test. Loading the CPU credit does not decrease. If it is a website with less access, we can operate even if packed 100 domains in one t2.micro.
- A supplement to this test. Even when there are burst accesses, you can digest the accumulated CPU credits and process it. It will not work at all once the CPU credit runs out, so you need to monitor it. And It is recommended stress test before use . When CPU credit runs out, the best practice is to spec up in t2 instance or not t2 instance . In fact we provide similar environments to our customers as well.
- second topic , I talk to resources offload from ec2 instance. Since the t2 instance has CPU credits, it is important to reduce CPU usage. From here, I will explain with some case.
- this case is using zabbix for monitoring AWS. we have a this issue. we excute AWS-SDK on Zabbix Server to get Cloudwatch metrics. But excution AWS-SDK need high cpu resource. We use t2.instance so the CPU credit will decrease.
- We rebuilt to this archtecting. Execute sdk with lambda not ec2. zabbix get Cloudwatch metrics via APIGateway. Wee offloaded cpu power from ec2 to lambda.
- This is the graph before and after the rebuilding. The CPU utilization became less than half. Lambda's usage fee is within the free tiar. We say Successful CPU offload with lambda!
- Next, traffic offload in cace of Distribution of WEB content with spikes. My customer Tess is maker of wheelchair. Together with the release of new products, we have built a website on AWS. There is no big budget. But it is covered by media like television , newspaper and web news. so it is necessary to prepare an environment that can withstand a lot of access.
- We adopted CloudFlare. CloudFlare is CDN service. They have free plan. we use it.
- We placed cloudflare front of ec2. The content of the website was mostly static content such as html, css, image andmovie. Therefore, the CDN side responces without access to the origin. AWS side is a little traffic.
- As a result We were able to deliver stably and speedy content even during peak times. There was almost no charge for traffic.
- Last point is using of Pay-per-use services. Pay-Per-Use service is used here in the sense that service charged only when processing.
- This is case of using lambda and s3 for Statistical work. My customer have this issue. They request to automate statistical processing that had been done by hand until then. Since it is not always work, they hoped for a system that would not generate fixed costs as much as possible.
- This is archtencting. User upload master data to s3. Lambda load master data from s3. Lambda performs data processing and connect with external API. Generate the results file to S3 and Notify to user SNS. User Download the results file.
- Easy operation thanks to full-managed service We provide system as a complete pay-per-use. Even if the amount of data increases, it automatically scales out. We got customers happy!
- fourth case is Notification system of AWS maintenance. If you have a large number of AWS accounts, you need a mechanism to notify events such as ec2 and RDS. You will miss it if only email. Since EC2 and Cron generates fixed costs, I do not want to use it.
- This is archtecting in this system. We use Lambda, Cloud watch event and Key Management Service(KMS).
- Invoke Lambda once a day at Cloud Watch Events periodically. Activated Lambda will receive each access key and secret key from KMS. Lambda is invoked to check for maintenance events in each aws accounts. If there are maintenance events, it will notify slack.
- Thanks to this system We have constructed a mechanism that notify someone with zero running cost. It is possible periodic reminds, so we never forget maintenance.
- Last one is case of Big Data analysis for local government in countryside. This customer is Misawa city where is my home town. Misawa city want to analyze and visualize tourism marketing to attract foreign tourists. But asking a large company will result in over budget.
- We analysis photos that tourists to Japan posted on social media by machine learning service. And analyzed interests and trends by country from Wikipedia access log and open data. Persistent data was downloaded from Data Lake and saved offline . We delete the AWS environment as soon as possible to reduce cost.
- This is archtecture. Using Lambda, DynamoDB, Spot instances,S3, finally we created a Data Lake for Redshift with more than sixty billion records. I use Cloud Vision API of Google because Amazon Rekognition don’t have some feature yet. I expect the same function will be added as soon as possible.
- We completed building the infrastructure, data collection, data processing, data aggregation and analysis in two weeks. We could realized it with cost of several hundred dollars. But it may cost several millions if it is the way so far. Local governments projects were implemented by local companies not large national companies. This is also important for the local economy.
- In summing up, there were some key points for countyside project. First I think There is an appropriate AWS usage in countryside. It is a little different from Large-scale projects. T2.instance is useful for small project to use the resources at low cost. By combining pay-per-use services, we can build inexpensive and reliable service!
- I believe It is very important for local to solve local issues by local companies in every way. Now we can do it by using Super Power of AWS. Last, I have one more important message. Please be gentle in the country people.
- Thanks for your kind attention. カムスハムニダ。