Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
AWS COST VISUALIZER HDE GIP - MTS JUNE 2014 Presented by Iskandar Setiadi & Saskya Irena
AWS COST VISUALIZER OVERVIEW 1. Problem Description 2. Analysis & Implementation 3. Result 4. Demonstration
ENORMOUS NUMBER OF PRODUCTS
UNDETAILED BILLING REPORT (UI)
COMPLICATED BILLING REPORT
CSV DETAILS InvoiceID,PayerAccountId,LinkedAccountId,RecordType,RecordId,ProductName,RateId,SubscriptionId,Pr "Estimated" ...
HOW MUCH IS THE EXPENSE FOR: 1. Each Customer 2. Each S3 Bucket 3. Each Infrastructure 4. Each Project (Tapirus, Nobita) 5...
SOLUTION CREATE YOUR OWN TOOL!
IMPLEMENTATION ENVIRONMENT 1. Python 2.6.9 2. Flask Framework & Redis for Database 3. Linux/UNIX based Operating System 4....
FIELDS OF DATA (CUSTOMER / CI / PROJECT) 1. Company Name 2. EC2 Instance Reserved 3. EC2 Instance Non-Reserved 4. EC2 EBS ...
FIELDS OF DATA (S3) 1. Bucket Name 2. Total Cost
FIELDS OF DATA (NON-CUSTOMER) 1. Product Name (SQS, etc) 2. Total Cost
PARSING CONVENTION EC2 Instance Reserved: BoxUsage, HeavyUsage EC2 Instance Non-Reserved: BoxUsage, SpotUsage EC2 EBS Stan...
26 DAYS STATISTICS 7.220.000++ LINES 1147 SECONDS (19 MINUTES 3 SECONDS)
KEY IDEAS CHECK THE LATEST 2 MONTHS REPORT RUN IT ONCE / TWICE PER DAY  (11 P.M. / 5 A.M.) USE SCRIPT / CRON TO HANDLE FAI...
NAMING CONVENTIONS / RESERVED KEYWORDS THERE'S NO UNDERSCORE '_' ON S3 BUCKET NAME PLEASE APPEND KEYWORD LIST AS BRIEF AS ...
ASSUMPTIONS All conventions in AWS Billing Report, Usage Report, and Company List are constant But the nature of productio...
DEMONSTRATION TIME!
ANY QUESTION? THANKS FOR YOUR ATTENTION! ありがとう ございました
AWS Cost Visualizer
Upcoming SlideShare
Loading in …5
×

AWS Cost Visualizer

2,051 views

Published on

A presentation which is used as GIP participants’ progress report. This presentation will explain how the custom-built AWS cost visualizer works.

Published in: Technology, Business
no profile picture user

  • Login to see the comments

AWS Cost Visualizer

  1. 1. AWS COST VISUALIZER HDE GIP - MTS JUNE 2014 Presented by Iskandar Setiadi & Saskya Irena
  2. 2. AWS COST VISUALIZER OVERVIEW 1. Problem Description 2. Analysis & Implementation 3. Result 4. Demonstration
  3. 3. ENORMOUS NUMBER OF PRODUCTS
  4. 4. UNDETAILED BILLING REPORT (UI)
  5. 5. COMPLICATED BILLING REPORT
  6. 6. CSV DETAILS InvoiceID,PayerAccountId,LinkedAccountId,RecordType,RecordId,ProductName,RateId,SubscriptionId,Pr "Estimated" ,"LineItem","21610877191184803585998665","AmazonElastic "Estimated" ,"LineItem","21942771010558985838334611","AmazonElastic BUT THERE ARE 8.000.000 (八百万) ENTRIES PER MONTH!
  7. 7. HOW MUCH IS THE EXPENSE FOR: 1. Each Customer 2. Each S3 Bucket 3. Each Infrastructure 4. Each Project (Tapirus, Nobita) 5. Other Expenses - DAILY / MONTHLY - - INSTANCE, ETC -
  8. 8. SOLUTION CREATE YOUR OWN TOOL!
  9. 9. IMPLEMENTATION ENVIRONMENT 1. Python 2.6.9 2. Flask Framework & Redis for Database 3. Linux/UNIX based Operating System 4. Amazon EC2 - t1.micro -> c3.large 5. Github (HDE/aws-cost-visualizer)
  10. 10. FIELDS OF DATA (CUSTOMER / CI / PROJECT) 1. Company Name 2. EC2 Instance Reserved 3. EC2 Instance Non-Reserved 4. EC2 EBS Standard 5. EC2 EBS PIOPS 6. EC2 Data Transfer 7. EC2 Others 8. S3 Usage Rate * 9. S3 ** 10. Others *= CUSTOMER ONLY ** = CI & PROJECT ONLY
  11. 11. FIELDS OF DATA (S3) 1. Bucket Name 2. Total Cost
  12. 12. FIELDS OF DATA (NON-CUSTOMER) 1. Product Name (SQS, etc) 2. Total Cost
  13. 13. PARSING CONVENTION EC2 Instance Reserved: BoxUsage, HeavyUsage EC2 Instance Non-Reserved: BoxUsage, SpotUsage EC2 EBS Standard: VolumeUsage, VolumeIOUsage, SnapshotUsage EC2 EBS PIOPS: VPIOPS, VolumeUsage EC2 Data Transfer: Regional, In-Bytes, Out-Bytes Others: Idle EIP, LoadBalancerUsage, Monitoring (Alarm, etc), DataProcessing, etc
  14. 14. 26 DAYS STATISTICS 7.220.000++ LINES 1147 SECONDS (19 MINUTES 3 SECONDS)
  15. 15. KEY IDEAS CHECK THE LATEST 2 MONTHS REPORT RUN IT ONCE / TWICE PER DAY  (11 P.M. / 5 A.M.) USE SCRIPT / CRON TO HANDLE FAILURE [PRODUCTION] IF AWS MONTHLY REPORT IS GETTING BIGGER (5 TIMES)  -> MIGRATE TO BETTER EC2 INSTANCE
  16. 16. NAMING CONVENTIONS / RESERVED KEYWORDS THERE'S NO UNDERSCORE '_' ON S3 BUCKET NAME PLEASE APPEND KEYWORD LIST AS BRIEF AS POSSIBLE
  17. 17. ASSUMPTIONS All conventions in AWS Billing Report, Usage Report, and Company List are constant But the nature of production environment is always changing, so it's needed to maintain the application If there's no specified date in AWS Billing Report, this application will map it to 1st date
  18. 18. DEMONSTRATION TIME!
  19. 19. ANY QUESTION? THANKS FOR YOUR ATTENTION! ありがとう ございました

×