1. 1
Automate web performance metrics
how to measure cdn
performance?
Marc Cortinas – Production Service - Webops - August 2014
2. 2
Agenda
Why?
Increase web preformance = Sales
Motivation PS-9063
1. How to measure web performance metrics?
2. How to automated web testing?
3. Combine: automate + web performance
4. What’s NodeJS? PhantomJS? QtWebKit?
5. Phantomas – Features
6. Phantomas – Metric Choice
7. Arquitecture
8. Grafana Dashboard
9. Which CDN are we goind to measure?
3. Tools used to measure web performance metrics
MAIN OBJECTIVE:
Enumerate different tools to calculate web performance in our application
Tools:
1. Pluguins/Browser features: Firebug, HTTP Tamper Data, Chrome
Developer Tools, Firefox Developer Tools, etc…
2. applications: Fiddler, Har 101(maybe gained by
phantomjs)+Tools(yslow),phantomas-grunt
3. $$ External Services $$ : Iplabel, Monitis, Pingdom, Apica, AlertSite
4. Others solutions:
3
1. Nagios + check_http
2. Time + Curl/wget
3. Monit + http check
4. Munin + http_loadtime
4. 4
Tools used to automate web testing
MAIN OBJECTIVE:
Enumerate different tools to automate web testing
Tools:
1. Selenium Grid + Jenkins
2. Others +++ Help me!!!
5. Combine: automate + web performance
MAIN OBJECTIVE:
Automate some web performance metric solution
Candidates: (all based on NodeJS)
1. PhantomJS + Yslow : HAR + YSLOW
Yslow: less features than phantomas
5
2. Grunt-phantomas :
Build-GUI from phantomas is so limited
Data stored in JSON file
3. Our solution: Grafana + Phantomas
1. Phantomas to collect data in CSV files
2. phan2graph.py Python Script to send to graphite
3. One sensor in each AWS EC2 region (blog account)
Micro instances is not allowed on VPC (ireland)
4. Sync whisper files(+backup) VS multiples datasources in
Grafana(+realtime)
6. What’s NodeJS? PhantomJS? QtWebKit?
MAIN OBJECTIVE:
Describe basic concepts to understand phantomas:
1. NodeJS: is a platform built on Chrome's JavaScript runtime for easily building
6
fast, scalable network applications
2. PhantomJS: PhantomJS is a headless WebKit scriptable with a JavaScript
API. It has fast and native support for various web standards: DOM handling,
CSS selector, JSON, Canvas, and SVG. Based on QtWebKit
3. QtWebKit: web content engine based on the Open Source WebKit project,
the same engine used by browsers such as Google Chrome and Safari from
Apple.
7. 7
Phantomas - Features
MAIN OBJECTIVE:
List of main useful features
Features:
1. Command line tool
2. Internal solution, I prefer that external solution
3. Full-load page,
4. 3rd parties calls can be discarted (explicit list of domain allowed)
5. Multiple runs to improve median datas
6. Differents output data formats: JSON, CSV, ElasticSearch,
7. Metrics can be sent to statds, elasticsearch (graphite too now! )
8. Timeout parameter
9. 2 web engine avaible: qtwebkit (default) or gecko (experimental)
10. User-agent switcher
11. HAR file export
12. Screenshot to SVG
8. 8
Phantomas – Metrics Choice
MAIN OBJECTIVE:
List of metrics: https://github.com/macbre/phantomas
Metrics selected to choice CDN:
1. Median Latency: median value of time to the first byte for all responses
2. Full Load Page Time: time it took to receive the last byte of the last HTTP
response
3. Number of assets not gziped: static assets that were not gzipped
4. Fastest Response: the time to the last byte of the fastest response
Metrics not analized yet but maybe is interesting:
timeToFirstCss: time it took to receive the last byte of the first CSS
timeToFirstJs: time it took to receive the last byte of the first JS
timeToFirstImage: time it took to receive the last byte of the first image
cachingNotSpecified: responses with no caching header sent (either Cache-Control or Expires)
cachingTooShort: responses with too short (less than a week) caching time
cachingDisabled: responses with caching disabled (max-age=0)
oldCachingHeaders: responses with old, HTTP 1.0 caching headers (Expires and Pragma)
9. 9
Architecture
CSV
phant2graph.py
Grafana
phant
AWS
Cogent
Colt
other..
Akamai
Fastly
webop
Graphite Graphite
rsync
whisper file whisper file
Code Name Graphite URL
ap-southeast-1 Singapore http://phant-singapore.services.odigeo.com/dashboard/ - Singapore
ap-southeast-2 Sydney http://phant-sydney.services.odigeo.com/dashboard/ - Sydney
eu-west-1 Ireland http://phant-ireland.services.odigeo.com/dashboard/ - Ireland
sa-east-1 Sao Paulo http://phant-saopaulo.services.odigeo.com/dashboard/ - Saopaulo
us-east-1 Northern Virginia http://phant-virginia.services.odigeo.com/dashboard/ - Virginia
us-west-1 Northern California http://phant-california.services.odigeo.com/dashboard/ - California
11. Which CDN are we goind to measure?
1. All content: the main domain browseable throught CDN
1. AWS Cloudfront
2. Fastly
3. Limelight
4. Akamai
5. EdgeCast
6. Own
7. …
8. Multi-CDN (JS detect the region and select CDN)
2. Only static: exclusive domain for static content
11
1. AWS Cloudfront
2. Fastly