In this talk from DevCon TLV we covered: ● The power of HTML5 APIs and how you can use them in your next modern Web Apps. ● On the server side how you can use: Google Cloud Endpoints to scale your API and gain more productivity. ● We did some live Demos and talked about Big Query interfaces.

1. Big Data And HTML5
@DevCon TLV
Ido Green
Developer Advocate
http://plus.ly/greenido
@greenido DevCon TLV
September 5th 2012
#devcon

2. ●The power of HTML5 APIs
●Modern Web Apps
●Google Cloud Endpoints
●Live Code & Demo
●Big Query
●Q&A
Agenda
#devcon

3. Modern Web Applications

4. Modern Web Applications
● Self Contained & Functional
● "Offline First"
● Client Side Architecture &
MVC Frameworks
● Device Aware / 60fps
#devcon

5. http://goo.gl/5cJwH
#devcon

6. http://goo.gl/Y6AG1
#devcon

7. Client Side Frameworks
● Ember.js - Don't waste time making trivial choices
● Backbone.js - Gives structure to web applications by
providing models with binding, collections and views
● Angular.js - AngularJS lets you extend HTML vocabulary
for your application
More: addyosmani.github.com/todomvc/
#devcon

8. ● Airplane, road trip, deserted island
● Flaky connections (e.g. cafes, car)
● Better performance
● Consolidates the concept of permanent app you will have
always available
* We will use: Lawnchair for our demo.
Offline - Why?
#devcon

9. ● Storing assets: AppCache
● Storing data: localStorage, IndexedDB, File API.
● Offline first:
○ Pretend that there's no internet connection
○ Implement a sync layer that works only when
online.
Offline - How?
navigator.onLine & window.(ononline|onoffline)
#devcon

10. Do More For Your Users
Web Intents is a framework for client-side
service discovery and inter-application
communication
#devcon

11. Google Chrome Frame
Chrome Frame is an open source plug-in that seamlessly
brings Google Chrome's open web technologies and
speedy JavaScript engine to IE
<meta http-equiv="X-UA-Compatible"
content="chrome=1">
X-UA-Compatible: chrome=1
#devcon

12. Modern Web Apps and The Server Conundrum
All modern web apps have to deal with a "home" server
○ Offload Computation
○ Sharing and Collaboration
But who wants to run a server
○ Spikey traffic
○ Client Server communication
○ Serialization
○ OAuth Dance
#devcon

13. ● Make it easy to build
● Easy to run and manage
● Easy to scale
● Free to get started, pay for what you use
● Not just for web serving, but also data persistence
Google App Engine To The Rescue
#devcon

14. Google Cloud Endpoints:
Business Logic
APIs for Mobile and Web Backends Made Easy!
Storage
(DataStore, SQL, Drive, etc)
Web APIs
Endpoints
Trusted
Tester
#devcon

15. Demo
https://github.com/greenido/backbone-bira
#devcon

16. Google Cloud Endpoints for Beer!
Business Logic
Beer Rating and Review Application
Storage
NoSql Datastore
Endpoints
Access Control,
Sort, Filter
Client UI for
managing Beers
#devcon

17. Demo!
Beers On!
https://birra-io2012.appspot.com/

18. Load Test Results - From laptop wifi
% ab -n9000 -c100 http://birra-io2012.appspot.com/
Percentage of the requests served within a certain time
(ms)
50% 263
66% 321
75% 370
80% 390
90% 449
95% 536
98% 649
99% 715
100% 3858 (longest request)
> 90% of requests below
1/2 a sec
#devcon

19. Load Test Results - From Compute Engine
% ab -n1000000 -c10 http://birra-io2012.appspot.com/
Percentage of the requests served within a certain time
(ms)
50% 6
66% 6
75% 6
80% 6
90% 7
95% 7
98% 11
99% 17
100% 3019 (longest request)
> 95% less than 7ms
#devcon

20. Screen shot of admin console
> 6k QPS!!

21. How BIG is big?

22. 1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
10 million rows?

23. 1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
100 million rows?

24. 1 million
1 million 1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
1 million
720 million rows!

25. Big Data at Google
70+ hours
100 million gigabytes
425 million users
#devcon

27. BigQuery gives you this power
Store data with reliability, redundancy and
consistency
Go from data to meaning
Quickly!
At scale & secure way...
#devcon

28. How are developers using it?
Game and social media analytics
Advertising campaign optimization
Sensor data analysis
Infrastructure monitoring
#devcon

29. Big Data at Google - Finding slow servers
SELECT
count(*) AS count, source_machine AS machine
FROM product.product_log.live
WHERE
elapsed_time > 4000
GROUP BY
source_machine
ORDER BY
count DESC
Result in ~20 seconds!
#devcon

30. Google Cloud
Storage
Upload your Data
It's An API
#devcon

31. BigQuery
Your data is loaded
Google Cloud
Storage
GCE
#devcon

32. Load your data into BigQuery
"jobReference":{
"projectId":"605902584318"},
"configuration":{
"load":{
"destinationTable":{
"projectId":"605902584318",
"datasetId":"my_dataset",
"tableId":"widget_sales"},
"sourceUris":[
"gs://widget-sales-data/2012080100.csv"],
"schema":{
"fields":[{
"name":"widget",
"type":"string"},
...
POST https://www.googleapis.com/bigquery/v2/projects/605902584318/jobs
#devcon

33. Query Away!
"jobReference":{
"projectId":"605902584318"},
"query":"SELECT TOP(widget, 50), COUNT(*) AS
sale_count
FROM widget_sales",
"maxResults":100,
"apiVersion":"v2"
}
POST https://www.googleapis.com/bigquery/v2/projects/605902584318/jobs
#devcon

34. ●Java
●Python
●.NET
●PHP
●JavaScript
●Apps Script
●More ...
Libraries
#devcon

35. Libraries - Example JavaScript query
var request = gapi.client.bigquery.jobs.query({
'projectId': project_id,
'timeoutMs': '30000',
'query': 'SELECT state, AVG(mother_age) AS theav
FROM [publicdata:samples.natality]
WHERE year=2000 AND ever_born=1
GROUP BY state
ORDER BY theav DESC;'
});
request.execute(function(response) {
console.log(response);
$.each(response.result.rows, function(i, item) {
...
#devcon

36. BigQuery UI
bigquery.cloud.google.com#devcon

37. Prepare your data / Batch Jobs
●App Engine MapReduce
●Commercial ETL tools
● Pervasive
● Informatica
● Talend
●UNIX command-line
Don't need interactive queries for some jobs?
● priority: "BATCH"
#devcon

38. Google Spreadsheets

39. Google Visualization API
#devcon

40. Commercial visualization tools
#devcon

41. Key Take Aways
Building modern applications with HTML5 and App Engine
AppEngine makes for easy deployment at scale
Use Big Query to 'feel' your data
Build interactivity by leveraging Cloud Endpoints
Exploit Modern Browser Features:
○ Offline
○ Geo
○ Web Intents
#devcon

42. Toda & Questions?
Ido Green
Developer Relations
Google Chrome Platform
plus.ly/greenido
App:: http://birra-io2012.appspot.com/
Code: https://github.com/greenido/
Slides: ido-green.appspot.com
Access: http://endpoints-trusted-tester.appspot.com
#devcon