Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Web as a data resource

1,730 views

Published on

Tech Talk at Georgia Tech

Hack U, 8 March 2011

  • Login to see the comments

Web as a data resource

  1. 1. Get me my Data !<br />Use the web as a data source<br />
  2. 2. Hack U @ Georgia Tech - March 8 – 12 2011<br />2<br />
  3. 3. Subramanyan Murali<br />yahoo<br />Mail Engineer <br />Hacker, Photographer, Traveler <br />@rmsguhan<br />
  4. 4. Data and hacks <br />4<br />
  5. 5. Putting something together in 24 hours can be a real rush !<br />5<br />
  6. 6. Internet is a data exchange medium <br />
  7. 7. Need innovative ways to make the data into information that is useful <br />7<br />
  8. 8. Any resource on the internet is a source of data … <br />8<br />
  9. 9. … that means that there is a lot of data <br />9<br />
  10. 10. Loads and loads of data and data services <br />10<br />
  11. 11. Data that is complex and in many formats <br />11<br />
  12. 12. Navigating through all the complexity can be hard !#$%&<br />12<br />
  13. 13. How to solve the data problem ?<br />13<br />
  14. 14. http://pipes.yahoo.comsaves the day!<br />14<br />
  15. 15. Graphical interface to play with data !<br />15<br />
  16. 16. WYSIWYG = pipes<br />16<br />
  17. 17. Predefined presentations !RSS, JSON, Map, badge<br />17<br />
  18. 18. Pipes is a visual tool, where is my command line ??<br />18<br />
  19. 19. Put your hands up for YQL !<br />19<br />
  20. 20. Common data access language across the webdeveloper.yahoo.com/yql<br />20<br />
  21. 21. SQL like syntaxStandard and easy to use<br />21<br />
  22. 22. select {what} from {service} where {condition} in ({sub-query} )<br />22<br />
  23. 23. Several services can be chained using in ()<br />23<br />
  24. 24. YQL Console<br />24<br />
  25. 25. Lots of Integrations and predefined tablesanyone can be part of the interface<br />25<br />
  26. 26. Fetching data from a resource has never been simpler !<br />26<br />
  27. 27. http://query.yahooapis.com/v1/ public/yql?q={query} &format={format} <br />27<br />
  28. 28. Irrespective of the source, output can be standardized …<br />28<br />
  29. 29. … thus giving you a solid & consistent data foundation<br />29<br />
  30. 30. Input can be XML, HTML, CSV, micro-formats, Atom feeds, JSON, RSS<br />30<br />
  31. 31. Output can be JSON or XML<br />31<br />
  32. 32. Mix and match YQL and Pipes ?<br />32<br />
  33. 33. Pipes provides you with processing capabilities<br />33<br />
  34. 34. YQL is another feed source<br />34<br />
  35. 35. Interlinking YQL and Pipes = fewer lines of code<br />35<br />
  36. 36. Not limited to Yahoo! sources <br />
  37. 37. select * from google.translate where q in (select text from twitter.search where q="georgia tech") and target='fr';<br />
  38. 38. Fire off multiple queries at once <br />
  39. 39. select * from query.multi where queries=‘{query1};{query2}’;<br />
  40. 40. Create your own tables with Open tables www.datatables.org/<br />
  41. 41. http://www.yqlblog.net/<br />
  42. 42. Go explore !developer.yahoo.com/yql/console/<br />
  43. 43. Happy Hacking !!<br />Photos @ http://flickr.com/photos/rmsguhan<br />

×