Introduction to YQL & Yahoo! APIs

2. YQL AND YAHOO! APIS
Saurabh Sahni
YDN Developer, Hacker, Evangelist
Twitter: @saurabhsahni

3. Hacking together
systems in 24 hours is
lot of fun

4. Data manipulation
Hacks = +
Data visualization

5. The web has lot of data around

6. ProgrammableWeb.Com – 3517 APIs

7. Yahoo! has opened
up its data

8. http://developer.yahoo.com/everything.html

9. THE TROUBLE WITH DATA
• You need to find data API
• Get Access – Signup for key
• Find data endpoint
• Read docs to learn what parameters you
have
• Get data in obscure format
• Use data after converting and filtering
• More APIs you use, more is your
annoyance

10. To make data access
easy on the web,
Yahoo! created YQL

11. YQL turns web
services and data on
the web into
databases.

12. select {what} from {where}
where {conditions}

13. You can select, filter,
sort and limit

14. You can even insert,
update and delete
from it.

15. FINDING VIDEOS ABOUT BANGALORE
SELECT * FROM youtube.search where
query='bangalore'

16. SELECTING PHOTOS OF HACKDAY
SELECT * FROM flickr.photos.search where
text="hackday"

17. INSERTING DATA
INSERT INTO bitly.shorten (login, apiKey, longUrl)
VALUES ('ME', 'API_KEY', 'http://yahoo.com')

18. UPDATING DATA
UPDATE social.profile.status
SET status="Using YQL UPDATE”
WHERE guid="NJFIDHVPVVISDX7UKED2WHU"

19. RETRIEVING MY CONTACTS
SELECT * FROM social.contacts WHERE
guid=me

20. ACCESSING PRIVATE DATA
http://query.yahooapis.com/v1/yql
Uses OAuth 1.0 for authorization
OAuth is complicated – use one of our SDKs at
https://github.com/yahoo

21. The easiest way to start
with YQL is to use the
console
http://developer.yahoo.com/yql/console

22. YQL: http://developer.yahoo.com/yql/console

26. URL STRUCTURE
http://query.yahooapis.com/v1/yql?[params]
http://query.yahooapis.com/v1/public/yql?[params]
Params
• q = [ YQL QUERY ]
• format = [ XML / JSON ]
• diagnostics = [ true / false ]
• debug = [ true / false ]
• callback = [ function name ]

27. LET’S SEE IT

28. QUERY EXAMPLES
Find hackday tweets:
SELECT * FROM twitter.search where q='hackday’
Search Yahoo! Answers for resolved questions about cars:
select * from answers.search where query="cars" and type="resolved”
Translate tweets into Hindi:
select * from google.translate where q in (select text from
twitter.search where q="hackday") and target='hi';
Extract important terms from top stories on Yahoo! news:
select * from search.termextract where context in (select description
from rss where url='http://rss.news.yahoo.com/rss/topstories')

29. QUERY EXAMPLES
Find distance between Bangalore and Mumbai:
select * from geo.distance where place1="bangalore" and
place2="mumbai”
Get stock quotes for Yahoo!, Apple, Google & Microsoft in one query:
select * from yahoo.finance.quotes where symbol in
("YHOO","AAPL","GOOG","MSFT")
Shorten a URL:
insert into yahoo.y.ahoo.it (url, keysize) values ('http://
www.javarants.com', 5)
Search apartments in criagslist:
select * from craigslist.search where location="bangalore" and
type="apa" and query="indiranagar”

30. QUERY EXAMPLES
Scrape news from Yahoo! Finance:
select * from html where url="http://finance.yahoo.com/q?
s=yhoo" and xpath='//div[@id="yfi_headlines"]/div[2]/ul/li/a’
Select, filter data from google spreadsheets:
select * from csv where url="https://
spreadsheets.google.com/pub?key=0ArYndzim-
lbrdF8wc3A5QWl1ZGRpdkxRZk80SU9zUXc&output=csv"
and col5 like 'Bangalore%’ ;

31. Let’s find
openhackindia
photos on flickr

33. MAKING REQUESTS: FLICKR URLS
<photo farm="3"
id="5708163920"
isfamily="0"
isfriend="0"
ispublic="1"
owner="31832337@N04"
secret="0075137487"
server="2496"
title="San Francisco"/>

34. MAKING REQUESTS: FLICKR URLS
Photo URL
http://farm{$farm}.static.flickr.com/{$server}/
{$id}_{$secret}.jpg
Photo Page URL
http://www.flickr.com/photos/{$owner}/{$id}
Photo Owner Profile URL
http://www.flickr.com/photos/{$owner}

36. https://github.com/saurabhsahni/Hacks/

37. Finding Recent Photos from flickr

38. Some YQL Hacks

39. ChromYQLip is a
chrome extension for
page scraping via YQL
Open
Hack
Bangalore
2010
Winner

40. http://bit.ly/chromeYQL

41. VIDEO CLIP

42. http://www.webmeme.in

43. WEBMEME.IN
Fetch multiple feeds in different formats like atom, RSS and
transform them into consistent RSS format
Select * from rss where url in (‘http://feeds.feedburner.com/
pluggd’, ‘http://quatrainman.blogspot.com/atom.xml’, ‘…’)
Filter news containing “india” from multiple feeds:
select * from rss where url in ('http://feeds.feedburner.com/
TechCrunch', 'http://www.readwriteweb.com/rss.xml','http://
gigaom.com/feed/') and description like '%india%’

44. YQL is open – you
can get your data
tables in our system

45. All you need to do is
write an XML
schema and put it
on Github.

46. http://github.com/yql/yql-tables

47. Here is the craigslist
search table
https://github.com/yql/yql-tables/tree/master/
craigslist/craigslist.search.xml

49. USE INSTANTLY BY UPLOADING ON YOUR SITE
USE 'http://www.mysite.com/my_table.xml'
AS mytable;
SELECT * FROM mytable
WHERE user='saurabh'

50. You can even write
server side javascript
to build a webservice
or augment one.
h#p://developer.yahoo.com/yql/guide/yql-­‐execute-­‐chapter.html

51. There are lot of
things you can do
with YQL.

52. Play yourself
http://developer.yahoo.com/yql/

53. RESOURCES
All Yahoo! APIs and Services
http://developer.yahoo.com/everything.html
YQL Documentation
http://developer.yahoo.com/yql
YQL Console
http://developer.yahoo.com/yql/console
YQL Github Account (Contribute Tables)
http://github.com/yql/yql-tables

54. ANY QUESTIONS?
http://www.slideshare.net/
saurabhsahni
Saurabh Sahni
Twitter: @saurabhsahni
Github: http://github.com/saurabhsahni
Web: http://www.saurabhsahni.com