Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Hacking RSS:        Filtering & Processing    Obscene Amounts of Information              #hackingRSS       Dawn FosterInt...
Information Overload                       CD Photo: http://www.flickr.com/photos/chefranden/2751354004/
Who Cares?●   Most of it is …    –   complete crap    –   out of date / obsolete    –   not interesting to you    –   irre...
You Want to Find the Needle                      Haystacks: http://www.flickr.com/photos/rasekh/4911673659/
RSS Alone is a Start●   Sources you care about delivered right to you. But …    –   Do you care about everything in each f...
Prioritize Your Reader●   Put things you care about at the top●   Categorize●   Dont try to read everything
Outsource / Crowdsource New Sources
The Real Magic is in Filtering RSS                       Complete Crap                         Interesting                ...
RSS Filtering Tools●   Yahoo Pipes (my favorite)    –   More powerful & fexible: options to filter any data found in      ...
Yahoo Pipes Filtering Example●   Input:    –   WebWorkerDaily    –   ReadWriteWeb●   Filter by content:    –   Collaborate...
PostRank●   Best Posts in a    feed●   Ranked on    engagement (links,    sharing, comments)●   Can get output as    RSS f...
Whats In a Feed? PostRank (Yahoo Pipes View)●   Content in feeds varies wildly depending on site.●   Common: title, author...
Yahoo Pipes PostRank Example●   Input PostRank    Feeds:    –   Engadget    –   CrunchGear    –   Boy Genius●   Filter by ...
Reformatting / Modifying RSS Feeds   Dont be satisfied with default RSS feed formats! Twitter Search Twitter RSS Feed     ...
Yahoo Pipes: Reformat Twitter Feed●   Input:    –   Twitter Search        feed●   Loop String Build:    –   Author    –   ...
Yahoo Pipes: Reformat PostRank Feed●   Input:    –   3 PostRank feeds●   Loop String Build:    –   PostRank    –   : (spac...
Using Web APIs 101●   Many API calls are basically URLs●   Constructing URLs    –   Use API documentation/examples to     ...
BackTweets (BackType API)●   Data about links on    Twitter●   Finds links regardless of    shortening service●   No RSS F...
Backtweets API + Twitter API + Yahoo Pipes●   What we want to do:    –   Start with a set of URLs (blog posts in a feed)  ...
BackTweets API: Get Tweet ID●   Take WebWorkerDaily Author Feed●   Use WWD URLs to build URLs for BackType API call●   Fet...
Twitter API: Get Data Based on Tweet ID●   Use BackType tweet ID to build URL for Twitter API●   Fetch data about Tweet & ...
BackType + Twitter API + Pipes Output●   Data from BackType + Twitter●   Built an RSS feed using Yahoo Pipes●   Included t...
Add Filters to BackType + Twitter Example●   Show only tweets from people with 1000+ followers
Admit it, we ALL do vanity searches ●   You can enter your search queries in Google, Twitter,     Flickr …       –   Add a...
How Should / Shouldnt You Use All of This?●   Do:    –   Use this for personal productivity    –   Play around and underst...
Q&AAbout Dawn:● Intel Community Manager for MeeGo● More Info: http://fastwonderblog.com● Dawn@FastWonder.com● @geekygirlda...
03/15/11   27
Upcoming SlideShare
Loading in …5
×

33

Share

Download to read offline

SXSW Hacking RSS: Filtering & Processing Obscene Amounts of Information

Download to read offline

Information overload is less about having too much information and more about not having the right tools and techniques to filter and process information to find the pieces that are most relevant for you. This presentation will focus on showing you a variety of tips and techniques to get you started down the path of looking at RSS feeds in a completely different light. The default RSS feeds generated by your favorite blog or website are just a starting point waiting to be hacked and manipulated to serve your needs. Most people read RSS feeds, but few people take the time to go one step further to hack on those RSS feeds to find only the most interesting posts. I combine tools like Yahoo Pipes, BackTweets, PostRank and more with some simple API calls to be able to find what I need while automatically discarding the rest. You start with one or more RSS feeds and then feed those results into other services to gather more information that can be used to further filter or process the results. This process is easier than it sounds once you learn a few simple tools and techniques, and no “real” programming experience is required to get started. This session will show you some tips and tricks to get you started down the path of hacking your RSS feeds.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

SXSW Hacking RSS: Filtering & Processing Obscene Amounts of Information

  1. 1. Hacking RSS: Filtering & Processing Obscene Amounts of Information #hackingRSS Dawn FosterIntel Community Manager for MeeGo dawn@fastwonder.com
  2. 2. Information Overload CD Photo: http://www.flickr.com/photos/chefranden/2751354004/
  3. 3. Who Cares?● Most of it is … – complete crap – out of date / obsolete – not interesting to you – irrelevant for you Junk Pile: http://www.flickr.com/photos/zen/4013525/
  4. 4. You Want to Find the Needle Haystacks: http://www.flickr.com/photos/rasekh/4911673659/
  5. 5. RSS Alone is a Start● Sources you care about delivered right to you. But … – Do you care about everything in each feed? – What about the feeds you arent subscribed to? – Can you keep up with what you have?
  6. 6. Prioritize Your Reader● Put things you care about at the top● Categorize● Dont try to read everything
  7. 7. Outsource / Crowdsource New Sources
  8. 8. The Real Magic is in Filtering RSS Complete Crap Interesting Maybe Relevant Yay!● In my Google Reader right now: – Analyst research blogs mentioning Online Community – Analyst research blogs mentioning MeeGo – Searches across social sites mentioning me, my projects, my websites etc. - filtering out things I dont care about – My favorite blogs filtered using PostRank to find only the ones with a lot of comments or social mentions
  9. 9. RSS Filtering Tools● Yahoo Pipes (my favorite) – More powerful & fexible: options to filter any data found in any field in the rss feed (URL, title, description, author …) – Downside: takes some time to learn & can be a little faky at times. Also a single point of failure if Yahoo ever killed it.● Other Options – FeedRinse: easy to use, not as fexible. Import RSS feeds, add filters, get new RSS feeds out. – RSS readers with filtering / alerts (FeedDemon) – Code: write your own filters – Note: many free RSS filtering services have gone out of business – can be bandwidth intensive & costly to host.
  10. 10. Yahoo Pipes Filtering Example● Input: – WebWorkerDaily – ReadWriteWeb● Filter by content: – Collaborate – Collaboration – Collaborative● Output: – 1 RSS Feed – Matching 3 keywords 2 Minute Yahoo Pipe Video How-tos: http://fastwonderblog.com/yahoo-pipes-and-rss-hacks/
  11. 11. PostRank● Best Posts in a feed● Ranked on engagement (links, sharing, comments)● Can get output as RSS feed● Feed includes postrank number as a field
  12. 12. Whats In a Feed? PostRank (Yahoo Pipes View)● Content in feeds varies wildly depending on site.● Common: title, author, pubDate, link, content, description● Site-specific: postrank, lat/long, image links, username, twitter source … (most RSS readers dont show these)● API: usually has additional data & can output RSS● If its in the feed, you can use it!
  13. 13. Yahoo Pipes PostRank Example● Input PostRank Feeds: – Engadget – CrunchGear – Boy Genius● Filter by content – Tablet● Sort: – PostRank● Output – 1 RSS feed – Best tablet posts
  14. 14. Reformatting / Modifying RSS Feeds Dont be satisfied with default RSS feed formats! Twitter Search Twitter RSS Feed Modify & more quickly scan key data
  15. 15. Yahoo Pipes: Reformat Twitter Feed● Input: – Twitter Search feed● Loop String Build: – Author – : (spacing) – Title● Loop Assign: – Store result back into title● Output: – 1 RSS feed – Efficient format
  16. 16. Yahoo Pipes: Reformat PostRank Feed● Input: – 3 PostRank feeds● Loop String Build: – PostRank – : (spacing) – Title● Loop Assign: – Store result back into title● Output: – 1 RSS feed – Efficient format
  17. 17. Using Web APIs 101● Many API calls are basically URLs● Constructing URLs – Use API documentation/examples to format the URL – http://api.twitter.com/1/statuses/show /ID.xml ● Version 1 of API show status for ID in .format● API keys – Tells API who you are (password)● Rate limiting – Only get so much & youre cut of – Limited by IP or API key – Chill out for a while & come back XKCD Comic: http://xkcd.com/844/
  18. 18. BackTweets (BackType API)● Data about links on Twitter● Finds links regardless of shortening service● No RSS Feeds● But … You can use API + Pipes to build one!
  19. 19. Backtweets API + Twitter API + Yahoo Pipes● What we want to do: – Start with a set of URLs (blog posts in a feed) – Find any tweet mentioning those URLs – Return the tweet and data about the person who posted it● Mission: Build feed using only data from these 2 APIs● BackType API provides Tweet ID (not humanly useful) – http://api.backtype.com/tweets/search/links.xml? q=URL&mode=batch&key=KEY – List of Twitter Status IDs for Tweets linking to URL – Note: I think this feature may be deprecated● Twitter API uses Tweet ID to get everything else – http://api.twitter.com/1/statuses/show/ID.xml – Returns a single status all relevant data for ID
  20. 20. BackTweets API: Get Tweet ID● Take WebWorkerDaily Author Feed● Use WWD URLs to build URLs for BackType API call● Fetch data from BackType URLs to get Tweet ID
  21. 21. Twitter API: Get Data Based on Tweet ID● Use BackType tweet ID to build URL for Twitter API● Fetch data about Tweet & User from Twitter API● Re-Build title to show “user (followers): tweet”
  22. 22. BackType + Twitter API + Pipes Output● Data from BackType + Twitter● Built an RSS feed using Yahoo Pipes● Included the information relevant for me● Could have included or filtered on: name, listed count, location, profile image, user URL, ...
  23. 23. Add Filters to BackType + Twitter Example● Show only tweets from people with 1000+ followers
  24. 24. Admit it, we ALL do vanity searches ● You can enter your search queries in Google, Twitter, Flickr … – Add a new project & have to update all of them – Can be hard to filter out some results – May have duplicates from multiple searches ● Yahoo Pipes – Update keywords in a CSV file – Use CSV file as input into a bunch of searches (RSS or API inputs) – Filter out what you dont want – Get 1 filtered RSS feed as output2 minute video: http://fastwonderblog.com/2009/05/01/keyword-csv-files-and-searching-2-minute-yahoo-pipes-demo/
  25. 25. How Should / Shouldnt You Use All of This?● Do: – Use this for personal productivity – Play around and understand the possibilities – Create prototypes for something you might want to build● Dont: Use in critical or production environments● Everything Ive done here could be done in most programming languages● For production use or putting data on websites: – Re-write in a real programming language with cached results and error checking XKCD Comic: http://xkcd.com/327/
  26. 26. Q&AAbout Dawn:● Intel Community Manager for MeeGo● More Info: http://fastwonderblog.com● Dawn@FastWonder.com● @geekygirldawn on Twitter 26Additional Reading:● http://fastwonderblog.com/yahoo-pipes-and-rss-hacks/ Photo of Dawn: http://www.flickr.com/photos/ahockley/3036575066/
  27. 27. 03/15/11 27
  • jjfscheffer

    Dec. 20, 2014
  • andeldar

    Apr. 5, 2014
  • DigitalVizir

    Sep. 5, 2013
  • rafarios

    Nov. 6, 2012
  • mcassi

    Apr. 5, 2012
  • arfatrulz

    Nov. 24, 2011
  • accloureiro

    Nov. 14, 2011
  • Patman162

    Oct. 5, 2011
  • reinhard

    Jun. 18, 2011
  • zarzecks

    Apr. 12, 2011
  • agustilopez

    Apr. 8, 2011
  • wlars

    Apr. 7, 2011
  • smlab

    Apr. 1, 2011
  • budesigns

    Mar. 30, 2011
  • jingo74

    Mar. 30, 2011
  • norelijn

    Mar. 30, 2011
  • plailleur

    Mar. 28, 2011
  • jroselln

    Mar. 28, 2011
  • paddyof

    Mar. 28, 2011
  • driessen

    Mar. 28, 2011

Information overload is less about having too much information and more about not having the right tools and techniques to filter and process information to find the pieces that are most relevant for you. This presentation will focus on showing you a variety of tips and techniques to get you started down the path of looking at RSS feeds in a completely different light. The default RSS feeds generated by your favorite blog or website are just a starting point waiting to be hacked and manipulated to serve your needs. Most people read RSS feeds, but few people take the time to go one step further to hack on those RSS feeds to find only the most interesting posts. I combine tools like Yahoo Pipes, BackTweets, PostRank and more with some simple API calls to be able to find what I need while automatically discarding the rest. You start with one or more RSS feeds and then feed those results into other services to gather more information that can be used to further filter or process the results. This process is easier than it sounds once you learn a few simple tools and techniques, and no “real” programming experience is required to get started. This session will show you some tips and tricks to get you started down the path of hacking your RSS feeds.

Views

Total views

29,352

On Slideshare

0

From embeds

0

Number of embeds

15,437

Actions

Downloads

285

Shares

0

Comments

0

Likes

33

×