SlideShare a Scribd company logo
1 of 56
Download to read offline
7
                                      drupal




Importing and syncing
content using Feeds

Peter Vanhee
2nd October 2011 - DrupalCamp Spain
Who am I?

‣ Consumer and contributor to Drupal for
  over 5 years
‣ Co-founder Youth Agora @youthagora
  and Nuvole @nuvoleweb
‣ Web consultant in Barcelona @pvhee
www.europeancampus.eu
Content is aggregated and synchronized using Feeds
Use Cases

‣ Build Activity streams using feeds from
  Twitter, Facebook, ...
‣ Batch import sets of users from file
‣ Import Commerce Products
‣ ...
What is Feeds?
Feeds is...

‣ Import / aggregate framework
‣ Create and update drupal entities out of
  the box: nodes, users, taxonomy
‣ Real-time feeds support
Related


‣ Migrate         ‣   Aggregator
‣ Table Wizard
‣ User import
‣ Node import
Feeds as the standard for piping content
Feeds usage


               drupal 6



                  drupal 7




              statistics from drupal.org
                Statistics from drupal.org
Fetcher

 Parser

Processor
Fetcher


‣ Http
‣ File upload




                     A complete list is at http://drupal.org/node/856644
Parser
‣ CSV, RSS1, RSS2, Atom
‣ OPML, Sitemap XML
‣ XPath for XML/HTML
‣ JSONPath for JSON
‣ YouTube, Flickr, Slideshare
‣ iCal

                        A complete list is at http://drupal.org/node/856644
Processor

‣ Node
‣ User
‣ Taxonomy
‣ Commerce Products



                      A complete list is at http://drupal.org/node/856644
2011.drupalcamp.es   my drupal site

      Example #1
Import and sync RSS Feeds
Feeds importers are configurations of
  Fetcher      Parser     Processor
create nodes of type Feed item
fields in RSS   fields in Feed item
And now... importing!
/import
We can do this with Drupal Core Aggregator too!
Let’s try spawning
                                 Feed A

                        User 1   Feed B
              Feed A

    User      Feed B             Feed C
              Feed C
1

                        User 2   Feed X
    Aggregator (core)                 Feeds
Spawning
Add Feed = Add Node
Example #2
Import   images
A node of type Story will hold a Flickr image
the Flickr XML feed




{   http://www.flickr.com/services/feeds/geo?format=rss_200&tags=drupalcamp
Parse XML with XPath expressions

              //item
               title
               link
        description




            geo:lat
           geo:long



media:thumbnail/@url
Fetcher             http



 Parser         XPath Parser


            Map targets from XPath
Processor      Node processor
select XPath XML parser
source is XPath Result   fields in Story
Importing
We have problems



        !
     Tags are not imported correctly
Feeds Tamper




     http://drupal.org/project/feeds_tamper
Enter Tampering
  Input Field                    Output Field
                       explode
drupal feeds sevilla             drupal
                                 feeds
                                 sevilla
                 html_encode
Me&lt;/a&gt;posted a        <a href=”.”>Me</a>
  photo:&lt;/p&gt;            posted a photo
Configure Tampering
More tamper actions
Add your own tamper plugins




            snippet from html_entity_decode.inc
After importing... tags are now
        correctly split
Importing Big Stuff

‣ Feeds has Batch support
‣ Feeds can run on cron jobs in the
  background
Field support
‣   Text       ‣ Email
‣   File       ‣ Multimedia
‣   Image      ‣ Address
‣   Link
‣   Geofield
‣   Taxonomy   Tip: check Feeds issue
‣   Password   queues for patches
‣   Date
‣   Integer
Feeds as a Framework
‣ Supports import/sync of Commerce
  Products (via Commerce Feeds)
‣ Straightforward to add custom fetchers,
  patchers and processors using Feed
  hooks (check feeds.api.php)
Finally...
when not to use Feeds


‣ Content does not need to be stored by
  Drupal
‣ Complex content migration tasks
Now... Real-Time?
                        Are we there yet?
                        Are we there yet?
                        Are we there yet?



Publisher                                     Subscriber
   Yes. Here you are.


                        Are we there yet?


                                     Slide taken from @alexb presentation on Feeds
Enter PubSubHubBub


Publisher          Hub           Subscriber

 What does this solve?
 1. Notifications to Subscribers managed by Hub
 2. Send only what is changed and when it gets
 changed
Euhm.. so what does
         Feeds do exactly?

  Feeds makes Drupal a PSHB
         Subscriber

Update feeds in real-time from PSHB Hubs
       e.g. Blogger, Wordpress, Superfeedr, ...
Thanks!


          Peter Vanhee
          @pvhee
Questions?


             Peter Vanhee
             @pvhee

More Related Content

Similar to Importing and synchronizing content using Feeds

Hadoop at Twitter (Hadoop Summit 2010)
Hadoop at Twitter (Hadoop Summit 2010)Hadoop at Twitter (Hadoop Summit 2010)
Hadoop at Twitter (Hadoop Summit 2010)
Kevin Weil
 
Introduction To Drupal
Introduction To DrupalIntroduction To Drupal
Introduction To Drupal
Lauren Roth
 
Best Practices for Migrating a Legacy-Based CMS to Drupal
Best Practices for Migrating a Legacy-Based CMS to DrupalBest Practices for Migrating a Legacy-Based CMS to Drupal
Best Practices for Migrating a Legacy-Based CMS to Drupal
Acquia
 
How to? Drupal developer toolkit. Dennis Povshedny.
How to? Drupal developer toolkit. Dennis Povshedny.How to? Drupal developer toolkit. Dennis Povshedny.
How to? Drupal developer toolkit. Dennis Povshedny.
DrupalCampDN
 
Drupal 6 to 7 migration guide
Drupal 6 to 7 migration guideDrupal 6 to 7 migration guide
Drupal 6 to 7 migration guide
Ebizon
 
Behavior & Specification Driven Development in PHP - #OpenWest
Behavior & Specification Driven Development in PHP - #OpenWestBehavior & Specification Driven Development in PHP - #OpenWest
Behavior & Specification Driven Development in PHP - #OpenWest
Joshua Warren
 

Similar to Importing and synchronizing content using Feeds (20)

Hadoop at Twitter (Hadoop Summit 2010)
Hadoop at Twitter (Hadoop Summit 2010)Hadoop at Twitter (Hadoop Summit 2010)
Hadoop at Twitter (Hadoop Summit 2010)
 
Introduction To Drupal
Introduction To DrupalIntroduction To Drupal
Introduction To Drupal
 
Drupal8 for Symfony developers - Dutch PHP
Drupal8 for Symfony developers - Dutch PHPDrupal8 for Symfony developers - Dutch PHP
Drupal8 for Symfony developers - Dutch PHP
 
Open event (show&tell april 2016)
Open event (show&tell april 2016)Open event (show&tell april 2016)
Open event (show&tell april 2016)
 
Drupal in Libraries
Drupal in LibrariesDrupal in Libraries
Drupal in Libraries
 
Hadoop and Pig at Twitter__HadoopSummit2010
Hadoop and Pig at Twitter__HadoopSummit2010Hadoop and Pig at Twitter__HadoopSummit2010
Hadoop and Pig at Twitter__HadoopSummit2010
 
Drupal 8 as a Drop-In Content Engine - SymfonyLive Berlin 2015
Drupal 8 as a Drop-In Content Engine - SymfonyLive Berlin 2015Drupal 8 as a Drop-In Content Engine - SymfonyLive Berlin 2015
Drupal 8 as a Drop-In Content Engine - SymfonyLive Berlin 2015
 
State of Drupal keynote, DrupalCon Vienna
State of Drupal keynote, DrupalCon ViennaState of Drupal keynote, DrupalCon Vienna
State of Drupal keynote, DrupalCon Vienna
 
Best Practices for Migrating a Legacy-Based CMS to Drupal
Best Practices for Migrating a Legacy-Based CMS to DrupalBest Practices for Migrating a Legacy-Based CMS to Drupal
Best Practices for Migrating a Legacy-Based CMS to Drupal
 
Best Practices and Tips on Migrating a Legacy-Based CMS to Drupal
Best Practices and Tips on Migrating a Legacy-Based CMS to DrupalBest Practices and Tips on Migrating a Legacy-Based CMS to Drupal
Best Practices and Tips on Migrating a Legacy-Based CMS to Drupal
 
Drupal8 for Symfony Developers
Drupal8 for Symfony DevelopersDrupal8 for Symfony Developers
Drupal8 for Symfony Developers
 
How to? Drupal developer toolkit. Dennis Povshedny.
How to? Drupal developer toolkit. Dennis Povshedny.How to? Drupal developer toolkit. Dennis Povshedny.
How to? Drupal developer toolkit. Dennis Povshedny.
 
Drupal 6 to 7 migration guide
Drupal 6 to 7 migration guideDrupal 6 to 7 migration guide
Drupal 6 to 7 migration guide
 
Getting started with WordPress development
Getting started with WordPress developmentGetting started with WordPress development
Getting started with WordPress development
 
Apigee deploy grunt plugin.1.0
Apigee deploy grunt plugin.1.0Apigee deploy grunt plugin.1.0
Apigee deploy grunt plugin.1.0
 
A winning combination: Plone as CMS and your favorite Python web framework as...
A winning combination: Plone as CMS and your favorite Python web framework as...A winning combination: Plone as CMS and your favorite Python web framework as...
A winning combination: Plone as CMS and your favorite Python web framework as...
 
Behavior & Specification Driven Development in PHP - #OpenWest
Behavior & Specification Driven Development in PHP - #OpenWestBehavior & Specification Driven Development in PHP - #OpenWest
Behavior & Specification Driven Development in PHP - #OpenWest
 
Evented applications with RabbitMQ and CakePHP
Evented applications with RabbitMQ and CakePHPEvented applications with RabbitMQ and CakePHP
Evented applications with RabbitMQ and CakePHP
 
Drupal 8 preview_slideshow
Drupal 8 preview_slideshowDrupal 8 preview_slideshow
Drupal 8 preview_slideshow
 
Introduction to Drupal, Wayne Eaker, Nov 11, 09
Introduction to Drupal, Wayne Eaker, Nov 11, 09 Introduction to Drupal, Wayne Eaker, Nov 11, 09
Introduction to Drupal, Wayne Eaker, Nov 11, 09
 

More from Peter Vanhee (7)

Going serverless at Comic Relief - July 2019
Going serverless at Comic Relief - July 2019Going serverless at Comic Relief - July 2019
Going serverless at Comic Relief - July 2019
 
Predicting peptide interactions using protein building blocks
Predicting peptide interactions using protein building blocksPredicting peptide interactions using protein building blocks
Predicting peptide interactions using protein building blocks
 
Going serverless at Comic Relief
Going serverless at Comic ReliefGoing serverless at Comic Relief
Going serverless at Comic Relief
 
Building reusable websites on Drupal 8: lessons learned from transforming red...
Building reusable websites on Drupal 8: lessons learned from transforming red...Building reusable websites on Drupal 8: lessons learned from transforming red...
Building reusable websites on Drupal 8: lessons learned from transforming red...
 
Building rednoseday.com on Drupal 8
Building rednoseday.com on Drupal 8Building rednoseday.com on Drupal 8
Building rednoseday.com on Drupal 8
 
Mapping in Drupal 7 using OpenLayers
Mapping in Drupal 7 using OpenLayersMapping in Drupal 7 using OpenLayers
Mapping in Drupal 7 using OpenLayers
 
Mapping in Drupal using OpenLayers
Mapping in Drupal using OpenLayersMapping in Drupal using OpenLayers
Mapping in Drupal using OpenLayers
 

Recently uploaded

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 

Recently uploaded (20)

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 

Importing and synchronizing content using Feeds