Migrating data into Drupal using the migrate module presentation at Oxford Drupal Camp, 22 June 2012

1. Migrating data into Drupal
using the migrate module
Oxford DrupalCamp
Friday 22 June, 2012
Johan Gant

2. What's involved?
Migrate me an army of
data!
Uh, oh!!

3. What's involved?
Planning/analysis
Tech/dev work Acceptance
Done!

4. Successful migrations
Move data from A to B in a planned, methodical manner.
Repeatable
Measurable
Work with realistic data
Safe
Use the right tools for the task
The complexity of your solution should be driven by
the task at hand, not your tech

5. When stuff goes wrong...
Un...
Safe
Reliable
Planned
Done
Upset...
Processes
Clients
Developers
RAGEBALLS FOR EVERYONE

6. Drupal migrate module
Provides a great framework for moving content into Drupal
Encourages good habits
Drush support
migrate_ui adds a nice front end
The Drupal way – so other Drupal devs can
pick up your work without too much bother.
Not a one stop shop for migrations
Use the right tools, or combination of
tools, to get the job done.
Let's look at some code

7. <?php
class DemoMigration extends Migration {
public function __construct() {
parent::construct();
$this->description = t('A sample migration module');
// Define the map between your source and destination
$this->map = new MigrateSQLMap(
$this->machineName, // defaults to your migration class name
array(
'id' => array(
'type' => 'int',
....
), http://drupal.org/node/1528934
),
MigrateDestinationNode::getKeySchema();
);
// Define the fields from your source
$source_fields = array(
'id' => t('description'),
'field1' => t('Node title'),
.....
);
$query = db_select('source_table_name', 'sql-alias')
->fields('alias', array_keys($source_fields))
->orderBy('id', 'ASC');
$this->source = new MigrateSourceSQL($query);
// Define what sort of destination you have
// - options include Nodes, Terms, Users and Comments
$this->destination = new MigrateDestinationNode('node_machine_name');
// Define your field mappings - nice options include default values (can include PHP),
// groupings, callbacks – see beer.inc example class for details
$this->addFieldMapping('source_body', 'field_body')
->defaultValue('Wibble'),
.....
}
}

8. /**
* Useful function to help 'massage' your awkward source data into shape
*/
public function prepareRow($row) {
// Convert ISO date to UNIX timestamp
if ($row->created_at) {
$row->created_at = strtotime($row->created_at);
}
}
Handling taxonomy terms
/**
* Term syntax/format a bit awkward,depends on whether you're using tid or name.
* Migrate module will match on tid or name and handle the rest for you :)
*/
$this->addFieldMapping('source_col_name', 'terms')
->arguments(array('source_type' => 'name')), // use tid if you've got term ids
->separator('$$'); // Used to split long string of term names in source
Handling node reference fields
/**
* Not documented when I used migrate, but expects a nid
*/
if ($row->some-identifier) {
$row->reference_nid = my_own_function_to_lookup_a_nid($row->some_identifier);
}

9. Migration!
Rails/PostgreSQL > Drupal 6
Export data into CSV, import into Drupal db via table
wizard module
Pre-migration script to create image nodes
Run migration
Client demo and deployment
5 days from start to finish

10. Problems!
Steep(ish) learning curve
Documentation a bit sparse in places
Awkward handling of taxonomy,
node reference, and domain data
Pre-migration fudges
Performance

12. Should I use Drupal migrate?
Yes / absolutely / do it now Probably not
● Want to move a reasonable ● Low volume or low complexity
volume of data INTO Drupal from
MySQL/XML/JSON/CSV
● Not familiar with coding
● Have complex data mappings that
● Already got something that works
are best expressed well
programmatically ● Trying to move data OUT of Drupal
● Make best use of Drupal tools and to somewhere else
existing code
● Want to recycle code between
projects

13. Drupal migrate - resources
● Migrate module page -
http://drupal.org/project/migrate
● Economist migration (great summary) -
http://drupal.org/node/915102
● Torchbox Team Drupal blog -
http://drupalblog.torchbox.com/