This document discusses site crawling tools and techniques. It provides information on various crawling tools, their uses, and pricing. Key things to look for when crawling a site include errors, redirects, duplicate titles and content, canonicals, image alt text, and PDF files. The document recommends crawling sites at minimum every quarter, before and after major updates, and if there are changes in organic traffic. Crawling can be used to test sitemaps, backlinks, and conduct content inventories.
7. @katemorris
Moz
Best For:
• In-House Marketers
• Beginning SEOs
Cost: $99/month+
User Score: 1-2
8. @katemorris
Raven
Best For:
• In-House Marketers
• Small Agencies
Cost: $99/month+
User Score: 3
9. Xenu Link Sleuth
@katemorris
Best For:
• Technical Experts
• Developers
Cost: Free
User Score: 4
10. Screaming Frog
@katemorris
Best For:
• Technical Marketers
• SEOs
• Developers
Cost: $160/year
User Score: 5
11. @katemorris
Deep Crawl
Best For:
• Expert SEOs
• Sites with Large
Footprints
Cost: $80/ month+
User Score: 6
12. Crawler User Scale
1 2 3 4
@katemorris
Unrelated to
SEO/Marketing
Novice Level
SEO
Knowledge,
Good SEO
Knowledge,
Non-Technical
Good
Technical,
Minimal SEO
5 6 7
Good SEO and
Technical
Knowledge
Expert Level SEO
Knowledge,
Minimal
Technical
Expert Technical
and SEO
Knowledge
13. Crawler User Scale
@katemorris
Unrelated to
SEO/Marketing
Novice Level
SEO
Knowledge,
Good SEO
Knowledge,
Non-Technical
Good
Technical,
Minimal SEO
Good SEO and
Technical
Knowledge
Expert Level SEO
Knowledge,
Minimal
Technical
BUILD
YOUR
OWN
Expert Technical
and SEO
Knowledge
35. Testing Backlinks
Issue: Backlink Analysis is time consuming,
especially checking backlink lists for live
links.
Solution: Make a text list of all the backlinks
and crawl. For Screaming Frog, set a
custom search for the client site.
@katemorris
41. @katemorris
Do This
Download and find your favorite crawler.
Make a calendar entry that reoccurs every
three months to remind you to crawl your
site.
At minimum, review:
– Errors
– Redirects
– Duplicated Titles
– XML Sitemaps