How to drill down into big-site index and crawl problems fast, and the tools to try to solve those problems (by Dr. Pete - BlueGlassX 2012)

1. Big Site SEO Triage
Dr. Peter J. Meyers
Marketing Scientist
SEOmoz

2. Are you a real doctor?

3. Why I’m a believer…

4. My philosophy…
Keyword
Density*
*just kidding

5. My philosophy…
Q: Is there a dupe
content penalty?

6. My philosophy…
A: I don’t give a shit.

7. I. Look at the damned site

8. Your top 2 SEO tools

9. Scope out the architecture

10. Navigate the main links

11. Quick-count internal links

12. Check for pagination

13. Scan search filters, sorts…

14. II. Drill down quickly

15. Master the basics

16. Expert’s handiwork

17. Amateur’s handlesswork

18. Two useful combos
site: + intitle:
site: + inurl:

19. Quick duplicate check
site:example.com
intitle:”unique phrase”

20. Spot-check URL parameters
site:example.com
inurl:page=

21. Spot-check the architecture
site:example.com/menu

22. Real-world example
11M - site:example.com
8.4M - site:example.com/shop
6.7M - site:…/shop inurl:option

23. How reliable is site:?
(a) site:seomoz.org
(b) site:…/blog
(c) site:…/blog intitle:spam

24. Experiment results…
(a) 67,700 – 114,000
(b) 4,580 – 8,650
(c) 40 – 40

25. Don’t forget the basics
 Robots.txt
 HTTP headers
 HTML headers

26. III. Bring in the firepower

27. GWT – HTML Improvements

28. GWT – Index Status (Advanced)

29. GWT – Tiered XML sitemaps
I. Main pages
II. Categories
III. Products

30. Screaming Frog – URL parameters

31. SF – Track bad internal links

32. IV. Choose your weapon

33. Robots.txt
+ Don’t need dev access
+ Good for prevention
- Still can be dangerous
- Bad for removal

34. Meta noindex
+ Extremely flexible
+ Fairly easy to reverse
- Occasionally ignored

35. 301 redirects
+ Still very powerful
+ Impact users + search
- Difficult to reverse
- Google can get suspicious

36. Rel=canonical
+ Fast and powerful
+ Very flexible
- Easy to screw up

38. Rel=prev, Rel=next
+ Low-risk option
+ Allows pages to rank
- Difficult to implement

39. Rule #1 for page-level cues
They can’t see what
They don’t crawl

40. GWT parameter handling
+ Requires no site access
- Inconsistently applied
- Google-specific

41. Technical SEO isn’t dead

42. Thank You!
Dr. Peter J. Meyers
peter@seomoz.org
@dr_pete