Can Google properly crawl and index JavaScript? SEO Experiments - Results and findings
•
14 likes•27,412 views
Slides from my session at Brighton SEO 2017
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (19)
Similar to Can Google properly crawl and index JavaScript? SEO Experiments - Results and findings
Similar to Can Google properly crawl and index JavaScript? SEO Experiments - Results and findings (20)
More from Onely
More from Onely (8)
Recently uploaded
Recently uploaded (20)
Can Google properly crawl and index JavaScript? SEO Experiments - Results and findings
- 1. Bartosz Góralewicz ELEPHATE Can Google properly crawl and index JavaScript? SEO Experiments - Results and findings @bart_goralewicz https://www.linkedin.com/in/goralewicz/
- 2. Why is JavaScript SEO such a complex topic?
- 3. DATANO Why is JavaScript SEO such a complex topic?
- 4. DATANO Making JavaScript SEO easy
- 5. JS SEO
- 6. JS SEO
- 7. What is the problem with JavaScript? HTML JS
- 8. What is the problem with JavaScript? HTML HTML
- 9. What is the problem with JavaScript? HTML HTML
- 10. JavaScript
- 11. and JavaScript
- 12. JavaScript & SEO
- 13. 50 shades of JavaScript
- 14. *more about it later not as “forgiving” as HTML*
- 15. It has all with…
- 17. Reverse engineering algorithms is no longer an option. Times have changed. Today, as long as you're not blocking Googlebot from crawling your JavaScript or CSS files, we are generally able to render and understand your web pages like modern browsers. To reflect this improvement, we recently updated our technical Webmaster Guidelines to recommend against disallowing Googlebot from crawling your site's CSS or JS files. ele.ph/ajax
- 18. ele.ph/scotch
- 20. Hulu.com JavaScript & SEO backfire
- 21. -37%
- 25. Google and JavaScript – difficult relationship
- 26. The only solution… Java Script Experiment
- 27. Kamil Grymuza
- 34. Content! OK.
- 35. ele.ph/AI
- 38. Inspect element
- 39. View source
- 40. First JavaScript rendering issues
- 41. Timeout, GSC bug or real ?
- 42. /test/
- 44. JavaScript links were not parsed by Googlebot Google kept us waiting…
- 46. Fail?
- 47. Re-fetched all URLs via GSC
- 50. /html/test/ (duh!) /plain-inline/test/ /jquery-inline/test/ /jquery-ajax/test/ /vue/test/ /react-inline/test/ And… that’s it! /test/
- 51. Let’s start with JS frameworks created by
- 52. Version 1
- 53. Checklist Check if ”framework/test/” URL was crawled Search for unique content from framework’s page Is URL indexed by Google? Fetch and render test (GSC)
- 54. Fetch and render test (GSC)
- 55. Is URL indexed by Google?
- 56. Search for unique content from framework’s page
- 57. Check if ”framework/test/” URL was crawled
- 65. Inline vs. External JS makes a huge difference Not all JavaScript frameworks are crawled and indexed in the same way Mindblowing findings JS generated links aren’t always crawled
- 66. ele.ph/angularU Also… Regardless of what your developers say, Angular JS 2 always has to be server rendered… according to Angular JS 2 community.
- 67. If you search for any competitive keyword terms, it’s always going to be server rendered sites. And the reason is because although Google does index client-side rendered HTML, it’s not perfect yet and other search engines don’t do it as well. So if you care about SEO, you still need to have server-rendered content. Jeff Whelpley Angular U conference, June 22-25, 2015, Hyatt Regency, San Francisco Airport “Angular 2 Server Rendering” ele.ph/angularU
- 68. ?How about Search Engines other than
- 70. 63,4% 22,8% 11,7% 1,3% 0,8% Google Bing Yahoo! Ask AOL Search Engine market share Source: https://www.statista.com/statistics/267161/market-share-of-search-engines-in-the-united-states/
- 71. 83,5% 11,1% 3,8% 0,8% 0,6% Google Bing Yahoo! MSN DuckDuckGo Search Engine market share Source: https://www.statista.com/statistics/280269/market-share-held-by-search-engines-in-the-united-kingdom/
- 73. ele.ph/jsforum
- 74. Tell me why I can’t… Clear your mind of questions *not the actual conversation
- 78. http://jsseo.expert/plain-external/test/ http://jsseo.expert/react-external/test/ http://jsseo.expert/jquery-external/test/ we found this URL from a link ca 3 weeks ago, but we haven't crawled it we picked this one up ~19 days ago, and similar to above, our systems just didn't feel like crawling it yet picked up ~21 days ago, also not crawled yet.
- 79. Thanks, John! You’re the best, Bartosz!* *he didn’t say that John Mueller’s feedback got me thinking…
- 80. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce est. Vivamus a tellus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Proin pharetra nonummy pede. Mauris et orci. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. Nunc viverra imperdiet enim. Fusce JavaScript vs. crawler budget Some JavaScript frameworks are indexable, but there is a fine print…
- 81. JavaScript
- 82. BUT… HTML
- 83. Do you already know where I am going with this?
- 85. Kamil Grymuza
- 86. Crawler budget experiment (early beta)
- 87. We repeated this experiment a few times and the results were always the same.
- 88. JavaScript, even when indexable, can kill your crawler budget…
- 89. JS is less effective than HTML JavaScript is not bad for SEO, if done right* *he said exactly that Ilya Grigorik
- 91. ele.ph/ilya
- 92. ele.ph/chrome41
- 93. ele.ph/chrome41 Download Chrome 41 here:
- 94. Using Chrome 41 for debugging JS indexing
- 97. JavaScript SEO isn’t only affecting technical
- 98. At the same times those links count just as HTML ones. Most link indexes (e.g. Majestic, MOZ etc.) can’t crawl JavaScript generated backlinks.
- 100. Also… ...there is more JavaScript generated links out there than I would imagine (I’ll tell you more about it in a second)
- 101. Problems? Manual penalties Tracking mentions Google Penguin Link audits Link building
- 102. Crawling JavaScript is expensive…
- 103. To execute JS for every page at our scale [would] require 10,000-15,000 servers, and we believe our customers are not ready to pay for that yet” Dmitry Gerasimenko, CEO of Ahrefs ~ July 2017
- 104. No hope?
- 105. Ahrefs guys changed their mind after reading my article… ele.ph/js-ahrefs
- 106. ele.ph/js-ahrefs
- 107. …We accepted the challenge and crawled this same website with our JS crawler, which rendered all of the pages that were there. ele.ph/js-ahrefs
- 108. ele.ph/js-ahrefs
- 109. ele.ph/js-ahrefs So earlier this year Dmitry (our Founder & CEO) decided that Ahrefs has to be the first among its rivals to crawl JavaScript links. Which we achieved with ease last month.
- 110. JS ele.ph/js-ahrefs
- 111. JS ele.ph/js-ahrefs
- 115. JS LINKS PER DAY!
- 116. Is there a perfect JavaScript SEO scenario?
- 117. YES
- 118. The setup for that is called Isomorphic JavaScript on React & Universal on Angular. That's what I'd recommend doing in general, if you can. Isomorphic JavaScript ele.ph/Iso-John
- 119. ele.ph/iso
- 120. What if it’s too late?
- 121. Prerender.io
- 124. JS Only Google and Ask can handle JavaScript Key takeways
- 125. JS Only Google and Ask can handle JavaScript Not all JavaScript frameworks are indexable Key takeways
- 126. Not all JavaScript frameworks are indexable Indexing JavaScript is only half the battle Key takeways
- 127. Indexing JavaScript is only half the battle JavaScript even when rendered and indexed, can kill crawler budget Key takeways
- 128. JavaScript even when rendered and indexed, can kill crawler budget Use Chrome 41 to debug JS Key takeways
- 129. Use Chrome 41 to debug JS If too late, use prerendering Key takeways
- 130. If too late, use prerendering Key takeways Isomorphic JS = perfect scenario
- 131. Not all JavaScript frameworks are indexable Indexing JavaScript is only half the battle JavaScript even when rendered and indexed, can kill crawler budget Use Chrome 41 to debug JS JS Only Google and Ask can handle JavaScript If too late, use prerendering Key takeways Isomorphic JS = perfect scenario