This is Onely's Bartosz Goralewicz's deck "Too Long; Didn’t Render - The State of JS and HTML Indexing" as presented at Digital Growth Unleashed in London on October 16, 2019.
13. Even after… Months of
publishing content.
It is 2019 and our
research shows that
there are hundreds
of thousands of
domains not fully
indexed.
14. How rendering works with Google
So, you look at the difference
between the initial HTML, and,
then, if after rendering you
see extra content?
Yeah.
15. There are certain heuristics, that,
if we see after a while, like, oh, this
page, actually, the renderer does not
diff as much or doesn’t diff, it looks
the way before
The Heuristics
I still haven’t fully, like, grasped
what exactly triggers the
heuristics
35. What usually relies on JavaScript?
01
Pagination
02
“You might be also
interested in” (internal linking)
03
Top products
04
Reviews
05
Comments
06
Main content (rarely)
36. Web development evolved. Most websites use ready-to-go components
Slider
Faceted navigation
Related Products
Comments
Main content
Menu
The roots of the problem
63. But other parts of technical SEO are important too.
JavaScript SEO is important.
Sample of
1300 URLs
100%
64. But other parts of technical SEO are important too.
JavaScript SEO is important.
Sample of
1300 URLs
100%
Number of pages indexed after 1 day:
98.5%
65. But other parts of technical SEO are important too.
JavaScript SEO is important.
Sample of
1300 URLs
100%
Number of pages indexed after 1 day:
98.5%
What about
the rest?
66. Some websites
aren't as lucky as The Guardian.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Indexed after 1 day Indexed after 2 days Indexed after 1 week Indexed after 2 weeks
The Guardian Target Reuters Eventbrite
98.5%
3.5%
68.5%
32%
99.9%
33%
88%
47%
99.9%
78%
91%
57%
99.9%
84%
91%
58%
68. Parts of
your website
depend
on JS
01
Webmaster updates
with new products
02
Google crawls your
website without
seeing all the links
69. Parts of
your website
depend
on JS
01
Webmaster updates
with new products
02
Google crawls your
website without
seeing all the links
03
Only crawls a part of the
domain without finding
products (valuable content)
05
Budget is too
low to render JS
04
Google indexer is
confused - crawler
budget falls
70. Which leads to this…
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Indexed after 1 day Indexed after 2 days Indexed after 1 week Indexed after 2 weeks
The Guardian Target Reuters Eventbrite
98.5%
3.5%
68.5%
32%
99.9%
33%
88%
47%
99.9%
78%
91%
57%
99.9%
84%
91%
58%
71. TO DO:
Go to onely.com/tools
and check your website
(100% free)
Crawl your website with
and without JS and
compare two crawls