Imagine having a big brand website with over 50% of its content not indexed by Google after weeks? Now imagine that the issue causing this severe decline in online visibility was self-induced? In the new digital marketing landscape, big brands losing money this way is shockingly normal.
This deck from a Searchmetrics webinar Bartosz Goralewicz did with Marcus Tober on December 4, 2019.
Want to watch the video? https://youtu.be/3XENVxVpyoo
2. 63% of Walmart’s pages are not indexed in Google.
H&M is losing 4,938,885 visits per month due to their indexing issues.
24% of Topshop products are not indexed in Google.
Now that I have your full attention – let’s begin J
3. Back in 2016 I published Hulu’s case study…
… that I wrote in SearchMetrics’ office in Berlin J
one.ly/hulu2016
9. In 2019
* This means that their content isn’t
visible with JavaScript switched off.
** this also means that Google
needs to render the page to see
that content.
Rendering is optional though.
of the websites use
JavaScript for their
crucial content.*80%
18. What usually relies on JavaScript?
01
Pagination
02
“You might be also
interested in” (internal linking)
03
Top products
04
Reviews
05
Comments
06
Main content (rarely)
19. The Disconnect
Google’s announcements this year didn’t help…
Statement:
Martin and John are nothing but helpful.
Google is a large organization though
and sometimes they make our life a little
bit too complex.
20. But… but 90% of the websites get
indexed in 5 seconds?
35. Web development evolved. Most websites use ready-to-go components
Slider
Faceted navigation
Related Products
Comments
Main content
Menu
The roots of the problem
43. Rendering is not Google’s
indexer default mode.
Rendering is triggered by... Heuristics
44. Rendering heuristics
So, you look at the difference
between the initial HTML, and,
then, if after rendering you
see extra content?
Yeah.
45. There are certain heuristics, that,
if we see after a while, like, oh, this
page, actually, the renderer does not
diff as much or doesn’t diff, it looks
the way before
The Heuristics
I still haven’t fully, like, grasped
what exactly triggers the
heuristics
79. Parts of
your website
depend
on JS
01
Webmaster updates
with new products
02
Google crawls your
website without
seeing all the links
03
Only crawls a part of the
domain without finding
products (valuable content)
05
Budget is too
low to render JS
04
Google indexer is
confused - crawler
budget falls
80. Which leads to this…
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Indexed after 1 day Indexed after 2 days Indexed after 1 week Indexed after 2 weeks
The Guardian Target Reuters Eventbrite
98.5%
3.5%
68.5%
32%
99.9%
33%
88%
47%
99.9%
78%
91%
57%
99.9%
84%
91%
58%
81. TO DO:
Go to onely.com/tools
and check your website
(100% free)
Crawl your website with
and without JS and
compare two crawls
82. We are going like: ”Oh, all right, we
are gonna skip rendering.” It is not
as frequently happening anymore.
(...) Many websites even if they do
not run JavaScript, they might still
go through the render phase,
because it doesn’t make a difference
as much
Crawling
and indexing
to come
together
I expect, eventually rendering,
crawling and indexing will come
closer together
83. Summary
Indexing your content
is one of the key
challenges for 2020
All indexing issues
we are seeing are
100% self-induced by
website owners
We need to redefine
how we are looking at
the crawler budget and
the factors affecting it
JavaScript SEO
= Technical SEO
84. To do:
1Closely monitor your
indexing stats (daily,
weekly, biweekly).
2
If your content relies on
JavaScript, monitor it closely
and make sure that:
• The code is light and efficient
(cheap to render)
• Your code doesn’t affect web
performance on low-end devices
3
Make sure that no
content within your
structure relies on
JavaScript (check all
the page types within
your domain).
4 Make sure that no meta data
is changed during rendering.
85. To do:
1Closely monitor your
indexing stats (daily,
weekly, biweekly).
2
If your content relies on
JavaScript, monitor it closely
and make sure that:
• The code is light and efficient
(cheap to render)
• Your code doesn’t affect web
performance on low-end devices
3
Make sure that no
content within your
structure relies on
JavaScript (check all
the page types within
your domain).
4 Make sure that no meta data
is changed during rendering.
Subscribe to Onely.com – be the
first one to know about our
crawl budget monitoring toolset.