The Elasticsearch Query DSL is a rich, flexible, powerful query language for full text and structured search, but with power comes complexity. Which of the 40 available queries should you use? What's a filter and when should you use it? How do you combine multiple filters, or multiple queries or queries with filters?
To most users, "relevance", and how it is affected by different queries, is a black box. Multi-field queries in particular can be difficult to get right if you don't understand how they work.
In this talk, I will explain the Query DSL from the ground up: how filters and queries use the inverted index to find matching documents, how the relevance score is calculated, how to combine the filter/query building blocks into complex statements. And finally, I will talk about the pitfalls of multi-field queries and how to avoid them.
1. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Clinton Gormley
@clintongormley
Elasticsearch Query DSL
… not just for wizards
2. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
3. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch.org/guide
4. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch
5. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch
• real-time
6. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch
• real-time
• distributed
7. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch
• real-time
• distributed
• search
8. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
elasticsearch
• real-time
• distributed
• search
• analytics
9. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
mapping
10. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
mapping
analysis
11. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
mapping
analysis query dsl
12. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
mapping
analysis query dsl
13. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
query dsl
14. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
query dsl
flexible, powerful
query language
15. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries
16. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries filters
17. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries
• relevance
filters
• boolean yes/no
18. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries
• relevance
• full text
filters
• boolean yes/no
• exact values
19. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries
• relevance
• full text
• not cached
filters
• boolean yes/no
• exact values
• cached
20. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries
• relevance
• full text
• not cached
• slower
filters
• boolean yes/no
• exact values
• cached
• faster
21. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
queries
• relevance
• full text
• not cached
• slower
filters
• boolean yes/no
• exact values
• cached
• faster
Filter first, then query remaining docs
22. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
23. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{...}
}
24. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"match":
{
"title":
"search"
}}
}
25. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
match_all:
{}}
}
26. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
match_all:
{}}
}
27. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"filtered":
{
"query":
{...},
"filter":
{...}
}
}
}
28. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"filtered":
{
"query":
{
"match":
{
"title":
"search"
}},
"filter":
{...}
}
}
}
29. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"filtered":
{
"query":
{
"match":
{
"title":
"search"
}},
"filter":
{
"term":
{
"status":
"active"
}}
}
}
}
30. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"filtered":
{
"query":
{
"match_all":
{}},
"filter":
{
"term":
{
"status":
"active"
}}
}
}
}
31. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"filtered":
{
"query":
{
"match_all":
{}},
"filter":
{
"term":
{
"status":
"active"
}}
}
}
}
32. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
how data is indexed
33. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
!
34. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
!
{
"title":
"Keeping
pets
healthy",
"content":
"My
quick
brown
fox
eats
rabbits
on
a
regular
basis"
}
35. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
!
{
"title":
"Keeping
pets
healthy",
"content":
"My
quick
brown
fox
eats
rabbits
on
a
regular
basis"
}
where content like
“%brown%fox%”
36. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
!
{
"title":
"Keeping
pets
healthy",
"content":
"My
quick
brown
fox
eats
rabbits
on
a
regular
basis"
}
slow & inflexible
37. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
!
{
"title":
"Keeping
pets
healthy",
"content":
"My
quick
brown
fox
eats
rabbits
on
a
regular
basis"
}
“analysis”
38. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
!
{
"title":
"Keeping
pets
healthy",
"content":
"My
quick
brown
fox
eats
rabbits
on
a
regular
basis"
}
39. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
[quick,brown,rabbits],
"content":
[brown,rabbits,are,commonly,seen]
}
!
{
"title":
[keeping,pets,healthy],
"content":
[my,quick,brown,fox,eats,rabbits,on,a,
regular,basis]
}
40. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc
1 Doc
2
a
are
basis
brown
commonly
eats
fox
my
on
quick
rabbits
regular
seen
field: content
sorted list of
unique terms
41. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc
1 Doc
2
a
are
basis
brown
commonly
eats
fox
my
on
quick
rabbits
regular
seen
field: content
where
they
occur
42. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc
1 Doc
2
a
are
basis
brown
commonly
eats
fox
my
on
quick
rabbits
regular
seen
field: content
43. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc
1 Doc
2
a
are
basis
brown
commonly
eats
fox
my
on
quick
rabbits
regular
seen
field: content
44. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
inverted index
45. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
inverted index
not just for text
46. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
inverted index
numbers, dates, bools, enums
geopoints, geoshapes, etc
47. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE
field
=
"value"
48. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE
field
CONTAINS
"value"
49. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE
field
CONTAINS
"value"
term filter
50. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"term":
{
"title":
"brown"
}
WHERE
field
CONTAINS
"value"
51. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
!
!
!
!
}
}
52. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"filtered":
{
!
!
}
}
}
53. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"filtered":
{
"query":
{
...
},
"filter":
{
...
}
}
}
}
54. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"filtered":
{
"query":
{
"match_all":
{}
},
"filter":
{
...
}
}
}
}
55. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"filtered":
{
"query":
{
"match_all":
{}
},
"filter":
{
...
}
}
}
}
56. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"filtered":
{
"query":
{
"match_all":
{}
},
"filter":
{
"term":
{
"title":
"brown"
}}
}
}
}
57. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc
1 Doc
2
brown
healthy
keeping
pets
quick
rabbits
field: title
58. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc
1 Doc
2
brown
healthy
keeping
pets
quick
rabbits
field: title
59. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"term":
{
"title":
"brown"
}
60. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"term":
{
"title":
"brown"
}
➔
result:
bitset[
1,
0
]
61. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"term":
{
"title":
"brown"
}
➔
result:
bitset[
1,
0
]
➔
cache
as:
"title:brown"
62. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE
field
IN
["val",…]
63. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE
field
IN
["val",…]
terms filter
64. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"terms":
{
"title":
["quick",
"pets"]
}
WHERE
field
IN
["val",…]
65. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc
1 Doc
2
brown
healthy
keeping
pets
quick
rabbits
field: title
66. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"terms":
{
"title":
["quick","pets"]
}
➔
result:
bitset[
1,
1
]
➔
cache
as:
"title:quick
title:pets"
67. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE
field
>=
"val1"
AND
field
<
"val2"
68. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE
field
>=
"val1"
AND
field
<
"val2"
range filter
69. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range":
{
"content":{
"gte":
"a",
"lt":
"m"
}
}
WHERE
field
>=
"val1"
AND
field
<
"val2"
70. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term Doc
1 Doc
2
a
are
basis
brown
commonly
eats
fox
my
on
quick
rabbits
regular
seen
field: content
71. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
➔
result:
bitset[
1,
1
]
➔
cache
as:
"content:[a
TO
m}"
"range":
{
"content":{
"gte":
"a",
"lt":
"m"
}
}
72. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range":
{
"date":{
"gte":
"2014-‐01-‐01",
"lt":
"2041-‐02-‐01"
}
}
73. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range":
{
"date":{
"gte":
"2014-‐01-‐01",
"lt":
"2041-‐02-‐01"
}
}
numeric/date fields
optimised
for range filters
74. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range":
{
"date":{
"gte":
"now
-‐
1h"
}
}
75. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range":
{
"date":{
"gte":
"now
-‐
1h"
}
}
not cached
76. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"range":
{
"date":{
"gte":
"now
-‐
1h
/
h"
}
}
cached
77. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE
field
IS
NOT
NULL
78. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE
field
has
any
term
79. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
WHERE
field
has
any
term
exists filter
80. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"exists":
{
"field":
"title"
}
WHERE
field
has
any
term
81. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
missing filter
WHERE
field
has
no
term
82. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"missing":
{
"field":
"title"
}
WHERE
field
has
no
term
83. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
AND
…
OR
…
NOT
84. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
AND
…
OR
…
NOT
bool filter
85. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
[
<filters>
],
"should":
[
<filters>
],
"must_not":
[
<filters>
]
}
86. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
[
<filters>
],
#
AND
"should":
[
<filters>
],
"must_not":
[
<filters>
]
}
87. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
[
<filters>
],
"should":
[
<filters>
],
#
OR
"must_not":
[
<filters>
]
}
88. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
[
<filters>
],
"should":
[
<filters>
],
"must_not":
[
<filters>
]
#
NOT
}
89. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
{
"term":
{
"title":
"rabbits"
}},
!
!
!
!
!
}
95. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
result
bitset
=
!
(title:rabbits
OR
content:rabbits)
AND
(title:quick
OR
content:quick)
AND
NOT
content:fox
96. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
filters
• boolean yes/no
• exact values
• cached
• faster
Filter first, then query
97. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
how relevant is this term?
98. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
how relevant is this term?
term query
99. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
≈ term filter + relevance
how relevant is this term?
term query
100. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"term":
{
"title":
"brown"
}
how relevant is this term?
101. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
!
!
!
}
}
102. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
GET
/_search
{
"query":
{
"term":
{
"title":
"brown"
}
}
}
103. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"_index":
"myindex",
"_type":
"mytype",
"_id":
"1",
"_score":
0.5,
"_source":
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
}
how relevant is this doc?
104. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
relevance score
105. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
relevance score
How common is the term in this doc?
➔ more is better
106. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
relevance score
How common is the term in this doc?
➔ more is better
How common is the term in ALL docs?
➔ less is better
107. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
relevance score
How common is the term in this doc?
➔ more is better
How common is the term in ALL docs?
➔ less is better
How long is this doc?
➔ shorter is better
108. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
lucene similarity
How common is the term in this doc?
➔ more is better
How common is the term in ALL docs?
➔ less is better
How long is this doc?
➔ shorter is better
109. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
lucene similarity
How common is the term in this doc?
➔ more is better
How common is the term in ALL docs?
➔ less is better
How long is this doc?
➔ shorter is better
110. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term frequency
!
How common is the term in ALL docs?
➔ less is better
How long is this doc?
➔ shorter is better
lucene similarity
111. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term frequency
!
How common is the term in ALL docs?
➔ less is better
How long is this doc?
➔ shorter is better
lucene similarity
112. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term frequency
!
Inverse document frequency
!
How long is this doc?
➔ shorter is better
lucene similarity
113. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term frequency
!
Inverse document frequency
!
How long is this doc?
➔ shorter is better
lucene similarity
114. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
Term frequency
!
Inverse document frequency
!
Length norm
lucene similarity
115. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
AND
…
OR
…
NOT
116. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
AND
…
OR
…
NOT
bool query
117. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
like bool filter, but different...
AND
…
OR
…
NOT
bool query
118. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
[
<queries>
],
#
AND
"should":
[
<queries>
],
"must_not":
[
<queries>
]
#
NOT
}
119. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
[
<queries>
],
#
AND
"should":
[
<queries>
],
#
Hmmm
"must_not":
[
<queries>
]
#
NOT
}
120. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
[
<queries>
],
#
AND
"should":
[
<queries>
],
#
Hmmm
"must_not":
[
<queries>
],
#
NOT
"minimum_should_match":
?
#
Hmmm
}
127. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bool
filter
➔
T/F
!
128. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bool
filter
➔
T/F
!
bool
query
➔
_score
129. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
_score
of
bool
query
=
130. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
_score
of
bool
query
=
sum(
_score
of
each
query)
131. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
_score
of
bool
query
=
sum(
_score
of
each
query)
*
num
of
matching
queries
132. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
_score
of
bool
query
=
sum(
_score
of
each
query)
*
num
of
matching
queries
/
num
of
queries
133. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
more matching should queries
134. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
more matching should queries
==
better relevance score
135. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
trim the long tail
"bool":
{
"should":
[
{
"term":
{
"title":
"quick"
}},
{
"term":
{
"title":
"brown"
}},
{
"term":
{
"title":
"rabbits"
}}
]
}
136. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
trim the long tail
"bool":
{
"should":
[
{
"term":
{
"title":
"quick"
}},
{
"term":
{
"title":
"brown"
}},
{
"term":
{
"title":
"rabbits"
}}
],
"minimum_should_match":
"75%"
}
137. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
trim the long tail
"bool":
{
"should":
[
{
"term":
{
"title":
"quick"
}},
{
"term":
{
"title":
"brown"
}},
{
"term":
{
"title":
"rabbits"
}}
],
"minimum_should_match":
"75%"
#
2
of
3
}
138. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
match query
139. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
match query
high level query
140. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
match query
high level query
understands mapping & analysis
141. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
match query
➔ analyze query string
➔ rewrite query
142. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
one word query
143. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"match":
{
"title":
"QUICK!"
}}
144. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"match":
{
"title":
"QUICK!"
}}
title:quick
145. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"match":
{
"title":
"QUICK!"
}}
title:quick
{
"term":
{
"title":
"quick"
}}
146. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
multi word query
147. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"match":
{
"title":
"QUICK
FOX!"
}}
148. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"match":
{
"title":
"QUICK
FOX!"
}}
title:quick
OR
title:fox
159. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
fuzzy queries
160. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
fuzzy queries
levenshtein edit distance
161. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bron
➔
brown
insertion
162. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bron
➔
brown
foxs
➔
fox
deletion
163. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bron
➔
brown
foxs
➔
fox
kiuck
➔
qiuck
substitution
164. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
bron
➔
brown
foxs
➔
fox
kiuck
➔
qiuck
➔
quick
transposition
165. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"match":
{
"title":
{
"query":
"KIUCK
BRON
FOXS!",
"fuzziness":
"AUTO"
}
}
166. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
phrase / proximity
167. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"match_phrase":
{
"title":"QUICK
BROWN
FOX!"
}
}
168. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"match_phrase":
{
"title":
{
"query":
"BROWN
QUICK
FOX!",
"slop":
"10"
}
}
}
169. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
combine queries
170. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
{
<min_should_match>
},
"should":
[
{
<fuzzy>
},
{
<proximity>
}
]
}
171. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
{
<min_should_match>
},
"should":
[
{
<fuzzy>
},
{
<proximity>
}
]
}
172. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
!
"match":
{
"title":
{
"query":
"<words>",
"minimum_should_match":
"75%"
}
}
!
173. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
{
<min_should_match>
},
"should":
[
{
<fuzzy>
},
{
<proximity>
}
]
}
174. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
!
"match":
{
"title":
{
"query":
"<words>",
"fuzziness":
"AUTO"
}
}
!
175. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
{
<min_should_match>
},
"should":
[
{
<fuzzy>
},
{
<proximity>
}
]
}
176. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
!
"match_phrase":
{
"title":
{
"query":
"<words>",
"slop":
"10"
}
}
!
177. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"must":
{
<min_should_match>
},
"should":
[
{
<fuzzy>
},
{
<proximity>
}
]
}
178. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
multi-field queries
179. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
180. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
easy!
182. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
183. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
hard!
184. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"should":
[
{
"match":
{
"title":
"quick
brown
fox"
}},
{
"match":
{
"content":
"quick
brown
fox"
}}
]
}
185. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
{
"title":
"Keeping
pets
healthy",
"content":
"My
quick
brown
fox
eats
rabbits
on
a
regular
basis"
}
186. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
{
"title":
"Keeping
pets
healthy",
"content":
"My
quick
brown
fox
eats
rabbits
on
a
regular
basis"
}
better match
187. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
{
"title":
"Keeping
pets
healthy",
"content":
"My
quick
brown
fox
eats
rabbits
on
a
regular
basis"
}
But 2 matches wins
188. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
dis_max query
189. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
all docs which match any query
dis_max query
190. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
all docs which match any query
dis_max query
_score
=
best
matching
query
191. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"bool":
{
"should":
[
{
"match":
{
"title":
"quick
brown
fox"
}},
{
"match":
{
"content":
"quick
brown
fox"
}}
]
}
192. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"dis_max":
{
"queries":
[
{
"match":
{
"title":
"quick
brown
fox"
}},
{
"match":
{
"content":
"quick
brown
fox"
}}
]
}
193. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
{
"title":
"Keeping
pets
healthy",
"content":
"My
quick
brown
fox
eats
rabbits
on
a
regular
basis"
}
194. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"title":
"Quick
brown
rabbits",
"content":
"Brown
rabbits
are
commonly
seen"
}
{
"title":
"Keeping
pets
healthy",
"content":
"My
quick
brown
fox
eats
rabbits
on
a
regular
basis"
}
195. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"dis_max":
{
"queries":
[
{
"match":
{
"title":
"quick
brown
fox"
}},
{
"match":
{
"content":
"quick
brown
fox"
}}
],
"tie_breaker":
0.2
}
196. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
all docs which match any query
dis_max query
_score
=
best
matching
query
+
tie_breaker
*
others
197. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
multi_match query
198. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
match query on multiple fields
multi_match query
199. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"dis_max":
{
"queries":
[
{
"match":
{
"title":
"quick
brown
fox"
}},
{
"match":
{
"content":
"quick
brown
fox"
}}
],
"tie_breaker":
0.2
}
200. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match":
{
"query":
"quick
brown
fox",
"fields":
[
"title",
"content"
]
"tie_breaker":
0.2
}
201. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match":
{
"query":
"quick
brown
fox",
"fields":
[
"title",
"content"
]
"tie_breaker":
0.2,
#
"type":
"best_fields"
}
202. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
best_fields
203. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
find whole "concept" in one field
best_fields
204. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"quick brown fox" in title or content
best_fields
205. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
best_fields
dis_max
"quick brown fox" in title or content
206. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"title":
{
"type":
"string"
}
207. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"title":
{
"type":
"string",
"fields":
{
!
!
!
!
!
!
!
}}
213. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match":
{
"query":
"quick
brown
fox",
"fields":
[
"title",
"title.stemmed",
"title.autocomplete"
]
"type":
"most_fields"
}
214. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
most_fields
215. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
most_fields
match same text
analyzed in different ways
216. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
most_fields
more matching fields = better
217. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
most_fields
bool
more matching fields = better
218. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
{
"first":
"Reginald",
"middle":
"Kenneth"
"last":
"Dwight"
}
219. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match":
{
"query":
"Reginald
Kenneth
Dwight",
"fields":
[
"first",
"middle",
"last"
]
!
}
220. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match":
{
"query":
"Reginald
Kenneth
Dwight",
"fields":
[
"first",
"middle",
"last"
]
"type":
"????"
}
221. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match":
{
"query":
"Reginald
Kenneth
Dwight",
"fields":
[
"first",
"middle",
"last"
]
"type":
"most_fields"
}
222. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match":
{
"query":
"Reginald
Kenneth
Dwight",
"fields":
[
"first",
"middle",
"last"
]
"type":
"most_fields"
}
223. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
224. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
field centric
225. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
(
first:
Reginald
OR
first:
Kenneth
OR
first:
Dwight
)
OR
(
middle:Reginald
OR
middle:Kenneth
OR
middle:Dwight
)
OR
(
last:
Reginald
OR
last:
Kenneth
OR
last:
Dwight
)
226. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
(
first:
Reginald
OR
first:
Kenneth
OR
first:
Dwight
)
OR
(
middle:Reginald
OR
middle:Kenneth
OR
middle:Dwight
)
OR
(
last:
Reginald
OR
last:
Kenneth
OR
last:
Dwight
)
227. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
228. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
operator: and
229. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
(
first:
Reginald
AND
first:
Kenneth
AND
first:
Dwight
)
OR
(
middle:Reginald
AND
middle:Kenneth
AND
middle:Dwight
)
OR
(
last:
Reginald
AND
last:
Kenneth
AND
last:
Dwight
)
230. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
(
first:
Reginald
AND
first:
Kenneth
AND
first:
Dwight
)
OR
(
middle:Reginald
AND
middle:Kenneth
AND
middle:Dwight
)
OR
(
last:
Reginald
AND
last:
Kenneth
AND
last:
Dwight
)
231. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
232. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
term frequencies
233. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
term frequencies
first:dwight
➔
common
234. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
problem
term frequencies
first:dwight
➔
common
last:
dwight
➔
uncommon
235. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
solution
236. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
index time solution
237. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
index time solution
single "fullname" field
238. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"first":
{
"type":
"string"
},
"middle":
{
"type":
"string"
},
"last":
{
"type":
"string"
}
!
241. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"match":
{
"full":
"Reginald
Kenneth
Dwight",
"minimum_should_match":
"75%"
}
242. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
query time solution
243. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
query time solution
term-centric query
244. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
(
first:
Reginald
AND
first:
Kenneth
AND
first:
Dwight
)
OR
(
middle:Reginald
AND
middle:Kenneth
AND
middle:Dwight
)
OR
(
last:
Reginald
AND
last:
Kenneth
AND
last:
Dwight
)
245. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
(
first:
Reginald
OR
middle:Reginald
OR
last:
Reginald
)
AND
(
first:
Kenneth
OR
middle:Kenneth
OR
last:
Kenneth
)
AND
(
first:
Dwight
OR
middle:Dwight
OR
last:
Dwight
)
246. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
blend(first,middle,last):Reginald
AND
blend(first,middle,last):Kenneth
AND
blend(first,middle,last):Dwight
247. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
blend(first,middle,last):Reginald
AND
blend(first,middle,last):Kenneth
AND
blend(first,middle,last):Dwight
blends term frequencies
248. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
cross_fields
249. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
cross_fields
query multiple fields
as if they were one
250. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
"multi_match":
{
"query":
"Reginald
Kenneth
Dwight",
"fields":
[
"first",
"middle",
"last"
]
"type":
"cross_fields"
}
252. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
best_fields:
whole concept in single field
most_fields:
same text, different analyzers
cross_fields:
treat multiple fields as one
253. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
understand the
building blocks
254. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
the rest is details
255. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
thank you
@clintongormley
256. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
thank you
@clintongormley
elasticsearch.org/downloads
elasticsearch.com/support
elasticsearch.com/jobs