Slide from the Helsinki Clojure meetup. June 2022

1. How we use Instaparse
Clojure meetup Helsinki, 21/06/2022
Julien Bille

2. Agenda
● Who I am?
● What is the problem?
● What is Instaparse?
● How we used it?

3. Who I am?
{:name "Julien Bille"
:from "France"
:living-in "The Netherlands"
:working-for "IPRally"
:job-title "Fullstack Developer"
}

4. IPRally
A search engine for patent
powered by unique AI knowledge
graph technology.

6. What is the problem?
Add boolean support to our product:
- Validate the syntax
- Being compatible with the most popular syntax (no standard)
- Query our database
- Prevent malicious query
(TACD:( ("electric bike” OR "electric $w4 bike” OR electricbike) AND (photovoltaic
OR solar) AND cell* ) AND AUTHORITY:( US OR JP OR EP OR WO ) AND
PBD_Y:[2000 to *]) NOT ( ALL_AN:Gazelle )
An example:

7. Boolean expression UI

8. What is Instaparse?
Instaparse is a Clojure library aims to be the simplest
way to build context free grammar parsers in Clojure.
First version release in 2013
https://github.com/Engelberg/instaparse
CLJC compatible

9. Context free grammar?
In practical term we want:
(defn text->tree [text-input rules])
(defn tree->sql [tree])

10. An example
(def as-and-bs
(insta/parser
"S = AB*
AB = A B
A = 'a'+
B = 'b'+"))
(as-and-bs "aaaaabbbaaaabb")
[:S
[:AB [:A "a" "a" "a" "a" "a"] [:B "b" "b" "b"]]
[:AB [:A "a" "a" "a" "a"] [:B "b" "b"]]]

11. A solution
(TACD:( ("electric bike” OR "electric $w4 bike” OR electricbike) AND (photovoltaic OR
solar) AND cell* ) AND AUTHORITY:( US OR JP OR EP OR WO ) AND PBD_Y:[2000 to *])
NOT ( ALL_AN:Gazelle )
Split into smaller pieces:
● word: photovoltaic
● quote-word: “electric bike”
● field-word: T:elect*
● operation: AND, OR, NOT
● list : (a b c)
● field-list: TA:(a b c)

12. A solution
(def boolean-parser
(insta/parser
"S = (exp )*
exp = word | op | quote-word | list | fields-list
word = fields?#'[a-zA-Z0-9_*/$#?-]+'
op = <space>'OR'<space> | <space>'AND'<space> | <space>'NOT'<space>
quote-word = fields? <'"'> (word<space> )* <'"'>
list = <lparen> (exp )* <rparen>
fields-list = fields list
fields = field <':'><space>
field = 'T' | 'TA' | 'A'
<lparen> = <'('><space>
<rparen> = <space><')'>
<space> = <#'[ ]*'>"
))

13. Examples
(boolean-parser "electric")
[:S [:exp [:word "electric"]]]
(boolean-parser "(electric)")
[:S [:exp [:list [:exp [:word "electric"]]]]]
(boolean-parser "(electric OR bike)"
)
[:S [:exp [:list
[:exp [:word "electric"]]
[:exp [:op "OR"]]
[:exp [:word "bike"]]]]]

14. Examples
(boolean-parser "(electric")
Parse error at line 1, column 10:
(boolean-parser "T:electric")
[:S [:exp [:word [:fields [:field "T"]] "electric"]]]
(boolean-parser "TA:electric")
[:S [:exp [:word [:fields [:field "TA"]] "electric"]]]

15. Examples
(boolean-parser "TAC:electric")
Parse error at line 1, column 4:
TAC:electric
^
Expected one of:
"A"
"TA"
"T"
"("
"""
"NOT"
"AND"
"OR"
#"[a-zA-Z0-9_*/$#?-]+"

16. Example
(boolean-parser "elec*")
[:S [:exp [:word "elec*"]]]
(boolean-parser "(TA:("electric bike" OR "electric $w4 bike"))"
)
[:S [:exp [:list [:exp [:fields-list
[
:fields [:field "TA"]]
[
:list
[
:exp [:quote-word [:word "electric"] [:word "bike"]]]
[
:exp [:op "OR"]]
[
:exp [:quote-word [:word "electric"] [:word "$w4"] [:word
"bike"]]]]]]]]]

17. Learning
1. Leaving room for human typo
2. Recursion comes for free
3. Don’t make it too complicated: used insta/transform, clojure.walk or
clojure.zip instead

18. Transform example
(->> (boolean-parser "(TA:("electric bike" OR "electric $w4
bike"))")
(insta/transform {
:quote-word transform-quote-word}))
[:S [:exp [:list [:exp [:fields-list
[
:fields [:field "TA"]]
[
:list
[
:exp [:quote-word {:words ("electric"
"bike")}]]
[
:exp [:op "OR"]]
[
:exp [:quote-word {:words ("electric" "$w4"
"bike")}]]]]]]]]

19. Questions?