A tour through the history of neural nets, and how the birth of neural nets is coupled with the birth of regular expressions.

1. Neural Nets
How Regular Expressions brought about Deep Learning

2. For millennia people have asked

3. Why?

4. What is mind?
What is consciousness?
How does my brain work?

5. We need a new science!

7. Exploring how neurons
work since 1890s™

8. Guillaume-Benjamin-Amand Duchenne (de Boulogne)

9. Fast forward to 1943,
just after

11. After Turing

12. After Turing

13. After Turing

14. in the USA

15. McCulloch
Neurosci
Pitts
Logician

16. published

17. During WW2

18. which tried to understand

19. During WW2

20. by modelling the neurons

22. they didn’t know it but...

23. Their paper had an enormous
impact on computer science

24. Shortly after in 1951

25. In 1956
Stephen Kleene

26. set out to prove what

28. could compute in

29. could compute

30. In 1956

31. proposed a
simple algebra

32. Kleene proposed a
simple algebra

33. (well, simple for algebra)

34. that defined a

35. Regular Language

36. WTF’s a Regular Language?

37. Let’s start with

38. an alphabet

39. {0,1,2,3,4,5,6,7,8,9}

40. some words

41. 123
012334
23848484

42. a language

43. A language

44. 004
005
007
0012
089

45. But what about a
Regular Language?

46. If you can decide if a “word”
is in a “language” by inspecting
its sequential “letters” in isolation

47. like

48. a bit like...

49. then it’s a
Regular Language

50. Isn’t everything regular?

51. Nope

52. for example

53. Palindromes are not Regular

54. aa
abba
abccba
abcddcba
abcdeedcba

55. So we call

56. a bit like...

57. Finite State Machine
or
Deterministic Finite Automaton

58. In 1956

59. proved that

60. a bit like...

61. is equivalent to

63. McCulloch
Neurosci
Pitts
Logician

64. He showed a Regular Language
can be defined using a
grammar with 3 operators

65. Concatenation
{"ab", "c"}{"d", "ef"}
= {"abd", "abef", "cd", "cef"}

66. Alternation
{"ab", "c"}|{"ab", "d", "ef"}
= {"ab", "c", "d", "ef"}

67. Kleene Star
{"ab", "c"}*
= {"", "ab", "c", "abab", "abc", "cab", "cc",
"ababab", "abcab", ... }.

68. Look familiar?

69. TADA!

71. A Regular Expression
defines a Regular Language

72. 00(1|2|3|4|5|6|7|8|9)(0|1|2|3|4|5|6|7|8|9)*

73. 00[1-9][0-9]*

74. 00[1-9]d*

75. But hold up….

76. In 1956

77. didn’t mention computers in

78. could compute

79. he was considering

80. During WW2

81. so how did they end up
central to Computer Science?

82. Fast-forward to 1968

83. Ken Thomson

84. published

85. published

86. and introduced Regular
Expressions in his port of the
editor QED for CTSS

87. and the next year

89. wrote “ed”

90. Fast-forward to 1968

91. which is still part of
*nux today

92. It has the command

93. g/{some command}/p

94. g/re/p

95. grep

96. Then in the 80s

97. Larry Wall

98. created Perl

99. It has a further extended syntax

100. with features like

101. capturing subgroups
(d{5})(?:-(d{4}))?

102. backrefs
(.*)1

103. lookarounds
(?<=a)bcd

104. recursion
(w)(?:(?R)|w?)1

105. which is more
expressive and flexible

106. so this is the default in
Perl, Java, JS, Ruby, PHP, etc.
(*features may be missing from some implementations)

107. These RegEx aren’t always
equivalent to a

108. a bit like...

109. for example: Palindromes

110. in fact

111. in fact in the
Noam Chomsky

112. defined a hierarchy that
splits up the languages

113. Perl “RegEx” covers Context-Free Languages
(and some other ones too)

114. so...

115. “RegEx” ⊃ Regular Expression

116. What about today?

117. RegEx is the workhorse
of data extraction

118. from
Log collation
to
Web Scraping

119. infact

121. may generate a RegEx
when you train extractors

122. We wanted to try making this
more accessible, so we put it up
as a labs API

123. regexpgen.import.io

124. Let’s check it out...

125. Remember I said about

126. McCulloch
Neurosci
Pitts
Logician

127. Their paper had an enormous
impact on computer science

128. Even though they weren’t thinking
about computers at all

129. That paper not only
gave rise to RegEx

130. but also

131. every neural network since,
including

132. “Deep Learning”

134. using technology such as

135. Recurrent Neural Nets

136. using technology such as

137. using technology such as

138. and of course I have to mention

140. In fact at

142. we use neural nets for complex
data extraction tasks

143. where a rule based system, or
even ML, is simply not practical

144. such as

145. working out the name of a
company from their website

146. Let’s have a look...

147. I want to leave you with
a final thought...

148. When you are training an
extractor

149. and it generates a RegEx

150. that’s a Regular Expression

151. it’s defining a Regular Language

152. which can be detected by a
Finite State Machine

153. which is equivalent to a
a McCulloch-Pitts neural net

154. which blows my mind

155. Thank you
matthew.painter@import.io