This document outlines the steps to build your own natural language processing (NLP) system, beginning with creating a streaming consumer, launching a message queue service, creating a data pre-processing service, serving an ML model, and publishing predictions to a messaging app. It discusses separating components for modularity and ease of testing/extensibility. The presenter recommends tools like Anaconda, Docker, Redis, Fast.ai and SpaCy and walks through setting up the environment and each step in a Jupyter notebook. The goal is to experiment with building your own end-to-end NLP system in a modular, reusable way.

1. @jeremimucha | https://create.ml7/12/2019
Build your own NLP system!
Michal Mucha, PyData London 2019

2. @jeremimucha | https://create.ml
Welcome!
Get ready to experiment
Golden Rule for Today: Try First, Study Second
Connect & collaborate with those around you!

3. @jeremimucha | https://create.ml
About me
Data Science and Data Engineering - consulting and training
Academic research (mobile phone data, smart meter data)
Commercial projects (decision simulation, revenue modeling,
visualization, building apps, data strategy)
Husband and dad
❤ boxing, cycling, hiking in the mountains ⛰ and traveling
Call me #$ Michael or % Me how 🙃

4. @jeremimucha | https://create.ml
Welcome!
Get ready to experiment
Rule for today: try first, study second
Connect & collaborate with those around you!

5. @jeremimucha | https://create.ml
High level steps
Create a Streaming Consumer
Launch and Integrate a Message Queue Service
Create the First Subscriber - a Data Pre-processing Service
Serve a Machine Learning Model
Publish or broadcast predictions to a Messaging App
Organize and bundle all services into a system

6. @jeremimucha | https://create.ml
Requirements
https://github.com/MichaMucha/pydata2019-nlp-system/
Software:
Anaconda Python
Git
Docker
Docker-compose
Telegram mobile app or desktop app
API keys and environment preparation
Check out this talk’s git repo
Create the Conda environment
Reddit CLIENT_ID and CLIENT_SECRET
Telegram Bot and API key
Voluntary - appreciated but not required:
Your own NLP model + Idea what you want to monitor in Reddit
Examine the conda-env.yml file that you used to create the new environment

7. @jeremimucha | https://create.ml
Benefits of Conda environments
Easy, self contained recipes
Installs binaries without building, no need for
dependencies
Makes shipping and sharing easier

8. @jeremimucha | https://create.ml
Step 1 - consumer
Navigate to the repository
Launch `jupyter lab`
Open the directory “step1”

9. @jeremimucha | https://create.ml
Step 1.1 - spawn Redis
Nice and clean - one line and we’re done
Not wasting time on things we don’t want to do!
Getting all the benefit

10. @jeremimucha | https://create.ml
Important idea
Separation of concerns
Modularity
Makes for easier…
Testing
Adding extensions
Monitoring
Teamwork

11. @jeremimucha | https://create.ml
Step 2 - Preprocessing
Open the directory “step2” in lab

12. @jeremimucha | https://create.ml
Step 3 - NLP models
BYOM today
Assumption:
your model is all trained and tested,
developed and signed off by important executives
Ready to use in the real world
Open “step3” in lab

13. @jeremimucha | https://create.ml
Important resources
https://fast.ai
Excellent course + framework
Releases the genius within you
https://spacy.io
Fantastic piece of engineering
Very widely used, open source

14. @jeremimucha | https://create.ml
Step 4 - beyond my lab

16. @jeremimucha | https://create.ml
Step 4 - beyond my lab
“Works on my machine” - o rly
ImportError - “just don’t move the files”
Another day another version
Dependency tracking

17. @jeremimucha | https://create.ml
Step 5 - Telegram
Go one extra step -
Make it easy for others to use your solutions!
Open “step5” in lab

18. @jeremimucha | https://create.ml
Step 6 - Orchestration
Making friends with the Operations team
Fast and easy prototyping
Configure and run sophisticated setups quickly
Build your own NLP system!

19. @jeremimucha | https://create.ml
Recap
What did you like most?
Write down three ideas to make it better!
Think of the one thing that you will take to your work

20. @jeremimucha | https://create.ml
Share your work!
Use your new knowledge to jumpstart your own solution
Please share what you built :)
Write a blog post!
Let’s stay in touch