Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Michal Mucha: Build and Deploy an End-to-end Streaming NLP Insight System | PyData London 2019

2,426 views

Published on

At this workshop, you will build your own messaging insights system - data ingestion from a live data source (Reddit), queueing, deploying a machine learning model, and serving messages with insights to your mobile phone!

Published in: Technology
  • Login to see the comments

Michal Mucha: Build and Deploy an End-to-end Streaming NLP Insight System | PyData London 2019

  1. 1. @jeremimucha | https://create.ml7/12/2019 Build your own NLP system! Michal Mucha, PyData London 2019
  2. 2. @jeremimucha | https://create.ml Welcome! Get ready to experiment Golden Rule for Today: Try First, Study Second Connect & collaborate with those around you!
  3. 3. @jeremimucha | https://create.ml About me Data Science and Data Engineering - consulting and training Academic research (mobile phone data, smart meter data) Commercial projects (decision simulation, revenue modeling, visualization, building apps, data strategy) Husband and dad ❤ boxing, cycling, hiking in the mountains ⛰ and traveling Call me #$ Michael or % Me how 🙃
  4. 4. @jeremimucha | https://create.ml Welcome! Get ready to experiment Rule for today: try first, study second Connect & collaborate with those around you!
  5. 5. @jeremimucha | https://create.ml High level steps Create a Streaming Consumer Launch and Integrate a Message Queue Service Create the First Subscriber - a Data Pre-processing Service Serve a Machine Learning Model Publish or broadcast predictions to a Messaging App Organize and bundle all services into a system
  6. 6. @jeremimucha | https://create.ml Requirements https://github.com/MichaMucha/pydata2019-nlp-system/ Software: Anaconda Python Git Docker Docker-compose Telegram mobile app or desktop app API keys and environment preparation Check out this talk’s git repo Create the Conda environment Reddit CLIENT_ID and CLIENT_SECRET Telegram Bot and API key Voluntary - appreciated but not required: Your own NLP model + Idea what you want to monitor in Reddit Examine the conda-env.yml file that you used to create the new environment
  7. 7. @jeremimucha | https://create.ml Benefits of Conda environments Easy, self contained recipes Installs binaries without building, no need for dependencies Makes shipping and sharing easier
  8. 8. @jeremimucha | https://create.ml Step 1 - consumer Navigate to the repository Launch `jupyter lab` Open the directory “step1”
  9. 9. @jeremimucha | https://create.ml Step 1.1 - spawn Redis Nice and clean - one line and we’re done Not wasting time on things we don’t want to do! Getting all the benefit
  10. 10. @jeremimucha | https://create.ml Important idea Separation of concerns Modularity Makes for easier… Testing Adding extensions Monitoring Teamwork
  11. 11. @jeremimucha | https://create.ml Step 2 - Preprocessing Open the directory “step2” in lab
  12. 12. @jeremimucha | https://create.ml Step 3 - NLP models BYOM today Assumption: your model is all trained and tested, developed and signed off by important executives Ready to use in the real world Open “step3” in lab
  13. 13. @jeremimucha | https://create.ml Important resources https://fast.ai Excellent course + framework Releases the genius within you https://spacy.io Fantastic piece of engineering Very widely used, open source
  14. 14. @jeremimucha | https://create.ml Step 4 - beyond my lab
  15. 15. @jeremimucha | https://create.ml Step 4 - beyond my lab “Works on my machine” - o rly ImportError - “just don’t move the files” Another day another version Dependency tracking
  16. 16. @jeremimucha | https://create.ml Step 5 - Telegram Go one extra step - Make it easy for others to use your solutions! Open “step5” in lab
  17. 17. @jeremimucha | https://create.ml Step 6 - Orchestration Making friends with the Operations team Fast and easy prototyping Configure and run sophisticated setups quickly Build your own NLP system!
  18. 18. @jeremimucha | https://create.ml Recap What did you like most? Write down three ideas to make it better! Think of the one thing that you will take to your work
  19. 19. @jeremimucha | https://create.ml Share your work! Use your new knowledge to jumpstart your own solution Please share what you built :) Write a blog post! Let’s stay in touch

×