Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using apache mx net in production deep learning streaming pipelines

106 views

Published on

As a Data Engineer I am often tasked with taking Machine Learning and Deep Learning models into production, sometimes in the cloud and sometimes at the edge. I have developed Java code that allows us to run these models at the edge and as part of a sensor/webcam/images/data stream. I have developed custom interfaces in Apache NiFi to enable real-time classification against MXNet models directly through the Java API or through DJL.AI's Java interface. I will demo running models on NVIDIA Jetson Nanos and NVIDIA Xavier NX devices as well as in the cloud.



# Technologies Utilized:

# Apache MXNet, DJL.AI, NVIDIA Jetson Nano, NVIDIA Jetson XAVIER, Apache NiFi, MiNIFi, Java, Python.

Published in: Software
  • Be the first to comment

Using apache mx net in production deep learning streaming pipelines

  1. 1. Using Apache MXNet in Production Deep Learning Streaming Pipelines Timothy Spann Principal DataFlow Field Engineer Cloudera @PaasDev
  2. 2. 2 Tim Spann Who am I? Cloudera Principal DataFlow Field Engineer @PaasDev DZone Zone Leader and Big Data MVB Future of Data Meetup Leader ex-Pivotal Field Engineer https://github.com/tspannhw https://www.datainmotion.dev/
  3. 3. 3 Welcome to Future of Data - Princeton - Virtual @PaasDev https://www.meetup.com/futureofdata-princeton/ From Big Data to AI to Streaming to Containers to Cloud to Analytics to Cloud Storage to Fast Data to Machine Learning to Microservices to ...
  4. 4. Apache MXNet Native Processor for Apache NiFi This is a beta, community release by me using the new beta Java API for Apache MXNet. https://github.com/tspannhw/nifi-mxnetinference-processor https://community.hortonworks.com/articles/229215/apache-nifi-processor-for-apache-mxnet-ssd-single.html https://www.youtube.com/watch?v=Q4dSGPvqXSA
  5. 5. • Great documentation • MXNet Models and Model Zoo • Fast Java • Action Recognition • Image Classification • Instance Segmentation • Object Detection • Pose Estimation • Q&A • Support also for PyTorch, Tensorflow and ONNX • Training https://djl.ai/ https://github.com/awslabs/djl/blob/master/mxnet/mxnet-model-zoo/README.md https://github.com/awslabs/djl/blob/master/mxnet/README.md https://d2l.djl.ai/ https://github.com/awslabs/djl/blob/master/docs/faq.md https://github.com/aws-samples/djl-demo https://github.com/awslabs/djl/releases/tag/v0.8.0 https://djl.ai/pytorch/pytorch-engine/
  6. 6. Apache MXNet Native Processor through DJL.AI for Apache NiFi This processor uses the DJL.AI Java Interface https://github.com/tspannhw/nifi-djl-processor https://dev.to/tspannhw/easy-deep-learning-in-apache-nifi-with-djl-2d79
  7. 7. DJL NiFi Processors - Sentiment Analysis https://www.datainmotion.dev/2020/09/using-djlai-for-deep-learning-based.html https://github.com/tspannhw/nifi-djlsentimentanalysis-processor probnegative 0.99 No value set probnegativeperc 99.44 No value set probpositive 0.01 No value set probpositiveperc 0.56 No value set rawclassification [class: "Negative", probability: 0.99440, class: "Positive", probability: 0.00559]
  8. 8. BERT QA through DJL.AI for Apache NiFi This processor uses the DJL.AI Java Interface https://github.com/tspannhw/nifi-djlqa-processor https://www.datainmotion.dev/2020/09/using-djlai-for-deep-learning-bert-q-in.html https://dev.to/tspannhw/easy-deep-learning-in-apache-nifi-with-djl-2d79 The pretrained model is DistilBERT model trained by HuggingFace using PyTorch. BERT QA
  9. 9. 9 https://blog.cloudera.com/benchmarking-nifi-performance-and-scalability/ NiFi Processing Billions of Events
  10. 10. Demo
  11. 11. 11 Edge AI to Cloud Streaming AI Pipeline Device Data SensorsEnergy Logs Weather Sensors Aggregates Energy SQL Analytics MiNiFi Agent Deep Learning Classification
  12. 12. Learn More
  13. 13. 13© 2020 Cloudera, Inc. All rights reserved. REFERENCES ● https://github.com/tspannhw/ApacheConAtHome2020 ● https://www.datainmotion.dev/2020/10/flank-streaming-edgeai-on-new-nvidia.html ● https://www.youtube.com/watch?v=h6mS08WDRHY&t=12s ● https://dev.to/tspannhw/using-djl-ai-for-deep-learning-based-sentiment-analysis-in-nifi-dataflow- 3c3a ● https://community.cloudera.com/t5/Community-Articles/Apache-NiFi-Processor-for-Apache-MX Net-SSD-Single-Shot/ta-p/249240 ● https://github.com/tspannhw/nifi-mxnetinference-processor ● https://github.com/tspannhw/nifi-djl-processor ● https://github.com/tspannhw/nifi-djlsentimentanalysis-processor
  14. 14. ● https://www.datainmotion.dev/2019/12/easy-deep-learning-in-apache-nifi-with.html ● https://github.com/tspannhw/ApacheConAtHome2020 ● https://docs.djl.ai/mxnet/mxnet-engine/index.html ● https://github.com/tspannhw/ApacheDeeplearning202 ● https://github.com/tspannhw/ApacheDeepLearning201 ● https://github.com/tspannhw/nifi-gluoncv-yolo3 ● https://github.com/tspannhw/ApacheDeepLearning101 REFERENCES
  15. 15. 15© 2020 Cloudera, Inc. All rights reserved. DEEPER CONTENT Title: "Using Apache MXNet in Production Deep Learning Streaming Pipelines" Abstract: As a Data Engineer I am often tasked with taking Machine Learning and Deep Learning models into production, sometimes in the cloud and sometimes at the edge. I have developed Java code that allows us to run these models at the edge and as part of a sensor/webcam/images/data stream. I have developed custom interfaces in Apache NiFi to enable real-time classification against MXNet models directly through the Java API or through DJL.AI's Java interface. I will demo running models on NVIDIA Jetson Nanos and NVIDIA Xavier NX devices as well as in the cloud. Technologies Utilized: Apache MXNet, DJL.AI, NVIDIA Jetson Nano, NVIDIA Jetson XAVIER, Apache NiFi, MiNIFi, Java, Python.
  16. 16. © 2020 Cloudera, Inc. All rights reserved. 16 TH N Y U

×