Azure Cognitive Services allow developers to build powerful AI-based solutions, enabling different capabilities in our software: vision. speech, search, text analytics, language understanding, and much more. Basically, the model is already built by Microsoft, you just need to do an API call to the Azure cloud and the service retrieves a result. For instance, you send a message and the Text Analytics API returns its sentiment score. However, there might be cases in which our customers need a local, non-cloud AI solution (either because of limited Internet access or data compliance). This is now possible thanks to the latest update of Azure Cognitive Services, which offers containerization support. Using containers, we can still deliver ML-driven solutions while keeping the data in-house. In this talk, we'll explore what it takes to configure and use containers in Azure Cognitive Services. Demos will be showcased as well for local Face and Text Cognitive Services.

1. Luis Beltran, Microsoft MVP, Researcher at TBU
Moderated By: Fernando Sibaja
Bringing AI to the Edge
On-premise Azure Cognitive Services

2. If you require assistance
during the session, type
your inquiry into the
question pane on the right
side.
Maximize your screen with
the zoom button on the
top of the presentation
window.
Please fill in the short
evaluation following the
session. It will appear in
your web browser.
Technical Assistance

3. PASS’ flagship event
November 6-9
Seattle, Washington
Free 1-day local
training events
Local user groups
around the world
Online special
interest user groups
Business analytics
training
Get involved
Explore
everything
PASS has
to offer
Free Online Resources
Newsletters
PASS.org

4. Luis Beltrán
Microsoft MVP, Researcher at TBU
About me
I’m a Microsoft MVP, PhD student at Tomas
Bata University in Zlin, Czech Republic,
Xamarin Certified Mobile Developer, and
lecturer at Instituto Tecnologico de Celaya in
Mexico
Interests
My main interest is to share my knowledge to
other people. As a lecturer, I really care about
my students and always want them to learn
the latest about technology, an ever-changing
environment
Main technologies
Xamarin
Azure
Artificial Intelligence
Databases
/luisantoniobeltran
@darkicebeam
Luis Beltran
luis@luisbeltran.mx
https://luisbeltran.mx

5. Luis Beltran, Microsoft MVP, Researcher at TBU
Moderated By: Fernando Sibaja
Bringing AI to the Edge
On-premise Azure Cognitive Services

6. • Azure Cognitive Services
• Docker Containers
• Container support in Azure Cognitive Services
• Requirements
• Demo
• Call to Action
Agenda

7. Azure Cognitive Services

8. Azure Cognitive Services
Applying AI to your business
Computer Vision
Face/Emotion Recognition
OCR/Handwriting
Custom Vision
Video Indexer
Content Moderator
Text-to-Speech
Speech-to-Text
Translator
Custom Speech
Language Understanding
PII Detection
Text Translator
Text Analytics
QnA Maker
Bing Custom Search
Bing Visual Search

10. Text Analytics API

13. A variety of real-world applications
Vision Speech Language
Natural Language Processing
Intent: PlayCall
Knowledge
Here are the top results:
The purpose of Customer Life-cycle Management (CLM)
is to maximize both customer retention and .... Predictive
trend analysis provides business visibility.
Oct 28, 2015 – Here are FIVE key trends in 2014 that
would help marketers in rolling ... Of late, marketers are
looking at customer lifecycle management (CLM)
Jan 5, 2016 – The top 10 customer service trends for
2016 that .... North American Consumer
Search
Here is what I found:
It also investigates the top three expected
Fraud Detection and Prevention programs, in
terms of demand in key markets…
First, let’s point out that there is not one
absolute answer—there are “pros” and “cons” to
each. Those who favor in-house…
Michael heads fraud prevention tool. Online and
mobile shopping are expected to continue
growing apace…

14. Docker Containers

15. Containers

17. Docker integration
https://hub.docker.com

18. Docker for Windows

19. Container Support in Azure
Cognitive Services

20. Considerations
• What if you can't upload an image or text to the cloud?
• Sending data outside your network might be subject to regulatory or privacy policies.
• If you could analyze the images or text locally, your application could benefit from
reduced latency and bandwidth.
Now, several of the Azure Cognitive Services APIs are available as Docker containers.

21. Support
• You can download a container that provides the exact same APIs as the cloud-based
services, and run it on a local Linux-based server or edge device.
• Images and text are processed directly in the container and never sent to the cloud.
• A connection to Azure is required only for billing, which is at the same rate as the
cloud-based services (including a free tier).
Container support in Azure Cognitive Services allows developers to use the same rich APIs
that are available in Azure, and enables flexibility in where to deploy and host the services
that come with Docker containers.

22. Features and benefits
• Run AI Cognitive Services on-premises
• Process lots of data (High throughput)
• No constraints in network connection (low latency)
• Portable architecture
• Control over data: Allow customers to choose where these Cognitive Services process
their data. This is essential for customers that cannot send data to the cloud but need
access to Cognitive Services technology. Support consistency in hybrid environments
– across data, management, identity, and security.

23. What is available?
Preview versions for of:
• Computer Vision
• Face
• Text Analytics
• and Language Understanding (LUIS).

26. Requirements

27. Prerequisites
• Docker Engine: Docker provides packages that configure the Docker environment on
macOS, Linux, and Windows. On Windows, Docker must be configured to support
Linux containers. Docker containers can also be deployed directly to Azure Kubernetes
Service or Azure Container Instances.
• Cognitive Service resource created on Azure: Docker must be configured to allow the
containers to connect with and send billing data to Azure.

28. https://docs.docker.com/toolbox/toolbox_install_windows/
docker-machine stop
docker-machine rm default
docker-machine create -d virtualbox --virtualbox-
cpu-count=2 --virtualbox-memory=2048 default

29. Prerequisites
• Container requirements: Individual containers can have their own requirements, as
well, including server and memory allocation requirements.

30. Demo

38. http://github.com/icebeam7/PASSMarathon

41. Call to Action

42. Resources
❑ Container support in Azure Cognitive Services
https://docs.microsoft.com/en-us/azure/cognitive-services/cognitive-services-
container-support
❑ Containers support of Cognitive Services (video)
https://azure.microsoft.com/en-us/resources/videos/containers-support-of-
cognitive-services/
❑ Cognitive Services
http://microsoft.com/cognitive

43. Resources
❑ Docker Images (Azure Cognitive Services)
https://hub.docker.com/_/microsoft-azure-cognitive-services
❑ Containers samples for Cognitive Services
https://github.com/Azure-Samples/cognitive-services-containers-samples
❑ Cognitive Services
https://azure.microsoft.com/en-us/blog/running-cognitive-service-containers/

44. Resources
❑ Install and Run Text Analytics containers
https://docs.microsoft.com/es-es/azure/cognitive-services/text-analytics/how-
tos/text-analytics-how-to-install-containers
❑ Running Cognitive Service containers
https://docs.microsoft.com/es-es/azure/cognitive-services/face/face-how-to-install-
containers

45. Questions?

46. Thank you for attending
@sqlpass #sqlpass @PASScommunity
Learn more from [Speaker Name]
@yourhandle email@company.com