What's Cooking in the Cloud?

Kasper Løvborg Jensen
Leafcastle Labs | Aarhus University
What’s Cooking in the Cloud?
Expanding the Menu for App Developers

Chief Wizard, Founder
Leafcastle Labs
Associate Professor
Aarhus University School of Engineering
Organizer
Google Developer Group Aarhus
@kasperljkasper@leafcastlelabs.com

GCP NEXT ‘16 Conference
•  Machine Learning for all!
•  Cloud Vision for all!
•  Cloud Speech API for all (soon)!
•  StackDriver (works with AWS)!
•  …lots more!

Outline
•  Why cloud?
•  Mobile backends
– Experiences
– Patterns
•  New powerful Tools/APIs
– Machine Learning
– Speech API
– Cloud Vision
•  Demo: TA-800

Why I am Excited about Cloud / GCP
•  Backends for mobile/wearable/web apps
–  Low/No Maintenance
–  Scalability
–  Reliability
–  Security
•  APIs / Services for applications
–  Integrate/synch with existing Services
–  Add powerful features/functionality through APIs
•  Great for startups
–  Business models and focus on creating value
–  Cost scales with growth
–  Speed of iteration and experiments

7

In My Steps

GCP Scalability

App Backend Needs
•  Storage
•  Computing
•  Network / serving
•  Cloud Messaging
•  Analytics
•  Tools
•  …

Google Cloud Platform
Source: GCP NEXT 2016 keynote

Mobile Backends?
Source: Google, Android, Apple, Chrome

Mobile Backends
“Building a backend service for a mobile app is
similar to building a web-based service, with
some additional requirements:
•  Limit on-device data storage
•  Synchronize data across multiple devices
•  Handle the offline case gracefully
•  Send notiﬁcations and messages
•  Minimize battery drain”
hAps://cloud.google.com/soluEons/mobile/mobile-app-backend-services

Backend Patterns
M.C. Escher

App Engine w. EndPoints
Modiﬁed from: hAps://cloud.google.com/soluEons/mobile/mobile-app-backend-services

Pure Firebase

Firebase + App Engine

Firebase + MVMs | Containers

Machine Learning
•  Machine Learning
•  Vision API
•  Speech API
•  Translate API
Source: Google

FROM THIS

TO THIS

Source: www.tensorﬂow.org

But wait, isn’t Machine Learning EVIL?

If by evil, you mean
AWESOME!

Cloud Speech APi

Speech API
•  Solves really hard problems!
– Natural language recognition
– Open vocabulary
– Noisy environments
– 80+ Different languages
hAps://cloud.google.com/speech/

Speech Horror Story ca. 2005
•  Distributed speech recognition in
app as part of my PhD
–  On PDA running Windows Mobile
with feature extraction through
custom DLL (c/c++)
–  Live stream features to Sphinx IV
DSR on Tomcat server that could
only handle one connection/request
at a time due to memory
requirements
–  Months of building/training speech,
grammar and acoustic models

Live or Batch?
•  Batch
– Upload audio sample(s) and get
results when recognition
complete
•  Live
– Stream audio to server and
receive partial results as they
become available
hAps://cloud.google.com/speech/

Cloud Vision API

Cloud Vision API
•  Solves really hard problems!
–  Extract and label objects in arbitrary image
without context
–  Can be anything
–  Find details like facial features, tilt and
“emotions“
–  Lighting conditions
–  Blurry pictures
–  Vast amount of cameras and settings
–  …

Case Study
•  Problem: Students falling asleep during
lectures
•  Solution: Need person some sort of
contraption to detect students falling
asleep and then wake them up
“Never send a man to
do a machine’s job”
-Gandalf

TA-800
•  IF we wanted to build a Terminator as a
teaching assistant
– Running Android
– Powered by Google Cloud / Skynet
•  Detect and “terminate” inactive students
during lectures
– High precision Nerf gatling gun

TA-800 Hardware
•  Nexus 6 or 6p
•  ATLAS prototype
•  Nerf minigun

TA-800 Behavior Protocol
While(true){
– Detect situation
•  Identify features and classify place/situation
– Acquire potential targets
•  Detect if sleeping
– TERMINATE Wake up targets
– Report to master
}

Architecture
Master TA-800

Demo

TA-800 Hack/Demo
•  Image processing based on Cloud Vision API
example app
https://github.com/GoogleCloudPlatform/
cloud-vision/tree/master/android/CloudVision
•  Firebase integration based on Firebase chat
example
https://github.com/ﬁrebase/AndroidChat
•  Total dev time:~6 hours (50%+ time on UI)
•  Will be on GitHub:

Cloud Vision API
hAps://cloud.google.com/vision/

Using Cloud Vision API
•  Specify request (JSON)
– Image
•  Content/source
– Features
•  Type
•  Max results
•  Handle result (JSON)
•  Android: use CloudVision API
–  com.google.apis:google-api-services-
vision:v1-rev2-1.21.0!

Cloud Vision Detection Types
DETECTION_TYPES = [!
'TYPE_UNSPECIFIED',!
'FACE_DETECTION',!
'LANDMARK_DETECTION',!
'LOGO_DETECTION',!
'LABEL_DETECTION',!
'TEXT_DETECTION',!
'SAFE_SEARCH_DETECTION',!
]!

Rich Response Data as JSON
•  Labels
–  Things/situations in the image
•  Faces
–  Faces, location in picture
–  Happy, angry, sad, surprised
–  Face “landmarks”: eyes, ears, …
–  Tilt, …
•  Logos
–  Known brands and logos
•  Landmarks
–  “Eiffel Tower”, “AROS museum”, …
–  Also including e.g. location metadata
•  Texts
–  OCR, finding text in the image

Your turn!
GCP:
https://cloud.google.com/
List of (most) products:
https://cloud.google.com/products/
Playground:
https://cloud-playground.appspot.com/playground/
Launcher:
https://cloud.google.com/launcher/
Firebase:
https://www.ﬁrebase.com/

THANK YOU!
Questions?
@kasperlj
kasper@leafcastlelabs.com

What's Cooking in the Cloud?

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (18)

Similar to What's Cooking in the Cloud?

Similar to What's Cooking in the Cloud? (20)

Recently uploaded

Recently uploaded (20)

What's Cooking in the Cloud?