EIA2017Portugal - Andres L. Martinez Ortiz - Curating Content with Google Machine Learning Application Programming Interfaces
•
3 likes•1,011 views
Andres L. Martinez Ortiz - Curating Content with Google Machine Learning Application Programming Interfaces
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to EIA2017Portugal - Andres L. Martinez Ortiz - Curating Content with Google Machine Learning Application Programming Interfaces
Similar to EIA2017Portugal - Andres L. Martinez Ortiz - Curating Content with Google Machine Learning Application Programming Interfaces (20)
More from European Innovation Academy
More from European Innovation Academy (20)
Recently uploaded
Recently uploaded (20)
EIA2017Portugal - Andres L. Martinez Ortiz - Curating Content with Google Machine Learning Application Programming Interfaces
- 1. Curating Online Content with Google ML APIs Andres L. Martinez a.k.a almo Google Developer Relations Manager @davilagrau
- 3. Unstructured data accounts for 90% of enterprise data* Cloud Machine Learning APIs help you make sense of it *Source: IDC
- 4. Why is this time different?Photo by Emily Morter on Unsplash
- 5. Photo by Sharon Pittaway on Unsplash Innovation
- 6. Photo by Alex Holyoake on Unsplash
- 7. Fast{er} Photo by Josh Calabrese on Unsplash
- 8. Focus on the user
- 9. The Machine Learning Spectrum TensorFlow Cloud Machine Learning Machine Learning APIs BYOML skills Pre-packaged ML
- 12. Voice Kit
- 13. A.I. Experiments
- 14. Greener Google
- 15. Machine Learning find new ways for our data center to save energy
- 18. Project Wing
- 19. Confidential & ProprietaryGoogle Cloud Platform 19 So…. Why APIs? { Google Cloud Platform } 1. We want to offer businesses the tools to differentiate by offering a powerful set of APIs that enable apps to see, hear and understand the world 2. Reduce your Time to Market (TMM) when launching your next-generation app 3. Provide you easy access to machine learning technology to give any developer the freedom to work in the language and tools they want 4. Provide virtually limitless scalability to your application without needing to manage back-end servers running deep learning
- 20. Pre-Trained Machine Learning Models Fully trained ML models from Google Cloud that allow a general developer to take advantage of rich machine learning capabilities with simple REST based services.
- 21. Confidential & ProprietaryGoogle Cloud Platform 21 Introducing Cloud Natural Language API Sentiment analysis and entity recognition for text
- 22. Confidential & ProprietaryGoogle Cloud Platform 22 Features Extract sentence, identify parts of speech and create dependency parse trees for each sentence Identify entities and label by types such as person, organization, location, events, products and media Understand the overall sentiment of a block of text Access via REST API. Text can be uploaded in the request or integrated with Google Cloud Storage Syntax Analysis Entity Recognition Sentiment Analysis Integrated REST API
- 23. Confidential & ProprietaryGoogle Cloud Platform 23 Cloud Vision API Insight from images with our powerful Cloud Vision API
- 24. Confidential & ProprietaryGoogle Cloud Platform 24 Faces: Faces, facial landmarks, emotions OCR: Read and extract text, with support for > 10 languages Photo credit Getty Images Label: Detect entities from furniture to transportation Logos: Identify product logos Landmarks & Image Properties Detect landmarks & dominant color of image Safe Search: Detect explicit content - adult, violent, medical and spoof Cloud Vision API Call API from anywhere, with support for embeddable images, and Google Cloud Storage
- 25. Let’s Party
- 28. Party planning ● Finding people @Twitter ● Cloud Vision API ● Custom classifier (k- means)
- 30. Google Cloud Console We need to have access so we can add hash tag to intro slide
- 34. Main shellplus_contacts = get_plus_contacts() print "Processing %d contacts" % len(plus_contacts) for plus_id in plus_contacts: plus_profile = get_plus_profile(plus_id) image_uri = plus_profile['image']['url'].replace("?sz=50","?sz=250") image_data = analyze_img(image_uri) if image_data is not None: print(image_uri) if 'labelAnnotations' in image_data['responses'][0]: for label in image_data['responses'][0]['labelAnnotations']: print label['description']; label['score']; image_uri
- 35. get_plus_contacts: oAuth storage = Storage('/home/almo/dev/keys/ex1/oAuth_credentials.dat') credentials = storage.get() if credentials is None or credentials.invalid: PEOPLE_API='https://www.googleapis.com/auth/contacts.readonly' flow = flow_from_clientsecrets('/home/almo/dev/keys/ex1/oAuth_key.json', scope=[PEOPLE_API]) credentials = run_flow(flow, storage) http = credentials.authorize(httplib2.Http()) service = build('people','v1',http=http) request = service.people().connections().list(resourceName='people/me', pageSize=500)
- 36. analyze_image api_key = json.load(open('/home/almo/dev/keys/ex1/api_key.json'))['api_key'] service = discovery.build('vision','v1',developerKey=api_key) service_request = service.images().annotate(body={ 'requests': [{ 'image': { 'content': image_content.decode('UTF-8') }, 'features': [{ 'type': 'LABEL_DETECTION', 'maxResults': 3 }] }] }
- 37. Data
- 38. face; 0,92830354; https://lh3.googleusercontent.com/-c3M1gn6ougg/AAAAAAAAAAI/AAAAAAAAAds/cTIrpGhktfw/photo.jpg?sz=250 text; 0,93046468; https://lh4.googleusercontent.com/-GFVyrVlgMy4/AAAAAAAAAAI/AAAAAAAAABE/u3xVd9eJgf8/photo.jpg?sz=250 font; 0,85384184; https://lh4.googleusercontent.com/-GFVyrVlgMy4/AAAAAAAAAAI/AAAAAAAAABE/u3xVd9eJgf8/photo.jpg?sz=250 line; 0,70535356; https://lh4.googleusercontent.com/-GFVyrVlgMy4/AAAAAAAAAAI/AAAAAAAAABE/u3xVd9eJgf8/photo.jpg?sz=250 eyebrow; 0,98022038; https://lh5.googleusercontent.com/-5c9gdP9nX9M/AAAAAAAAAAI/AAAAAAAAGt4/FoZEEVA8F68/photo.jpg?sz=250 hair; 0,96653992; https://lh5.googleusercontent.com/-5c9gdP9nX9M/AAAAAAAAAAI/AAAAAAAAGt4/FoZEEVA8F68/photo.jpg?sz=250 face; 0,95101357; https://lh5.googleusercontent.com/-5c9gdP9nX9M/AAAAAAAAAAI/AAAAAAAAGt4/FoZEEVA8F68/photo.jpg?sz=250 person; 0,92170084; https://lh4.googleusercontent.com/-yVWpXcqQfXU/AAAAAAAAAAI/AAAAAAAAB5w/rqxRrJHgk_0/photo.jpg?sz=250 news; 0,63342041; https://lh4.googleusercontent.com/-yVWpXcqQfXU/AAAAAAAAAAI/AAAAAAAAB5w/rqxRrJHgk_0/photo.jpg?sz=250 professional; 0,61274487; https://lh4.googleusercontent.com/- yVWpXcqQfXU/AAAAAAAAAAI/AAAAAAAAB5w/rqxRrJHgk_0/photo.jpg?sz=250 drawer; 0,80023241; https://lh6.googleusercontent.com/-Qf9SSsIUktA/AAAAAAAAAAI/AAAAAAAAABg/u6zPUNXCYFs/photo.jpg?sz=250 furniture; 0,79278195; https://lh6.googleusercontent.com/-Qf9SSsIUktA/AAAAAAAAAAI/AAAAAAAAABg/u6zPUNXCYFs/photo.jpg?sz=250 product; 0,76023591; https://lh6.googleusercontent.com/-Qf9SSsIUktA/AAAAAAAAAAI/AAAAAAAAABg/u6zPUNXCYFs/photo.jpg?sz=250 eyewear; 0,97702742; https://lh3.googleusercontent.com/-ihQNk3ewmzQ/AAAAAAAAAAI/AAAAAAAAAMk/EEEylEyriNE/photo.jpg?sz=250 hair; 0,96766639; https://lh3.googleusercontent.com/-ihQNk3ewmzQ/AAAAAAAAAAI/AAAAAAAAAMk/EEEylEyriNE/photo.jpg?sz=250 sunglasses; 0,96445274; https://lh3.googleusercontent.com/-ihQNk3ewmzQ/AAAAAAAAAAI/AAAAAAAAAMk/EEEylEyriNE/photo.jpg?sz=250 person; 0,92747426; https://lh4.googleusercontent.com/--_BxhkQPYfA/AAAAAAAAAAI/AAAAAAAAACA/1pN6-Chy8EI/photo.jpg?sz=250 person; 0,96007371; https://lh3.googleusercontent.com/-sX8l_lv_-7w/AAAAAAAAAAI/AAAAAAAAAPU/ApQpBMPbcdc/photo.jpg?sz=250 face; 0,95332307; https://lh3.googleusercontent.com/-sX8l_lv_-7w/AAAAAAAAAAI/AAAAAAAAAPU/ApQpBMPbcdc/photo.jpg?sz=250 Raw Data
- 39. 160 different labels Max Freq.: 200 Min Freq. : 1
- 40. person 200 0,9320951099 hair 140 0,9609928544 face 139 0,9489352931 font 136 0,7606724908 text 130 0,925080287 blue 114 0,9112923658 facial hair 36 0,8802876539 nose 34 0,8859786603 profession 30 0,569073382 hairstyle 25 0,7532089968
- 41. glasses 20 0,8234816515 facial expression 14 0,9502550086 eyebrow 12 0,9559630675 black and white 11 0,9199305709 eyewear 11 0,9767648145 logo 11 0,7749610755
- 46. "hair", 0.9559916, "person", 0.94347906, "face", 0.92830354 "text", 0.9304647, "font", 0.85384184, "line", 0.70535356 "eyebrow", 0.9802204, "hair", 0.9665399, "face", 0.9510135 "person", 0.92170084, "news", 0.63342035, "professional", 0.61274487 "drawer", 0.8002325, "furniture", 0.792782, "product", 0.760235 "eyewear", 0.9770274, "hair", 0.9676664, "sunglasses", 0.96445274 "person", 0.92747426, "https://lh4.googleusercontent.com/-- _BxhkQPYfA/AAAAAAAAAAI/AAAAAAAAACA/1pN6-Chy8EI/photo.jpg?sz=250" "green", 0.9307698, "text", 0.92834556, "font", 0.8631033 "hair", 0.98155975, "face", 0.95545304, "eyebrow", 0.93590355 "face", 0.9523797, "person", 0.94760686, "hair", 0.94507515 "hair", 0.9731342, "face", 0.94925183, "person", 0.9371813 "hair", 0.94741917, "person", 0.9436425, "hairstyle", 0.7414854 "person", 0.925232, "people", 0.9086431, "male", 0.83032143 "person", 0.95530343, "face", 0.94757956, "nose", 0.86752254 "face", 0.96074444, "hair", 0.9606222, "eyebrow", 0.9451414 "face", 0.9664352, "hair", 0.9561741, "nose", 0.9222636 "phenomenon", 0.94444287, "celestial event", 0.53744316, "aurora", 0.52995497 "face", 0.9625666, "hair", 0.9514838, "facial expression", 0.94977105 "product", 0.80306137, "font", 0.77923214, "logo", 0.69078964 "black and white", 0.9267871, "person", 0.8998944, "photography", 0.8296365
- 47. Training mode!
- 48. “invited”, "hair", 0.9559916, "person", 0.94347906, "face", 0.92830354 “excluded”, "text", 0.9304647, "font", 0.85384184, "line", 0.70535356 “excluded”, "eyebrow", 0.9802204, "hair", 0.9665399, "face", 0.9510135 “invited”, "person", 0.92170084, "news", 0.63342035, "professional", 0.61274487 “excluded”, "drawer", 0.8002325, "furniture", 0.792782, "product", 0.760235 “excluded”, "eyewear", 0.9770274, "hair", 0.9676664, "sunglasses", 0.96445274 “excluded”, "green", 0.9307698, "text", 0.92834556, "font", 0.8631033 “excluded”, "hair", 0.98155975, "face", 0.95545304, "eyebrow", 0.93590355 “invited”, "face", 0.9523797, "person", 0.94760686, "hair", 0.94507515 “invited”, "hair", 0.9731342, "face", 0.94925183, "person", 0.9371813 “invited”, "hair", 0.94741917, "person", 0.9436425, "hairstyle", 0.7414854 “invited”, "person", 0.925232, "people", 0.9086431, "male", 0.83032143 “invited”, "person", 0.95530343, "face", 0.94757956, "nose", 0.86752254 “excluded”, "face", 0.96074444, "hair", 0.9606222, "eyebrow", 0.9451414 “excluded”, "face", 0.9664352, "hair", 0.9561741, "nose", 0.9222636
- 49. Prediction Mode!
- 50. "hair", 0.9559916, "person", 0.94347906, "face", 0.92830354 “invited” "text", 0.9304647, "font", 0.85384184, "line", 0.70535356 “excluded” "eyebrow", 0.9802204, "hair", 0.9665399, "face", 0.9510135 “excluded” "person", 0.92170084, "news", 0.63342035, "professional", 0.61274487 “invited” "drawer", 0.8002325, "furniture", 0.792782, "product", 0.760235 “excluded” "eyewear", 0.9770274, "hair", 0.9676664, "sunglasses", 0.96445274 “excluded” "green", 0.9307698, "text", 0.92834556, "font", 0.8631033 “excluded” "hair", 0.98155975, "face", 0.95545304, "eyebrow", 0.93590355 “excluded” "face", 0.9523797, "person", 0.94760686, "hair", 0.94507515 “invited” "hair", 0.9731342, "face", 0.94925183, "person", 0.9371813 “invited” "hair", 0.94741917, "person", 0.9436425, "hairstyle", 0.741485 4 “invited” "person", 0.925232, "people", 0.9086431, "male", 0.83032143 “invited” "person", 0.95530343, "face", 0.94757956, "nose", 0.86752254 “invited” "face", 0.96074444, "hair", 0.9606222, "eyebrow", 0.9451414 “excluded” "face", 0.9664352, "hair", 0.9561741, "nose", 0.9222636 “excluded”
- 53. Curating Online Content with Google ML APIs Andres L. Martinez a.k.a almo Google Developer Relations Manager @davilagrau
Editor's Notes
- TensorFlow Open sourced in November 2015 Make clear that it's not only for Neural Nets, but rather it's a generalized ML lib for all sorts of machine learning problems (decision trees, support vector machines, reinforcement learning, and more) Goal is to emulate the neural networks inside your brain using a data flow framework A tensor is a multidimensional array, and TF is a data flow framework for tensors. With TF you can represent and perform computations on multi-dimensional data flowing through a graph. Developers + researchers can build and train their own models Input => algorithms => output Runs on many platforms: data centers, CPUs and GPUs, mobile phones Get started with the MNIST example, basically the “Hello World” of machine learning. It uses a dataset of bitmap images of handwritten numbers - you use this to train a model to recognize numbers from handwritten images Needs more background, understanding of ML and math, more code Cloud ML Fully managed service Train using a custom TensorFlow graph Batch and online predictions, at scale Integrated Datalab experience Regression and classification tasks Great use case if you have a pile of data, a custom classification, and you don't want to hire (or become) a machine learning expert (aka get a PhD) When you’d want to use TF vs. Cloud ML: the largest challenge for NN users now is the scalability for distributed training. It's so hard to build a GPU cluster. Cloud ML solves that. The cool thing about the vision + speech APIs is that you don’t need to be an ML expert to use them. You just need to be able to make a REST request! Machine Learning APIs, meant for people building applications Access fully-trained models...as a service Black box the ML part, and let you move on with your life: If you can make a REST API call, you can use it. Examples include vision api, speech api, and translate api. More to come! <CLICK> Vision API = Friendly Machine Learning, not the acronym (FML?)
- faces photo is from our mobile vision website: https://developers.google.com/vision/ People pictured above are Google employees, from the London developer relations team. ship photo is from Getty Images Receipt Photo is from istock photos Logo Image: New Google Logo 2015 - YouTube - www.youtube.com1270 × 720Search by image
- Let me walk you through what Vision API can really do: Label Detection: The API can detect broad sets of categories within an image, ranging from modes of transportation to animals. Face Detection: The API can detect multiple faces within an image, along with the associated key facial attributes like emotional state or wearing headwear. OCR: The API can detect and extract text within an image, with support for a broad range of languages. Explicit Content Detection: We can detect explicit content like adult content or violent content within an image. Landmark Detection: The APi can detect popular natural and manmade structures within an image. Logo Detection: We can detect popular product logos within an image.