TensorFlow 2.0 is the latest release aimed at user convenience, API simplicity, and scalability across multiple platforms. In addition, TensorFlow 2.0, along with a variety of new projects in the TensorFlow ecosystem, TFX, TF-Agent, and TF federated, can help you quickly and easily create a wide variety of machine learning models in more environments. This talk will introduce TensorFlow 2.0 and discusses how to develop and optimize machine learning workflows based on TensorFlow 2.0 and projects within the various TensorFlow ecosystems. This slide was presented at GDG DevFest Songdo on November 30, 2019.

1. Boosting machine learning workflow
with TensorFlow 2.0
Jeongkyu Shin
Lablup Inc.
Google Developers Experts (ML/DL)

2. Jeongkyu Shin
Lablup Inc.
Making open-source machine
learning cluster platform:
Backend.AI
https://www.backend.ai
Google Developer Expert
ML / DL GDE, Sprint Master

3. Elements: Framework
● TensorFlow 2.0
● TensorFlow Extended
● TensorFlow Lite
● ML Kit
● TF-Agents
● TF Model Analysis
● TensorFlow Federated
● TF-NSL
● TensorFlow.js
● Swift for TensorFlow
● MLIR

4. TensorFlow
An end-to-end open source machine learning platform
Easy model building
Robust machine learning production
Powerful experimentation for research

5. TensorFlow: Summary
Statistics
> 30,000 commits since Dec. 2015
> 1,400 contributors
> 7,000 pull requests
> 24,000 forks (last 12 months)
> 6,400 TensorFlow related GitHub
Current
Complete ML model prototyping
Distributed training
CPU / GPU / TPU / Mobile support
TensorFlow Serving
Easier inference service
XLA compiler (1.0~)
Various target platform / performance tuning
Keras API Support (1.2~)
High-level programming API (Keras-compatible)
Eager Execution (1.4~)
Interactive mode
TensorFlow.Data (1.4~)
Pipeline for data sources

6. TensorFlow 2.0
Now in Live!
Keras as default grammar
Eager Execution as default runtime mode
Deprecates session-based execution
tf.compat.v1 for legacy codes
2.1 Soon!
TPU support is back
Mixed precision training

7. TensorFlow 2.0
Consolidate APIs
Keep one API for single behavior
Increase developer convenience
Debug with eager execution
Keep performance with AutoGraph
Keep namespace consistency
Remove global variable references
Easier large-scale training
Merge distributed TensorFlow features

8. # TensorFlow 1.X
outputs = session.run(f(placeholder), feed_dict={placeholder: input})
# TensorFlow 2.0
outputs = f(input)
TensorFlow 2.0: Optimization
Problems with Session.run ()
Not a function but works like a function and used like a function
Difficult to optimize session code
Change
Written like regular Python code
Decorator changes to optimized TensorFlow code at execution
(AutoGraph)

9. @autograph.convert()
def my_dynamic_rnn(rnn_cell, input_data, initial_state,
seq_len):
outputs = tf.TensorArray(tf.float32, input_data.shape[0])
state = initial_state
max_seq_len = tf.reduce_max(seq_len)
for i in tf.range(max_seq_len):
new_output, new_state = rnn_cell(input_data[i], state)
output = tf.where(i < seq_len, new_output,
tf.zeros_like(new_output))
state = tf.where(i < sequence_length, new_state, state)
outputs = outputs.write(i, output)
return tf.transpose(outputs.stack(), [1, 0, 2]), state
def tf__my_dynamic_rnn(rnn_cell, input_data, initial_state, sequence_length):
try:
with tf.name_scope('my_dynamic_rnn'):
outputs = tf.TensorArray(tf.float32, ag__.get_item(input_data.shape,
0, opts=ag__.GetItemOpts(element_dtype=None)))
state = initial_state
max_sequence_length = tf.reduce_max(sequence_length)
def extra_test(state_1, outputs_1):
with tf.name_scope('extra_test'):
return True
def loop_body(loop_vars, state_1, outputs_1):
with tf.name_scope('loop_body'):
i = loop_vars
new_output, new_state = ag__.converted_call(rnn_cell, True, False,
False, {}, ag__.get_item(input_data, i, opts=ag__.GetItemOpts
(element_dtype=None)), state_1)
output = tf.where(tf.less(i, sequence_length), new_output, tf.
zeros(new_output.shape))
state_1 = tf.where(tf.less(i, sequence_length), new_state, state_1)
outputs_1 = outputs_1.write(i, output)
return state_1, outputs_1
state, outputs = ag__.for_stmt(tf.range(max_sequence_length),
extra_test, loop_body, (state, outputs))
return tf.transpose(outputs.stack(), ag__.new_list([1, 0, 2])), state
except:
ag__.rewrite_graph_construction_error(ag_source_map__)

10. TensorFlow 2.0: Distribution Strategy
Supporting Strategies
MirroredStrategy (1.11)
All-reduce
Synchronized training
TPUStrategy (1.12)
Strategies on Google Cloud
CollectiveAllReduceStrategy (2.0)
Multi-node based MirroredStrategy
ParameterServerStrategy (2.0)
TPUStrategy (2.1)

11. Example: MirroredStrategy
strategy = tf.distribute.MirroredStrategy()
config = tf.estimator.RunConfig(
train_distribute=strategy, eval_distribute=strategy)
regressor = tf.estimator.LinearRegressor(
feature_columns=[tf.feature_column.numeric_column('feats')],
optimizer='SGD',
config=config)
def input_fn():
return tf.data.Dataset.from_tensors(({"feats":[1.]}, [1.]))
.repeat(10000).batch(10)
regressor.train(input_fn=input_fn, steps=10)
regressor.evaluate(input_fn=input_fn, steps=10)

12. Example: MirroredStrategy
ResNet50 Training code
train_dataset = tf.data.Dataset(...)
eval_dataset = tf.data.Dataset(...)
model = tf.keras.applications.ResNet50()
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)
model.compile(loss="categorical_crossentropy", optimizer=optimizer)
model.fit(train_dataset, epochs=10)
model.evaluate(eval_dataset)

13. Example: MirroredStrategy
ResNet50 Training code
Added MirroredStrategy: other codes are same.
train_dataset = tf.data.Dataset(...)
eval_dataset = tf.data.Dataset(...)
model = tf.keras.applications.ResNet50()
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)
strategy = tf.contrib.distribute.MirroredStrategy()
model.compile(loss="categorical_crossentropy", optimizer=optimizer,
distribute=strategy)
model.fit(train_dataset, epochs=10)
model.evaluate(eval_dataset)

14. Machine Learning Pipeline
Training is only a small part
Different story for production environment
Data
Ingestion
Data
Analysis + Validation
Data
Transformation Trainer
Model Evaluation
and Validation
Serving Logging
Shared Utilities for Garbage Collection, Data Access Controls
Pipeline Storage
Tuner
Shared Configuration Framework and Job Orchestration
Integrated Frontend for Job Management, Monitoring, Debugging, Data/Model/Evaluation Visualization

15. Machine Learning Pipeline
Training is only a small part
Different story for production environment
Diverse environments / situation make it so difficult
Data
Ingestion
Data
Analysis + Validation
Data
Transformation
Model Evaluation
and Validation
Serving Logging
Shared Utilities for Garbage Collection, Data Access Controls
Pipeline Storage
Tuner
Shared Configuration Framework and Job Orchestration
Integrated Frontend for Job Management, Monitoring, Debugging, Data/Model/Evaluation Visualization
TensorFlow
Core

16. TensorFlow Extended: Before
End-to-end machine learning platform by
Making pipeline with TensorFlow components
Orchestrate various computation resources
Data
Ingestion
Data
Transformation
Model Evaluation
and Validation
Logging
Shared Utilities for Garbage Collection, Data Access Controls
Pipeline Storage
Tuner
Shared Configuration Framework and Job Orchestration
Integrated Frontend for Job Management, Monitoring, Debugging, Data/Model/Evaluation Visualization
TensorFlow
Core
TensorFlow
Serving
TFDV

17. TensorFlow Extended: Now
Data
Ingestion
TFDV
TensorFlow
Transform
TensorFlow
Core
TensorFlow
Model Analysis
TensorFlow
Serving
Logging
Shared Utilities for Garbage Collection, Data Access Controls
Pipeline Storage
Tuner
AirFlow / KubeFlow
AirFlow / KubeFlow with various tools
End-to-end machine learning platform by
Making pipeline with TensorFlow components
Orchestrate various computation resources

18. Kubeflow Runtime
Example
Gen
Statistic
sGen
Schema
Gen
Example
Validator
Transform Trainer
Evaluator
Model
Validator
Pusher
TFX Config
Metadata Storage
Training +
Validation
Data
TensorFlow
Serving
TensorFlow
Hub
TensorFlow
Lite
TensorFlow JS
Airflow Runtime
TensorFlow Extended: Overview

19. TFX: Pipeline Orchestration
Airflow Kubeflow

20. TensorFlow Lite
TensorFlow for on-device environments
Simplifies on-device production challenges
Limited resources: CPU, memory, power consumption
Heterogeneous accelerators: various ASICs
Use cases
Android devices
Google assistant
Mobile ML platform

21. TensorFlow Lite with
TensorFlow 2.0
Model compatibility
TensorFlow 1.X / 2.0
GPU / NPU support
Less-buggy TFLite model
converter
With MLIR

22. ML Kit :
ML SDK for diverse environments

23. ML Kit :
ML SDK for diverse environments

24. ML Kit :
ML SDK for diverse environments
Turn-key solution for a specific task
Component to help with your own models
Server-based powerful features with Firebase
+Increases accuracy
+More categories
-Data network needed

25. ML Kit :
ML SDK for diverse environments
Will be more network-independent solution
On-device SDK: offline-ready
Firebase features?

26. TF-Agents:
library for reinforcement learning
Jupyter Notebook Examples
Integration with TensorFlow / pybullet
Examples and Docs
Supports both TensorFlow 1.14 and 2.0

27. TF-Agents:
library for reinforcement learning
Jupyter Notebook Examples
Integration with TensorFlow / pybullet
Examples and Docs
Supports both TensorFlow 1.14 and 2.0
Gfootball opensource training env.
https://github.com/google-
research/football

29. Machine Learning Fairness
Transparency Framework
Data card
Statistics for data bias

30. What-If Tool
Trackback results
Simulate the difference

31. ML Fairness
Part of TensorFlow Model Analysis
Automatic bias monitoring
Evaluate the performance impact of
adjustments
Case studies and benchmarks

32. Federated Learning
Distributed training with
minimal privacy violations
Data island + edge computing

33. Federated Learning
Save resources / traffic
Training without exposing local data
Keeping privacy
Demo
Action / emoticon prediction

34. TensorFlow Federated
Federated Learning API
Unified Core API
Local runtime for simulation
*Sources: Federated Learning: Machine Learning on Decentralized Data (Google I/O 19)

35. node.js based server-side training
Supports online prebuilt models / new TF Hub
TensorFlow.js

36. node.js based server-side training
Nvidia GPU support on desktop (driver needed)
Supports online prebuilt models / new TF Hub
TensorFlow.js

37. TensorFlow Neural Structured Learning
My recent interest!
Are you familiar with Taxonomy or semantics?
How about Semantic Web or Knowledge graph?

38. TensorFlow Neural Structured Learning
New learning paradigm
Training ‘relationship’ of inputs
NSL: Generalized training method to
Neural graph learning
Adversarial learning

39. TensorFlow Neural Structured Learning
import tensorflow as tf
import neural_structured_learning as nsl
# Prepare data.
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
# Create a base model -- sequential, functional, or subclass.
model = tf.keras.Sequential([
tf.keras.Input((28, 28), name='feature'),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation=tf.nn.relu),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
# Wrap the model with adversarial regularization.
adv_config = nsl.configs.make_adv_reg_config(multiplier=0.2, adv_step_size=0.05)
adv_model = nsl.keras.AdversarialRegularization(model, adv_config=adv_config)
# Compile, train, and evaluate.
adv_model.compile(optimizer='adam’,
loss='sparse_categorical_crossentropy’,
metrics=['accuracy'])
adv_model.fit({'feature': x_train, 'label': y_train}, batch_size=32, epochs=5)
adv_model.evaluate({'feature': x_test, 'label': y_test})

40. Swift for TensorFlow
Every components with one language
Only swift from Backend to Frontend
C++ + Python + (TF Lite) + …
Swift
LLVM-based platform independency support
Low huddle (more link pythonic + interpreter mode)
Future
MLIR: framework-independent ML accelerating with LLVM

41. MLIR
“Multi-Level Intermediate Representation" Compiler Infrastructure
Global improvements to TensorFlow infrastructure
Abstraction layer for accelerators
Support Heterogenous, distributed, mobile, custom ASICs
Urgency driven by the “end of Moore’s law”
As a part of LLVM: Other frameworks can benefit

42. MLIR
TensorFlow to TensorFlow Lite Converter
First MLIR-based improvements
Enhance edge device support by simplifying transforms and expressibility
Nov. 2019

43. Elements: Devices
● Coral, Edge TPU
● TPU Pods V3
● Cloud TPU Pods Beta

44. TitleCoral: Edge TPU based hardware

45. Coral
Edge TPU Brand
Dev Board / USB
Also PCI-E, SOM
Software Stack
Mendel OS (Debian Fork)
Edge TPU compiler for TF Lite
model compilation
Python SDK

46. Coral: Dev Board
CPU i.MX 8M SoC w/ Quad-core A53
GPU Integrated GC7000 Lite GPU
TPU Google Edge TPU
RAM 1GB LPDDR4 RAM
Storage 8 GB eMMC
Security/Crypto
eMMC secure block for TrustZone MCHP
ATECC608A Crypto Chip
Power 5V 3A via Type-C connector
Connectors USB-C, RJ45, 3.5mm TRRS, HDMI
OS Mendel Linux (Debian derivative) Android
ML TensorFlow Lite

47. Coral: USB accelerator
TPU Google Edge TPU
Power 5V 3A via Type-C connector
Connectors USB 3.1 (gen 1) via Type-C
Supported OS
Debian 6.0 or higher / Other Debian
derivatives
Supported
Architectures
x86-64, ARMv8
Supported ML TensorFlow Lite

48. Coral: SDK and market
I/O `19: New SDK
Model compiler for custom
model serving
Limited ops. Support
Competition in Edge AI ASIC market
Neural ComputeStick / Jetson
Nano
RP4 + Add-on
More to come!

50. Cloud TPU Pods
Full TPU Pod on Google Cloud
What topics need these big resources?
XLNet (June 2019)
Outperforms BERT language model
Cloud TPU Pod training (< 2.5 days)
260k US dollar (~312,000,000 won) for 2.5 days use
https://arxiv.org/abs/1906.08237

51. Cloud TPU Pods
Full TPU Pod on Google Cloud
What topics need these big resources?
Google T5 (Oct 2019)
Unified text-to-text transformer
Outperforms XLNet language model (?!)
Cloud TPU Pod training (~2 weeks)
Guess the cost! https://arxiv.org/abs/1910.10683

52. Elements: Framework
● TensorFlow 2.0
● TensorFlow Extended
● TensorFlow Lite
● TF-Agents
● ML Kit
● TF Model Analysis
● TensorFlow Federated
● TF-NSL
● TensorFlow.js
● Swift for TensorFlow
● MLIR
Elements: Devices
● Coral, Edge TPU
● TPU Pods V3
● Cloud TPU Pods Beta

53. Thank You!
facebook/jeongkyu.shininureyes@gmail.com
inureyesgithub/inureyes