Tensorflow 2.0 and Keras: What's new, what's shared, what's different
•
3 likes•694 views
Talk given at ODSC East, Boston, May 1, 2019. Tensorflow 2.0 makes Keras the default API for model definition. This is a big change. It makes Tensorflow more accessible to beginners and newcomers and it also disrupts consolidated patterns and habits for experienced Tensorflow programmers. This workshop is aimed to both audiences and it covers how to define models in Tensorflow 2.0 using the tf.keras API. It also covers the commonalities and differences between the open source Keras package and tf.keras, explaining pros and cons of each of the two. If you are getting started with Tensorflow or you're puzzled by the changes in Tensorflow 2.0, come and learn how easy it is to design models using Keras.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Tensorflow 2.0 and Keras: What's new, what's shared, what's different
Similar to Tensorflow 2.0 and Keras: What's new, what's shared, what's different (20)
More from Francesco Mosconi
More from Francesco Mosconi (10)
Recently uploaded
Recently uploaded (20)
Tensorflow 2.0 and Keras: What's new, what's shared, what's different
- 1. Catalit LLC
- 3. Catalit LLC ABOUT US www.catalit.com Catalit Data Science Zero to Deep Learning bootcamp.zerotodeeplearning.com
- 4. Catalit LLC TODAY • Part 1:Tensorflow 2.0 • Part 2: Migration fromTF 1.x and from Keras
- 5. Catalit LLC ASSUMPTIONS • Used Keras and/orTensorflow 1.x before • Curious aboutTensorflow 2.0
- 6. Catalit LLC PART 1 Tensorflow 2.0
- 7. Catalit LLC PART 1 • Motivation • AboutTF 2.0 • Public design process • Eager Execution • API Cleanup
- 9. Catalit LLC FRAMEWORKS IN 2017 https://towardsdatascience.com/battle-of-the-deep-learning-frameworks-part-i-cff0e3841750
- 10. Catalit LLC https://towardsdatascience.com/deep-learning-framework-power-scores-2018-23607ddf297a
- 11. Catalit LLC THE BATTLE IS OVER • My opinion: in 2019, only 2 DL Frameworks: • Tensorflow 2.0 (with tf.keras) • Pytorch (with Caffe2)
- 14. Catalit LLC COMMITS Tensorflow Keras CNTK MXNet
- 15. Catalit LLC COMMITS Tensorflow Keras CNTK MXNet
- 19. Catalit LLC LEGACY REASONSTO USE KERAS • 2 reasons to use Keras until now: • Simple API • Multiple backends
- 20. Catalit LLC LEGACY REASONSTO USE KERAS • 2 reasons to use Keras until now: • Simple API ==> Adopted byTF 2.0 • Multiple backends ==> No more true/relevant
- 21. Catalit LLC REASONSTO USETF.KERAS • Better integration withTensorflow Ecosystem • Support forTPUs (Google proprietary)
- 22. Catalit LLC CONCLUSION • LearnTF 2.0 (and Pytorch if you do research) • Vendor lock-in to Google is issue, not many viable alternatives
- 24. Catalit LLC SOMETIME IN AUGUST 2018 …
- 25. Catalit LLC DEV SUMMIT (MARCH 6-7) https://www.tensorflow.org/alpha
- 27. Catalit LLC TF 2.0 KEY POINTS • Requests for comments (RFCs) • Eager Execution • API Cleanup
- 28. Catalit LLC PUBLIC DESIGN PROCESS
- 29. Catalit LLC PUBLIC 2.0 DESIGN PROCESS https://groups.google.com/a/tensorflow.org/forum/#!forum/discuss https://github.com/tensorflow/community/blob/master/governance/TF-RFCs.md https://github.com/tensorflow/community/tree/master/rfcs
- 30. Catalit LLC SOME ACCEPTED RFC • TensorForest Estimator • Attention for Dense networks on Keras • TensorFlow 2.0: Optimizer unification • Unify RNN Inteface • Check https://github.com/tensorflow/community/tree/master/rfcs
- 32. Catalit LLC EAGER EXECUTION • Available since late 2017 • Following Pytorch and Chainer • Imperative • Define-by-run • No graph & session
- 33. Catalit LLC EAGER EXECUTION • Debugging with Python tools • Dynamic models with Python control flow • Support for custom and higher-order gradients
- 34. Catalit LLC IS GRAPH GOING AWAY? • No, you can still define models using the traditional static graph • tf.compat.v1.disable_eager_execution()
- 36. Catalit LLC Standard Tensorflow Eager Execution
- 38. Catalit LLC Standard Tensorflow Eager Execution
- 39. Catalit LLC Model is a function Gradient Tape
- 40. Catalit LLC WHAT IT MEANS FORYOU • Debug like Numpy, scale likeTensorflow • Easier to build and test custom models
- 43. Catalit LLC MODEL APIS INTF 1.X • tf.core • tf.layers • tf.keras.layers • tf.estimator.Estimator • tf.contrib.learn.Estimator • …
- 44. Catalit LLC MODEL APIS INTF 2.0 • tf.core ====> stripped down • tf.layers ====> Deprecated • tf.keras ====> Primary API • tf.estimator.Estimator ====> API same, compatible with tf.keras • tf.contrib.learn.Estimator ====> Deprecated • …
- 45. Catalit LLC ALSO • Unified metrics ==> tf.keras.metrics • Unified optimizers ==> tf.keras.optimizers • Unified losses ==> tf.keras.losses • ====> Single implementation for each
- 51. Catalit LLC PART 2 Other features & Migration
- 52. Catalit LLC PART 2 • Other features inTF 2.0 • Migration fromTF 1.x • Migration from Keras
- 54. Catalit LLC IMPROVED DOC • Historically: • Keras Documentation => Awesome • Tensorflow Documentation => Pretty bad
- 55. Catalit LLC IMPROVED DOC • As of May 2019 • Keras Documentation => Stale, no new releases since October 2018 • Tensorflow 2.0 Alpha Documentation => Pretty good, full of examples and tutorials
- 57. Catalit LLC NO MORE GLOBALS • Keep track of your variables! If you lose track of a tf.Variable, it gets garbage collected.
- 58. Catalit LLC FUNCTIONS & AUTOGRAPH • Functions to bundle things in the same graph • Autograph to convert flow control
- 59. Catalit LLC MIGRATION FROMTF 1.X
- 60. Catalit LLC MIGRATION FROMTF 1.X • Use upgrade script • tf.Session.run ==> Python function • tf.get_variable ==> tf.Variable • Training loops ==> model.fit • Read https://www.tensorflow.org/alpha/guide/migration_guide
- 61. Catalit LLC UPGRADE SCRIPT • https://www.tensorflow.org/alpha/guide/upgrade • tf_upgrade_v2 --infile tensorfoo.py --outfile tensorfoo-upgraded.py
- 63. Catalit LLC 1.X COMPATIBILITY • import tensorflow.compat.v1 as tf • has everything except tf.contrib
- 64. Catalit LLC MIGRATION FROM KERAS
- 65. Catalit LLC MIGRATION FROM KERAS RRR Framework: 1. Replace 2. Refactor 3. Rewrite
- 66. Catalit LLC MIGRATION FROM KERAS RRR Framework: 1. Replace 2. Refactor 3. Rewrite
- 67. Catalit LLC REPLACE • from keras import ... • from tensorflow.keras import ...
- 68. Catalit LLC MIGRATION FROM KERAS RRR Framework: 1. Replace 2. Refactor 3. Rewrite
- 69. Catalit LLC REFACTOR • Import keras.backend as K • K.get_session() • Problem: no more session in Eager execution !!!
- 70. Catalit LLC SOLUTIONS • tf.compat.v1.disable_eager_execution () • Can you achieve the same result with different code?
- 71. Catalit LLC MIGRATION FROM KERAS RRR Framework: 1. Replace 2. Refactor 3. Rewrite
- 72. Catalit LLC REWRITE • Functionality overlap • Best practices inTF 2.0
- 73. Catalit LLC REWRITE EXAMPLES • tf.dataVS Generators • Saved Model • Distribution strategies
- 78. Catalit LLC CONSIDERATIONS • Image augmentation easier in ImageDataGenerator • Tf.data preferable when: • Big Data (HDF, Lots of files,TFRecords, …) • Complex preprocessing • Not images (text, sequences, binary files, movies, etc.)
- 80. Catalit LLC REWRITE: EXPORT • Keras save model • Reload in Keras (e.g. flask app)
- 81. Catalit LLC REWRITE: EXPORT • Save model in Json • Save weights in h5 • Useful for porting to other frameworks
- 82. Catalit LLC REWRITE: EXPORT • Save model inTF • Exported to Protobuf • Useful deployment
- 85. Catalit LLC KERAS MULTI GPU • Data Parallel • Issue with LSTM layer in TF 2.0 alpha
- 86. Catalit LLC TF DISTRIBUTE • Many strategies • Distribute on devices • Distribute on clusters • Data parallel • Model parallel • ….
- 89. Catalit LLC CONCLUSIONS • Tensorflow 2.0 is coming • It will replace Keras sooner thanTensorflow 1.x • Migration from Keras andTensorflow 1.x quite easy • New opportunities • Vendor lock-in
- 90. Catalit LLC bootcamp.zerotodeeplearning.com June 3 - 7 San Francisco 10% DISCOUNT CODE: ODSC10
- 91. Catalit LLC THANKYOU Francesco Mosconi @framosconis Catalit Data Science Zero to Deep Learning