SlideShare a Scribd company logo
1 of 13
Introduction to TensorFlow and PyTorch
Kendall Chuang and David Clark
February 16, 2017
Acknowledgements
Thank you to Tubular Labs for hosting this workshop!
Who We Are
● David Clark
○ Data Science Consultant
○ http://www.dmclark5.com/
● Kendall Chuang
○ Senior Python Engineer at Ayasdi
Objectives
● Learn basic concepts of neural networks
● Work through tutorial on Tensor Flow
● Work through tutorial on PyTorch
● Compare/contrast each deep learning framework
What is Tensor Flow?
● Developed by the Google Brain Team and open-sourced in
November 2015
○ Graph based
○ Nodes are operations
○ Edges are multi-dimensional arrays called tensors
● All operations are done outside of Python
● Inputs are stored in a placeholder() or Variable()
○ placeholder(): fixed input
○ Variable(): variable input
● Inputs are populated during a Tensor Flow session
• Vector: One-dimension
• Matrix: Two-dimensions
• Tensor: n-dimensions
What is a tensor?
Image from http://noaxiom.org/tensor.
• Tensor Flow stores data in tensors
What is PyTorch?
● Python wrapper over Torch (C++ Library) released in Jan 2017
● Also stores data and variables as Tensors
● Developed by Facebook and other companies
○ Graph based
○ Nodes are operations
○ Edges are multi-dimensional arrays called tensors
● Deep Learning operations are done outside of Python
● Tensor and Variable objects
○ Tensor(): inputs/outputs
○ Variable(): variable
● Does not have the concept of a session
• Machine learning based on how
the brain works
• Input is passed through neurons.
• Weights and biases applied.
• Decisions made using activation
function.
What is a neural network?
Neural Networks and Deep Learning
What is a neural network?
● Many inputs can pass through a single neuron
● Hidden layers
○ Layer of neurons between inputs and outputs
○ Increased modeling complexity
● Inputs, hidden layers, outputs => neural network
● Deep learning
○ Many hidden layers
○ Can make very complex decisions
Neural Networks and Deep Learning
Softmax Regression
● Regression for j = 1,... K
● Algorithm for categorical data
● Evidence: Weighted sum of chemical properties.
○ Negative weight: is not of that quality.
○ Positive weight: is of that quality.
● Convert evidence to predicted probabilities using softmax equation
Cost/Loss Function
• Model trained by minimizing the cost function
• Cross-entropy cost function:
• y = softmax(E)
• y’ = labels
• More Info:
https://rdipietro.github.io/friendly-intro-to-cross-entropy-loss/
Back Propagation
• Backpropagation is used to evaluate how variables affect the
minimization of the cost function
Training Loop
1. Forward Propagation (feed in variables forward,
calculate the results)
2. Calculate Loss at the Output
3. Back Propagation (propagate loss backwards,
estimate error)
4. Optimization with Gradient Descent Algorithm
5. Repeat until done!

More Related Content

What's hot

Language translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlowLanguage translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlowS N
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to ChainerShunta Saito
 
TensorFlow Dev Summit 2017 요약
TensorFlow Dev Summit 2017 요약TensorFlow Dev Summit 2017 요약
TensorFlow Dev Summit 2017 요약Jin Joong Kim
 
TensorFlow Dev Summit 2018 Extended: TensorFlow Eager Execution
TensorFlow Dev Summit 2018 Extended: TensorFlow Eager ExecutionTensorFlow Dev Summit 2018 Extended: TensorFlow Eager Execution
TensorFlow Dev Summit 2018 Extended: TensorFlow Eager ExecutionTaegyun Jeon
 
Teaching Recurrent Neural Networks using Tensorflow (May 2016)
Teaching Recurrent Neural Networks using Tensorflow (May 2016)Teaching Recurrent Neural Networks using Tensorflow (May 2016)
Teaching Recurrent Neural Networks using Tensorflow (May 2016)Rajiv Shah
 
Scalable Deep Learning Using Apache MXNet
Scalable Deep Learning Using Apache MXNetScalable Deep Learning Using Apache MXNet
Scalable Deep Learning Using Apache MXNetAmazon Web Services
 
An Introduction to TensorFlow architecture
An Introduction to TensorFlow architectureAn Introduction to TensorFlow architecture
An Introduction to TensorFlow architectureMani Goswami
 
Applying your Convolutional Neural Networks
Applying your Convolutional Neural NetworksApplying your Convolutional Neural Networks
Applying your Convolutional Neural NetworksDatabricks
 
Deep learning for molecules, introduction to chainer chemistry
Deep learning for molecules, introduction to chainer chemistryDeep learning for molecules, introduction to chainer chemistry
Deep learning for molecules, introduction to chainer chemistryKenta Oono
 
Electricity price forecasting with Recurrent Neural Networks
Electricity price forecasting with Recurrent Neural NetworksElectricity price forecasting with Recurrent Neural Networks
Electricity price forecasting with Recurrent Neural NetworksTaegyun Jeon
 
Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Oswald Campesato
 
Scaling Python to CPUs and GPUs
Scaling Python to CPUs and GPUsScaling Python to CPUs and GPUs
Scaling Python to CPUs and GPUsTravis Oliphant
 
Neural networks and google tensor flow
Neural networks and google tensor flowNeural networks and google tensor flow
Neural networks and google tensor flowShannon McCormick
 
深層学習フレームワーク概要とChainerの事例紹介
深層学習フレームワーク概要とChainerの事例紹介深層学習フレームワーク概要とChainerの事例紹介
深層学習フレームワーク概要とChainerの事例紹介Kenta Oono
 
What is TensorFlow? | Introduction to TensorFlow | TensorFlow Tutorial For Be...
What is TensorFlow? | Introduction to TensorFlow | TensorFlow Tutorial For Be...What is TensorFlow? | Introduction to TensorFlow | TensorFlow Tutorial For Be...
What is TensorFlow? | Introduction to TensorFlow | TensorFlow Tutorial For Be...Simplilearn
 
Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowSri Ambati
 
Neural Networks with Google TensorFlow
Neural Networks with Google TensorFlowNeural Networks with Google TensorFlow
Neural Networks with Google TensorFlowDarshan Patel
 

What's hot (20)

Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to Chainer
 
Language translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlowLanguage translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlow
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to Chainer
 
TensorFlow Dev Summit 2017 요약
TensorFlow Dev Summit 2017 요약TensorFlow Dev Summit 2017 요약
TensorFlow Dev Summit 2017 요약
 
TensorFlow Dev Summit 2018 Extended: TensorFlow Eager Execution
TensorFlow Dev Summit 2018 Extended: TensorFlow Eager ExecutionTensorFlow Dev Summit 2018 Extended: TensorFlow Eager Execution
TensorFlow Dev Summit 2018 Extended: TensorFlow Eager Execution
 
Teaching Recurrent Neural Networks using Tensorflow (May 2016)
Teaching Recurrent Neural Networks using Tensorflow (May 2016)Teaching Recurrent Neural Networks using Tensorflow (May 2016)
Teaching Recurrent Neural Networks using Tensorflow (May 2016)
 
Scalable Deep Learning Using Apache MXNet
Scalable Deep Learning Using Apache MXNetScalable Deep Learning Using Apache MXNet
Scalable Deep Learning Using Apache MXNet
 
An Introduction to TensorFlow architecture
An Introduction to TensorFlow architectureAn Introduction to TensorFlow architecture
An Introduction to TensorFlow architecture
 
Applying your Convolutional Neural Networks
Applying your Convolutional Neural NetworksApplying your Convolutional Neural Networks
Applying your Convolutional Neural Networks
 
Deep learning for molecules, introduction to chainer chemistry
Deep learning for molecules, introduction to chainer chemistryDeep learning for molecules, introduction to chainer chemistry
Deep learning for molecules, introduction to chainer chemistry
 
TensorFlow
TensorFlowTensorFlow
TensorFlow
 
Electricity price forecasting with Recurrent Neural Networks
Electricity price forecasting with Recurrent Neural NetworksElectricity price forecasting with Recurrent Neural Networks
Electricity price forecasting with Recurrent Neural Networks
 
Machine Intelligence at Google Scale: TensorFlow
Machine Intelligence at Google Scale: TensorFlowMachine Intelligence at Google Scale: TensorFlow
Machine Intelligence at Google Scale: TensorFlow
 
Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)
 
Scaling Python to CPUs and GPUs
Scaling Python to CPUs and GPUsScaling Python to CPUs and GPUs
Scaling Python to CPUs and GPUs
 
Neural networks and google tensor flow
Neural networks and google tensor flowNeural networks and google tensor flow
Neural networks and google tensor flow
 
深層学習フレームワーク概要とChainerの事例紹介
深層学習フレームワーク概要とChainerの事例紹介深層学習フレームワーク概要とChainerの事例紹介
深層学習フレームワーク概要とChainerの事例紹介
 
What is TensorFlow? | Introduction to TensorFlow | TensorFlow Tutorial For Be...
What is TensorFlow? | Introduction to TensorFlow | TensorFlow Tutorial For Be...What is TensorFlow? | Introduction to TensorFlow | TensorFlow Tutorial For Be...
What is TensorFlow? | Introduction to TensorFlow | TensorFlow Tutorial For Be...
 
Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlow
 
Neural Networks with Google TensorFlow
Neural Networks with Google TensorFlowNeural Networks with Google TensorFlow
Neural Networks with Google TensorFlow
 

Viewers also liked

20150306 파이썬기초 IPython을이용한프로그래밍_이태영
20150306 파이썬기초 IPython을이용한프로그래밍_이태영20150306 파이썬기초 IPython을이용한프로그래밍_이태영
20150306 파이썬기초 IPython을이용한프로그래밍_이태영Tae Young Lee
 
High Performance Distributed TensorFlow with GPUs - NYC Workshop - July 9 2017
High Performance Distributed TensorFlow with GPUs - NYC Workshop - July 9 2017High Performance Distributed TensorFlow with GPUs - NYC Workshop - July 9 2017
High Performance Distributed TensorFlow with GPUs - NYC Workshop - July 9 2017Chris Fregly
 
Introduction to Deep Learning with TensorFlow
Introduction to Deep Learning with TensorFlowIntroduction to Deep Learning with TensorFlow
Introduction to Deep Learning with TensorFlowTerry Taewoong Um
 
TensorFlow Tutorial
TensorFlow TutorialTensorFlow Tutorial
TensorFlow TutorialNamHyuk Ahn
 
Environmental Audit and Environmental Impact Assessment
Environmental Audit and Environmental Impact AssessmentEnvironmental Audit and Environmental Impact Assessment
Environmental Audit and Environmental Impact AssessmentEffah Effervescence
 
Seminar on Environmental Impact Assessment
Seminar on Environmental Impact AssessmentSeminar on Environmental Impact Assessment
Seminar on Environmental Impact Assessmentashwinpand90
 
Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessmentSayyid Ina
 
ENVIRONMENTAL IMPACT ASSESSMENT - EIA
ENVIRONMENTAL IMPACT ASSESSMENT - EIAENVIRONMENTAL IMPACT ASSESSMENT - EIA
ENVIRONMENTAL IMPACT ASSESSMENT - EIASakthivel R
 
Environmental Impact Assessment - University of Winnipeg
Environmental Impact Assessment - University of WinnipegEnvironmental Impact Assessment - University of Winnipeg
Environmental Impact Assessment - University of WinnipegJohn Gunter
 
Deep Learning with PyTorch
Deep Learning with PyTorchDeep Learning with PyTorch
Deep Learning with PyTorchMayur Bhangale
 
Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessmentKashmeera N.A.
 
13 environmental impact assessment
13 environmental impact assessment13 environmental impact assessment
13 environmental impact assessmentPrabha Panth
 
Eia - environmental impact assessment
Eia - environmental impact assessmentEia - environmental impact assessment
Eia - environmental impact assessmentParth Patel
 
PyTorch Tutorial for NTU Machine Learing Course 2017
PyTorch Tutorial for NTU Machine Learing Course 2017PyTorch Tutorial for NTU Machine Learing Course 2017
PyTorch Tutorial for NTU Machine Learing Course 2017Yu-Hsun (lymanblue) Lin
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기NAVER Engineering
 
Environmental impact assessment methodology by Dr. I.M. Mishra Professor, Dep...
Environmental impact assessment methodology by Dr. I.M. Mishra Professor, Dep...Environmental impact assessment methodology by Dr. I.M. Mishra Professor, Dep...
Environmental impact assessment methodology by Dr. I.M. Mishra Professor, Dep...Arvind Kumar
 
Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessmentJini Rajendran
 
Planning and environmental impact assessment (EIA)
Planning and environmental impact assessment (EIA)Planning and environmental impact assessment (EIA)
Planning and environmental impact assessment (EIA)tesdev
 

Viewers also liked (20)

Intro to Python
Intro to PythonIntro to Python
Intro to Python
 
20150306 파이썬기초 IPython을이용한프로그래밍_이태영
20150306 파이썬기초 IPython을이용한프로그래밍_이태영20150306 파이썬기초 IPython을이용한프로그래밍_이태영
20150306 파이썬기초 IPython을이용한프로그래밍_이태영
 
High Performance Distributed TensorFlow with GPUs - NYC Workshop - July 9 2017
High Performance Distributed TensorFlow with GPUs - NYC Workshop - July 9 2017High Performance Distributed TensorFlow with GPUs - NYC Workshop - July 9 2017
High Performance Distributed TensorFlow with GPUs - NYC Workshop - July 9 2017
 
Introduction to Deep Learning with TensorFlow
Introduction to Deep Learning with TensorFlowIntroduction to Deep Learning with TensorFlow
Introduction to Deep Learning with TensorFlow
 
TensorFlow Tutorial
TensorFlow TutorialTensorFlow Tutorial
TensorFlow Tutorial
 
Environmental Audit and Environmental Impact Assessment
Environmental Audit and Environmental Impact AssessmentEnvironmental Audit and Environmental Impact Assessment
Environmental Audit and Environmental Impact Assessment
 
Seminar on Environmental Impact Assessment
Seminar on Environmental Impact AssessmentSeminar on Environmental Impact Assessment
Seminar on Environmental Impact Assessment
 
Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessment
 
ENVIRONMENTAL IMPACT ASSESSMENT - EIA
ENVIRONMENTAL IMPACT ASSESSMENT - EIAENVIRONMENTAL IMPACT ASSESSMENT - EIA
ENVIRONMENTAL IMPACT ASSESSMENT - EIA
 
Environmental Impact Assessment - University of Winnipeg
Environmental Impact Assessment - University of WinnipegEnvironmental Impact Assessment - University of Winnipeg
Environmental Impact Assessment - University of Winnipeg
 
Deep Learning with PyTorch
Deep Learning with PyTorchDeep Learning with PyTorch
Deep Learning with PyTorch
 
Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessment
 
13 environmental impact assessment
13 environmental impact assessment13 environmental impact assessment
13 environmental impact assessment
 
Eia - environmental impact assessment
Eia - environmental impact assessmentEia - environmental impact assessment
Eia - environmental impact assessment
 
PyTorch Tutorial for NTU Machine Learing Course 2017
PyTorch Tutorial for NTU Machine Learing Course 2017PyTorch Tutorial for NTU Machine Learing Course 2017
PyTorch Tutorial for NTU Machine Learing Course 2017
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
 
Environmental impact assessment methodology by Dr. I.M. Mishra Professor, Dep...
Environmental impact assessment methodology by Dr. I.M. Mishra Professor, Dep...Environmental impact assessment methodology by Dr. I.M. Mishra Professor, Dep...
Environmental impact assessment methodology by Dr. I.M. Mishra Professor, Dep...
 
Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessment
 
Planning and environmental impact assessment (EIA)
Planning and environmental impact assessment (EIA)Planning and environmental impact assessment (EIA)
Planning and environmental impact assessment (EIA)
 
Environmental impact assessment
Environmental impact assessmentEnvironmental impact assessment
Environmental impact assessment
 

Similar to Intro to TensorFlow and PyTorch Workshop at Tubular Labs

Artificial Intelligence Chapter 9 Negnevitsky
Artificial Intelligence Chapter 9 NegnevitskyArtificial Intelligence Chapter 9 Negnevitsky
Artificial Intelligence Chapter 9 Negnevitskylopanath
 
Deep_Learning_Frameworks_CNTK_PyTorch
Deep_Learning_Frameworks_CNTK_PyTorchDeep_Learning_Frameworks_CNTK_PyTorch
Deep_Learning_Frameworks_CNTK_PyTorchSubhashis Hazarika
 
An Introduction to Deep Learning
An Introduction to Deep LearningAn Introduction to Deep Learning
An Introduction to Deep LearningPoo Kuan Hoong
 
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
MDEC Data Matters Series: machine learning and Deep Learning, A PrimerMDEC Data Matters Series: machine learning and Deep Learning, A Primer
MDEC Data Matters Series: machine learning and Deep Learning, A PrimerPoo Kuan Hoong
 
Deep Learning Tutorial
Deep Learning Tutorial Deep Learning Tutorial
Deep Learning Tutorial Ligeng Zhu
 
Entity embeddings for categorical data
Entity embeddings for categorical dataEntity embeddings for categorical data
Entity embeddings for categorical dataPaul Skeie
 
Cvpr 2018 papers review (efficient computing)
Cvpr 2018 papers review (efficient computing)Cvpr 2018 papers review (efficient computing)
Cvpr 2018 papers review (efficient computing)DonghyunKang12
 
Fundamental of deep learning
Fundamental of deep learningFundamental of deep learning
Fundamental of deep learningStanley Wang
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learningVishwas Lele
 
Deep Learning Made Easy with Deep Features
Deep Learning Made Easy with Deep FeaturesDeep Learning Made Easy with Deep Features
Deep Learning Made Easy with Deep FeaturesTuri, Inc.
 
Machine Learning, Deep Learning and Data Analysis Introduction
Machine Learning, Deep Learning and Data Analysis IntroductionMachine Learning, Deep Learning and Data Analysis Introduction
Machine Learning, Deep Learning and Data Analysis IntroductionTe-Yen Liu
 
NLP and Deep Learning for non_experts
NLP and Deep Learning for non_expertsNLP and Deep Learning for non_experts
NLP and Deep Learning for non_expertsSanghamitra Deb
 
DSRLab seminar Introduction to deep learning
DSRLab seminar   Introduction to deep learningDSRLab seminar   Introduction to deep learning
DSRLab seminar Introduction to deep learningPoo Kuan Hoong
 
Reproducing and Analyzing Adaptive Computation Time in PyTorch and TensorFlow
Reproducing and Analyzing Adaptive Computation Time in PyTorch and TensorFlowReproducing and Analyzing Adaptive Computation Time in PyTorch and TensorFlow
Reproducing and Analyzing Adaptive Computation Time in PyTorch and TensorFlowUniversitat Politècnica de Catalunya
 
Deep Learning Interview Questions And Answers | AI & Deep Learning Interview ...
Deep Learning Interview Questions And Answers | AI & Deep Learning Interview ...Deep Learning Interview Questions And Answers | AI & Deep Learning Interview ...
Deep Learning Interview Questions And Answers | AI & Deep Learning Interview ...Simplilearn
 
Deep Learning in Recommender Systems - RecSys Summer School 2017
Deep Learning in Recommender Systems - RecSys Summer School 2017Deep Learning in Recommender Systems - RecSys Summer School 2017
Deep Learning in Recommender Systems - RecSys Summer School 2017Balázs Hidasi
 

Similar to Intro to TensorFlow and PyTorch Workshop at Tubular Labs (20)

Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
 
Artificial Intelligence Chapter 9 Negnevitsky
Artificial Intelligence Chapter 9 NegnevitskyArtificial Intelligence Chapter 9 Negnevitsky
Artificial Intelligence Chapter 9 Negnevitsky
 
Development of Deep Learning Architecture
Development of Deep Learning ArchitectureDevelopment of Deep Learning Architecture
Development of Deep Learning Architecture
 
Deep_Learning_Frameworks_CNTK_PyTorch
Deep_Learning_Frameworks_CNTK_PyTorchDeep_Learning_Frameworks_CNTK_PyTorch
Deep_Learning_Frameworks_CNTK_PyTorch
 
Icon18revrec sudeshna
Icon18revrec sudeshnaIcon18revrec sudeshna
Icon18revrec sudeshna
 
An Introduction to Deep Learning
An Introduction to Deep LearningAn Introduction to Deep Learning
An Introduction to Deep Learning
 
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
MDEC Data Matters Series: machine learning and Deep Learning, A PrimerMDEC Data Matters Series: machine learning and Deep Learning, A Primer
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
 
Deep Learning Tutorial
Deep Learning Tutorial Deep Learning Tutorial
Deep Learning Tutorial
 
Entity embeddings for categorical data
Entity embeddings for categorical dataEntity embeddings for categorical data
Entity embeddings for categorical data
 
Cvpr 2018 papers review (efficient computing)
Cvpr 2018 papers review (efficient computing)Cvpr 2018 papers review (efficient computing)
Cvpr 2018 papers review (efficient computing)
 
Fundamental of deep learning
Fundamental of deep learningFundamental of deep learning
Fundamental of deep learning
 
Deep learning
Deep learningDeep learning
Deep learning
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
 
Deep Learning Made Easy with Deep Features
Deep Learning Made Easy with Deep FeaturesDeep Learning Made Easy with Deep Features
Deep Learning Made Easy with Deep Features
 
Machine Learning, Deep Learning and Data Analysis Introduction
Machine Learning, Deep Learning and Data Analysis IntroductionMachine Learning, Deep Learning and Data Analysis Introduction
Machine Learning, Deep Learning and Data Analysis Introduction
 
NLP and Deep Learning for non_experts
NLP and Deep Learning for non_expertsNLP and Deep Learning for non_experts
NLP and Deep Learning for non_experts
 
DSRLab seminar Introduction to deep learning
DSRLab seminar   Introduction to deep learningDSRLab seminar   Introduction to deep learning
DSRLab seminar Introduction to deep learning
 
Reproducing and Analyzing Adaptive Computation Time in PyTorch and TensorFlow
Reproducing and Analyzing Adaptive Computation Time in PyTorch and TensorFlowReproducing and Analyzing Adaptive Computation Time in PyTorch and TensorFlow
Reproducing and Analyzing Adaptive Computation Time in PyTorch and TensorFlow
 
Deep Learning Interview Questions And Answers | AI & Deep Learning Interview ...
Deep Learning Interview Questions And Answers | AI & Deep Learning Interview ...Deep Learning Interview Questions And Answers | AI & Deep Learning Interview ...
Deep Learning Interview Questions And Answers | AI & Deep Learning Interview ...
 
Deep Learning in Recommender Systems - RecSys Summer School 2017
Deep Learning in Recommender Systems - RecSys Summer School 2017Deep Learning in Recommender Systems - RecSys Summer School 2017
Deep Learning in Recommender Systems - RecSys Summer School 2017
 

Recently uploaded

Lecture 1: Basics of trigonometry (surveying)
Lecture 1: Basics of trigonometry (surveying)Lecture 1: Basics of trigonometry (surveying)
Lecture 1: Basics of trigonometry (surveying)Bahzad5
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid BodyAhmadHajasad2
 
me3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Ame3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Akarthi keyan
 
Nodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxNodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxwendy cai
 
cloud computing notes for anna university syllabus
cloud computing notes for anna university syllabuscloud computing notes for anna university syllabus
cloud computing notes for anna university syllabusViolet Violet
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical SensorTanvir Moin
 
Multicomponent Spiral Wound Membrane Separation Model.pdf
Multicomponent Spiral Wound Membrane Separation Model.pdfMulticomponent Spiral Wound Membrane Separation Model.pdf
Multicomponent Spiral Wound Membrane Separation Model.pdfGiovanaGhasary1
 
nvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptxnvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptxjasonsedano2
 
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxSUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxNaveenVerma126
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvementVijayMuni2
 
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Sean Meyn
 
The relationship between iot and communication technology
The relationship between iot and communication technologyThe relationship between iot and communication technology
The relationship between iot and communication technologyabdulkadirmukarram03
 
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...soginsider
 
solar wireless electric vechicle charging system
solar wireless electric vechicle charging systemsolar wireless electric vechicle charging system
solar wireless electric vechicle charging systemgokuldongala
 
Design of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxDesign of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxYogeshKumarKJMIT
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfJulia Kaye
 

Recently uploaded (20)

Lecture 1: Basics of trigonometry (surveying)
Lecture 1: Basics of trigonometry (surveying)Lecture 1: Basics of trigonometry (surveying)
Lecture 1: Basics of trigonometry (surveying)
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
 
me3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Ame3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part A
 
Nodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxNodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptx
 
cloud computing notes for anna university syllabus
cloud computing notes for anna university syllabuscloud computing notes for anna university syllabus
cloud computing notes for anna university syllabus
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical Sensor
 
Multicomponent Spiral Wound Membrane Separation Model.pdf
Multicomponent Spiral Wound Membrane Separation Model.pdfMulticomponent Spiral Wound Membrane Separation Model.pdf
Multicomponent Spiral Wound Membrane Separation Model.pdf
 
nvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptxnvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptx
 
Lecture 4 .pdf
Lecture 4                              .pdfLecture 4                              .pdf
Lecture 4 .pdf
 
Litature Review: Research Paper work for Engineering
Litature Review: Research Paper work for EngineeringLitature Review: Research Paper work for Engineering
Litature Review: Research Paper work for Engineering
 
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxSUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvement
 
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
 
Lecture 2 .pptx
Lecture 2                            .pptxLecture 2                            .pptx
Lecture 2 .pptx
 
The relationship between iot and communication technology
The relationship between iot and communication technologyThe relationship between iot and communication technology
The relationship between iot and communication technology
 
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
 
solar wireless electric vechicle charging system
solar wireless electric vechicle charging systemsolar wireless electric vechicle charging system
solar wireless electric vechicle charging system
 
Présentation IIRB 2024 Chloe Dufrane.pdf
Présentation IIRB 2024 Chloe Dufrane.pdfPrésentation IIRB 2024 Chloe Dufrane.pdf
Présentation IIRB 2024 Chloe Dufrane.pdf
 
Design of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxDesign of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptx
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
 

Intro to TensorFlow and PyTorch Workshop at Tubular Labs

  • 1. Introduction to TensorFlow and PyTorch Kendall Chuang and David Clark February 16, 2017
  • 2. Acknowledgements Thank you to Tubular Labs for hosting this workshop!
  • 3. Who We Are ● David Clark ○ Data Science Consultant ○ http://www.dmclark5.com/ ● Kendall Chuang ○ Senior Python Engineer at Ayasdi
  • 4. Objectives ● Learn basic concepts of neural networks ● Work through tutorial on Tensor Flow ● Work through tutorial on PyTorch ● Compare/contrast each deep learning framework
  • 5. What is Tensor Flow? ● Developed by the Google Brain Team and open-sourced in November 2015 ○ Graph based ○ Nodes are operations ○ Edges are multi-dimensional arrays called tensors ● All operations are done outside of Python ● Inputs are stored in a placeholder() or Variable() ○ placeholder(): fixed input ○ Variable(): variable input ● Inputs are populated during a Tensor Flow session
  • 6. • Vector: One-dimension • Matrix: Two-dimensions • Tensor: n-dimensions What is a tensor? Image from http://noaxiom.org/tensor. • Tensor Flow stores data in tensors
  • 7. What is PyTorch? ● Python wrapper over Torch (C++ Library) released in Jan 2017 ● Also stores data and variables as Tensors ● Developed by Facebook and other companies ○ Graph based ○ Nodes are operations ○ Edges are multi-dimensional arrays called tensors ● Deep Learning operations are done outside of Python ● Tensor and Variable objects ○ Tensor(): inputs/outputs ○ Variable(): variable ● Does not have the concept of a session
  • 8. • Machine learning based on how the brain works • Input is passed through neurons. • Weights and biases applied. • Decisions made using activation function. What is a neural network? Neural Networks and Deep Learning
  • 9. What is a neural network? ● Many inputs can pass through a single neuron ● Hidden layers ○ Layer of neurons between inputs and outputs ○ Increased modeling complexity ● Inputs, hidden layers, outputs => neural network ● Deep learning ○ Many hidden layers ○ Can make very complex decisions Neural Networks and Deep Learning
  • 10. Softmax Regression ● Regression for j = 1,... K ● Algorithm for categorical data ● Evidence: Weighted sum of chemical properties. ○ Negative weight: is not of that quality. ○ Positive weight: is of that quality. ● Convert evidence to predicted probabilities using softmax equation
  • 11. Cost/Loss Function • Model trained by minimizing the cost function • Cross-entropy cost function: • y = softmax(E) • y’ = labels • More Info: https://rdipietro.github.io/friendly-intro-to-cross-entropy-loss/
  • 12. Back Propagation • Backpropagation is used to evaluate how variables affect the minimization of the cost function
  • 13. Training Loop 1. Forward Propagation (feed in variables forward, calculate the results) 2. Calculate Loss at the Output 3. Back Propagation (propagate loss backwards, estimate error) 4. Optimization with Gradient Descent Algorithm 5. Repeat until done!