SlideShare a Scribd company logo

From REINFORCE to PPO

•
•
Woong won Lee
Woong won Lee

RLKorea의 프로젝트인 피지여행에서 진행한 내용을 정리한 것입니다. 피지여행은 DeepRL에서 중요한 Policy Gradient를 쭉 정리해보는 프로젝트입니다. PG의 처음 시작인 REINFORCE 부터 현재 새로운 baseline이 된 PPO까지 이론과 코드를 함께 살펴봅니다.

Engineering
1 of 75
Download to read offline
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO
From REINFORCE to PPO

Recommended

Doing Deep Reinforcement learning with PPO
Doing Deep Reinforcement learning with PPODoing Deep Reinforcement learning with PPO
Doing Deep Reinforcement learning with PPO이 의령
 
가깝고도 먼 Trpo
가깝고도 먼 Trpo가깝고도 먼 Trpo
가깝고도 먼 TrpoWoong won Lee
 
강화학습 알고리즘의 흐름도 Part 2
강화학습 알고리즘의 흐름도 Part 2강화학습 알고리즘의 흐름도 Part 2
강화학습 알고리즘의 흐름도 Part 2Dongmin Lee
 
강화학습의 흐름도 Part 1
강화학습의 흐름도 Part 1강화학습의 흐름도 Part 1
강화학습의 흐름도 Part 1Dongmin Lee
 
RLCode와 A3C 쉽고 깊게 이해하기
RLCode와 A3C 쉽고 깊게 이해하기RLCode와 A3C 쉽고 깊게 이해하기
RLCode와 A3C 쉽고 깊게 이해하기Woong won Lee
 
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기NAVER D2
 
Introduction to SAC(Soft Actor-Critic)
Introduction to SAC(Soft Actor-Critic)Introduction to SAC(Soft Actor-Critic)
Introduction to SAC(Soft Actor-Critic)Suhyun Cho
 
[2017 PYCON 튜토리얼]OpenAI Gym을 이용한 강화학습 에이전트 만들기
[2017 PYCON 튜토리얼]OpenAI Gym을 이용한 강화학습 에이전트 만들기[2017 PYCON 튜토리얼]OpenAI Gym을 이용한 강화학습 에이전트 만들기
[2017 PYCON 튜토리얼]OpenAI Gym을 이용한 강화학습 에이전트 만들기이 의령
 

Recommended

Doing Deep Reinforcement learning with PPO
Doing Deep Reinforcement learning with PPODoing Deep Reinforcement learning with PPO
Doing Deep Reinforcement learning with PPO이 의령
 
가깝고도 먼 Trpo
가깝고도 먼 Trpo가깝고도 먼 Trpo
가깝고도 먼 TrpoWoong won Lee
 
강화학습 알고리즘의 흐름도 Part 2
강화학습 알고리즘의 흐름도 Part 2강화학습 알고리즘의 흐름도 Part 2
강화학습 알고리즘의 흐름도 Part 2Dongmin Lee
 
강화학습의 흐름도 Part 1
강화학습의 흐름도 Part 1강화학습의 흐름도 Part 1
강화학습의 흐름도 Part 1Dongmin Lee
 
RLCode와 A3C 쉽고 깊게 이해하기
RLCode와 A3C 쉽고 깊게 이해하기RLCode와 A3C 쉽고 깊게 이해하기
RLCode와 A3C 쉽고 깊게 이해하기Woong won Lee
 
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기NAVER D2
 
Introduction to SAC(Soft Actor-Critic)
Introduction to SAC(Soft Actor-Critic)Introduction to SAC(Soft Actor-Critic)
Introduction to SAC(Soft Actor-Critic)Suhyun Cho
 
[2017 PYCON 튜토리얼]OpenAI Gym을 이용한 강화학습 에이전트 만들기
[2017 PYCON 튜토리얼]OpenAI Gym을 이용한 강화학습 에이전트 만들기[2017 PYCON 튜토리얼]OpenAI Gym을 이용한 강화학습 에이전트 만들기
[2017 PYCON 튜토리얼]OpenAI Gym을 이용한 강화학습 에이전트 만들기이 의령
 
Control as Inference.pptx
Control as Inference.pptxControl as Inference.pptx
Control as Inference.pptxssuserbd1647
 
Let's do Inverse RL
Let's do Inverse RLLet's do Inverse RL
Let's do Inverse RLDongmin Lee
 
[study] pointer networks
[study] pointer networks[study] pointer networks
[study] pointer networksGyuhyeon Nam
 
파이썬과 케라스로 배우는 강화학습 저자특강
파이썬과 케라스로 배우는 강화학습 저자특강파이썬과 케라스로 배우는 강화학습 저자특강
파이썬과 케라스로 배우는 강화학습 저자특강Woong won Lee
 
안.전.제.일. 강화학습!
안.전.제.일. 강화학습!안.전.제.일. 강화학습!
안.전.제.일. 강화학습!Dongmin Lee
 
Introduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningIntroduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningNAVER Engineering
 
강화학습의 개요
강화학습의 개요강화학습의 개요
강화학습의 개요Dongmin Lee
 
강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)
강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)
강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)Curt Park
 
Natural Policy Gradient 직관적 접근
Natural Policy Gradient 직관적 접근Natural Policy Gradient 직관적 접근
Natural Policy Gradient 직관적 접근Sooyoung Moon
 
강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)
강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)
강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)Euijin Jeong
 
Model based rl
Model based rlModel based rl
Model based rlSeolhokim
 
Trust Region Policy Optimization, Schulman et al, 2015
Trust Region Policy Optimization, Schulman et al, 2015Trust Region Policy Optimization, Schulman et al, 2015
Trust Region Policy Optimization, Schulman et al, 2015Chris Ohk
 
Matrix calculus
Matrix calculusMatrix calculus
Matrix calculusSungbin Lim
 
2018 06-11-active-question-answering
2018 06-11-active-question-answering2018 06-11-active-question-answering
2018 06-11-active-question-answeringWoong won Lee
 
An introduction to deep reinforcement learning
An introduction to deep reinforcement learningAn introduction to deep reinforcement learning
An introduction to deep reinforcement learningBig Data Colombia
 
Deep Reinforcement Learning
Deep Reinforcement LearningDeep Reinforcement Learning
Deep Reinforcement LearningMeetupDataScienceRoma
 
[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기
[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기
[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기JungHyun Hong
 
Self-Attention with Linear Complexity
Self-Attention with Linear ComplexitySelf-Attention with Linear Complexity
Self-Attention with Linear ComplexitySangwoo Mo
 
Actor critic algorithm
Actor critic algorithmActor critic algorithm
Actor critic algorithmJie-Han Chen
 
Soft Actor-Critic Algorithms and Applications 한국어 리뷰
Soft Actor-Critic Algorithms and Applications 한국어 리뷰Soft Actor-Critic Algorithms and Applications 한국어 리뷰
Soft Actor-Critic Algorithms and Applications 한국어 리뷰태영 정
 

More Related Content

What's hot

Control as Inference.pptx
Control as Inference.pptxControl as Inference.pptx
Control as Inference.pptxssuserbd1647
 
Let's do Inverse RL
Let's do Inverse RLLet's do Inverse RL
Let's do Inverse RLDongmin Lee
 
[study] pointer networks
[study] pointer networks[study] pointer networks
[study] pointer networksGyuhyeon Nam
 
파이썬과 케라스로 배우는 강화학습 저자특강
파이썬과 케라스로 배우는 강화학습 저자특강파이썬과 케라스로 배우는 강화학습 저자특강
파이썬과 케라스로 배우는 강화학습 저자특강Woong won Lee
 
안.전.제.일. 강화학습!
안.전.제.일. 강화학습!안.전.제.일. 강화학습!
안.전.제.일. 강화학습!Dongmin Lee
 
Introduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningIntroduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningNAVER Engineering
 
강화학습의 개요
강화학습의 개요강화학습의 개요
강화학습의 개요Dongmin Lee
 
강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)
강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)
강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)Curt Park
 
Natural Policy Gradient 직관적 접근
Natural Policy Gradient 직관적 접근Natural Policy Gradient 직관적 접근
Natural Policy Gradient 직관적 접근Sooyoung Moon
 
강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)
강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)
강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)Euijin Jeong
 
Model based rl
Model based rlModel based rl
Model based rlSeolhokim
 
Trust Region Policy Optimization, Schulman et al, 2015
Trust Region Policy Optimization, Schulman et al, 2015Trust Region Policy Optimization, Schulman et al, 2015
Trust Region Policy Optimization, Schulman et al, 2015Chris Ohk
 
2018 06-11-active-question-answering
2018 06-11-active-question-answering2018 06-11-active-question-answering
2018 06-11-active-question-answeringWoong won Lee
 
An introduction to deep reinforcement learning
An introduction to deep reinforcement learningAn introduction to deep reinforcement learning
An introduction to deep reinforcement learningBig Data Colombia
 
[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기
[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기
[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기JungHyun Hong
 
Self-Attention with Linear Complexity
Self-Attention with Linear ComplexitySelf-Attention with Linear Complexity
Self-Attention with Linear ComplexitySangwoo Mo
 
Actor critic algorithm
Actor critic algorithmActor critic algorithm
Actor critic algorithmJie-Han Chen
 
Soft Actor-Critic Algorithms and Applications 한국어 리뷰
Soft Actor-Critic Algorithms and Applications 한국어 리뷰Soft Actor-Critic Algorithms and Applications 한국어 리뷰
Soft Actor-Critic Algorithms and Applications 한국어 리뷰태영 정
 

What's hot (20)

Control as Inference.pptx
Control as Inference.pptxControl as Inference.pptx
Control as Inference.pptx
 
Let's do Inverse RL
Let's do Inverse RLLet's do Inverse RL
Let's do Inverse RL
 
[study] pointer networks
[study] pointer networks[study] pointer networks
[study] pointer networks
 
파이썬과 케라스로 배우는 강화학습 저자특강
파이썬과 케라스로 배우는 강화학습 저자특강파이썬과 케라스로 배우는 강화학습 저자특강
파이썬과 케라스로 배우는 강화학습 저자특강
 
안.전.제.일. 강화학습!
안.전.제.일. 강화학습!안.전.제.일. 강화학습!
안.전.제.일. 강화학습!
 
Introduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningIntroduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement Learning
 
강화학습의 개요
강화학습의 개요강화학습의 개요
강화학습의 개요
 
강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)
강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)
강화학습 기초부터 DQN까지 (Reinforcement Learning from Basics to DQN)
 
Natural Policy Gradient 직관적 접근
Natural Policy Gradient 직관적 접근Natural Policy Gradient 직관적 접근
Natural Policy Gradient 직관적 접근
 
강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)
강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)
강화학습 기초_2(Deep sarsa, Deep Q-learning, DQN)
 
Model based rl
Model based rlModel based rl
Model based rl
 
Trust Region Policy Optimization, Schulman et al, 2015
Trust Region Policy Optimization, Schulman et al, 2015Trust Region Policy Optimization, Schulman et al, 2015
Trust Region Policy Optimization, Schulman et al, 2015
 
Matrix calculus
Matrix calculusMatrix calculus
Matrix calculus
 
2018 06-11-active-question-answering
2018 06-11-active-question-answering2018 06-11-active-question-answering
2018 06-11-active-question-answering
 
An introduction to deep reinforcement learning
An introduction to deep reinforcement learningAn introduction to deep reinforcement learning
An introduction to deep reinforcement learning
 
Deep Reinforcement Learning
Deep Reinforcement LearningDeep Reinforcement Learning
Deep Reinforcement Learning
 
[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기
[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기
[GomGuard] 뉴런부터 YOLO 까지 - 딥러닝 전반에 대한 이야기
 
Self-Attention with Linear Complexity
Self-Attention with Linear ComplexitySelf-Attention with Linear Complexity
Self-Attention with Linear Complexity
 
Actor critic algorithm
Actor critic algorithmActor critic algorithm
Actor critic algorithm
 
Soft Actor-Critic Algorithms and Applications 한국어 리뷰
Soft Actor-Critic Algorithms and Applications 한국어 리뷰Soft Actor-Critic Algorithms and Applications 한국어 리뷰
Soft Actor-Critic Algorithms and Applications 한국어 리뷰