SlideShare a Scribd company logo
1 of 34
# 
Frank Compagner 
Guerrilla Games
# 
How we learned to stop worrying and love 
the development cycle 
• Background 
• Iteration 
• Teamwork 
• Stability 
• Releasing 
Iterate 
Launch
# 
• Killzone Shadow Fall – PS4 launch title 
• 200 local users, 150 remote 
• Head revision: 1.8 TB (1.6 M files) 
• Depot size: 8 TB 
• Peak change: 180 GB / week 
• 60 dedicated build servers 
• All in Perforce since 2004
# 
Server 
Proxy Proxy Proxy Proxy 
Build Farm (60) 
Users (200) 
Proxy 
Cambridge 
Proxy 
Guilford 
Proxy 
San Diego 
Proxy 
Amsterdam Shanghai
# 
Commit 
Server 
Edge 
Server 
Build Farm (60) Users (200) 
Cambridge 
Proxy 
Guilford 
Proxy 
San Diego 
Proxy 
Amsterdam Shanghai 
Edge 
Server 
Edge 
Server
# 
Submits 
GB / week
# 
• Monitors all vital stats of server and proxies 
• Simple Python script, produces simple html 
• Works for Windows and Linux servers 
• Proxies cleaned by another Python script 
• Both available in the Perforce Workshop
#
#
# 
Code 
Compile 
Test Programmer 
Link
# 
Code 
Compile 
IDE and Tools 
Test Programmer 
Link 
Distribute 
Incremental linker 
Code 
Hotloading
# 
Model 
Artist 
Test Process 
Hotloading
# 
Model 
Tools 
Artist 
Test Process 
Hotloading Distribute
# 
Programmer 
Rigger 
Tester 
Team 
Designer Animator
# 
Programmer 
Rigger 
Animator 
Animator 
Tester 
Designer 
Modeler 
Programmer 
Programmer 
VFX artist 
Designer 
Audio Engineer 
Modeler 
Designer 
Animator
#
# 
• Does not look like this
# 
• It’s more like this
# 
• OK, like this
# 
• Personal branch is fine (sandbox / streams / Git) 
• Feature branches sometimes work 
• Give control over your environment 
• But create distance to others 
• Team work requires frequent branch switching
# 
• For us, branch switching is always expensive: 
– Amount of data and change 
– Unmergeable files 
– Code-Data dependencies 
• Branches add complexity, we already have that 
• Team effort, we don’t want distance 
• Almost everybody works on the trunk 
• So why isn’t it always broken?
#
# 
Programmer 
Rigger 
Tester 
Team 
Designer Animator
# 
Programmer 
Submit Sync 
Rigger 
Tester 
Team 
Submit 
Submit 
Designer Animator 
Submit 
Submit 
Sync 
Sync 
Sync 
Sync
# 
Programmer 
Test 
Submit Sync 
Rigger 
Tester 
Team 
Submit 
Submit 
Designer Animator 
Submit 
Submit 
Sync 
Sync 
Sync 
Sync 
Test 
Test 
Test 
Test
# 
• Test pre submit (user) 
• Test post submit (builder) 
• Test in parallel 
• Test as fast as possible 
• Still accidents do happen 
User B 
Sync 
Sync Build Game Submit 
Sync Test Game 
Sync Build Tools Submit 
Sync Cook Data 
< 30 minutes 
User A 
Submit Sync 
Test Game 
… 
Sync 
Upload 
Cook Data Upload 
Test
# 
Build machines 
write 
Label 42: change 1200 
Label 41: change 1180 
Label 40: change 1155 
Label 39: change 1150 
… 
Labels.xml 
read 
Safe-Sync Tool 
• Each game build creates a label 
• Stored in Labels.xml file in depot 
• Builds add their result to the label 
• Builds can add artifacts to a label 
• Nobody gets latest (not safe) 
• Everybody uses safe-sync tool
Select 
tests 
Sync to 
get label 
# 
Labels that 
pass tests 
Labels that 
fail tests 
• Custom safe-sync tool shows labels and test results 
• Use it to sync to a known good label 
– This includes artifacts from other builds
#
# 
• We branch for every release (often every week) 
• Nobody works on the release branch 
• Everybody works on the trunk 
• Changes are merged up from main to release 
release 
main 
= release 
= change
• Cherry picking rules! 
• Gives you a lot of control 
• But also lots of confusion 
• And mistakes are easy 
• Good tools can fix this 
• Give flexibility 
• And control 
# 
(See whitepaper for details)
# 
• Iterate as fast as you can 
• Don’t branch too much 
• Test everything 
• Sync should be safe 
• Release all the time 
• Control what you release 
Speed 
Control
# 
Frank Compagner 
frank@guerrilla-games.com
# 
RESOURCES 
Perforce Workshop (P4 monitor, cache cleaner): 
https://swarm.workshop.perforce.com/files/guest/frank_compagner 
ReleaseManager: 
http://www.perforce.com/resources/presentations/user-conference-talks/ 
tools-for-smooth-release-cycle 
http://www.perforce.com/sites/default/files/guerrilla-games-whitepaper.pdf

More Related Content

What's hot

FME World Tour 2016: FME and continuous integration
FME World Tour 2016: FME and continuous integrationFME World Tour 2016: FME and continuous integration
FME World Tour 2016: FME and continuous integrationGIM_nv
 
Real World Elixir Deployment
Real World Elixir DeploymentReal World Elixir Deployment
Real World Elixir DeploymentPete Gamache
 
Source version control using subversion
Source version control using subversionSource version control using subversion
Source version control using subversionMangesh Bhujbal
 
Version control
Version controlVersion control
Version controlvisual28
 
Manage your environment with DSC
Manage your environment with DSCManage your environment with DSC
Manage your environment with DSCGian Maria Ricci
 
DockerCon US 2016 - Extending Docker With APIs, Drivers, and Plugins
DockerCon US 2016 - Extending Docker With APIs, Drivers, and PluginsDockerCon US 2016 - Extending Docker With APIs, Drivers, and Plugins
DockerCon US 2016 - Extending Docker With APIs, Drivers, and PluginsArnaud Porterie
 
Infrastructure as Code
Infrastructure as CodeInfrastructure as Code
Infrastructure as CodeLeandro Rosa
 
Deploying puppet code at light speed
Deploying puppet code at light speedDeploying puppet code at light speed
Deploying puppet code at light speedTomas Doran
 
Deploying Puppet Code At Light Speed - Puppet Camp Silicon Valley
Deploying Puppet Code At Light Speed - Puppet Camp Silicon ValleyDeploying Puppet Code At Light Speed - Puppet Camp Silicon Valley
Deploying Puppet Code At Light Speed - Puppet Camp Silicon ValleyPuppet
 
2013 t-dose - libre office easyhacks
2013   t-dose - libre office easyhacks2013   t-dose - libre office easyhacks
2013 t-dose - libre office easyhacksRob Snelders
 
Joe Damato
Joe DamatoJoe Damato
Joe DamatoOntico
 
Asterisk 11to16, What could go wrong
Asterisk 11to16, What could go wrongAsterisk 11to16, What could go wrong
Asterisk 11to16, What could go wrongJöran Vinzens
 
What's new in ASP.NET vNext
What's new in ASP.NET vNextWhat's new in ASP.NET vNext
What's new in ASP.NET vNextGunnar Peipman
 
Modern Commandline Tool
Modern Commandline ToolModern Commandline Tool
Modern Commandline ToolYuji Shimada
 
UEMB260: Provisioning: Under the Hood
UEMB260: Provisioning: Under the HoodUEMB260: Provisioning: Under the Hood
UEMB260: Provisioning: Under the HoodIvanti
 
How to setup a development environment for ONAP
How to setup a development environment for ONAPHow to setup a development environment for ONAP
How to setup a development environment for ONAPVictor Morales
 

What's hot (20)

FME World Tour 2016: FME and continuous integration
FME World Tour 2016: FME and continuous integrationFME World Tour 2016: FME and continuous integration
FME World Tour 2016: FME and continuous integration
 
Real World Elixir Deployment
Real World Elixir DeploymentReal World Elixir Deployment
Real World Elixir Deployment
 
Source version control using subversion
Source version control using subversionSource version control using subversion
Source version control using subversion
 
Keynote - Fluentd meetup v14
Keynote - Fluentd meetup v14Keynote - Fluentd meetup v14
Keynote - Fluentd meetup v14
 
Version control
Version controlVersion control
Version control
 
Manage your environment with DSC
Manage your environment with DSCManage your environment with DSC
Manage your environment with DSC
 
DockerCon US 2016 - Extending Docker With APIs, Drivers, and Plugins
DockerCon US 2016 - Extending Docker With APIs, Drivers, and PluginsDockerCon US 2016 - Extending Docker With APIs, Drivers, and Plugins
DockerCon US 2016 - Extending Docker With APIs, Drivers, and Plugins
 
Infrastructure as Code
Infrastructure as CodeInfrastructure as Code
Infrastructure as Code
 
Ansible E2E Testing
Ansible E2E TestingAnsible E2E Testing
Ansible E2E Testing
 
Deploying puppet code at light speed
Deploying puppet code at light speedDeploying puppet code at light speed
Deploying puppet code at light speed
 
Deploying Puppet Code At Light Speed - Puppet Camp Silicon Valley
Deploying Puppet Code At Light Speed - Puppet Camp Silicon ValleyDeploying Puppet Code At Light Speed - Puppet Camp Silicon Valley
Deploying Puppet Code At Light Speed - Puppet Camp Silicon Valley
 
2013 t-dose - libre office easyhacks
2013   t-dose - libre office easyhacks2013   t-dose - libre office easyhacks
2013 t-dose - libre office easyhacks
 
Joe Damato
Joe DamatoJoe Damato
Joe Damato
 
Symfony aws
Symfony awsSymfony aws
Symfony aws
 
How to deploy your Apps in serverless-way using App Engine.pptx
How to deploy your Apps in serverless-way using App Engine.pptxHow to deploy your Apps in serverless-way using App Engine.pptx
How to deploy your Apps in serverless-way using App Engine.pptx
 
Asterisk 11to16, What could go wrong
Asterisk 11to16, What could go wrongAsterisk 11to16, What could go wrong
Asterisk 11to16, What could go wrong
 
What's new in ASP.NET vNext
What's new in ASP.NET vNextWhat's new in ASP.NET vNext
What's new in ASP.NET vNext
 
Modern Commandline Tool
Modern Commandline ToolModern Commandline Tool
Modern Commandline Tool
 
UEMB260: Provisioning: Under the Hood
UEMB260: Provisioning: Under the HoodUEMB260: Provisioning: Under the Hood
UEMB260: Provisioning: Under the Hood
 
How to setup a development environment for ONAP
How to setup a development environment for ONAPHow to setup a development environment for ONAP
How to setup a development environment for ONAP
 

Similar to How Guerrilla Games Achieves Speed and Control in their Development Cycle

Testing sync engine
Testing sync engineTesting sync engine
Testing sync engineIlya Puchka
 
Maximize Your Production Effort (English)
Maximize Your Production Effort (English)Maximize Your Production Effort (English)
Maximize Your Production Effort (English)slantsixgames
 
Ci of js and apex using jasmine, phantom js and drone io df14
Ci of js and apex using jasmine, phantom js and drone io   df14Ci of js and apex using jasmine, phantom js and drone io   df14
Ci of js and apex using jasmine, phantom js and drone io df14Kevin Poorman
 
Inside the IT Territory game server / Mark Lokshin (IT Territory)
Inside the IT Territory game server / Mark Lokshin (IT Territory)Inside the IT Territory game server / Mark Lokshin (IT Territory)
Inside the IT Territory game server / Mark Lokshin (IT Territory)DevGAMM Conference
 
Escaping Test Hell - ACCU 2014
Escaping Test Hell - ACCU 2014Escaping Test Hell - ACCU 2014
Escaping Test Hell - ACCU 2014Wojciech Seliga
 
BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!
BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!
BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!Andreas Grabner
 
Supersize your production pipe enjmin 2013 v1.1 hd
Supersize your production pipe    enjmin 2013 v1.1 hdSupersize your production pipe    enjmin 2013 v1.1 hd
Supersize your production pipe enjmin 2013 v1.1 hdslantsixgames
 
Introduction to jenkins
Introduction to jenkinsIntroduction to jenkins
Introduction to jenkinsAbe Diaz
 
Ship It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails EcosystemShip It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails EcosystemYi-Ting Cheng
 
What is this agile thing anyway
What is this agile thing anywayWhat is this agile thing anyway
What is this agile thing anywayLisa Van Gelder
 
deliver:agile - Enable your Agile Team with Continuous Delivery Pipelines
deliver:agile - Enable your Agile Team with Continuous Delivery Pipelinesdeliver:agile - Enable your Agile Team with Continuous Delivery Pipelines
deliver:agile - Enable your Agile Team with Continuous Delivery PipelinesEsteban Garcia
 
JavaOne 2016 - CON3080 - Testing Java Web Applications with Selenium: A Cookbook
JavaOne 2016 - CON3080 - Testing Java Web Applications with Selenium: A CookbookJavaOne 2016 - CON3080 - Testing Java Web Applications with Selenium: A Cookbook
JavaOne 2016 - CON3080 - Testing Java Web Applications with Selenium: A CookbookJorge Hidalgo
 
Build automation best practices
Build automation best practicesBuild automation best practices
Build automation best practicesCode Mastery
 
Getting Started with Serverless Architectures
Getting Started with Serverless ArchitecturesGetting Started with Serverless Architectures
Getting Started with Serverless ArchitecturesAmazon Web Services
 
DCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityDCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityGeoff Harcourt
 
MyHeritage - QA Automations in a Continuous Deployment environment
MyHeritage -  QA Automations in a Continuous Deployment environmentMyHeritage -  QA Automations in a Continuous Deployment environment
MyHeritage - QA Automations in a Continuous Deployment environmentMatanGoren
 
Supersize Your Production Pipe
Supersize Your Production PipeSupersize Your Production Pipe
Supersize Your Production Pipeslantsixgames
 
Designing a Highly Available Environment Using Methods of Modern IT Infrastru...
Designing a Highly Available Environment Using Methods of Modern IT Infrastru...Designing a Highly Available Environment Using Methods of Modern IT Infrastru...
Designing a Highly Available Environment Using Methods of Modern IT Infrastru...Perforce
 
Heavenly hell – automated tests at scale wojciech seliga
Heavenly hell – automated tests at scale   wojciech seligaHeavenly hell – automated tests at scale   wojciech seliga
Heavenly hell – automated tests at scale wojciech seligaAtlassian
 
"Using Automation Tools To Deploy And Operate Applications In Real World Scen...
"Using Automation Tools To Deploy And Operate Applications In Real World Scen..."Using Automation Tools To Deploy And Operate Applications In Real World Scen...
"Using Automation Tools To Deploy And Operate Applications In Real World Scen...ConSol Consulting & Solutions Software GmbH
 

Similar to How Guerrilla Games Achieves Speed and Control in their Development Cycle (20)

Testing sync engine
Testing sync engineTesting sync engine
Testing sync engine
 
Maximize Your Production Effort (English)
Maximize Your Production Effort (English)Maximize Your Production Effort (English)
Maximize Your Production Effort (English)
 
Ci of js and apex using jasmine, phantom js and drone io df14
Ci of js and apex using jasmine, phantom js and drone io   df14Ci of js and apex using jasmine, phantom js and drone io   df14
Ci of js and apex using jasmine, phantom js and drone io df14
 
Inside the IT Territory game server / Mark Lokshin (IT Territory)
Inside the IT Territory game server / Mark Lokshin (IT Territory)Inside the IT Territory game server / Mark Lokshin (IT Territory)
Inside the IT Territory game server / Mark Lokshin (IT Territory)
 
Escaping Test Hell - ACCU 2014
Escaping Test Hell - ACCU 2014Escaping Test Hell - ACCU 2014
Escaping Test Hell - ACCU 2014
 
BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!
BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!
BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!
 
Supersize your production pipe enjmin 2013 v1.1 hd
Supersize your production pipe    enjmin 2013 v1.1 hdSupersize your production pipe    enjmin 2013 v1.1 hd
Supersize your production pipe enjmin 2013 v1.1 hd
 
Introduction to jenkins
Introduction to jenkinsIntroduction to jenkins
Introduction to jenkins
 
Ship It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails EcosystemShip It ! with Ruby/ Rails Ecosystem
Ship It ! with Ruby/ Rails Ecosystem
 
What is this agile thing anyway
What is this agile thing anywayWhat is this agile thing anyway
What is this agile thing anyway
 
deliver:agile - Enable your Agile Team with Continuous Delivery Pipelines
deliver:agile - Enable your Agile Team with Continuous Delivery Pipelinesdeliver:agile - Enable your Agile Team with Continuous Delivery Pipelines
deliver:agile - Enable your Agile Team with Continuous Delivery Pipelines
 
JavaOne 2016 - CON3080 - Testing Java Web Applications with Selenium: A Cookbook
JavaOne 2016 - CON3080 - Testing Java Web Applications with Selenium: A CookbookJavaOne 2016 - CON3080 - Testing Java Web Applications with Selenium: A Cookbook
JavaOne 2016 - CON3080 - Testing Java Web Applications with Selenium: A Cookbook
 
Build automation best practices
Build automation best practicesBuild automation best practices
Build automation best practices
 
Getting Started with Serverless Architectures
Getting Started with Serverless ArchitecturesGetting Started with Serverless Architectures
Getting Started with Serverless Architectures
 
DCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityDCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production Parity
 
MyHeritage - QA Automations in a Continuous Deployment environment
MyHeritage -  QA Automations in a Continuous Deployment environmentMyHeritage -  QA Automations in a Continuous Deployment environment
MyHeritage - QA Automations in a Continuous Deployment environment
 
Supersize Your Production Pipe
Supersize Your Production PipeSupersize Your Production Pipe
Supersize Your Production Pipe
 
Designing a Highly Available Environment Using Methods of Modern IT Infrastru...
Designing a Highly Available Environment Using Methods of Modern IT Infrastru...Designing a Highly Available Environment Using Methods of Modern IT Infrastru...
Designing a Highly Available Environment Using Methods of Modern IT Infrastru...
 
Heavenly hell – automated tests at scale wojciech seliga
Heavenly hell – automated tests at scale   wojciech seligaHeavenly hell – automated tests at scale   wojciech seliga
Heavenly hell – automated tests at scale wojciech seliga
 
"Using Automation Tools To Deploy And Operate Applications In Real World Scen...
"Using Automation Tools To Deploy And Operate Applications In Real World Scen..."Using Automation Tools To Deploy And Operate Applications In Real World Scen...
"Using Automation Tools To Deploy And Operate Applications In Real World Scen...
 

More from Perforce

How to Organize Game Developers With Different Planning Needs
How to Organize Game Developers With Different Planning NeedsHow to Organize Game Developers With Different Planning Needs
How to Organize Game Developers With Different Planning NeedsPerforce
 
Regulatory Traceability: How to Maintain Compliance, Quality, and Cost Effic...
Regulatory Traceability:  How to Maintain Compliance, Quality, and Cost Effic...Regulatory Traceability:  How to Maintain Compliance, Quality, and Cost Effic...
Regulatory Traceability: How to Maintain Compliance, Quality, and Cost Effic...Perforce
 
Efficient Security Development and Testing Using Dynamic and Static Code Anal...
Efficient Security Development and Testing Using Dynamic and Static Code Anal...Efficient Security Development and Testing Using Dynamic and Static Code Anal...
Efficient Security Development and Testing Using Dynamic and Static Code Anal...Perforce
 
Understanding Compliant Workflow Enforcement SOPs
Understanding Compliant Workflow Enforcement SOPsUnderstanding Compliant Workflow Enforcement SOPs
Understanding Compliant Workflow Enforcement SOPsPerforce
 
Branching Out: How To Automate Your Development Process
Branching Out: How To Automate Your Development ProcessBranching Out: How To Automate Your Development Process
Branching Out: How To Automate Your Development ProcessPerforce
 
How to Do Code Reviews at Massive Scale For DevOps
How to Do Code Reviews at Massive Scale For DevOpsHow to Do Code Reviews at Massive Scale For DevOps
How to Do Code Reviews at Massive Scale For DevOpsPerforce
 
How to Spark Joy In Your Product Backlog
How to Spark Joy In Your Product Backlog How to Spark Joy In Your Product Backlog
How to Spark Joy In Your Product Backlog Perforce
 
Going Remote: Build Up Your Game Dev Team
Going Remote: Build Up Your Game Dev Team Going Remote: Build Up Your Game Dev Team
Going Remote: Build Up Your Game Dev Team Perforce
 
Shift to Remote: How to Manage Your New Workflow
Shift to Remote: How to Manage Your New WorkflowShift to Remote: How to Manage Your New Workflow
Shift to Remote: How to Manage Your New WorkflowPerforce
 
Hybrid Development Methodology in a Regulated World
Hybrid Development Methodology in a Regulated WorldHybrid Development Methodology in a Regulated World
Hybrid Development Methodology in a Regulated WorldPerforce
 
Better, Faster, Easier: How to Make Git Really Work in the Enterprise
Better, Faster, Easier: How to Make Git Really Work in the EnterpriseBetter, Faster, Easier: How to Make Git Really Work in the Enterprise
Better, Faster, Easier: How to Make Git Really Work in the EnterprisePerforce
 
Easier Requirements Management Using Diagrams In Helix ALM
Easier Requirements Management Using Diagrams In Helix ALMEasier Requirements Management Using Diagrams In Helix ALM
Easier Requirements Management Using Diagrams In Helix ALMPerforce
 
How To Master Your Mega Backlog
How To Master Your Mega Backlog How To Master Your Mega Backlog
How To Master Your Mega Backlog Perforce
 
Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...
Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...
Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...Perforce
 
How to Scale With Helix Core and Microsoft Azure
How to Scale With Helix Core and Microsoft Azure How to Scale With Helix Core and Microsoft Azure
How to Scale With Helix Core and Microsoft Azure Perforce
 
Achieving Software Safety, Security, and Reliability Part 2
Achieving Software Safety, Security, and Reliability Part 2Achieving Software Safety, Security, and Reliability Part 2
Achieving Software Safety, Security, and Reliability Part 2Perforce
 
Should You Break Up With Your Monolith?
Should You Break Up With Your Monolith?Should You Break Up With Your Monolith?
Should You Break Up With Your Monolith?Perforce
 
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...Perforce
 
What's New in Helix ALM 2019.4
What's New in Helix ALM 2019.4What's New in Helix ALM 2019.4
What's New in Helix ALM 2019.4Perforce
 
Free Yourself From the MS Office Prison
Free Yourself From the MS Office Prison Free Yourself From the MS Office Prison
Free Yourself From the MS Office Prison Perforce
 

More from Perforce (20)

How to Organize Game Developers With Different Planning Needs
How to Organize Game Developers With Different Planning NeedsHow to Organize Game Developers With Different Planning Needs
How to Organize Game Developers With Different Planning Needs
 
Regulatory Traceability: How to Maintain Compliance, Quality, and Cost Effic...
Regulatory Traceability:  How to Maintain Compliance, Quality, and Cost Effic...Regulatory Traceability:  How to Maintain Compliance, Quality, and Cost Effic...
Regulatory Traceability: How to Maintain Compliance, Quality, and Cost Effic...
 
Efficient Security Development and Testing Using Dynamic and Static Code Anal...
Efficient Security Development and Testing Using Dynamic and Static Code Anal...Efficient Security Development and Testing Using Dynamic and Static Code Anal...
Efficient Security Development and Testing Using Dynamic and Static Code Anal...
 
Understanding Compliant Workflow Enforcement SOPs
Understanding Compliant Workflow Enforcement SOPsUnderstanding Compliant Workflow Enforcement SOPs
Understanding Compliant Workflow Enforcement SOPs
 
Branching Out: How To Automate Your Development Process
Branching Out: How To Automate Your Development ProcessBranching Out: How To Automate Your Development Process
Branching Out: How To Automate Your Development Process
 
How to Do Code Reviews at Massive Scale For DevOps
How to Do Code Reviews at Massive Scale For DevOpsHow to Do Code Reviews at Massive Scale For DevOps
How to Do Code Reviews at Massive Scale For DevOps
 
How to Spark Joy In Your Product Backlog
How to Spark Joy In Your Product Backlog How to Spark Joy In Your Product Backlog
How to Spark Joy In Your Product Backlog
 
Going Remote: Build Up Your Game Dev Team
Going Remote: Build Up Your Game Dev Team Going Remote: Build Up Your Game Dev Team
Going Remote: Build Up Your Game Dev Team
 
Shift to Remote: How to Manage Your New Workflow
Shift to Remote: How to Manage Your New WorkflowShift to Remote: How to Manage Your New Workflow
Shift to Remote: How to Manage Your New Workflow
 
Hybrid Development Methodology in a Regulated World
Hybrid Development Methodology in a Regulated WorldHybrid Development Methodology in a Regulated World
Hybrid Development Methodology in a Regulated World
 
Better, Faster, Easier: How to Make Git Really Work in the Enterprise
Better, Faster, Easier: How to Make Git Really Work in the EnterpriseBetter, Faster, Easier: How to Make Git Really Work in the Enterprise
Better, Faster, Easier: How to Make Git Really Work in the Enterprise
 
Easier Requirements Management Using Diagrams In Helix ALM
Easier Requirements Management Using Diagrams In Helix ALMEasier Requirements Management Using Diagrams In Helix ALM
Easier Requirements Management Using Diagrams In Helix ALM
 
How To Master Your Mega Backlog
How To Master Your Mega Backlog How To Master Your Mega Backlog
How To Master Your Mega Backlog
 
Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...
Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...
Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...
 
How to Scale With Helix Core and Microsoft Azure
How to Scale With Helix Core and Microsoft Azure How to Scale With Helix Core and Microsoft Azure
How to Scale With Helix Core and Microsoft Azure
 
Achieving Software Safety, Security, and Reliability Part 2
Achieving Software Safety, Security, and Reliability Part 2Achieving Software Safety, Security, and Reliability Part 2
Achieving Software Safety, Security, and Reliability Part 2
 
Should You Break Up With Your Monolith?
Should You Break Up With Your Monolith?Should You Break Up With Your Monolith?
Should You Break Up With Your Monolith?
 
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...
 
What's New in Helix ALM 2019.4
What's New in Helix ALM 2019.4What's New in Helix ALM 2019.4
What's New in Helix ALM 2019.4
 
Free Yourself From the MS Office Prison
Free Yourself From the MS Office Prison Free Yourself From the MS Office Prison
Free Yourself From the MS Office Prison
 

Recently uploaded

Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 

Recently uploaded (20)

Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 

How Guerrilla Games Achieves Speed and Control in their Development Cycle

  • 1. # Frank Compagner Guerrilla Games
  • 2. # How we learned to stop worrying and love the development cycle • Background • Iteration • Teamwork • Stability • Releasing Iterate Launch
  • 3. # • Killzone Shadow Fall – PS4 launch title • 200 local users, 150 remote • Head revision: 1.8 TB (1.6 M files) • Depot size: 8 TB • Peak change: 180 GB / week • 60 dedicated build servers • All in Perforce since 2004
  • 4. # Server Proxy Proxy Proxy Proxy Build Farm (60) Users (200) Proxy Cambridge Proxy Guilford Proxy San Diego Proxy Amsterdam Shanghai
  • 5. # Commit Server Edge Server Build Farm (60) Users (200) Cambridge Proxy Guilford Proxy San Diego Proxy Amsterdam Shanghai Edge Server Edge Server
  • 6. # Submits GB / week
  • 7. # • Monitors all vital stats of server and proxies • Simple Python script, produces simple html • Works for Windows and Linux servers • Proxies cleaned by another Python script • Both available in the Perforce Workshop
  • 8. #
  • 9. #
  • 10. # Code Compile Test Programmer Link
  • 11. # Code Compile IDE and Tools Test Programmer Link Distribute Incremental linker Code Hotloading
  • 12. # Model Artist Test Process Hotloading
  • 13. # Model Tools Artist Test Process Hotloading Distribute
  • 14. # Programmer Rigger Tester Team Designer Animator
  • 15. # Programmer Rigger Animator Animator Tester Designer Modeler Programmer Programmer VFX artist Designer Audio Engineer Modeler Designer Animator
  • 16. #
  • 17. # • Does not look like this
  • 18. # • It’s more like this
  • 19. # • OK, like this
  • 20. # • Personal branch is fine (sandbox / streams / Git) • Feature branches sometimes work • Give control over your environment • But create distance to others • Team work requires frequent branch switching
  • 21. # • For us, branch switching is always expensive: – Amount of data and change – Unmergeable files – Code-Data dependencies • Branches add complexity, we already have that • Team effort, we don’t want distance • Almost everybody works on the trunk • So why isn’t it always broken?
  • 22. #
  • 23. # Programmer Rigger Tester Team Designer Animator
  • 24. # Programmer Submit Sync Rigger Tester Team Submit Submit Designer Animator Submit Submit Sync Sync Sync Sync
  • 25. # Programmer Test Submit Sync Rigger Tester Team Submit Submit Designer Animator Submit Submit Sync Sync Sync Sync Test Test Test Test
  • 26. # • Test pre submit (user) • Test post submit (builder) • Test in parallel • Test as fast as possible • Still accidents do happen User B Sync Sync Build Game Submit Sync Test Game Sync Build Tools Submit Sync Cook Data < 30 minutes User A Submit Sync Test Game … Sync Upload Cook Data Upload Test
  • 27. # Build machines write Label 42: change 1200 Label 41: change 1180 Label 40: change 1155 Label 39: change 1150 … Labels.xml read Safe-Sync Tool • Each game build creates a label • Stored in Labels.xml file in depot • Builds add their result to the label • Builds can add artifacts to a label • Nobody gets latest (not safe) • Everybody uses safe-sync tool
  • 28. Select tests Sync to get label # Labels that pass tests Labels that fail tests • Custom safe-sync tool shows labels and test results • Use it to sync to a known good label – This includes artifacts from other builds
  • 29. #
  • 30. # • We branch for every release (often every week) • Nobody works on the release branch • Everybody works on the trunk • Changes are merged up from main to release release main = release = change
  • 31. • Cherry picking rules! • Gives you a lot of control • But also lots of confusion • And mistakes are easy • Good tools can fix this • Give flexibility • And control # (See whitepaper for details)
  • 32. # • Iterate as fast as you can • Don’t branch too much • Test everything • Sync should be safe • Release all the time • Control what you release Speed Control
  • 33. # Frank Compagner frank@guerrilla-games.com
  • 34. # RESOURCES Perforce Workshop (P4 monitor, cache cleaner): https://swarm.workshop.perforce.com/files/guest/frank_compagner ReleaseManager: http://www.perforce.com/resources/presentations/user-conference-talks/ tools-for-smooth-release-cycle http://www.perforce.com/sites/default/files/guerrilla-games-whitepaper.pdf