Net-A-Porter talk about how they've built their Continuous Delivery pipeline to deliver their online service on Software Defined Infrastructure. Video: https://www.youtube.com/watch?v=ew6PUlVS5yM Net-A-Porter talk about how they've built their Continuous Delivery pipeline to deliver their online service on Software Defined Infrastructure. Slides: TODO Join DevOps Exchange London here: http://www.meetup.com/DevOps-Exchange... Follow DOXLON on twitter http://www.twitter.com/doxlon

1. Thomas Eskenazi
Thomas Eskenazi
@tomeskenazi
@tomeskenazi
DOXLON – 2015
DOXLON – 2015
Yann Vigara
Yann Vigara
@yannvigara
@yannvigara
We did it our way!
We did it our way!

2. The Mission
The Mission
4 Monolithic apps
4 Monolithic apps
2 Jenkins servers / 7000+
2 Jenkins servers / 7000+
jobs
jobs
1 puppet master with all
1 puppet master with all
modules in one git repo and
modules in one git repo and
one environment
one environment
Cobbler and multiple puppet
Cobbler and multiple puppet
classes
classes
4 weeks release cycle times
4 weeks release cycle times
30 Service Oriented Apps
30 Service Oriented Apps
(more than 100 in DEV)
(more than 100 in DEV)
Scalable synced Jenkins servers
Scalable synced Jenkins servers
~800 jobs + 3500 still in legacy
~800 jobs + 3500 still in legacy
Scalable puppet stack,
Scalable puppet stack,
4 environments,
4 environments,
dedicated module repositories
dedicated module repositories
Foreman and one role per server
Foreman and one role per server
On-demand deployment to Prod
On-demand deployment to Prod

3. Our new recruit
Our new recruit

4. CD Pipelines - Overview
CD Pipelines - Overview
Jenkins-based, multiple
Jenkins-based, multiple
servers in sync
servers in sync
●
Manual or SCM-triggered
Manual or SCM-triggered
●
Manual or auto-
Manual or auto-
promotions (multiple
promotions (multiple
gating criteria, user
gating criteria, user
authentication)
authentication)
●
Integrated to Jira, Sonar,
Integrated to Jira, Sonar,
Artifactory, Confluence,
Artifactory, Confluence,
PuppetDB, Pulp...
PuppetDB, Pulp...
●
Multiple KPI, Metrics and
Multiple KPI, Metrics and
Dashboards extracted
Dashboards extracted
from Jenkins
from Jenkins
●
Build and deployment
Build and deployment
configuration stored in
configuration stored in
project codebase
project codebase

5. CD Pipelines - Dashboard
CD Pipelines - Dashboard
Real-time Data
Real-time Data
Extracted from Jenkins
Extracted from Jenkins
REST API (JSON)
REST API (JSON)
PHP page, query
PHP page, query
parameters defines
parameters defines
Jenkins job target, what
Jenkins job target, what
to display
to display
Source Code available
Source Code available
to dev/testers for
to dev/testers for
contribution
contribution

6. CD Pipelines - KPI & Metrics
CD Pipelines - KPI & Metrics

7. CD Pipelines - Generation
CD Pipelines - Generation
Jenkins Admin Job to generate all pipelines, manual & nightly-
Jenkins Admin Job to generate all pipelines, manual & nightly-
scheduled
scheduled
●
Single Input Parameter: SCM/Git URL
Single Input Parameter: SCM/Git URL
●
Digests the build.yaml and deploy.yaml + FM templates
Digests the build.yaml and deploy.yaml + FM templates
●
Automatically detects types of builds (mvn, gradle, scala, custom)
Automatically detects types of builds (mvn, gradle, scala, custom)
●
2 jobs: MASTER & DEV
2 jobs: MASTER & DEV

8. CD Pipelines - Definition
CD Pipelines - Definition
Build Config DSL: Build.yaml
Build Config DSL: Build.yaml
●
Build Environment (node,
Build Environment (node,
JDK…)
JDK…)
●
Artefact Types (package,
Artefact Types (package,
dependency, systest…)
dependency, systest…)
●
Notifications (email, test
Notifications (email, test
reports…)
reports…)

9. CD Pipelines - Definition
CD Pipelines - Definition
Deployment Config DSL:
Deployment Config DSL:
Deploy.yaml
Deploy.yaml
●
Artifact Definitions: names, types
Artifact Definitions: names, types
(puppet/aws...), puppet modules,
(puppet/aws...), puppet modules,
hieradata roles
hieradata roles
●
Environment (DEV, STG, PROD...)
Environment (DEV, STG, PROD...)
●
Integ./System Tests Definition
Integ./System Tests Definition
(targets, nodes, url…)
(targets, nodes, url…)
●
Deployment Orchestration (promo
Deployment Orchestration (promo
trigerring/transitioning)
trigerring/transitioning)

10. CD Pipelines - Job Example
CD Pipelines - Job Example

11. Puppet Module Structure
Puppet Module Structure
Role based configuration
Role based configuration
Base modules
Base modules
Common modules
Common modules
Naming convention
Naming convention

12. Environment and promotion
Environment and promotion
4 puppet environments : dev, qa, stg, prd
4 puppet environments : dev, qa, stg, prd
Referencing the git tag per environment in a modules.conf file
Referencing the git tag per environment in a modules.conf file
Set of bash scripts to manage the promotion
Set of bash scripts to manage the promotion
Puppet masters syncing on modules.conf changes
Puppet masters syncing on modules.conf changes

13. Foreman
Foreman
Server provisioning
Server provisioning
Audit reports
Audit reports
1 puppet class from
1 puppet class from
nap_role_*
nap_role_*
Facts used by hieradata
Facts used by hieradata
●
system_env
system_env
●
system_role
system_role
●
system_group
system_group
hieradata/
hieradata/
●
global.yaml
global.yaml
●
roles/
roles/
●
nap_hiera_<$system_role>/
nap_hiera_<$system_role>/
●
default.yaml
default.yaml
●
<$system_env>.yaml
<$system_env>.yaml
●
<$system_env>.<$system_group>.yaml
<$system_env>.<$system_group>.yaml
●
domain/
domain/
●
dom1.yaml
dom1.yaml
●
dom2.yaml
dom2.yaml
●
hosts/
hosts/
●
dom1/host1.yaml
dom1/host1.yaml

14. PuppetDB
PuppetDB
No exported resources
No exported resources
Query PuppetDB and use ERB templates
Query PuppetDB and use ERB templates
Host discovery in Jenkins during promotion
Host discovery in Jenkins during promotion

15. Pulp
Pulp
Central repository
Central repository
1 Repository per role/service per environment
1 Repository per role/service per environment

16. Ongoing challenges
Ongoing challenges
Puppet lifecycle testing
Puppet lifecycle testing
Training
Training
Heritage migration
Heritage migration
Hybrid Cloud
Hybrid Cloud
(Openstack/AWS)
(Openstack/AWS)
Docker
Docker
Security/Network/SDN
Security/Network/SDN

17. ?
?

18. We are hiring
We are hiring
yann.vigara@net-a-porter.com
yann.vigara@net-a-porter.com
thomas.eskenazi@net-a-porter.com
thomas.eskenazi@net-a-porter.com