SlideShare a Scribd company logo
1 of 64
DEVOPS TOOLS FOR
WINNING AGILITY
KELLYN POT’VIN-GORMAN
TECHNICAL INTELLIGENCE MANAGER,
DELPHIX
Kellyn Pot’Vin-Gorman
Technical Intelligence Manager, Delphix
• Multi-platform DBA, (Oracle, MSSQL, MySQL, Sybase,
PostgreSQL, Informix…)
• Oracle ACE Director, (Alumni)
• Oak Table Network Member
• Idera ACE 2018
• APEX Women in Technology Award, CTA
• STEM education with Raspberry Pi and Python, including
DevOxx4Kids, Oracle Education Foundation and Girl Geek
Dinners
• President, Rocky Mtn Oracle User Group
• President, Denver SQL Server User Group
• Technical author, instructor and presenter.
• Author, blogger, (http://dbakevlar.com)
THE DEVOPS JOURNEY
Did you just automate testing?
It can be a longer trip than planned.
THE LIFE OF A DEVOPS ENGINEER
 Automation everything possible through scripting
 Orchestration between environments, including data sources
 Security, (PII, HIPAA, GDPR, SOX)
 Optimize projects, improving on each pass
 Collaborate and Communicate
 CULTURE, CULTURE, CULTURE!
COST OF CUSTOM SCRIPTING
In a 1000-person IT organization Avg. time each developer
spends scripting and maintaining deployment and release
management related tasks 8%
• Number of developers per team 8
• Number of development teams 100
• Average fully loaded cost per FTE/year $125,000
Annual cost of scripting $8,000,000
*Black hole of DevOps
6
WHAT IS THE MOST
MAJOR RISK TO
DEVOPS
• Silo’d teams continuing as they
were.
• No unified vision.
• Introduction of the cloud, especially
and SaaS, (Software as a Service)
hasn’t been easy.
• Lack of knowledge in the industry
due to grassroots and infancy of the
industry.
• Lacking tools that remove human
intervention and manual tasks.
• FORGETING THE DATA.
7
A TERMINOLOGY RECAP-
• Methodology: Scrum, Lean, Crystal, Kanban…
• Build Automation, Pipelines, Toolchains,
Automation
• Continuous Delivery/Continuous Integration
• Functional Testing/Unit Testing/Test Driven
Development
• Monitoring/Alerting
• Packages/Containers/Virtualization
• Release Coordination/Orchestration
https://xebialabs.com/glossary/
8
BUILD AUTOMATION
Requires scripting and automation experience for
development, testing, releasing, monitoring and alerting.
9
CONFIGURATION MANAGEMENT
Concept &
Development
Testing &
Verification
Operations &
Support
Deployment to
Production
10
CONTINUOUS DELIVERY/CONTINUOUS
INTEGRATION
The risk is low if “fail first, fail fast” occurs in development and
test only. The idea is to continuously deliver code in small
batches. Small releases make it easier to develop, test and
identify issues, releasing later successful software.
https://continuousdelivery.com/
11
RAPID DEPLOY
Software Content
Enablement Service Need or
Focus
Area
All Tiers of
Specific
Environment
12
RELEASE COORDINATION/ORCHESTRATION
Release coordination and orchestration is like a symphony. If the
coordination of the instruments are done well, the music will be
released in a melodious and successful manner. If it’s not, then it
can be disastrous.
13
THE GOALS OF ALL OF THESE ARE THE SAME-
• Shorter Delivery Cycles
• Higher Success Rate on Releases
• Shorten Time to Address Issues
• Continued Improvement in Communication
• Increased Revenue
• The one thing they don’t take into consideration, is
the data!
14
LET’S TALK DATA GRAVITY AND IT’S IMPACT
• Data is HEAVY.
• Relational databases, (or any data sources) are heavily
weighted and can be impactful to DevOps implementations.
15
EMBRACE VIRTUALIZATION…
A technical approach in which users and applications do not use
physical machines, but simulated systems running on actual, “real”
hardware. Virtualization can be used to eliminate resource usage
and enable savings for databases, network, file and application
management, along with server infrastructure.
Virtualize and Deployed▶
▶
MANY TYPES OF VIRTUALIZATION
Storage Pool for Delphix
QA
DEV PATCH TEST
PRODUCTION
Database/App Tier
1 TB
1 TB
600GB
Read From
Production
Each Virtual Database takes up around 5-10Gb upon creation, (dependent upon
parameters)
TEST
Read AND Write
DATA VIRTUALIZATION: FROM PROD TO
VIRTUAL
Source Environment
SCSI/SSL
DATA VIRTUALIZATION: PROD TO VIRTUAL
Source
Environment
SCSI/SSL
Any
Storage
Validated
Sync
Environment
DATA VIRTUALIZATION: SPACE
SAVINGS
Delphix Virtualization EngineValidated Sync Target
Dozen Virtual Database Copies,
(VDBs)
DATA VIRTUALIZATION: HOW IS IT POSSIBLE?
Delphix Virtualization Engine
Any
Storage
21
WHY DATABASE VIRTUALIZATION IS DIFFERENT
• DBA is “only as good as their last backup”.
• Many database tools take considerable time to recover or
refresh.
• DevOps is often about automation- automate an “undo” for
development and testing that includes data rewind.
• Include a self-service tool that will allow for rewind without
DBA intervention.
• Allows for data version control and DataOps, the next step in
DevOps
22
TIME TO TEST IN DEVOPS
Task Time Needed
Requirement Gathering 4-8hrs
Test Case Creation 4-8hrs
Test Data Generation Several Hours/Days
Test Script Generation 3hrs-16hrs
Test Execution- Manaul 1 day-2 Days
Test Results 1-2hrs
https://autonomiq.io/
23
HOW TO ENHANCE DEVOPS DATA
Timeline in a Project
Waterfall- 5 Releases
January February March April
Agile with Virtualization- 12 Releases
January February March April
Database Refresh
Development Production Release
Testing Cycle
The Goal
of Agile
Data Holdup to
Development Cycle
25
FURTHER- DATA PODS/CONTAINERS
Containers offer the ability to isolate application code and/or the
whole infrastructure stack into a package able entity to ease
deployment, even from the same kernel. This is a powerful tool for
DevOps to ease deployment for complex tiers, applications and
multiple data stores. Data Pods is the next step, where a pod is
created from virtualized environments.
26
DATA PODS
Database
Unstructured
Data
Application
DATA PODS FOR EASE OF DELIVERY
Rsync (UNIX/Linux)
Robocopy (Windows)
QA
NFS
iSCSI
Development
NFS
iSCSI
Testing
NFS
iSCSI
Delphix Virtualization Engine
Same Storage Size as Production
Deployed using Jenkins
Create “Container”
Branching and Bookmarking
• The ability to mark each iteration of development
with a bookmark
• Simplify to lock and deliver while testing a
consistent image via a virtual database, (VDB)
• If a something goes wrong, the ability to
“bookmark”, (and subsequent snapshot) to deliver
to development to address.
Administration Ease
Self-Service
Release
1.2
Release
1.3
Branch
1.3
Branch
1.1
DevOps and Source Control
32
MATURITY
36
DEVOPS TOOLS
37
38
THE PERIODIC TABLE OF DEVOPS TOOLS
https://xebialabs.com/periodic-table-of-devops-tools/
39
ANSIBLE
• Learn about Playbooks
• Powerful collection of steps, (aka plays)
• Plays consist of a list of tasks
• Handlers allow for remote operations
• YAML Syntax
• Security configurations based on hosts and users
• Ability to run as secondary users, (privilege escalation)
• Huge support community and best practices are mature
40
EXAMPLE OF AN ANSIBLE CALL
- name: Transfer and execute a script.
hosts: server
remote_user: test_user
sudo: yes
tasks:
- name: Transfer the script
copy: src=test.sh dest=/home/test_user mode=0777
- name: Execute the script
command: sh /home/test_user/test.sh
41
JENKINS
Jenkins is an open source automation server-
• Easiest executed via a jetty servlet
• Has an easy to access URL, (commonly host:8080)
• Automation is done via individual projects.
• Has numerous plugins to support environments
• Supports multiple scripting languages, including shell, batch
commands and Ant scripts.
• Supports robust revision control and compatible with version
control systems like CVS and Subversion.
• Supports notifications like qmail and sendmail.
42
JENKINS GROOVY SCRIPT EXAMPLE
// prepare variables for subscript execution:
String subscript = "cdmserver-monitor-instance-
status.groovy";
println "executing subscript ${subscript}"
// variables to be passed to the subscript
status = (action == "start" ? "started" : "stopped")
def s =
ScriptlerConfiguration.getConfiguration().getScriptById(su
bscript)
File scriptSrc = new
File(ScriptlerManagment.getScriptDirectory(),
s.getScriptPath());
43
JENKINS PLUGIN FOR DELPHIX
https://github.com/delphix/delphix-jenkins-plugin
44
PUPPET
• Review, automate, deploy and invest in all your software.
• Control and enforce consistency across environments and
platforms.
• Share, test and enforce changes on-premise and in the cloud.
Similar to Puppet
• Chef
• Jenkins
45
KUBERNETES
• Open-source system for automating
deployments, scaling and containerizing.
• Resource level isolation at the kernel level,
making it lighter on systems.
• It works with docker and other Containers,
DevOps tools and applications.
• Used by Google and IBM cloud containers
• Oracle Cloud Infrastructure, (OCI) uses it
for Oracle Linux bare metal
• Red Hat for OpenShift
• 1 or more containers = pod
46
QUALI
https://community.quali.com/spaces/12/index.html
47
JIRA FOR TESTING
48
JIRA FOR TESTING
49
GITHUB
Git is an open-source version control system that was started by Linus
Trovalds
GitHub Alternatives
• Subversion
• CVS
• Mercurial
• Git
50
THOSE WITH TESTING IN MIND
51
ZEPHYR
52
AUTONOMIQ
53
ZAPTEST
54
XEBIALABS DEVOPS PLATFORM
FULL ARA PLATFORM
• Not just about testing automation
• But doesn’t forget testing OR the data
• Has plugins for the entire enterprise lifecycle of DevOps
• Reports on the release pipeline
• Even does risk assessment
56
SO, KNOW YOUR TOOLS
Incredible variation of tools and knowing tool
classifications doesn’t mean that the tools are alike. Find
out what each group is using from the tools and embrace
coverage that incorporates success.
57
PROACTIVELY ASSESS IT ALL
• Build out a hierarchy of all environments and teams
involved.
• Group projects into bronze, silver, gold, platinum
• Begin to isolate requirements for automation,
collaboration and teams.
• Identify and document existing tools, platforms and
programming languages used.
• Create goals and timelines
• Build your DevOps Project team.
• Update applications, data loads and processes towards an
approach to the cloud and heterogenous toolsets.
• Eliminate additional data jumps, using staging and single
load processes. Attempt to use a golden copy whenever
applicable.
• Use the right management and monitoring tools for the
focus, heterogenous development tools for increased
success.
• Locate a methodology that makes the whole team more
successful.
Know Thy Enemy…
• All IaaS solutions provide encryption in-flight and encryption
at-rest
• But encryption doesn’t protect data as much as it needs to be .
• Europe already requires data masking, not just data encryption
for any confindential data, (GDPR):
• http://ec.europa.eu/justice/data-protection/article-29/documentation/opinion-
recommendation/files/2014/wp216_en.pdf
CONFIDENTIAL DATA
GENERAL DATA PROTECTION REGULATIONS
The GDPR (General Data Protection
Regulation) seeks to create a harmonised
data protection law framework across the EU
and aims to give citizens back the control of
their personal data, whilst imposing strict
rules on those hosting and ‘processing’ this
data, anywhere in the world.
WHO SHOULD HAVE ACCESS TO WHAT?
• Granule control in tools is essential
• Automation may remove need for access to
critical data
• Deter need to bring data out of secured
systems to determine requirements, etc.
Do they need access to production data at all?
• Encryption is reversible data obfuscation, which is very
different from masking data.
• Data masking is non-reversible.
• It solves the issue at the data level.
• Is authentication and authorization in non-production in
compliance with security goals?
• All organizations will soon need to review if critical data in non-production
environments be accessible to developers, testers and users.
CONFIDENTIAL DATA
• Masking personally-identifiable, (PII, HIPAA, PCI, etc.)
information renders it useless from a security
standpoint
• Resolves both the technical and personal responsibility
issue.
• The data can be masked before it moves to non-
production, removing unnecessary risk.
WHY MASKING IS PART OF THE ANSWER
CONFIDENTIAL DATA
Production
Non-
Production
Exposure
CONFIDENTIAL DATA
Production
Non-
Production
Exposure
Encryption
Masking
Solution
66
HOW TO SUCCEED WITH DEVOPS TOOLS
Culture is one of the biggest challenges in DevOps:
• Automating repeatable and complex processes, (scripting
and automation tools)
• Get data out of the way of DevOps, (Containers, Data Pods
and Virtualization)
• Locate heterogeneous tools/scripting languages that can
support multiple tiers of technology.
• Understanding that automation does not lessen the value of
the contributors, but increases it.
• Read- The Phoenix Project and the DevOps Handbook!
THANK YOU!
Email: dbakevlar@gmail.com
Twitter: @DBAKevlar
Website: http://dbakevlar.com

More Related Content

What's hot

How bigtop leveraged docker for build automation and one click hadoop provis...
How bigtop leveraged docker for build automation and  one click hadoop provis...How bigtop leveraged docker for build automation and  one click hadoop provis...
How bigtop leveraged docker for build automation and one click hadoop provis...Evans Ye
 
What Big Data Folks Need to Know About DevOps
What Big Data Folks Need to Know About DevOpsWhat Big Data Folks Need to Know About DevOps
What Big Data Folks Need to Know About DevOpsMatt Ray
 
Introduction into Docker Containers, the Oracle Platform and the Oracle (Nati...
Introduction into Docker Containers, the Oracle Platform and the Oracle (Nati...Introduction into Docker Containers, the Oracle Platform and the Oracle (Nati...
Introduction into Docker Containers, the Oracle Platform and the Oracle (Nati...Lucas Jellema
 
BYOP: Custom Processor Development with Apache NiFi
BYOP: Custom Processor Development with Apache NiFiBYOP: Custom Processor Development with Apache NiFi
BYOP: Custom Processor Development with Apache NiFiDataWorks Summit
 
OpenShift Enterprise 3.1 vs kubernetes
OpenShift Enterprise 3.1 vs kubernetesOpenShift Enterprise 3.1 vs kubernetes
OpenShift Enterprise 3.1 vs kubernetesSamuel Terburg
 
Managing Complexity at Velocity
Managing Complexity at VelocityManaging Complexity at Velocity
Managing Complexity at VelocityMatt Ray
 
DCEU 18: Continuous Delivery with Docker Containers and Java: The Good, the B...
DCEU 18: Continuous Delivery with Docker Containers and Java: The Good, the B...DCEU 18: Continuous Delivery with Docker Containers and Java: The Good, the B...
DCEU 18: Continuous Delivery with Docker Containers and Java: The Good, the B...Docker, Inc.
 
Transforming Application Delivery with PaaS and Linux Containers
Transforming Application Delivery with PaaS and Linux ContainersTransforming Application Delivery with PaaS and Linux Containers
Transforming Application Delivery with PaaS and Linux ContainersGiovanni Galloro
 
What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December...
What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December...What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December...
What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December...Lucas Jellema
 
NetflixOSS for Triangle Devops Oct 2013
NetflixOSS for Triangle Devops Oct 2013NetflixOSS for Triangle Devops Oct 2013
NetflixOSS for Triangle Devops Oct 2013aspyker
 
Jelastic - DevOps for Java with Docker Containers - Madrid 2015
Jelastic - DevOps for Java with Docker Containers - Madrid 2015Jelastic - DevOps for Java with Docker Containers - Madrid 2015
Jelastic - DevOps for Java with Docker Containers - Madrid 2015Jelastic Multi-Cloud PaaS
 
Running Oracle Database on Docker
Running Oracle Database on DockerRunning Oracle Database on Docker
Running Oracle Database on Dockergvenzl
 
Apache Spark: Lightning Fast Cluster Computing
Apache Spark: Lightning Fast Cluster ComputingApache Spark: Lightning Fast Cluster Computing
Apache Spark: Lightning Fast Cluster ComputingAll Things Open
 
Perforce Helix Never Dies: DevOps at Bandai Namco Studios
Perforce Helix Never Dies: DevOps at Bandai Namco StudiosPerforce Helix Never Dies: DevOps at Bandai Namco Studios
Perforce Helix Never Dies: DevOps at Bandai Namco StudiosPerforce
 
Structor - Automated Building of Virtual Hadoop Clusters
Structor - Automated Building of Virtual Hadoop ClustersStructor - Automated Building of Virtual Hadoop Clusters
Structor - Automated Building of Virtual Hadoop ClustersOwen O'Malley
 
"[WORKSHOP] K8S for developers", Denis Romanuk
"[WORKSHOP] K8S for developers", Denis Romanuk"[WORKSHOP] K8S for developers", Denis Romanuk
"[WORKSHOP] K8S for developers", Denis RomanukFwdays
 
Docker in Open Science Data Analysis Challenges by Bruce Hoff
Docker in Open Science Data Analysis Challenges by Bruce HoffDocker in Open Science Data Analysis Challenges by Bruce Hoff
Docker in Open Science Data Analysis Challenges by Bruce HoffDocker, Inc.
 
컨테이너 기술 소개 - Warden, Garden, Docker
컨테이너 기술 소개 - Warden, Garden, Docker컨테이너 기술 소개 - Warden, Garden, Docker
컨테이너 기술 소개 - Warden, Garden, Dockerseungdon Choi
 
Ports, pods and proxies
Ports, pods and proxiesPorts, pods and proxies
Ports, pods and proxiesLibbySchulze
 

What's hot (20)

How bigtop leveraged docker for build automation and one click hadoop provis...
How bigtop leveraged docker for build automation and  one click hadoop provis...How bigtop leveraged docker for build automation and  one click hadoop provis...
How bigtop leveraged docker for build automation and one click hadoop provis...
 
What Big Data Folks Need to Know About DevOps
What Big Data Folks Need to Know About DevOpsWhat Big Data Folks Need to Know About DevOps
What Big Data Folks Need to Know About DevOps
 
Introduction into Docker Containers, the Oracle Platform and the Oracle (Nati...
Introduction into Docker Containers, the Oracle Platform and the Oracle (Nati...Introduction into Docker Containers, the Oracle Platform and the Oracle (Nati...
Introduction into Docker Containers, the Oracle Platform and the Oracle (Nati...
 
BYOP: Custom Processor Development with Apache NiFi
BYOP: Custom Processor Development with Apache NiFiBYOP: Custom Processor Development with Apache NiFi
BYOP: Custom Processor Development with Apache NiFi
 
OpenShift Enterprise 3.1 vs kubernetes
OpenShift Enterprise 3.1 vs kubernetesOpenShift Enterprise 3.1 vs kubernetes
OpenShift Enterprise 3.1 vs kubernetes
 
Managing Complexity at Velocity
Managing Complexity at VelocityManaging Complexity at Velocity
Managing Complexity at Velocity
 
DCEU 18: Continuous Delivery with Docker Containers and Java: The Good, the B...
DCEU 18: Continuous Delivery with Docker Containers and Java: The Good, the B...DCEU 18: Continuous Delivery with Docker Containers and Java: The Good, the B...
DCEU 18: Continuous Delivery with Docker Containers and Java: The Good, the B...
 
Transforming Application Delivery with PaaS and Linux Containers
Transforming Application Delivery with PaaS and Linux ContainersTransforming Application Delivery with PaaS and Linux Containers
Transforming Application Delivery with PaaS and Linux Containers
 
What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December...
What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December...What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December...
What is Kafka & why is it Important? (UKOUG Tech17, Birmingham, UK - December...
 
NetflixOSS for Triangle Devops Oct 2013
NetflixOSS for Triangle Devops Oct 2013NetflixOSS for Triangle Devops Oct 2013
NetflixOSS for Triangle Devops Oct 2013
 
Jelastic - DevOps for Java with Docker Containers - Madrid 2015
Jelastic - DevOps for Java with Docker Containers - Madrid 2015Jelastic - DevOps for Java with Docker Containers - Madrid 2015
Jelastic - DevOps for Java with Docker Containers - Madrid 2015
 
Big data and Kubernetes
Big data and KubernetesBig data and Kubernetes
Big data and Kubernetes
 
Running Oracle Database on Docker
Running Oracle Database on DockerRunning Oracle Database on Docker
Running Oracle Database on Docker
 
Apache Spark: Lightning Fast Cluster Computing
Apache Spark: Lightning Fast Cluster ComputingApache Spark: Lightning Fast Cluster Computing
Apache Spark: Lightning Fast Cluster Computing
 
Perforce Helix Never Dies: DevOps at Bandai Namco Studios
Perforce Helix Never Dies: DevOps at Bandai Namco StudiosPerforce Helix Never Dies: DevOps at Bandai Namco Studios
Perforce Helix Never Dies: DevOps at Bandai Namco Studios
 
Structor - Automated Building of Virtual Hadoop Clusters
Structor - Automated Building of Virtual Hadoop ClustersStructor - Automated Building of Virtual Hadoop Clusters
Structor - Automated Building of Virtual Hadoop Clusters
 
"[WORKSHOP] K8S for developers", Denis Romanuk
"[WORKSHOP] K8S for developers", Denis Romanuk"[WORKSHOP] K8S for developers", Denis Romanuk
"[WORKSHOP] K8S for developers", Denis Romanuk
 
Docker in Open Science Data Analysis Challenges by Bruce Hoff
Docker in Open Science Data Analysis Challenges by Bruce HoffDocker in Open Science Data Analysis Challenges by Bruce Hoff
Docker in Open Science Data Analysis Challenges by Bruce Hoff
 
컨테이너 기술 소개 - Warden, Garden, Docker
컨테이너 기술 소개 - Warden, Garden, Docker컨테이너 기술 소개 - Warden, Garden, Docker
컨테이너 기술 소개 - Warden, Garden, Docker
 
Ports, pods and proxies
Ports, pods and proxiesPorts, pods and proxies
Ports, pods and proxies
 

Similar to DevOps tools for winning agility

Devoxx 2016 - Docker Nuts and Bolts
Devoxx 2016 - Docker Nuts and BoltsDevoxx 2016 - Docker Nuts and Bolts
Devoxx 2016 - Docker Nuts and BoltsPatrick Chanezon
 
Oscon London 2016 - Docker from Development to Production
Oscon London 2016 - Docker from Development to ProductionOscon London 2016 - Docker from Development to Production
Oscon London 2016 - Docker from Development to ProductionPatrick Chanezon
 
Intro Docker october 2013
Intro Docker october 2013Intro Docker october 2013
Intro Docker october 2013dotCloud
 
Continuous Integration - Mobile Practice
Continuous Integration - Mobile PracticeContinuous Integration - Mobile Practice
Continuous Integration - Mobile PracticeHARMAN Services
 
Unified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsUnified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsXebiaLabs
 
Intro to Docker October 2013
Intro to Docker October 2013Intro to Docker October 2013
Intro to Docker October 2013Docker, Inc.
 
DevOps and the DBA- 24 Hours of Pass
DevOps and the DBA-  24 Hours of PassDevOps and the DBA-  24 Hours of Pass
DevOps and the DBA- 24 Hours of PassKellyn Pot'Vin-Gorman
 
Cloud for agile_sw_projects-final
Cloud for agile_sw_projects-finalCloud for agile_sw_projects-final
Cloud for agile_sw_projects-finalAlain Delafosse
 
Fits docker into devops
Fits docker into devopsFits docker into devops
Fits docker into devopsEvans Ye
 
What's New in Docker - February 2017
What's New in Docker - February 2017What's New in Docker - February 2017
What's New in Docker - February 2017Patrick Chanezon
 
Cloudsolutionday 2016: DevOps workflow with Docker on AWS
Cloudsolutionday 2016: DevOps workflow with Docker on AWSCloudsolutionday 2016: DevOps workflow with Docker on AWS
Cloudsolutionday 2016: DevOps workflow with Docker on AWSAWS Vietnam Community
 
Unified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsUnified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsCompuware
 
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013dotCloud
 
Docker open stack boston
Docker open stack bostonDocker open stack boston
Docker open stack bostondotCloud
 
DCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityDCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityGeoff Harcourt
 

Similar to DevOps tools for winning agility (20)

Devoxx 2016 - Docker Nuts and Bolts
Devoxx 2016 - Docker Nuts and BoltsDevoxx 2016 - Docker Nuts and Bolts
Devoxx 2016 - Docker Nuts and Bolts
 
Oscon London 2016 - Docker from Development to Production
Oscon London 2016 - Docker from Development to ProductionOscon London 2016 - Docker from Development to Production
Oscon London 2016 - Docker from Development to Production
 
DevOps and DBA- Delphix
DevOps and DBA-  DelphixDevOps and DBA-  Delphix
DevOps and DBA- Delphix
 
Intro Docker october 2013
Intro Docker october 2013Intro Docker october 2013
Intro Docker october 2013
 
Webinar Docker Tri Series
Webinar Docker Tri SeriesWebinar Docker Tri Series
Webinar Docker Tri Series
 
Continuous Integration - Mobile Practice
Continuous Integration - Mobile PracticeContinuous Integration - Mobile Practice
Continuous Integration - Mobile Practice
 
Unified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsUnified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOps
 
Intro to Docker October 2013
Intro to Docker October 2013Intro to Docker October 2013
Intro to Docker October 2013
 
DevOps and the DBA- 24 Hours of Pass
DevOps and the DBA-  24 Hours of PassDevOps and the DBA-  24 Hours of Pass
DevOps and the DBA- 24 Hours of Pass
 
Cloud for agile_sw_projects-final
Cloud for agile_sw_projects-finalCloud for agile_sw_projects-final
Cloud for agile_sw_projects-final
 
Fits docker into devops
Fits docker into devopsFits docker into devops
Fits docker into devops
 
What's New in Docker - February 2017
What's New in Docker - February 2017What's New in Docker - February 2017
What's New in Docker - February 2017
 
Cloudsolutionday 2016: DevOps workflow with Docker on AWS
Cloudsolutionday 2016: DevOps workflow with Docker on AWSCloudsolutionday 2016: DevOps workflow with Docker on AWS
Cloudsolutionday 2016: DevOps workflow with Docker on AWS
 
Devops
DevopsDevops
Devops
 
Unified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsUnified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOps
 
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
 
OpenStack Summit
OpenStack SummitOpenStack Summit
OpenStack Summit
 
OpenStack Boston
OpenStack BostonOpenStack Boston
OpenStack Boston
 
Docker open stack boston
Docker open stack bostonDocker open stack boston
Docker open stack boston
 
DCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production ParityDCRUG: Achieving Development-Production Parity
DCRUG: Achieving Development-Production Parity
 

More from Kellyn Pot'Vin-Gorman

Redgate_summit_atl_kgorman_intersection.pptx
Redgate_summit_atl_kgorman_intersection.pptxRedgate_summit_atl_kgorman_intersection.pptx
Redgate_summit_atl_kgorman_intersection.pptxKellyn Pot'Vin-Gorman
 
SQLSatOregon_kgorman_keynote_NIAIMLEC.pptx
SQLSatOregon_kgorman_keynote_NIAIMLEC.pptxSQLSatOregon_kgorman_keynote_NIAIMLEC.pptx
SQLSatOregon_kgorman_keynote_NIAIMLEC.pptxKellyn Pot'Vin-Gorman
 
Turning ADHD into "Awesome Dynamic Highly Dependable"
Turning ADHD into "Awesome Dynamic Highly Dependable"Turning ADHD into "Awesome Dynamic Highly Dependable"
Turning ADHD into "Awesome Dynamic Highly Dependable"Kellyn Pot'Vin-Gorman
 
Cepta The Future of Data with Power BI
Cepta The Future of Data with Power BICepta The Future of Data with Power BI
Cepta The Future of Data with Power BIKellyn Pot'Vin-Gorman
 
Pass Summit Linux Scripting for the Microsoft Professional
Pass Summit Linux Scripting for the Microsoft ProfessionalPass Summit Linux Scripting for the Microsoft Professional
Pass Summit Linux Scripting for the Microsoft ProfessionalKellyn Pot'Vin-Gorman
 
Power BI with Essbase in the Oracle Cloud
Power BI with Essbase in the Oracle CloudPower BI with Essbase in the Oracle Cloud
Power BI with Essbase in the Oracle CloudKellyn Pot'Vin-Gorman
 
ODTUG Leadership Talk- WIT and Sponsorship
ODTUG Leadership Talk-  WIT and SponsorshipODTUG Leadership Talk-  WIT and Sponsorship
ODTUG Leadership Talk- WIT and SponsorshipKellyn Pot'Vin-Gorman
 
Taming the shrew, Optimizing Power BI Options
Taming the shrew, Optimizing Power BI OptionsTaming the shrew, Optimizing Power BI Options
Taming the shrew, Optimizing Power BI OptionsKellyn Pot'Vin-Gorman
 

More from Kellyn Pot'Vin-Gorman (20)

Redgate_summit_atl_kgorman_intersection.pptx
Redgate_summit_atl_kgorman_intersection.pptxRedgate_summit_atl_kgorman_intersection.pptx
Redgate_summit_atl_kgorman_intersection.pptx
 
SQLSatOregon_kgorman_keynote_NIAIMLEC.pptx
SQLSatOregon_kgorman_keynote_NIAIMLEC.pptxSQLSatOregon_kgorman_keynote_NIAIMLEC.pptx
SQLSatOregon_kgorman_keynote_NIAIMLEC.pptx
 
Boston_sql_kegorman_highIO.pptx
Boston_sql_kegorman_highIO.pptxBoston_sql_kegorman_highIO.pptx
Boston_sql_kegorman_highIO.pptx
 
Oracle on Azure IaaS 2023 Update
Oracle on Azure IaaS 2023 UpdateOracle on Azure IaaS 2023 Update
Oracle on Azure IaaS 2023 Update
 
IaaS for DBAs in Azure
IaaS for DBAs in AzureIaaS for DBAs in Azure
IaaS for DBAs in Azure
 
Being Successful with ADHD
Being Successful with ADHDBeing Successful with ADHD
Being Successful with ADHD
 
Azure DBA with IaaS
Azure DBA with IaaSAzure DBA with IaaS
Azure DBA with IaaS
 
Turning ADHD into "Awesome Dynamic Highly Dependable"
Turning ADHD into "Awesome Dynamic Highly Dependable"Turning ADHD into "Awesome Dynamic Highly Dependable"
Turning ADHD into "Awesome Dynamic Highly Dependable"
 
PASS Summit 2020
PASS Summit 2020PASS Summit 2020
PASS Summit 2020
 
DevOps in Silos
DevOps in SilosDevOps in Silos
DevOps in Silos
 
Azure Databases with IaaS
Azure Databases with IaaSAzure Databases with IaaS
Azure Databases with IaaS
 
How to Win When Migrating to Azure
How to Win When Migrating to AzureHow to Win When Migrating to Azure
How to Win When Migrating to Azure
 
Securing Power BI Data
Securing Power BI DataSecuring Power BI Data
Securing Power BI Data
 
Cepta The Future of Data with Power BI
Cepta The Future of Data with Power BICepta The Future of Data with Power BI
Cepta The Future of Data with Power BI
 
Pass Summit Linux Scripting for the Microsoft Professional
Pass Summit Linux Scripting for the Microsoft ProfessionalPass Summit Linux Scripting for the Microsoft Professional
Pass Summit Linux Scripting for the Microsoft Professional
 
Taming the shrew Power BI
Taming the shrew Power BITaming the shrew Power BI
Taming the shrew Power BI
 
Power BI with Essbase in the Oracle Cloud
Power BI with Essbase in the Oracle CloudPower BI with Essbase in the Oracle Cloud
Power BI with Essbase in the Oracle Cloud
 
ODTUG Leadership Talk- WIT and Sponsorship
ODTUG Leadership Talk-  WIT and SponsorshipODTUG Leadership Talk-  WIT and Sponsorship
ODTUG Leadership Talk- WIT and Sponsorship
 
GDPR- The Buck Stops Here
GDPR-  The Buck Stops HereGDPR-  The Buck Stops Here
GDPR- The Buck Stops Here
 
Taming the shrew, Optimizing Power BI Options
Taming the shrew, Optimizing Power BI OptionsTaming the shrew, Optimizing Power BI Options
Taming the shrew, Optimizing Power BI Options
 

Recently uploaded

[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesThousandEyes
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 

Recently uploaded (20)

[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyesAssure Ecommerce and Retail Operations Uptime with ThousandEyes
Assure Ecommerce and Retail Operations Uptime with ThousandEyes
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 

DevOps tools for winning agility

  • 1. DEVOPS TOOLS FOR WINNING AGILITY KELLYN POT’VIN-GORMAN TECHNICAL INTELLIGENCE MANAGER, DELPHIX
  • 2. Kellyn Pot’Vin-Gorman Technical Intelligence Manager, Delphix • Multi-platform DBA, (Oracle, MSSQL, MySQL, Sybase, PostgreSQL, Informix…) • Oracle ACE Director, (Alumni) • Oak Table Network Member • Idera ACE 2018 • APEX Women in Technology Award, CTA • STEM education with Raspberry Pi and Python, including DevOxx4Kids, Oracle Education Foundation and Girl Geek Dinners • President, Rocky Mtn Oracle User Group • President, Denver SQL Server User Group • Technical author, instructor and presenter. • Author, blogger, (http://dbakevlar.com)
  • 3. THE DEVOPS JOURNEY Did you just automate testing? It can be a longer trip than planned.
  • 4. THE LIFE OF A DEVOPS ENGINEER  Automation everything possible through scripting  Orchestration between environments, including data sources  Security, (PII, HIPAA, GDPR, SOX)  Optimize projects, improving on each pass  Collaborate and Communicate  CULTURE, CULTURE, CULTURE!
  • 5. COST OF CUSTOM SCRIPTING In a 1000-person IT organization Avg. time each developer spends scripting and maintaining deployment and release management related tasks 8% • Number of developers per team 8 • Number of development teams 100 • Average fully loaded cost per FTE/year $125,000 Annual cost of scripting $8,000,000 *Black hole of DevOps
  • 6. 6 WHAT IS THE MOST MAJOR RISK TO DEVOPS • Silo’d teams continuing as they were. • No unified vision. • Introduction of the cloud, especially and SaaS, (Software as a Service) hasn’t been easy. • Lack of knowledge in the industry due to grassroots and infancy of the industry. • Lacking tools that remove human intervention and manual tasks. • FORGETING THE DATA.
  • 7. 7 A TERMINOLOGY RECAP- • Methodology: Scrum, Lean, Crystal, Kanban… • Build Automation, Pipelines, Toolchains, Automation • Continuous Delivery/Continuous Integration • Functional Testing/Unit Testing/Test Driven Development • Monitoring/Alerting • Packages/Containers/Virtualization • Release Coordination/Orchestration https://xebialabs.com/glossary/
  • 8. 8 BUILD AUTOMATION Requires scripting and automation experience for development, testing, releasing, monitoring and alerting.
  • 9. 9 CONFIGURATION MANAGEMENT Concept & Development Testing & Verification Operations & Support Deployment to Production
  • 10. 10 CONTINUOUS DELIVERY/CONTINUOUS INTEGRATION The risk is low if “fail first, fail fast” occurs in development and test only. The idea is to continuously deliver code in small batches. Small releases make it easier to develop, test and identify issues, releasing later successful software. https://continuousdelivery.com/
  • 11. 11 RAPID DEPLOY Software Content Enablement Service Need or Focus Area All Tiers of Specific Environment
  • 12. 12 RELEASE COORDINATION/ORCHESTRATION Release coordination and orchestration is like a symphony. If the coordination of the instruments are done well, the music will be released in a melodious and successful manner. If it’s not, then it can be disastrous.
  • 13. 13 THE GOALS OF ALL OF THESE ARE THE SAME- • Shorter Delivery Cycles • Higher Success Rate on Releases • Shorten Time to Address Issues • Continued Improvement in Communication • Increased Revenue • The one thing they don’t take into consideration, is the data!
  • 14. 14 LET’S TALK DATA GRAVITY AND IT’S IMPACT • Data is HEAVY. • Relational databases, (or any data sources) are heavily weighted and can be impactful to DevOps implementations.
  • 15. 15 EMBRACE VIRTUALIZATION… A technical approach in which users and applications do not use physical machines, but simulated systems running on actual, “real” hardware. Virtualization can be used to eliminate resource usage and enable savings for databases, network, file and application management, along with server infrastructure.
  • 16. Virtualize and Deployed▶ ▶ MANY TYPES OF VIRTUALIZATION Storage Pool for Delphix QA DEV PATCH TEST PRODUCTION Database/App Tier 1 TB 1 TB 600GB Read From Production Each Virtual Database takes up around 5-10Gb upon creation, (dependent upon parameters) TEST Read AND Write
  • 17. DATA VIRTUALIZATION: FROM PROD TO VIRTUAL Source Environment SCSI/SSL
  • 18. DATA VIRTUALIZATION: PROD TO VIRTUAL Source Environment SCSI/SSL Any Storage Validated Sync Environment
  • 19. DATA VIRTUALIZATION: SPACE SAVINGS Delphix Virtualization EngineValidated Sync Target Dozen Virtual Database Copies, (VDBs)
  • 20. DATA VIRTUALIZATION: HOW IS IT POSSIBLE? Delphix Virtualization Engine Any Storage
  • 21. 21 WHY DATABASE VIRTUALIZATION IS DIFFERENT • DBA is “only as good as their last backup”. • Many database tools take considerable time to recover or refresh. • DevOps is often about automation- automate an “undo” for development and testing that includes data rewind. • Include a self-service tool that will allow for rewind without DBA intervention. • Allows for data version control and DataOps, the next step in DevOps
  • 22. 22 TIME TO TEST IN DEVOPS Task Time Needed Requirement Gathering 4-8hrs Test Case Creation 4-8hrs Test Data Generation Several Hours/Days Test Script Generation 3hrs-16hrs Test Execution- Manaul 1 day-2 Days Test Results 1-2hrs https://autonomiq.io/
  • 23. 23 HOW TO ENHANCE DEVOPS DATA
  • 24. Timeline in a Project Waterfall- 5 Releases January February March April Agile with Virtualization- 12 Releases January February March April Database Refresh Development Production Release Testing Cycle The Goal of Agile Data Holdup to Development Cycle
  • 25. 25 FURTHER- DATA PODS/CONTAINERS Containers offer the ability to isolate application code and/or the whole infrastructure stack into a package able entity to ease deployment, even from the same kernel. This is a powerful tool for DevOps to ease deployment for complex tiers, applications and multiple data stores. Data Pods is the next step, where a pod is created from virtualized environments.
  • 27. DATA PODS FOR EASE OF DELIVERY Rsync (UNIX/Linux) Robocopy (Windows) QA NFS iSCSI Development NFS iSCSI Testing NFS iSCSI Delphix Virtualization Engine Same Storage Size as Production Deployed using Jenkins Create “Container”
  • 28. Branching and Bookmarking • The ability to mark each iteration of development with a bookmark • Simplify to lock and deliver while testing a consistent image via a virtual database, (VDB) • If a something goes wrong, the ability to “bookmark”, (and subsequent snapshot) to deliver to development to address.
  • 31. DevOps and Source Control
  • 34. 37
  • 35. 38 THE PERIODIC TABLE OF DEVOPS TOOLS https://xebialabs.com/periodic-table-of-devops-tools/
  • 36. 39 ANSIBLE • Learn about Playbooks • Powerful collection of steps, (aka plays) • Plays consist of a list of tasks • Handlers allow for remote operations • YAML Syntax • Security configurations based on hosts and users • Ability to run as secondary users, (privilege escalation) • Huge support community and best practices are mature
  • 37. 40 EXAMPLE OF AN ANSIBLE CALL - name: Transfer and execute a script. hosts: server remote_user: test_user sudo: yes tasks: - name: Transfer the script copy: src=test.sh dest=/home/test_user mode=0777 - name: Execute the script command: sh /home/test_user/test.sh
  • 38. 41 JENKINS Jenkins is an open source automation server- • Easiest executed via a jetty servlet • Has an easy to access URL, (commonly host:8080) • Automation is done via individual projects. • Has numerous plugins to support environments • Supports multiple scripting languages, including shell, batch commands and Ant scripts. • Supports robust revision control and compatible with version control systems like CVS and Subversion. • Supports notifications like qmail and sendmail.
  • 39. 42 JENKINS GROOVY SCRIPT EXAMPLE // prepare variables for subscript execution: String subscript = "cdmserver-monitor-instance- status.groovy"; println "executing subscript ${subscript}" // variables to be passed to the subscript status = (action == "start" ? "started" : "stopped") def s = ScriptlerConfiguration.getConfiguration().getScriptById(su bscript) File scriptSrc = new File(ScriptlerManagment.getScriptDirectory(), s.getScriptPath());
  • 40. 43 JENKINS PLUGIN FOR DELPHIX https://github.com/delphix/delphix-jenkins-plugin
  • 41. 44 PUPPET • Review, automate, deploy and invest in all your software. • Control and enforce consistency across environments and platforms. • Share, test and enforce changes on-premise and in the cloud. Similar to Puppet • Chef • Jenkins
  • 42. 45 KUBERNETES • Open-source system for automating deployments, scaling and containerizing. • Resource level isolation at the kernel level, making it lighter on systems. • It works with docker and other Containers, DevOps tools and applications. • Used by Google and IBM cloud containers • Oracle Cloud Infrastructure, (OCI) uses it for Oracle Linux bare metal • Red Hat for OpenShift • 1 or more containers = pod
  • 46. 49 GITHUB Git is an open-source version control system that was started by Linus Trovalds GitHub Alternatives • Subversion • CVS • Mercurial • Git
  • 52. FULL ARA PLATFORM • Not just about testing automation • But doesn’t forget testing OR the data • Has plugins for the entire enterprise lifecycle of DevOps • Reports on the release pipeline • Even does risk assessment
  • 53. 56 SO, KNOW YOUR TOOLS Incredible variation of tools and knowing tool classifications doesn’t mean that the tools are alike. Find out what each group is using from the tools and embrace coverage that incorporates success.
  • 54. 57 PROACTIVELY ASSESS IT ALL • Build out a hierarchy of all environments and teams involved. • Group projects into bronze, silver, gold, platinum • Begin to isolate requirements for automation, collaboration and teams. • Identify and document existing tools, platforms and programming languages used. • Create goals and timelines • Build your DevOps Project team.
  • 55. • Update applications, data loads and processes towards an approach to the cloud and heterogenous toolsets. • Eliminate additional data jumps, using staging and single load processes. Attempt to use a golden copy whenever applicable. • Use the right management and monitoring tools for the focus, heterogenous development tools for increased success. • Locate a methodology that makes the whole team more successful. Know Thy Enemy…
  • 56. • All IaaS solutions provide encryption in-flight and encryption at-rest • But encryption doesn’t protect data as much as it needs to be . • Europe already requires data masking, not just data encryption for any confindential data, (GDPR): • http://ec.europa.eu/justice/data-protection/article-29/documentation/opinion- recommendation/files/2014/wp216_en.pdf CONFIDENTIAL DATA
  • 57. GENERAL DATA PROTECTION REGULATIONS The GDPR (General Data Protection Regulation) seeks to create a harmonised data protection law framework across the EU and aims to give citizens back the control of their personal data, whilst imposing strict rules on those hosting and ‘processing’ this data, anywhere in the world.
  • 58. WHO SHOULD HAVE ACCESS TO WHAT? • Granule control in tools is essential • Automation may remove need for access to critical data • Deter need to bring data out of secured systems to determine requirements, etc. Do they need access to production data at all?
  • 59. • Encryption is reversible data obfuscation, which is very different from masking data. • Data masking is non-reversible. • It solves the issue at the data level. • Is authentication and authorization in non-production in compliance with security goals? • All organizations will soon need to review if critical data in non-production environments be accessible to developers, testers and users. CONFIDENTIAL DATA
  • 60. • Masking personally-identifiable, (PII, HIPAA, PCI, etc.) information renders it useless from a security standpoint • Resolves both the technical and personal responsibility issue. • The data can be masked before it moves to non- production, removing unnecessary risk. WHY MASKING IS PART OF THE ANSWER
  • 63. 66 HOW TO SUCCEED WITH DEVOPS TOOLS Culture is one of the biggest challenges in DevOps: • Automating repeatable and complex processes, (scripting and automation tools) • Get data out of the way of DevOps, (Containers, Data Pods and Virtualization) • Locate heterogeneous tools/scripting languages that can support multiple tiers of technology. • Understanding that automation does not lessen the value of the contributors, but increases it. • Read- The Phoenix Project and the DevOps Handbook!
  • 64. THANK YOU! Email: dbakevlar@gmail.com Twitter: @DBAKevlar Website: http://dbakevlar.com

Editor's Notes

  1. Setting off on the DevOps journey can be unnerving. If you only came here to learn how to use DevOps for testing, you’re only part way there. DevOps often overlooks testing, (dev/ops, yes?) but it needs to have a complete Development cycle solution. What’s it going to take to implement DevOps across the enterprise? As many who have gone before you, that means you have to plan anywhere from a small team to possibly hundreds or more resources, applications, data sources and hardware. How many scripts and tools will be required to make your pilot projects work? Consider those to support infrastructure, manage deployments, even orchestrate release pipelines or the DevOps toolchain. How will you take this on without having DevOps consume your teams completely? This is what we’ll discuss here and what tools can ease the transition.
  2. Deployment can be simple if you’re deploying a single application to a single environment that only plans on changing on a regular interval or include a small portion of your business. What do you do when it comprises of large data sources and complex organizations with constant change? This is the norm for today’s enterprise applications, which deploy multiple times to different environments, even hourly. Deployment has traditionally meant writing and testing scripts, customizing and configuring them for different applications and environments, and updating them every time something changes… in a very manual way, but that isn’t scalable to DevOps. Introducing the DevOps model- A more modern and scalable approach is to automatically generate standardized deployment plans based on a model of your application and environment. Think of it like a “template”, with different baselines and sizes for enivornments to use. Relying on a model means that your deployment plans are more likely to execute in a repeatable manner across the enterprise, even with extensive custom scripting, as it’s transparent due to tools utilized. When deployment plans are generated automatically, development teams can spend less time writing and maintaining scripts and more time innovating.
  3. The larger and more complex your environments are, the more benefit you’ll see from advanced deployment approaches such as blue-green deployments, canary deployments, and rolling releases. Implementing these patterns with a provisioning, configuration management, or Continuous Integration tool requires extensive manual work and scripting, which means more complicated maintenance… forever. Take advantage of the pre-built DevOps best practices in an Application Release Application (ARA)tool, rather than trying to script, in other words- don’t reinvent the wheel! When you have built-in support for sophisticated release and deployment approaches, adapting a deployment for a particular situation is as easy as selecting a different pattern. As teams mature in their Continuous Delivery adoption, they can experiment with the patterns that are best for their goals, while still benefitting from standardized processes and pipelines.
  4. Keep in mind that there are many terms used for the concepts on this slide. I’ve chosen the most common ones, but depending on the choice in Agile and DevOps methodology, the words may change, but the goal is the same. DevOps is a software engineering practice that aims to bring Development and Operations teams closer together to achieve shorter development cycles and more frequent, more reliable software deployments. As companies venture off into galaxies unknown, the DevOps journey usually brings big changes to a company’s culture, processes, and tools. Ultimately, DevOps is about delivering products, features, and services to customers and to the market faster—and automation of the complete application lifecycle plays a key role. Reliably promoting software through a delivery pipeline, step by step and from environment to environment, typically requires a toolchain of myriad software tools designed to carry out a variety of specialized delivery functions, from Continuous Integration to configuration management, from build to Application Release Automation… and often many more. Application Release Automation (ARA) orchestrates the overall build, test, provisioning, configuration management, and deployment tools in your software delivery pipeline to automate the release process, starting with the initial idea of a feature and ending with the ultimate delivery of that feature to the user. ARA also encompasses automating the deployment of applications across multiple environments in the pipeline, from Development to Testing to Staging to Production.
  5. Build automation is the process of automating the creation of a software build and the associated processes including: compiling computer source code into binary code, packaging binary code, and running automated tests.
  6. Configuration management (CM) is a systems engineering process for establishing and maintaining consistency of a product's performance, functional, and physical attributes with its requirements, design, and operational information throughout its life. A DBA’s desire for low risk and stability assists here as we desire routine that results in expected outcomes.
  7. Continuous delivery (CD) is a software engineering approach in which teams produce software ... incremental updates to applications in production. A straightforward and repeatable deployment process is important for continuous delivery.
  8. This is another area that introduces risk, so DBAs can be very adverse to it, but the focus on a single feature, helps minimize the impact and often isolate any issues.
  9. Release Orchestration is the use of tools like XLRelease which manage software releases from the development stage to the actual software release itself.
  10. OK, yours may currently may not be the same. We need to talk about how you can become aligned with everyone else’s goals. It doesn’t mean you have to give up your first database. You can be part of the goals of the company and still protect the data, all of the data and the database. Enterprises have many DevOps tasks that can’t be automated and require manual intervention, and scripts are inherently not suited for many of these activities, but this is why we incorporate tools into the solution to begin with. You will be required to have the insight and management skill to audit your end-to-end development process and delivery pipeline to include all manual and automated processes, automating anything that isn’t already and should be. You will need to help identify and build out a shared, purpose-built ARA platform that everyone can utilize as part of your Continuous Delivery solution.
  11. The concept was first coined just a few years ago by a Senior VP Platform Engineer, Dave McCrory. It was an open discussion aimed at understanding how data impacted the way technology changed when connected with network, software and compute. He discusses the basic understanding that there’s a limit in “the speed with which information can get from memory (where data is stored) to computing (where data is acted upon) is the limiting factor in computing speed.” called the Von Newmann Bottleneck. These are essential concepts that I believe all DBAs and Developers should understand, as data gravity impacts all of us. Its the reason for many enhancements to database, network and compute power. Its the reason optimization specialists are in such demand. Other roles such as backup, monitoring and error handling can be automated, but the more that we drive logic into programs, nothing is as good as true skill in optimization when it comes to eliminating much of data gravity issues. Less data, less weight- it’s as simple as that.
  12. In computing, virtualization means to create a virtual version of a device or resource, such as a server, storage device, network or even a database. The framework divides the resource into one or more execution environments. For data, this can result in a golden copy or source that is used for a centralized location and removal of duplicated data. For read and writes, having unique data for that given copy, while duplicates are kept to singular. So why is this so important to me? I was a database administrator before becoming DevOps and I continually see data as the biggest hurdle for most successes.
  13. Point out the engine and size after we’ve compressed and de-duplicated. Note that each of the VDBs will take approximately 5-10G vs. 1TB to offer a FULL read/write copy of the production system It will do so in just a matter of minutes. That this can also be done for the application tier!
  14. Using any storage and only fraction of space Syncs with native or third-party database/snapshot backups Can maintain weeks of data changes. Managed just like any full copy of a database Users can instantly provision a read/write virtual copy of a database Can be used for replication, ETL, mirroring, and change data capture (CDC).
  15. How do we “rewind” data and code changes now? Why should the DBA rewind changes made in dev and test? Why should you be the one to do this in test? Virtualization removes this. The Virtual databases are read and write, so even maintenance tasks, like DBCC’s can be offloaded to one. Ability to version control, not just the meta data, but the user data!
  16. This is the time to be eliminated.
  17. DBA has to commandeer a database for patch testing. This has to be performed for EACH environment, 100’s or 1000’s of databases! Most are not synchronized with production, different outcomes when released to production. Bugs occurring in one, not another!
  18. Over 80% of time is waiting for RDBMS, (relational databases) or other data source/test data to be refreshed. Developers and Testers are waiting for data to do their primary functions. This allows for faster and less costly migrations to the cloud, too.
  19. Package software into standardized units for development, shipment and deployment. A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings.
  20. A data pod is a set of virtual data environments and controls built then delivered to users for self-service data consumption. It allows for self-management without the need for DBAs to manage standard processing, automate rebuilds and even remove need for backout scripts when development, testing and promotion goes wrong.
  21. We refer to a container as a template in our product. Note that a data pod can be moved here or to the cloud…
  22. This is a cornerstone to developers and testers, so as DBAs, we know the pain when a developer comes to us to flashback a database and before that, recover or logically recover, (import or datapump) independent objects. What is The developer/tester could do this for themselves?
  23. If you rely on custom, handwritten scripts, you won’t have automated rollback when things go wrong—leaving you scrambling to resolve the failure and restore your application to a usable state- rarely if ever is the application the only thing that’s changed- external files and even more complex- data sources have changed and need to be rolledback as well. You could script this and it often is just this once you get to the release- just means the number of scripts you’re dealing with multiplies exponentially, and the overhead to make sure those rollbacks scripts are all properly implemented increases dramatically. Depending on where the failure occurs… your rollback script might fail, too. This is where self-service portals to eliminate the need in non-prod environments can make a real difference, removing the need from the development/testing cycle until the test of the actual release/rollback.
  24. This is the interface for Developers and testers- they can bookmark before important tasks or rewind to any point in the process. They can bookmark and branch for full development/testing needs.
  25. This may appear to be a traffic disaster of changes, but for developers with Agile experience, a “sprint” looks just like this. You have different sprints that are quick runs and merges where developers are working separately on code that must merge successfully at the correct intersection and be deployed. Versioning with source control is displayed at the top, using Virtual images. You can see each iteration of the sprints. In the middle section is the branches of that occur during the development process. A virtual can be spun from a virtual, which means that it’s easier for developers to work from the work another developer has produced. Stopping points and release via a clone is simply minutes vs. hours or days.
  26. The maturity of the DevOps environment will decide how silo’d or how blended the role you’ll have in DevOps vs. your standard role as a DBA.
  27. Methods provide a format or guide to work from. Hybrid approaches often implement best. Collaboration methods ensure that communication continues when team members return to their desks Deployment tools help with documenting and lessons learned Build tools help with automation and orchestration
  28. Scrum focuses on features, bug fixes and backlog debt. Serves very large teams, including those 800+ Lean’s goal is to eliminate all waste, over demand on resources and ability to deliver faster and more effectively each time. XP is one of the most controversial due to the ability to deliver even to large companies every couple minutes if required.. Very disciplined approach. Crystal is often known under Crystal Clear, Yellow Orange and others.
  29. What does a sprint look like?
  30. These are just a few, but the ones we’ll commonly see as DBAs These are some of the most common software products that you’ll be using as a DevOps DBA.
  31. Ant is another java based built tool that’s part of Apache open-source project. Similar to Make and written in XML.
  32. This Groovy script executes another script, making it valuable in environments that already have a number of mature scripts in place that should be reused in automation.
  33. This is our plugin- that’s how important we find these tools that we’ve built it into Delphix….
  34. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.. It groups containers that make up an application into logical units for easy management and discovery. Originally by Google and now run by Cloud Native Computing Foundation. Name stands for pilot. Kubernetes is being used in a number of vendor solutions including Mesosphere for its DC/OS product[12], Rancher Labs for its container management platform[13], Pivotal for its PKS product[14], Red Hat for its OpenShift product,[15][16] CoreOS for its Tectonic product, Mirantis for its Mirantis Cloud Platform[17], and IBM for its IBM Cloud Container Service [18][19] and IBM Cloud Private product.[20][21][22] Oracle joined the Cloud Native Computing Foundation as a platinum member on September 13, 2017. Oracle open sourced a Kubernetes installer for Oracle Cloud Infrastructure and released Kubernetes on Oracle Linux.[23]
  35. Any cloud, application and a huge number of plugins, this product is focused on automation and from a DevOps tier. Allows for lab automation, CI/CD, Cloud Transformation, Network transformation and… Cyber Ranges: Reduce the effort required to achieve comprehensive security compliances to make cyber training more accessible for companies. This product also deploys virtually, which means a reduction in the impact to cloud costs, (storage and sprawl)
  36. Jira isn’t just for bugs, (that’s what DBAs think, anyway… )
  37. The dashboard is clear and concise, displaying the success and failure rates of test cases and progression.
  38. DevOps automation, scale, legacy retirements Weaker on the Security, as it still discusses accessing data vs. masking data, but that’s still a common problem with most test data. Full integration with JIRA, Confluence, Crowd, LDAP, Continuous Integration tools like Jenkins, Bamboo and Automation tools like Selenium, QTP/UFT etc.
  39. Test Data, Test Case and Test Script Generation Test Management It has orchestrators and automator for scripting. This is a very test-focused tool.
  40. Script against any UI, Document with one-click, Execute in parallel, and host your test environment with in-house private cloud streamline test automation to do regression testing Just-in-Time Any applications, desktop automation with Cross-Platform execution, Early Test Automation right at the design phase
  41. Automation Release Application Tools are Enhanced for testing why?
  42. The status of a release—and the chance that it’s going to fail—isn’t just a technical concern. All stakeholders need tools that will take their “must have” requirements into scope. Trying to hand-code and maintain scripts is highly error-prone, especially as you scale. Any scripts used should focus on a tool that can eliminate human intervention from them and release with a secondary push file. Many tools today can serve unlimited programming languages and plugins for different DevOps tools. Use these to determine where new tools should be introduced with the focus on longevity.
  43. Main points are: Collection of data on EU citizens, including IP Addresses, cookies, URL and user names online are all subject to GDPR. The citizen has a right to have proof of why you require their data, have it validated, audited, corrected if it isn’t correct And the right to be forgotten. Fines are up to 4% annual revenue or 20 million euro.
  44. As development teams are empowered to deploy applications as part of their Continuous Delivery process, a simple approach to user management and granular access control becomes more important to ensure that code that doesn’t belong in Production doesn’t end up there. If access control is defined in code, then the burden of configuring and maintaining it falls to technical staff. Public companies and organizations in regulated industries are often specifically required to ensure that access control is carefully managed and that separation of duties is enforced.
  45. Or does it shift the problem toward authentication and authorization?
  46. Two books at the bottom…they’re homework