Moving from manual processes for development and deployment to a more automated approach requires a great deal of work and knowledge. In this all day seminar we will go through the steps to help you along this journey.
We will start with understanding how source control works and end with automated deployments across environments
You’ll learn about processes and tools that not only make it easier and faster to move database changes, but add protection to your production information.
We will discuss tools, process and the fundamental changes in culture necessary to take your database development and deployment into a high functioning DevOps team.
DevEX - reference for building teams, processes, and platforms
DevOps for the DBA
1. DEVOPS FOR THE DBA
GRANT FRITCHEY
PRODUCT ADVOCATE
REDGATE SOFTWARE
2. GOALS
• Discuss tools, process and the fundamental changes in culture necessary to
make your database development and deployment a part of a high
functioning DevOps team
5. DEVOPS
• Gartner:
Devops represents a change in IT culture, focusing on rapid IT service delivery
through the adoption of agile, lean practices in the context of a system-
oriented approach. DevOps emphasizes people (and culture), and seeks to
improve collaboration between operations and development teams. DevOps
implementations utilize technology – especially automation tools that can
leverage an increasingly programmable and dynamic infrastructure from a
life cycle perspective.
6. DEVOPS
• Gartner:
Devops represents a change in IT culture, focusing on rapid IT service delivery
through the adoption of agile, lean practices in the context of a system-oriented
approach. DevOps emphasizes people (and culture), and seeks to improve
collaboration between operations and development teams. DevOps implementations
utilize technology – especially automation tools that can leverage an increasingly
programmable and dynamic infrastructure from a life cycle perspective.
7. DEVOPS
• Gartner:
Devops represents a change in IT culture, focusing on rapid IT service
delivery through the adoption of agile, leanpractices in the context of
a system-oriented approach. DevOps emphasizes people (and culture), and seeks to
improve collaboration between operations and development teams. DevOps
implementations utilize technology – especially automation tools that can leverage an
increasingly programmable and dynamic infrastructure from a life cycle perspective.
8. DEVOPS
• Gartner:
Devops represents a change in IT culture, focusing on rapid IT service delivery through the
adoption of agile, leanpractices in the context of a system-oriented approach. DevOps
emphasizes people(and culture), and seeks to improve collaboration between operations and
development teams. DevOps implementations utilize technology – especially automation tools that can
leverage an increasingly programmable and dynamic infrastructure from a life cycle perspective.
9. DEVOPS
• Gartner:
Devops represents a change in IT culture, focusing on rapid IT service delivery through the
adoption of agile, leanpractices in the context of a system-oriented approach. DevOps
emphasizes people(and culture), and seeks to improve collaboration
between operations and development teams. DevOps implementations utilize technology – especially
automation tools that can leverage an increasingly programmable and dynamic infrastructure from a life cycle
perspective.
10. DEVOPS
• Gartner:
Devops represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile,
leanpractices in the context of a system-oriented approach. DevOps emphasizes people(and culture), and seeks to
improve collaborationbetween operations and development teams. DevOps implementations utilize technology –
especially automation toolsthat can leverage an increasingly programmable and dynamic infrastructure
from a life cycle perspective.
11. DEVOPS
•An agile culture that better supports
collaboration between people that uses
automation and tooling in support of
process.
20. FUD IS THE
PROBLEM
DevOps only works for new systems
‘SA’ for developers
NoOps
DevOps is incompatible with security and compliance
Agile means no design
Elimination of process
24. WHY
DEVOPS FOR
THE DBA
Improved communications
Better integration with dev, ops and the
business
Added protections through superior
processes
Better support for the organization
Faster and safer deployments
27. COMMUNICATION IS CORE TO DEVOPS
Eliminating barriers
Mitigate, reduce, and eliminate choke points
Successful implementation of DevOps involves all the teams
29. CULTURAL SHIFT
Transparency
What do you do?
How do you do it?
Write it all down
Collaboration
Tear down the wall
We’re all on the same team
Clarity
Definitions matter
Language matters
35. AUTOMATED TESTING PROTECTS
PRODUCTION
• When something breaks deploying in dev, build a test
• When something breaks deploying in CI, build a test
• When something breaks deploying to QA, build a test
• Every test improves protection for production
• Testing combined with “shift-left” improves production
safety
36. FREQUENT DEPLOYMENTS MEANS PRACTICE
No script should
run first in
production
Every deployment
is a test
Practice is
fundamental to
mastery
“Shift-left” makes
practice
deployments
mean more
All this adds to
protection for
production
37. MONITORING ENSURES BEHAVIOR
Understand behavior prior to deployment
Know when deployments occur
Understand behavior after a deployment
Monitoring drives innovation
Protection starts with monitoring
48. BUILDING A DEPLOYMENT PROCESS
WORK BACKWARDS
FROM PRODUCTION
IF IT’S NOT IN
SOURCE CONTROL,
IT DOESN’T EXIST
ENSURE
COMPLIANCE IS
ALWAYS COVERED
TEST WHAT BREAKS
49. BUILDING A
DEPLOYMENT
PROCESS
Production deployment script
Test script on copy of production
Generate script for production from trunk
Merge code in source control
Add changes to source control
Make changes to code
Get code from source control
Branch code in source control
57. DATABASE TOOLING, THE MAGIC
Microsoft
SQL Server
Data Tools
Redgate SQL
Change
Automation
DBMaestro DbUp
Datical Apex/Idera
DevOps Flyway
58. HOW THE MAGIC WORKS
State
• Compare between
database &
database
• Compare between
database & source
control
• Compare between ?
& ?
Migrations
• Scripts build on
scripts
• Which build on
scripts
Hybrid
• Mix & match
60. SCENARIO
ONE
• We have a single database under
development, but we use linked servers for
reporting. How do we:
• A) Automate our processes across multiple
environments
• B) Prevent use of production data in linked
servers
61. SCENARIO
TWO
• We have multiple teams developing within a
single database. How do we:
• A) Support completely independent development
and release cycles
• B) Ensure that functionality from one team doesn’t
affect another
• C) Get each teams functionality to the other
teams
62. SCENARIO
THREE
• Our system consists of four different,
interrelated databases. Each one is on it’s
own for development and release, but, we
need to have all four while we develop.
How do we:
• A) Keep four databases on every developers
machine
• B) Ensure that different development teams don’t
break others code
63. SCENARIO
FOUR
• We’re pretty happily chugging along with
our DevOps processes, however, since
moving to Azure SQL Database and
implementing automated tuning, we’re
seeing indexes in production that don’t exist
elsewhere. How do we:
• A) See what has changed prior to deployment
• B) Ensure that we can incorporate these changes
into our own processing
67. SOURCE CONTROL FUNCTIONALITY
Keep track of code changes
Maintain a history of changes
Multiple people working simultaneously
Isolate code through branching
68. SOURCE CONTROL FUNCTIONALITY
Merge code from different branches on command
Show conflicts on merges in order to resolve them
Allow reversion to a previous state
Works within your toolset
75. A SHORT DISCUSSION ON DATABASE BRANCHING
ONE DATABASE PER
BRANCH ON AN
INSTANCE
ONE INSTANCE PER
BRANCH
ONE CONTAINER
PER BRANCH
JUST BE READY TO
REBUILD DATABASES
AS NEEDED
79. GENERAL TESTING PRACTICES
TEST EARLY AND
OFTEN
TESTING STARTS
IN DEVELOPMENT
SLOWLY ADD
TESTS OVER TIME
VALIDATE
STANDARDS
DOCUMENT YOUR
INTENTIONS
80. HOW DO
WE TEST?
Automatically
Define a known starting state
• Generated test data
• Modified production data
• Always work within Compliance requirements
Test data must be provided
85. WHAT ARE WE AUTOMATING
Getting code into source controlGetting
Retrieving code from source controlRetrieving
Generating an artifact (aka SQL)Generating
86. DATABASE TOOLING, THE MAGIC
Microsoft
SQL Server
Data Tools
Redgate SQL
Change
Automation
DBMaestro DbUp
Datical Apex/Idera
DevOps Flyway
87. STATE VS. MIGRATIONS
State
• Compare between
database &
database
• Compare between
database & source
control
• Compare between ?
& ?
Migrations
• Scripts build on
scripts
• Which build on
scripts
Hybrid
• Mix & match as
needed
88. STATE: STRENGTHS & WEAKNESSES
Strengths
• Simple to understand
• Merge is easy
• Tools generate scripts
• Visible history
Weaknesses
• Domain of problems not
solved
• Tool ordering of changes
can be problematic
• Tool choices for changes
can be an issue
89. MIGRATIONS: STRENGTHS & WEAKNESSES
Strengths
• All changes work
• Control over method and
changes
• All domain problems
handled
Weaknesses
• Harder to understand and
work with
• History is hard to assemble
• Merges may be difficult
• All changes in Dev occur in
Prod
90. THE NEED FOR AN ARTIFACT
Separate the concept of a build from a deploy
Build generates an artifact
Deploy uses the artifact to create/update a database
Automate both
92. CONTINUOUS INTEGRATION
• Get your code into source control
• Pick a tool for the build
• Pick a flow control tool
• Build your CI process
• You’ve now started down the DevOps path
100. PRODUCTION DRIFT
There will
ALWAYS be
production fixes
Prod changes
must go to VCS
Drift may block
deployments
Tooling detect
drift
Track versions of
the db yourself
Drift detection is
needed
101. ROLLBACKS
Once changes
committed, can't
be rolled back
Old version of
code Be prepared
Pre-write rollback
scripts
Have old code
versions ready
(snapshot
production)
Be ready to
restore
102. ROLLING BACK CODE
The previous version
of
procs/functions/views
can be used
Double check logical
function with the rest
of the application
Can we roll back
independently?
Is application code
rollback needed?
Previous version from
production
Previous released
version from VCS
Auto-generate
rollback code
103. ROLLBACK
TABLE AND
DATA
CHANGES
Undoing data
changes is
complex and
risky
Pre-write scripts
Include DML
changes
Test in
intermediate
environments
May need to
execute manually
Copy old data
prior to changes
Use scripts to
restore old data
104. ROLL FORWARD
Faster
• Faster than a
rollback
Safer
• Safer than a
rollback
Easier
• Easier than a
rollback
Harder
• Harder to
convince
management
to support
105. POST RELEASE
Any deployment
impacts the
system
Lack of impact is
information
Monitoring
provides a
baseline of
resource usage
Compare the
baseline to new
resource and
business use
108. SCENARIO
ONE
• We have a single database under
development, but we use linked servers for
reporting. How do we:
• A) Automate our processes across multiple
environments
• B) Prevent use of production data in linked
servers
109. SCENARIO
TWO
• We have multiple teams developing within a
single database. How do we:
• A) Support completely independent development
and release cycles
• B) Ensure that functionality from one team doesn’t
affect another
• C) Get each teams functionality to the other
teams
110. SCENARIO
THREE
• Our system consists of four different,
interrelated databases. Each one is on it’s
own for development and release, but, we
need to have all four while we develop.
How do we:
• A) Keep four databases on every developers
machine
• B) Ensure that different development teams don’t
break others code
111. SCENARIO
FOUR
• We’re pretty happily chugging along with
our DevOps processes, however, since
moving to Azure SQL Database and
implementing automated tuning, we’re
seeing indexes in production that don’t exist
elsewhere. How do we:
• A) See what has changed prior to deployment
• B) Ensure that we can incorporate these changes
into our own processing
112. BONUS
SCENARIO
• We’re using state-based delivery of our
databases. Because we have multiple
development streams, we have multiple QA
systems. We have an emergency release.
How do we:
• A) Manage getting our code from dev to QA to
preprod to production
115. DEVOPS FUNDAMENTALS
One Team More Backups
Added
Protection
Automated
Protection
Practice
Deployments
Monitoring
116. GOALS
• Discuss tools, process and the fundamental changes in culture necessary to
make your database development and deployment a part of a high
functioning DevOps team
117. RESOURCES
• State of Database Devops 2019
• The Phoenix Project
• Redgate University
• Microsoft Learning Resources for DevOps
• Microsoft DevOps Resource Center
• Steve Jones
• Kendra Little