1. Database migrations - the
missing piece in Java EE
Rikard Thulin

2. 3 billion Java developers
lives was ruined on
August 29, 1997
The survivors of the
deployment called the war
Judgment Day.

3. They lived only to face a new
nightmare, the war against the
DBA.
Skynet, the computer which
controlled releases, sent
two engineers back through
time

4. Their mission: to solve
the problems of
schema migrations…
Opening “quote” from “Terminator 2: Judgment Day”

5. You willlearnhow to
behappieras you can spend
more timewriting code!

6. And we will have alookinto
thefutureand the
possibilities at hand

7. Roadmap
• Introduction to the topic and why it is so important
• The current situation, the tools of the trade
• Future possibilities at hand to solve this problem

8. Part 1
Introduction to the topic
and why it is so important

9. Finally,
we where ready to go
IT WAS RELEASE DAY

10. Just get the luggage

11. There might be a small burden

12. Let the team lead or the smart guy do it!
Or if you are a manager, it translates to the expensive guy

13. The team goes…

14. This is your complete team

15. change recall refactoring applied some marked specified
better run units the tagged along more SQL precondition pain
state both statements contains production Sets applying
database nested things executes identified context leave test
one management Generally apply HOPE change stable
individual ran database just IT generate fails different
changes uniquely per machines change written
productivity log attribute Set change sets development author
files WORKS Contexts describes by descriptive
someone regardless Major else environments column
queries local Each yet either WORK change ANYMORE set
Preconditions DATABASECHANGELOG unexpected control
file !? Concepts stop store execution arbitrarily location auto
commit executed Developers

16. change recall refactoring applied some marked specified
better run units the tagged along more SQL precondition pain
state both statements contains production Sets applying
database nested things executes identified context leave test
one management Generally apply HOPE change stable
individual ran database just IT generate fails different
changes uniquely per machines change written
productivity log attribute Set change sets development author
files WORKS Contexts describes by descriptive
someone regardless Major else environments column
queries local Each yet either WORK change ANYMORE set
Preconditions DATABASECHANGELOG unexpected control
file !? Concepts stop store execution arbitrarily location auto
commit executed Developers

17. change recall refactoring applied some marked specified
better run units the tagged along more SQL precondition pain
state both statements contains production Sets applying
database nested things executes identified context leave test
one management Generally apply HOPE change stable
individual ran database just IT generate fails different
changes uniquely per machines change written
productivity log attribute Set change sets development author
files WORKS Contexts describes by descriptive
someone regardless Major else environments column
queries local Each yet either WORK change ANYMORE set
Preconditions DATABASECHANGELOG unexpected control
file !? Concepts stop store execution arbitrarily location auto
commit executed Developers

18. Finally
we are set to go…
…but…

19. DBA

20. Fill in the forms…

21. You are trapped in the Jar
(aka jar-hell)

23. It breaks andYou look like this

24. You gently communicate
with your fellow co-worker

25. This was embarrassing
and unprofessional

27. Time to be a hero!

28. Engineers likes to be heroes
This has to be a good
thing, right?

29. You (the smart guy) spend the
weekend

31. Your boss turns to this
Are you nuts? OVERTIME?!

32. The obvious solution to
professionally solve the problem

33. We add pre-release meeting!

34. change remember refactoring applied some marked
specified better run units the tagged along more SQL
precondition and state both statements contains production
Sets last database nested things executes identified context
leave test one management Generally apply suddenly
change stable individual ran time just a generate fails different
STUFF uniquely per machines change co-worker
productivity log attribute Set change sets development author
files DOES Contexts describes added descriptive NOT
regardless Major a environments column queries local Each
yet either WORK change ANYMORE set Preconditions
DATABASECHANGELOG unexpected control file !? Concepts
stop store execution arbitrarily location auto commit executed
Developers

35. change remember refactoring applied some marked
specified better run units the tagged along more SQL
precondition and state both statements contains production
Sets last database nested things executes identified context
leave test one management Generally apply suddenly
change stable individual ran time just a generate fails different
STUFF uniquely per machines change co-worker
productivity log attribute Set change sets development author
files DOES Contexts describes added descriptive NOT
regardless Major a environments column queries local Each
yet either WORK change ANYMORE set Preconditions
DATABASECHANGELOG unexpected control file !? Concepts
stop store execution arbitrarily location auto commit executed
Developers

36. change remember refactoring applied some marked
specified better run units the tagged along more SQL
precondition and state both statements contains production
Sets last database nested things executes identified context
leave test one management Generally apply suddenly
change stable individual ran time just a generate fails different
STUFF uniquely per machines change co-worker
productivity log attribute Set change sets development author
files DOES Contexts describes added descriptive NOT
regardless Major a environments column queries local Each
yet either WORK change ANYMORE set Preconditions
DATABASECHANGELOG unexpected control file !? Concepts
stop store execution arbitrarily location auto commit executed
Developers
$ git pull

37. change remember refactoring applied some marked
specified better run units the tagged along more SQL
precondition and state both statements contains production
Sets last database nested things executes identified context
leave test one management Generally apply suddenly
change stable individual ran time just a generate fails different
STUFF uniquely per machines change co-worker
productivity log attribute Set change sets development author
files DOES Contexts describes added descriptive NOT
regardless Major a environments column queries local Each
yet either WORK change ANYMORE set Preconditions
DATABASECHANGELOG unexpected control file !? Concepts
stop store execution arbitrarily location auto commit executed
Developers

38. change remember refactoring applied some marked
specified better run units the tagged along more SQL
precondition and state both statements contains production
Sets last database nested things executes identified context
leave test one management Generally apply suddenly
change stable individual ran time just a generate fails different
STUFF uniquely per machines change co-worker
productivity log attribute Set change sets development author
files DOES Contexts describes added descriptive NOT
regardless Major a environments column queries local Each
yet either WORK change ANYMORE set Preconditions
DATABASECHANGELOG unexpected control file !? Concepts
stop store execution arbitrarily location auto commit executed
Developers

39. Instead

41. The aha moment

42. From

43. To

44. The smiling team

45. Productivity increased

46. You will learn how to
behappieras you can spend
more timewriting code!

47. Sweden

48. Friendly

49. neighbor

50. to

51. Norway

52. unlike

53. Norway

54. we

55. do

56. not

57. have

58. a

59. storage

60. problem

61. Proven national gas reserves (cubic meters)
0
2,007,000,000,000
Proven national oil reserves (barrels)
0
5,320,000,000
NO STORAGE PROBLEM

63. Development
One developer, life is easy!

64. Development
Obvious we need production
Production

65. Another
joins

66. Now each developer can work
Development Production

67. Time to
bring out the engineer
from the closet

68. Development ProdCI
for the engineers
to cooperate
we need Continuos Integration

69. Time to
really
bring out the engineer
from the closet

70. ProdDev CI
someone told the boss that
things should not only be tested
on end users
QA

71. Time to
really, really
bring out the engineer
from the closet

72. StagingDev CI
Now when we know about quality assurance
we should stage
our production environment
QA
Prod

73. More
joins
As we do everything by the book
things ($) goes well

74. Staging clusterCI
How do we keep
everything CONSISTENT?
QA
Staging cluster
Prod clusterProd cluster
Dev

75. {code}
VCS Build server
How do we keep
{ code }
CONSISTENT?
artifacts
Repository
push pull
publish
release
Dev
QA
promote
DevStaging
Prod

76. How do we keep
{ code + database }
CONSISTENT?

77. Two entities

80. Really?
Is the database EVIL?

81. artifact manual, script, …
Source Database

82. the problem is to keep
them consistent

83. HOW

84. artifact manual, script, …
Source Database
db deltas,
automatic

85. artifact
Source

86.
{code +
migrations}
VCS Build server
How do we keep
{ code + database }
CONSISTENT?
artifacts
{code +
migrations}
Repository
push pull
publish
release
Dev
QA
promote
DevStaging
Prod

87. Shiny Happy People
You will get

88. Schema Migration
DEMO DEMO DEMO DEMO

89. Schema Migration
DEMO DEMO DEMO DEMO
Migrations are applied automatically
during deployment

90. Schema Migration
DEMO DEMO DEMO DEMO
Spring based application

91. Schema Migration
DEMO DEMO DEMO DEMO
What you will see

92. Shiny Happy People
DEMONSTRATION OF

93. LOCAL DEV PROD
Version 0 Version 0 Version 0

94. LOCAL DEV PROD
Version 1 Version 0 Version 0

95. LOCAL DEV PROD
Version 1 Version 1 Version 0

96. LOCAL DEV PROD
Version 2 Version 1 Version 0

97. LOCAL DEV PROD
Version 2 Version 3 Version 0

98. LOCAL DEV PROD
Version 2 Version 3 Version 3

99. What can we do?
create/drop TABLE
create/drop INDEX
add/remove CONSTRAINT
alter SQUENCE
STOREDPROCEDURES
create/drop VIEWS
insert DATA
and more!

100. What
is the trick?

101. When
is the trick?

102. Database Change Management
Real Life Tricks
• Possible to use on an existing database
• Quick and simple to implement
• Database copies mix and match
• Wash copies of production database
• DBA approval / review process
• One click release possible

103. Part 2
The current situation
tools of the trade

105. Tools of the Trade
ActiveRecord Ruby (Ruby)
Entity Framework Migrations (.net)
COMPETITION
HAS IT

106. Tools of the Trade
Hibernate & JPA
Schema Generation

107. Tools of the Trade
FlyWay & Liquibase
Stronger execution order
Supports different DB / environment
Stereotype of being “heavyweight"
Automatic rollback
Convention over configuration
Gaining popularity

108. Tools of the Trade
FlyWay
Convention over configuration
Get up and running in 5 minutes
Gaining popularity

109. Tools of the Trade
Liquibase
Stronger execution order
Supports different DB for different environment
Stereotype of being “heavyweight"
Automatic rollback

110. Use one of the tools Use one of the tools Use one of the
tools Use one of the tools Use one of the tools Use one
of the tools Use one of the tools Use one of the tools Use
one of the tools Use one of the tools Use one of the tools
Use one of the tools Use one of the tools Use one of the
FlyWay & Liquibase
Use one of the tools Use one of the tools Use one of the
tools Use one of the tools Use one of the tools Use one
of the tools Use one of the tools Use one of the tools Use
one of the tools Use one of the tools Use one of the tools
Use one of the tools Use one of the tools Use one of the
tools Use one of the tools Use one of the tools Use one

111. Tools of the Trade
summary
Schema migration is
•Not solved by Java EE today
•Solved by 3PP
•Already in competitors standards

112. Part 3
Future possibilities
to solve this problem

113. Reasons to add Migrations to Java EE
We are behind the competition
Schema migrations is not well know
to the broad developer community
Increased Quality and Productivity
Adding complexity to the platform

114. Advanced tooling

115. the future?
class User {
}

116. @Migration(id="1", author="duke")
class CreateUserTable {
@CreateTable(User.class)
@AddColumn(User.class)
long id;
@AddColumn(entity = User.class, size = "255")
String name;
}
the future?

117. the future?
@Migration(id="1", author="duke")
class CreateUserTable {
@AddColumn(User.class)
long id;
…
}
@Entity
class User {
@Id
long id;
…
}
Migration2Code
Generator

118. @Entity
class User {
@Id
long id;
@Column(size="255")
String name;
}
the future?

119. @Migration(id="2", author="duke")
class AddColumnsToUser {
@DropColumn(User.class)
String name;
@AddColumn(entity = User.class, size="80")
String firstName;
@AddColumn(entity = User.class, size="80")
String lastName;
@AddColumn(entity = User.class, size="255")
String email;
}
the future?

120. @Entity
class User {
@Id
long id;
@Column(size="80")
String firstName;
@Column(size="80")
String lastName;
@Column(size="255")
String email;
}
the future?

121. The future
my proposal
Add schema migration to
Java EE

122. The future
my proposal
Who? WE JCP JSR!
What? Lets figure it out!
When? Now. Lets start!

123. Wrap up

124. Wrap up
1. Start using one of the tools!
2. We need to figure out if
database migrations is a
candidate to be included in
Java EE
3. Now it the time to take action!

125. You can be the Hero
Hero wrap up

126. Q & A
rikard.thulin@squeed.com