The Information Technology have led us into an era where the production, sharing and use of information are now part of everyday life and of which we are often unaware actors almost: it is now almost inevitable not leave a digital trail of many of the actions we do every day; for example, by digital content such as photos, videos, blog posts and everything that revolves around the social networks (Facebook and Twitter in particular). Added to this is that with the "internet of things", we see an increase in devices such as watches, bracelets, thermostats and many other items that are able to connect to the network and therefore generate large data streams. This explosion of data justifies the birth, in the world of the term Big Data: it indicates the data produced in large quantities, with remarkable speed and in different formats, which requires processing technologies and resources that go far beyond the conventional systems management and storage of data. It is immediately clear that, 1) models of data storage based on the relational model, and 2) processing systems based on stored procedures and computations on grids are not applicable in these contexts. As regards the point 1, the RDBMS, widely used for a great variety of applications, have some problems when the amount of data grows beyond certain limits. The scalability and cost of implementation are only a part of the disadvantages: very often, in fact, when there is opposite to the management of big data, also the variability, or the lack of a fixed structure, represents a significant problem. This has given a boost to the development of the NoSQL database. The website NoSQL Databases defines NoSQL databases such as "Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open source and horizontally scalable." These databases are: distributed, open source, scalable horizontally, without a predetermined pattern (key-value, column-oriented, document-based and graph-based), easily replicable, devoid of the ACID and can handle large amounts of data. These databases are integrated or integrated with processing tools based on the MapReduce paradigm proposed by Google in 2009. MapReduce with the open source Hadoop framework represent the new model for distributed processing of large amounts of data that goes to supplant techniques based on stored procedures and computational grids (step 2). The relational model taught courses in basic database design, has many limitations compared to the demands posed by new applications based on Big Data and NoSQL databases that use to store data and MapReduce to process large amounts of data. Course Website http://pbdmng.datatoknowledge.it/ Contact me to download the slides

1. The
Gangs
of
Three
Ciao
ciao
Vai
a
fare
ciao
ciao
Dr. Fabio Fumarola
Maven,
Gradle
and
SBT

2. Java
Project
related
ques>ons
• How
do
we
create
a
java
project?
• How
do
we
add
dependencies?
• How
do
we
create
a
runnable
jar?
• How
do
we
create
a
war?
• How
do
we
execute
tests?
1

3. Contents
• Maven
• Gradle
• Sbt
2

4. What
is
Maven?
• Apache
Maven
is
a
soIware
project
management.
• It
is
based
on
the
concept
of
a
Project
Object
Model
(POM)
• Maven
can
mange
a
project’s
build,
repor>ng
a
documenta>on
form
a
central
piece
of
informa>on
But
it
isn’t
a
mere
build
tool
3

5. Maven
features
• Dependency
System
• Mul>-­‐module
builds
• Consistent
project
structure
• Consistent
build
model
• Plugin
oriented
• Project
generated
sites
4

6. Advantages
over
Ant
• Eliminate
complicate
scripts
• All
the
func>onality
required
to
build
your
project,
i.e.,
clean,
compile,
copy,
resources,
install,
deploy
• Cross
Project
Reuse
–
Ant
has
no
convenient
way
to
reuse
target
across
projects.
5

7. How
Maven
Works?
6

8. Maven
keywords
• POM
• Archetype
• Ar>fact
or
dependency
• Plugin
• Goal
7

9. POM
• Unit
of
work
in
Maven
• It
is
an
XML
file
that
contains
the
informa>on
and
the
configura>on
details
used
by
Maven
to
build
the
project
8

10. Archetype
• It
is
a
template
of
a
project
with
is
combined
with
some
user
input
to
produce
a
working
Maven
project
• There
are
hundreds
of
archetypes
to
help
us
get
started
– $
mvn
–version
– $
mvn
archetype:generate
-­‐
DgroupId=com.mycompany.app
-­‐Dar>factId=my-­‐app
-­‐
DarchetypeAr>factId=maven-­‐archetype-­‐quickstart
-­‐
Dinterac>veMode=true
– $
mvn
eclipse:eclipse
9

11. Ar>fact
• An
ar>fact
is
a
module
obtained
by
another
ar>fact
deployed
to
a
maven
repository
• Each
ar>fact
belongs
to
a
group
• Each
group
can
have
more
ar>facts
10
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.6</version>
</dependency>

12. Goals
and
Plugins
• Are
an
extension
of
the
standard
maven
lifecycles
steps.
– Clean,
compile,
test,
package,
install
and
deploy
• There
are
plugin
to
do
other
steps
such
as
11
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.5.3</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
[...]
</plugin> </plugins>

13. What
is
Gradle?
• Gradle
is
a
general
purpose
build
system
• It
comes
with
a
rich
build
descrip>on
• language
(DSL)
based
on
Groovy
• It
supports
”build-­‐by-­‐conven>on”
principle
• But
it
is
very
flexible
and
extensible
• It
has
built-­‐in
plug-­‐ins
for
Java,
Groovy,
Scala,
Web
• Groovy
as
a
base
language
allows
impera>ve
programming
in
the
build
file.
12

14. Advanced
features
• Parallel
unit
test
execu>on
• Dependency
building
• Incremental
build
support
• Dynamic
tasks
and
task
rules
• Gradle
daemon
13

15. Hello,
Gradle
• Create
a
file
build.gradle
task
hello
<<
{
println
'Hello
World'
}
• Run
$
gradle
hello
• Edit
the
file
as
below
task
hello
<<
{
print
'Hello
'
}
task
world(dependsOn:
hello)
<<
{
println
'World!"
}
• Run
$
gradle
–q
world
14

16. Build
scripts
are
code
task
upper
<<
{
String
someString
=
'mY_nAmE'
println
"Original:
"
+
someString
println
"Upper
case:
"
+
someString.toUpperCase()
}
-­‐
$gradle
upper
4.>mes
{
counter
-­‐>
task
"task$counter"
<<
{
println
"I'm
task
number
$counter"
}
}
15

17. Gradle
is
Groovy
that
is
Java
JAVA
import
java.ioFile;
…
String
parentDir
=
new
File(“test.txt”).getAbsoluteFile()
.getParentPath();
Groovy
def
parentDir
=
new
File(“test.txt”).absoluteFile.parentPath
Gradle
parentDir
=
file(“test.txt”).absoluteFile.parentPath
16

18. Building
a
Java
project
apply
plugin:
‘java’
17

19. Gradle
support
all
ant
task
task
hello
<<
{
String
gree>ng
=
"hello
from
Ant"
ant.echo(message:
gree>ng)
}
task
list
<<
{
def
path
=
ant.path
{
fileset(dir:
'libs',
includes:
'*.jar')
}
path.list().each
{
println
it
}
}
task
zip
<<
{
ant.zip(des{ile:
'archive.zip')
{
fileset(dir:
'src')
{
include(name:
'**.xml')
exclude(name:
'**.java')
}
}
}
18

20. Overriding
conven>ons
Version
=
1.0
Group
=
‘org.mygroup’
task
release(dependsOn:
assemble)
<<
{
println
'We
release
now'
}
build.taskGraph.whenReady
{
taskGraph
-­‐>
if
(taskGraph.hasTask(':release'))
{
version
=
'1.0’
}
else
{
version
=
'1.0-­‐SNAPSHOT’
}
}
19

21. Referencing
files
&
file
collec>ons
• Groovy-­‐like
syntax:
File
configFile
=
file('src/config.xml')
• Create
a
file
collec>on
from
a
bunch
of
files:
FileCollec>on
collec>on
=
files(
'src/file1.txt',
new
File('src/file2.txt'),
['src/file3.txt',
'src/file4.txt'])
• Create
a
files
collec>on
by
referencing
project
proper>es:
collec>on
=
files
{
srcDir.listFiles()
}
• Opera>on
on
collec>ons
def
union
=
collec>on
+
files('src/file4.txt')
def
different
=
collec>on
-­‐
files('src/file3.txt')}
20

22. Using
file
collec>ons
as
input
• Use
a
File
object
to
specify
the
source
directory.
compile
{
source
=
file('src/main/java')
}
• Using
a
closure
to
specify
the
source
files.
compile
{
source
=
{
file(‘src’).listFiles()
.findAll
{
it.name.endsWith('.zip')
}
.collect
{
zipTree(it)
}
}
}
}
21

23. Copying
files
• Using
Gradle
task
type:
task
copyTask(type:
Copy)
{
from
'src/main/webapp‘
into
'build/explodedWar‘
include
'**/*.jsp‘
exclude
{
details
-­‐>
details.file.name.endsWith('.html')
&&
details.file.text.contains('staging')
}
}
22

24. DEPENDENCY
MANAGEMENT
23

25. Repository
configura>on
• Remote
Repos
repositories
{
mavenCentral()
mavenCentral
name:
'mul>-­‐jar-­‐repos',
urls:
["h‚p://repo.mycompany.com/jars1",
"h‚p://repo.mycompany.com/jars1"]
}
• Local
Repos
repositories
{
flatDir
name:
'localRepository',
dirs:
'lib'
flatDir
dirs:
['lib1',
'lib2']
}
24

26. Referencing
dependencies
dependencies
{
run>me
files('libs/a.jar',
'libs/b.jar’)
run>me
fileTree(dir:
'libs',
includes:
['*.jar'])
}
dependencies
{
compile
'org.springframework:spring-­‐webmvc:3.0.0.RELEASE'
testCompile
'org.springframework:spring-­‐test:3.0.0.RELEASE'
testCompile
'junit:junit:4.7'
}
25

27. Referencing
dependencies
List
groovy
=
["org.codehaus.groovy:groovy-­‐all:1.5.4@jar",
"commons-­‐cli:commons-­‐cli:1.0@jar",
"org.apache.ant:ant:1.7.0@jar"]
List
hibernate
=
['org.hibernate:hibernate:3.0.5@jar',
'somegroup:someorg:1.0@jar']
dependencies
{
run>me
groovy,
hibernate
}
26

28. PLUGINS
27

29. Extending
your
build
• Any
Gradle
script
can
be
a
plug-­‐in:
apply
from:
'otherScript.gradle'
apply
from:
'h‚p://mycomp.com/otherScript.gradle’
• Use
many
of
the
standard
or
3rd-­‐party
plug-­‐ins:
apply
plugin:
'java'
apply
plugin:
'groovy'
apply
plugin:
'scala'
apply
plugin:
'war'
28

30. Standard
plugins
29
http://www.gradle.org/docs/current/userguide/standard_plugins.html

31. Example
java
and
scala
project
• h‚p://www.gradle.org/docs/current/userguide/
tutorial_java_projects.html
• h‚p://www.gradle.org/docs/current/userguide/
scala_plugin.html
• h‚p://plugins.gradle.org/
30

32. SBT
The
interac>ve
build
tool
31

33. Sbt
• It
is
a
tool
to
define
task
and
then
run
them
from
the
shell
• To
install
on
linux
grab
the
– rpm:
h‚ps://dl.bintray.com/sbt/rpm/sbt-­‐0.13.7.rpm
– deb:
h‚ps://dl.bintray.com/sbt/rpm/sbt-­‐0.13.7.deb
32

34. Star>ng
with
sbt
• It
is
based
on
the
file
build.sbt
• Create
a
directory
hello_scala
• Edit
a
file
and
add
lazy
val
root
=
(project
in
file(".")).
se„ngs(
name.:=("hello”),
version
:=
"1.0",
scalaVersion
:=
"2.11.4"
)
• Create
an
HelloWorld
class
33

35. Project
Structure
• Sbt
is
base
on
maven
project
structure
34

36. Running
sbt
• It
has
a
shell
via
$sbt
• Or
we
can
run
task
in
batch
– $
sbt
clean
compile
"testOnly
TestA
TestB”
• It
also
support
con>nuos
delivery
and
run
via
– >
~compile
– >
~run
35

37. Crea>ng
a
project
defini>on
• To
create
a
project
in
the
build.sbt
file
lazy
val
root
=
(project
in
file(.)).
se„ngs(
organiza>on
:=
"com.example",
version
:=
"0.1.0",
scalaVersion
:=
"2.11.4"
)
• The
build.sbt
define
a
Project
which
holds
a
list
of
scala
expression
called
se)ngs
• Moreover
a
build.sbt
can
have
vals,
lazy
vals
and
defs
36

38. Se„ngKey,
TaskKey
and
InputKey
lazy
val
root
=
(project
in
file(".")).
se„ngs(
name.:=("hello")
)
• .:=
is
a
method
that
takes
a
parameter
and
return
s
Se„ng[String]
lazy
val
root
=
(project
in
file(".")).
se„ngs(
name
:=
42
//
will
not
compile
)
• Does
it
run?
37

39. Types
of
key
• Se„ngKey[T]:
a
key
for
a
value
computed
once
(the
value
is
computed
when
loading
the
project,
and
kept
around).
• TaskKey[T]:
a
key
for
a
value,
called
a
task,
that
has
to
be
recomputed
each
>me,
poten>ally
with
side
effects.
• InputKey[T]:
a
key
for
a
task
that
has
command
line
arguments
as
input.
Check
out
Input
Tasks
for
more
details.
• The
built-­‐in
key
are
field
of
the
object
Keys
(h‚p://
www.scala-­‐sbt.org/0.13/sxr/sbt/Keys.scala.html)
38

40. Custom
Keys
• Can
be
created
with
their:
– se„ngKey,
taskKey
and
inputKey
lazy
val
hello
=
taskKey[Unit](“An
example
task”)
• A
TaskKey[T]
can
be
used
to
define
task
such
as
compile
or
package.
39

41. Defining
tasks
and
se„ngs
• For
example,
to
implement
the
hello
task
lazy
val
hello
=
taskKey[Unit]("An
example
task”)
lazy
val
root
=
(project
in
file(".")).
se„ngs(
hello
:=
{
println("Hello!")
}
)
• Imports
in
build.sbt
import
sbt._
import
Process._
import
Keys._
40

42. Adding
library
dependencies
val
derby
=
"org.apache.derby"
%
"derby"
%
"10.4.1.3"
lazy
val
commonSe„ngs
=
Seq(
organiza>on
:=
"com.example",
version
:=
"0.1.0",
scalaVersion
:=
"2.11.4"
)
lazy
val
root
=
(project
in
file(".")).
se„ngs(commonSe„ngs:
_*).
se„ngs(
name
:=
"hello",
libraryDependencies
+=
derby
)
41

43. Dependencies
• To
add
an
unmanaged
dependency
there
is
the
task
unmanagedBase
:=
baseDirectory.value
/
"custom_lib”
• To
add
an
managed
dependency
there
is
the
task
libraryDependencies
+=
groupID
%
ar>factID
%
revision
• Or
libraryDependencies
++=
Seq(
groupID
%
ar>factID
%
revision,
groupID
%
otherID
%
otherRevision
)
42

44. Resolvers
• To
add
a
resolver
1. resolvers
+=
name
at
loca>on
2. resolvers
+=
Resolver.mavenLocal
3. resolvers
++=
Seq(name1
at
loca>on1,
name2
at
loca>on2)
• Is
it
possible
also
to
define
if
the
dependency
is
for
test
libraryDependencies
+=
"org.apache.derby"
%
"derby"
%
"10.4.1.3"
%
Test
43

45. Plugins
• To
declare
a
plugin
add
a
directory
project/Build.sbt
44