Bundling Packages and Deploying Applications with RPM

Bundling Packages and
Deploying Applications
with RPM
Alexander Shopov
<ash@kambanaria.org>

[ash@edge ~]$ whoami
By day: Software Engineer at Cisco {,Nov. 28}
By night: OSS contributor
Coordinator of Bulgarian Gnome TP
Contacts:
E-mail: ash@kambanaria.org
Jabber: al_shopov@jabber.minus273.org
LinkedIn: http://www.linkedin.com/in/alshopov
SlideShare: http://www.slideshare.net/al_shopov
GitHub: https://github.com/alshopov
Web: Just search “al_shopov”

Please Learn And Share
License: Creative Commons Attribution 4.0
International
CC-BY v4.0

As Number of Components Grows
Complexity (Interactions) Grows as
Power of 2

As Years of Maintenence Grow
Complexity Grows at Least Linearly

As Both Number of Components
and Years Grow – Complexity
Explodes

Why RPM
● Widely used packaging format;
● Used to mean: “Red Hat Package Manager”, now
it is “RPM Package Manager”;
● Popular with Enterprise-ish distributions:
– Fedora;
– Red Hat;
– CentOS;
– Oracle's Linux;
– SUSE.
● The other popular format in Linux is “deb” used in
Debian/Ubuntu.

Where Does RPM Stand?
Inno, NullSoft, InstallShiled,
Wix, Windows Installer,
Installer for Mac OS
Compiling
Source to Binary
Make, imake, auto-tools,
project/solution files, etc.
Packaging
Binary to Packages

Where Does RPM Stand?
RPM
Compiling
Packaging
Source to Binary
Binary to Packages Make, imake, auto-tools,
Inno, NullSoft, InstallShiled,
project/solution files, etc.
Wix, Windows Installer,
Installer for Mac OS

The Packaging Process
SPECS
SOURCES
BUILD
BUILDROOT
RPMS

The Packaging Process
SOURCES
BUILD
BUILDROOT
RPMS
SPECS
%prep
%install
%files
● Uncompress & patch
– rpmbuild -bp
● Configure/compile
– rpmbuild -bc
● Install to a virtual file
system – place, copy
– rpmbuild -bi
● List files to be packaged
– rpmbuild -bb/-bs/-ba

Structure of RPM file
● Lead/identifier/magic ( starts with abed dbee - hex)
● Signature - optional
● Header – tagged structure –
tags/type/number/size/data for them
● Payload – archive. gzipped cpio (1977 by Dick Haight
UNIX 1.0) → gzipped tar (UNIX 7th ed. – 1979)
Payload
Identifier
Signature
Header

Structure of RPM Package Name
● name-version-release.architecture.rpm
● gzip-1.3.12-19.el6_4.x86_64.rpm
● Name – what package is or does and purpose
● Version – upstream version
● Release – version of RPM (prefix for distro)
● Architecture of the rpm
– noarch
– src

Common and Rarer Archiectures
Suffix Architecture Fedora RedHat CentOS Oracle
i686 32bit Intel x x x x
x86_64 64bit AMD x x x x
ia64 64bit Intel x x x
s390 24/31/32bit G4/5/6 x x
s390x IBM System Z – 64bit z900 x x
ppc 32bit PowerPC x x
ppc64 64bit PowerPC BE x x
ppc64le 64bit PowerPC LE (
arm 32bit ARM5-7 )
armhfp 32bit ARM7 x
aaarch64 64bit ARM8+ x

Important Packages to Install
● redhat-rpm-config
● rpm-build
● rpmdevtools
● rpmlint
● Other if you want to contribute to
Fedora/Redhat/CentOS. Oracle have not
published for them

redhat-rpm-config
● /etc/rpm/macros* - others also install here -
macros here perl,python, nodejs, java
● /usr/lib/rpm/redhat – macros and shell scripts

rpm-build
● rpmbuild – command to actually build
● rpmspec – query spec files
● gendiff – diff wrapper

rpmlint
● rpmlint – check errors in specs and packages
● rpmdiff – check differences between packages

rpmdevtools
● rpmfile, rpmls, rpmpeek, rpmelfsym, rpmargs
● /etc/rpmdevtools – spec templates
● rpmdev-bumpspec – auto increment and
maintain spec files
● rpmdev-setuptree – create the spec tree
● rpmdev-newinit, rpmdev-newspec

Let's Build A Package!
● Let's build Tomcat 8 package
● Proper ownership of files
● Add a startup/shtdown script
● Deploy a small package in Tomcat
● Make the package take the place of ROOT
context
● Switch provider of JPA – from Derby to Oracle
RDBMS

Prepare The Build Environment
● yum install rpmdevtools
● Dependencies:
– fakeroot
– fakeroot-libs
– rpm-build

FAKE IT!
LEAVE IT TO
FAKEROOT

Create Designated User for Building
of RPM
● useradd packager (will make a user group with
same name)
● passwd packager
● Do not add designated user to groups:
– root/wheel
– adm/sys/shutdown/sshd, others
– oinstall/dba/oper (if you have Oracle RDBMS on
the same machine for some weird reason)

Create RPM Building Environment
● rpmdev-setuptree
● ~/.rpmmacros
– %_topdir %(echo $HOME)/rpmbuild
● ls ~/rpmbuild/
– BUILD RPMS SOURCES SPECS SRPMS
– mkdir ~/rpmbuild/BUILDROOT
● rpm --eval '%buildroot'
– /home/packager/rpmbuild/BUILDROOT/%{name}-%
{version}-%{release}.i386

Starting the Build
rpmdev-newspec -o ~/rpmbuild/SPECS/counterbean.spec
preamble
%prep
%build
%install
%clean
%files
%changelog
Name: counterbean
Version:
Release: 1%{?dist}
Summary:
Group:
License:
URL:
Source0:
BuildRequires:
Requires:
%description
%prep
%setup -q
%build
%configure
make %{?_smp_mflags}
%install
rm -rf $RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT
%clean
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root,-)
%doc
%changelog

Fix the Preamble
Name: counterbean
Version: 2.0.1
Release: 1%{?dist}
Summary: Keep a keen eye on beans for counting purposes
# one of '/usr/share/doc/rpm-*/GROUPS'
Group: Applications/Productivity
License: ASL 2.0
URL: https://github.com/alshopov/counterbean
Source0: apache-tomcat-8.0.14.tar.gz
# BuildRequires: TODO
# Requires: TODO
# no need to set BuildRoot/٪buildroot/$RPM_BUILD_ROOT
# it is set for us
# BuildRoot:

Fix the Preamble
Name: counterbean
Version: 2.0.1
Release: 1%{?dist}
Summary: Keep a keen Comments eye on beans start for with counting “#”
purposes
# one of '/usr/share/doc/rpm-*/Place GROUPS'
comments on lines
License: ASL 2.0
# Requires: TODO
# it is set for us
# BuildRoot:

Fix the Preamble
Name: counterbean
Version: 2.0.1
Release: 1%{?dist}
# one of '/usr/share/doc/rpm-*/Group: Applications/Productivity
The GROUPS'
Voldemort Character –
License: ASL 2.0
Do not name “%”
# Requires: TODO
# it is set for us
# BuildRoot:

Fix the Preamble
Name: counterbean
Version: 2.0.1
Release: 1%{?dist}
Summary: Keep a keen We eye will fix on these beans later
for counting purposes
# one of '/usr/share/doc/rpm-*/GROUPS'
License: ASL 2.0
# Requires: TODO
# it is set for us
# BuildRoot:

%define prefix /opt/%{name}
%description
A sample Java application for packaging demo purposes.
You can also use it to count:
* beans;
* kogs;
* minions.
%prep
# -q: no output, -c: create directory
%setup -q -c
%install
install -m 0755 -d ${RPM_BUILD_ROOT}%{prefix}
cp -r apache-tomcat-8.0.14/* ${RPM_BUILD_ROOT}%{prefix}
%clean
rm -fr ${RPM_BUILD_ROOT}
%files
%defattr(-,root,root,-)
/opt/counterbean
%doc
%changelog
* Mon Oct 13 2014 Alexander Shopov <ash@kambanaria.org>
- Initial version containing only tomcat.

[root@localhost ashopov]# yum install
~packager/rpmbuild/RPMS/i386/counterbean-2.0.1-1.el6.i386.rpm
Loaded plugins: refresh-packagekit, security
Setting up Install Process
Examining /home/packager/rpmbuild/RPMS/i386/counterbean-2.0.1-1.el6.i386.rpm:
counterbean-2.0.1-1.el6.i386
Marking /home/packager/rpmbuild/RPMS/i386/counterbean-2.0.1-1.el6.i386.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package counterbean.i386 0:2.0.1-1.el6 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
======================================================================================
Package Arch Version Repository Size
======================================================================================
Installing:
counterbean i386 2.0.1-1.el6 /counterbean-2.0.1-1.el6.i386 12 M
Transaction Summary
======================================================================================
Install 1 Package(s)
Total size: 12 M
Installed size: 12 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : counterbean-2.0.1-1.el6.i386 1/1
Verifying : counterbean-2.0.1-1.el6.i386 1/1
Installed:
counterbean.i386 0:2.0.1-1.el6
Complete!

Try To Start
[packager@localhost ~]$ /opt/counterbean/bin/startup.sh
Neither the JAVA_HOME nor the JRE_HOME environment variable is defined
At least one of these environment variable is needed to run this program
[packager@localhost ~]$

We Forgot Dependencies
[root@localhost ~]# yum provides '*/java'
⋮⋮⋮
graphviz-java-2.26.0-7.el6.i686 : Java extension for graphviz
Repo : public_ol6_latest
Matched from:
Filename : /usr/lib/graphviz/java
⋮⋮⋮
1:java-1.6.0-openjdk-1.6.0.0-6.1.13.4.el6_5.i686 : OpenJDK Runtime Environment
Matched from:
Filename : /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/bin/java
⋮⋮⋮
[packager@localhost ~]$ yum provides jre
⋮⋮⋮
1:java-1.6.0-openjdk-1.6.0.0-5.1.13.3.el6_5.i686 : OpenJDK Runtime Environment
Matched from:
Other : jre
⋮⋮⋮
1:java-1.7.0-openjdk-1.7.0.51-2.4.4.1.0.1.el6_5.i686 : OpenJDK Runtime Environment
Matched from:
Other : jre
⋮⋮⋮

Add Dependencies
Release: 2%{?dist}
Requires: jre
%changelog
Add jre as runtime dependency
Initial version containing only tomcat.

[root@localhost ashopov]# yum install
~packager/rpmbuild/RPMS/i386/counterbean-2.0.1-2.el6.i386.rpm
Setting up Install Process
Examining /home/packager/rpmbuild/RPMS/i386/counterbean-2.0.1-2.el6.i386.rpm:
counterbean-2.0.1-2.el6.i386
Marking /home/packager/rpmbuild/RPMS/i386/counterbean-2.0.1-2.el6.i386.rpm as an
update to counterbean-2.0.1-1.el6.i386
Resolving Dependencies
---> Package counterbean.i386 0:2.0.1-1.el6 will be updated
---> Package counterbean.i386 0:2.0.1-2.el6 will be an update
--> Processing Dependency: jre for package: counterbean-2.0.1-2.el6.i386
---> Package java-1.7.0-openjdk.i686 1:1.7.0.65-2.5.1.2.0.1.el6_5 will be installed
--> Finished Dependency Resolution
Dependencies Resolved

Parametrize the SPEC
Name: counterbean
Version: 2.0.1
Release: 3%{?dist}
License: ASL 2.0
Requires: jre
%define tc_ver 8.0.14
Source0: apache-tomcat-%{tc_ver}.tar.gz
%description
…
%define prefix /opt/%{name}
%prep
…
%install
cp -r apache-tomcat-%{tc_ver}/* ${RPM_BUILD_ROOT}%{prefix}
%clean
…
%files
…
%changelog
…

Root Owns Everything – Let's Fix
Release: 4%{?dist}
%files
%defattr(-,%{name},%{name},-)
/opt/%{name}
%changelog
* Tue Oct 14 2014 Alexander Shopov <ash@kambanaria.org>
Make 'counterbean' user own everything
…

But When We Install…
warning: user counterbean does not exist - using root
warning: group counterbean does not exist - using root

We Must First Create The User
Release: 5%{?dist}
Requires(pre): shadow-utils
%pre
getent group %{name} >/dev/null || groupadd -r %{name}
getent passwd %{name} >/dev/null ||
useradd -r -g %{name} -d %{prefix} -s /sbin/nologin
-c "Account to own and run %{name}" %{name}
exit 0
%changelog
* Tue Oct 14 2014 Alexander Shopov <ash@kambanaria.org>
Create user and group for 'counterbean'
…

Create Template
[packager@localhost rpmbuild]$ rpmdev-newinit --help
⋮⋮⋮
The template used is /etc/rpmdevtools/template.init.
[packager@localhost rpmbuild]$ rpmdev-newinit
/rpmbuild/SOURCES/counterbean.init
Skeleton init script has been created to
"/home/packager/rpmbuild/SOURCES/counterbean.init".
# The template used is /etc/rpmdevtools/template.init.

#!/bin/sh
#
# counterbean - <summary>
#
# chkconfig: <default runlevel(s)> <start> <stop>
# description: <description, split multiple lines
# with a backslash>
# http://fedoraproject.org/wiki/FCNewInit/Initscripts
### BEGIN INIT INFO
# Provides:
# Required-Start:
# Required-Stop:
# Should-Start:
# Should-Stop:
# Default-Start:
# Default-Stop:
# Short-Description:
# Description:
### END INIT INFO
# Source function library.
. /etc/rc.d/init.d/functions
exec="/usr/sbin/counterbean"
prog=$(basename $exec)
[ -e /etc/sysconfig/$prog ] &&
. /etc/sysconfig/$prog
lockfile=/var/lock/subsys/$prog
start() {
echo -n $"Starting $prog: "
# if not running, start it up here, usually something like "daemon $exec"
retval=$?
echo
[ $retval -eq 0 ] && touch $lockfile
return $retval
}
Template Contents
stop() {
echo -n $"Stopping $prog: "
# stop it here, often "killproc $prog"
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
stop
start
}
case "$1" in
start|stop|restart)
$1
;;
force-reload)
restart
;;
status)
status $prog
;;
try-restart|condrestart)
if status $prog >/dev/null ; then
restart
fi
;;
reload)
# If config can be reloaded without restarting, implement it here,
# remove the "exit", and add "reload" to the usage message below.
# For example:
# status $prog >/dev/null || exit 7
# killproc $prog -HUP
action $"Service ${0##*/} does not support the reload action: " /bin/false
exit 3
;;
*)
echo $"Usage: $0 {start|stop|status|restart|try-restart|force-reload}"
exit 2
esac

Startup Script
#!/bin/sh
#
# counterbean - count bean, kogs, minions
#
# chkconfig: 345 98 02
# description: small demo RDBMS Java web app
# Source function library.
. /etc/rc.d/init.d/functions
start_cmd="runuser counterbean -g counterbean -s
/bin/sh -c '/opt/counterbean/bin/startup.sh' "
stop_cmd="runuser counterbean -g counterbean -s
/bin/sh -c '/opt/counterbean/bin/shutdown.sh' "
prog=counterbean
lockfile=/var/lock/subsys/$prog
start() {
echo -n $"Starting $prog: "
$start_cmd
retval=$?
echo
[ $retval -eq 0 ] && touch $lockfile
return $retval
}
stop() {
echo -n $"Stopping $prog: "
$stop_cmd
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
stop
start
}
case "$1" in
start|stop|restart)
$1
;;
status)
status $prog
;;
*)
echo $"Usage: $0 {start|stop|restart|status}"
exit 2
esac

Use The Script
Release: 6%{?dist}
Requires: coreutils, jre, chkconfig, initscripts
Requires(pre): shadow-utils
Source1: %{name}.init
%install
install %{SOURCE1} -D ${RPM_BUILD_ROOT}/etc/rc.d/init.d/%{name}
/opt/%{name}
%attr(-,root,root) /etc/rc.d/init.d/%{name}
%changelog
* Wed Oct 15 2014 Alexander Shopov <ash@kambanaria.org>
Proper startup scripts
…

Use The Script
%pre
getent group %{name} >/dev/null || groupadd -r %{name}
getent passwd %{name} >/dev/null ||
useradd -r -g %{name} -d %{prefix} -s /sbin/nologin
-c "Account to own and run %{name}" %{name}
exit 0
%post
/sbin/chkconfig --add %{name}
if [ "$1" = 1 ]; then
/sbin/service %{name} start > /dev/null 2>&1
fi
exit 0
%preun
if [ "$1" = 0 ]; then
/sbin/service %{name} stop > /dev/null 2>&1
fi
exit 0
%postun
if [ "$1" -ge 1 ]; then
/sbin/service %{name} restart > /dev/null 2>&1
fi
exit 0

%pre, %post, %preun,
%postun?
● Scripts that get executed on install, upgrade,
remove;
● There are others:
– %triggerin, %triggerun, %triggerpostun:
ADVANCED – dependencies on packages;
– %pretrans, %posttrans: VERY ADVANCED,
avoid (will fail in chroot, initial install, usually written
in Lua.

Execution of Scripts
install upgrade uninstall
%pre $1 == 1 $1 == 2 Not run
%post $1 == 1 $1 == 2 Not run
%preun Not run $1 == 1 $1 == 0
%postun Not run $1 == 1 $1 == 0
1 2 3 4 5 6
%pre of new
package
package
install
%post of
new
package
%preun of
old package
removal of
old
package
%postun of
old package
$1 == 2 $1 == 2 $1 == 1 $1 == 1

Introducing CounterBean –
Enterprise version

Very Modern App
HTML5
Bootstrap 3.2

Very Modern App
Servlet 3.0
No web.xml
Servlet 3.0
Annotations

Very Modern App
In-Memory Derby
JPA 2.1
With EclipseLink 2.5.x

When Deploying It
● Remove default applications from Tomcat
● Make it the ROOT application
– Expand as ROOT in Tomcat's webapps folder
– Remove context.xml
● Migrate to Oracle RDBMS
– Fix persistence.xml
– Add Oracle's driver

Make Counterbean the Default App
Release: 7%{?dist}
%define cb_ver 1.0-SNAPSHOT
Source1: %{name}-%{cb_ver}.war
%install
# Remove default applications
rm -fr ${RPM_BUILD_ROOT}%{prefix}/webapps
# Ensure ROOT exists and unzip counterbean to it
install -m 0755 -d ${RPM_BUILD_ROOT}%{prefix}/webapps/ROOT
unzip %{SOURCE1} -d ${RPM_BUILD_ROOT}%{prefix}/webapps/ROOT
# Sets the context, we need the default
rm -fr ${RPM_BUILD_ROOT}%{prefix}/webapps/ROOT/META-INF/context.xml

The DBA Prepares the RDBMS
SQL> CREATE USER counterbean IDENTIFIED BY counterbean;
User created.
SQL> GRANT RESOURCE, CONNECT TO counterbean;
Grant succeeded.
SQL> ALTER USER counterbean QUOTA 50M ON USERS;
User altered.

The DBA Prepares the RDBMS
AKA “Mr. Generous DBA”
SQL> CREATE USER counterbean IDENTIFIED BY counterbean;
User created.
SQL> GRANT RESOURCE, CONNECT TO counterbean;
Grant succeeded.
SQL> ALTER USER counterbean QUOTA 50M ON USERS;
User altered.

Use Oracle RDBMS
Release: 8%{?dist}
⋮⋮⋮
Source3: ojdbc7.jar
Source4: persistence.xml
%install
⋮⋮⋮
# Sets the context, we need the default
rm -fr ${RPM_BUILD_ROOT}%{prefix}/webapps/ROOT/META-INF/context.xml
# Replace JDBC driver
rm ${RPM_BUILD_ROOT}%{prefix}/webapps/ROOT/WEB-INF/lib/derby-*.jar
cp %{SOURCE3} ${RPM_BUILD_ROOT}%{prefix}/webapps/ROOT/WEB-INF/lib/
# Replace persistence.xml
cp %{SOURCE4} ${RPM_BUILD_ROOT}%{prefix}
/webapps/ROOT/WEB-INF/classes/META-INF/persistence.xml

Final File Touches
Release: 9%{?dist}
%files
/opt/%{name}
%attr(-,root,root) /etc/rc.d/init.d/%{name}
%doc /opt/%{name}/LICENSE
%doc /opt/%{name}/NOTICE
%doc /opt/%{name}/RELEASE-NOTES
%doc /opt/%{name}/RUNNING.txt
%attr(755,%{name},%{name}) %ghost /opt%{name}/conf/Catalina/
%ghost /opt/%{name}/logs/
%ghost /opt/%{name}/work/
%config /opt/%{name}/webapps/ROOT/WEB-INF/classes/META-INF/persistence.xml

Publishing RPMs As Yum
Repositories
yum install createrepo
mkdir -p /somewhere/{SRPMS,i386,x86_64}
# copy RPMs to the directories
for arch in /somewhere/*; do
pushd ${arch}
createrepo .
popd
done

Using the New Repository
cat /etc/yum.repos.d/example.repo
[example]
name=Example Repository
baseurl=ftp://example.com/somewhere/$basearch
enabled=1

Popular RPM Commands
rpm -ivh counterbean-2.0.1-10.el6.i386
# Install, Verbose, Hashes show progress
rpm -U *.rpm # Upgrade
rpm -F *.rpm # Freshen – install if exists
rpm -qa | grep -i counterbean # Query All
rpm -qpR counterbean-2.0.1-10.el6.i386
# Query what the package Provide and Requires
rpm -ql counterbean # Query List files in package
rpm -e counterbean # Erase
rpm -qf /etc/passwd # Query for package of File
rpm -qi counterbean # infomration
# do the query on a package rather than the RPM DB
rpm -qip counterbean-2.0.1-10.el6.i386 # or -qilp
rpm -Vp # verify package – changes to files
rpm --rebuilddb # rebuild database

Popular YUM Commands
yum install counterbean # install, -y automatically
yum remove counterbean # uninstalls
yum update -y # auto upgrades all
yum search counterbean # find such a package
yum info counterbean # info about a package
yum list # list packages
# available|installed|extras|updates|obsoletes
yum group[list|info|install|update|remove] # groups
yum provides /etc/passwd
yum repolist # enables repositories
yum repolist all # all
yum --[enable|disable]repo=repo_id # enable/disable

Further Resources
● Max RPM Book: http://www.rpm.org/max-rpm/
● GuruLabs RPM Guide:
https://www.gurulabs.com/downloads/GURULABS-RPM-LAB/GUR
ULABS-RPM-GUIDE-v1.0.PDF
● Fedora Project
– How to Create an RPM Package:
https://fedoraproject.org/wiki/How_to_create_an_RPM_package
– Packaging Guidelines:
https://fedoraproject.org/wiki/Packaging:Guidelines
● Clint Savage, How to Make an RPM:
https://www.youtube.com/watch?v=4J_Iksu1fgo
● Jeff's Linux Vids:
https://www.youtube.com/channel/UC96flr-XcwMTeyZOhGPfkLw

Bundling Packages and Deploying Applications with RPM

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Bundling Packages and Deploying Applications with RPM

Similar to Bundling Packages and Deploying Applications with RPM (20)

More from Alexander Shopov

More from Alexander Shopov (10)

Recently uploaded

Recently uploaded (20)

Bundling Packages and Deploying Applications with RPM