Building a large scale, cloud hosted, multi device product that changes the way students are taught in high schools, turns out to be not entirely trivial. We have been building PulseOn for the last two years. We faced many technical challenges, and have tried many different technologies. Some ideas turned out to be extremely helpful, some other ideas that looked promising turned out to be bad practices. In this talk we will discuss what we have learned about building modular, scalable web applications. We will go into different areas of the technology stack, from storage and (nosql) databases to a modular RESTful backend and a multi device HTML5 based frontend. We will also go into the cloud, and discuss topics such as auto scaling and failover. We will discuss what works and what doesn’t, and hopefully help other to make the right decisions. Some topics that we will touch upon: - Modular architecture with OSGi - Failing at linked data and semantic databases - Using MongoDB from OSGi - Release process and semantic versioning - Continuous deployment - JavaScript frameworks Bios: Paul Bakker Paul is a software architect for Luminis Technologies and the author of “Building Modular Cloud Apps With OSGi”. He believes that modularity and the cloud are the two main challenges we have to deal with to bring technology to the next level, and is working on making this possible for mainstream software development. Today he is working on educational software focussed on personalised learning for high school students in the Netherlands. Paul is an active contributor on open source projects such as Amdatu, Apache ACE and BndTools. He has a background as a trainer on Java related technology and is a regular speaker on conferences such as JavaOne, Devoxx and JFokus. Jago de Vreede Jago is a software engineer at Luminis Technologies, as a software engineer he has seen a broad-spectrum of projects and he has been working on a large OSGi project for the last year. His work is not exclusive to java development but also does front-end development, and the integration between these. Performance tuning and optimizations are also part of his work.

1. Lessons learned
from a large scale OSGi web app
Jago de Vreede
Paul Bakker

2. Paul Bakker
@pbakker

3. Jago deVreede

4. Agenda
!
High level architecture
Modular architecture
Modularity with OSGi
Cloud deployments with Apache
ACE
Frontend frameworks

5. Ask whenever
you want!

6. The case
PulseOn
!
Educational system
focussed on personalized
learning
!
Used in high schools in
the Netherlands
!
Expand to other countries
in the near future

7. Cloud provisioning
http://ace.apache.org/
bndtools
Eclipse OSGi plugin
http://bndtools.org/
Eclipse
http://eclipse.org/ Source control
Tools
Stash, Jira, Bamboo
http://atlassian.com/
WebStorm
http://jetbrains.com/
webstorm/

8. HTML 5 + JavaScript
RESTful services
OSGi services
Mongo
Apache Felix
S3
High level architecture
Requirements:
!
Agile and modular
Modern web app
UI mostly offloaded to
clients or devices
Integration via REST
API
Horizontally scalable
A
m
d
a
t
u

9. Auth
Blob stores
MongoDB
Multi-tenancy
OpenSocial
Search
Remote Services
REST
Template
Web
...
Components

10. Load
Load Balancer
PulseOn
PulseOn
PulseOn
School B
Mongo
Mongo
Mongo
Deployment
Load
Load Balancer
PulseOn
PulseOn
PulseOn
School A
Mongo
Mongo
Mongo
Load
Load Balancer
DAMS
DAMS
DAMS
Content backend
Mongo
Mongo
Mongo

11. Profiles Rest
Profiles API
Profiles
Service
MongoDB
Progress Rest
Progress API
Progress
Service
MongoDB
Curriculum
API
Curriculum
Service
MongoDB
... Rest
... API
...
Service
services
all the way down

12. A service should only
do onesingle thing

13. Services are the
(reusable) building
blocks of our
application

14. Benefits of a services based
architecture
!
Small services are easy to
maintain
Small services are
composable
Services are easily testable

15. Packaging services
Services are packaged in small
bundles
Related bundles may be generated
from a single Bndtools project

16. Data Storage

17. Semantic data store
(triple store)
MySQL
(with JPA)
MongoDB
NowBeginning

18. Relational databases
in a modular system
A part of data should be
owned by 1 service
How to set boundaries in a relational world!?

19. Relational databases
in the modern web
!
ORM is REALLY difficult to get right
Relational databases don’t scale well

20. MongoDB: our database of choice
!
Object Mapping trivial
Easy to scale
Powerful aggregation/map-reduce
framework

21. Code example: MongoDB with Amdatu
Amdatu
Mongo Service
Setup Object
Mapper
Execute query

22. Release & Deployment

23. Binaries
Release to release repo
Release to
release OBR

24. JavaScript minification
closure compiler
deterministic? => fixed March 3, 2014
Baseline OBR Release OBR
CI Build Server

25. Deployment
Software distribution framework
Manages the installation and upgrade
of bundles, configuration, etc. to
heterogenous targets

26. Load
Load Balancer
PulseOn
PulseOn
PulseOn
School B
Mongo
Mongo
Mongo
Deployment
Load
Load Balancer
PulseOn
PulseOn
PulseOn
School A
Mongo
Mongo
Mongo
Load
Load Balancer
DAMS
DAMS
DAMS
Content backend
Mongo
Mongo
Mongo

27. Deployment
Load
Load Balancer
PulseOn
PulseOn
PulseOn
School
Load
Load Balancer
DAMS
DAMS
DAMS
Content backend

28. Deployment
Load
Load Balancer
Target
Target
Target
School
Load
Load Balancer
Target
Target
Target
Content backend
Release OBR
Apache ACE
Baseline OBR
CI Build Server

29. Continuous Deployment
Load
Load Balancer
Target
School
Load
Load Balancer
Target
Content backend
Snapshot OBR
Apache ACE
CI Build Server
Like to know more?
Continuous Automated Deployment with Apache ACE
youtube.com/watch?v=4S_zvgG_MLw

31. Unit testing OSGi code
is trivial
But what if we want
more?

32. Integration testing turns
out to be trivial as well!
Automated
dependency
injection of
service
Setup
Config Admin
Junit style
asserts

33. Frontend frameworks

34. jQuery Pros
Can do a
lot with not
much code
Lot of
helpful
plugins
available
Easy to get
started

35. jQuery
Hard to
maintain
Cons
Hard to test

36. Backbone.js Pros
Designed
towards
consuming
REST data
Well
structured

37. Backbone.js Cons
Hard to
maintain
A LOT OF
CODE!!
Testable

38. AngularJS

39. AngularJS Pros
Designed
towards
consuming
REST data
Well
structured
No
guessing
in htmlTestable

40. TypeScript Pros
Well
structured
Code
completion
compile
time checks

41. TypeScript Cons
Well
structured
Slow-ish
Need to
have a
definition
to work

42. !
A modular architecture gives us:!
Maintainability
Extensibility
Freedom to change
Wrap up

43. But what if I want
Spring
EJB
Hibernate?
Just don’t… You really don’t need to.

44. Cloud provisioning
http://ace.apache.org/
Eclipse OSGi plugin
http://bndtools.org/
That’s us
http://luminis-
technologies.com
Cloud OSGi services
http://www.amdatu.org/
Amdatu
Paul Bakker
paul.bakker@luminis.eu
@pbakker
Jago de Vreede
jago.devreede@luminis.eu

45. Learn more?
!
Friday 10.20 - 12.50 or
14.45 - 17.15
!
Tutorial “Developing modular cloud
applications with OSGi”