Codemotion Berlin 2017

1. Evolution of message brokers.
From helpers to cores.
Artem Demchenkov

2. 2
Artem Demchenkov
● CTO at Billie
● ardemchenkov@gmail.com
● https://www.linkedin.com/in/artem-
demchenkov-76b69934/

3. Billie
We are an innovative financial platform,
specialized on factoring.
Some numbers:
● Start: 01.12.2016
● Closed beta: 01.05.2017
● Public launch: 01.06.2017
● Team: 18
● Engineering crew: 8

4. is an intermediary program module that translates a message from the
formal messaging protocol of the sender to the formal messaging
protocol of the receiver.
Boring, but necessary theoretical information
4
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Message-oriented middleware
(MOM)
is software or hardware infrastructure supporting sending and
receiving messages between distributed systems.
Message broker

5. A Universe of Message Brokers
5
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
And more...

6. PAST
6
PRESENT FUTURE
?

7. Message broker is a helper on a side of an infrastructure
7
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Website RabbitMQ Consumer
Past

8. 8
PAST PRESENT FUTURE
?

9. Message broker is a core element of an infrastructure.
9
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Message
broker
Internal tool
Microservice
Microservice
Present

10. Real-life example. Google Cloud Platform.
Present
10
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017

11. Use case 1. Synchronization of data between parts of a system
11
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Website
Google Pub/Sub
Salesforce Partner portal

12. Distribution
Values
12
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Allows getting rid of a central data storage.
Each application can use an own one.
Real-time task forwarding between different
systems, responsible people or departments.
Improves data consistency.
Instantness
Consistency

13. Use case 2. Safe and secure triggering of microservices
13
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Backend Google Pub/Sub Microservice

14. Values
14
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Safety You won’t loose a message.
Each message can be secured.Security

15. Use case 3. Persistent data storage
15
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Backend Apache Kafka
AWS Cloud Storage
Google Cloud Storage
Microsoft Azure Blob Storage

16. Values
16
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Persistence You have a full set of historical data.
You have a right to choose how to store data,
where and in which format.
Cloud storage services provide a permission
management mechanism to secure data.
Flexibility
Security

17. Use case 4. Log aggregation. Commit log
17
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Backend Apache Kafka
Instance 1 Storage 1
Instance 2 Storage 2

18. Values
18
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Replicability Message broker log allows replicating any
state of a system at any moment in time
You have an full set of historical data.
Message broker log can be used by many
different clients.
Persistence
Multifunctionality

19. Use case 5. Log aggregation. Fulltext search
19
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Backend Apache Kafka Google Cloud Storage Elastic

20. Use case 5. Log aggregation. Fulltext search
20
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Backend Apache Kafka Apache Solr Cloudera Hue

21. Values
21
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Searchability Any information in the past could be found.
It’s relatively easy to detect wrong messages
and define when and how it has been
consumed.
Discoverability

22. Use case 6. Delayed messaging
22
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Website RabbitMQ External data provider

23. Values
23
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Effectiveness Allows consuming a message at the most
effective time.
Saves traffic, memory usage and disk space.Thriftiness

24. Use case 7. Multi-cloud application
24
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Google Cloud Storage Google Pub/Sub AWS Cloud Storage

25. Values
25
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Distribution Allows synchronising data across
applications running on other clouds.
Allows backing up data across applications
running on other clouds.
Provides more flexibility in terms of choosing
right services for an architecture.
Security
Variety

26. 26
PAST PRESENT FUTURE
?

27. Future
27
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Let’s look again at the current situation.

28. 28
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Google Pub/Sub
RabbitMQ Apache Kafka
My prediction is the following: soon projects will use more than one message broker,
communicating to each other.
Future

29. 29
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Future
Let’s see in 3-4 years, whether I was right

30. 30
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
Future
Conclusion

31. 31
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
“Because it’s there”
George Mallory - English mountaineer, answering a question why did he want to climb Everest. Made 3
attempts and died during the 3rd one in 1924. It is still a mystery whether he had reached the top of it or
not.

32. 32
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
“If you don’t know that something exists, you, probably, don’t need
it”
Tenzing Norgay - Nepali Sherpa mountaineer. One of the first two people reached the top of Everest in
1953.

33. 33
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
“No matter what you’ve seen in that presentation, think twice
whether you product needs it”
Artem Demchenkov - CTO at Billie. I hope, successfully presented these slides on Codemotion Berlin 2017.

34. Thank you
● ardemchenkov@gmail.com
● https://www.linkedin.com/in/artem-
demchenkov-76b69934/

35. We have just started, but there will be interesting:
Check out our “Billie Engineering Corner” on Medium
35
Artem Demchenkov. Evolution of message brokers: from helpers to cores. Codemotion Berlin 2017
https://medium.com/billie-finanzratgeber/tagged/engineering