Talk on 17/08/2021 for Java London Community

1. Triangle of
Cassandra &
Solr & Kafka
IRINA KAMALOVA
SENIOR SOFTWARE ENGINEER
ALVEO TECHNOLOGY, UK
17/08/2021

2. Motivation
To give a brief introduction to Apache Cassandra and
some things that you will miss from relational database
To give a brief introduction to Apache Solr
To give an example of integrating these products
with Apache Kafka

3. Business Case
We need to:
• Store tweets
• Verify tweets
• Indexing for
complex queries
“The problem of Tweets”- 500 million per day

4. Apache
Cassandra

5. Features
 Peer-to-Peer architecture
 Data durability
 Fault tolerance
 High availability
 Elastic Scalability
 Tunable Consistency for reads/writes
 Column oriented and schema-free

6. You will miss about:
 Cassandra = No Joins
 No subqueries for selecting data
 No queries “like” expression for selecting data
 No commit/rollback features
 Adding new columns is not possible in some cases
*More to read can be find in my article: https://iren-kamalova.medium.com/prevent-yourself-from-the-pitfalls-of-using-cassandra-cbf03c84f22c

7. Cassandra architecture

8. You will miss about:
 Insert with “if exist” condition is extremely expensive
 Update with “if” condition is extremely expensive

9. Paxos protocol

10. Paxos protocol

11. Paxos protocol
4 round trips across the cluster!

12. Well, this is what I
need. But… What if I
need more complex
query on some data?

13. Apache Solr

14. Features:
Full-text indexing and search
REST-like HTTP/XML and JSON APIs
Statistics
Well-documented
Powerful UI

16. Solr Cloud
Peer-to-Peer architecture
Fault tolerance
Central configuration for the entire cluster
Automatic load balancing
Fail-overs for queries
ZooKeeper integration for cluster
coordination and configuration

18. Well, this is what we need.
But… How to integrate it
together?

19. Business Case
We need to:
• Store tweets
• Verify tweets
• Indexing for
complex queries
“The problem of Tweets”- 500 million per day

20. Data flow
Verification

21. Data flow
Verification
? Where to store events if Solr is not responding

22. Apache Kafka

23. Features:
Event-driven
architecture
High throughput
Fault-tolerant
High availability
Data durability
Elastic Scalability

24. Fault-tolerant & High Availability

25. Fault-tolerant & High Availability

26. Data flow
Verification
Indexer

27. Features of the whole system:
Peer-to-Peer architecture for components
Faut-tolerance
Hight Availability
Elastic Scalability
Data durability
Fast writes
Eventual consistence
Powerful search

28. Questions?
Follow me:
Github
https://github.com/irenkamalova
Gitpages:
https://irenkamalova.github.io/
Twitter: https://twitter.com/IrenKamalova
Medium:
https://iren-kamalova.medium.com/
LinkedIn
https://www.linkedin.com/in/irina-kamalova/

29. Thank You !