Presented at #PHPLX 11 July 2013
When you need to do some heavy processing how do you scale you application?
You can use Redis and ZeroMQ to leverage the heavy work for you!
With this presentation we will know more about this two technologies and how they can be used to help solve problems with the performance and scalability of your application.
2. @rjsmelo 2
RICARDO MELO
● CTO @ DRI
● PHP, Mysql, Linux and lots of other
OSS
● ZCE, RHCE, LPI 3, ITIL, etc
● +10 years building (and breaking)
things
3. @rjsmelo 3
About
● 14 Year old academic spin-off
● Pragmatic OSS Orientation
● PHP, Mysql, SugarCRM, Drupal,
JavaScript, Linux, etc.
● Crafters, Integrators
● Always looking for software developers
– Yes, right now!
4. 1999 - 2013 DRI. Some Rights Reserved. 4
Outline
● Redis
● ZeroMQ
● Use Cases
● Conclusions
5. 1999 - 2013 DRI. Some Rights Reserved. 5
Redis
“Redis is an open source, BSD licensed,
advanced key-value store. It is often
referred to as a data structure server
since keys can contain strings, hashes,
lists, sets and sorted sets.”
source: http://redis.io
12. 1999 - 2013 DRI. Some Rights Reserved. 12
ZeroMQ
“ØMQ is a high-performance asynchronous
messaging library aimed at use in scalable
distributed or concurrent applications. It provides a
message queue, but unlike message-oriented
middleware, a ØMQ system can run without a
dedicated message broker. With bindings for 30+
languages”
source: https://en.wikipedia.org/wiki/%C3%98MQ
13. 1999 - 2013 DRI. Some Rights Reserved. 13
PHP Module - zmq
#
# ZMQ instalation - http://www.zeromq.org/intro:get-the-software
#
tar xzvf zeromq-3.2.2.tar.gz
cd zeromq-3.2.2
./configure
make
make install
echo "/usr/local/lib" > /etc/ld.so.conf.d/usr_local_lib.conf
ldconfig
#
# ZMQ PHP Binding Instalation
#
pear channel-discover pear.zero.mq
pecl install pear.zero.mq/zmq-beta
echo "extension=zmq.so" > /etc/php.d/zmq.ini
14. 1999 - 2013 DRI. Some Rights Reserved. 14
Socket Types
● Threads in a process (inproc://)
● Processes in a box (ipc://)
● Processes over the network (tcp://)
● Multicast group (pgm://)
15. 1999 - 2013 DRI. Some Rights Reserved. 15
Pattern: Request - Reply
23. 1999 - 2013 DRI. Some Rights Reserved. 23
Pattern: Pipeline
24. 1999 - 2013 DRI. Some Rights Reserved. 24
Use Case: Service Cluster
Reference: http://zguide.zeromq.org/page:all#Service-Oriented-Reliable-Queuing-Majordomo-Pattern
25. 1999 - 2013 DRI. Some Rights Reserved. 25
Use Case: “Unix Style” Application
● Lots of simple, “focused” programs
– Or in different servers, different languages, etc
● All components collaborate to get the
job done
– cat file | sort | uniq -c
● Glue everything with ZeroMQ
26. 1999 - 2013 DRI. Some Rights Reserved. 26
Use Case: Cache
● We need to cache some values for
speed
● Use SET / GET on Redis
● I could use memcache, but this is a
Redis Talk :-)
● They have similar performance
– http://redis.io/topics/benchmarks
27. 1999 - 2013 DRI. Some Rights Reserved. 27
Use Case: Buffer
● Use Redis as a FIFO (or LIFO)
● Hidden Goods
– Decoupling
– Flow control
– rpoplpush
28. 1999 - 2013 DRI. Some Rights Reserved. 28
Use Case: Background Tasks
● Your application needs to do some
heavy work
– Send Email
– Image Resizing
– Mega Huge map-reduce query to your pentabyte cluster :-)
● You don't want to keep your user
waiting
● Send things to background
29. 1999 - 2013 DRI. Some Rights Reserved. 29
Use Case: Background Tasks
● Use Redis as your Job Queue
– Your application should send job to be run and parameters to the
queue
● Use workers to POP jobs from the
queue and do the heavy work.
● Don't reinvent the well
– Look for php-resqueue
30. 1999 - 2013 DRI. Some Rights Reserved. 30
Conclusions
● Both ZeroMQ and Redis are extremely
fast
● ZeroMQ is great to “glue things” as well
as adding flexibility to scale dynamical
● Redis is great as a queue and allows
you to cope with your load.