More Related Content Similar to Theres a rabbit on my symfony (20) More from Alvaro Videla (20) Theres a rabbit on my symfony1. There's a Rabbit
on my Symfony
RabbitMQ and Symfony2 Integration
Álvaro Videla
Symfony Live 2011
Thursday, March 3, 2011
3. About Me
• Development Manager at TheNetCircle.com
• Blog: http://videlalvaro.github.com/
• Twitter: @old_sound
Thursday, March 3, 2011
4. About Me
• Developer at Liip
• Blog: http://videlalvaro.github.com/
• Twitter: @old_sound
Thursday, March 3, 2011
5. About Me
Co-authoring
RabbitMQ in Action
http://bit.ly/rabbitmq
Thursday, March 3, 2011
6. Why Do I need
RabbitMQ?
Thursday, March 3, 2011
8. Why Do I need
Messaging?
Thursday, March 3, 2011
11. I don’t want to wait
till your app resizes
my image!
Thursday, March 3, 2011
13. Can we also notify the
user friends when she
uploads a new image?
Thursday, March 3, 2011
14. Can we also notify the
user friends when she
uploads a new image?
I forgot to mention we need it for tomorrow…
Thursday, March 3, 2011
17. Dumb! You’re delivering
full size images!
The bandwidth bill has
tripled!
We need this fixed for yesterday!
Thursday, March 3, 2011
19. I need to call your PHP
stuff but from Python
Thursday, March 3, 2011
20. I need to call your PHP
stuff but from Python
And also Java starting next week
Thursday, March 3, 2011
26. AMQP
• Advanced Message Queuing Protocol
• Suits Interoperability
• Completely Open Protocol
• Binary Protocol
• AMQP Model
• AMQP Wire Format
Thursday, March 3, 2011
27. AMQP Model
• Exchanges
• Message Queues
• Bindings
• Rules for binding them
Thursday, March 3, 2011
29. Message Flow
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/chap-Messaging_Tutorial-Initial_Concepts.html
Thursday, March 3, 2011
36. Usage Scenarios
• Batch Processing
• Image Uploading
Thursday, March 3, 2011
37. Usage Scenarios
• Batch Processing
• Image Uploading
• Distributed Logging
Thursday, March 3, 2011
38. Scenario
Batch Processing
Thursday, March 3, 2011
41. Requirements
• Generate XML
• Distribution Over a Cluster
Thursday, March 3, 2011
42. Requirements
• Generate XML
• Distribution Over a Cluster
• Elasticity - Add/Remove new workers
Thursday, March 3, 2011
43. Requirements
• Generate XML
• Distribution Over a Cluster
• Elasticity - Add/Remove new workers
• No Code Changes
Thursday, March 3, 2011
45. Publisher Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$msg = new AMQPMessage($video_info,
array('content_type' => 'text/plain',
'delivery_mode' => 2));
$channel->basic_publish($msg, 'video-desc-ex');
$channel->close();
$conn->close();
Thursday, March 3, 2011
46. Publisher Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$msg = new AMQPMessage($video_info,
array('content_type' => 'text/plain',
'delivery_mode' => 2));
$channel->basic_publish($msg, 'video-desc-ex');
$channel->close();
$conn->close();
Thursday, March 3, 2011
47. Publisher Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$msg = new AMQPMessage($video_info,
array('content_type' => 'text/plain',
'delivery_mode' => 2));
$channel->basic_publish($msg, 'video-desc-ex');
$channel->close();
$conn->close();
Thursday, March 3, 2011
48. Publisher Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$msg = new AMQPMessage($video_info,
array('content_type' => 'text/plain',
'delivery_mode' => 2));
$channel->basic_publish($msg, 'video-desc-ex');
$channel->close();
$conn->close();
Thursday, March 3, 2011
49. Publisher Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$msg = new AMQPMessage($video_info,
array('content_type' => 'text/plain',
'delivery_mode' => 2));
$channel->basic_publish($msg, 'video-desc-ex');
$channel->close();
$conn->close();
Thursday, March 3, 2011
50. Publisher Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$msg = new AMQPMessage($video_info,
array('content_type' => 'text/plain',
'delivery_mode' => 2));
$channel->basic_publish($msg, 'video-desc-ex');
$channel->close();
$conn->close();
Thursday, March 3, 2011
51. Consumer Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$channel->queue_declare('video-desc-queue', false, true,
false, false);
$channel->queue_bind('video-desc-queue', 'video-desc-ex');
$channel->basic_consume('video-desc-queue', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
52. Consumer Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$channel->queue_declare('video-desc-queue', false, true,
false, false);
$channel->queue_bind('video-desc-queue', 'video-desc-ex');
$channel->basic_consume('video-desc-queue', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
53. Consumer Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$channel->queue_declare('video-desc-queue', false, true,
false, false);
$channel->queue_bind('video-desc-queue', 'video-desc-ex');
$channel->basic_consume('video-desc-queue', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
54. Consumer Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$channel->queue_declare('video-desc-queue', false, true,
false, false);
$channel->queue_bind('video-desc-queue', 'video-desc-ex');
$channel->basic_consume('video-desc-queue', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
55. Consumer Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$channel->queue_declare('video-desc-queue', false, true,
false, false);
$channel->queue_bind('video-desc-queue', 'video-desc-ex');
$channel->basic_consume('video-desc-queue', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
56. Consumer Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$channel->queue_declare('video-desc-queue', false, true,
false, false);
$channel->queue_bind('video-desc-queue', 'video-desc-ex');
$channel->basic_consume('video-desc-queue', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
57. Consumer Code
$conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST);
$channel = $conn->channel();
$channel->exchange_declare('video-desc-ex', 'direct', false,
true, false);
$channel->queue_declare('video-desc-queue', false, true,
false, false);
$channel->queue_bind('video-desc-queue', 'video-desc-ex');
$channel->basic_consume('video-desc-queue', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
58. Scenario
Upload Pictures
Thursday, March 3, 2011
61. Requirements
• Upload Picture
• Reward User
Thursday, March 3, 2011
62. Requirements
• Upload Picture
• Reward User
• Notify User Friends
Thursday, March 3, 2011
63. Requirements
• Upload Picture
• Reward User
• Notify User Friends
• Resize Picture
Thursday, March 3, 2011
64. Requirements
• Upload Picture
• Reward User
• Notify User Friends
• Resize Picture
• No Code Changes
Thursday, March 3, 2011
68. Publisher Code
$channel->exchange_declare('upload-pictures', 'fanout', false,
true, false);
$metadata = json_encode(array(
'image_id' => $image_id,
'user_id' => $user_id,
‘image_path' => $image_path));
$msg = new AMQPMessage($metadata, array('content_type' =>
'application/json', 'delivery_mode' => 2));
$channel->basic_publish($msg, 'upload-pictures');
Thursday, March 3, 2011
69. Publisher Code
$channel->exchange_declare('upload-pictures', 'fanout', false,
true, false);
$metadata = json_encode(array(
'image_id' => $image_id,
'user_id' => $user_id,
‘image_path' => $image_path));
$msg = new AMQPMessage($metadata, array('content_type' =>
'application/json', 'delivery_mode' => 2));
$channel->basic_publish($msg, 'upload-pictures');
Thursday, March 3, 2011
70. Publisher Code
$channel->exchange_declare('upload-pictures', 'fanout', false,
true, false);
$metadata = json_encode(array(
'image_id' => $image_id,
'user_id' => $user_id,
‘image_path' => $image_path));
$msg = new AMQPMessage($metadata, array('content_type' =>
'application/json', 'delivery_mode' => 2));
$channel->basic_publish($msg, 'upload-pictures');
Thursday, March 3, 2011
71. Publisher Code
$channel->exchange_declare('upload-pictures', 'fanout', false,
true, false);
$metadata = json_encode(array(
'image_id' => $image_id,
'user_id' => $user_id,
‘image_path' => $image_path));
$msg = new AMQPMessage($metadata, array('content_type' =>
'application/json', 'delivery_mode' => 2));
$channel->basic_publish($msg, 'upload-pictures');
Thursday, March 3, 2011
72. Publisher Code
$channel->exchange_declare('upload-pictures', 'fanout', false,
true, false);
$metadata = json_encode(array(
'image_id' => $image_id,
'user_id' => $user_id,
‘image_path' => $image_path));
$msg = new AMQPMessage($metadata, array('content_type' =>
'application/json', 'delivery_mode' => 2));
$channel->basic_publish($msg, 'upload-pictures');
Thursday, March 3, 2011
73. Consumer Code
$channel->exchange_declare('upload-pictures', 'fanout',
false, true, false);
$channel->queue_declare('resize-picture', false, true,
false, false);
$channel->queue_bind('resize-picture', 'upload-pictures');
$channel->basic_consume('resize-picture', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
74. Consumer Code
$channel->exchange_declare('upload-pictures', 'fanout',
false, true, false);
$channel->queue_declare('resize-picture', false, true,
false, false);
$channel->queue_bind('resize-picture', 'upload-pictures');
$channel->basic_consume('resize-picture', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
75. Consumer Code
$channel->exchange_declare('upload-pictures', 'fanout',
false, true, false);
$channel->queue_declare('resize-picture', false, true,
false, false);
$channel->queue_bind('resize-picture', 'upload-pictures');
$channel->basic_consume('resize-picture', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
76. Consumer Code
$channel->exchange_declare('upload-pictures', 'fanout',
false, true, false);
$channel->queue_declare('resize-picture', false, true,
false, false);
$channel->queue_bind('resize-picture', 'upload-pictures');
$channel->basic_consume('resize-picture', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
77. Consumer Code
$channel->exchange_declare('upload-pictures', 'fanout',
false, true, false);
$channel->queue_declare('resize-picture', false, true,
false, false);
$channel->queue_bind('resize-picture', 'upload-pictures');
$channel->basic_consume('resize-picture', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
78. Consumer Code
$channel->exchange_declare('upload-pictures', 'fanout',
false, true, false);
$channel->queue_declare('resize-picture', false, true,
false, false);
$channel->queue_bind('resize-picture', 'upload-pictures');
$channel->basic_consume('resize-picture', $consumer_tag,
false, false, false, false, $consumer);
while(count($channel->callbacks)) {
$channel->wait();
}
Thursday, March 3, 2011
79. Consumer Code
$consumer = function($msg){
$meta = json_decode($msg->body, true);
resize_picture($meta['image_id'], $meta['image_path']);
$msg->delivery_info['channel']->
basic_ack($msg->delivery_info['delivery_tag']);
};
Thursday, March 3, 2011
80. Consumer Code
$consumer = function($msg){
$meta = json_decode($msg->body, true);
resize_picture($meta['image_id'], $meta['image_path']);
$msg->delivery_info['channel']->
basic_ack($msg->delivery_info['delivery_tag']);
};
Thursday, March 3, 2011
81. Consumer Code
$consumer = function($msg){
$meta = json_decode($msg->body, true);
resize_picture($meta['image_id'], $meta['image_path']);
$msg->delivery_info['channel']->
basic_ack($msg->delivery_info['delivery_tag']);
};
Thursday, March 3, 2011
82. Consumer Code
$consumer = function($msg){
$meta = json_decode($msg->body, true);
resize_picture($meta['image_id'], $meta['image_path']);
$msg->delivery_info['channel']->
basic_ack($msg->delivery_info['delivery_tag']);
};
Thursday, March 3, 2011
83. Consumer Code
$consumer = function($msg){
$meta = json_decode($msg->body, true);
resize_picture($meta['image_id'], $meta['image_path']);
$msg->delivery_info['channel']->
basic_ack($msg->delivery_info['delivery_tag']);
};
Thursday, March 3, 2011
84. Scenario
Distributed Logging
Thursday, March 3, 2011
86. Requirements
• Several Web Servers
Thursday, March 3, 2011
87. Requirements
• Several Web Servers
• Logic Separated by Module/Action
Thursday, March 3, 2011
88. Requirements
• Several Web Servers
• Logic Separated by Module/Action
• Several Log Levels:
Thursday, March 3, 2011
89. Requirements
• Several Web Servers
• Logic Separated by Module/Action
• Several Log Levels:
• Info, Warning, Error
Thursday, March 3, 2011
90. Requirements
• Several Web Servers
• Logic Separated by Module/Action
• Several Log Levels:
• Info, Warning, Error
• Add/Remove log listeners at will
Thursday, March 3, 2011
96. Publisher Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$msg = new AMQPMessage('some log message',
array('content_type' => 'text/plain'));
$channel->basic_publish($msg, 'logs',
server1.user.profile.info');
Thursday, March 3, 2011
97. Publisher Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$msg = new AMQPMessage('some log message',
array('content_type' => 'text/plain'));
$channel->basic_publish($msg, 'logs',
server1.user.profile.info');
Thursday, March 3, 2011
98. Publisher Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$msg = new AMQPMessage('some log message',
array('content_type' => 'text/plain'));
$channel->basic_publish($msg, 'logs',
server1.user.profile.info');
Thursday, March 3, 2011
99. Publisher Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$msg = new AMQPMessage('some log message',
array('content_type' => 'text/plain'));
$channel->basic_publish($msg, 'logs',
server1.user.profile.info');
Thursday, March 3, 2011
100. Consumer Code
Get messages sent by host:
server1
Thursday, March 3, 2011
101. Consumer Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$channel->queue_declare('server1-logs', false, true,
false, false);
$channel->queue_bind('server1-logs', 'logs', 'server1.#');
Thursday, March 3, 2011
102. Consumer Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$channel->queue_declare('server1-logs', false, true,
false, false);
$channel->queue_bind('server1-logs', 'logs', 'server1.#');
Thursday, March 3, 2011
103. Consumer Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$channel->queue_declare('server1-logs', false, true,
false, false);
$channel->queue_bind('server1-logs', 'logs', 'server1.#');
Thursday, March 3, 2011
104. Consumer Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$channel->queue_declare('server1-logs', false, true,
false, false);
$channel->queue_bind('server1-logs', 'logs', 'server1.#');
Thursday, March 3, 2011
105. Consumer Code
Get all error messages
Thursday, March 3, 2011
106. Consumer Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$channel->queue_declare('error-logs', false, true,
false, false);
$channel->queue_bind('error-logs', 'logs', '#.error');
Thursday, March 3, 2011
107. Consumer Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$channel->queue_declare('error-logs', false, true,
false, false);
$channel->queue_bind('error-logs', 'logs', '#.error');
Thursday, March 3, 2011
108. Consumer Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$channel->queue_declare('error-logs', false, true,
false, false);
$channel->queue_bind('error-logs', 'logs', '#.error');
Thursday, March 3, 2011
109. Consumer Code
$channel->exchange_declare('logs', 'topic', false,
true, false);
$channel->queue_declare('error-logs', false, true,
false, false);
$channel->queue_bind('error-logs', 'logs', '#.error');
Thursday, March 3, 2011
113. RabbitMQ Bundle
Publish a Message:
$msg = array('user_id' => 1235,
'image_path' => '/path/to/new/
pic.png');
$this->get('rabbitmq.upload_picture_producer')->
publish(serialize($msg));
Thursday, March 3, 2011
114. RabbitMQ Bundle
Consume a Message:
$ ./app/console_dev rabbitmq:consumer -m 1 upload_picture
Thursday, March 3, 2011
115. RabbitMQ Bundle
Consume 50 Messages:
$ ./app/console_dev rabbitmq:consumer -m 50 upload_picture
Thursday, March 3, 2011
116. RabbitMQ Bundle
Consume infinite* Messages:
$ ./app/console_dev rabbitmq:consumer -m -1 upload_picture
*messages may not be infinite
Thursday, March 3, 2011
117. RabbitMQ Bundle
Configuration:
# app/config/config.yml
rabbitmq.config:
connections:
default:
host: 'localhost'
port: 5672
user: 'guest'
password: 'guest'
vhost: '/'
Thursday, March 3, 2011
118. RabbitMQ Bundle
Configuration:
# app/config/config.yml
rabbitmq.config:
connections:
...
producers:
upload_picture:
connection: default
exchange_options: {name: 'upload-picture',
type: direct}
Thursday, March 3, 2011
119. RabbitMQ Bundle
Configuration:
# app/config/config.yml
rabbitmq.config:
connections:
...
consumers:
upload_picture:
connection: default
exchange_options: {name: 'upload-picture',
type: direct}
queue_options: {name: 'upload-picture'}
callback: upload_picture_service
Thursday, March 3, 2011
120. RabbitMQ Bundle
Using a Producer:
public function indexAction($name)
{
...
$msg = array('user_id' => 1235,
'image_path' => '/path/to/new/pic.png');
$this->get('rabbitmq.upload_picture_producer')->
publish(serialize($msg));
...
}
Thursday, March 3, 2011
121. RabbitMQ Bundle
Consumer Class:
class UploadPictureConsumer extends ContainerAware
implements ConsumerInterface
{
public function execute($msg)
{
// Process picture upload.
// $msg will be what was published
// from the Controller.
}
}
Thursday, March 3, 2011
123. RabbitMQ Bundle
Recap:
• Add an entry for the consumer/producer in
the configuration.
Thursday, March 3, 2011
124. RabbitMQ Bundle
Recap:
• Add an entry for the consumer/producer in
the configuration.
• Implement your Callback.
Thursday, March 3, 2011
125. RabbitMQ Bundle
Recap:
• Add an entry for the consumer/producer in
the configuration.
• Implement your Callback.
• Start the consumer from the CLI.
Thursday, March 3, 2011
126. RabbitMQ Bundle
Recap:
• Add an entry for the consumer/producer in
the configuration.
• Implement your Callback.
• Start the consumer from the CLI.
• Add code to publish messages.
Thursday, March 3, 2011
128. RabbitMQ Bundle
There’s more:
• RPC clients and servers
Thursday, March 3, 2011
129. RabbitMQ Bundle
There’s more:
• RPC clients and servers
• Anonymous consumers
Thursday, March 3, 2011
130. RabbitMQ Bundle
There’s more:
• RPC clients and servers
• Anonymous consumers
• Parallel RPC
Thursday, March 3, 2011
131. RabbitMQ Bundle
Fork it at
http://github.com/videlalvaro/RabbitMqBundle
Thursday, March 3, 2011
133. RabbitMQ
• Enterprise Messaging System
• Open Source MPL
• Written in Erlang/OTP
• Commercial Support
Thursday, March 3, 2011
134. Features
• Reliable and High Scalable
• Easy To install
• Easy To Cluster
• Runs on: Windows, Solaris, Linux, OSX
• AMQP 0.8 - 0.9.1
Thursday, March 3, 2011
135. Client Libraries
• Java
• .NET/C#
• Erlang
• Ruby, Python, PHP, Perl, AS3, Lisp, Scala,
Clojure, Haskell
Thursday, March 3, 2011
136. Docs/Support
• http://www.rabbitmq.com/documentation.html
• http://dev.rabbitmq.com/wiki/
• #rabbitmq at irc.freenode.net
• http://www.rabbitmq.com/email-archive.html
Thursday, March 3, 2011
139. Conclusion
• Flexibility
Thursday, March 3, 2011
140. Conclusion
• Flexibility
• Scalability
Thursday, March 3, 2011
141. Conclusion
• Flexibility
• Scalability
• Interoperability
Thursday, March 3, 2011
142. Conclusion
• Flexibility
• Scalability
• Interoperability
• Reduce Ops
Thursday, March 3, 2011
144. Thanks!
Álvaro Videla
http://twitter.com/old_sound
http://github.com/videlalvaro
http://github.com/tnc
http://www.slideshare.net/old_sound
Thursday, March 3, 2011