Real Time Recommendations Using WebSockets and Redis - Ninad Divadkar, Inuit
•Download as PPTX, PDF•
9 likes•9,951 views
WebSockets connect the browser to your app server. But what if the processing happens on some other server? In that case you need to connect the worker process to the app process via a messaging system. After experimenting with RabbitMQ, we settled on Redis as a great pub sub and a caching system. This presentation will describe the architecture of the system and how we use spring-websockets and spring-data-Redis to power the system. As a bonus, we will show a great way to find out in real time how many users are currently using your system.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Real Time Recommendations Using WebSockets and Redis - Ninad Divadkar, Inuit
Similar to Real Time Recommendations Using WebSockets and Redis - Ninad Divadkar, Inuit (20)
More from Redis Labs
More from Redis Labs (20)
Recently uploaded
Recently uploaded (20)
Real Time Recommendations Using WebSockets and Redis - Ninad Divadkar, Inuit
- 6. Recommend rules based on user behavior • Backend analyzes your behavior, and recommends rules.
- 7. • Increased the ‘wow’ factor of our product. • People who accept the recommended rules are more likely to subscribe. • > 60% acceptance rate Recommending rules
- 8. This is enabled by… Communication between: • Browser and app servers using Web sockets • Back end processes using Redis Pub/Sub
- 9. WebSockets WebSockets is an application protocol. Enables two way communication between Server and Client. Supported by the latest browsers and web servers. Spring supports websockets.
- 10. Redis Pub/Sub Publish to a channel Subscribe to a pattern or a channel Linear complexity for most operations O(N+M)
- 11. Architecture
- 12. Spring on the backend.. Spring-session Spring-mvc Spring-websockets Spring-data-redis Spring-*
- 13. And AngularJS on the front AngularJS SockJS – JS web socket client StompJS – Text protocol over web socket
- 14. The sequence Browser WebApp servers Worker processes Create web socket connection Subscribe to queue Create Redis Pub Sub channel for user Connect a Bank Relay bank connection request to worker Listen for bank connection publish event Connect to bank, Import transactions, etc. Publish to web socketInform user of successful connection Publish connection successful event ① ② ③ ④ ⑤ ⑥⑦⑧
- 15. Why Redis for Pub/Sub? Our use case – Create and close channels dynamically, 100s of times a minute. 1000s of channels (users) live at any given time. For our use case, RabbitMQ did not perform.
- 17. When a user launches the app • Web socket connection created. • Subscribe to the user’s topic.
- 18. The front end _connect() – Connects the web socket, subscribes to a queue called “/user/{username}/queue/user_rec ommendation”. fireCallbacks() – fire callbacks to UI subscribers, informing them about events.
- 19. Some spring web socket config • Assign “/queue” endpoint to web sockets. • Assign “/ws” to annotated methods. • Stomp endpoint is “/ws/ws- init”
- 20. When a web socket subscription occurs … we create a Redis PUBSUB channel. • Every user has a unique PUBSUB channel. • Only the App process that listens to the websocket subscribes to the Redis channel.
- 21. And this is what you see in Redis We do end up 1000s of channels, but Redis doesn’t blink! … RabbitMQ did not scale above 200 queues
- 22. User starts categorizing transactions
- 23. When it detects a possible rule, it publishes a message to the user’s topic. In the worker process
- 24. … and publishes the same message on the Web Socket App Server receives Message
- 25. A rule recommendation. And the user sees…
- 26. “PUBSUB CHANNELS” gives an exact count of how many users are using the system at a given time. Why DevOps loves this
- 27. What about mobile? Web socket supported in iOS and Android. iOS – StarScream Android - android-websockets Stomp support not so great… but it’s there: iOS – WebSocketStompKit Android – Gozirra You can use Web Sockets without stomp.
- 28. iOS Demo • Heavy lifting done by the back end. • Same User Experience across all devices.
- 29. Q and A
- 30. Contact Github - http://github.com/nsdiv Email – ninad.divadkar @ gmail.com
Editor's Notes
- Hello everybody, thank you for coming and welcome to my presentation. My name is Ninad Divadkar, and I’m going to talk about Real time recommendations using Web Sockets and Redis.
- A little bit about my self. I work for Intuit Inc in the QuickBooks Self Employed division. I’m a full stack developer, and although I have used quite a few different programming languages over the years, I’d say my backend expertise is in JAVA and .NET, and front end in AngularJS. I’m also a Spring contributor. My latest contribution is to the spring-data-redis project. Redis is introducing Geo commands in the next version, and I have added support for those in spring-data-redis. In addition, I am active on stackoverflow, mainly in JavaScript, Java and Spring topics.
- Before we dive into the presentation, let me tell you about QuickBooks Self Employed, because it will help us understand our use case better. QuickBooks Self Employed helps self employed people – like artists, dancers, uber drivers, etc find tax deductions and be complaint with IRS rules and regulations. We do that by connecting to their bank and importing the bank transactions. Users then categorize these transactions as business or personal. Anything transaction categorized as a business expense can be a tax deduction. We have an interface so that users can pay the IRS their estimated taxes directly through our software. And we integrate with TurboTax for hassle free filing. In addition to our web presence, our iOS and Android apps help self employed people do mileage tracking. We launched a year and a half ago in the US, and have also launched the product in the UK, with more countries to follow shortly.
- Let me demo you our iOS app to show what the product does. Users categorize these transactions as business or personal. Users track their mileage, so, for example if they are working for Uber they can deduct the miles they drove their car for work. We give the users an overview of their finances, because many users do not have a great way of know whether they have a profit or not. And we integrate with TurboTax (which is also an Intuit product), so that users can easily file their taxes.
- Now, I’m going to demo two workflows in QuickBooks Self Employed that demonstrate how we use web sockets and Redis Pub/Sub in tandem. I’m just going to demo the UI, and as go further into the presentation, I’ll show you the architecture and deeper layers of our product. The first workflow is the Bank Connection Flow. After launching the app, the user is led to the ‘connect a bank account’ screen. The user selects a bank and enters his credentials. We support 10,000 plus banks and credit unions in the US. Connecting to a bank takes anywhere from 30 seconds to 5 minutes. During this time we’re communicating with a backend service that actually logs in to the bank and downloads transactions. This is a critical process in the user flow, almost a make or break point. We need to WOW the user, and make sure the user understands the value of our product. At this moment, we want the UI to be updated with what stage the back end is at in the process of connecting a bank.
- After the user has started categorizing transactions, the product learns from the categorization, and recommends rules to help the user save time. So if, for example, the user categorizes 3 Wal-mart transactions as business, we understand that he or she is going to categorize the rest as business as well. So the product pops up a recommendation to the user.
- Recommending rules to user has made our users very happy. When they start using the product, they learn the value right away. We have an over 60% rate of acceptance with our rule recommendation engine, and it is going up all the time. Best of all, it is a unified experience across web and mobile. The web as well as mobile apps use web-sockets to get the recommendations.
- These two workflows are enabled by….. Web Sockets for communicating between the front end (browser) and the back end. Redis Pub/Sub for communicating between back end processes.
- Most of you may be familiar with what websockets are, but let me spend a couple of minutes explaining: WebSockets is a application protocol, that works over TCP. The idea behind the WebSocket protocol consists of reusing the established TCP connection between a Client and Server. After the HTTP handshake, the Client and Server start speaking WebSocket protocol by exchanging WebSocket envelopes. What this means for us is that it allows two way communication in the browser. This allows us to communicate with the browser in message bus type fashion. Before web socket support, we used to have long polling and other stop gap measures, but they are not required anymore.
- Now, let’s move to Redis Pub/Sub. Most of us use Redis for caching, but Redis also has a very good pub/sub messaging framework built in. The terminology that Redis uses is a ‘channel’ instead of a ‘queue’ or a ‘topic’. A publisher publishes to a channel. A Subscriber subscribes to a channel or a pattern. These commands are super fast, and all of the pub/sub commands take linear time, O(n+m), where N is the number of clients subscribed directly to the channel and M is the number subscribed patterns.
- The architecture of our site. We have web app processes that serve requests coming in through our web app. We have API processes that serve API requests coming in through our mobile apps. We run embedded Tomcat 8 in both servers. We have worker processes who do the heavy lifting, allowing us to scale without having to increase our app and api processes. Intra process communication is done using Redis pub/sub, i.e. Web app and API processes communicate with the worker process using Redis Pub/Sub
- Spring is used extensively on the backend. We use a lot more Spring projects than shown above.
- SockJS is a browser JavaScript library that provides a WebSocket like object. Under the hood SockJS tries to use native WebSockets first. If that fails it can use a variety of browser-specific transport protocols and presents them through WebSocket-like abstractions. STOMP is a simple text messaging protocol, and stompJS implements that protocol. Spring supports it, and so we use it.
- The big question, why redis for Pub Sub. Our use case is a little different from regular use cases for RabbitMQ. RabbitMQ can handle thousands of messages a second, but it is not built to handle hundreds of queues being created dynamically in a minute. We went to production with RabbitMQ. We have multiple app servers, but a web socket is created to only one app server. That server needs to listen to a message sent by the worker process. So we need a unique queue per user. RabbitMQ performance dropped drastically after around 200 queues. CPU spiked and we had to drop the idea of using RabbitMQ. Instead of being able to create ~50 queues / sec, throughput dropped to ~2 queues / sec.
- Now, we’ll go through the entire workflow, of a user launching the app and seeing a recommendation.
- Creating a web socket connection is not enough. Web-sockets connection opens a pipe to the server. However you want to publish to a topic. That way different parts of the code can use different topics, just like on the back end.
- EnableWebSocketMessageBroker is the Spring annotation to enable broker backed mesasaging over webSocket. The MessageBrokerRegistry configuration reserves certain endpoints for websocket broker Stomp endpoint is ws-init, for regular cases
- We’re implementing an ApplicationListener where we are listening to the SessionSubscribeEvent. When a SessionSubscribeEvent occurs, we create a Redis Pub Sub channel for the user. Every user has a unique PUBSUB channel. And only the App process that listens to the websocket subscribes to the Redis channel. For example, if there are 5 app server taking requests, the web socket is connected to only 1 app server, and only that app server will subscribe to the user’s PUBSUB channel.
- During tax peak we had over 4k simultaneous users on our site.
- When the worker process detects a possible rule, it publishes a message to the user’s topic.
- An unexpected benefit of creating Redis channels whenever a user launches our app is that we can count the exact number of users who are on our site. Our DevOps has written a script that runs every minute and captures the number of live channels in Redis. The picture above is of a Splunk search where we have charted the number of users using our app over a 7 day period. Mind you this is after tax season is over. During tax season we had record traffic. Things like these help us understand how many users are using our web app or mobile apps at any given time, and how long they are staying on the site as well.
- Websockets are supported in both iOS and Android. There are quite a few open source libraries that work great. We are using https://github.com/daltoniam/Starscream in iOS and https://github.com/koush/android-websockets for android. There isn’t good support for Stomp. But that is not an issue. Spring has support for just Websockets without stomp. You’ll just have to make your own protocol, and for our use case it is not a problem. http://stackoverflow.com/questions/27158106/websocket-with-sockjs-spring-4-but-without-stomp
- Redis channel limits - https://groups.google.com/forum/#!topic/redis-db/G-rFG0k7NiY. At least up to 10k. We have tested until 5k simultaneous channels, our peak traffic has reached 4k simultaneous connections. Websockets for iOS and android – WebsocketStompKit (iOS), gozirra (android) while there is support, it is not well maintained. XMPP vs websocket - http://stackoverflow.com/questions/31108780/stomp-or-xmpp-over-websocket . Stomp is easier to implement in Spring