MongoDB Days Silicon Valley: Using MongoDB with Adobe AEM Communities
•Download as PPTX, PDF•
4 likes•2,503 views
Presented by Kevin Nennig, AEM Technical Instructor, Adobe Experience level: Introductory In Adobe Experience Manager 6.1, a new approach has been taken for user generated content (UGC) for better security, higher availability, and ease of use for community members and administrators. The Social Resource Provider API for AEM Communities can be implemented 3 different ways to create a UGC cloud, two of which use a MongoDB backend. Utilizing MongoDB, companies can implement an on prem UGC common store with a Mongo and a Solr server. UCG is then stored and accessed from this UGC cloud. In this session we’ll explore the Social Resource Provider framework and implement the Mongo Social Resource Provider (MSRP) for AEM Communities.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (19)
Similar to MongoDB Days Silicon Valley: Using MongoDB with Adobe AEM Communities
Similar to MongoDB Days Silicon Valley: Using MongoDB with Adobe AEM Communities (20)
More from MongoDB
More from MongoDB (20)
Recently uploaded
Recently uploaded (20)
MongoDB Days Silicon Valley: Using MongoDB with Adobe AEM Communities
- 1. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Using Mongo with Adobe AEM Communities Kevin Nennig | Corporate Technical Instructor
- 2. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. ADOBE TRAINING Adobe Media Optimizer Adobe Social Adobe Target Adobe Audience Manager Adobe Experience Manager Adobe Campaign Adobe Analytics Adobe Primetime 2 Adobe Marketing Cloud Solution
- 3. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. ADOBE TRAINING Adobe Media Optimizer Adobe Social Adobe Target Adobe Audience Manager Adobe Experience Manager Adobe Campaign Adobe Analytics Adobe Primetime 3 Adobe Marketing Cloud Solution
- 4. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. AEM Modules
- 5. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. AEM Modules
- 6. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Using Mongo in AEM Persistence of entire database User generated content only
- 7. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Using Mongo in AEM Persistence of entire database User generated content only
- 8. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Why use Mongo for AEM Communities? • Flexible JSON document model • Efficient searching with indexing • Built in replication for high availability • Scaleable • High concurrent writes • Reduces operational overhead
- 9. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Lets have a social opinion! • http://mongosv.kevinnennig.com • AEM Community with Mongo common store (MSRP) • “Mongo Opinions” is based upon Yahoo Answers • Sign up and give your opinion to a few questions!
- 10. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Communities 6.1 10 Publish farms are preferred 1 http://mongosv.kevinnennig.com
- 11. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Content Replication 11 AEM Publish Farm AEM Author Dispatcher
- 12. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Content Replication 12 AEM Publish Farm AEM Author Dispatcher • Recommended approach • Performance • Scalability on read access • Failover • What about UGC?
- 13. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Communities 6.1 13 Publish farms are preferred 1 Replicating UGC isn’t good enough 2 http://mongosv.kevinnennig.com
- 14. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. AEM Replication of UGC (earlier versions of AEM) 1 AEM Publish AEM Publish Comment, rating, forum post 2 Stored in repository and in Replication Outbox 3 Check and fetch Outbox content 4 Workflow-based moderation and spam check AEM Author Replication to all publish Internal Network DMZ 5 5
- 15. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. AEM Replication of UGC (earlier versions of AEM) • Everything has to go through author • UGC is not immediately available everywhere • Slow 1 AEM Publish AEM Publish Comment, rating, forum post 2 Stored in repository and in Replication Outbox 3 Check and fetch Outbox content 4 Workflow-based moderation and spam check AEM Author Replication to all publish Internal Network DMZ 5 5
- 16. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Communities 6.1 16 Publish farms are preferred 1 Replicating UGC isn’t good enough 2 Need to be Fast and consistent 3 http://mongosv.kevinnennig.com
- 17. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 17 The Solution: Common Store for UGC http://mongosv.kevinnennig.com
- 18. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Implementing the Common Store 18 • Social Resource Provider (SRP) • An API that abstracts the storage • Implemented three different ways • MongoMK JCR SRP (JSRP) • Adobe SRP (ASRP) • Mongo SRP (MSRP)
- 19. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. JSRP: JCR Social Resource Provider 19 19 19 AEM Author • Publish instances are clustered with MongoMK • Default storage mechanism • Easy to setup UGC • UGC is only available on publish instances • Publish Farm is not utilized MongoMK AEM Publish 3 AEM Publish 2 AEM Publish 1
- 20. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. ASRP: Adobe Social Resource Provider 20 AEM Publish Farm Adobe Social Cloud AEM Author • Recommended for managed UGC • No setup needed • Integrates with Adobe Social Analytics Pipeline and Moderation • Fast
- 21. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. MSRP: Mongo Social Resource Provider 21 AEM Author AEM Publish Farm • Recommended for on prem UGC • Required to setup a mongo replset and solr cloud • Fast and scaleable
- 22. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. What’s stored in the Common Store? 22 • Contextual • Comments • Reviews • Ratings • Votes/Likes Adobe Social Cloud MongoMK • Standalone • Forums • Calendars • File Sharing • Activities
- 23. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. What isn’t stored in the Common Store? 23 • External user accounts • External groups • Community website Adobe Social Cloud MongoMK
- 24. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. What isn’t stored in the Common Store? 24 • External user accounts • External groups • Community website Adobe Social Cloud MongoMK Replicated among publish instances but not author Replicated from the author instance to publish farm
- 25. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Comparing SRP 25 Adobe Social Cloud MongoMK Social Resource Provider JSRP ASRP MSRP Microkernel MongoMK TarMK TarMK Site Content Repository Publish Server Publish Server Publish Server UGC Repository Publish Server Adobe Social Cloud MongoDB When it should be Default storage. Not recommended for production, only development. Customers using the ASC or who do not Requirements of data being on prem
- 26. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Dispatcher 26 UGC is dynamic content. Can it work with Dispatcher? http://mongosv.kevinnennig.com
- 27. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Dispatcher 27 AEM Publish Farm AEM Author Dispatcher
- 28. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Dispatcher 28 AEM Publish Farm Dispatcher User info
- 29. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Performance increase using Mongo SRP 29 “Mixed transactions” test case: Based on customer data Reverse Replication JSP components (deprecated in 6.0) CRX2 2 node active/passive cluster 5 transactions per second
- 30. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Performance increase using Mongo SRP 30 “Mixed transactions” test case: Based on customer data Reverse Replication JSP components (deprecated in 6.0) CRX2 2 node active/passive cluster 5 transactions per second Mongo SRP SCF components MSRP 4 node farm Dispatcher 100 tps per instance
- 31. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 31 Setting up MSRP http://mongosv.kevinnennig.com
- 32. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. MSRP: Mongo Social Resource Provider 32 AEM Author 27019 2701827017 aem AEM Publish Farm
- 33. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. MSRP: Mongo Social Resource Provider 33 AEM Author 27019 2701827017 aem Database: msrp-communities Collection: collection1 AEM Publish Farm
- 34. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Starting a replset Mongo 34 Start Mongo Servers mongoServer1$ mongod --bind_ip nennig-osx.local --logpath logs/mongo.log --dbpath db --replSet aem --port 27017 mongoServer2$ mongod --bind_ip nennig-osx.local --logpath logs/mongo.log --dbpath db --replSet aem --port 27018 mongoServer3$ mongod --bind_ip nennig-osx.local --logpath logs/mongo.log --dbpath db --replSet aem --port 27019
- 35. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Starting a replset Mongo 35 Create the Replica Set Connect to the mongodb you want to be Primary $ mongo nennig-osx.local:27017 >
- 36. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Starting a replset Mongo 36 Initialize the Replica Set > rs.initiate() { "info2" : "no configuration explicitly specified -- making one", "me" : "nennig-osx.local:27017", "info" : "Config now saved locally. Should come online in about a minute.", "ok" : 1 } aem:PRIMARY> rs.conf() { "_id" : "aem", "members" : [ { "_id" : 0, "host" : "nennig-osx.local:27017" } ] }
- 37. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Starting a replset Mongo 37 Initialize the Replica Set > rs.initiate() { "info2" : "no configuration explicitly specified -- making one", "me" : "nennig-osx.local:27017", "info" : "Config now saved locally. Should come online in about a minute.", "ok" : 1 } aem:PRIMARY> rs.conf() { "_id" : "aem", "members" : [ { "_id" : 0, "host" : "nennig-osx.local:27017" } ] } You can also use: rs.status() This will show the current status of each server including who’s PRIMARY/SECONDARY
- 38. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Starting a replset Mongo 38 Add members aem:PRIMARY> rs.add(“nennig-osx.local:27018”) { "ok" : 1 } aem:PRIMARY> rs.add(“nennig-osx.local:27019”) { "ok" : 1 }
- 39. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Starting a replset Mongo 39 aem:PRIMARY> rs.conf() { "_id" : "aem", "version" : 3, "members" : [ { "_id" : 0, "host" : "nennig-osx.local:27017" }, { "_id" : 1, "host" : "nennig-osx.local:27018" }, { "_id" : 2, "host" : "nennig-osx.local:27019" } ] }
- 40. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Start the Solr Server 40 Download config files from the AEM server Start the Solr Server with the conf files on port 8983 solrServer$ java –jar start.jar
- 41. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. Configure the Social Resource Provider on AEM 41
- 42. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 42 Demo http://mongosv.kevinnennig.com
- 43. © 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 43 Q&AKevin Nennig nennig@adobe.com Slideshare: http://www.slideshare.net/KevinNennig/aem-communities-61-mongosv-15 Community Topologies: https://docs.adobe.com/docs/en/aem/6-1/deploy/communities/topologies.html MSRP: https://docs.adobe.com/docs/en/aem/6-1/administer/communities/srp/msrp.html http://mongosv.kevinnennig.com
Editor's Notes
- Hello everyone and welcome to this presentation on using Mongo with AEM Communities My name is Kevin Nennig and I’m a Corporate Technical Instructor for Adobe. I’ve been working with Adobe Experience Manager for almost 2 years My job gives me a lot of opportunities to work with new companies looking to implement our AEM. One of the topics a lot of clients want to hear about is the Communities feature of AEM The Current release of AEM is 6.1 and with the release comes an updated Communities Module. In this communities module, 2 out of the 3 implementations use Mongo as the data storage mechanism. In this presentation we will discuss each of these options and understand why Mongo is a good solution for AEM.
- The Adobe Marketing Cloud is a service for digital marketers, advertisers, and publishers that works in conjunction with the Adobe Creative Cloud. The Adobe Marketing Cloud consists of these capabilities : 1. A scalable, secure and open platform to host data & content 2. A multi-channel analytics and decision capability 3. Optimization to increase visitor acquisition, improve conversion and maximize the value of audiences These capabilities are enabled through the following 8 solutions… (name and describe them) The focus of today’s course is on _______.
- AEM is the main focus of this presentation. AEM is a comprehensive content management system for building websites, mobile apps, and forms. It makes it easier to manage marketing content and digital assets
- Within AEM there are 5 main modules – Sites, Assets, Apps, Forms, Communities.
- This presentation is focused on AEM communities 6.1 It allows up to build thriving communities and inspire engaging conversations across audiences from customers to employees to partners.
- Within AEM, there are two different scenarios to use Mongo You can use Mongo at the Microkernel layer to persist all of AEM’s DB to Mongo OR you can use Mongo to store only User Generated Content for a company’s online communities
- Within AEM, there are two different scenarios to use Mongo You can use Mongo at the Microkernel layer to persist all of AEM’s DB to Mongo OR you can use Mongo to store only User Generated Content for a company’s online communities
- The are a few reasons why Mongo works well with storing data for AEM. First off, flexible JSON document model – this is actually what AEM’s persistence microkernel layer is based upon efficient searching with indexing - Compared to other noSQL solutions Built in replication for high availability – Mongo uses replica sets which allow the primary MongoDB to replicate to secondaries for redundancy of the data layer Scaleable – Mongo allows for both horizontal and vertical scaling High concurrent writes - which is good for high traffic on a community site reduces operational overhead — AEM doesn’t have to worry about persistence
- Anyone in the audience that wants to participate can go to this website This Communities website uses MSRP or Mongo Social resource provider to store all user generated content Feel free to answer
- When we start to talk about AEM Communities, there are 3 things that we need to consider, 1 – This allows for content to be served up in the fastest manner possible.
- This is a basic AEM deployment. Your author server is your internal server where you manage all of your content. The publish farm is your publish instances that hold published content of your website. Since AEM builds your pages dynamically for every request, there is a static webserver we call a dispatcher in from of your publish farm The dispatcher caches your static content so that your site is faster
- This is a basic AEM deployment. Your author server is your internal server where you manage all of your content. The publish farm is your publish instances that hold published content of your website. Since AEM builds your pages dynamically for every request, there is a static webserver we call a dispatcher in from of your publish farm The dispatcher caches your static content so that your site is faster
- The second idea that was considered was replicating content wasn’t good enough for last scale solutions 2 – replicating content backwards allows for publish instances to be out of sync for short periods of time
- This concept allow transfer of content from the publish server to the author for review, moderation, and spam check without opening the firewall from the DMZ to the author server.
- This concept allow transfer of content from the publish server to the author for review, moderation, and spam check without opening the firewall from the DMZ to the author server.
- The third idea is that read/write had to be fast for a large scale of transitions and there cannot be data collisions 3 – When dealing with large community site that have hundreds of transactions per second
- Create a common store for user generated content. Where all UGC created is stored in a central repository and is only accessed on request. Where the website itself is still on the publish instance and only the UGC on the current page is needed for rendering
- The Social Resource provider is the solution for the common store This allows for us to have multiple solutions to store UGC depending on a company’s requirements JSRP - uses clustering to have a common database among publish instances where all content can be stored ASRP – Managed and hosted by Adobe MSRP – A mongo replset is used for on prem deployment
- JCR Social Resource Provider. New in 6.1 Default configuration for Communities UGC persisted in publisher JCR repository. Cluster assumed if multiple publishers. Suitable for customers who can run a publish cluster on MongoMK or RdbmsMK UGC not available on author Uses Oak indices. Querying necessary for communities. Sort by date, helpfulness, number of votes. Can’t avoid queries. All of our SRPs have good/flexible querying No more bucketing. With Oak, buckets aren’t necessary, and it made indexing simpler without them, so we are eliminating them. Migration required for all SRPs Open Source tool will be available to export and import to *SRP
- The Adobe Social Resource Provider. Publishers can be a farm. Point all of your instances, including author, at Adobe Social Cloud. All instances look at same copy of data. Shipped in AEM 6.0 UGC is persisted via a cloud service provided and supported by Adobe Integrates with the Adobe Social Analytics Pipeline and Moderation No need to invest in infrastructure to hold UGC
- The Mongo Social Resource Provider or MSRP. New in 6.1. A whole lot like the last picture, except UGC is persisted in a local and dedicated MongoDB and Solr. Suitable for a large volume of UGC Compatible with on-prem publish farm topologies The external solr server is used to create non-biased indexers between the publish instances and the Mongo UGC cloud
- Only the actualy UGC is stored in the Common store. Things such as:
- The other aspects of a community would be users, groups, and the actual site itself. Non of these are stored in the UGC,
- External users and groups should not be available on the internal author server And therefore are only replicated on the publish servers so users can gain the proper rights to the community site on any publish server The Community site itself is managed on the author instance and then replicated to the publish farm just like any normal piece of content
- External users and groups should not be available on the internal author server And therefore are only replicated on the publish servers so users can gain the proper rights to the community site on any publish server The Community site itself is managed on the author instance and then replicated to the publish farm just like any normal piece of content ASC – Adobe Social Cloud
- The next question concerning UGC is the dispatcher….
- If we recall, the dispatcher is used to cache static content from our dynamic AEM instances. Which means that UGC specific information can’t be stored on the dispatcher, which means community pages pose a problem
- If we recall, the dispatcher is used to cache static content from our dynamic AEM instances. Which means that UGC specific information can’t be stored on the dispatcher, which means community pages pose a problem
- Utilizing Mongo as the common store, we can see how much performance increases for User Generated content As a benchmark we found that reverse replication allows for about 5 transactions per second, no matter how many active nodes Where when we use Mongo SRP, we find that we can massively increase transactions by 20x per instance. Which means transactions are linearly scaleable by the amount of AEM publish nodes in the front end.
- Utilizing Mongo as the common store, we can see how much performance increases for User Generated content As a benchmark we found that reverse replication allows for about 5 transactions per second, no matter how many active nodes Where when we use Mongo SRP, we find that we can massively increase transactions by 20x per instance. Which means transactions are linearly scaleable by the amount of AEM publish nodes in the front end.
- If a customer wants to utilize the fastness of a publish farm and wants to have an on prem solution, MSRP is the best solution for the UGC. Lets see how we can set one up
- The collection used for UGC is called msrp-communities, which can be configured. The solr cloud I’m using the default collection collection1 for indexing
- The collection used for UGC is called msrp-communities, which can be configured. The solr cloud I’m using the default collection collection1 for indexing
- We first need to start 3 separate mongo instances listening for an aem replica set
- Then we need to connect to the mongo instance we want to become our primary
- We then can initialize our replica set and check to make sure it’s configured correctly Once the replica set has been initialized your prompt will change to show the current replica set and the current servers state
- We then can initialize our replica set and check to make sure it’s configured correctly Once the replica set has been initialized your prompt will change to show the current replica set and the current servers state
- Once our replica set, aem, is setup we can add the other two mongo instances as members. Remember the least amount of configuration for a replica set requires 3 mongo members
- We then want to check the configurations to make sure all 3 mongo instances are added to the replset
- There are 2 Solr config files that need to be added to the solr configuration before starting These can be obtained from any AEM instance in the communities section Once copied, you can start the solr server on the default port 8983
- Once the mongo replica set and solr server is up and running, The SRP configuration needs to be setup on every AEM server that will be using the UGC We can see an example of that configuration here Notice that we are going to persist UGC to a collection called msrp-communities
- Let’s take a moment to see a demo in action!
- Let’s take a moment to see a demo in action!