Hello everyone and welcome to this presentation on using Mongo with AEM Communities
My name is Kevin Nennig and I’m a Corporate Technical Instructor for Adobe.
I’ve been working with Adobe Experience Manager for almost 2 years
My job gives me a lot of opportunities to work with new companies looking to implement our AEM. One of the topics a lot of clients want to hear about is the Communities feature of AEM
The Current release of AEM is 6.1 and with the release comes an updated Communities Module.
In this communities module, 2 out of the 3 implementations use Mongo as the data storage mechanism.
In this presentation we will discuss each of these options and understand why Mongo is a good solution for AEM.
The Adobe Marketing Cloud is a service for digital marketers, advertisers, and publishers that works in conjunction with the Adobe Creative Cloud.
The Adobe Marketing Cloud consists of these capabilities :
1. A scalable, secure and open platform to host data & content
2. A multi-channel analytics and decision capability
3. Optimization to increase visitor acquisition, improve conversion and maximize the value of audiences
These capabilities are enabled through the following 8 solutions… (name and describe them)
The focus of today’s course is on _______.
AEM is the main focus of this presentation.
AEM is a comprehensive content management system for building websites, mobile apps, and forms.
It makes it easier to manage marketing content and digital assets
Within AEM there are 5 main modules – Sites, Assets, Apps, Forms, Communities.
This presentation is focused on AEM communities 6.1
It allows up to build thriving communities and inspire engaging conversations across audiences from customers to employees to partners.
Within AEM, there are two different scenarios to use Mongo
You can use Mongo at the Microkernel layer to persist all of AEM’s DB to Mongo
OR you can use Mongo to store only User Generated Content for a company’s online communities
Within AEM, there are two different scenarios to use Mongo
You can use Mongo at the Microkernel layer to persist all of AEM’s DB to Mongo
OR you can use Mongo to store only User Generated Content for a company’s online communities
The are a few reasons why Mongo works well with storing data for AEM. First off,
flexible JSON document model – this is actually what AEM’s persistence microkernel layer is based upon
efficient searching with indexing - Compared to other noSQL solutions
Built in replication for high availability – Mongo uses replica sets which allow the primary MongoDB to replicate to secondaries for redundancy of the data layer
Scaleable – Mongo allows for both horizontal and vertical scaling
High concurrent writes - which is good for high traffic on a community site
reduces operational overhead — AEM doesn’t have to worry about persistence
Anyone in the audience that wants to participate can go to this website
This Communities website uses MSRP or Mongo Social resource provider to store all user generated content
Feel free to answer
When we start to talk about AEM Communities, there are 3 things that we need to consider,
1 – This allows for content to be served up in the fastest manner possible.
This is a basic AEM deployment.
Your author server is your internal server where you manage all of your content.
The publish farm is your publish instances that hold published content of your website.
Since AEM builds your pages dynamically for every request, there is a static webserver we call a dispatcher in from of your publish farm
The dispatcher caches your static content so that your site is faster
This is a basic AEM deployment.
Your author server is your internal server where you manage all of your content.
The publish farm is your publish instances that hold published content of your website.
Since AEM builds your pages dynamically for every request, there is a static webserver we call a dispatcher in from of your publish farm
The dispatcher caches your static content so that your site is faster
The second idea that was considered was replicating content wasn’t good enough for last scale solutions
2 – replicating content backwards allows for publish instances to be out of sync for short periods of time
This concept allow transfer of content from the publish server to the author for review, moderation, and spam check without opening the firewall from the DMZ to the author server.
This concept allow transfer of content from the publish server to the author for review, moderation, and spam check without opening the firewall from the DMZ to the author server.
The third idea is that read/write had to be fast for a large scale of transitions and there cannot be data collisions
3 – When dealing with large community site that have hundreds of transactions per second
Create a common store for user generated content. Where all UGC created is stored in a central repository and is only accessed on request.
Where the website itself is still on the publish instance and only the UGC on the current page is needed for rendering
The Social Resource provider is the solution for the common store
This allows for us to have multiple solutions to store UGC depending on a company’s requirements
JSRP - uses clustering to have a common database among publish instances where all content can be stored
ASRP – Managed and hosted by Adobe
MSRP – A mongo replset is used for on prem deployment
JCR Social Resource Provider. New in 6.1
Default configuration for Communities
UGC persisted in publisher JCR repository. Cluster assumed if multiple publishers.
Suitable for customers who can run a publish cluster on MongoMK or RdbmsMK
UGC not available on author
Uses Oak indices. Querying necessary for communities. Sort by date, helpfulness, number of votes. Can’t avoid queries. All of our SRPs have good/flexible querying
No more bucketing. With Oak, buckets aren’t necessary, and it made indexing simpler without them, so we are eliminating them.
Migration required for all SRPs
Open Source tool will be available to export and import to *SRP
The Adobe Social Resource Provider.
Publishers can be a farm. Point all of your instances, including author, at Adobe Social Cloud. All instances look at same copy of data.
Shipped in AEM 6.0
UGC is persisted via a cloud service provided and supported by Adobe
Integrates with the Adobe Social Analytics Pipeline and Moderation
No need to invest in infrastructure to hold UGC
The Mongo Social Resource Provider or MSRP. New in 6.1.
A whole lot like the last picture, except UGC is persisted in a local and dedicated MongoDB and Solr.
Suitable for a large volume of UGC
Compatible with on-prem publish farm topologies
The external solr server is used to create non-biased indexers between the publish instances and the Mongo UGC cloud
Only the actualy UGC is stored in the Common store. Things such as:
The other aspects of a community would be users, groups, and the actual site itself.
Non of these are stored in the UGC,
External users and groups should not be available on the internal author server
And therefore are only replicated on the publish servers so users can gain the proper rights to the community site on any publish server
The Community site itself is managed on the author instance and then replicated to the publish farm just like any normal piece of content
External users and groups should not be available on the internal author server
And therefore are only replicated on the publish servers so users can gain the proper rights to the community site on any publish server
The Community site itself is managed on the author instance and then replicated to the publish farm just like any normal piece of content
ASC – Adobe Social Cloud
The next question concerning UGC is the dispatcher….
If we recall, the dispatcher is used to cache static content from our dynamic AEM instances.
Which means that UGC specific information can’t be stored on the dispatcher, which means community pages pose a problem
If we recall, the dispatcher is used to cache static content from our dynamic AEM instances.
Which means that UGC specific information can’t be stored on the dispatcher, which means community pages pose a problem
Utilizing Mongo as the common store, we can see how much performance increases for User Generated content
As a benchmark we found that reverse replication allows for about 5 transactions per second, no matter how many active nodes
Where when we use Mongo SRP, we find that we can massively increase transactions by 20x per instance.
Which means transactions are linearly scaleable by the amount of AEM publish nodes in the front end.
Utilizing Mongo as the common store, we can see how much performance increases for User Generated content
As a benchmark we found that reverse replication allows for about 5 transactions per second, no matter how many active nodes
Where when we use Mongo SRP, we find that we can massively increase transactions by 20x per instance.
Which means transactions are linearly scaleable by the amount of AEM publish nodes in the front end.
If a customer wants to utilize the fastness of a publish farm and wants to have an on prem solution,
MSRP is the best solution for the UGC.
Lets see how we can set one up
The collection used for UGC is called msrp-communities, which can be configured.
The solr cloud I’m using the default collection collection1 for indexing
The collection used for UGC is called msrp-communities, which can be configured.
The solr cloud I’m using the default collection collection1 for indexing
We first need to start 3 separate mongo instances listening for an aem replica set
Then we need to connect to the mongo instance we want to become our primary
We then can initialize our replica set and check to make sure it’s configured correctly
Once the replica set has been initialized your prompt will change to show the current replica set and the current servers state
We then can initialize our replica set and check to make sure it’s configured correctly
Once the replica set has been initialized your prompt will change to show the current replica set and the current servers state
Once our replica set, aem, is setup we can add the other two mongo instances as members.
Remember the least amount of configuration for a replica set requires 3 mongo members
We then want to check the configurations to make sure all 3 mongo instances are added to the replset
There are 2 Solr config files that need to be added to the solr configuration before starting
These can be obtained from any AEM instance in the communities section
Once copied, you can start the solr server on the default port 8983
Once the mongo replica set and solr server is up and running,
The SRP configuration needs to be setup on every AEM server that will be using the UGC
We can see an example of that configuration here
Notice that we are going to persist UGC to a collection called msrp-communities