Planning Very Large Scale Document Repositories with High Availability in SharePoint 2013

Planning Very Large Scale Document
Repositories with High Availability (HA) in
SharePoint Server 2013
Wired2Win Webinar Series

WinWire Technologies Inc.
2350 Mission College Boulevard,
Suite 480, Santa Clara, California 95054
Phone/Fax: +1 (888) 849 7339
Email : info@winwire.com

WinWire Technologies, Inc. Confidential

© 2010 WinWire Technologies

@WinWire



2

Who We Are
WinWire Technologies is a
specialized IT solutions
company focused on helping
business and technology
leaders achieve an “on-themove” business environment
leveraging pre-built
collaborative and analytic
solution accelerators and
cross-platform mobile
technologies.



3

This is the story of Bob.



4

Why is Bob sad?



5

Bob deployed SharePoint.



6

Things were pretty good.



7

So what’s the problem?



8

He needs to provide the following functionality to his organization.

Very Large Scale
Document
Repositories

Lowest possible
maintenance and
operations costs


Highest possible
availability

Predictable
“Elastic”
architecture that
can grow with the
deployment


9

He’s got many folks to please,
including IT Support, legal and
compliance.



10

Users are concerned.
They don’t want it to be hard.



11

SharePoint seems to provide the
right capabilities!



12

Or does it?



13

Bob isn’t sure how to proceed.



14

Let’s help Bob with
Planning Very Large Scale
Document Repositories with
High Availability (HA) in
SharePoint



15

Agenda



16

Agenda



17

Enterprise Content Management

Create

Control

Create and organize
content easily


Manage content policy,
information architecture
and taxonomy


18

Protect
Reduce risk and manage
compliance with
centralized tools

SharePoint 2013 Designing Your Farm
•

Motivation
• End-user experience is what really
matters.
• Tail end of performance is sticky.

•

Basic Idea
• Categorize applications into SLA
buckets.
• Homogenize front-end, backend,
and data layers.
• Prevent proliferation of app server
deployments.



19

Designing for better SLA
Latency

Request
Management
Front-End

Resource
Utilization

Desired
Behavior

Very low
< 5 msec

Very High

Medium

Consistent
Latency

Low
<500 msec

Distributed
Cache

Throughput

Medium

Low-Medium

Consistent
Latency

Low
<500 msec

Medium

Low – High

Fairly
Consistent
Latency

Batch Processing Layer

High
> 1 min

High

High-Very
High

High
Throughput

Database Layer

Very Low
< 5 msec

Very High

Low-Medium

Consistent
Latency

Specialized
Workload


Search


20

Evolution of topology
Schematic Diagram…
Machines could be virtual or physical.
Topologies are rough estimates.

<100 Users
< 10KUsers
1,000 Users

Routing &
Caching
Front-End

More & More Users

Search

Specialized

More Usage
Small Resources
Very Content
MoreLight And Simple
Workload
Large Content
Fault Tolerance
< 10M Items

Query

Batch
Processing

Multiple Index Partitions
Advanced Routing

Crawl

Database


More Workloads
Simple Workload
Evaluation
More Workloads


21

What defines your SharePoint?
WORKLOAD
Facts: purpose, services, # of users,
concurrency, rps, location
Tuning options: satellite farms, disable
services, block clients …

Facts: # of servers, hw spec, roles, network
Tuning Options: add server, scale-up, tune
settings …

Facts: #of site collections, DBs, data size

Tuning Options: split site collections,
balance content…



22

Microsoft’s 2013 Dogfood Farm
WORKLOAD

HARDWARE

15,000 users

Role

HW Spec

8,800 active users per week

Routing & Caching

VM, 4 cores, 14GB RAM

1.7M requests per day

Front End


Collab, Social, Doc Mgmt, Project

Batch Processing


Database

8 cores, 64 GB RAM

204,106 profiles in UPA
1 Web App

DATASET

SLAs (During Peak Hours)

1.3 TB total dataset size

Availability

1,001,141 documents

99.9%

10 content DBs
Largest content DB - 290 GB

Role

Avg CPU/ Mem

Routing & Caching

12% / 8 GB

511 ms

Front End

45% / 11GB

Batch Processing

80% / 12 GB

Database

11% / 46 GB

Active Users RPS
(hr)

8,297 Site collections

2,500

Largest site collection - 275 GB


Latency
(99th ntile)


23

50

Capacity Planning Resources



24

Components – Scaling Cheat sheet

Component
Search administration
Crawler
Content processing (CPC)
Analytics processing (APC)

Index
Query processing (QPC)

CPU



3

3




Network




4


5


Disk


2




Memory
1








1.

Admin: Memory footprint increase with topology size

4.

2.

Crawler: Crawled documents are temporarily stored to
disk

APC: Network bandwidth increase with scale out, but
only in-between APC nodes

5.

QPC: Network IOPS increases linearly with number of
index partitions and QPS

3.

CPC and APC are scheduled with “below normal”
priority



25

Sample Worksheet
Current Size

Recommended
storage

Notes

Configuration

2 GB

The configuration database grows very slowly. Approximate growth is 40 MB per 50,000 Site
Collections

Central Administration
Content

1 GB

Very slow growth. Not expected to grow beyond 1 GB.

Content (one or more)

???
???
???
???

•
•
•
•

Functional Area (PROD)
Portal and Self Service
Legacy Data to be migrated. When migration of legacy data is performed, database sizing and
Site Collection design has to take place.
Cross Functional Area Collaboration

Usage and Health Data

10 GB

Dependent on logging level selected

Subscription Settings Service

5 GB

Dependent on number of alerts created by users

Search Administration

10 GB

Search Administration is typically small

Crawl (one or more)

50 GB

Crawl = 0.046 x (sum of content databases)

Property (one or more)

15 GB

Property = 0.015 x (sum of content databases)

Profile

10 GB

Approximately 1 MB per user profile

Synchronization

5 GB

Approximately 650 kb per user profile

Social Tagging

2 GB

Approximately 0.009 MB per tag

Staging

5 GB

Web Analytics Service

Reporting

5 GB

Web Analytics Service

Secure Store

1 GB

Word Automation Services

1 GB

Managed Metadata

10 GB

The size of the database is affected by the number of content types and keywords.

Total
WinWire Technologies, Inc. Confidential ??? © 2010 WinWire Technologies

26

Agenda



27

Results from SharePoint 2013 Deployments
• 50% faster server response
• 4 X faster profile sync.
•

10 X faster w/ Active Directory import

• 40% more efficient use of

bandwidth
• 80% less file i/o related SQL IOPS
• More scalable SharePoint farms



28

SharePoint Farms Scale More than Ever

Product
does have
its limits



29

Limits are
for each
Dimension

Each
dimension
needs to
stay within
limits

SharePoint 2013 Performance Benchmarks

SharePoint 2010
benchmark on
SharePoint 2013

SharePoint 2010
SharePoint 2013

4 front-end VMs w/ 4 CPUs, 12 GB RAM

Green Zone: < 60% CPU on FEs, < 0.5 sec server response time

1 physical SQL Server w/ 8 CPUs, 32 GB RAM

Max Zone: 90+% CPU on FEs, < 1 sec server response time



30

SharePoint 2013 Scale Divisional Benchmark

Collaboration + social + communities Green Zone: < 60% CPU on FEs, < 0.5 sec server response time
+ meta-data navigation
Max Zone: 90+% CPU on FEs, < 1 sec server response time



31

Agenda



32

SharePoint Security Model: Conceptual

Identity

Authentication

Authorization

Identities

DS

Group

Authentication
Method

Authentication
Provider

Permissions

F

User

STS

Web App

TOKEN

Permission
Level

Role
Assignment

Site

USER POLICY

Token


R

ANONYMOUS
ACCESS & POLICY

User
Groups
Roles
Claims


D

C
DS

User

Policies

33

Security Scope

Group

End User Security Recommendations - Inheritance

Site
Role Assignment

• Roles assignments are inherited by child objects
• Subsite, list, library, folder, item or document

Library

• Best Practice
• Use inheritance wherever possible
Role Assignment

W



34

End User Security Recommendations - Inheritance
1

Web Object

Site

Scope 1

1

Document Library Object

SPGroup Object FullGP

+ FullControlGP (Full Control)
Role Assignment

Folder Object
Item 1 Object

+ User 4

+ ReaderGP (Reader)

1

Item 2 Object

+ User 3

+ ContributorGP (Contributor)

1

1

SPGroup Object ContributeGP
Scope 2

Library

+ User 1

User 5 (Reader)

+ User 2

+ User 2 (Limited Access)
Item 3 Object

SPGroup Object ReaderGP
+ User 5

1

Scope 3

+ AD Group X

User 1 (Contributor)
Scope 4
Role Assignment

Scope 5

W

AD Group X (Reader)
User 4 (Full Control)

● Roles assignments are inherited by child objects
– Subsite, list, library, folder, item or document



35

End User Security Recommendations - Fine-grained Permissions
Site
Role Assignment

Library

X
Role Assignment

W

You can break inheritance
– Assign explicit permissions
– Dynamic security changing code redesign
– Recommendation is to modify scope architecture so
that scope membership does not cause ACL
recalculation at the parent document library and
Web. Only possible using custom code.



36

End User Security Recommendations - Fine-grained Permissions

Scope 1

Site
1

Web Object
Role Assignment

Folder Object

Library

1

Document Library Object

Item 1 Object

User 2 (Reader)
+ AccessGP1 (Limited Access)

2
Scope 2

3

User 5 (Reader)

X

Item 2 Object


4

SPGroup Object AccessGP1
Scope 3
Item 3 Object

5

+ User 1
Scope 4

Role Assignment

Scope 5
AD Group X (Reader)

W



+ User 5

37

+ User 2
+ AD Group X
+ User 3
+ User 4

Agenda



38

Production Farm Disaster Recovery Architecture



39

Disaster Recovery Options
Standby
Type

Definition

Description

Pros

Cons

Hot Standby

A second data
center that can
provide availability
within seconds or
minutes.

Multiple data centers

Relatively fast to recover

Can be quite expensive
to configure and
maintain

Warm
Standby

A second data
center that can
within minutes or
hours

Virtual server images
shipped to local and
regional disaster
recovery farms

Relatively inexpensive to
recover, because a virtual
server farm can require little
configuration upon recovery

Can be very expensive
and time consuming to
maintain

Cold
Standby

A second data
center that can
within hours or days

Backups are shipped to
support bare metal
recovery to local and
regional offsite storage
on a regular basis, and
contracts are in place
for emergency server
rentals in another region

Often the cheapest option to
maintain, operationally

The slowest option to
recover


Content and services
through only one data
center


40

Often an expensive option
to recover, because it
requires that virtual servers
be configured correctly after
a disaster has occurred

Agenda



41

High availability (HA) with RBS
Effects of RBS on SQL Server Database Size

Database size (100 GB)
Database data file size
(100 GB)
Database transaction log
file size (100 GB)
Size of RBS externalized
data
Database size (1 TB)
Database data file size (1
TB)
Database transaction log
file size (1 TB)
Size of RBS externalized
data

Size (GB)
Without
With
RBS
RBS
217.2
7.0

Reduction
96.8%

106.9

3.2

97.0%

111.6

3.8

96.6%

-2,292

96.2
26

-98.9%

1,120

6.5

99.4%

1,173

20

98.3%

--

1,115

--



42

Q&A
Next Webinar
Topic: Line of Business (LOB) + Enterprise Search
When: December 05, 2013
Time: 8AM PST
Presented by: Bill Baer and Cem Aykan - Microsoft



Thank You

Reach out to us at marketing@winwire.com
Visit us: www.winwire.com
http://www.winwire.com/winwire-blog/
www.twitter.com/winwire


Planning Very Large Scale Document Repositories with High Availability in SharePoint 2013

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Planning Very Large Scale Document Repositories with High Availability in SharePoint 2013

Similar to Planning Very Large Scale Document Repositories with High Availability in SharePoint 2013 (20)

More from WinWire Technologies Inc

More from WinWire Technologies Inc (20)

Recently uploaded

Recently uploaded (20)

Planning Very Large Scale Document Repositories with High Availability in SharePoint 2013