Working with large lists (over 5,000 items) in SharePoint 2013 and Office 365 can prove challenging to both the administrator and the end use without devising a plan for effectively displaying data. This introductory to medium-level presentation explains why SharePoint has a list item view threshold and why it’s a bad idea to ignore it. Several examples will be demonstrated for how to provide the best UX (user experience) for users while maintaining performance in the farm. Some highlights of what we will discuss include: · Creating filtered views by using simple and compound indexes. · Using managed metadata and key filters to assist users in finding what they need. · Leveraging client-side tools such as Excel and Access 2013 to work with and report on lists and libraries · Effective use of folders for managing large libraries · Managing and monitoring thresholds through Central Admin, PowerShell and REST SharePoint Consultant Accudata Systems, Inc. www.accudatasystems.com twitter: @sharepointfly blog: http://get-spsteve.com This topic was presented at SharePoint TechFest Houston on Oct. 24, 2015.

1. Working with
Large Lists in
SharePoint 2013
WHY YOU NEED TO PAY ATTENTION TO THRESHOLDS AND METADATA

2. Steve Stewart
Accudata Systems, Inc.
7906 North Sam Houston Pkwy W | Suite 300 | Houston, TX 77064
Direct: 281.897.6394 | Fax: 281.897.5001
www.accudatasystems.com
blog.get-spsteve.com
@SharePointFly
sstewart@accudatasystems.com

3. Services we offer:
• Advisory Services
• Data Center Infrastructure
• Security and Mobility
• Managed Services
• Enterprise Applications
• Network
• Unified Communications
• Assessment & Compliance

4. Feedback & Other Important Stuff
 Use #HoustonSPTF
 Register to use the Mobile App at SPTF2015.com to give presentation
feedback
 Afternoon Reception runs from 2:45-3:30 p.m. in the Exhibit Hall.
Refreshments are free and there will be prizes!

5. Have you ever seen this?

6. List View Threshold
 The List View Threshold is set by default to 5,000 items for a web
application in Central Administration
 Administrators with special permissions have a default list view
threshold of 20,000 items
 Must be administrator on local server and on farm
 Several other thresholds also set (List View Lookup, Permission, etc.)
 Administrative Window – Time period for working on large lists
without limit

7. Central Admin - Resource Throttling
Resource throttling settings are accessed in Central Admin from
Application Management>Manage Web Applications>Web Application>Settings

8. CA Resource Throttling Settings

9. Some Important Notes
 The List View Threshold calculation includes items in the
Recycle Bin. If you delete 1,000 items, they still count
against the item limit until they are deleted from the
recycle bin.
 Search is not affected by the threshold. You can always
search for items and get results. Even if the list is well over
the threshold
 A common misunderstanding is that a view that has
been filtered to contain less than the threshold number
of items will display items when the list exceeds the
threshold. Only views filtered with indexed columns work
this way.

10. What Actions are Blocked When
the Threshold is Exceeded?
Operation Description
Add/remove/update a list column
All columns, including lookup and calculated columns, in addition
to many kinds of updates, such as a type change or a uniqueness
change. Some updates, such as a name change, are not
blocked because they do not affect every item in the list.
Add/remove/Update a list content type
Affects every item in the list so it is blocked for any list that has
more items than the list view threshold.
Create/remove an index
Affects every item in the list so it is blocked for any list that has
more items than the list view threshold.
Manage files which have no checked-in version
A non-indexed recursive query that fails for any list that has more
items than the list view threshold.
Non-indexed recursive queries
Includes filters and some sorts. This operation fails when the list size
is greater than the list view threshold. Because there is no index, it
does a full scan against the entire list. Also, it returns all items, and
it ignores folders.
Cross list query
Includes queries by the Content Query Web Part and follows the
list view threshold setting for auditors and administrators, which by
default is 20,000 items. If the operation involves more than 20,000
items, the query fails.

11. Blocked Operations, cont.
Operation Description
Lookup columns that enforce relationship behavior
You cannot create lookup columns that enforce relationship behavior
when the list that is referenced contains more items than the list view
threshold.
Deleting a list
Affects every item in the list so it is blocked for any list that has more items
than the list view threshold.
Deleting a site
If the sum of all items in a site is greater than the list view threshold,
deleting the site is prevented because it affects too many items.
Save list as template with data
Affects every item in the list so it is blocked for any list that has more items
than the list view threshold.
Showing totals in list views
Performs a query against every item in the list so it is blocked for any list
that has more items than the list view threshold.
Enable/disable attachments in a list
Affects every item in the list so it is blocked for any list that has more items
than the list view threshold.

12. What’s Happening in a Nutshell
 List View Threshold is a SQL limitation, not a SharePoint limitation
 To ensure accurate updates without affecting other users accessing the
same table, SQL locks the row being updated
 When more than 5,000 rows need to be locked due to the same query,
it is more efficient for SQL to lock the entire table … that is all the rows.
 It’s there to ensure optimized performance for ALL users
 So think of it this way. SQL normally eats one tiny piece of pie at a time
except when there are more than 5,000 pieces ordered. In this case,
SQL eats the entire pie!

13. “
”
So, just up the
number. SharePoint
can handle it right?
Actually, it’s not really a good idea to increase the
threshold number.

14. List View Threshold Throughput
0 5 10 15 20 25 30 35 40 45 50
5,000
10000
50000
100000
Request per Second
ListViewThreshold
RPS*
*Values are approximate and for illustration only.
As the size of the list threshold increases,
the number of requests per second
SharePoint can handle decreases

15. Does this mean my list can’t be larger
than 5,000 items? Of course not.
Feature Max Value Type
Number of items in a list
or library
30 million Supported Limit
Size of an individual file
item or item attachment
2 Gigabytes Boundary

16. Boundaries, Thresholds, and
Supported Limits
 Boundary - An absolute that cannot be exceeded by
design
 Threshold - A value that is set to a default and cannot be
exceeded unless the default is changed
 Supported Limit - Values determined by testing and
which represent known limits of the product. Exceeding
these limits may result in performance degradation,
errors, and other problems.

17. Thresholds and Limits
Threshold
or Limit
Default
value
Description
List View Threshold 5,000
Specifies the maximum number of list or library items that a database operation, such
as a query, can process at one time. Operations that exceed this limit are blocked.
To give you time to make alternative plans, You are warned on the List Settings page
when your list has exceeded 3,000 items. The warning contains a help link to this topic.
Unique permissions
limit
50,000
Specifies the maximum number of unique permissions allowed for a list or library.
Every time you break the inheritance of permissions for an item or folder, it is counted
as 1 unique permission toward this limit. If you try to add an item that would lead to
exceeding this limit, you are prevented from doing so.
Row size limit 6
Specifies the maximum number of table rows internal to the database used for a list
or library item. To accommodate wide lists with many columns, each item is wrapped
over several internal table rows, up to 6 rows and up to a total of 8,000 bytes
(excluding attachments).
For example, if you have a list with many small columns, one that contains hundreds
of Yes/No columns, then you could exceed this limit, in which case you would not be
able to add more Yes/No columns to the list, but you still may be allowed to add
columns of a different type.
Note Administrators can only set this limit by using the object model, not through the
user interface.

18. Thresholds and Limits
Threshold
or Limit
Default
value
Description
List View Lookup
Threshold
8
Specifies the maximum number of join operations, such as those based on lookup,
Person/Group, or workflow status columns.
If the query uses more than eight columns, the operation is blocked. However, it is
possible to programmatically select which columns to use by using maximal view,
which can be set through the object model.
List View Threshold
size for auditors and
administrators
20,000
Specifies the maximum number of list or library items that a database operation, such
as a query, can process at one time when performed by an auditor or administrator
with appropriate permissions. This setting works in conjunction with Allow Object
Model Override.
Allow Object Model
Override
Y
Specifies whether or not developers can perform database operations, such as
queries, that request an override of the List View Threshold to the higher limit specified
by the List View Threshold for auditors and administrators. An administrator must
enable an object model override, and then developers with appropriate permission
may then programmatically request that their query use the higher List View Threshold
to take advantage of it.

19. Daily Administrative Time Window
 Specifies a time period during which resource thresholds and limits
are ignored. An administrator can configure a time window during
"off-peak" hours in 15 minute increments and up to 24 hours, such as,
6:00 PM to 10:00 PM or 1:30 AM to 5:15 AM.
 Note A database operation or query started within the daily time
window continues until completion (or failure) even if it doesn't finish
within the specified time window.
 Time can be used to run queries and conduct other administrative tasks
such as indexing columns, creating views and deleting large numbers of
items.
 The administrative window is set in Central Administration for a web
application

20. Is the Threshold Really All or Nothing
for the Web Application
 Yes and no.
 You can only change the threshold for the entire web application using
Central Admin
 But, you can also use PowerShell to disable the threshold for a single list
$web = Get-SPWeb https://domain.com/web/
$list = $web.Lists[“List Name"]
$list.EnableThrottling = $false
Additional PowerShell commands are available for setting List thresholds at the web
application level. See MSDN for more information:
https://msdn.microsoft.com/en-
us/library/office/Microsoft.SharePoint.Administration.SPWebApplication_properties.aspx

21. REST API
 You can use the REST API to quickly view the list view threshold
settings for a particular site collection without opening Central
Admin by accessing the following URL:
 <site collection URL>/_api/site/maxItemsPerThrottledOperation

 Results will be returned that look like this:

22. Demonstrations

23. Views with Indexed Columns
Supported Column Types Unsupported Column Types
Single line of text Multiple lines of text
Choice (single value) Choice (multi-valued)
Number Calculated
Currency Hyperlink or Picture
Date and Time Custom Columns
Lookup (single value)
Indexing the column in the other
list or library does not improve
performance of the lookup
operation.
Lookup (multi-valued)
Person or Group (single value) Person or Group (multi-valued)
Yes/No External data
Managed Metadata

24. Filter Ideas
To filter by: Index the: For example:
Recently changed items Modified column
To view only items that have changed in the past week, apply the filter
Modified is greater than [Today]-7.
New items Created column
To create a view of only those items that were added in the past week,
apply the filter Created is greater than [Today]-7.
My items Created By column
To create a view of only those items that you added, apply the filter
Created By is equal to [Me].
Items due today
Due Date column
(created by you in a list
or library)
To create a view of only those items with a due date of today, apply
the filter Due Date is equal to [Today].
Discussion board updates Last Updated column
To create a view of only the discussions that were updated in the past
month, apply the filter Last Updated is greater than [Today]-30.
Archive files in a document library Date Modified
To create a view of documents not changed since the year 2006 (that
you then want to archive), apply the filter Date Modified is less than
December 31st 2006.
Find a subset of financial data
Region, Year (as two
simple indexes)
To create a view of financial data for the NorthEast region in 2008,
apply the filter Region equals "NE" AND Year equals 2008 because the
Region column is likely to have less values than the Year column.

25. Index a Column and Create a
View Based on it (Demo)
1. Go to list > List Settings
2. Scroll to the bottom of the
columns list
3. Click Indexed columns
4. Click Create a new index
5. Select Columns to index
6. Normally, you will pick only a
primary column from the drop
down. However, you may with
some fields select a secondary to
create a compound Index. For
example, Modified by and
Modified (date). Compound
indexes can be used with
managed metadata.
7. Click Create (max 20 indexes)
8. Indexed columns appear in View
builder filter area in the Indexed
Columns group.

26. Create Index Columns

27. Using InfoPath to Break the Rules
Trick: You cannot use a
calculated column as an indexed
column, but you can calculate a
value in InfoPath and use a rule to
transfer the value to a supported
column type.

28. Use Key Filters
 Key Filters work with managed metadata to help parse a list
 Prerequisite for using this feature is activation of the Metadata
Navigation and Filtering site feature.
 Fields available for Key Filtering are:
 Content Type
 Choice Field
 Managed Metadata Field
 Person or Group Field
 Date and Time Field
 Number Field

29. Steps to Create a Key
Filter
1. Go to List
Settings
2. Click Metadata
Navigation
Settings
3. Add Navigation
Hierarchies
4. Add Key Filters
5. Click OK

30. Folders – mainly for libraries
 You can divide items into folders to get around view thresholds
 This is best suited for document libraries
 Either use initial view with all folders visible or create navigation (quick
launch) to go directly to folder
 Use Per Location View Settings to customize default and available views
in folders
 In lists there is no default way to move items into folders after creation
 Document Center

31. Key Filters Get Us Part Way There,
But an Index is Still Needed

32. Datasheet
1. From the Ribbon, click Create View
2. Select Datasheet View as the view type
3. Under View Settings, expand folders and select Show all items
without folders
4. Expand Item Limit and select either Display All Items or Limit the
total number of items returned to option.
5. Type 30 in the box below it (this number must be below the list view
threshold)
6. Save the view
7. Click OK when you receive the warning about flat views.
8. Edit the view again, now change

33. Data Sheet View Settings
 Remember unless you are referencing indexed columns, no filters,
sorting, totals, etc., can be selected.

34. Export to Excel
 From the List/Library Ribbon click Export to Excel
 You can use the Export to Excel feature to create a read only
version of your list which can be used to perform queries
 Remove unwanted columns from the Excel worksheet without
affecting your view or the list – columns will not reappear upon
refresh
 Use the Refresh button to get new additions to the list.
 You can use Excel tools to generate reports, etc.

35. Open in Access
 You need Access 2013 to use this feature in SharePoint 2013
 From the Ribbon click Open in Access to create a client that can be
used for:
 CRUD (Create, Read, Update, Delete)
 Creating forms
 Queries
 Reporting

36. Even More Powerful – Access Views
1. From the Ribbon select Create View
2. Select Access View

37. Select View Style
3. Select an Access View Style
4. Click OK
5. Make adjustments to the form
and list design, as desired.
6. Use Access to update lists.

38. Save the Access View App
Anywhere
 Users can update your list from another site collection
 You can control permission based on the location where the
document is saved
 Think about adding a link to the app on the Quick Launch
 Users can download a copy for more efficiency

39. Questions

40. References
 Designing large lists and maximizing list
performance
 Techniques for managing large lists
 Manage lists and libraries with many items
 SharePoint 2010/2013: List view Lookup threshold
uncovered
 Software Boundaries and Limits for SharePoint
2013

41.  © Copyright 2015 Stephen D. Stewart. All rights reserved.