SPUnite17 Big Move - Learning from the Shell O365 Migration
•
2 likes•390 views
SharePoint Unite 2017 Session
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (18)
Similar to SPUnite17 Big Move - Learning from the Shell O365 Migration
Similar to SPUnite17 Big Move - Learning from the Shell O365 Migration (20)
More from NCCOMMS
More from NCCOMMS (20)
Recently uploaded
Recently uploaded (20)
SPUnite17 Big Move - Learning from the Shell O365 Migration
- 1. A THOUSAND DAYS A Deep Dive into one Office 365 SharePoint Online Migration Project. Activity Decision Blocker Update Activity End Robert Tucker & Daniel McPherson
- 3. Migration – 1000 Days Platform Sizing 25 Microsoft teams involved 40 in-house team members 200 in-house UAT testers 497 databases 7.5K SharePoint workflows republished 10.5K Site Collections 90K Sites 166K users 17.5M ACLs, identity transformed 170M objects indexed 180TB of migrated content 20TB Start Date June 2014 BPOS-D SharePoint 2013 Target is Dedicated vNext
- 7. Pre-Requisites • Full Directory and AAD Connect • FTC Removal • URL’s Provided • Network Tasks • Site Collections Upgraded • Office 2010 -> 2013+ • IE 11 or later
- 15. Identities § Make use of the reports provided by Microsoft to identify the variance in your on-premises users, and your synchronised Azure Active directory users and groups § All users (and optionally groups) must be synchronized to AAD before migration if you want to persist use of these for permissions on your migrated content § Licenses should be applied to all users. NB: In general licence checking for SharePoint Online (does not apply to Exchange or OneDrive) may not initially be enabled but must eventually be enabled to ensure compliance § Users that you identified who are in the Variance Report are users that will no longer be accessing SharePoint content. These will not be mapped resulting in these accounts being unable to access the new environment § Accounts listed within the Variance Report must be validated against the Security Identifiers (SIDs) only. Ensure you exclude historical SID’s of deleted accounts.
- 16. The principal goal of Identity Migration is to synch all desired users and groups in your AAD and produce a Validated Identity Mapping File and Skip List used in the Migration • Questionnaire • AAD Setup and Synch Timeline • Schedule Review Kickoff • Initial AAD Synch • Initial Scan to understand Identity Variance w/Users & Groups 9 months to Migration • Identity Variation Report #1 10 weeks to Migration • Identity Variation Report #2 8 weeks to Migration • Identity Variation Report #3 4 weeks to Migration • Identity Variation Report : FINAL 1 week to Migration • Validated Identity Mapping File and Skip List Migration Weekend The Identity Variation Report (IVR) is a new information tool that surfaces unique IDs (aka Identity Objects, SIDs) that Microsoft is uncertain about whether to migrate or skip. The goal of the IVR is to facilitate analysis, review and final customer disposition about whether to migrate the SID (SID is in mapping file) or skip the SID (SID is not in the mapping file) AAD Identity Migration Process Overview
- 17. Typical Permissions Mappings for SP2013 to Office 365 SPO back User Now Mapped To Expected after mapping on SPO Everyone [c:0(.s|true] Everyone [c:0(.s|true] Everyone [c:0(.s|true] NT AuthorityAuthenticated Users [c:0!.s|windows] NT AuthorityAuthenticated Users [c:0- .f|rolemanager|spo-grid-all- users/db1e96a8-a3da-442a-930b- 235cac24cd5c] Everyone except external users [c:0- .f|rolemanager|spo-grid-all-users/db1e96a8-a3da- 442a-930b-235cac24cd5c] ACME [c:0ǿ.t|partners|pdo] ACME [c:0ǿ.t|partners|pdo] DG-ACME-ZZ-SPO-EXTRN-ACME-CORPORATION [c:0-.f|rolemanager|s-1-5-21-3335339047- 1235679043-2628806996-288605] – a newly created Azure Active Directory Group. All Authenticated Users [c:0(.s|true] All Authenticated Users [c:0(.s|true] Everyone [c:0(.s|true] All Users – External (Custom Claim) [c:0!.s|trusted%3apartners] All Users – External [c:0!.s|trusted%3apartners] <Will be removed – Custom Claims are not supported>
- 19. Connected Applications § What is a “Connected Application”? § Why did they need remediation? (192) § Azure AD Auth versus ACS § Least privilege.
- 20. back
- 21. RECAP
- 22. Keeping Our Eyes On Storage 2015 View Point – Take Away Monthly Growth Delta 1.05TB Run out Estimate July 2016 Go Live Data Oct 2015 We needed to ensure that the glidepath for storage was in line with our predicted migration date to Office 365. 1. BPOS-D tenant extracts were not complete enough and did not represent the true statistics on available or used storage 2. Recovered storage from site deletions and splits-shrinks was not consistent nor did it represent the storage which was being ‘deleted’
- 23. Predicted Storage Through 2016 115 135 155 175 195 215 235 255 Sep-15 Oct-15 Nov-15 Dec-15 Jan-16 Feb-16 M ar-16 Apr-16 M ay-16 Jun-16 Jul-16 Aug-16 Sep-16 Oct-16 Nov-16 Dec-16 Jan-17 Feb-17 TB Used and Reserved Storage Month Predicted Storage Through 2015 - Week 28 - 2015 STORAGE Week 28 Jul-16 Monthly Growth Delta 1.15TB Run out Estimate July 2016 Go Live Data Oct 2015 Oct-15
- 24. Things to Pay Attention to § Ensure you minimise reserved storage § Ensure you have visibility on content database size – helps to know if advanced remediation will be required – work with Microsoft on this § Expedite housekeeping and remove stale sites (or archive them) § If you plan large scale data migration to SPO after migration – inform Microsoft well in advance of the migration volumes to ensure they build out your target platform correctly based on your predicted or planned volumes back § The advice is to choose automated storage management on the tenant rather than manual management – simplifies your administration. § Understand how storage reporting is shown on SharePoint Online site collections – its slightly different than on-premises storage metrics
- 25. Predicted Storage Through 2017 115 135 155 175 195 215 235 255 Sep-15 Oct-15 Nov-15 Dec-15 Jan-16 Feb-16 M ar-16 Apr-16 M ay-16 Jun-16 Jul-16 Aug-16 Sep-16 Oct-16 Nov-16 Dec-16 Jan-17 Feb-17 TB Used and Reserved Storage Month Predicted Storage Through 2015 - Week 47 - 2015 STORAGE Week 47 Apr-17 Sep-17 Monthly Growth Delta 1.16TB Run out Estimate April 2017 Go Live Data Sep 2016
- 26. Keeping Our Eyes On Issues 2015 View Point – Take Away Monthly Growth Delta 1.05TB Run out Estimate July 2016 Go Live Data Oct 2015 We had several high risk, high impact issues and a lot of medium risk, medium impact. Difficult to close: 1. Issues were being added faster than we were clearing them 2. Revelations about storage, the platform, connectivity, search, permissions, new capabilities, removal of old capabilities, and the impact on landscape complexity with SharePoint Add-Ins were all contributing.
- 40. DTAP DEVELOPMENT, TEST, ACCEPTANCE AND PRODUCTION. § O365 has not been designed with DTAP in mind. § Why are you implementing DTAP? § What implementation protects you from what risks? § Why did Shell implement DTAP? § Slows things down, tooling may be required.
- 42. Large lists and the “Happy Hour” § First a bit of background. § Don’t believe everything you read. § Myths and misinformation. § Types of Threshold (List versus Lookup). § Key to it is: § Column Indexing § Metadata Navigation § Office
- 44. • 8785 lists > 5000 items • 57,000 views which were impacted and needed remediating • Fix consisted of: Adding indexed fields to columns based on the view definitions • Collect all view definitions • Define the new indexed fields • Index the fields and update the views • 5000 views which could not be fixed • Approx. 26,000 would work but could be fixed quickly • Approx. 25,000 required fixing or were not impacted • Across 10,000 site collections, 90k sites. Scale of the Problem
- 52. Workflows § Understand and work with the Microsoft reports to identify all workflows which may be impacted. § To avoid unnecessary workflow restarts it is best to complete in- flight workflows before the migration event when your content is moved to the vNext environment. § Follow the advice provided in the Microsoft reports as part of your migration preparation – especially on which workflows will be impacted and how you can remediate them
- 53. Prepare for Migration – 2010 Workflows Extract From Microsoft Documentation § All workflows with email activity will need manual remediation. § Where identities are present, follow Microsoft guidance for scenarios where (this list is not exhaustive): § The activities with the identities were processed on SPO-D prior to migration § If the workflow instance has an invalid email § The workflow shows as In Progress, but doesn’t progress § If the workflows show Error Occurred § Where Conditional rules exist § Where the workflow is Checked Out § The account which published the workflow is no longer present § Emails which use mail enabled security groups § Activities that Embed Identity – 2010 (not exhaustive) § Send an Email § Look Up Manager of a User § Assign a Form to a Group § Assign a to-do Item § Collect Data From a User § Created by a Specific Person § Modified by a Specific Person § Person is a valid SharePoint User
- 54. Prepare for Migration – 2013 workflows – Extract From Microsoft Documentation § All workflows with email activity will need remediation. You must manually correct this within the workflow. § No workflow state information is retained e.g. they will all stop after migration § Workflow history - workflow history and task history are not retained. § Identity transformation for email accounts is required § For read only mode during migration, Workflow status is no longer accessible. Activities That Embed Identity - 2013 § Send an Email § Look Up Manager of a User § Assign a Form to a Group § Assign a to-do Item § Collect Data from a User § Created by a Specific Person § Modified by a Specific Person § Person is a valid SharePoint User § Start Approval Process § Start Custom Task Process § Start Feedback Process
- 55. Summary Comparison of Workflow Impacts back 2010 Workflows § History will be retained § Workflow logic § With User Identity information WONT WORK § Without User Identity information – WILL WORK § Workflows with an user impersonation step persists identity of the user account that published the workflow 2013 Workflows § History, although migrated, no longer associated with Workflow § Workflow logic § User Identity information will be broken § Will lose state information § Must be restarted post migration § UAT Testing is available for both 2010 and 2013 Workflows § Final Migration – Testing of 2013 Workflows occurs after DNS cutover § Change driven by an architecture Change for 2013 Workflows § Reports are available from Microsoft about statuses of workflows – use these!
- 56. Plan for UAT 4 Most working - new problems discovered from UAT1/2 & 3. For Mar Go - We plan another dry run in Feb 17 Dec-16 Workflow Problem Extends Discover workflow migration impact (they would stop) for any workflow with 2 other conditions (15k) Nov-16 All connected app firewalls configured for UAT Complete implementation of firewall changes for all connected apps and testers in global locations Nov-16 Delay Migration Migration postponed: Businesses had critical process in process and could not defer them - therefore migration in Dec was postponed to Mar 17 Dec-16 UAT Search still not 100% Key connected apps still cannot test. Dec-16
- 57. Search § Biggest single impact for migration is the re-indexing time – work with Microsoft to understand what that will look like § Know your search configuration § Understand what will be available within your UAT environment to prepare your business testing
- 58. Things To Pay Attention To § Implementing hybrid search early may prevent usage of your target Dedicated Office 365 tenant for UAT Testing § Understand and document your search configuration. Configuration should be checked during each UAT for all managed property mappings § Custom properties should be revalidated (there is a limit on SharePoint Online of the number of custom managed properties allowed) back § Prepare for re- indexing time – full results may be unavailable for a certain period depending on your migration approach and source tenant size § Migration of custom search landing pages will require careful handling
- 62. Know Your § Critical Business Processes § Environment and how its Unique § Permission Model § Critical Tools § Critical Features Test Plans Key to Success Do § Provide Critical Scenario Coverage § Work with Release Managers § Ask Others about their Experiences § Supply business critical items Do Not § Assume it will work § Assume it has already been tested § Test using admin accounts only
- 63. Things to pay attention to § It is important to identify a team of business and technical users that can support UAT as testers § Allocate enough time depending on the size and complexity of your environment § Improper user acceptance testing can result in missed deadlines, wasted resources, and added cost. § Ensure you re-check any issues highlighted in previous iterations of UAT’s § Ensure connectivity to your target UAT environment. § Ensure all connected apps can authenticate to the tenant (see connected apps) § Ensure your testers are aware of their responsibilities and predicted outcomes. § Have a valid test plan for standard capabilities, customised sites, applications, connected apps § Use all of the UAT availability time § Capture and address each finding with Microsoft or your business teams
- 64. Test Plan Your Test Plan should include the following categories for UAT: § Schedule § Coverage § Personnel (call out any 3rd parties you may need to rely on to support or conduct your testing) § Focus areas (categorizing the use-cases as one of either business critical or non-business critical) § Administrator test cases § Functional business test cases § Device test matrix back § Remediation § Status reporting § Sign off criteria and process § Communications § Accessing the UAT environment § Problem step recorder § Acceptance criteria § Sign off
- 66. Key Takeaways from this project • Understand your current environment • Work with Microsoft to remove all blockers and meet the Office 365 requirements – get access to the migration preparation reports and the migration advice from Microsoft as early in your timeline as you can • Execute some serious testing and document and re-test through your UAT’s • Establish your exposure to the workflow impacts and remediate early – develop parameterised workflows where identities are removed • Engage with business early on the impact of the migration event • Establish early on your approach to Office 365 connectivity • Ideally, keep your existing environment as free of customisations as possible • Get good advice on large lists remediation, and understand your permissions landscape in respect of user accounts and Active directory groups • Plan for re-indexing time for search and prepare business for the impact
- 68. Web App Policy • Work towards removing policies in SPO-D • Policies are collapsed if migrated • World is different in the destination - Single web app with Host Header Site Collections WHY? • No ability to edit policies in vNext – Only remove policies • Unintended access issues Recommend • Start this process early!
- 69. App Catalog • Web apps are collapsed on migration – does not affect instances • Customers need to choose what URL will hold the site • Any app not registered in the catalog of records needs re-installed in DvNext • Only those registered in vNext will be available for consumption in vNext Why? • You can only have 1 App Catalog in DvNext Guidance: • Choose the URL that contains the most Apps in your environment
- 70. WorkFlow • Testing in place is available for both 2010 and 2013 Workflows • Final Migration – Testing of 2013 Workflows occurs after DNS cutover • Architecture Change for 2013 Workflows • Reports are available in the SSP about statuses of workflows
- 71. Comparison of Workflows • History, although migrated, no longer associated with Workflow • Workflow logic • User Identity information will be broken • Will lose state information • Must be restarted post migration • History will be retained • Workflow logic • With User Identity information – WILL NOT WORK • Without User Identity information – WILL WORK • Workflows with an user impersonation step persists identity of the user account that published the workflow
- 72. Full Trust Code • Is it gone yet on your 2013 D farm? • Sandbox solutions – No longer supported in D or DvNext as of December 2017 • ISV Free • CAM Apps ready
- 73. Test Plans = Win Plans Know Your • Critical Business Processes • Environment and how its Unique • Permission Model • Critical Tools • Critical Features Do • Provide Critical Scenario Coverage • Work with Release Managers • Ask Others about their Experiences • Supply business critical items Do Not • Assume it will work • Assume it has already been tested • Test using admin accounts only
- 74. S.M.A.T. • The SharePoint Migration Assessment Tool • Download it >here< • NB: An additional tool will be released later this year which will also help identify issues with mapping of identities on your SharePoint farm for SharePoint Online. The SharePoint Migration assessment tool (SMAT) is a simple command line executable that will scan the contents of your SharePoint farm to help identify the impact of migrating your server to SharePoint Online with Office 365. Because the tool is designed to run without impacting your environment, you may observe the tool requires one to two days to complete a scan of your environment. During this time, the tool will report progress in the console window. After the scan is complete, you can find output files in the Logs directory. This is where you will find the summary and more detailed insights into the scenarios that could be impacted by migration. To improve the quality of Microsoft products and services, the tool will report anonymous statistical information back to Microsoft. Optionally, you can identify your organization when prompted at the end of the scan. If the tool is not able to connect to the internet to report this information, the tool will still function as otherwise expected
- 77. Preparation for Migration to D-vNext – Notable. • Selecting Hybrid search and UAT environment (C+) • MS built a separate UAT farm • Required local host files for testers • Required firewall routing rules across multiple countries • Initially UAT didn’t support workflows, emails, hybrid search • Still doesn’t support webapps (check if this has been fixed?) • Changes to connected application authentication (B-) • Identify connected apps – possible through logs but not perfect. We left a lot behind which were identified post migration • Challenge to move away from service account approach = cloud identities (we consider cloud identities a risk) • Needed to retain ACS because of lack of granularity in Azure • Readiness of MS tooling & Reporting (B+) • Identified and fixed permissions mapping issue • Identified workflow issues (2010 do migrate state, 2013 do not; any workflow with email accounts (and 3 other scenarios) need re- publishing and lose state • Reporting is important and we had specific requirements not met with the then current MS reports. Frequency and consistency is key. This is now much better • Web App policies disappear • Work with Microsoft to manage this during your migration
- 78. Preparation for Migration to D-vNext • Search and Search configuration (B-) • Hybrid search and our scale brought issues for UAT – initially we could not replicate search results across all the corpus. Some apps depended on this for validating solutions. Now Fixed. • Content DB state (A-) • We had constant resizing of content DB’s as we had some very large ones. MS tech has now been further developed to minimise the need for this. • Customisations Impact (B-) • We have a very customised end user experience on hundreds of sites. We developed a framework (we think Microsoft copied our approach with SharePoint UX!) to standardise this and drive consistency away from multiple JS files, libraries etc. Challenges on the new UX • Overall impact was not high (Access DB issues now fixed) • Authentication for Users (A-) • New Experience, STS setup and we couldn’t really test at scale the capabilities of the STS during UAT’s • We have mixed Office 2010 and Pro Plus (roll out continues) which was challenging but not high impact (more end user comms and knowledge) • Change and Comms (B+) • We were late out the gate here but it is very important to have both the correct C&C, target the right audience, get the right material. • Need multiple levels of engagement, from user experience, connected applications, customisations
- 79. Preparation for Migration to D-vNext • Identity Mapping (A-) • Process will report on user and group objects within your legacy tenant • Its required to identify those which are missing from a mapping perspective. E.g. not in local AD, not in AAD • Note: • reports uses SIDS, you need to consider that one object may have historical SIDS when checking • AD query reporting to process these Microsoft lists is required across your entire AD forest • You need AD query tools to finalise this process – take time to build a repeatable process as you do this several times per UAT • Identify site configurations which will change (B-) • Access request emails will need replacing with new vNext format [manual editing or script] • Access request options change [script] • Custom search configurations and search settings URL’s may need changing [script] • Loss of web app policies (A-) • All web app policies from all your legacy web apps will migrate (collapsed) • We found it best to remove and manage through AAD groups • Example here is eDiscovery groups • Do not rely on web app policies in AAD – target removal early and re-use on AAD groups • One group to look to retain temporarily are any DENY ALL policies (you will need to replace with license management in future) • Understand changes on SPO (B-) • auditing controls, sharing options and user experience (e.g. folders), Access requests process, managing new UX options, etc.
- 80. Recommendations • Do 3 or even 4 UAT’s • You need to do a lot of testing for customisations, connected apps, workflows, search configuration • Use your target tenant if possible for UAT (minimises local network configuration changes) • Plan for your workflows to stop in advance • Change direct inclusion of emails to parameterisation • Inform business that identified SP2010 and 2013 workflows may be impacted depending on archetype • Publicise the changes in authentication for connected apps • Stress the importance to move to tokens, clientID/secret and not rely on cloud identities • There are differences in REST API on SPO than on BPOS-D • Examine the need for URL’s within IE trusted zone for AUTHN • Performance testing is key – ensure all locations undergo performance testing to identify en-route network configurations • If you use pac files for local config – plan change well ahead • Build a comprehensive Communications & Change approach across your business • Engage early, engage big • Understand the reports you have from Microsoft and identify what you need for your migration • challenge for change! • Understand how the permissions mapping impacts if you have custom claims – especially the default implicit groups • Define your tenant configuration upfront (Site storage, OneDrive, profile permissions, SCA options) early • Plan for search index downtime – search results will take time • Ensure support teams have the right level of access post migration • Define clear policies for new features before migration • Engage through Yammer, mail, publications and set clear statements on what is allowed and what isn’t • Develop AD query and reporting skills • Gain skills in Dynamic AAD group configurations
- 81. Summary • Test, test, test • Test again • Engage with businesses on connected apps, customisations and test thoroughly • Inform business through communications • Drive policies around customisations, and new features • Re-examine workflows – assume they will all lose state if involving email, elevated permissions • Do not expect webapps to work in UAT • Check EOP configurations for inbound alerts and workflow emails • Plan AAD connect • Prepare and examine permissions mapping