Market Trends: What new developments are shaping the way teams work? Replacing HP Quality Center?: What hurdles are typically faced in replacing legacy Test Management? Moving Beyond HP Unified Functional Tester?: What options exist to move to more modern automation tools? Migration Best Practices: How are leading companies making the switch?

1. Modernizing Your Testing
Tools
Kevin Dunne & Antony Edwards | 10.11.2017

2. INSERT PHOTO ON TOP
Change before you have to.
Jack Welch

3. Agenda
Market Trends
• What new developments are shaping the way teams work?
Replacing HP Quality Center?
• What hurdles are typically faced in replacing legacy Test Management?
Moving Beyond HP Unified Functional Tester?
• What options exist to move to more modern automation tools?
Migration Best Practices
• How are leading companies making the switch?
Q&A

4. Market Trends

5. The percentage of respondents’
organizations that practice agile
2 0 15 95%
HOW MANY?
&
Percentage of Teams Using Agile
Only 4% of respondents work in a completely
traditional / non-agile development organization.
Contrast this with the 2009 report, in which (31%) of the respondents
worked where there were two teams or less practicing agile!
53%
Less than ½ of our
teams are agile
34%
More than ½of
our teams are
agile
9%
All of our
teams are agile
4 %
None of our
teams are
agile
TOP 3 BENEFITS
87%
Ability to manage
changing priorities
85%
Increased team
productivity
8 4 %
Improved project
visibility
*State of Agile, 2016 Report, VersionOne
Agile is Mainstream

6. 65%Of software developers, manager, and
executives report that their organizations
have started down the path to
ContinuousDelivery.
Faster time to market
Better quality of product
Competitive advantage
Customer satisfaction
Reduced cost of development
Ranking of Benefits
*https://www.perforce.com/pdf/continuous-delivery-report.pdf
1
2
3
4
5
DevOps is Catching On

7. HP - Legacy “End to End” Incumbent
Tools Amount of Companies
HPQC 4,618
HPQC + JIRA 1,953
HPQC + Unified Functional Tester 2,176
HPQC + Unified Functional Tester + LoadRunner 1,398
HPQC + Unified Functional Tester + LoadRunner + JIRA 901

8. HP Acquisition By Microfocus
- Gartner Report, 11/10/16
GARTNER SUGGESTED IMPACTS
● Adding Hewlett Packard Enterprise's (HPE) ITOM and ADM offerings to Micro Focus' portfolio creates
significant, complex and uncertain overlaps that I&O leaders must plan for now.
● Customers basing their information management and governance strategies on HPE's products
should carefully monitor investment, as long-term strategies under Micro Focus are unknown.
● Spinning off HPE's security software portfolio to Micro Focus creates uncertainty for I&O leaders about
future strategic investments in those products
GARTNER RECOMMENDATIONS
● Create migration options from HPE ITOM and ADM software...

9. Shifting to Best of Breed
The market is shifting to more agile best of breed tooling from HP
incumbent (stats as of 2012, for Australia):

10. Explosion of Tools

11. Best in Class JIRA Integration
Superior Customer Support
Robust Automation Support
Better Usability
Product Roadmap and Vision
In The Cloud On-Premise
Enterprises Demand Modern TCM

12. New Age Demands New Automation

13. Benefits of Best of Breed
Smaller, more agile
best of breed
vendors supporting
the latest
technology and
trends to keep you
up to date
Ability to tailor
solutions to meet the
needs of individual
teams and
applications, with
better commercial
support and
partnerships
Today’s
technological
advancements
make it easier than
ever to integrate the
various elements of
a solution
Modern
Functionality
Solve Specific
Problems
Leverage
Modern API’s

14. Replacing HP Quality Center

15. Teams Face Challenges Migrating
Transferring
Attachments
Splitting
Objects into
Systems
Live Migration
vs. One Time
Migration in
Phases
Switching
Deployment
Supporting
Traceability
Migration at
Scale

16. QAS Offers a Wealth of Migration Tools
Start Fresh
HP Migration
Tool
Excel Import Custom

17. Starting Fresh
Many more companies than you may imagine would decide to start fresh on a
tool, leaving behind their data in the old system:
Pros
• No cost of migration
• Utilize features in new tool
Cons
• Need to recreate any objects
required
• Lose audit trail (or need to maintain
old system)
Good For: Smaller teams in non-regulated industries, with few relevant
objects

18. Excel Import
Like many other tools, QASymphony provides the ability to import objects via
excel upload
Pros
• No cost (but time consuming)
• Migrate major objects and folders
• Some ability to do a live migration
(although kludgy)
Cons
• Lose traceability, attachments
• Must be done 1 project at a time
• Cannot import test labs, results
• Risk of missing/incomplete data
Good For: Smaller teams needing only migrate 1 type of object (usually test
cases), with little or no attachments

19. HP Migration Tool
QASymphony partners with several partners to deliver a tool assisted
migration via our proprietary migration utility
Pros
• Low cost
• Complete Migration of objects and
traceability
• Ability to migrate project by project
Cons
• One-time migration, need to cut
over
• Cannot support a big bang
approach, about 5-10 projects/day
Good For: Larger teams looking to migrate completely, a few projects at a
time with hard cutover

20. Our HP QC Migration Experience
Migrate your testing assets to QASymphony, regardless of
deployment or database type:
Requirements Test Cases / Run
History
Linkages
Defects Users Fields / Values

21. Custom Migration
For customers that need an usual type of migration, potentially involving a high
level of complexity, there is ability for custom migration via partners
Pros
• Build an integration to
accommodate in house systems or
unusual/complex requirements
Cons
• Development lead time
• High level of prof. services cost
• Potential to introduce unnecessary
complexity into new tools
Good For: Very large teams with highly complex or unusual requirements or
home grown systems

22. Custom Migration Methods
There are a few ways to build custom migration utilities depending on the
deployment methods of each system and available access
DB to DB API to DB API to API

23. Comparison of Methods
Start Fresh Excel Import Migration Tool Custom
Transfer Attachments 𝗫 𝗫 ✔ ✔
Split into Systems 𝗫 ✔ 𝗫 ✔
Live Migration 𝗫 1/2 𝗫 ✔
Phased Migration 𝗫 ✔ ✔ ✔
Switch Deployments 𝗫 ✔ ✔ ✔
Traceability 𝗫 𝗫 ✔ ✔
Scale 𝗫 𝗫 ✔ ✔
Cost $0 $0 (+ time) $$ $$$$+

24. • Purchase date: June 2015
• Method: Migration Tool
• Package: qTest Elite
• Users: ~500
• Hosting: On-Premise
• HP QC Projects Migrated: 8
• HP QC Objects Migrated: ~200,000
• Migration Time Frame: ~2 weeks
• Migration Hands On Time: ~ 12 hours
• Training: QASymphony on site, 2 days/16 hours + additional remote
sessions
• QASymphony Product Influence: Docker installation support, support
for HP QC Migration for Version 10.x, concurrent licensing
Case Study: Nordstrom

25. Successful Test Automation

26. This Year Next Year
Test Automation is a Challenge

27. 1. Establish Key success factors for
implementing test automation.
2. Shift focus on testing the user
experience
3. Integrate into a modern
continuous testing environment.
How to Shake Legacy Automation Demons

28. Key Principle
Implementing test automation is a change
project and needs to be managed like one.
… There will be issues. Work through them.
… Don’t do too much change at once.
… Ensure someone day-to-day involved in the project
has done it before.

29. Script Creation
Source Code
Repository
CI (e.g. Jenkins)
Test Management
Test
Environment
Test Execution Engines
Test Automation Key Success Factors

30. 1. Set up your test environment.
2. Define your test architecture.
3. Set up your test management.
4. Set up your CI and automated
execution infrastructure.
5. Set up your SCM.
6. Train the team.
7. Define roles and ensure all test
activities are covered.
8. Setup relationships with other
teams (e.g. dev, ops).
Script Creation
Source Code
Repository
CI (e.g. Jenkins)
Test Management
Test
Environment
Test Execution Engines
Test Automation Key Success Factors

32. Testing Gaps
Visibility Gap
56%
Automation Gap
Can’t use test automation for a
significant part of their testing
Overall IT
budget spent on
QA
QA teams say that
they can’t keep up
30%
89%
Productivity Gap
CEO CTO
Dev Ops
User
18%
Meeting their
test objectives
Actually meeting
customer
expectations
86%
User Experience Gap
4%
App still used
after 1 month
Confidence and Predictability Gap

33. Software Has Changed

34. Software Has Changed
 Digital Experience
 DevOps
 Mobile and IoT
 Micro-services Monolithic
Architecture
Micro-services
Architecture
(Cloud(s) & Edge)

35. Test Automation is Focused Only on Test-Execution
Huge improvements to be made by expanding
automation beyond test-execution to the rest of
the testing process
Define
Test
Objectives
Define
Test
Cases
Create &
Maintain
Test
Scripts
Define
Test Runs
Setup
Test Env
Review
Test
Results
Feedback
Execute
Tests
(auto)

36. The 5 Principles of Digital Automation Intelligence
Transform testing from ‘code compliance checking’ to helping teams create amazing digital
experiences that drive user adoption, engagement, and revenue.
Test through the eyes of the user & service APIs
Continuously test all aspects of the digital experience
Expand automation beyond test execution using AI, ML, and analytics
Report quality in terms of the user with predictive analytics
0100 0001
0100 0101
0100 0101
Use data to guide all parts of the test process and create feedback loops

37. Migration Best Practices

38. Do Some Housekeeping
Artifacts
• Migration is a good time to figure out what artifacts are truly needed
Process
• Evaluate compliance, reporting and SDLC needs
Project infrastructure
• Evaluate active users, projects, integrations

39. Minimize Unnecessary Work
• Leverage prebuilt solutions that can automate the majority of the
work to be done
• Look for tools that can be customized to fit the needs of your individual
migration
• Measure twice, cut once
• Plan the process in advance to minimize unnecessary rework and reimports
• Separate “Nice to Have” from “Must Have” migration items and
migrate in priority order

40. Protect Against Scope Creep
Size Effort Up Front, Work Backwards
• Identify Timelines, Migration files sizes, Hardware needs
Phase out approach
• Make sure to get most important data migrated up front
Set Realistic Timelines
• Add 20-30% contingency for migrations, make sure to account for "dead time"
Enforce Project Plan
• If not doing live migration, set hard cutoff dates for transition

41. Execute & Iterate
Perform iterative
migrations and
validation
• Complete the
migration and testing
on each iteration
Implement SDLC
refinement
• Incorporate any SDLC
process changes that
resulted from
housekeeping
Implement
Integrations
• Integrate any tools
that is not available
out box by qTest
Monitor Team
Adoption
• Be Agile, make
changes as needed

42. Q&A