Доклад Mitko Mitev на конференции SQA Days-19, 20-21 мая 2016 г., Санкт-Петербург

1. The Challenge of
Accepting Software

2. Agenda
• What is Acceptance Testing?
• Types of Acceptance Testing
• Areas of Application
• AT as part of different lifecycles
• Challenges
• Guidelines to success
• Outsourcing of AT

3. Agenda
• What is Acceptance Testing?
• Types of Acceptance Testing
• Areas of Application
• AT as part of different lifecycles
• Challenges
• Guidelines to success
• Outsourcing of AT

4. What is Acceptance Testing?
Formal testing with respect to:
• user needs,
• requirements and
• business processes
conducted to determine:
• whether or not a system satisfies the acceptance criteria
• whether or not the customer to accept the system.

5. Acceptance testing is…
Other Types of Testing: the intent is principally to reveal errors
Acceptance Testing
1. Measures compliancy to business objectives;
2. Expose business logic problems that unit testing and system
testing have missed out
Acceptance testing – considered as a “validation” process.

6. Agenda
• What is Acceptance Testing?
• Types of Acceptance Testing
• Areas of Application
• AT as part of different lifecycles
• Challenges
• Guidelines to success
• Outsourcing of AT

7. Types of Acceptance Testing

8. Types of Acceptance Testing
• Process of verifying that a solution works for the user.
• It is NOT System testing!
• Focus is on the journey and not on technical or system
User Acceptance Testing

9. Types of Acceptance Testing
• Checks operational readiness
• Non – functional testing.
• Acceptance of the system by the operational team, including:
• Testing of backup/ restore;
• Disaster recovery;
• User management;
• Maintenance tasks;
• Data load and migration tasks;
• Periodic checks of security vulnerabilities;
Operational Acceptance Testing

10. Types of Acceptance Testing
• Performed against a contract’s acceptance criteria for
producing custom–developed software.
• Acceptance Criteria should be defined in the contract.
• Regulation AT - performed against any regulations that must
be adhered to, such as
• government,
• legal or
• safety regulations.
Contract and regulation Acceptance Testing

11. Types of Acceptance Testing
• to get feedback from potential or existing customers before
the software product goes live.
• Alpha testing - performed at the developing organization’s site
but not by the developing team.
• Beta testing (or field testing) is performed by customers or
potential customers at their own locations.
Alpha and Beta Testing

12. Acceptance Testing
Acceptance Testing – Entry criteria
1. Business requirements must be available;
2. Application Code should be fully developed;
3. All previous Test phases should be completed;
4. No Show Stoppers or Major defects;
5. Regression Testing - Done;
6. AT Environment - Ready;

13. Acceptance Testing
Acceptance Testing – stages

14. Acceptance Testing
Test Cases - approaches:
• Requirements–based
• Business process (workflow) or user scenario
• Data–driven

15. Agenda
• What is Acceptance Testing?
• Types of Acceptance Testing
• Areas of Application
• AT as part of different lifecycles
• Challenges
• Guidelines to success
• Outsourcing of AT

16. Areas of Application
Acceptance
Testing
BFSI
Health
care
Embedded
Gaming
ERP/
CRM/
SCM/
Retail
Aerospace

17. Agenda
• What is Acceptance Testing?
• Types of Acceptance Testing
• Areas of Application
• AT as part of different lifecycles
• Challenges
• Guidelines to success
• Outsourcing of AT

18. AT in lifecycles
Acceptance Testing – in Waterfall

19. AT in lifecycles
Acceptance Testing – in a Simplified V - model

20. AT in lifecycles
Acceptance Testing – in Traditional and Agile

21. AT in lifecycles
Value & Risk - Waterfall vs Agile

22. Agenda
• What is Acceptance Testing?
• Types of Acceptance Testing
• Areas of Application
• AT as part of different lifecycles
• Challenges
• Guidelines to success
• Outsourcing of AT

23. The status of IT Projects
According to CHAOS Report - 2015

24. Why IT projects fail?

25. Acceptance Testing - challenges
Challenge 1
“I was too busy to complete the testing”:
• Users do the acceptance testing in addition to their busy
schedules as the acceptance testing is the final testing phase;
How to avoid it?
• Start as early as possible

26. Acceptance Testing - challenges
Challenge 2
“It’s too late in the project, it doesn’t matter what issues I find –
the team will never incorporate our feedback”:
• acceptance testing is the last phase when the system is
“almost” ready to go live
• acceptance testing can easily turn out to be a formality
How to avoid it?
• Users test on their own with less devotion to the project

27. Acceptance Testing - challenges
Challenge 3
“I am here, what do you want me to do?”:
• Lack of understanding how the system works
• users join late and not prepared
• Experienced and Skilled people needed
How to avoid it?
• Users should be motivated to test on their own with test cases
they have created

28. Acceptance Testing - challenges
Other reasons why it fails:
1. No collaboration
2. Focusing on how, not on what
3. Expecting acceptance tests to be a full regression suite
4. Focusing on tools
5. No management buy–in

29. Agenda
• What is Acceptance Testing?
• Types of Acceptance Testing
• Areas of Application
• AT as part of different lifecycles
• Challenges
• Guidelines to success
• Outsourcing of AT

30. Guidelines to success
Important factors:
• Real world scenarios
• derived directly from how the user operates
• built by involving users from the very beginning of the project
• As users add input, they define requirements that might have
been missed
• Fresh eyes
• have little understanding of the product,
• have no experience working with the dev and test team
• not worried about protecting relationships.
• provide real, open and honest feedback

31. Guidelines to success
Important factors:
• Client engagement
• projects are often running behind schedule
• clients are also busy
• it is very important to organize the process so that it is very easy
for the client to engage them effectively.
• Collaboration platform
• client and testers exchange information.
• not only accelerating testing but also build a better product.

32. Guidelines to success
Other success factors
1. Staff to test – choose carefully!
2. Make time to test!
3. Create new test cases!
4. AT environment: identical, but separate from the production!
5. Realistic Test data – not “Typical”!
6. Smoke test before user has access to it. Always!
7. Avoid confusion with “regression”

33. Agenda
• What is Acceptance Testing?
• Types of Acceptance Testing
• Areas of Application
• AT as part of different lifecycles
• Challenges
• Guidelines to success
• Outsourcing of AT

34. Outsourcing Acceptance Testing
Usual model:

35. Outsourcing Acceptance Testing
Managers:
• are reluctant to lose control
• strongly believe that the internal team has a better
understanding of the end-user requirements.
Practice, though, shows that outsourcing the acceptance phase is
leading to a better output.
Why not outsource Acceptance Testing?

36. Certifications
iSQI® (the International Software Quality Institute) provides over
30 certifications in the following IT-areas
(including ISTQB, IREB, UXQB and more): www.isqi.org
iSQI

37. Questions?