This is a free module from my course ISTQB CTAL Test Manager revised to 2012 syllabus. If you need full training feel free to contact me by email (amraldo@hotmail.com) or by mobile (+201223600207).
2. Objectives
Build on ISTQB CTFL in the area of test manager
Prepare for the ISTQB CTAL – Test Manager exam
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 2
3. Prerequisites
ISTQB CTFL or equivalent
Practical experience in SW testing
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 3
4. Notes
Ask any time.
Turn your cell silent.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 4
5. References
ISTQB CTAL – TM syllabus version 2012
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 5
6. Outline
Testing Process
Test Management
Reviews
Defect Management
Improving the Test Process
Test Tools and Automation
People Skills – Team Composition
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 6
7. Outline
Testing Process
Test Management
Reviews
Defect Management
Improving the Test Process
Test Tools and Automation
People Skills – Team Composition
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 7
8. Testing Process
Introduction
Test Planning, Monitoring and Control
Test Analysis
Test Design
Test Implementation
Test Execution
Evaluating Exit Criteria and Reporting
Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 8
9. Testing Process
Introduction
Test Planning, Monitoring and Control
Test Analysis
Test Design
Test Implementation
Test Execution
Evaluating Exit Criteria and Reporting
Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 9
10. Fundamental Test Process in ISTQB
CTFL Syllabus
Can be executed:
Sequentially
In parallel (exploratory)
Or overlapping
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 10
P
A
&
D
I
&
E
E
&
R
C
C
11. Refinement of Fundamental Test
Process in ISTQB CTAL Syllabi
Activities are considered separately in order:
To provide additional refinement and optimization of the processes
To better fit the software development lifecycle
To facilitate effective test monitoring and control
Tailoring these main activities within the context of the system and the
project is usually required.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 11
12. Refinement of Fundamental Test
Process in ISTQB CTAL Syllabi cont’d
It is important to understand the other steps in the test process.
Majority of the TM’s work usually is done during the planning, monitoring,
and control of the testing project.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 12
P
A D I E E & R
C
M & C
13. Testing Process
Introduction
Test Planning, Monitoring and Control
Test Analysis
Test Design
Test Implementation
Test Execution
Evaluating Exit Criteria and Reporting
Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 13
14. Learning Objectives
TM-1.2.1 (K4) Analyze the test needs for a system in order to plan test
activities and work products that will achieve the test objectives
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 14
15. Test Planning
Identification of:
Activities and resources to meet mission and
objectives defined in test strategy
Methods for gathering and tracking metrics that:
Guide the project
Determine adherence to the plan
Assess achievement of objectives
Tools, training and documentation guidelines based on
me metrics defined in the planning
For each level, starts @ test process initiation and
continues till closure activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 15
16. Test Strategy Affects Test Planning
Strategy(ies) determine tasks that should occur during the planning.
In risk-based testing strategy , risk analysis guides planning regarding
mitigating activities of identified product risks and help contingency
planning.
Test effort allocation
Testing levels needed
Test activities prioritization
Test techniques selection
Need for static testing
In a reactive strategy, planning for the creation of test charters and tools
for dynamic testing techniques such as exploratory testing may be
warranted.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 16
17. Examples: Risk-Analysis Guiding Test
Planning
If a number of likely and serious potential security defects identified, a
significant amount of effort should be spent developing and executing
security tests.
If serious defects are usually found in the design specification, the test
planning process could result in additional static testing (reviews) of the
design specification.
If performance is a high risk, performance testing may be conducted as
soon as integrated code is available.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 17
18. Example: DO-178 B Needed Coverage
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 18
Used in an airborne environment
Failure Level Description Testing Technique
A: Catastrophic
Failure may cause lack of critical function
needed to safely fly or land the plane
MC/DC
B: Hazardous
Failure may have a large –ve impact on safety
or performance
Decision Testing
and MC/DC
optional
C: Major
Failure is significant, but less serious than A or
B
Statement testing
@ minimum
D: Minor
Failure is noticeable, but with less impact than
C
E: No effect Failure has no impact on safety
19. Test Management PoV
Test Basis, Test Conditions, and Test
Cases Complex Relationship
Need to be understood for effective
planning, monitoring and control
Affects tool decisions
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 19
Test Development PoV
Test Object
Test Items
Test Basis
Test Conditions
Test Cases
Test Procedures
Test
Analysis
Test
Design
Test
Implem
entation
20. Relation with the Development Team
Impacts Test Planning
Development work products affect traceability and traceability matrix
requirements.
Development work products must be approved before specific testing
activities can start.
Development life cycle model determines which and how information
should be exchanged between development and testing teams.
Test plan lists SW features both in and outside testing scope w/
appropriate project formality and documentation.
TM defines initial test environment specification
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 20
21. Test Planning Considers Dependencies
External dependencies and associated SLA’s must be identified and initial
contact should be made.
Example dependencies:
Outside groups
Other projects
External vendors or development partners
Deployment team
Database administrators
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 21
22. Example: Test Planning in Sequential
Models
Has many test planning issues
that must be considered:
1. Quality and delivery trade-off
due to schedule compression @
the end
2. Unstable or untestable SW
delivered to test team
3. Failure to include all activities
shown in the model
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 22
23. Example: Test Planning in Iterative
Models
Has many test planning issues
that must be considered:
1. Regression testing
2. Planning for bugs
3. Lack of rigorous testing
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 23
24. Example: Test Planning in Agile
Models (Iterative Models ++)
Challenges
Volume and speed o change
Keeping w/ the pace
Increased regression risk
Inadequate unit testing
Poor test oracles and test basis
Scrum or scrums
Over expectation
Opportunities
Automated unit testing
Static code analysis
Code coverage
Continuous integration
Automated functional tests
Informal reviews and power o 3
Control of technical debt
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 24
25. Example: Test Planning in Spiral
Models
Has many test planning issues
that must be considered:
1. Testing must be very flexible
2. Testing objective varies along
the development lifecycle (early
testing is for knowing the
unknown not for finding
defects).
3. Unpredictable schedule
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 25
26. Test Monitoring
Providing feedback and visibility about all test activities
Efficient test control requires defining a schedule and monitoring
framework.
This framework should include detailed measures and targets needed to
relate test work products and activities to plan and strategic objectives.
Easy for small or less complex projects
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 26
27. Defining Monitoring Objectives and
Targets
Test monitoring framework must be defined a manner that is
understandable and relevant to the project and business stakeholders
(residual risks vs. defects and tests cases for example).
Test conditions are a key player as they link test basis to test work
products.
Properly configured traceability wit properly defined test conditions make
the complex relationships more transparent and comprehensible.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 27
28. Involvement of Stakeholders in Defining
Monitoring Frameworks
Sometimes, stakeholders require targets that are not related to system
functionality or specifications (common if no or lacking formal
documentation).
Involvement stakeholders @ early stage help define measures and targets
that provide better control and influence testing activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 28
30. Test Control
Ongoing comparing of actual progress against plan and implementing
corrective actions when needed
Guides testing to fulfill mission, strategies, and objectives
Appropriate reactions to the control data depend (decision taking, test re-
prioritizing, test rescheduling or changing entry criteria) on detailed
planning information (traceability between test basis, test conditions and
test work products).
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 30
31. Exercise: Test Planning, Monitoring
and Control
Assume, you are a TM who is planning test for a project that will follow
iterative SDLC.
Your requirements are prioritized into 5 levels (VH, H, M, L, VL).
Follow the ISTQB fundamental test process and outline the test activities
across each iteration.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 31
32. Testing Process
Introduction
Test Planning, Monitoring and Control
Test Analysis
Test Design
Test Implementation
Test Execution
Evaluating Exit Criteria and Reporting
Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 32
33. Learning Objectives
TM-1.3.1 (K3) Use traceability to check completeness and consistency of
defined test conditions with respect to the test objectives, test strategy,
and test plan
TM-1.3.2 (K2) Explain the factors that might affect the level of detail at
which test conditions may be specified and the advantages and
disadvantages for specifying test conditions at a detailed level
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 33
34. Test Analysis
Test analysis = defining “what” to test (test conditions)
Test design = defining “how” to test (test cases)
Test analysis and design can be implemented as parallel, integrated, or
iterative activities.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 34
35. Test Conditions
Identified by analysis of the test basis, test objectives, and product risks
Viewed as detailed measures and targets for success (exit criteria)
Test conditions coverage, test conditions achievement % ...
Should bi-directionally traceable
To test basis, test objectives, risks or any other project or stakeholder criteria
for success
To test design and resulting test work products
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 35
36. When and How Test Analysis?
As soon as test basis is established for a test level (test basis and test
conditions will differ according to test level)
Formal test techniques and analytical techniques can identify test
conditions
Test conditions may vary in level o detail.
Specifying test conditions in a detailed fashion will tend to result in a
larger number of test conditions.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 36
37. How much Detail in Test Conditions?
Answer depends on some factors:
Testing level
Detail and quality of test basis
System/software complexity
Risks
The relationship between the test basis, what is to be tested and how it is to
be tested
SDLC in use
Test management tool being utilized
Level at which test design and other test work products are to be specified
and documented
Skills and knowledge of the test analysts
The level of maturity of the test process and the organization itself
Availability of other project stakeholders for consultation
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 37
38. Exercise: Test Analysis
Perform a complete test analysis for the given screenshot.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 38
39. Detailed Test Conditions Advantages
Facilitates more flexibility in relating other test work products to test basis
and test objectives yielding better monitor and control eventually
Contributes to defects prevention
Relates testing work products to stakeholders in terms that they can
understand
Helps influence and direct testing and development activities
More efficient coverage of detailed measures and targets
Basis for clearer horizontal traceability within a test level
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 39
40. Detailed Test Conditions Usage
Effective in the following situations:
Lightweight test design documentation methods are being used to
accommodate SDLC, cost and/or time constraints or other factors
Little or no formal requirements or other development work products are
available as the test basis
The project is large-scale, complex or high risk
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 40
41. Disadvantages of Detailed Test
Conditions
Potentially time-consuming
Maintainability can become difficult in a changing environment.
Level of formality needs to be defined and implemented across the team.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 41
42. General Test Conditions Usage
Effective in the following situations:
CT
Less complex projects where simple hierarchical relationships exist between
what is to be tested and how it is to be tested
AT where use cases can be utilized to help define tests
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 42
43. Testing Process
Introduction
Test Planning, Monitoring and Control
Test Analysis
Test Design
Test Implementation
Test Execution
Evaluating Exit Criteria and Reporting
Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 43
44. Learning Objectives
TM-1.4.1 (K3) Use traceability to check completeness and consistency of
designed test cases with respect to the defined test conditions
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 44
45. Test Implementation
Fulfillment of the test design
Checking against explicit and implicit entry criteria
Creating automated tests
Organizing tests (both manual and automated) into execution order
Finalizing test data and test environment
Forming a test execution schedule, including resource allocation
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 45
46. Test Design
Test design = defining “how” to test (test cases) by stepwise elaboration of
the identified test conditions or test basis using test techniques defined
during test planning
Test design includes test data design and test environment design as well.
After the TA’s do their work, TM must check the design exit criteria before
moving to the next step.
Test design exit criteria include review and approval from relevant
stakeholders, coverage, cost, time ...
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 46
47. Test Design Traceability
Test cases can be related directly to test basis or indirectly through test
conditions (preferably detailed ones).
Test basis items could be individual requirements (atomic?!), risk item,
strategic objectives, test objectives or project or stakeholder success
criterion.
Traceability resolution loss should be avoided.
Must be bi-directional to allow TM to relate test results in terms of project
aspects
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 47
48. Inputs and Outputs for Test Design
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 48
Test Plan
Test
Conditions
Test
strategy
Quality
Risks
Test
Oracles
Concrete
or
Logical?
Test Cases w/
Traceability
Test Data
Design
Test
Environment
Design
TA’s
49. Concrete or Logical Test Cases?
Concrete Test Cases
Specific information
Preconditions, inputs, outputs, post
conditions, and procedures
Useful when:
Requirements are well-defined.
Testers are inexperienced
External verification like audits is
needed.
Benefit is excellent reproducibility.
Maintenance is a significant issue and
limits testers creativity.
Logical Test Cases
Guidelines for what should be tested
TAs can vary data and/or procedure
Useful when:
Requirements are not well-defined.
Testers are experienced in testing and
the product.
Formality is not needed.
Benefits are better coverage due to
variation and lower costs of creation
and maintenance.
Reproducibility is a significant issue.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 49
50. Sequential or Iterative Test
Development Process?
Decision is influenced by the test level and test strategy.
In ST, SIT and AT, it is more sequential as test basis are available much
earlier than the test execution start, while in CIT and CT, it is more
iterative.
Analytical strategies are sequential while reactive tend are iterative by
nature.
Even in sequential development, some overlap may be beneficial
especially test data generation after test design and before test
implementation can reveal problems in the test design.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 50
51. Example: IEC 61508 Test Implications
Risk-analysis is required.
Risk-based testing is a mandatory risk mitigation technique.
All risk-relevant test work products must be formally documented.
SIL mandates selection of test techniques, required structural coverage
and test levels to be used.
Reviews are mandatory even for test work products.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 51
52. Exercise: Test Design
Derive minimal number of test cases from previously defined test
conditions.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 52
53. Testing Process
Introduction
Test Planning, Monitoring and Control
Test Analysis
Test Design
Test Implementation
Test Execution
Evaluating Exit Criteria and Reporting
Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 53
54. Learning Objectives
TM-1.5.1 (K3) Use risks, prioritization, test environment and data
dependencies, and constraints to develop a test execution schedule which
is complete and consistent with respect to the test objectives, test
strategy, and test plan
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 54
56. Test Execution Schedule
Tests must be scheduled and sequenced during test implementation
taking into considerations:
Order of automated and manual test execution
Prioritization assigned to tests from requirements or risks
Execution constraints like tester or environment availability or test
environment reconfiguration
TM should carefully review schedule to avoid any test execution blockage.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 56
57. Ready for Test Execution?
Trust, but verify the test environment and test data.
If the test environment to evolve during execution, double check all changes
that will occur in accordance with the test execution schedule.
Contingencies should be defined if these anticipated changes did not occur for
a reason or another.
Tests must be reviewed and verified before running.
Double check execution schedule to make sure every thing will be in place.
Make sure SW is ready for testing.
Smoke tests are effective in checking the test execution readiness.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 57
59. Detail and Complexity
Same argument considered for test conditions apply for test cases, test
data description and test environment description.
Using testware as a way of system documentation, testware reuse or
having testers w/ different skill set mandates documenting more details.
Regulatory rules may also mandates documentation details.
Project complexity can also determine level of details needed in
implementation.
TM must defined implementation level of details to guide testers during
implementation.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 59
60. Early Test Implementation Pros
Concrete tests provide worked examples of how SW should behave, if
written in accordance with test basis.
Business domain experts verify concrete tests easier than abstract
business rules, and thereby identify further weaknesses in SW
specifications.
Verified tests may provide illuminating illustrations of required behavior
for SW designers and developers.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 60
61. Early Test Implementation Cons
Changing test basis may lead to change in the testware already developed.
In Agile SDLC, embracing changes is the norm!.
In sequential SDLC, poorly managed projects can lead to unhappy
surprises.
Before embarking on an extensive test implementation, it is wise to
understand SDLC and the predictability of SW features that will be
available for testing.
In conclusion, late implementation may be favored.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 61
62. Testing Process
Introduction
Test Planning, Monitoring and Control
Test Analysis
Test Design
Test Implementation
Test Execution
Evaluating Exit Criteria and Reporting
Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 62
63. Learning Objectives
TM-1.6.1 (K3) Use traceability to monitor test progress for completeness
and consistency with the test objectives, test strategy, and test plan
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 63
64. Test Execution
Begins once test object is delivered and entry criteria to test execution are
satisfied (or waived carefully)
Criteria examples are:
Tests should be designed
Tools should be in place
Test results tracking should be working and tracked data is understandable by
all team members.
Standards for test logging and defect reporting should be available and
published.
Bottom line, do whatever it takes to make execution proceeds efficiently.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 64
65. Blending Reactive Test Strategies
10-20% of testing should be reactive (even more in case formal testing is
automated).
Blending offsets weaknesses in test strategies as they complement each other.
Reactive testing blend is done by giving permissions to explore other options
during formal execution.
Time boxing can help.
Experienced testers should do experience- and defect-based testing more.
Time boxing and chartered testing can help.
Reproducibility for unscripted test should be considered.
Capture-play back tools can help.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 65
66. TM Role in Test Execution
Being there all day during test execution
when needed unless he is in meetings
Daily team debriefing of progress with the
entire team
Take proper control actions if needed
Checks and ensures proper bi-directional
traceability carried out by the team
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 66
Pass 2 Defects Status Count
Verified 47
To be verified 1
Needs approval 5
Deferred 21
Failed verification 3
Not a defect 8
New 13
Total 103
67. Exercise: Metrics
Describe one or more metric that you can use to report residual risks
during test execution.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 67
68. Testing Process
Introduction
Test Planning, Monitoring and Control
Test Analysis
Test Design
Test Implementation
Test Execution
Evaluating Exit Criteria and Reporting
Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 68
69. Learning Objectives
TM-1.7.1 (K2) Explain the importance of accurate and timely information
collection during the test process to support accurate reporting and
evaluation against exit criteria
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 69
70. Testing as a Source of Information
Testing produces lots of information (relevant or not).
TM must ensure that right information is captured and delivered to
appropriate stakeholders.
TM should be able to relate the test results to the exit criteria.
Test management tools must be properly selected and in place to support
accurately and timely gather information so as to facilitate effective
evaluation and reporting.
TM should never use gathered info for personal appraisals.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 70
71. Proper Information Collection
Requires clear definition of what to be collected, by whom and when
Should be documented during test planning
Frequent check on gathered information is a must (never assume they are
OK).
Team should be trained and aware of how to gather/report information w/
or w/o tools.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 71
72. Reporting Frequency and Detail
There is no one size fits all reporting approach.
Close monitoring of exit criteria does not necessitate detailed reporting.
Close monitoring = proper control
Reporting = providing supporting information
Details and type of reporting should differ according to recipient
stakeholders.
They need different information to do their work.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 72
73. Testing Process
Introduction
Test Planning, Monitoring and Control
Test Analysis
Test Design
Test Implementation
Test Execution
Evaluating Exit Criteria and Reporting
Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 73
74. Learning Objectives
TM-1.8.1 (K2) Summarize the four groups of test closure activities
TM-1.8.2 (K3) Implement a project retrospective to evaluate processes
and discover areas to improve
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 74
75. Test Closure Activities
Once execution is complete, key outputs should be captured and either
passed to the relevant person or archived.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 75
Test Closure
Activities
Test
Completion
Check
Test Artifacts
Handover
Lessons
Learned
Archiving
76. Test Closure Activities cont’d
These tasks are important, often missed, and should be explicitly included
as part of the test plan.
It is common for one or more of these tasks to be omitted due to
premature reassignment or dismissal of project team members, resource
or schedule pressures on subsequent projects, or team burnout.
On projects carried out under contract, such as custom development, the
contract should specify the tasks required.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 76
77. Example: Test Closure Checklists
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 77
Test Completion
• All planned tests
run or
deliberately
skipped?
• All known
defects fixed,
deferred or
accepted?
Test Artifacts
Handover
• Known defects
communicated
to operation
and support
team?
• Test and
environment to
maintenance
team?
• Regression tests
documented?
Lessons Learned
• Were relevant
stakeholders
involved in risk
analysis?
• Were estimates
accurate?
• RCA done and
actions defined?
• Process
improvement?
• Any
unanticipated
deviation?
Archiving
• All test work
product
archived in CM?
• If the project to
be reopened, do
we have every
thing we need?
78. Project Retrospective
A meeting held at the end of each iteration/increment
To discuss:
What went well?
What can be improved?
How to improve?
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 78
79. Exercise: Retrospective
Assume that 25% of defects opened on a certain feature were rejected as
not actual problems.
If rejection rate is more than 5%, is unacceptable.
List 5 actions, that could investigate the root cause of the high defect
rejection rate.
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 79
80. Summary
Introduction
Test Planning, Monitoring and Control
Test Analysis
Test Design
Test Implementation
Test Execution
Evaluating Exit Criteria and Reporting
Test Closure Activities
24-Sep-15ISTQB Advanced Level 2012 - Test Manager 80
Test object is the component or system to be tested.
Test item is the individual element to be tested. There usually is one test object and many test items.
Test basis is all documents from which the requirements of a component or system can be inferred. The documentation on which the test cases are based. If a document can be amended only by way of formal amendment procedure, then the test basis is called a frozen test basis.
Test condition is an item or event that could be verified by one or more test cases (function, transaction, quality C/C or structural element).
Test case is a set of input values, execution preconditions, expected results and execution post conditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement.
Test procedure is a sequence of actions for the execution of a test.
Tracking detailed design specification from system designers, business requirements from business analysts, and the test work products from testing team.
Detailed design documents to be approved before detailed test case creation can start.
In Agile lifecycle, informal transfer-of-information sessions convey information between teams prior to the start of testing.
Each feature that is within scope may need a corresponding test design specification.
W/ architects to verify resources availability, to ensure people configuring environment are committed, and to understand cost/delivery timescales and the work required to deliver the test environment
Normally puts test manager under pressure to sign-off releases then followed by blame if the release is proven to be buggy in the field.
Leads to schedule slack, retroactive unit testing done by test teams instead of development or unstable deliveries at the end.
A test management weakness .
A business stakeholder may be more interested in establishing coverage against an operational business cycle even though the specification is defined in terms of system functionality.
We will have project initiation, 5 development iterations and 1 post mortem.
Project initiation will have P, MnC, and TA.
5 iterations will have complete process activities.
Post mortem will have MnC as retrospective and closure activities.
In formally-documented contexts, test implementation is the activity in which test designs are implemented as concrete test cases, test procedures, and test data. Some organizations following the IEEE 829 [IEEE829] standard define inputs and their associated expected results in test case specifications and test steps in test procedure specifications. More commonly, each test’s inputs, expected results, and test steps are documented together. Test implementation also includes the creation of stored test data (e.g., in flat files or database tables).
Traceability resolution loss occurs when multiple requirements trace to a single test condition which in turn traces to multiple test cases. If a single test fails, it is unclear what is the impacted requirement.
Check requirements of that feature for any requirements problems.
Review test cases and test conditions.
Review implemented automated test scripts.
Rerun sample test cases that resulted in rejected defects.
Review rejected defects.
Train testers on the feature.