Joel Montvelisky, Chief Solution Architect at PractiTest, gave a webinar on reporting principles for QA managers. He discussed decision-driven test management (DDTM), where testing is planned based on the information stakeholders need to make decisions. He emphasized the need for repetition and using multiple reporting channels like dashboards and emails. Montvelisky recommended "numberless reporting" using labels rather than numbers, and provided tips for effective reporting including starting with the bottom line, keeping reports simple, and being willing to correct yourself.

1. Joel Montvelisky
Chief Solution Architect
Simplify Test Management
PractiTest Webinar:
Reporting Principles for Every QA Manager

2. What do we want to cover today?
• DDTM – Decision Driven Test Management
• The need for repetition & multiple reporting channels
• Numberless reporting
• Do’s & Don’ts for successful reporting
• Questions and Answers
Simplify Test Management

3. A little about me…
 Testing:
- Tester
- QA Manager
- Blogger
- Speaker
- Solution
Architect
 Some numbers:
- 1 wife
- 3 kids
- 1 dog & 2 fish
- 18 years testing
 PractiTest
- Chief Solution Architect
- Test Manager
Simplify Test Management

4. About
 SaaS ALM Solution
 Simple & Intuitive – yet – Methodological & Flexible
 Integrates with Jira, Selenium, Pivotal Tracker, Jenkins, Redmine,
Bugzilla, QTP, Ranorex, SoapUI, etc…
 Best Darn Support in the Business 
Simplify Test Management

5. DDTM
Decision-Driven
Test Management
Simplify Test Management

6. Why Do We Test?
Simplify Test Management
To provide our stakeholders with visibility
into the status of the product and process,
so they can make the correct decisions.
DDTM
Decision-Driven Test Management
Planning and running your tests based on the information
needed by your stakeholders to make their decisions.

7. DDTM –
Reverse Engineering your Testing Process:
1. Identify your stakeholders and their decisions
2. Understand the information needed to answer questions
3. Plan your reports, metrics and statistics
4. Define the tests to capture data
5. Run your tests based on timelines required for data capturing
6. Continuously monitor for new stakeholders & their changing
needs
Simplify Test Management

8. A Word of Caution About
Stakeholders’ Decisions
 Stakeholders come in many types & sizes.
 Decisions tend to happen at the beginning
or the end of the project.
 Decisions are not only based on user stories
or requirements.
 Each stakeholder will make
different decisions,
at different times,
based on different information.
Simplify Test Management

9. The Need for Repetition
&
Multiple Channels
Simplify Test Management

10. Question:
Simplify Test Management
Which of the following reporting channels do you use?
 Web-based Dashboards
 Live Monitors
 High or Low Level Testing Reports
 Emails with Status Updates
 Update Meetings
 Periodic Management Presentations
 Other

11. Need for Repetition
(Theory of Effective Frequency)
Simplify Test Management
Herbert E. Krugman (GE 1960’s):
“People need to be exposed 3 times
to act on an idea.”
1st time: What is it? (Curiosity)
2nd time: What of it? (Recognition)
3rd time: Take action. (Decision)

12. Using Multiple Channels
Simplify Test Management
People absorb information
differently:
 Graphically
 Textually
 Statistically
 Comparatively
etc.
People access information via
multiple media:
 Computer
 Phone
 Verbally
 Presentations
etc.
It is more effective to combine multiple
channels & formats to communicate information

13. Simplify Test Management
And so...
In the case of Reports, More is actually Better.
More Repetitions x More Formats
=
Better Results!!

14. Numberless Reporting
Simplify Test Management

15. Question:
Simplify Test Management
Do you use exact numbers as part of
your reports?
e.g. Number of Tests, Number of Bugs, etc

16. Numberless Reporting
Simplify Test Management
Reminder
Don’t make
me think!
Cheers,
The Boss
If you already know what interests
your stakeholders, why not provide
them with the processed information?

17. Numberless Reporting - example
Simplify Test Management
Version 7.2 Status Overview:
Feature A.1
Feature B.2
Feature C.3
Feature D.4
Platform E.5
Platform F.6
Stability & Load G.7
Overall project projection
Stability
New features
Version Projection
Main risks
 Major delays on feature D.4
 Lack of stability in Feature C.3
 4 Team members moving to
project XYZ next week

18. Numberless Reporting
Simplify Test Management
 Color code the main areas (features?) of your project
 List the most important things to know
 Provide projections
 Make sure people know where things are good and
where they are not good
 Be ready to provide numbers to back your numberless
conclusions.

19. Reporting Do’s & Don’ts
Simplify Test Management

20. Do 1: Always start from the bottom line
then you can explain
Simplify Test Management
 Place the most important information on the top and
center.
 Add data to support your conclusions,
but always at the end.
 You telling the News, and not writing a Novel!

21. Do 2: Work S.M.A.R.T.
Simplify Test Management
 Simple
 Measurable
 Actionable
 Repeatable
 Timely

22. Do 3: Trust your gut
Simplify Test Management
Do 4: Focus
 Express your feelings even if you sometimes lack hard
data to back them up.
 Be concrete and focused on what you are trying to
say.
Don’t beat around the bush.

23. Don’t 1: Don’t fall in love with your data
Simplify Test Management
Don’t 2: Don’t be afraid to correct yourself
Don’t 3: Don’t be afraid to say I don’t know
(and get back to people with the answer later)

24. Questions & Answers

25. About
 SaaS ALM Solution
 Simple & Intuitive – yet – Methodological & Flexible
 Integrates with Jira, Selenium, Jenkins, Redmine, Bugzilla, QTP,
Ranorex, SoapUI, etc…
 Best Darn Support in the Business 
Simplify Test Management

26. Thank You!
joel@practitest.com
http://qablog.practitest.com
Simplify Test Management