This document provides an overview of testing methods for mainframe applications. It discusses how testing needs have changed from the past where testing involved examining batch job outputs or online screen outputs, to modern approaches like test automation, virtualization, and DevOps. It covers different levels of testing maturity and factors to consider when planning tests. Black box and white box testing techniques are described, such as model-based testing, use case testing, and debugging. The presentation emphasizes the importance of testing to ensure stability, reliability, and compliance with changing requirements.

1. FOR MORE INFORMATION PLEASE CONTACT
Xact Consulting A/S
Arnold Nielsens Boulevard 68A
DK-2650 Hvidovre
+45 7023 0100
info@Xact.dk
www.Xact.dk
Enterprise Modernization
S505. How to test a Mainframe Application
GSE Nordic Conference
May 2017
Michael Erichsen,
Xact Consulting A/S,
michael.erichsen@Xact.dk
Welcome to a presentation about how to test a mainframe application.
1

2. Disclaimers
• This is NOT a presentation
about how to replatform
from a mainframe to a
Microsoft platform
• If you want to hear about
that subject, then you could
spend the next hour much
better anywhere else!
There has been some confusion in the conference invitation about the presentation I
was invited to give.
If this had led you to expect a presentation on how to change platforms from z/OS to
Microsoft, then I am going to disappoint you.
2

3. More Disclaimers
IANAT
– I Am Not A Tester…
But I am co-author of one of the
testing tools discussed
This is a presentation about testing.
I am not a tester myself, but I do spend most of my time these days setting up test
environments and building test tools.
3

4. Remember back in the 2. Millennium?
Let us start with a flash back to the recent past.
Do you remember back in the second millennium, where we used to sail in Viking ships?
We discovered America, but were careful enough not to tell anybody about it.
4

5. Remember back in the 2. Millennium?
That was when applications were monolithic.
5

6. Remember back in the 2. Millennium?
Code was spaghetti.
6

7. Remember back in the 2. Millennium?
7
And the user interface was Leporello lists in batch.
7

8. Remember back in the 2. Millennium?
8
And green screens on line.
8

9. Testing – Old Style
Batch programs were tested by resetting the master file or database.
Then you created an input file.
You submitted the batch job.
And finally, you examined the output and compared it to the expected result or the
output of a previous run.
9

10. Testing – Old Style
Online programs were tested by starting a transaction, entering data, pressing “Enter”,
and then examining the output on the screen.
Or perhaps taking a screen shot for documentation.
10

11. This was good enough for your Grandfathers,
but probably not for you?
This way of testing was perhaps sufficient in the millennium, where we sailed the long
ships with dragon’s heads.
But today we need something else.
11

12. Testing in the 3. Millennium
• Stability and reliability have
always been the hallmarks
of the mainframe
– Changing test requirements
– Changed environment
– Shorter time to market
– Developers, testers and
operations staff are not
sitting close to each other
any more
Testing is changing completely now, and it has suddenly become very interesting.
Stability and reliability have always been the hallmarks of the mainframe.
But that is not enough anymore.
12

13. Testing in the 3. Millennium
• Stability and reliability have
always been the hallmarks
of the mainframe
– Changing test requirements
– Changed environment
– Shorter time to market
– Developers, testers and
operations staff are not
sitting close to each other
any more
We have moved from green screens to client/server, monolithic and spaghetti to
modular and structured, from small tight groups to globally dispersed, virtual teams.
And compliance requirements are added on top of it all.
We live and work in a time of liberalization, economic challenges, concentration of
corporations and a focus on shareholder value rather than using the revenue for
company building.
13

14. Testing in the 3. Millennium
• Stability and reliability have
always been the hallmarks
of the mainframe
– Changing test requirements
– Changed environment
– Shorter time to market
– Developers, testers and
operations staff are not
sitting close to each other
any more
• Instead we got DevOps
• Reduce time used to provide
new solutions
• Test faster and more
completely
– Test automation
– Test virtualization
– More flexible test environments
– Promotion depending on test
and quality assurance
DevOps is the current buzzword.
It means that we need to be faster and better.
All the time.
Testing needs to be automated to speed deliveries up.
This requires more and more parallel test environments, and it requires that we use test
virtualization to make them more independent of each other.
14

15. Testing in the 3. Millennium
• Stability and reliability have
always been the hallmarks
of the mainframe
– Changing test requirements
– Changed environment
– Shorter time to market
– Developers, testers and
operations staff are not
sitting close to each other
any more
• Instead we got DevOps
• Reduce time used to provide
new solutions
• Test faster and more
completely
– Test automation
– Test virtualization
– More flexible test environments
– Promotion depending on test
and quality assurance
We now have new flexible and modern test environments available, like Micro Focus
Enterprise, Compuware Topaz, and the IBM products formerly known as Rational.
The Rational products includes the IBM developer, the IBM debugger, Rational Team
Concert, all the tools in the Test Workbench, and zD&T, which is a complete mainframe
on Intel hardware.
15

16. Testing in the 3. Millennium
• Stability and reliability have
always been the hallmarks
of the mainframe
– Changing test requirements
– Changed environment
– Shorter time to market
– Developers, testers and
operations staff are not
sitting close to each other
any more
• Instead we got DevOps
• Reduce time used to provide
new solutions
• Test faster and more
completely
– Test automation
– Test virtualization
– More flexible test environments
– Promotion depending on test
and quality assurance
Finally, promotion is increasingly combined with test and quality assurance.
Nobody will be allowed to put anything in production, unless they can prove that it is
tested, that all the changes are covered by the test, and that coding standards have been
followed.
16

17. Testing Maturity
Introduction to Software Testing (Ammann & Offutt)
Level 0
•There’s no difference
between testing and
debugging
Level 1
•The purpose of testing
is to show that the
software works
Level 2
•The purpose of testing
is to show that the
software doesn’t work
Level 3
•The purpose of testing
is not to prove
anything specific, but
to reduce the risk of
using the software
Level 4
•Testing is a mental
discipline that helps
all IT professionals
develop higher quality
software
So, why do we test in the first place?
This is an attempt to discuss testing maturity.
If you sit there thinking that testing and debugging is the same thing, you are given a
round zero.
If you test to prove that the code that I have written does not work, you are some levels
more mature. But not too respectful, I might say.
If you focus on risk reduction and quality improvement, you score the highest marks.
17

18. Typical Software Quality Factors
William C. Hetzel, The Complete Guide to Software Testing
• Functionality (exterior quality)
– Correctness
– Reliability
– Usability
– Integrity
• Engineering (interior quality)
– Efficiency
– Testability
– Documentation
– Structure
• Adaptability (future quality)
– Flexibility
– Reusability
– Maintainability
Here are some good factors to consider when you plan your tests.
The list itself demonstrates how far modern testing has moved.
18

19. Typical Software Quality Factors
Hetzel, William C., The Complete Guide to Software Testing
• Functionality (exterior quality)
– Correctness
– Reliability
– Usability
– Integrity
• Engineering (interior quality)
– Efficiency
– Testability
– Documentation
– Structure
• Adaptability (future quality)
– Flexibility
– Reusability
– Maintainability
•How many
factors
do we
actually
test for?
And it puts the big question to all of us:
How many factors do we actually test for?
19

20. Release Culture
Company A (Old story,
possibly an urban legend):
– It is better economy to ship early
and often
– We have a monopoly, and it is
cheaper to pay damages (and
lawyers) than to have better
quality
Company B (Real life):
– Every version must be both
forward and backward compatible
– Thorough testing
– Wide beta programs
The culture aspect is important.
It is really a part of your corporate culture.
If all that matters are shareholder values for the current quarter, then you risk losing
focus on your customers.
And in the longer run:
Losing focus on your real business.
Company A cocktail party.
Company B is how IBM is expected to work.
20

21. Where do we validate our Input?
• The 3270 model leaves no choice
• The distributed model leaves nothing but choices
– In the client?
– At entry to the mainframe?
– At entry to every module?
The first question we meet is:
Where do you validate your input?
This is traditionally different from mainframes to distributed platforms.
But it might be worth reconsidering?
21

22. Where do we validate our Input?
• The 3270 model leaves no choice
• The distributed model leaves nothing but choices
– In the client?
– At entry to the mainframe?
– At entry to every module?
• An all too true story
– Mainframe developers have validated certain data for decades
– Front end developers do not ask
• They just code the same test in the front end
In many shops, there is no consensus in this matter.
This results in the same validation happening in different layers of the applications.
And even a service catalog will not help you, if nobody is using it, anyway.
22

23. Quote of the Day
• “If debugging is the process
of removing software bugs,
then programming must be
the process of putting them
in”
– Edsger Dijkstra
• Stolen from
http://www.marcofolio.net/
Debugging is not the same as testing.
But this quote probably goes for both efforts.
23

24. The Real Architecture Model Cycle
Experience
shows
limitations
Before we dig into some of the many methods, techniques and tools for testing, I would
like to remind how architecture models evolve.
First somebody gets annoyed by the limitations in their practical work.
24

25. The Real Architecture Model Cycle
Experience
shows
limitations
Improvement
ideas
Then someone gets some good ideas for improvement.
25

26. The Real Architecture Model Cycle
Experience
shows
limitations
Improvement
ideas
Bestseller
with new
terms for
everything
If one of them is ambitious enough, then he gives it a name and writes a book about it.
It is always a “he”, and sometimes the book becomes a best seller.
26

27. The Real Architecture Model Cycle
Experience
shows
limitations
Improvement
ideas
Bestseller
with new
terms for
everything
Part of
curriculum
This is new and exciting, so some universities put in into their curriculum to show that
they are on the leading edge.
It is beaten into the heads of the poor students.
27

28. The Real Architecture Model Cycle
Experience
shows
limitations
Improvement
ideas
Bestseller
with new
terms for
everything
Part of
curriculum
Vendors
create new
tools
No tool vendor wants to be left behind, so they all develop or buy a tool to support it.
28

29. The Real Architecture Model Cycle
Experience
shows
limitations
Improvement
ideas
Bestseller with
new terms for
everything
Part of
curriculum
Vendors
create new
tools
Students
become
architects
In the meantime the students graduate.
They get jobs, and since they are on the beat with the newest trend, they become
architects in the companies, where you work.
29

30. The Real Architecture Model Cycle
Experience
shows
limitations
Improvement
ideas
Bestseller
with new
terms for
everything
Part of
curriculum
Vendors
create new
tools
Students
become
architects
Everybody
must use new
terms and
tools
Now everybody else must use the new tools and terms, or they are fast becoming
obsolete.
30

31. The Real Architecture Model Cycle
Experience
shows
limitations
Very soon somebody gets annoyed by the limitations in their practical work, and the
cycle starts all over.
So, I will not have a slide in this presentation about Test Driven Development, because
that might have become obsolete since I started speaking.
31

32. Processes
Waterfall?
– Test planning
• Finding an error before it has been
made is brilliant
• Finding errors late in the project is
expensive
Agile?
– ”Shift Left”?
• Finding an error as soon as it has
been made
Can they be combined?
– Or will that be too expensive?
The big movement in the development and testing process today is going from waterfall
to agile.
We know the limitations of the waterfall model, and we are annoyed by them.
It is too rigid and too slow for the requirements of today. Errors found in the test phase
become expensive, and they can delay a project.
But we should try not to throw out the baby with the waterfall.
I have seen test planners finding errors that the coders had not yet made, because of the
helicopter view of the testers.
32

33. Quote of the Day
• “Walking on water and
developing software from a
specification are easy if both
are frozen”
– Edward V Berard
• Stolen from
http://www.marcofolio.net/
If an environment is too agile, you might fall through.
33

34. Testing Methods
• Static testing
– Review
– Walk through
– Inspection
– Verification
• Dynamic testing
– Black box
• Treats software under test as a black-box without knowing its internals.
• Tests are using software interfaces and trying to ensure that they work as expected
– White box
• Looks inside the software that is being tested
• Uses that knowledge as part of the testing process
To start to discuss and categorize testing methods, we have a distinction between static
and dynamic testing.
Static is what we do at code review meetings, at the desk top, or at home with a print
list and a glass of good single malt.
Dynamic is computer aided.
It comes in two flavours: Black box, and white box.
34

35. Black Box Testing
Viktor Farcic: Technology Conversations
• Efficient for large segments
of code
• Code access is not required
• Separation between user’s
and developer’s
perspectives
This is a listing of some of the advantages of black box testing.
35

36. Black Box Testing
Viktor Farcic: Technology Conversations
• Efficient for large segments
of code
• Code access is not required
• Separation between user’s
and developer’s
perspectives
• Limited coverage since only a
fraction of test scenarios is
performed
• Inefficient testing due to
tester’s lack of knowledge
about software internals
• Blind coverage since tester has
limited knowledge about the
application
And some of the disadvantages.
36

37. Black Box Testing
Model-based testing
Use case testing
Specification-based testing
All-pairs testing
Fuzz testing
Exploratory testing
These are the main methods for black box testing.
They are much more advanced than just entering data into a green screen.
They all look at both how to develop your test cases and how to automate them.
Model-based testing is used by a method called “Model Driven Design” by defining test
cases to test that the model is implemented.
37

38. Black Box Testing
Model-based testing
Use case testing
Specification-based testing
All-pairs testing
Fuzz testing
Exploratory testing
Use case testing uses test cases based on analysis of success and some failure scenarios.
Specification-based testing uses Boundary Value Analysis, Decision Tables, and State
Transitioning tests.
All-pairs testing tests all possible discrete pairs of input parameters.
Fuzz testing is an automatic test of invalid, unexpected or random data.
And Exploratory testing will develop test cases as you go and get new ideas from
experience.
This could be the basis of a long list of requirements to your test tool vendors.
38

39. White Box Testing
Viktor Farcic: Technology Conversations
• Efficient in finding errors and
problems
• Required knowledge of internals
of the software under test is
beneficial for thorough testing
• Allows finding hidden errors
• Programmers introspection
• Helps optimizing the code
• Due to required internal
knowledge of the software,
maximum coverage is obtained
This is a listing of some of the advantages of white box testing.
39

40. White Box Testing
Viktor Farcic: Technology Conversations
• Efficient in finding errors and
problems
• Required knowledge of internals
of the software under test is
beneficial for thorough testing
• Allows finding hidden errors
• Programmers introspection
• Helps optimizing the code
• Due to required internal
knowledge of the software,
maximum coverage is obtained
• Might not find unimplemented
or missing features
• Requires high level knowledge
of internals of the software
under test
• Requires code access
And some of the disadvantages.
40

41. White Box Testing
• Debugging (single stepping)
• Post-mortem debugging
(Dump analysis)
• Unit testing
• Integration testing
• Component Interface
testing
• System testing
• Acceptance testing
• Regression testing
• Usability testing
• Accessibility testing
• Security testing
• Internationalization testing
• Code coverage
• Fault injection
White box testing calls for a discussion about testing techniques.
We will only touch it very lightly, but go into more details about some of the tools.
41

42. Code Coverage Criteria
• Function coverage
• Statement coverage
• Branch coverage
• Condition coverage (or
predicate coverage)
Code coverage was one of the first methods for systematic software testing.
Modern tools often report on code coverage after testing on a function and statement
basis, but we might consider more advanced criteria as well.
Branch coverage checks that each branch of each control structure has been executed.
For example, given an if statement, have both the true and false branches been
executed?
Condition coverage (or predicate coverage) checks in the same way if each Boolean sub-
expression has been evaluated both to true and false.
42

43. Fault Injection
Parasoft JTest as an example
(AD 1999)
– Analyse Java class using
introspection
– Generate input data intended to
break the code
– Shoot at it
– Create a report on what was
broken and how to fix it
This depends on the wish to
validate on the unit level
Fault Injection is wide spread in Java development.
Java has a language feature called introspection, which reports which methods can be
called with which arguments.
This is used to generate automated test cases, that analyses a program, injects all
interesting input, and reports on missing validations and error handling.
I have once used the tool mentioned here to make a critical Java application bullet proof.
43

44. Test Automation
Front end tools
– Hiperstation
– IDz Generic Service Client
– TestRunner
– SoapUI
Host tools
– Headless Code Coverage
Lifecycle tools
– Embed in lifecycle management
Automated tests are important for speeding up the development and test life cycle.
We can use front end tools that only need to know about the interfaces.
Code coverage can run as part of automated test as what is called headless Code
Coverage, where it is running as a batch job without a graphical user interface.
And they can and should be embedded in your lifecycle management as parts of the
promotion procedure.
44

45. Quote of the Day
• “Always code as if the guy
who ends up maintaining
your code will be a violent
psychopath who knows
where you live”
– Rick Osborne
• Stolen from
http://www.marcofolio.net/
Some programmers claim that if a program has been hard to code, it should also be hard
to read.
Maintainers do not agree.
Neither do white box testers.
45

46. Security Testing
• Authentication and
authorization
• Resilience (DoD attacks,
injections etc.)
– Chaos Monkey (Simian Army)
A different area that needs testing is security.
We now know that mainframes can be attacked and penetrated.
We need to develop formalized tests of authentication, authorization and resilience.
This is not only a task for network and firewall departments.
46

47. Security Testing
• Authentication and
authorization
• Resilience (DoD attacks,
injections etc.)
– Chaos Monkey (Simian Army)
An interesting tool has been developed by Netflix engineers to test the resiliency and
recoverability of their Amazon Web Services.
It was called “Chaos Monkey”, and now they have a complete suite, so they called it
“Simian Army”.
Chaos Monkey is a resiliency tool that helps applications tolerate random instance
failures.
47

48. Security Testing
• Authentication and
authorization
• Resilience (DoD attacks,
injections etc.)
– Chaos Monkey (Simian Army)
• Privacy
– The EU ePrivacy Directive
– The EU General Data Protection
Regulation
– Privacy By Design
– Privacy By Default
– The Right to be Forgotten
But not only must we protect our companies and our systems.
We also need to protect the security and privacy of our current customers and of others
that use our systems and might be our future customers.
48

49. Security Testing
• Authentication and
authorization
• Resilience (DoD attacks,
injections etc.)
– Chaos Monkey (Simian Army)
• Privacy
– The EU ePrivacy Directive
– The EU General Data Protection
Regulation
– Privacy By Design
– Privacy By Default
– The Right to be Forgotten
This has long been an ethical requirement, but thanks to visionary and resilient
politicians in the European Union we now have “The EU ePrivacy Directive” and “The
General Data Protection Regulation”.
It must become law in each member state by May 6th 2018.
Some of the important principles are “Privacy By Design”, “Privacy By Default”, and “The
Right to be Forgotten”.
These principles must be obeyed by our application design and our test data
management.
49

50. Test Data Management
• The Achilles’ heel of many test
teams
• Test data creation
– By hand
– By tool
• CA Test Data Manager (Datamaker)
• Compuware’s Test Data Management
• InfoSphere Optim Test Data
Management
– From production data
• Masking/Anonymization/Pseudonym
ization
– EU directive
• Test data backup and restore
Test Data Management is a very big discussion in itself.
There are other presentations on that subject at the conference, such as S503 - The
Impact of Inefficient, Ineffective and unsecure Test Data By Kim Mortensen and Gerald
Pfeiffer, and S510 - Where does your test data come from? By Bill Alexander.
50

51. Test Data Management
• The Achilles’ Heel of many test
teams
• Test data creation
– By hand
– By tool
• CA Test Data Manager (Datamaker)
• Compuware’s Test Data Management
• InfoSphere Optim Test Data
Management
– From production data
• Masking/Anonymization/Pseudonym
ization
– EU directive
• Test data backup and restore
• Synchronizing code changes,
data structure changes, and
test data
– Synchronization Big Iron 
zD&T
I will just mention that when you are expanding your test environments to a farm of
zD&T mainframe systems running on Intel, one of the biggest challenges is precisely test
data management:
How do you create a consistent and sufficiently large set of test data?
And how do you update your test data, when applications change?
51

52. z series Development and Test Environment
• zD&T, RD&T, RDzUT,
zPDT, ADCD…
• Fast and economic
deployment of test
environments
Z Series Development and Test Environment is the mainframe on Intel hardware.
It is a less expensive and much faster way of setting up all the parallel test environments
that we need today.
Setting up a new z/OS system with out-of-the-box settings can be done in a few days
from scratch, and even faster when you have a golden image to clone.
52

53. z series Development and Test Environment
• zD&T, RD&T, RDzUT,
zPDT, ADCD…
• Fast and economic
deployment of test
environments
– The heavy lift is load
and adaptation of
data, applications,
configurations, and
customizations
What takes time and effort is to identify and extract applications, data, configurations,
and customizations on the big iron, and then to adapt them to the new test
environment.
53

54. Tools
• Service test
• Performance test
• Unit test
Let us now look at some test tools.
54

55. Service test
Product Company
HiperStation Compuware
IBM Developer for z Service Client feature IBM
LoadRunner HPE
Rational Test Workbench IBM
SoapUI SmartBear
Service test is done by tools running outside the mainframe.
They are doing a black box test to check the interfaces and API’s you have exposed to
front end clients.
These are some of the most popular tools.
55

56. Performance Test
Product Company
Application Performance Analyzer IBM
FreezeFrame Macro4
InTune BMC
Mainframe Application Tuner CA
SmartTune ASG
Startool Serena
Strobe Compuware
Performance test is done by your usual on line monitors like MainView, Omegamon,
Tmon, and Sysview.
And with profiler products such as these on this list.
56

57. Unit test
Product Company
Topaz for Total Test Compuware
XaTester Xact
zUnit IBM
Unit test tools have been top of the mainframe testers’ wish list for years.
Now they are finally becoming available.
IBM has zUnit, which is part of IDZ, IBM Developer for z systems. I will point a
presentation about that one.
Compuware recently announced Topaz for Total Test. I will point to a presentation on
that one too.
And Xact also recently announced XaTester, which I will go a bit more in detail on in a
moment.
57

58. Quote of the Day
“A bug is not an error,
it is a test that was not written”
– Extreme programming
So, with a bit of luck, it might be time to say goodbye to some of the bugs?
58

59. The New Kids on the Block
• Rational Integration Tester
• zUnit
• Topaz
• XaTester
Let us go into details om one integration tester and three unit testers.
They are two different categories, but in practical use, they seem to overlap.
Rational Integration Tester can be used for test virtualization at interfaces like a call to
MQ or a distributed program link in CICS.
59

60. The New Kids on the Block
• Rational Integration Tester
• zUnit
• Topaz
• XaTester
ZUnit tests single modules in batch.
Topaz test single modules, but can also provide test virtualization for some calls.
XaTester was built to test single modules, but user requirements have made it evolve
into some integration testing and quite a bit of test virtualization.
60

61. Rational Integration Tester
Rational Integration Tester is a scripting-free environment for developing tests for
service-oriented architecture (SOA) messaging and business process integration
projects.
It can virtualize several messaging solutions (for example, JMS, TIBCO, MQ, CICS
distributed program link, and CICS Transaction Gateway) by using a plug-in for the
different environments.
61

62. A Sample Application
Imagine an application that is called by a front end, and which communicates with
another application using MQ.
It could look like this.
62

63. Test Virtualization
What you would like to do was to test your application independently of the other
application.
So, you would like to stub or mock up the MQ call and get the expected result back,
even though the other application was unavailable.
63

64. RIT Installation
You must set up a server for Rational Integration Tester with a repository.
You must install an exit in your MQ system.
And, you must install the Rational Test Workbench on your workstation.
You can then define that certain MQ calls should be intercepted by the exit and that the
requests and responses are recorded by the server when running the application.
64

65. RIT Simulation
Now you can define that all requests are redirected to the server, and the proper
responses are returned.
You can also use an editor to define which data is returned in which fields by the server.
65

66. Rational Integration Tester
The user interface is the Rational Test Workbench, which is shown here.
I have been using it in a project, where the response time for the remote application was
two to three days.
The test virtualization reduced testing time to seconds.
66

67. More Sample Applications
The limitations of the integration tester are seen, when you want to test single modules
in batch or online.
67

68. Unit Test
The unit tester is similar to the integration tester, but it operates at the module level. So,
we need some more packaging around the module to test it.
This is a generic view of a unit tester.
68

69. Unit Test
Unit testers typically have graphical user interfaces and tools to record and edit test
data.
They can have parses that analyses the data definitions of sources of the modules under
test.
And they can have repositories to save test cases, test suites, and test data.
69

70. zUnit
zUnit is not a separate product, but a feature of Developer for z Systems.
It provides a code-driven unit testing framework for Enterprise COBOL and PL/I.
zUnit is an adaptation of the xUnit framework, which is derived from the Java
environment.
It generates COBOL or PL/I programs that can call the modules under test.
Bill Alexander will tell you much more about it in his presentation.
70

71. Topaz for Total Test
Topaz for Total Test from Compuware automates both the creation and the execution of
unit tests
It integrates tightly with Xpediter, but can run stand alone or integrated with IBM
Developer.
It generates data stubs, program stubs and test result assertions.
It can call subprograms with parameters collected by Xpediter
It can integrate to tools like SonarCube, and Code Coverage can be used during the test-
run.
Kim Mortensen will tell you more in his presentation.
71

72. XaTester
XaTester is a unit test and integration test tool for COBOL, PL/I, EGL, Fortran, RPG and
Assembler programs on Mainframe and iSeries.
This is the Eclipse based user interface, which can run stand-alone or embedded in IBM
Developer for z.
72

73. XaTester Web Interface
This is the web browser interface of XaTester.
73

74. XaTester Infrastructure
And this is the infrastructure needed to support all the features.
74

75. XaTester Installation
To install XaTester you need a separate server for the test engine and the repository.
The test engine includes a scheduling feature.
You also need to install a host agent, when running on z/OS.
75

76. XaTester Installation
If you want to run XaTester in CICS, you also need to install some CICS components.
IMS is handled as batch.
If you want to run XaTester from Eclipse or from IBM Developer, you also need to install
a plug-in on your workstation.
76

77. XaTester Extraction
First, you must get the data definitions of the modules under test.
COBOL, EGL and RPG can be extracted by XaTester and stored in the repository for later
use.
Structures for other languages like PL/I, Fortran and assembler can be defined in an
editor in XaTester until somebody requires us to build a parser for this language. Or they
plug in their own parser.
77

78. XaTester Testing
You add the module or modules you want to test in an editor window, where you also
define input data.
This can be typed directly or you can define an SQL query to retrieve the data.
If you have more than one module, you can define the output from one module as input
to the next.
Then you run the test and examine the result.
78

79. XaTester Regression Testing
You can save the result from at test in the repository.
Next time you run the test, the results will be compared to the results of the previous
run.
Fields like time and date can be excluded from the comparison.
79

80. XaTester Code Coverage
If you have RDz or IDz on your system, and you have configured the headless code
coverage feature, then XaTester can use it to display, which statements have not been
tested.
80

81. XaTester Mock up
When you test a module that calls submodules, you can ask XaTester to generate a mock
up module, which can be called to return the recorded values, if that module is not
available.
This is supported both in batch and in CICS.
81

82. XaTester Demo
But let us just spend a few minutes to see what such a unit tester looks like in a web
browser interface.
It is a test of a COBOL program that calculate a risk factor based on the age of the
customer.
82

83. XaTester Demo
83

84. XaTester Demo
Let us spend a few minutes more to see what it looks like in Eclipse.
84

85. XaTester Demo
85

86. Feature Comparison
3 Unit Testers, 1 Integration Tester
Category XaTester TopazforTT zUnit RIT
Runtime Main batch, called
batch, CICS, IMS, iSeries
Main batch Batch Any
User Interfaces Eclipse, Browser Eclipse Eclipse Workbench and
browser
Programming
Languages
COBOL, PL/I, assembler,
EGL, RPG
COBOL COBOL & PL/I Any
Execution Environments PC, host and a server PC and host PC and host Server and host
Execution integration Stand alone, Idz, local
scheduling, submitting
to night batch
Stand alone, Idz and
Topaz
Workbench/Xpediter
Idz N/A
Testing Test suites, Code
coverage, Advanced test
script support,
SonarCube, External
reporting, repository
with test artefacts
Test suites, code
coverage, SonarCube,
External reporting
Test suites Test suites, Repository
with test artefacts
This is a short comparison chart between the products.
Since all vendors are presenting at the conference, you can dig deeper yourselves, if you
are interested.
None of the tools support DB2 Stored Procedures or Websphere application server.
86

87. Feature Comparison
3 Unit Testers, 1 Integration Tester
Category XaTester TopazforTT zUnit RIT
Runtime Main batch, called
batch, CICS, IMS, iSeries
Main batch Batch Any
User Interfaces Eclipse, Browser Eclipse Eclipse Workbench and
browser
Programming
Languages
COBOL, PL/I, assembler,
EGL, RPG
COBOL COBOL & PL/I Any
Execution Environments PC, host and a server PC and host PC and host Server and host
Execution integration Stand alone, Idz, local
scheduling, submitting
to night batch
Stand alone, Idz and
Topaz
Workbench/Xpediter
Idz N/A
Testing Test suites, Code
coverage, Advanced test
script support,
SonarCube, External
reporting, repository
with test artefacts
Test suites, code
coverage, SonarCube,
External reporting
Test suites Test suites, Repository
with test artefacts
Nor do they support an ISPF user interface, which I would not expect anybody to want
anyway.
None of them support C or C++.
Both Topaz and XaTester can be invoked from tools like Jenkins.
For XaTester external requests can be made using a REST API.
87

88. Feature Comparison
3 Unit Testers, 1 Integration Tester
Category XaTester TopazforTT zUnit RIT
Virtualization Submodules and DB2
(including rollback)
Submodules and
VSAM/QSAM
(Automated using
Xpediter)
Subroutines, VSAM Really many
Integration Apache Ant, CentraSite
ActiveSOA, CICS DPL,
CTG, DB2, Java, JDBC,
Jenkins, MQ, SAG
WebMethods, SAP,
Tibco, WAS Service
Integration Bus
Test data creation File import, recording File import, recording File import File import, recording
When reading this chart, you should keep in mind that the integration tester does not
really compare directly to the unit testers.
88

89. The Final Test
We are almost done now, but it is never over without a final test.
89

90. The Final Test
Well known
fact
A study at Manchester Metropolitan University involving dropping 100 slices under
laboratory conditions established that toast typically lands on the floor butter-side-down
because of way it is typically dropped from a table, and the aerodynamic drag caused by
the air pockets within the bread.
90

91. The Final Test
Well known
fact
The toast is invariably butter-side-up when dropped.
As it falls, it rotates; given the typical speed of rotation and the typical height of a table,
a slice of toast that began butter-side-up on the table will land butter-side-down on the
floor in 81% of cases.
91

92. The Final Test
Well
known
Fact: Cat
righting
reflex
Cats possess the ability to turn themselves right side up in mid-air if they should fall
upside-down, known as the cat righting reflex. This enables them to land on their feet if
dropped from sufficient height, about 30 cm.
92

93. The Final Test
So, what happens if you tie a slice of toast to the back of a cat and push it down from a
table?
And how do you design a practical test for it?
93

94. To Summarize
• Automated testing and quality assurance is a matter of survival
• We are expanding the scope of our testing rapidly
– But many areas are still more or less unhandled
• The main issue is not tools, but processes and focus
• Pinpoint the needs of your business
– Then define your processes and methods
– And finally select the tools that suit your requirements
To summarize the presentation.
Automated testing and quality assurance is not just something nice to have in this day
and time.
It is a matter of survival.
The scope of our testing is expanding rapidly, but many areas are still more or less
unhandled.
The main issue is not the tools, but to have the right processes and to keep the focus.
You should pinpoint the needs of your business.
Then you should define your processes, methods and best practices.
And finally, you could select the tools that suit your requirements.
94

95. Hand-on Session and more Information
• If you want to know more about zUnit, go to session S510, or
catch Bill Alexander
• If you want to know more about Topaz for Total Test, go to
session S608, or catch Kim Mortensen
• If you want to play with XaTester, you can join the hands-on
session at 15.15, or catch Steen Brahe at the demonstration
booth in the coffee break area
95

96. FOR MORE INFORMATION PLEASE CONTACT
Xact Consulting A/S
Arnold Nielsens Boulevard 68A
DK-2650 Hvidovre
+45 7023 0100
info@Xact.dk
www.Xact.dk
Enterprise Modernization
Moderniza-
tion
Conversion
Business
Software
Consultants
Thank you for listening to this presentation about testing.
96