A complete overview on VSAM (Virtual Storage Access Methods)-by rohitkapa

1. VSAM
Complete Overview

2. Sessions Outline
• Session 1 : Introduction to VSAM
• Session 2 : Structure of VSAM Datasets
• Session 3 : Access Method Services
• Session 4 : Demo/Exercise Discussion
2

3. Objectives
• Familiarization with
– VSAM concepts
– Space Allocation and types of data sets supported by VSAM
– VSAM Characteristics
• Identify the general uses for Access Method Services
3

4. Introduction to VSAM
(Virtual Storage Access Method)
•
Session 1
4

5. Topics Covered
• VSAM Basics
• VSAM Dataset Organization
5

6. What is VSAM?
• VSAM is a high-performance access method used in
OS/390, MVS/ESA and z/OS operating systems
• Used as a Dataset in conjunction with programming
languages (COBOL or PL/I) to create Applications
• Used by CICS to store and retrieve data
• Not a database management system – does not
provide a relationship among data (such as DB2 or IMS)
6

7. VSAM Characteristics
• VSAM supports three types of data access:
Sequential, Random (also called Direct access)
and Skip Sequential
• Protection of data against unauthorized access is
an inherent part of VSAM
• Easily portable to AS/400, the PC or non-IBM
computers (cross system compatibility)
• A format for storing data independently of the
type of direct access storage device on which it is
stored
7

8. VSAM Characteristics (Contd.)
• Options for optimizing performance
• A multifunction service program (Access
Method Services - IDCAMS) for setting up
catalog records and maintaining data sets
• Transactional VSAM - allows VSAM data set
sharing in batch/online and batch/batch
environments
8

9. VSAM Dataset Organizations
• ESDS Entry Sequenced Data Set
• KSDS Key Sequenced Data Set
• RRDS (Fixed Length) Relative Record
Data Set
• VRRDS Variable Length Relative Record
Data Set
• LDS Linear Data Set
9

10. NON-VSAM Access Methods
• BSAM : Basic Sequential Access Method
• QSAM : Queued Sequential Access Method
• ISAM : Indexed Sequential Access
Method
• BDAM : Basic Direct Access Method
10

11. Similarities between VSAM & NON-VSAM
Datasets
• VSAM type Similar to
– KSDS ISAM
– ESDS BSAM/QSAM
– RRDS BDAM
11

12. VSAM Dataset Organization
•
12

13. Topics Covered
• VSAM Catalog Structure
– VSAM Terminology
• VSAM Record Storage
– Control Intervals and Control Areas
13

14. VSAM Terminology
• Catalog Management • Record management
• Data component
• Logical Record
• Master/User Catalog
• Key Field
• Data Space
• Physical Record • Control Interval and Control Area
• Cluster • Alternate Indexes
• Sphere • Spanned Records
• Index component
14

15. Catalog Management
• VSAM maintains extensive information about
data sets and direct access storage space in an
integrated catalog facility (ICF) catalog.
• The catalog’s collection of information about a
data set defines that data set’s characteristics. All
VSAM files
must be defined in an ICF catalog.
15

16. Record Management
• The record management part of VSAM
contains the access method code.
16

17. Logical record
• Unit of information used to store data in a VSAM data set.
• Set of bytes containing a logical description of an item
processed
by an application program.
• A logical record can be of a fixed size or a variable size
depending on the
business requirements.
17

18. Key field
• Identifies the item associated with the logical record.
• Important Field whose contents can be used to retrieve the specific logical
record
18

19. Physical record
• The data is usually a set of logical records (packed together) transferred
from or to memory by just one Read or Write CCW. The access method
uses the BLKSIZE parameter to determine the length of the physical record.
A large block size results in fewer gaps in the track but larger buffer in
memory to keep the data.
• A physical record is device dependent, its size is calculated at the time the
data set is defined.
• All physical records have the same length.
19

20. Cluster
• Collection of physical datasets that make up
one logical data set.
• Consists of Data Component or Data
Component & Index Component
20

21. Sphere
• A sphere is a base VSAM cluster and its
associated clusters.
• These associated clusters are the alternate
indexes (AIXs) of the base cluster.
21

22. Data and Index Component
• A component is an individual part of a VSAM data set. Each
component has a name, an entry in the catalog and an entry in the
VTOC.
• The KSDS and VRRDS organizations have data and index
components. ESDS, RRDS and LDS organizations only have data
components.
• The data component is the part of a VSAM data set, alternate
index, or catalog that contains the data records.
• Using the Index component VSAM is able to randomly retrieve a
record from the data component when a request is made for a
record with a certain key. The key determines the record’s position
in the data set.
22

23. Control Interval and Control Area
• It is the fundamental building block of every VSAM file.A CI is a
contiguous area of direct access storage that VSAM uses to store
data records and control information that describes the records.
• A CI is the unit of information that VSAM transfers between the
storage device and the process or during one I/O operation.
Whenever a record is retrieved from direct access storage, the
entire CI containing the record is read into a VSAM I/O buffer in
virtual storage. The desired record is transferred from the VSAM
buffer to a user-defined buffer or work area.
• A CA is formed by two or more CIs put together into fixed-length
contiguous areas of direct access storage. A VSAM data set is
composed of one or more CAs. CAs are needed to implement the
concept of splits. The size of a VSAM file is always a multiple of its
CA. VSAM files are extended in units of CAs. A spanned record
cannot be larger than a CA.
23

24. Alternate Indexes
• Alternate indexes (AIXs) allows logical records of a KSDS or
of an ESDS to be accessed sequentially and directly by more
than one key field.
• AIXs eliminate the need to store the same data in different
sequences in multiple data sets for the purposes of various
applications. Each alternate index is a KSDS cluster
consisting of an index component and a data component.
24

25. Spanned Records
• Spanned records are needed when the application
requires very long logical records. A spanned record
may be the data component of an AIX cluster. If
spanned records are used for KSDS, the primary key
must be within the first control interval.
• Spanned records are logical records that are larger than
the CI size. To have spanned records, the file must be
defined with the SPANNED attribute at the time it is
created. Spanned records are allowed to extend across
or span control interval boundaries.
• A spanned record must always begin on a control
interval boundary and fills one or more control
intervals within a single control area. A spanned record
cannot share the CI with any other records.
25

26. Control Intervals and Control Areas
•
26

27. Control Interval
• VSAM Basic unit of work – the minimum
amount of data that is transferred via
buffers, between DASD and Main Memory
• VSAM determines the size of the physical
Record (BLOCK) based on CI Size
• The CI size can be from 512 byes to 32 KB
27

28. Control Interval & Control Area
• CA is a fixed length area of auxiliary storage
space in which VSAM stores records
• Logical records of a VSAM dataset are
grouped into a number of CIs.
• CIs of a dataset are grouped into one or more
CAs
• The size of a VSAM file is always a multiple of
its CA
28

29. General format of a Control Interval
29

31. Control Information Field of a CI
• CIF consists of two subfields:
• Control Information Definition Field (CIDF)
– It is a 4 byte field
– Contains the Offset and Amount of Free space
• Record Definition Field (RDF)
– It is a 3-byte field.
– Length of records (based on the spread of data)
– Number of adjacent records that are of same
length
31

32. Control Information Fields in a CI
Record Type Number of Records/CI Control Information (bytes)
Fixed Length Multiple 10
Fixed Length 1 7
Variable Length Depends on the spread 4 (CIDF) and 3 (RDF)
of records
32

33. Format of Index Control Interval
•
Header Entry1 Entry2 Entryn
CIF
KSDS Control Interval
Control Information Field
33

34. Structure of VSAM Datasets
Session 2
•
34

35. Topics Covered
• Types of VSAM Datasets
• More on KSDS Index Structure
35

36. Objective
• Understand the types and internal
organization of VSAM datasets
• Identify appropriate applications for VSAM
data sets
• Understand KSDS Index Search Techniques
36

37. Types of VSAM Datasets
•
37

38. VSAM Datasets Types
• Entry Sequenced Data Set (ESDS)
• Relative Record Data Set (RRDS)
• Key Sequenced Data Set (KSDS)
• Variable Length Relative Record Data Set (VRRDS)
• Linear Data Set (LDS)
38

39. ESDS CA Structure
•
39

40. COBOL example – ESDS data set
• SELECT <Cobol application file>
• ASSIGN TO <AS-ddname>
• ORGANIZATION IS SEQUENTIAL
• ACCESS MODE IS SEQUENTIAL
• FILE STATUS IS STATUS-KEY
40

41. RRDS Structure
41

42. COBOL example - Sequential Retrieval
• SELECT <vsamfile>
• ASSIGN TO <ddname>
• ORGANIZATION IS RELATIVE
• ACCESS IS SEQUENTIAL
• RELATIVE KEY IS RR-NUMBER
42

43. COBOL example - Direct Retrieval
• SELECT <vsamfile>
• ASSIGN TO <ddname>
• ORGANIZATION IS RELATIVE
• ACCESS IS RANDOM
• RELATIVE KEY IS RR-NUMBER.
43

44. KSDS Structure
•
44

45. COBOL example - Sequential Retrieval
• SELECT <vsamfile>
• ASSIGN TO <ddname>
• ORGANIZATION IS INDEXED
• ACCESS IS SEQUENTIAL
• RECORD-KEY IS EMP-NUM
• FILE-STATUS IS STATUS-KEY.
45

46. COBOL example - Direct Retrieval
• SELECT <vsamfile>
• ASSIGN TO <ddname>
• ORGANIZATION IS INDEXED
• ACCESS IS RANDOM
• RECORD-KEY IS EMP-NUM
• FILE-STATUS IS STATUS-KEY.
46

47. LDS Structure
47

48. Comparison of VSAM DS
•
48

49. Comparison of VSAM DS (Contd.)
•
49

50. More on KSDS Index Structure
•
50

51. KSDS Structure
51

52. General Format of Index Set
52

53. General format of Sequence Set
53

54. Index Search Techniques
• Horizontal Search
– For a Sequential Search, search is performed with the key value
less than or equal to the sequence set record
• Vertical Search
– For a Random Access, search is performed from the Index set
– Once the key <= condition is met, then it follows a downward
pointer to the corresponding Sequence set
54

55. Example-1
How Index & Sequence Sets work
55

56. Example-2
Data Component Structure
Data Records
56

57. Example-2 (Contd.) Data Component
Structure
Index Set Record
57

58. Share option in VSAM
• Share option in VSAM is defined as
Concurrent Usage of Dataset by several People at the
same time. We can use this option for sharing the
files in the same region and in other system. The
SHAREOPTION has two parameters, the inter-system
parameter and intra-system parameter.
• Syntax:SHAREOPTIONS(crossregion,crossystem)
58

59. Share option cont:....
• If the source data set is allocated with DISP=SHR, there is a
risk that it could be updated by a region other than the FOR. If this
happened, the data table would no longer match the source data set. To
minimize this risk, the VSAM cross-region SHAREOPTION should be set
to 1 or 2.
• 1 means that either one region can have update access to the
data set or many regions can have read-only access.
• 2 means that one region can have update access to the data
set and, at the same time, many regions can have read-only access.
• Regardless of the setting of DISP, a warning message is issued if the
cross-region SHAREOPTION is 3 or 4, or if it is 2 but the CICS-maintained
data table has read-only access (which means another region might be
able to update the data set).
59

60. Syntax and attributes
60

61. Access Method Services
Session 3
•
61

62. Topics Covered
• Introduction to IDCAMS
• IDCAMS Commands
• Define and Load a VSAM dataset
• VSAM Advanced Topics
62

63. Objectives
• Use Access Method Services (AMS )
• Identify some useful AMS Commands
• Define and Load a Basic Cluster with IDCAMS
• Define and Build Alternate Index
63

64. Introduction to IDCAMS
•
64

65. IDCAMS Utility
• Multifunction utility program supplied
with VSAM (IDCAMS) to manage and
maintain datasets
65

66. IDCAMS Features
• IDCAMS can be used to
– Define, Alter and Delete datasets and allocate space
for them
– List and Maintain VSAM Catalogs
– Load, Print and Copy datasets
– Reorganize datasets
– Define and build alternate indexes
– Facilitate backup and recovery
– Provide security and integrity functions
66

67. Invoking IDCAMS
1. As a Job or Job step
• Example
• //YOURJOB JOB (ACCT),'IDCAMS JOB',
• //STEP1 EXEC PGM=IDCAMS
• //SYSPRINT DD SYSOUT=*
• //SYSIN DD *
• AMS commands and their parameters
• /*
67

68. Invoking IDCAMS (Contd.)
• 2. TSO can be used with VSAM and access
method services to
– Execute access method services commands
– Execute a program to call access method services
3. From an application Program
68

69. IDCAMS Commands
•
69

70. COMMANDS
• Modal Commands
– Used to control execution and establish options
– Modal commands allow the conditional execution
of functional commands.
• Functional Commands
– Used to invoke access method services
70

71. COMMANDS (Contd..1)
• COMMAND specifies the type of service requested
• Default statement margins are positions 2 through 72.
• Command stmt can be continued to the next line with
‘-’ as the last parameter
• Comment may be embedded in Command statements
between /* and */
71

72. COMMANDS (Contd..2)
72

73. Condition Codes
• IDCAMS returns a condition code following
the execution of each Command
• Can be used to selectively execute or skip
subsequent commands
• Initialized to zero at the start of the execution
73

74. Condition Codes (Contd..1)
• LASTCC - specifies the condition code value
that resulted from the immediately preceding
function command
• MAXCC - specifies the maximum condition code
value that has been established by any previous
function command
74

75. Condition Codes (Contd..2)
• Code Explanation
• 0 Function executed successfully
• 4 Some problem was met in executing
the function but it was possible to
continue. The continuation might not provide
• exact results, but no permanent
harm will have been done
75

76. Condition Codes (Contd..3)
• Code Explanation
• 8 A requested function was
completed but major specifications
were unavoidably bypassed
• 12 The requested function could not
be performed, as a result of a logical
error
• 16 A severe error occurred that caused the
Job Step to terminate
76

77. Modal Commands
• IF-THEN-ELSE :
– Used to control command execution on the basis
of condition codes
• DO-END :
– Used to specify the group of commands as a
single unit
77

78. Modal Commands (Contd.)
• NULL : Used to indicate that no action is to
be taken
• PARM : Specifies processing options to be
used during execution
• SET : Used to change or reset a
previously defined condition code
78

79. Modal Commands - Syntax
• IF {LASTCC|MAXCC} {operator} {numeric value}
• THEN {command }
• DO
• {command set}
• END
• ELSE {command}
• DO
• {command set}
• END
79

80. Functional Commands
• DEFINE Used to define the following objects
– Alias
– Alternate index
– Cluster
– Dataspace
– User Catalog and Master Catalog
– Path
– Catalog entry for a Generation Data Group
– Catalog entry for a non-VSAM dataset
80

81. Functional Commands (Contd..1)
• DELETE Used to delete catalogs, VSAM and
non-VSAM datasets
• EXAMINE Analyzes and reports on the
structural consistency of the index component
and/or data component of KSDS cluster
• LISTCAT Used to list catalog entries
81

82. Functional Commands (Contd..2)
• IMPORT Connects user catalogs and imports VSAM
datasets
• EXPORT Disconnects user catalogs and exports VSAM
datasets.
• PRINT Used to print VSAM and non-VSAM datasets
and catalogs
82

83. Functional Commands (Contd..3)
• ALTER - Used to alter attributes of datasets
and catalog that have already been defined
• Alter
– Freespace
– Volume
– Prevent temporary updates to a VSAM dataset
83

84. Functional Commands (Contd..4)
• BLDINDEX - Builds alternate indexes for existing
datasets
• Keys will be actually built from the base cluster with
the BLDINDEX command
• VERIFY : Used to cause a catalog to correctly reflect
the end of a dataset after an error occurred in closing a
VSAM dataset. The error may have caused the catalog
to be incorrect.
84

85. Functional Commands (Contd..5)
• REPRO
• Loads an empty VSAM cluster with records
• Creates backup of a VSAM dataset on a
sequential dataset and restores the dataset
• Merges data from two VSAM datasets
85

86. Example: Modal & Functional
Commands
• //JOBNAME JOB ...
• //STEP1 EXEC PGM=IDCAMS
• //SYSPRINT DD SYSOUT=*
• //SYSIN DD *
• DEFINE CLUSTER -
• IF LASTCC = 0 THEN -
• REPRO -
• ELSE -
• DO
• SET LASTCC = 0
• DELETE CLUSTER ...
• DEFINE CLUSTER ...
• END
• /*
86

87. Define and Load a VSAM Dataset
•
87

88. DEFINE CLUSTER
• The DEFINE CLUSTER command can be used
to define a cluster and specify attributes for
the cluster as a whole and for the components
of the cluster. The general format is :
88

89. LISTCAT
• The LISTCAT command lists catalog entries.
– Freespace, CI/CA split, Data Attributes, Statistics.
Allocation information
• All parameters of the LISTCAT command are
optional
• LISTCAT can be used under TSO
89

90. LISTCAT Syntax
• LISTCAT ALIAS | ALTERNATEINDEX |
• CLUSTER | NONVSAM | PATH |
ENTRIES(entryname/password entryname/password..)
|
• LEVEL(level)
• EXPIRATION(days)
• NAME | HISTORY | VOLUME|
• ALLOCATION | ALL
• OUTFILE(ddname)
90

91. REPRO
• The REPRO command copies VSAM and non-
VSAM data sets, copies catalogs.
91

92. REPRO Syntax
• REPRO {INFILE(ddname)/password
• INDATASET(entryname)/password
• {OUTFILLE(ddname/password
• OUTDATASET(entryname/password}
• ( CHARACTER | DUMP | HEX )
• ( FROMKEY(key) |
• ( FROMADDRESS(address) |
• ( FROMNUMBER(number) )
• SKIP(number)
• ( TOKEY(key) |
• TOADDRESS(address) |
• TONUMBER(number) )
• COUNT(Number)
92

93. DELETE
• The DELETE command deletes catalogs, VSAM
datasets, non-VSAM datasets and objects.
93

94. DELETE Command Syntax
• DELETE (entryname/password
entryname/password..)
• ALIAS | ALTERNATEINDEX | CLUSTER |
NONVSAM |
• PATH | USERCATALOG
• ERASE | NOERASE
• PURGE| NOPURGE
• FORCE | NOFORCE
94

95. PRINT
• PRINT: The PRINT command prints VSAM
datasets, non-VSAM datasets, and catalogs.
The syntax is:
95

96. PRINT Syntax
• PRINT {INFILE(ddname /password )
• INDATASET(entryname /password ) }
• CHARACTER| DUMP | HEX
• FROMKEY(key) |
• FROMADDRESS(address) |
• FROMNUMBER(number) }
• SKIP(number)
• OUTFILE(ddname)
• TOKEY(key) |
• TOADDRESS(address) |
• TONUMBER(number)|
• COUNT(Number)
96

97. VERIFY
• The VERIFY command causes a catalog to
correctly reflect the end of a VSAM data set
after an error occurs while closing a VSAM
data set
• The syntax is:
VERIFY {FILE(ddname/password) |
DATASET(entryname/password)}
97

98. VERIFY Example
• Example of a typical VERIFY usage
• Job TRGJJJJ has terminated abnormally and VSAM
dataset TRGXXX.MAST.CLUSTER is not closed correctly.
This job is followed by another job that uses this file for
an IDCAMS copy. Following are the set of return codes
that will appear in the Job Spool
• In JESMSGLG the error will be
IEC161I 056-084,TRGGXXXX,JS010,IFILE01,,,
• In SYSPRINT of the step failed the error will be
IDC3300I ERROR OPENING TRGXX.MAST.CLUSTER
IDC3351I ** VSAM OPEN RETURN CODE IS 118
• Solution: A VERIFY issued on VSAM file
TRGXXX.MAST.CLUSTER will correct this error
98

99. ALTERNATE INDEX
• Allows for a KSDS to be accessed in a different
order other than the primary key
• Allows the creation of an index for a non-
indexed object, such as an ESDS cluster
99

100. Steps to create an ALTINDX
• DEFINE command to create the alternate
index
• BLDINDEX command to build the keys for the
alternate index
• DEFINE command to create a PATH relating
the alternate index to the base cluster
100

101. BLDINDX
• The BLDINDEX command builds alternate
indexes for existing data sets
• Alternate Index can be built only after its base
has been loaded with at least one record
101

102. DEFINE PATH
• After defining an alternative index, a PATH
must be defined to access the data set. This
forms the connection between the alternative
index and the base cluster, and the connection
is stored in the catalog
• When a program opens a path for
processing, both the base cluster and the
alternate index are opened
102

103. GDG
• A group of chronologically or functionally related data
sets are called Generation Data Group (GDG)
• Each data set within a GDG is called a Generation Data
Set or simply a Generation
• They are processed periodically, often by adding a new
generation, retaining previous generations, and
sometimes discarding the oldest generation
• Each generation data set in the group will have the
name as GDG name.GxxxxVyy, where “xxxx” is the
generation number and “yy” is the version number
103

104. GDG (Contd..)
• GDGs can be referred in a JCL by means of an
absolute name or relative name
• Absolute name is by coding the complete
Generation name as GDGname.G0001V00
• Relative name is a signed integer used to refer
to the latest (0), the next to the latest (−1),
and so forth, generation. The relative number
can also be used to catalog a new generation
(+1)
104

105. GDG Advantages
• The advantages of grouping related data sets are:
– All of the data sets in the group can be referred to by a
common name
– The operating system is able to keep the generations
in chronological order
– Outdated or obsolete generations can be
automatically deleted by the operating system
105

106. GDG Usage
• One of the important usage of a GDG is for cataloging
periodic backups of VSAM datasets
– Example, a GDG of limit 7 can be used to take backup of
VSAM data sets from the batch job for all 7 days of the
week
– The GDG base after data sets are cataloged will be as
follows
• GDGname.G0001V00
• GDGname.G0001V01
• ………………………
• GDGname.G0001V06
• Another important usage of GDG is for cataloging
periodic reports (weekly, monthly, etc). Example, a
GDG with limit 12 can hold monthly reports for a year
106

107. GDG Syntax
• DEFINE GENERATIONDATAGROUP
(NAME(entryname)
LIMIT(limit)
[EMPTY | NOEMPTY]
[SCRATCH | NOSCRATCH]
[TO(date) | FOR(days)])
107

108. Utility U.5 in ISPF
• A user interface is available under option U.5.
This provides 2 options
– Access method services
– VSAM maintenance utility
108

109. U.5.1 Option
• Access method services (option u.5.1) provides
• interface for the following functions
– Define Cluster
– Define alternate index
– Define path
– Alter VSAM dataset characteristics
– Rename a dataset
– Copy/Load a VSAM dataset
– Delete a VSAM dataset
– Print a VSAM dataset
– Verify endfile markers
109

110. VSAM Advanced Topics
•
110

111. CICS VSAM Transparency for z/OS
• CICS VSAM Transparency for z/OS can help
you unlock value in data used by your legacy
applications and extend its use to new DB2
applications, online and e-business
111

112. VSAM to DB2 migration
• 24x7 availability
• Running adhoc queries
• Ability to perform data analysis (mgt
reporting, analytics)
• Exploitation of database maintenance tools
• Integration with new applications based on DB2
technology
112

113. Other advanced topics in VSAM
• VSAM Extended Format & Extended Addressability
• Data striping
• Data compression
• Partial space release
• System-managed buffering (SMB)
• VSAM Record Level Sharing (RLS)
• Utilities: File-Aid, DITTO, STARTOOL
113

114. THANK YOU
114