Archive for category MVS

How the system starts and manages batch jobs

Batch processing is the most fundamental function of z/OS. Many batch jobs are run in parallel and JCL is used to control the operation of each job. Correct use of JCL parameters (especially the DISP parameter in DD statements) allows parallel, asynchronous execution of jobs that may need access to the same data sets.

The job entry subsystem (JES) helps z/OS receive jobs, schedule them for processing, and determine how job output is processed.

An initiator is a system program that processes JCL, sets up the necessary environment in an address space, and runs a batch job in the same address space. Multiple initiators (each in an address space) permit the parallel execution of batch jobs.

One goal of an operating system is to process work while making the best use of system resources. To achieve this goal, resource management is needed during key phases to do the following:

-    Before job processing, reserve input and output resources for jobs.

-    During job processing, manage spooled SYSIN and SYSOUT data.

-    After job processing, free all resources used by the completed jobs, making the resources available to other jobs.

z/OS shares with the Job Entry Subsystem (JES) the management of jobs and resources. JES receives jobs into the system, schedules them for processing by z/OS, and controls their output processing. JES is the manager of the jobs waiting in a queue. It manages the priority of the jobs and their associated input data and output results. The initiator uses the statements in the JCL records to specify the resources required of each individual job after it is released (dispatched) by JES.

During the life of a job, both JES and the z/OS base control program control different phases of the overall processing. Jobs are managed in queues: Jobs that are waiting to run (conversion queue), currently running (execution queue), waiting for their output to be produced (output queue), having their output produced (hard-copy queue), and waiting to be purged from the system (purge queue).

No Comments

System IPL Sequence

The initialization process begins when the System Programmer or Operator selects the LOAD function at the Hardware Management Console (HMC).

To successfully IPL z/OS, the system programmer needs to supply the following information:

-    The device address of the IPL volume

-    The LOADxx member that contains pointers to system parameters

-    The IODF data set that contains the configuration information

-    The device address of the IODF volume

z/OS locates all usable central storage that is online and available, and begins creating the various system areas.

Not all disks attached to a CPU have loadable code on them. A disk that does is generally referred to as an ″IPLable″ disk, and more specifically as the SYSRES volume.

IPLable disks contain a bootstrap module at cylinder 0 track 0. At IPL, this bootstrap is loaded into storage at real address zero and control is passed to it. The bootstrap then reads the IPL control program IEAIPL00 (also known as IPL text) and passes control to it. This in turn starts the more complex task of loading the operating system and executing it.

After the bootstrap is loaded and control is passed to IEAIPL00, IEAIPL00 prepares an environment suitable for starting the programs and modules that make up the operating system, as follows:

1. It clears central storage to zeros before defining storage areas for the master scheduler.

2. It locates the SYS1.NUCLEUS data set on the SYSRES volume and loads a series of programs from it known as IPL Resource Initialization Modules (IRIMs).

3. These IRIMs begin creating the normal operating system environment of control blocks and subsystems.

IPL Resource Initialization Modules (IRIMs)

Some of the more significant tasks performed by the IRIMs are as follows:

-    Read the LOADPARM information entered on the hardware console at the time the IPL command was executed.

-    Search the volume specified in the LOADPARM member for the IODF data set. IRIM will first attempt to locate LOADxx in SYS0.IPLPARM. If this is unsuccessful, it will look for SYS1.IPLPARM, and so on, up to and including SYS9.IPLPARM. If at this point it still has not been located, the search continues in SYS1.PARMLIB. (If LOADxx cannot be located, the system loads a wait state.)

-    If a LOADxx member is found, open and read information including the nucleus suffix (unless overridden in LOADPARM), the Master Catalog name, and the suffix of the IEASYSxx member to be used.

-    Load the operating system’s Nucleus.

-    Initialize virtual storage in the Master Scheduler address space for the System Queue Area (SQA), the Extended SQA (ESQA), the Local SQA (LSQA), and the Prefixed Save Area (PSA). At the end of the IPL sequence, the PSA will replace IEAIPL00 at real storage location zero, where it will then stay.

-    Initialize real storage management, including the segment table for the master scheduler, segment table entries for common storage areas, and the page frame table.

The last of the IRIMs then loads the first part of the Nucleus Initialization Program (NIP), which invokes the Resource Initialization Modules (RIMs), one of the earliest of which starts up communications with the NIP console defined in the IODF.

Nucleus Initialization Program (NIP)

During the NIP stage, the system might prompt the system programmer or operator to provide system parameters that control the operation of z/OS. The system also issues informational messages about the stages of the initialization process. IEASYSnn, a member of PARMLIB, contains parameters and pointers that control the direction that the IPL takes. The system programmer or operator may alter these parameters as necessary.

IEASYSnn, a member of PARMLIB, contains parameters and pointers that control the direction that the IPL takes.

The system continues the initialization process, interpreting and acting on the system parameters that were specified. NIP carries out the following major initialization functions:

-    Expands the SQA and the extended SQA by the amounts specified on the SQA system parameter.

-    Creates the pageable link pack area (PLPA) and the extended PLPA for a cold start IPL; resets tables to match an existing PLPA and extended PLPA for a quick start or a warm start IPL.

-    Loads modules into the fixed link pack area (FLPA) or the extended FLPA. Note that NIP carries out this function only if the FIX system parameter is specified.

-    Loads modules into the modified link pack area (MLPA) and the extended MLPA. Note that NIP carries out this function only if the MLPA system parameter is specified.

-    Allocates virtual storage for the common service area (CSA) and the extended CSA. The amount of storage allocated depends on the values specified on the CSA system parameter at IPL.

-    Page-protects the NUCMAP, PLPA and extended PLPA, MLPA and extended MLPA, FLPA and extended FLPA, and portions of the nucleus. An installation can override page protection of the MLPA and FLPA by specifying NOPROT on the MLPA and FIX system parameters.

Address space creation and subsystem initialization

In addition to initializing system areas, z/OS establishes system component address spaces. It establishes an address space for the Master Scheduler and other system address spaces for various subsystems and system components. Some of the component address spaces are: *MASTER*, ALLOCAS, APPC, CATALOG, and so on.

The Master Scheduler address space is the first system component address space to be created (ASID=1). Then, the Master Scheduler may start the primary job entry subsystem (JES2 or JES3). On many production systems, JES is not started immediately; instead, an automation package starts all tasks in a controlled sequence.

Then other defined subsystems are started. All subsystems are defined in the PARMLIB library, in member IEFSSNxx. These subsystems are secondary subsystems.

The private areas are available only to that address space, but common areas are available to all.

After the system is initialized and the job entry subsystem is active, the installation can submit jobs for processing by using the START, LOGON, or MOUNT command. When a job is activated through START (for batch jobs), LOGON (for time-sharing jobs), or MOUNT, a new address space is allocated.

No Comments

VTOC (Volume Table of Contents)

z/OS uses a Catalog and a Volume Table of Contents (VTOC) on each DASD to manage the storage and placement of data sets

The VTOC lists the data sets that reside on its volume, along with information about the location and size of each data set, and other data set attributes.

A standard z/OS utility program, ICKDSF, is used to create the label and VTOC. When a disk volume is initialized with ICKDSF, the owner can specify the location and size of the VTOC. The size can be quite variable, ranging from a few tracks to perhaps 100 tracks, depending on the expected use of the volume. More data sets on the disk volume require more space in the VTOC.

The VTOC also has entries for all the free space on the volume. Allocating space for a data set causes system routines to examine the free space records, update them, and create a new VTOC entry.

Data sets are always an integral number of tracks (or cylinders) and start at the beginning of a track (or cylinder).

You can also create a VTOC with an index. The VTOC index is actually a data set with the name SYS1.VTOCIX.volser, which has entries arranged alphabetically by data set name with pointers to the VTOC entries. It also has bitmaps of the free space on the volume.

A VTOC index allows the user to find the data set much faster.

No Comments

What is a Catalog?

A catalog describes data set attributes and indicates the volumes on which a data set is located.

When a data set is cataloged, it can be referred to by name without the user needing to specify where the data set is stored. Data sets can be cataloged, uncataloged, or recataloged. All system-managed DASD data sets are cataloged automatically in a catalog.

Cataloging of data sets on magnetic tape is not required, but doing so can simplify users’ jobs.

In z/OS, the Master Catalog and User Catalogs store the locations of data sets. Both disk and tape data sets can be cataloged. To find a data set that you have requested, z/OS must know three pieces of information:

-    Data set name

-    Volume name

-    Unit (the volume device type, such as a 3390 disk or 3590 tape)

You can specify all three values on ISPF panels or in JCL. However, the unit device type and the volume are often not relevant to an end user or application program. A system catalog is used to store and retrieve UNIT and VOLUME location of a data set. In its most basic form, a catalog can provide the unit device type and volume name for any data set that is cataloged. A system catalog provides a simple look-up function. With this facility the user need only provide a data set name.

Master Catalog and User Catalog

A z/OS system always has at least one Master Catalog. If it has a single catalog, this catalog would be the Master Catalog and the location entries for all data sets would be stored in it. A single catalog, however, would be neither efficient nor flexible, so a typical z/OS system uses a master catalog and numerous User Catalogs connected to it.

A User Catalog stores the name and location of a data set (dsn/volume/unit). The master catalog usually stores only a data set High-Level Qualifier (HLQ) with the name of the user catalog, which contains the location of all data sets prefixed by this HLQ. The HLQ is called an alias.

As a general rule, all user data sets in a z/OS installation are cataloged. Uncataloged data sets are rarely needed and their use is often related to recovery problems or installation of new software. Data sets created through ISPF are automatically cataloged.

No Comments

What is batch processing?

Batch processing is a method of running a program or a series of programs in which one or more records (a batch) are processed with little or no action from the user or operator.

A program that reads a large file and generates a report, for example, is considered to be a batch job.

To enable the processing of a batch job, z/OS professionals use Job Control Language (JCL) to tell z/OS which programs are to be executed and which files will be needed by the executing programs.

JCL allows the user to describe certain attributes of a batch job to z/OS, such as:

-    Who the user is (the submitter of the batch job)

-    What program to run

-    Where input and output are located

-    When a job is to run

After the user submits the job to the system, there is normally no further human interaction with the job until it is complete.

No Comments

IPL Types

You may use one of three types of initialization: Cold start, quick start, or warm start.

The primary difference between the three types is whether or not certain storage is treated.

Cold start

An IPL that loads (or reloads) the Pageable Link Pack Area (PLPA) and clears the Virtual Input/Output (VIO) data set pages.

The first IPL after system installation is always a cold start because the PLPA is initially loaded. Subsequent IPLs are cold starts when the PLPA is reloaded, either to alter its contents or to restore its contents if they were lost. This is usually done when changes have been made to the LPA (for example, when a new SYSRES containing maintenance is being loaded).

VIO is a method of using memory to store small temporary data sets for rapid access. However, unlike a RAM disk on a PC, these are actually backed up to disk and so can be used as a restart point. Obviously there should not be too much data stored in this way, so the size is restricted.

Quick start

This is an IPL that does not reload the PLPA, but clears the VIO data set pages. The system resets the page and segment tables to match the last-created PLPA. This is usually done when there have been no changes to LPA, but VIO must be refreshed. This prevents the warm start of jobs that were using VIO data sets.

Warm start

An IPL that does not reload the PLPA, and preserves journaled VIO data set pages. This will allow jobs that were running at the time of the IPL to restart with their journaled VIO data sets.

Often, the preferred approach is to do a cold start IPL (specifying CLPA). The other options can be used, but extreme care must be taken to avoid unexpected change or backout of change. A Warm start could be used when you have long-running jobs which you want to restart after IPL, but an alternative approach is to break down those jobs into smaller pieces which pass real data sets rather than use VIO. Modern disk controllers with large cache memory have reduced the need for VIO data to be kept for long periods.

Also, do not confuse a cold start IPL (CLPA would normally be used rather than the term ″cold start″) with a JES cold start. Cold starting JES is something that is done extremely rarely, if ever, on a production system, and totally destroys the existing data in JES.

No Comments

IPL (Initial Program Load)

z/OS initialization, or an Initial Program Load (IPL), is the act of loading a copy of the operating system from disk into the processor’s real storage and executing it.

This process essentially consists of:

-    System and storage initialization, including the creation of system component address spaces

-    Master scheduler initialization and subsystem initialization

z/OS systems are designed to run continuously with many months between reloads, allowing important production workloads to be continuously available. Changes are the usual reason for a reload, and the level of changes on a system dictates the reload schedule.

Bear in mind that:

-    A test system may be IPLed daily or even more often.

-    A high-availability system (from a bank for example), may only be reloaded once a year, or even less frequently, to refresh the software levels.

-    Outside influences may often be the cause of IPLs, such as the need to test and maintain the power systems in the machine room.

-    Sometimes faulty software uses up system resources that can only be replenished by an IPL. This sort of incident is normally the subject of investigation and correction so that it does not happen again.

Shutting down the system usually requires a single command, which results in the removal of most tasks except for the automation task itself. The automation task is closed manually, followed by any commands needed to remove the system from a Sysplex or serialization ring.

No Comments

How to compress LLA-managed libraries

Sometimes a PDS library needs to be compressed to reclaim the free space caused by member deletions. The recommended procedure for compressing an LLA-managed library is:

1. Create a new CSVLLAxx member that includes a REMOVE statement identifying the library that needs to be compressed (ex: CSVLLA02).

REMOVE(SYS1.EXAMPLE.FILE)

2. Then issue the F LLA,UPDATE=xx command (ex: F LLA,UPDATE=02).

3. Compress the library. You can either use “Z” against the data set you want to compress from the ISPF panel or submit a job to do the compress.

4. Create a new CSVLLAxx member that includes a LIBRARIES statement to return the compressed library to LLA management (ex: CSVLLA03).

LIBRARIES(SYS1. EXAMPLE.FILE)

5. Then issue the F LLA,UPDATE=xx command (ex: F LLA,UPDATE=03).

NOTE:

CSVLLAxx is a member of SYS1.PARMLIB.

No Comments

The Paging Process

In addition to the DAT hardware and the segment and page tables required for address translation, paging activity involves a number of system components to handle the movement of pages and several additional tables to keep track of the most current version of each page.

To understand how paging works, assume that DAT encounters an invalid page table entry during address translation, indicating that a page is required that is not in a real storage frame. To resolve this page fault, the system must bring the page in from auxiliary storage. First, however, it must locate an available real storage frame. If none are available, the request must be saved and an assigned frame freed. To free a frame, the system copies its contents to auxiliary storage and marks its corresponding page table entry as invalid. This operation is called a page-out.

After a frame is located for the required page, the contents of the page are copied from auxiliary storage to real storage and the page table invalid bit is set off. This operation is called a page-in.

Paging can also take place when z/OS loads an entire program into virtual storage. z/OS obtains virtual storage for the user program, and allocates a real storage frame to each page. Each page is then active and subject to the normal paging activity; that is, the most active pages are retained in real storage while the pages not currently active might be paged out to auxiliary storage.

Page stealing

z/OS tries to keep an adequate supply of available real storage frames on hand. When a program refers to a page that is not in real storage, z/OS uses a real storage page frame from a supply of available frames.

When this supply becomes low, z/OS uses page stealing to replenish it, that is, it takes a frame assigned to an active user and makes it available for other work. The decision to steal a particular page is based on the activity history of each page currently residing in a real storage frame. Pages that have not been active for a relatively long time are good candidates for page stealing.

Unreferenced interval count

z/OS uses a sophisticated paging algorithm to efficiently manage virtual storage based on which pages were most recently used. An unreferenced interval count indicates how long it has been since a program referenced the page. At regular intervals, the system checks the reference bit for each page frame. If the reference bit is off — that is, the frame has not been referenced — the system adds to the frame’s unreferenced interval count. It adds the number of seconds since this address space last had the reference count checked. If the reference bit is on, the frame has been referenced and the system turns it off and sets the unreferenced interval count for the frame to zero. Those frames with the highest unreferenced interval counts are those most likely to be stolen.

z/OS also uses various storage managers to keep track of all pages, frames, and slots in the system.

No Comments

Frames, Pages and Slots

When a program is selected for execution, the system brings it into virtual storage, divides it into pages of 4 kilobytes (4K), and transfers the pages into real storage for execution. To the programmer, the entire program appears to occupy contiguous space in storage at all times. Actually, not all pages of a program are necessarily in real storage, and the pages that are in real storage do not necessarily occupy contiguous space.

The pieces of a program executing in virtual storage must be moved between real and auxiliary storage. To allow this, z/OS manages storage in units, or blocks, of 4 kilobytes.

The following blocks are defined:

- A block of real storage is a frame.

- A block of virtual storage is a page.

- A block of auxiliary storage is a slot.

A page, a frame, and a slot are all the same size: 4096 bytes (4 kilobytes). An active virtual storage page resides in a real storage frame. A virtual storage page that becomes inactive resides in an auxiliary storage slot (in a paging data set).

No Comments