Admin Alert: Better Subsystem Throughput Via Multiple Job Queues, Part One

Published: February 28, 2007

by Joe Hertvik

Scheduling system i5 batch job execution can be an art form. Some groups of jobs (such as monthly processing) must run in sequence to maintain data integrity while other jobs (such as nightly reports) can be simultaneously run to achieve better throughput. Luckily, there are specific techniques for creating subsystem job queues to meet these needs. This week and next, I'll explain how to create multiple subsystem job queues to help with your throughput issues.

Why Would My Subsystem Need More Job Queues?

All i5/OS subsystems use job queues to stage work for processing. Job queues are assigned to a subsystem through the use of job queue entries. Job queue entries also help regulate how many of the job queue's current jobs can run in the subsystem at the same time. Although there is a practical limit on the number of job queues that can be attached to a subsystem, you can theoretically add 9999 job queue entries to a single subsystem. There are several good reasons for attaching more than one job queue to a subsystem, including:

• To provide separate batch job scheduling environments for different departments so that any one department is not slowed down by having to wait for another department's batch jobs to finish.
• To set up a single-threaded job queue so that sequence-dependent batch jobs (such as a group of jobs that close out your Accounting books at month-end) end correctly before starting the next job. Jobs submitted to a single-threaded job queue will run one at a time, and the next job in the job queue will not process until the previous job submitted from that same job queue finishes processing.
• To allow different types of jobs (such as reports or file rebuilds) to run together in the same batch subsystem, so that shorter running jobs can complete even while a longer running job is executing.
• To prevent a job error message that is waiting to be answered from stopping other jobs from entering your subsystem.

Three Steps to Creating Multiple Job Queues for the Same Subsystem

While there are many reasons for creating and assigning different job queues to a subsystem, the techniques for creating these queues are the same no matter how many job queues you are assigning to a subsystem. To create multiple job queues in the same subsystem, I usually follow these three steps:

1. Reconfigure the target subsystem to run the proper number of simultaneous jobs that I want to service. My target subsystem must be able to handle the workload that I plan to throw at it. If I'm planning to run six simultaneous live jobs in the QBATCH subsystem, with the subsystem accepting two jobs at a time from the QBATCH job queue and four other jobs from different departmental job queues, I need to reconfigure QBATCH to handle that workload.
2. Create the new job queues and assign them to the target subsystem through the use of job queue entries. Obviously, this is self explanatory.
3. Reconfigure any affected user profiles, scheduled jobs, etc. to submit their jobs into the new job queues. The new job queues are no good if no one is using them. Next issue, I'll cover how to perform this function for any type of batch job in the system.

As an example of how to configure and assign additional job queues to a subsystem, I'll show you how to reconfigure the QBATCH subsystem so that it can run up to six separate batch jobs at the same time from four different departmental job queues (Accounting, Marketing, Production, and Sales) as well as to run two batch jobs from the QBATCH job queue. Although this example deals with adding new job queues dedicated to departmental functions, all of the configurations shown here can be applied when creating and adding job queues to process additional work in any i5/OS subsystem.

Here's how I would configure my QBATCH subsystem to handle this example workload.

Step One: Prep the Subsystem for the Work It's Going to Receive

The first step is to make sure that my subsystem can handle the workload that I want to throw at it. In my case, this means that QBATCH needs to be able to run six simultaneous jobs from five different sources. To determine whether QBATCH is configured to run six jobs at once, I will check its subsystem description's Maximum jobs in subsystem parameter. I can find this parameter by displaying my subsystem configuration for QBATCH with the following Display Subsystem Description command (DSPSBSD).

DSPSBSD SBSD(QBATCH)

This command shows me the following menu for viewing my subsystem configuration parameters:

1. Operational attributes
2. Pool definitions
3. Autostart job entries
4. Work station name entries
5. Work station type entries
6. Job queue entries
7. Routing entries
8. Communications entries
9. Remote location name entries
10. Prestart job entries

If I select option 1, Operational attributes, I will be able to view my Maximum jobs in subsystem (MAXJOBS) parameter. If it's not set to my target value of six jobs, I can change it by running the following Change Subsystem Description command (CHGSBSD).

CHGSBSD SBSD(QBATCH2) MAXJOBS(6)

For subsystem descriptions, I can specify that the subsystem can run from between 1-1000 simultaneous jobs or I can change MAXJOBS to *NOMAX, which means that the subsystem can run as many active jobs as it can handle.

Step two: Creating Departmental Job Queues and Attaching Them to the Subsystem

It's easy to create a specialized job queue for a subsystem. Job queue creation is done on the green-screen through the use of the Create Job Queue command (CRTJOBQ). For my example, where I want to add job queues for four different departments (Accounting, Marketing, Production, and Sales) to the QBATCH subsystem, I would first create my job queues by using these CRTJOBQ commands.

CRTJOBQ JOBQ(QGPL/ACCOUNTING) TEXT('Accounting department job queue')

CRTJOBQ JOBQ(QGPL/MARKETING) TEXT('Marketing department job queue')

CRTJOBQ JOBQ(QGPL/PRODUCTION) TEXT('Production department job queue')

CRTJOBQ JOBQ(QGPL/SALES) TEXT('Sales department job queue')

When adding user-defined system objects such as job queues, I usually create them in the QGPL library, which is IBM's default library for user-defined system objects. I placed them here instead of in IBM's QSYS system library (where the QBATCH subsystem description resides) because objects in QSYS sometimes get overwritten or swapped out when performing a system upgrade. QGPL objects are usually kept intact when the system is upgraded. So I use QGPL to store my job queues because it's a safer location for retaining these objects in the future.

Once I have my job queues, I can attach them to my QBATCH subsystem description by the use of Job Queue entries. Inside a subsystem description, job queue entries identify which job queues the subsystem will accept work (i.e., jobs) from. Subsystem job queue entries have four components.

• Entry sequence number. This is a unique number between one and 9999 that sequences job queue entries inside a subsystem. Job queue sequence numbers are important because the subsystem will always accept available work (jobs) first from lower sequenced job queue entries before it will accept jobs from higher sequenced job queues.
• Job queue name and library. This tells the subsystem where the entry's job queue is located. Because the library is part of the name, I could conceivably create many different job queues with the same name in different libraries and use each of those job queues to feed work to a different subsystem.
• Maximum active jobs. This number specifies how many jobs from the designated job queue can run in the attached subsystem at the same time. For my purposes, I am single threading the departmental job queues (only one job can run from each queue at a time) while I am allowing two jobs from the QBATCH job queue to process at one time.
• Maximum active jobs by priority. If I am allowing multiple simultaneous jobs to run from my job queue, I can further break down the Maximum active jobs parameter so that the subsystem would regulate work coming from this job queue based on the incoming job's priority level. For example, if I specified that there is a set number of jobs that can be submitted to the subsystem from this job queue, I could further break down that number to say that no more than five jobs of priority 1 could run at the same time, no more than three jobs of priority 2 could run together, one job from priority 3, etc. This parameter offers another way to segment and order work coming into the subsystem.

Before I add my new job queue entries to the QBATCH subsystem, it's worthwhile to first look at my existing subsystem entries. If I once again entered the DSPSBSD command and entered option 6, Job queue entries, I might get a display similar to the following:

Subsystem description:   QBATCH       Status:   INACTIVE               
                                                                         
 Seq  Job                       Max   ---------Max by Priority---------- 
 Nbr  Queue       Library     Active   1   2   3   4   5   6   7   8   9 
  10  QBATCH      QSYS             3   *   *   *   *   *   *   *   *   * 
  20  QS36EVOKE   QGPL        *NOMAX   *   *   *   *   *   *   *   *   * 
  50  QTXTSRCH    QGPL        *NOMAX   *   *   *   *   *   *   *   *   * 

To set up this subsystem to allow one job each from the four departmental job queues as well as two jobs from the QBATCH job queue to run simultaneously, I would run the following Add Job Queue Entry (ADDJOBQE), Change Job Queue Entry (CHGJOBQE), and Remove Job Queue Entry (RMVJOBQE) commands.

To change the QBATCH job queue entry so that the subsystem would only run two jobs from that queue at the same time, I would run the following Change Job Queue entry command.

CHGJOBQE SBSD(QBATCH) JOBQ(QBATCH) MAXACT(2) SEQNBR(10)

The MAXACT parameter defines the Maximum active jobs value for the job queue described above.

To remove the QS36EVOKE and QTXTSRCH job queue entries because they will not be used by my subsystem, I would run the following Remove Job Queue Entry commands.

RMVJOBQE SBSD(QBATCH) JOBQ(QGPL/QS36EVOKE)

RMVJOBQE SBSD(QBATCH) JOBQ(QGPL/QTXTSRCH)

To add job queues entries to the QBATCH subsystem for my four departmental job queues so that QBATCH will only accept one job at a time for processing from each job queue, I would enter the following Add Job Queue Entry commands.

ADDJOBQE SBSD(QBATCH) JOBQ(QGPL/ACCOUNTING) MAXACT(1) SEQNBR(20)

ADDJOBQE SBSD(QBATCH) JOBQ(QGPL/MARKETING) MAXACT(1) SEQNBR(30)

ADDJOBQE SBSD(QBATCH) JOBQ(QGPL/PRODUCTION) MAXACT(1) SEQNBR(40)

ADDJOBQE SBSD(QBATCH) JOBQ(QGPL/SALES) MAXACT(1) SEQNBR(50)

And once all these commands were entered, my job queue entries list for the QBATCH subsystem would now look like this:

Subsystem description:   QBATCH         Status:   ACTIVE                 
                                                                         
 Seq  Job                       Max   ---------Max by Priority---------- 
 Nbr  Queue       Library     Active   1   2   3   4   5   6   7   8   9 
  10  QBATCH      QGPL             2   *   *   *   *   *   *   *   *   * 
  20  ACCOUNTING  QGPL             1   *   *   *   *   *   *   *   *   * 
  30  MARKETING   QGPL             1   *   *   *   *   *   *   *   *   * 
  40  PRODUCTION  QGPL             1   *   *   *   *   *   *   *   *   * 
  50  SALES       QGPL             1   *   *   *   *   *   *   *   *   *

Since I reconfigured QBATCH to run a maximum of six jobs at one time, this means that I can now reach my target goal of being able to simultaneously run six jobs in the subsystem at one time from these different sources.

At this point, I have set up my QBATCH subsystem so that it can accept work from various job queues and regulate that work so that it runs according to my example. However, the best designed subsystem job queue entry architecture will not do me any good unless I've set up the rest of my system to take advantage of it. I'll cover that issue next week in part two of this article and explain how to set up your user profiles, scheduled job entries, and submitted jobs to send work to the new job queues.

About Our Testing Environment

All configurations described in this article were tested on an i5 box running i5/OS V5R3. However, most of these commands and features are also available in i5/OS V5R4 and most earlier versions of OS/400 V4R5 and below running on AS/400 and iSeries machines.

Why Can't I Do This in OpsNav?

All the configurations shown in this article are performed by using 5250 green-screen commands. I am not using iSeries Navigator (OpsNav) functionality for this setup because I could not find equivalent functions for creating job queues and modifying a subsystem description using OpsNav features. System i5 subsystem configuration technology is still only contained in i5/OS and OS/400 commands; it has not migrated to OpsNav yet. As a result, these configurations can only be accomplished by running the green-screen commands listed here.

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot