Admin Alert: Reorganizing IBM i Files To Improve Disk Performance, Part 2

June 25, 2014 Joe Hertvik

Last issue, I began putting together a template for detecting and efficiently reorganizing physical files on IBM i partitions. I covered the most common reasons administrators reorganize files and demonstrated a procedure for detecting which files on your system need reorganization. This week, I’ll complete the process by reviewing different ways to reorganize files and look at some techniques for file reorganization while the file is active.

Steps To Effectively Reorganizing Files

As I discussed last time, here are the steps to follow for an effective disk reorganization strategy.

Set up a procedure for identifying files and members that need reorganization.
Properly setting up your Reorganize Physical File Member (RGZPFM) statements.
Identifying times that critical files can be reorganized.
Rinse and repeat.

Last issue, I covered step #1. This time, I’ll discuss steps 2 through 4.

Step 2: Properly setting up your Reorganize Physical File Member (RGZPFM) statements

There are a number of techniques for effectively running physical file reorganizations. In the IBM i operating system, file reorgs can be accomplished by running any one of the following Reorganize Physical File Member command versions:

RGZPFM FILE(library/filename) /* version 1 */
RGZPFM FILE(library/filename) ALWCANCEL(*YES) /* version 2 */
RGZPFM FILE(library/filename) RBDACCPTH(*OPTIMIZE) ALWCANCEL(*YES) /*version 3*/

Here’s how each of these file reorganizations work and the benefits each technique provides.

Version 1 RGZPFM–Just the basics

The version 1 command is a standard RGZPFM command that just removes deleted records and reorganizes the file. It is the most common way of running RGZPFM. Version 1 is easy to run and seldom causes any problems.

The downside is that with a version 1 reorg, you’ll lose any in-process reorganization work if you cancel a file reorganization before it completes. If you cancel a version 1 RGZPFM command from either the green screen or inside a batch job, any reorg changes that were made to the target file will be discarded and the file will contain the same number of deleted records it had when it started.

Version 2 RGZPFM–Cancelling a reorg without penalties

The version 2 RGZPFM command allows you to cancel the reorg as needed, either on the green screen or inside a batch job. Because its Allow Cancel (ALWCANCEL) parameter is set to *YES, cancelling a version 2 RGZPFM command will retain all the file reorganization changes that have been completed so far and your file will be partially reorganized. The next time you start the RGZPFM command on a partially reorganized file, the operating system will attempt to restart the reorganization at the point it was cancelled. If too many changes occur after a cancelled reorganization, the operating system may opt to restart the reorganization at the beginning of the file rather than attempting to continue from the last reorganization attempt.

Version 3 RGZPFM–Getting a jump start on rebuilding associated logical file access paths

The version 3 RGZPFM command adds some flexibility when related file logical file access paths are rebuilt as their associated physical file is reorganized. It does this through the Rebuild Access Paths (RBDACCPTH) parameter.

Related logical file reorganizations are controlled through the access path rebuild list that’s maintained and controlled through the Edit Rebuild of Access Paths (EDTRBDAP) command. This can add some time to your access path rebuilds as, by default, there are only a set number of logical files that can have their access paths rebuilt by the system at the same time when using the EDTRBDAP list. All of which adds time to your reorganization, depending on how many logical files are waiting in line to be reorganized via EDTRBDAP.

In our RGZPFM command versions 1 and 2, RBDACCPTH is set to *YES (its default setting). Setting RBDACCPTH to *YES designates that all dependent logical file access paths are synchronously reorganized after the RGZPFM command finishes (no logical file reorganization happens until the physical is reorged).

There are two ways you can speed up IBM i access path rebuilds: manually or by setting the RGZPFM’s RBDACCPTH to *OPTIMIZE, which is demonstrated in the version 3 RGZPFM statement.

You can manually increase the number of simultaneous access path rebuilds by using a technique I outlined in an earlier article. This technique requires you to monitor and change the EDTRBDAP list, and to submit batch jobs to speed up multiple logical file rebuilds.

The second way to increase the number of simultaneous access path rebuilds is to set the Rebuild Access Path (RBDACCPTH) parameter on your RGZPFM command to *OPTIMIZE. As shown in my RGZPFM version 3 command, setting RBDACCPTH to *OPTIMIZE gives the operating system some leeway on how to rebuild associated access paths. When using *OPTIMIZE, the IBM i operating system will rebuild access paths for a reorganized file in one of the following ways: Associated access paths will be rebuilt asynchronously when the reorganization operation on the physical file ends; or associated access paths will be maintained and rebuilt during the reorganization.

The operating system decides how to rebuild access paths, based on which method it calculates will finish the access path rebuilds faster (optimizing the rebuilds).

There are other parameters you can set for your RGZPFM commands, but these are the basic ones that affect the file’s ability to recover from a cancelled reorg and to speed up access path rebuilds after your reorganization.

Step 3: Identifying times that critical files can be reorganized

The next step determines when you want to or are able to reorganize system files with excessive deleted records.

Because a RGZPFM operation requires file locks, you’ll usually have to arrange for your file reorganizations to occur when the files aren’t in use or configure your reorganizations to occur while the file is active.

Reorganizing files to account for record locking can be accomplished in the following ways:

1. By quieting your IBM i system so that any applications that usually lock your reorged files are not running. This would involve running your reorganizations during a maintenance window when application files are not in use and the system is quiet for rebuilds.

2. By specifying whether the RGZPFM command should exclusively lock the file that it’s reorganizing or whether the target file should be shared for reading or updating (reorg-while-active). This is done through the Lock state parameter on your RGZPFM command (LOCK). LOCK can be set to any one of three different setting as shown by these RGZPFM command examples

RGZPFM FILE(library/filename) LOCK(*EXCL) /* version 4 */
RGZPFM FILE(library/filename) LOCK(*EXCLRD) ALWCANCEL(*YES) /* version 5 */
RGZPFM FILE(library/filename) LOCK(*SHRUPD) ALWCANCEL(*YES) /*version 6*/

In the version 4 RGZPFM command, the LOCK parameter is set to its default setting of Exclusive (*EXCL). In this iteration, the RGZPFM command must be able to get an exclusive lock on the file it is reorganizing and no other process can access the file while it is being reorganized. The file must be in a quiet state (no other users) in order to use the version 4 RGZPFM command.

The LOCK Exclusive Read (*EXCLRD) parameter in the version 5 RGZPFM command allows other users to read but not write, update, or delete records in the file being reorganized. In order to use *EXCLRD for your LOCK parameter, you also need to set the Allow Cancel parameter to *YES.

For the version 6 RGZPFM command, LOCK is set to Shared Updated (*SHRUPD) which allows users to read, write, update, and delete individual records in the file as it is being reorganized. Like the version 5 RGZPFM command, you also need to set the Allow Cancel parameter to *YES. It’s important to note that if you’re reorganizing the file in key file sequence or need to maintain the records in arrival sequence, you shouldn’t use the LOCK = *SHRUPD parameter with this command.

When you set the LOCK parameter to either *EXCLRD or *SHRUPD, you are performing a reorg-while-active function, which allows the file to be accessed while the reorganization is occurring.

Step 4: Rinse and repeat

After you’ve performed the other three steps, the final step is to run your file reorganizations on a regular basis.

This means setting up scheduled jobs that will run all your required reorganization commands on a regular basis. As I mentioned last issue, file reorganizations are a temporary disk recovery technique. You’ll be able to reduce disk space by running all your designated reorganizations at one time. But after you reorganize, the number of deleted records will slowly start to climb again, and you’ll have to run your target reorgs on a regular basis to enjoy the full benefits of your file reorganization strategy.

Putting together a file reorganization strategy like this isn’t a one-time event. But once the strategy is conceived and implemented, it can produce a number of benefits in reducing the amount of used disk space and improving production efficiency.

Joe Hertvik is an IBM i subject matter expert (SME) and the owner of Hertvik Business Services, a content strategy company that provides white papers, case studies, blogging, and social media services for B2B software companies. Joe also runs a data center and help desk for several companies. Joe has written the Admin Alert column for IT Jungle since 2002.