Admin Alert: Corralling i/OS Storage Hogs, Part 2

March 16, 2011 Joe Hertvik

Last week, I published part 1 of my series detailing the seven biggest culprits of unnecessary i/OS storage consumption. This week, let’s continue with the remainder of my list and see what you can do to keep your storage problems under control.

Again, the Seven Sinister Storage Hogs

While not inclusive of all the ways processes can consume disk on your machine, I usually focus my attention on the big seven culprits of unnecessary storage consumption when an issue appears on one of my iSeries, System i, or Power i systems.

Excessive spooled files and other system objects
Walking Dead libraries
Just In Case files
Large files that need reorganization
Files with obsolete members
Cleaning up utility program data
Controlling your journaling

Last issue, I discussed the first three hogs in the pen. This week, let’s look at the lower numbers in the list.

Hog #4: Large files that need reorganization

Outside of duplicated program and data objects (storage hogs #2 and #3), the most insidious i/OS storage gulper is DB2 file members that have an extremely large number of deleted records. This commonly happens with active files such as order entry files or work files that contain a large number of new, modified, and deleted records. Unfortunately, deleted records consume as much file space as active records and if not controlled, they will consume greater and greater amounts of system disk.

Once identified, it’s not difficult to reorganize a physical file member to remove its deleted records, but the problem is identifying those files. i/OS doesn’t provide any easy way to quickly identify which large files that have the highest number of deleted records.

To solve this issue, I wrote a utility to create a system reorganization file called REORGMAP. After being properly loaded by its companion program (REORGCREAT), REORGMAP becomes a database of all physical file members on your system that can be sorted by total file size and total deleted records. Once you have this information, it’s a simple matter to sort your file and determine which file members can be reorganized to regain hard drive space. Even though I created REORGMAP and REORGCREAT several years ago, this technique still works surprisingly well today.

You can see my original article on how to create your own REORGMAP file for identifying data file storage hogs with a high number of deleted records here. Note, however, that the original article wasn’t complete in that it didn’t show you which database members in each bloated file had the highest number of deleted records. To adjust your REORGMAP file to properly identify obese file members with deleted records, see part 2 of the article, which corrects the process to list deleted records by file member.

Hog #5: Files with obsolete members

This problem usually occurs with applications that exchange a large amount of data, such as EDI file transfers. What frequently happens is that an EDI program uses one physical file for data transfers but for each transfer, it adds an additional member that is specific to that transfer. If these work members aren’t occasionally pruned from your physical file, you can easily wind up with a file containing several thousand obsolete members that are just taking up space. The key is to identify which i/OS data files have the largest number of data members and then come up with a reasonable purge routine for deleting members that are no longer needed.

As with deleted records, the key is to identify which files are stockpiling work members like firewood. As far as I know, there is no internal i/OS technique for identifying how many data files have excessive members. But if you create a REORGMAP file as mentioned above, you can come up with a quick query or SQL statement to run over REORGMAP to display how many members each file has.

Hog #6:: Cleaning up utility program data

Another area you should review is how long your utility programs keep data. For example, if you have a fax utility running on your iSeries, System i, or Power i partition, check to see how long faxes are kept on your system. You may find that your fax program is keeping old fax messages for six months, when a more reasonable retention time may fit your organization’s criteria. A similar thing can happen with applications that email spooled files, especially if you don’t have your spooled file retention times under control (storage hog #1).

To reclaim storage space, you may want to check and adjust your retention times for these other types of utility program data.

Log files that record all transactions occurring inside a specific package
Shipping confirmations, such as you might receive from an EDI program or a shipping company
EDI acknowledgements
Alert messages such as you might receive from a third-party messaging system such as Help/Systems’ Robot/ALERT or Bytware’s Messenger Console. This can also fill up your storage if you have a large number of monitors on your system and you’re keeping message activity for a number of months

Some of these items may take up excessive space in the AS/400 Integrated File System (AS/400 IFS), which provides another area to check for excessive storage usage.

Hog #7: Controlling your journaling

Storage consumption problems can also occur due to journaling activity on your system. Excessive journaling issues can occur in two separate areas.

When you are logging i/OS security events in journal receivers that are associated with the QAUDJRN journal in the QYS library. Depending on which security events you are logging, journal receiver size can quickly grow and be overlooked because many systems automatically detach full journal receivers when exchanging them for new receivers. To read more about this situation and what you can do about it, consult this article by Steve Martinson on iSeries Security Journal Receiver Management.
For data recovery, application, compliance, and data access issues, you may also be required to journal several active data files on your system. Similar to what happens with security auditing, user data file journals may also be set up to automatically swap out journal receivers when they are full. If those receivers are not properly managed, the old receivers may remain on your system storage for many months (or even years) without purging.

To see how many old journal receivers may still be sitting on your system, run the following Work with Journal Receivers (WRKJRNRCV) command.

WRKJRNRCV JRNRCV(*ALL/*ALL)

This will show you all the journal receivers on your system, and help you determine whether you have excessive journal receivers. Once you have this information, you can decide whether to manually purge old journal receivers or to set up system processes to automatically purge receivers without manual intervention.