Midrange Stuff, OS/400 Edition

DCSoftware last week launched the ReorgWizard, a new OS/400 file reorganization utility that gives users greater control over the amount of resources consumed by file reorganization. Instead of falling victim to file reorganizations that incur substantial overhead and deteriorate overall system performance over a period of hours or days, administrators can slow file reorganization to one file per second, without requiring users to log off the system.

Reorganizing files is an important OS/400 house-keeping task that frees disk space and makes applications run faster. As database records are created by an application, they're sequentially numbered, starting from the top. When a record has been deleted because it is no longer needed, the record's data is removed, but that record's placeholder remains, and additional records continue to be piled onto the end of the list. To reclaim that unused placeholder, a file reorganization must be done. In addition to freeing disk space, eliminating those placeholders speeds application response time, because of the unique way that OS/400 stores data in its memory and storage system.

While file reorganizations usually make applications run faster, they can also create their own performance problem. The problem has to do with the overhead incurred by OS/400 while it is "reshuffling" the data in the database. As DCSoftware developer Dave Shea explains, this overhead is created by OS/400 constantly recalculating the access paths, to make sure that the logical files continue to point to the correct physical files, as the reorganization runs.

Changing the job priority of the file reorganization process has little effect on the fraction of system resources consumed by the reorganization because it doesn't count the overhead created by OS/400. "Even if you monkey around with the job priority, you might still be in trouble," Shea says. "It's the overhead that's killing it." Another approach would be to put a hold on the job, but then you risk locking up the record that was being worked on.

Shea knew there had to be a better way, so he developed his own widget, the ReorgWizard. The ReorgWizard features a "throttle" menu option that lets administrators assign how many records the ReorgWizard should process before taking a one-second break. With a setting of 0, the process runs full speed ahead (incurring substantial overhead along the way). If the throttle setting is 1, the ReorgWizard will perform a complete record reorganization at the rate of one record per second, and the overhead will decrease accordingly." You are controlling the run rate from inside the job, not by externally trying to control the job using a job priority," Shea says.

Shea says this throttle feature will be especially beneficial to companies that rely heavily on journaling, such as those running mirroring software for high availability or for data warehousing projects. "In high availability or data warehouse environments where journaling is used to keep a mirrored database synchronized, this throttle feature can be critical in allowing the mirrored machine to keep up with the reorg activity," he says.

Another feature that Shea says distinguishes his product from similar offerings on the market is that it doesn't create a duplicate copy of a file or directory, in order to prevent users from having to be logged off the system to perform a file reorganization. Duplicating libraries can exacerbate an already tenuous DASD situation. Shea says the ReorgWizard reorganizes records "in place," and only requires users to log off the system when the product needs an exclusive lock to delete the unused record placeholder. IBM's own OS/400 command Reorganize Physical File Members (RGZPFM) requires users to log off the system during the entire reorganization.

Not every shop needs a reorg tool as clever as the ReorgWizard, Shea says, which is impressive in its own right. Shops that can get by with reorganizing files during off hours are fine with the RGZPFM command. But companies that don't have the luxury of a large weekend window should consider getting more control of their reorgs with the Wizard, he adds.

The ReorgWizard is the first new product from Hopkinton, Massachusetts, based DCSoftware since ARCTOOLS/400, the company's flagship purge and archive utility. Shea says ReorgWizard and ARCTOOL/400 make a nice one-two punch for purging old records from the box, then reclaiming that space using ReorgWizard. To that end, DCSoftware is offering to knock 50 percent off the cost of the ReorgWizard to both new and existing ARCTOOLS/400 users.

ReorgWizard supports OS/400 V3R2 through V5R2. License fees range from $499 for a P05 box to $5,999 for a P60 box (IBM has eliminated the P60 tier with the new iSeries servers). For more information, go to www.arctools.com.