Maxava IFS Replication Makes Performance Leap

March 9, 2015 Dan Burger

In the high availability world, the replication of IFS data has been a bit of a bottleneck. Companies that require 24×7 continuous operations have no tolerance for downtime, so backing up all systems and data, creating a failover system, and having access to everything is of the upmost importance. One of the keys to this is throughput performance and Maxava has addressed this with multi-threaded replication architecture for the IBM i Integrated File System, or IFS, which has just become available in its enterprise HA product.

Multi-threaded IFS replication lets organizations run multiple IFS replication processes in parallel, increasing the throughput. The key to this performance upgrade from single-thread replication is multiple dynamic apply groups and automated trigger points, which provides the faster replication of data on backup systems so that IFS data is not still sitting in a queue waiting to be processed.

Support for the multi-threaded apply process is not new. Maxava used this technology for replicating DB2 for i data and IBM i objects before applying it to the IBM i IFS, which is a derivative of the OS/2 parallel file system that IBM grafted onto OS/400 twenty years ago to give it the capability of supporting PC-style ASCII files. The database being a higher priority than the IFS led to this natural selection. That and the fact that IFS files for some companies are increasing in number and size at a rate never seen before. Where it was once common to be replicating “big” files in the 10 GB to 20 GB range, there are files now in the hundreds of gigabytes range.

“A lot of the stuff we learned from doing multi-threading and multi-processing applied data was used to assist this project,” says Pete Kania, technical services and development director at Maxava. “But the IFS is quite a different beast than the DB2 for i database.”

The commonalities between database and IFS are items such as knowing how many objects are involved and the size of the library, understanding what needs to be managed, and what stuff needs to be cleaned up.

The difference is that applications are storing more stuff in the IFS. Kania says one of the challenges now is getting a grasp on what needs to be replicated and what doesn’t. Some of the stuff being stored in the IFS is transitory or temporary and doesn’t need to be replicated. Sending unnecessary data and objects down the pipe long distances will slow the process and increase costs.

“When we first released our single-apply IFS replication years ago, we believed it was performing at an optimum speed,” Kania says. “We knew the speed was superior to competitors’ systems that we were replacing. But as customers began using more data and the size of the IFS increased, we started to look at ‘future-proofing’ our product.”

Part of that future proofing, Kania says, is that the multi-threading technology is applied so that as volumes increase more “apply groups” can be added dynamically to adjust for heavier and lighter loads.

Handling more jobs and at a faster pace has its limitations. If you overload a machine with too many jobs, you’ll run out of memory, CPW, and I/O. Every machine and workload has a sweet spot where it runs as fast as possible. So what you can do in terms of running multiple jobs will vary from one system to the next.

Other variables to take into account for the performance-minded multi-threader are factors the number of updates required and how much the network can handle. There’s more to this than just expecting multi-threading to be the answer. Jobs run fast or slow depending on how well the programs are coded, so that is a factor to take into account as well.

With the much faster processing that IBM has brought to Power8–faster disk and I/O–companies can take advantage of features like multi-threading and handle large volumes of data on their systems.

“In general, the more applies (threads) you use, the faster it is going to go up to the capabilities of the box and the network. But we can find the sweet spot,” Kania says. “In IBM i shops it’s been tradition to tune the system for high performance, particularly by the larger customers who push the system to its limits and who have teams devoted to performance tuning. You have systems handling multiple files that are hundreds of gigs. Applications are updating, reading or modifying the database and files sizes are increasing. Inefficiencies in the code show up quickly in terms of run times not being what they should be. It’s a problem for some customers,” he says.

But the database and the IFS are the two places multi-threading is making a performance difference. The multi-threading capabilities from Maxava are available now in the company’s Enterprise+ high availability software.

Table of Contents

Content archive

Recent Posts

Subscribe

Pages

Search