Admin Alert: Six Things To Do Before Switching Production Processing To Your CBU

May 1, 2013 Joe Hertvik

Switching production processing to an IBM i Capacity BackUp (CBU) system involves more than just running the High Availability (HA) software commands that initiate the switch. It also involves preparing your source and target machines to switch places, so that data is correctly replicated and you can confidently run production processing on your CBU. Today, I’ll describe six steps you can take to ensure a smooth CBU switch over.

Setting The Stage

This article lists out tasks you should consider adding to your HA run book to produce the best results when switching production processing to an IBM i CBU system. These tasks are intended for traditional HA environments using replication software. They may not be appropriate if you’re using more cloud-like technologies for HA and disaster recovery, such as Live Partition Mobility.

These steps are written for planned switch overs, and may not be appropriate or feasible to use during an unplanned or emergency switch.

My recommended six things to do before switching production processing to a CBU are:

Contact IBM and third-party vendors to get any temporary keys needed for the switch over.
Quiet the source system.
Make sure transactions haven’t come in after you turn off replication.
Make sure all pending replicated transactions have been applied to your target system before switching over.
Back up both the source and target systems, if you can.
Check data integrity between source and target machines.

Here’s each recommended task in detail.

To Do A Few Days To A Week Before Your Planned Switch

1. Contact IBM and third-party vendors to get any temporary keys needed for the switch over–If you don’t have permanent or disaster recovery keys for your CBU licensed software, most vendors will provide temporary keys that you can use for switch tests and temporary switch overs (30 days or less). Be sure to contact any relevant vendors before the switch over, to make sure their software will work on your CBU machine.

Things To Do Immediately Before Your Planned Switch

2. Quiet the source system–Make sure there are no active IBM i processes that can update the source database. In HA switchovers, you want to ensure that all transactions have been properly transmitted and applied to the target system before switching over. Some of the items you’ll need to shut down are:

Batch jobs that are currently running. Make sure they complete before quieting the system.
Job schedulers such as Robot/SCHEDULE or the IBM Job Scheduler, that submit new batch jobs to the system.
Web servers that can update database files.
Companion servers that are able to connect to your system and update data.
All IBM i third-party software such as credit card servers or other specialized software that updates the database.
QINTER and other interactive subsystems.
Utility jobs that periodically wake up and perform some database function.

The trick here is to shut off processing while keeping replication up (see step 3). So you have to be diligent in turning off anything that can update the database.

3. Make sure transactions haven’t come in after you turn off replication–Check your run book steps to ensure that you turn off replication after you quiet the system, not before. I’ve seen run books and exercises where the HA admin turns off replication first, then quiets the system, only to later discover that several orders or transactions have entered the source system that have not been replicated to the target system. Your HA replication should be the last item turned off on your source system before switching processing to your target CBU system.

4. Make sure all pending replicated transactions have been applied to your target system before switching over–Use your HA monitoring software to ensure all transactions from the source system have been sent over to the target system and applied. The goal is for the target system database to exactly match the production system database.

5. Back up both the source and target systems, if you can–Get a full system backup (GO SAVE, option 21) of your production and CBU data before the switch, so that you can restore critical files if unexpected data corruption occurs.

You should also make sure that your source system startup program doesn’t restart after the backup. Option 21 saves automatically after you restart your IBM i startup program upon completion, which will allow users, Web sites, and outside servers to attach to and start sending data to your database. So you’ll want to change your run book to add steps to keep the system from restarting after the backup. Check out this article on Preventing Your System From Restarting After a Full Backup for more information.

If you plan to IPL your source machine before switching over, check out this article on the Secrets of the IBM i IPL Parameters. IBM i systems have an IPL parameter called Start TCP/IP (STRTCP), which automatically starts TCP/IP processing after an IPL. The problem is that many of your TCP/IP servers may be configured to autostart whenever TCP/IP is started. If you decide to IPL your source system before the switch over, you could accidentally restart your TCP/IP servers (including your HTTP servers), causing changes to your database that may not be replicated before switch over. So check out the IPL parameters article, if your switch includes a source system IPL.

6. Check data integrity between source and target machines–I recommend that anyone preparing to switch to a CBU create a data integrities report that allows them to compare financial totals between your source and target system. The drill here is to run the same data integrity report on both systems before switching roles and compare the results. If the data integrity totals match, you have more confidence that there are no problems in the database. If the totals don’t match, it gives you an early warning that something is wrong with the data and you can investigate and resynchronize the systems before switching over. I’ve had clients who implemented data integrity checking, and it has caught mistakes both in their switch processing run book and in their replication scheme. It’s a good process to add to your run book.

Using Incremental IFS Backups With BRMS

In March, I wrote a tip on speeding up AS/400 Integrated File System (IFS) backups by using incremental backups. I demonstrated how to run an incremental IFS backup using green-screen commands. An Admin Alert reader wrote in and asked how to implement an incremental IFS backup using IBM’s Backup, Recovery, and Media Services (BRMS) licensed program. As I don’t use BRMS in my shop, I posed this question to the Admin Alert readership.

Reader Felipe Rodriguez wrote in with this tip on setting up IFS incremental backups in BRMS.

I do this within the BRMS control group (WRKCTLGBRM). I can confirm that this saves well over a million objects during a full save, and anywhere from a few hundred to a few thousand objects during incremental backups.

The BRMS control group attributes show the type of incremental save you want.

I use *CUML for the Incremental type parameter, as shown below. Or you could define it in the Backup Policy and use *BKUPCY.

Incremental type . . . . . . . . . . . . *CUML *CUML, *INCR, *BKUPCY

Then within the instructions in the control group itself, I select “I” for incremental in the Weekly Activity column, showing the type of backups to perform each weekday. Shown below, I do an Incremental save Sun-Fri, then a Full save on Sat.

Thanks, Felipe. So give this a try if you’re interested in performing IFS incremental backups in BRMS.

IBM Working On IBM i Software Licensing Scheme Change

If you haven’t read it yet, check out Alex Woodie’s article on possible new software licensing schemes that IBM is working on for IBM i shops. According to Alex, “IBM executives confirmed at the COMMON conference this month that they are working on at least one new licensing scheme, ostensibly to help MSPs sell cloud solutions, but perhaps for the wider IBM i world.”

This is something to keep an eye on, especially if you’ve been struggling with the intricacies of IBM i licensing. Alex also discusses software licensing from a third-party software prospective, which for many shops is a much bigger (and more expensive) problem than IBM licensing.

Check out Alex’s article for more details. Hopefully, we’ll hear more about this soon.

Follow Me On My Blog, On Twitter, And On LinkedIn

Check out my blog at joehertvik.com, where I focus on computer administration and news (especially IBM i); vendor, marketing, and tech writing news and materials; and whatever else I come across.

You can also follow me on Twitter @JoeHertvik and on LinkedIn.

Joe Hertvik is the owner of Hertvik Business Services, a service company that provides written marketing content and presentation services for the computer industry, including white papers, case studies, and other marketing material. Email Joe for a free quote for any upcoming projects. He also runs a data center for two companies outside Chicago. Joe is a contributing editor for IT Jungle and has written the Admin Alert column since 2002.