Discovering Which Locked Object is Holding Up Your Job
December 15, 2010 Hey, Joe:
I have a recurring problem with file locks on my system, where critical jobs are stuck in lock waiting status (LCKW) waiting for an object before crashing on a file allocation error. How can I tell which object the job is waiting for before it crashes and which user has that object locked? I’m running i5/OS V5R4M5.
There are two ways to figure out which locked object a job is waiting for. One way to locate it is by using i5/OS green screen commands; the other is by using iSeries Navigator (OpsNav). Here’s the drill for locating your locked object or record using either technique.
On the green screen, use the Work with Job (WRKJOB) command to view the job that is waiting on a record or file lock.
On the Work with Job screen that appears, select option 12 (Work with Locks) to see all the file locks the job is currently holding or requesting. You’ll see a screen that looks something like this.
Look for any entry that has a status of REQ (Request) with a member lock that says WAIT. In this case, my job is waiting for the SOURCE file in library JOEHTST. You can then run the following Work with Object Lock (WRKOBJLCK) command to determine who is holding the lock.
WRKOBJLCK OBJ(library/object_name) OBJTYPE(*FILE)
And this should tell you the name of the job that is locking your file. However, WRKOBJLCK doesn’t always provide the name of the locking job. There have been times when I’ve run WRKOBJLCK and it doesn’t let me see the name of the locking job. It just hangs and I can’t get any information. In that case, you just have to wait for the job to resolve the lock or try to locate the locked object by using OpsNav.
To perform the same process in OpsNav V5R4Mx, open the Work Management→Active Jobs node under your target partition. You’ll see a screen that looks something like this.
Right-click on the job that’s waiting for a lock and select Details→Locked Objects from the pop-up menu that appears.
Like its green-screen counterpart, the OpsNav Locked Objects screen will also show you the list of objects that the job is either currently holding or requesting.
Look for the object that has a Lock Status of Requested. Right-click on the object and select Lock Holders from the pop-up menu that appears. This will display a screen that shows which jobs are currently locking the object your job is trying to access. You can then make a decision on how best to proceed.
If you have the programming knowledge or third-party software that can detect and react to job messages when they occur, you can also configure your system to automatically answer your file or record allocation errors whenever they appear. We recently set up a system to do this by using Bytware’s MessengerPlus. Inside MessengerPlus, we set up a QSYSOPR message queue monitor to look for a specific error message that occurred whenever a job crashed because it was unable to obtain a locked record. We monitored for the following error message:
RNQ1218 - Unable to allocate a record in file &7 (R C G D F)
We configured the monitor such that when MessengerPlus detected an RNQ1218 allocation error message in QSYSOPR, it would respond with up to three R (Retry) responses within 15 minutes before alerting the operations staff that there was a problem with the system. We set the monitor up to check for RNQ1218 messages every minute to catch these errors as they occurred. If three R responses didn’t solve the problem, MessengerPlus would text the Operations staff for help.
If you don’t have third-party software that can be used to auto-answer these messages, you may be able to write a custom program that will retrieve and send retries R responses to the QSYSOPR messages, the same way that I was sending retries through my MessengerPlus software.
If you decide to set up an automatic retry routine for your file allocation messages, there’s an easy trick for viewing all your QSYSOPR inquiry messages along with how each message was answered, as well as who answered each message. To get a listing of all your inquiry messages and their replies, run the following green-screen Display Message (DSPMSG) command.
DSPMSG MSGQ(QSYSOPR) OUTPUT(*PRINT) MSGTYPE(*INQ)
This will produce a spooled file containing all the current and answered inquiry messages on your system, as well as what reply was sent to each message and who sent the reply. This spooled file can come in handy when you’re trying to determine if your auto-answer routine is working correctly. For more information on creating and interpreting the QSYSOPR inquiry message spooled file, check out my article on Solving i5/OS Inquiry Message Whodunits.