Stuff
OS/400 Edition
Volume 1, Number 10 -- May 23, 2002

Data Queue Basics


by Kevin Vandever

[The code for this article is available for download.]

Everyone has used Elmer's glue, right? You remember the commercial, don't you? It boasted 101 uses for the gooey white stuff. Well, I use a tool that just may be the Elmer's glue of iSeries programming. It can't mend your broken glasses or hold a meeting together, but it does allow you to pass data to and from programs, trigger events, and create flexible subfile applications. That tool is the data queue.

What Are Data Queues?

Data queues are a type of system object (type *DTAQ) that you can create and maintain using OS/400 commands and APIs. They provide a means of fast asynchronous communication between two jobs, or within the same job, because they use less system resources than database files, message queues, or data areas. You can attach a sender ID to each message being placed on a data queue. The sender ID, an attribute of the data queue that is established when the queue is created, contains the qualified job name and current user profile. Also built into data queues is the ability to set the length of time a job will wait for an entry before continuing its processing. A negative wait parameter will tell the job to wait indefinitely for an entry before processing. A wait parameter of 0 to 99999 will tell the job to wait that number of seconds before processing. When reading messages from a data queue, you have the option to peek at the entry, meaning that your application receives the message but leaves it on the queue; or, you can destroy the entry, meaning that your application receives the entry and removes it from the queue. And, as of V5R1, you can now journal data queue entries.

High-level language programs can send data to a data queue using the QSNDDTAQ (send to a data queue) API and receive data using the QRCVDTAQ (receive from a data queue) API, as well as others. Data queues can be read in a first in, first out (FIFO) sequence, or in a last in, first out (LIFO) sequence, or in keyed sequence. Keyed data queues allow the programmer to specify a specific order in which to receive messages on the data queue or to retrieve only data queue messages that meet a criterion. That is, the programmer can receive a data queue message that is equal to (EQ), greater than (GT), greater than or equal to (GE), less than (LT), or less than or equal to (LE) a search key.

Why Use Them?

As is the case with Elmer's glue, there are a bunch of uses for data queues. I don't want to limit you by trying to list all of them here, because you may already possess, or come up with, your own set of uses. However, I will mention a few standard tasks for which data queues are perfect. For one, they are great for passing data between programs. I realize that you can pass data between programs using call parameters, but if you're interested in decoupling--that is, decreasing the dependency that programs have on each other--data queues allow you to accomplish that. Also, if you want to pass data between programs on different machines, even if one of those programs resides on a machine other than the iSeries, the data queue is a great option. Another twist on the passing-data objective is the client/server application. In the client/server scenario, you have many programs (or clients) writing different types of data to the same data queue. These clients can reside locally or remotely and can be written in many different languages. The server receives the various data queue messages and processes them accordingly, which may mean calling other programs or triggering other events. I will show you a simple client/server application later in this article. Another place where data queues come in handy is in subfile applications. I know, it sounds a little odd, but using data queues with subfiles allows you to create very flexible applications with relative ease. Curious? Check out "Keyed Data Queues: The Key to Flexible Subfiles." For now, let's move on to some more data queue basics.

Commands and APIs

There are three commands and five APIs that you can use with data queues. I am going to list all of them but focus only on the few that you will use most of the time. Also, if you use TAA tools, there are a couple of helpful commands included that I like to use. The following is a list of the commands and APIs:

  • CRTDTAQ—The create data queue command is used to create data queues.

  • DLTDTAQ—The delete data queue is used to delete data queues. Following so far?

  • WRKDTAQ—The work with data queues allows you to list one to many queues. From this screen, you can delete data queues, change the descriptions of data queues, and add new data queues.

  • CLRDTAQ—This is not a standard OS/400 command. It is part of the TAA tools and is used to clear a data queue's messages.

  • DSPDTAQ—This is also part of the TAA tools and not a standard OS/400 command. It is used to display the contents of a data queue.

  • DSPDTAQD—Are you starting to see that the best data queue commands are not standard OS/400 commands? This command is also part of the TAA tools and allows you the view the details of a data queue, including how many entries currently exist.

  • QCLRDTAQ—This API allows you to clear a data queue's messages. If you don't have TAA tools, you could create your own CLRDTAQ command using this API.

  • QRCVDTAQ—This API allows you to receive data queue entries.

  • QMHQRDQD—This API allows you to retrieve data queue information. You could use this API to create your own DSPDTAQD command if you don't have TAA tools.

  • QMHRDQM—This API allows you to receive an entry from the data queue without removing that message from the data queue. Use this API to create your own DSPDTAQ command.

  • QSNDDTAQ—This API allows you to write messages to a data queue.

Now I will further discuss the CRTDTAQ command and the QSNDDTAQ and QRCVDTAQ APIs. The CRTDTAQ command is not complicated, but there are a few options from which you can choose, so I would like to discuss those with you. Check out the figure below. After typing in the data queue name and library, the next entry is data queue type. You have two choices: *STD or *DDM. A standard (*STD) data queue is one that is created on the local machine.


A DDM data queue is the same as a DDM file. It is a logical view of a data queue on another machine.

A *STD data queue is created on one iSeries machine and you create a DDM data queue on another.

A DDM data queue points to the *STD data queue on the other machine. After the type parameter comes the maximum entry length. It is here that you define the maximum length of your data queue message. The maximum length you can use is 64512 bytes, or 64K. The next parameter I want to talk about is the sequence parameter. You have three choices: *FIFO, *LIFO, or *KEYED. If you select *KEYED, you are then prompted for the key length. Just as with a database file, the key length is the length of the data you are going to use as the key to the data queue. Next, you determine whether or not you want to include the sender ID, which is the job name and user's profile, with each message written to the queue. The next parameter is broken up into two parts and is used to determine the maximum size of the data queue and the initial number of entries. The maximum size can go to 2 gig. The initial number of entries tells OS/400 how much space to reserve when the queue is created. You can extend the queue up to the maximum size, but, depending on your application, you may want to reserve a specific amount of space to start off. The default is 16. A new parameter and feature of data queues introduced in V5R1 is the automatic reclaim feature. This allows OS/400 to reclaim unused space in a data queue when that queue is empty. The need for this feature comes from the fact that the data queue size extends as you add messages beyond what you set in the initial number of entries parameter. Answering *YES to this parameter causes OS/400 to automatically reclaim the storage whenever the data queue is empty. The size of the data queue will then go back to the initial size at creation, which is determined by the maximum message size and number of initial entries parameters. One note of caution if you use this feature is that the data queue is locked while this reclaim takes place.

Client/Server Application

To better illustrate the APIs, I thought I would show them to you in action. DQClient and DQServer are RPG programs that employ data queue APIs. DQClient writes messages to a data queue using the QSNDDTAQ API. In our make-believe client/server world, it is one of many clients that will write to the same data queue, all with different messages. DQServer will use the QRCVDTAQ API to receive messages from all the clients and process them accordingly. The rest of the RPG is pretty straightforward stuff, so I'm just going to concentrate on the API code.

Take a look at DQClient. It calls the QSNDDTAQ API, which accepts four parameters. If I would have used *KEYED data queues, I would have added two more parameters to determine the key and key length, but in this example I am writing to a *FIFO data queue so that the server can receive entries in the order in which they are written. Below is the code, with an explanation in parentheses, that writes to the data queue.

 C   call 'QSNDDTAQ'
C  parm      queue (Name of data queue)
C  parm      lib   (Library Containing data queue)
C  parm      len   (Length of data being written to data queue)
C  parm      data  (Actual data being written to data queue)

This particular client will write a message type in the first two bytes, followed by the actual data that is required. In this case the two-byte type is OH, for order header, and the rest of the data is the actual order number. As you'll see shortly, the server will read that message and, based on the message type, process that message accordingly. Let's take a look at the server.

DQServer is a never-ending job, in that it will wait on the data queue, process the entry when one is written from one of the clients, and wait for the next entry. The job will only end normally when a client sends a special shut-down request. To run this program, you would submit it using the SBMJOB command. You could call it from a command line, but since it is a never-ending job, it will lock up your interactive session. This program uses the QRCVDTAQ API with a wait parameter of –1. Remember earlier in the article I mentioned that a negative number in the wait parameter meant that the program would wait indefinitely on the queue? Below is the code, with an explanation in parentheses, to receive from the queue.

C  call 'QRCVDTAQ'
C  parm   queue (Name of data queue)
C  parm   lib (Library containing data queue)
C  parm   len (Length of data received from data queue)
C  parm   data (Actual data received from data queue)
C  parm   wait (Time to wait before continuing processing)

As you will see in my other article, there are other parameters you can use with this API, but since this is a *FIFO queue containing no sender ID, these five parameters are all you need. Once the message is received, the program interrogates the first two bytes to determine the message type and processes the appropriate logic. In this case, my client sent an OH, so a program is called to create and order header record.

Sniff This!

I have provided you some very basic data queue concepts upon which to build. However, even though this is basic stuff, it has a practical, real-world application. Most data queue processing I do includes *FIFO data queues. Check out my other article in this issue to see how to use *KEYED data queues in a subfile application. Also, mess around with the other commands and APIs I mentioned in this article. Soon, you too will see why data queues are the Elmer's glue of iSeries programming.


Sponsored By
ALDON COMPUTER GROUP

Free Whitepaper on Application Modernization Through Multi-Platform Development.

Aldon Computer Group, the industry leader in multi-platform change management, offers a cutting-edge solution for companies seeking e-business success. Aldon Affiniti is the only application development tool that allows you to manage all your code, whether RPG, JAVA, COBOL, C++, or anything else from ONE single screen.


Find out more today at www.aldon.com. (And while you're at it, don't forget to sign up for a FREE online seminar on multi-platform development.)



THIS ISSUE
SPONSORED BY:

ASNA
Aldon Computer Group
LANSA
Advanced Systems Group
Profound Logic Software
WorksRight Software


BACK ISSUES

TABLE OF CONTENTS
JavaScript and the SHELL Command

Data Queue Basics

Simplify Java Web App Deployment with WAR Files

The Art of Globbing

Keyed Data Queues: The Key to Flexible Subfiles

Work with Active Jobs from Operations Navigator

Editors
Shannon O'Donnell
Kevin Vandever

Managing Editor
Shannon Pastore

Contributing Editors:
Howard Arner
Joe Hertvik
Ted Holt
David Morris
Richard Shaler

Contact the Editors
Do you have a gripe, inside dope or an opinion?
Email the editors:
editors@itjungle.com



Last Updated: 5/22/02
Copyright © 1996-2008 Guild Companies, Inc. All Rights Reserved.