Dynamically Sized Arrays

Published: May 10, 2006

Hey, Ted:

I have created an array to sort customer shipment data in the correct order for all part numbers that need to ship on a particular day. I want to define the array to grow as needed, as we anticipate our business growing. I don't want the program blowing up when the number of elements is exceeded. Is it possible to dynamically size an array in an RPG program, rather than specify the number of elements at compile time?

--Jennifer

I have two ideas for you, Jennifer. The first is to use RPG's memory allocation features to get memory as you need it. Define the array to have some very large number of elements, more than you will ever use, but allocate only enough memory to hold the number of elements you need. If you run out of room, ask for more memory. The idea I'm proposing here is similar to the way physical files are allocated. When the file is full, the system extends the physical file size according to the increment value in the file description.

Here is an example I threw together to illustrate this approach.

H option(*srcstmt: *nodebugio)

FQCustCdt  if   e             disk    prefix(Cus_)

  // Pointer needed for memory allocation operations
D pInfo           s               *

  // Dynamically allocated array
D Info            ds                  based(pInfo)
D CustInfo                            dim(1000)
D  Name                         14a   overlay(CustInfo:*next)
D  Number                        6p 0 overlay(CustInfo:*next)
D  State                         2a   overlay(CustInfo:*next)

  // number of elements that have been allocated for the 
array to use
D NbrAllocated    s             10u 0
  // number of elements that have been loaded with values
D NbrInUse        s             10u 0
  // Initial number of array elements
D NbrInit         c                   const(10)
  // Number of elements to extend the array by
D NbrIncr         c                   const(5)

 /free
    *inlr = *on;

    // Initial allocation of array
    NbrAllocated = NbrInit;
    pInfo = %alloc(NbrAllocated*%size(CustInfo));

    // Load the customer file into the array
    dow '1';
       read CusRec;
  if %eof();
     leave;
  endif;
  NbrInUse += 1;

  // If there is no room left in the array, extend it.
       if NbrInUse > NbrAllocated;
          NbrAllocated += NbrIncr;
          pInfo = %realloc(pInfo:NbrAllocated*%size(CustInfo));
       endif;

       // Load the next array element.
       Name(NbrInUse) = %trimr(Cus_LstNam) + ',' + Cus_Init;
       Number(NbrInUse) = Cus_CusNum;
       State(NbrInUse) = Cus_State;
    enddo;

    // Sort the array into the proper sequence
    sorta %subarr(Name: 1: NbrInUse);
    sorta %subarr(State: 1: NbrInUse);

    return;
 /end-free

This program reads three fields of the QCUSTCDT file, which is in library QIWS, into array CUSTINFO. I've allowed for 1,000 elements in array CustInfo. Since CustInfo is in a data structure that is based on pointer pInfo, I have to use memory allocation operations to assign memory to the array.

Take a look at the calculations. The %ALLOC function assigns enough memory to hold an initial 10 elements. As I read each record of QCUSTCDT, I determine whether there is enough memory in the array to hold another element. Whenever the array fills up, I use the %REALLOC function to reallocate the array, adding memory for another five elements.

Once the array is loaded, I can sort it. You'll see a couple of examples of sorting toward the end of the program. Notice that the %SUBARR function, which was introduced with V5R3, allows me to sort only the portion of the array that I have loaded.

I have to give the credit for the second approach to Barbara Morris of IBM. She suggested using an automatically extending user space to hold the array. I didn't know such an animal existed.

The idea is that you first create a user space. Here's a quick program I threw together to create a user space called DYNARR in library QTEMP.

D CreateUserSpace...
D                 pr                  extpgm('QUSCRTUS')
D   NameAndLib                  20a   const
D   ExtAttr                     10a   const
D   InitSize                    10i 0 const
D   InitValue                    1a   const
D   PublicAut                   10a   const
D   Description                 50a   const
D   Replace                     10a   const
D   ErrorStruct                 16a
D
D ChangeUserSpaceAttributes...
D                 pr                  extpgm('QUSCUSAT')
D   LibraryName                 10a
D   NameAndLib                  20a   const
D   Attributes                  10a   const
D   ErrorStruct                 16a
D
D AttrList        ds                  qualified
D  Size                         10i 0
D  Key                          10i 0
D  DataLength                   10i 0
D  Data                          1a

D SpaceName       s             20a   inz('DYNARR    QTEMP')
D RtnLib          s             10a
D ErrorStruct     s             16a   inz(*allx'00')

 /free
    *inlr = *on;
    CreateUserSpace (SpaceName: *blanks : 32767:
                     X'00': '*ALL': *blanks: '*YES': ErrorStruct);

    AttrList.Size = 1;  // 1 attribute will be changed
    AttrList.Key = 3;   // 3=extensibility attribute
    AttrList.DataLength = 1;
    AttrList.Data = '1'; // user space is extensible
    ChangeUserSpaceAttributes (RtnLib: SpaceName: AttrList: ErrorStruct);

    return;

The program calls two APIs--one to create the user space and one to make it automatically extensible.

The second program loads and sorts the array.

H option(*srcstmt: *nodebugio)

FQCustCdt  if   e             disk    prefix(Cus_)

 // Pointer to the auto-extending user space
D pInfo           s               *

  // Dynamically allocated array stored in the user space
D Info            ds                  based(pInfo)
D CustInfo                            dim(1000)
D  Name                         14a   overlay(CustInfo:*next)
D  Number                        6p 0 overlay(CustInfo:*next)
D  State                         2a   overlay(CustInfo:*next)

  // number of elements that have been loaded with values
D NbrInUse        s             10u 0

  // API to retrieve a pointer to the user space
D RtvPtrToUsrSpc  pr                  extpgm('QUSPTRUS')
D   Name                        20a   const
D   Ptr                           *

 /free
    *inlr = *on;

    // Assign the array to the auto-extending user space
    RtvPtrToUsrSpc ('DYNARR    QTEMP': pInfo);

    // Load the customer file into the array
    dow '1';
       read CusRec;
       if %eof();
         leave;
       endif;

       NbrInUse += 1;

       // Load the next array element.
       Name(NbrInUse) = %trimr(Cus_LstNam) + ',' + Cus_Init;
       Number(NbrInUse) = Cus_CusNum;
       State(NbrInUse) = Cus_State;
    enddo;

    // Sort the array into the proper sequence
    sorta %subarr(Name: 1: NbrInUse);
    sorta %subarr(State: 1: NbrInUse);

    return;
 /end-free

Notice that the memory allocation logic is gone, replaced by a call to the QUSPRTUS API, which retrieves a pointer to the user space and assigns it to pointer pInfo.

I should mention a couple more things. First, in these quickly thrown together examples, I did not check to make sure that I never exceed the number of elements in the DIM keyword. I've written these short programs with the assumption that I will never get near that number, and that assumption was valid for my example because file QCUSTCDT only has a dozen records. However, in a production application, you should check for this possibility.

Second, I wrote two programs to illustrate the second approach--one to create the user space and one to process it. I only did that for convenience. You may want to combine the two into one. However, a better idea may be to implement the first one as a utility program. It seems to me that extensible user spaces could come in handy for more applications than this one.

--Ted