Quick And Handy RPG Input

Life was so much simpler when terminals and printers ran on twinax, Java was an island or coffee, and all my data was structured in records. I don’t have to fool with wiring at present, and the Java programming language is to me only an occasional, plodding nuisance, but data is a different story.

Unstructured, non-relational data comes at me from all directions at an ever-increasing velocity, and no wonder, as stream I/O is the standard for much of the computing world. (For the benefit of anyone who’s interested, I’ve added a second article to the end of this article. It is an inadequate contrast of record and stream I/O. Two for the price of one! What a bargain!)

But to return to the matter at hand, making sense of unstructured data is, fortunately, not difficult for a craftsman like you.

IBM has given us some help with non-relational data by means of the XML-processing op codes–XML-SAX and XML-INTO. These are fine for XML, but what if you have to read JSON or CSV? Maybe then you use another wonderful tool from IBM: RPG Open Access. But then again, maybe all you really need is an easily written RPG subprocedure.

I can illustrate what I’m talking about with two simple (and I do mean simple) routines that I wrote and have used and re-used to deal with unstructured input.

1. A certain Web service with which I had to communicate sent responses as plain-text files, which I stored in the IFS. I had to extract information from the text and update the database. I used a subprocedure similar to this one:

D String          s           1024a   varying template

  // ===================================================================
  // Retrieve a value from a string
  //
  // This routine retrieves whatever's between two strings.
  //
  // For example:
  //
  //     Buffer:     'You owe us $7,538,266.95.  Pay now or else!'
  //     LeadDelim:  'owe us '
  //     TrailDelim: '.  Pay'
  //     Returns:    '$7,538,266.95
  
  // ===================================================================
P ExtractString   b
D                 pi                  like(String)
D  inBuffer                   4096a   const
D  inLeadDelim                        like(String) const
D  inTrailDelim                       like(String) const
  // locals
D LeadPos         s             10i 0
D TrailPos        s             10i 0
D DataPos         s             10i 0

D EmptyString     c                   const('')

 /free
     LeadPos = %scan(inLeadDelim: inBuffer);
     if (LeadPos <= *zero);
        return EmptyString;
     endif;

     DataPos = LeadPos + %len(inLeadDelim);
     TrailPos = %scan(inTrailDelim: inBuffer: DataPos);
     if (TrailPos <= *zero)
     or (TrailPos = DataPos);
        return EmptyString;
     endif;

     return %subst(inBuffer: DataPos: TrailPos - DataPos);

 /end-free
P                 e

Suppose that the contents of the IFS file are in a variable called Text, and that Text has this value:

Order 12345 was added to the database.

Extracting the order number requires nothing more than a simple call:

AmtOwed = ExtractString (Text: 'Order ': ' was');

2. Non-relational data are often separated by all sorts of values, e.g., commas, colons, semicolons, and various two-character combinations. Here’s a routine for that sort of data.

D String          s           1024a   varying template

  // ============================================================
  // Divide a string into two parts using a string to divide them
  
  // Example:
  //   DivideString (SomeString: ',': String1: String2);
  //       SomeString = 'ABC,XYZ'
  //       String1    = 'ABC'
  //       String2    = 'XYZ'
  // ============================================================
P DivideString    b
D                 pi
D  inString                           like(String) const
D  inDivider                          like(String) const
D  ouLeftString                       like(String)
D  ouRightString                      like(String)
D // locals
D pos             s             10i 0
D LeftString      s                   like(String)
D RightString     s                   like(String)
D EmptyString     c                   const('')

 /free
     LeftString = EmptyString;
     RightString = EmptyString;

     pos = %scan(inDivider: inString);

     select;
        when pos = *zero;
           LeftString = inString;
        when pos = 1;
           if %len(inString) > %len(inDivider);
              RightString = %subst(inString: %len(inDivider)+1);
           endif;
        when pos + %len(inDivider) >= %len(inString);
           LeftString = %subst(inString:1:pos-1);
        other;
           LeftString = %subst(inString:1:pos-1);
           RightString = %subst(inString:pos+%len(inDivider));
     endsl;

     ouLeftString  = LeftString;
     ouRightString = RightString;
 /end-free
P                 e

You might use this routine to peel off the values of a CSV file one by one. Or you might use it with a file in what I call configuration format:

Customer=12345
AmtDue=73747.00
DateDue=2014-03-19

Calling this routine within a loop extracts keywords and values.

DivideString (Text: '=': KeyWord: Value);

Like the example unstructured output routine I presented last week, these are very simple routines, yet they make quick work of common tasks.

Does your shop have a library of reusable subprocedures? (Tip: TOOLKIT makes a good name. TOOLBOX is also good.)

Does your tools library have a binding directory listing the subprocedures and modules to which your developers can bind? (Tip: You can name the binding directory the same as the library.)

Does your tools library have a source physical file for procedure prototypes for your reusable routines? (Tip: PROTOTYPES makes a good, descriptive source physical file name.)

If so, you’re doing great! You can inform anyone who asks that you’re prepared to process unstructured, non-relational data, whether input or output, at a moment’s notice.

If not, why not?

by Ted Holt

When I set out to get my computer science degree, the only programming I knew was business programming using COBOL and RPG II. That means the only I/O I had any experience with was with data structured in fixed-format records. Soon I was writing Pascal and was adept with another type of I/O: stream I/O.

There’s no need for me to explain the organization of record-based data to you, an astute reader of this august publication. You already know that one record represents one entity–a customer, an employee, an invoice, a line of an invoice, etc. You know that records are divided into fields, which are attributes of the entity. And you know that a collection of records of the same type is called a file. This is the world we live in, although nowadays we use relational database terms: table, row, and column.

Stream data is almost always unstructured. (I say almost, because it doesn’t have to be unstructured.) It can take multitudinous forms. The data may or may not be divided into lines. Lines may be terminated with any character or combination of characters, the most common of which are carriage return and line feed. A line may or may not be divided into fields, which are typically of no certain length and may be separated from one another by commas, tabs, or any creation of someone’s productive imagination.

Stream I/O is the norm for many, primarily non-business, computing environments. The following table gives a few examples of stream operations.