• The Four Hundred
  • Subscribe
  • Media Kit
  • Contributors
  • About Us
  • Contact
Menu
  • The Four Hundred
  • Subscribe
  • Media Kit
  • Contributors
  • About Us
  • Contact
  • The Efficiency of Varying Length Character Variables

    September 10, 2008 Jon Paris

    Remember the bad old days when dinosaurs still roamed the earth and the only way to build strings in RPG involved playing silly games with arrays? Or worse still, obscure combinations of MOVE operations? Thankfully those days are far behind us–although sadly there are still a few RPG/400 dinosaurs coding away!

    RPG IV introduced many powerful new string handling options, such as the %TRIMx family of BIFs, but even now there are capabilities in the language that few programmers fully exploit. One of my favorites is variable length fields. There are many good reasons to use these fields, but in this tip we’re going to focus mainly on performance.

    For those of you unfamiliar with varying length fields, the following D specs show how they are defined and illustrate the constituent parts.

    Varying length fields have two components: the current length that is represented by a 2-byte integer in the first two positions, followed by the actual data. They are differentiated from regular character fields by the use of the keyword “Varying.” (See (A) in the code that follows.)

    You should train yourself to always code the INZ keyword to ensure that the length field is set correctly. This is critical when varying length fields are incorporated in data structures. Why? Because by default, data structures are initialized to spaces (hex 40) and that causes havoc when interpreted as the field length! At (B) and (C) in the code example that follows, I have defined the two components as separate fields–overlaying varyField–to demonstrate the layout.

         D varyingStruct   DS 
    (A)  D  varyField                   256a   Varying Inz
    // Following fields are defined just to show the layout of a varying field
    (B)  D   length      5i 0 Overlay(varyField)
    (C)  D   data      256a   Overlay(varyField: *Next)
    

    Whenever the content of a varying length field is changed, the compiler adjusts the length to reflect the new content. Note that you should always use %Trimx when loading data from a fixed length field into a varying length field, otherwise any trailing blanks will be counted in the field length. Any time you want to know how long the field is, use the %Len() built-in function to obtain the current value.

    Now that we’ve reviewed the basics of variable length fields, let’s see how they can be used to boost the performance of some types of string operation. Take a look at the following two pieces of code. Both of them build a string of 100 comma separated values. At first glance there is very little difference in the logic, but would you believe that the second one can run hundreds or even thousands of times faster?

           For i = 1 to 10;
             For j = 1 to 10;
    (D)        fixedField = %Subst(baseString: i: j );         
    (E)        longFixed = %TrimR(longFixed) + ',' + fixedField; 
             EndFor;
           EndFor;
    
           For i = 1 to 10;
             For j = 1 to 10;
               fixedField = %Subst(baseString: i: j ); 
    (F)        longVarying += ',' + %TrimR(fixedField);
             EndFor;
           EndFor;
    

    The reason is simple. The second one (F) makes use of a varying length field to build up the result string! This difference in speed is easy to understand if you think about what is going on under the hood. The first version (E) uses a fixed length target string so these are the steps that take place:

    • Work out where the last non-space character is.
    • Add the comma in the next position.
    • Add the content of fixedField in the next and subsequent positions.
    • If longFixed is not yet full, add blanks to fill it.

    This process is repeated for each new value added to the string. Notice that having carefully padded the string with blanks (4), the very next thing we do (1) is to work out how many there are so that we can ignore them!

    Contrast this with the mechanics of the version using the variable length field (F):

    • Increment the field length by 1, and place the comma in that position.
    • Determine the length of the field to add (i.e., ignoring trailing spaces).
    • Copy the new data in starting at the field length + 1 position incrementing the field length.

    Much simpler! And the resulting speed differences can be staggering. In tests I ran while preparing this tip, even with a target field length as small as 256 characters, the varying length field version took only half the time of the fixed length version. When I raised the field length to 25,600, which is a much more realistic size when building a CSV, HTML or XML string, the speed difference rose to 1,300 to 1!

    Another point to consider is that the code shown above (E) is already much more efficient than much of the code I have seen in customers’ programs. The two variants below are both very common and both even less efficient. In the first case (G) the field being added is being trimmed of blanks, which are immediately added back if it does not fill the target field! In the second case (H) the separation of the two functions means that the calculations for the effective length of the target field and the subsequent blank filling occur twice for each loop. You can imagine what that does to the speed. And yes, I have seen cases where people combine both G and H!

    (G)        longFixed = %TrimR(longFixed) + ',' + %TrimR(fixedField); 
    
    (H)        longFixed = %TrimR(longFixed) + ','; 
               longFixed = %TrimR(longFixed) + fixedField; 
    

    That’s all for this first look at variable length fields. In a future tip we’ll look at their uses and abuses in the database.

    P.S. For those of you wondering what the purpose of the code at (D) is, it is simply used to generate fields of different effective lengths (one to 10 characters) to act as the test data to be added to the target string.

    Jon Paris is one of the world’s most knowledgeable experts on programming on the System i platform. Paris cut his teeth on the System/38 way back when, and in 1987 he joined IBM’s Toronto software lab to work on the COBOL compilers for the System/38 and System/36. He also worked on the creation of the COBOL/400 compilers for the original AS/400s back in 1988, and was one of the key developers behind RPG IV and the CODE/400 development tool. In 1998, he left IBM to start his own education and training firm, a job he does to this day with his wife, Susan Gantner–also an expert in System i programming. Paris and Gantner, along with Paul Tuohy, are co-founders of System i Developer, which hosts the new RPG & DB2 Summit conference. Send your questions or comments for Jon to Ted Holt via the IT Jungle Contact page.



                         Post this story to del.icio.us
                   Post this story to Digg
        Post this story to Slashdot

    Share this:

    • Reddit
    • Facebook
    • LinkedIn
    • Twitter
    • Email

    Tags:

    Sponsored by
    PERFSCAN

    Revolutionary Performance Management Software

    At Greymine, we recognize there is a void in the IT world for a dedicated performance management company and also for a performance management tool that’s modern, easy to use, and doesn’t cost an arm and a leg. That’s why we created PERFSCAN.

    PERFSCAN is designed to make your job easier. With revolutionary technology, an easy-to-read report and graphics engine, and real time monitoring, tasks that used to take days can now take minutes. This means you will know your system better and will be able to provide better service to your customers.

    OUR FEATURES

    PERFSCAN is full of robust features that don’t require you to take a three-day class in order to use the product effectively.

    Customizable Performance Reporting

    Whether you are troubleshooting a major system problem or simply creating a monthly report, PERFSCAN lets you select any combination of desired performance metrics (CPU, Disk, and Memory).

    User Defined Performance Guidelines

    No matter if you are a managed service provider managing complex systems in the cloud or a customer analyzing your on-premises solution, PERFSCAN gives you the flexibility to define all mission critical guidelines how they need to be.

    Understanding The Impact Of Change

    Tired of all the finger pointing when performance is suffering? PERFSCAN’s innovative What’s Changed and Period vs. Period analysis creates a culture of proof by correlating known environmental changes with system performance metrics.

    Comprehensive Executive Summary

    Creating performance graphs is easy. Understanding what they mean is another thing. With one mouse click, PERFSCAN includes an easy-to-understand executive summary for each core metric analyzed.

    Combined Real-Time Monitor And Performance Analysis Tool

    With PERFSCAN’s combined built in enterprise real-time monitor and historical performance analysis capability, you will always know how your mission-critical systems are performing.

    Cloud Performance Reporting Is Easy

    Managing performance for production systems in the cloud can be a black hole to many system administrators. The good news is PERFSCAN analyzes all core metrics regardless of the location. That’s why MSPs and customers love PERFSCAN.

    Detailed Job Analysis

    PERFSCAN shows detailed top job analysis for any desired period. All metrics are displayed in two ways: Traditional Report and Percentage Breakdown Pie Chart. This toggle capability instantly shows the jobs using the most system resources.

    Save Report Capability

    Your boss lost the report you gave to him on Friday. Now what do you do? With PERFSCAN’s save report capability, any report can be retrieved in a matter of seconds.

    Professional PDF Reporting With Branding

    Creating professional looking reports for your customers has never been easier with PERFSCAN. Branding for our partners and service provider customers is easy with PERFSCAN.

    Check it out at perfscan.com

    Share this:

    • Reddit
    • Facebook
    • LinkedIn
    • Twitter
    • Email

    Sponsored Links

    ProData Computer Services:  Access remote databases from RPG, Webinar, Sept. 17, 2 p.m. CDT
    MoshiMoshi:  Episode Three now showing! Watch and Win!
    COMMON:  Join us at the Focus 2008 workshop conference, October 5 - 8, in San Francisco, California

    IT Jungle Store Top Book Picks

    Easy Steps to Internet Programming for AS/400, iSeries, and System i: List Price, $49.95
    Getting Started with PHP for i5/OS: List Price, $59.95
    The System i RPG & RPG IV Tutorial and Lab Exercises: List Price, $59.95
    The System i Pocket RPG & RPG IV Guide: List Price, $69.95
    The iSeries Pocket Database Guide: List Price, $59.00
    The iSeries Pocket Developers' Guide: List Price, $59.00
    The iSeries Pocket SQL Guide: List Price, $59.00
    The iSeries Pocket Query Guide: List Price, $49.00
    The iSeries Pocket WebFacing Primer: List Price, $39.00
    Migrating to WebSphere Express for iSeries: List Price, $49.00
    iSeries Express Web Implementer's Guide: List Price, $59.00
    Getting Started with WebSphere Development Studio for iSeries: List Price, $79.95
    Getting Started With WebSphere Development Studio Client for iSeries: List Price, $89.00
    Getting Started with WebSphere Express for iSeries: List Price, $49.00
    WebFacing Application Design and Development Guide: List Price, $55.00
    Can the AS/400 Survive IBM?: List Price, $49.00
    The All-Everything Machine: List Price, $29.95
    Chip Wars: List Price, $29.95

    Food Companies Can Save Millions on Recalls, Lawson Says Look for an Interim i Release Next Year, and i 6.2 in Early 2010

    Leave a Reply Cancel reply

Volume 8, Number 31 -- September 10, 2008
THIS ISSUE SPONSORED BY:

Help/Systems
ProData Computer Services
System i Developer

Table of Contents

  • The Efficiency of Varying Length Character Variables
  • SQL Assist: Powerful Interactive SQL
  • Admin Alert: Getting into a i5/OS Restricted State

Content archive

  • The Four Hundred
  • Four Hundred Stuff
  • Four Hundred Guru

Recent Posts

  • Power10 Boosts NVM-Express Flash Performance
  • Fortra Completes Postmortem Of GoAnywhere Vulnerability
  • Guru: Binding Directory Entries
  • How Does Your Infrastructure Spending Stack Up To The World?
  • IBM i PTF Guide, Volume 25, Number 22
  • Fortra Issues 20th State of IBM i Security Report
  • FNTS Launches Managed Services for Power Servers in IBM Cloud
  • Total LTO Shipped Capacity Up Slightly in 2022
  • Four Hundred Monitor, May 24
  • Update On Critical Security Vulnerability In PowerVM

Subscribe

To get news from IT Jungle sent to your inbox every week, subscribe to our newsletter.

Pages

  • About Us
  • Contact
  • Contributors
  • Four Hundred Monitor
  • IBM i PTF Guide
  • Media Kit
  • Subscribe

Search

Copyright © 2023 IT Jungle