Guru: Watch Out For This Pitfall When Working With Integer Columns
May 22, 2023 Gregory Simmons
Remember that awesome jungle game where you had to guide the hero through a series of increasingly hard obstacles to gather treasure? Jumping over snakes, scorpions, and rolling logs, swinging on vines over alligators –your timing had to be just right. Pitfall Harry survived in a world of 255 screens, each of which were 160 x 192 pixels and a dazzling 128 colors! Well today, I want to make you aware of a pitfall which caught me off guard a while back. I hope this article will help you avoid my pitfall.
There I was, writing my code, happy as a lark, when all of a sudden, testing brought a bug to my attention. When I debugged and tracked down the issue, I was astonished to find that my integer field was topping out at 999,999,999 instead of the expected 2,147,483,647. I will demonstrate the issue and how to correct it. Just as the beloved Atari game Pitfall from my youth, this trap has three alligators or gotchas to be aware of and avoid.
So, let’s create a quick table to play with. This is a simple table which just contains three columns of the three types of integers
Create or Replace Table My_Int_File For System Name MyIntFile ( My_Small_Int For MySmallInt SmallInt Default 0 Not Null, My_Int For MyInt Int Default 0 Not Null, My_Big_Int For MyBigInt BigInt Default 0 Not Null ) RcdFmt rMyIntFile;
This table above should have three fields, capable of holding data values as such:
|SMALLINT||Used to store small integers with a precision of 15 bits.||-32,768 to +32,767|
|INT||Used to store large integers with a precision of 31 bits.||-2,147,483,648 to +2,147,483,647|
|BIGINT||Used to store big integers with a precision of 63 bits.||-9223372036854775808 to +9223372036854775807|
Now, let’s write a small RPG program to experiment with these integer columns. For this example, I will just play with the myInt column.
1 **free 2 Dcl-ds dsMyIntFile Extname('MYINTFILE') Qualified End-ds; 3 Dcl-s myInt10 Int(10); 4 dsMyIntFile.myInt = *hival; 5 myInt10 = *hival; 6 *InLr = *On; 7 Return;
Line 1: **Free – We’ll be going 100 percent RPG free for this program.
Line 2: I’ve defined a data structure like the file that we just created.
Line 3: For later comparison, I’ve created a standalone variable of type Int(10).
Line 4: Assign *hival to the field column in my data structure, mySmallInt.
Line 5: Assign *hival to the stand alone variable, myInt10.
Now for the interesting part. When I run this program and stop it in debug on line 6 and add my two fields to the monitors tab in the RDi debugger perspective. This is what I get:
Did any of you cock your head to the side as I did when I first saw this? Why didn’t dsMyIntFile.MyInt yield the same as MyInt10, 2147483647? After some head scratching and research, I discovered that by default, RPG treats integer fields in externally described files and data structures as BINDEC or Binary Decimal fields.
Thankfully, this is an easy fix. There is a control option available to make integers in externally described files and data structures to behave the same as internally defined standalone variables. So, one small alteration to my test program:
1 **free 2 Ctl-Opt ExtBinInt(*Yes); 3 Dcl-ds dsMyIntFile Extname('MYINTFILE') Qualified End-ds; 4 Dcl-s myInt10 Int(10); 5 dsMyIntFile.myInt = *hival; 6 myInt10 = *hival; 7 *InLr = *On; 8 Return;
Line 2: The ExtBinInt(*YES) control option allows your program to make use of the full range of numbers for that type of integer.
Now, when I rerun my program in debug, these are the values I get:
All better! Ever since this discovery, I use this control option in all of my RPG programs to keep me protected from this pitfall.
Until next time, happy coding.
Gregory Simmons is a software engineer with PC Richard & Son. He started on the IBM i platform in 1994, graduated with a degree in Computer Information Systems in 1997 and has been working on the OS/400 and IBM i platform ever since. He has been a registered instructor with the IBM Academic Initiative since 2007, and holds a COMMON Application Developer certification. When he’s not trying to figure out how to speed up legacy programs, he enjoys speaking at COMMON and other technical conferences, running, backpacking, SCUBA diving, hunting, and fishing.
Thanks, didn’t know that. Never stumble upon it to be honest… it could be that – for some reason – many files in DDS are defined with ZONED or PACKED and rarely using integer, at least in many programs I see on the platform… but… good to know especially when SQL DDL is employed (i.e. auto numerators etc.).
I remember when IBM added ExtBinInt(*YES) to RPG IV. I thought “finally!” Integers that behaved like Integers! Just wish the list of CTL-OPT keywords we all have to use in most of our programs were implemented as standards in a new version of RPG… but I can live with it as is I suppose. Good recommendation, Greg.
I think this is fascinating. I tried your code for the small integer. Thinking the upper limit is +32,767 when in fact the upper limit is only +9,999 is mind blowing.