• The Four Hundred
  • Subscribe
  • Media Kit
  • Contributors
  • About Us
  • Contact
Menu
  • The Four Hundred
  • Subscribe
  • Media Kit
  • Contributors
  • About Us
  • Contact
  • Johnny Can’t Read and RPG Can’t Do Math

    October 25, 2002 Timothy Prickett Morgan

    Hey, Ted:

    I ran into an anomaly and was wondering if you could shed some light on it.

    A subprocedure that I wrote doesn’t return the correct value. The problem is caused by an expression that doesn’t return the value that I think it should.

    Why does the expression, which returns the correct value (.3) in the Test4 evaluation, apparently return an incorrect value (.29999) when combined in another expression?

    H EXPROPTS(*MAXDIGITS)
    D X               S              5  0 Inz(1)
    D Y               S              5  0 Inz(6)
    D  ABC            S             15A   Inz('13077.39')
    D  NUM            S             15  5 Inz(0000013077.00000)
    D  Digits         S             10    Inz('0123456789')
    D   Test1         S             15  5
    D   Test2         S             15  5
    D   Test3         S             15  5
    D   Test4         S             15  5
    D   Rslt1         S             15  5
    D   Rslt2         S             15  5
    
     /Free
          Test1= (.1**X);
          Test2= (%Scan(%Subst(%Trim(ABC):X+Y:1):Digits)-1);
          Test3= (Test1)*(Test2);
          Test4= (.1**X)*(%Scan(%Subst(%Trim(ABC):X+Y:1):Digits)-1);
          Rslt1= NUM + (Test1)*(Test2);
          Rslt2= NUM + (.1**X)*(%Scan(%Subst(%Trim(ABC):X+Y:1):Digits)-1);
          *InLR= *On;
     /End-Free
    

    Running the program in debug produces the following results:

    TEST1 = 0000000000.10000
    TEST2 = 0000000003.00000
    TEST3 = 0000000000.30000
    TEST4 = 0000000000.30000
    RSLT1 = 0000013077.30000
    RSLT2 = 0000013077.29999
    

    RSLT1 and RSLT2 should yield the same value but don’t.

    The problem does not seem to be precision-related, since the incorrect value requires a precision of 5 decimal places to produce and the correct value is only 1 decimal place. I tried both EXPROPTS(*MAXDIGITS) and EXPROPTS(*RESDECPOS) with no effect.

    Any ideas?

    — Paul

    Welcome to the wonderful world of floating point arithmetic, Paul. Your problem reminds me of some of the war stories I used to hear from old timers when I was just getting into this business. Before COBOL was introduced, businesses were using FORTRAN. Rounding errors were an everyday occurrence.

    Here’s some information I received from Barbara Morris of the RPG compiler development team at IBM‘s labs in Toronto.

    Since 1**X is a floating point expression, the whole expression is done in floating point. When you use ** or any floating point value in an expression, you should consider whether you want the whole expression to be done in floating point. If not, use the %inth or %dech built-in functions to return to decimal arithmetic.

    Also, half adjust (eval(h) or %inth or %dech) should always be used with floating point, unless truncation is really wanted.  Using eval(h) for the expressions in this case gives .30000 all the time.

    — Ted

    Sponsored By
    RJS SOFTWARE SYSTEMS

    Implement Document Imaging on your iSeries-AS/400 in 30 minutes or less

    Image Server/400 is a Web browser-based document image management system for the iSeries.

    Documents can be quickly scanned and stored in IFS folders, and then located and retrieved for viewing via any Web browser. Integrate with other iSeries-AS/400 applications.

    Visit us at COMMON, Booth 418, call us at 888-RJS-SOFT, or download a FREE fully functional demo from our Web site at

    www.rjssoftware.com

    Share this:

    • Reddit
    • Facebook
    • LinkedIn
    • Twitter
    • Email

    Tags: Tags: mgo_rc, Volume 2, Number 82 -- October 25, 2002

    Sponsored by
    WorksRight Software

    Do you need area code information?
    Do you need ZIP Code information?
    Do you need ZIP+4 information?
    Do you need city name information?
    Do you need county information?
    Do you need a nearest dealer locator system?

    We can HELP! We have affordable AS/400 software and data to do all of the above. Whether you need a simple city name retrieval system or a sophisticated CASS postal coding system, we have it for you!

    The ZIP/CITY system is based on 5-digit ZIP Codes. You can retrieve city names, state names, county names, area codes, time zones, latitude, longitude, and more just by knowing the ZIP Code. We supply information on all the latest area code changes. A nearest dealer locator function is also included. ZIP/CITY includes software, data, monthly updates, and unlimited support. The cost is $495 per year.

    PER/ZIP4 is a sophisticated CASS certified postal coding system for assigning ZIP Codes, ZIP+4, carrier route, and delivery point codes. PER/ZIP4 also provides county names and FIPS codes. PER/ZIP4 can be used interactively, in batch, and with callable programs. PER/ZIP4 includes software, data, monthly updates, and unlimited support. The cost is $3,900 for the first year, and $1,950 for renewal.

    Just call us and we’ll arrange for 30 days FREE use of either ZIP/CITY or PER/ZIP4.

    WorksRight Software, Inc.
    Phone: 601-856-8337
    Fax: 601-856-9432
    Email: software@worksright.com
    Website: www.worksright.com

    Share this:

    • Reddit
    • Facebook
    • LinkedIn
    • Twitter
    • Email

    Type2 IFS Directories Reader Feedback and Insights: Thanks for the Information

    Leave a Reply Cancel reply

MGO Volume: 2 Issue: 82

This Issue Sponsored By

    Table of Contents

    • Reader Feedback and Insights: SQL to the Rescue!
    • Another Way to Dump an IFS File
    • Johnny Can’t Read and RPG Can’t Do Math

    Content archive

    • The Four Hundred
    • Four Hundred Stuff
    • Four Hundred Guru

    Recent Posts

    • Liam Allan Shares What’s Coming Next With Code For IBM i
    • From Stable To Scalable: Visual LANSA 16 Powers IBM i Growth – Launching July 8
    • VS Code Will Be The Heart Of The Modern IBM i Platform
    • The AS/400: A 37-Year-Old Dog That Loves To Learn New Tricks
    • IBM i PTF Guide, Volume 27, Number 25
    • Meet The Next Gen Of IBMers Helping To Build IBM i
    • Looks Like IBM Is Building A Linux-Like PASE For IBM i After All
    • Will Independent IBM i Clouds Survive PowerVS?
    • Now, IBM Is Jacking Up Hardware Maintenance Prices
    • IBM i PTF Guide, Volume 27, Number 24

    Subscribe

    To get news from IT Jungle sent to your inbox every week, subscribe to our newsletter.

    Pages

    • About Us
    • Contact
    • Contributors
    • Four Hundred Monitor
    • IBM i PTF Guide
    • Media Kit
    • Subscribe

    Search

    Copyright © 2025 IT Jungle