• The Four Hundred
  • Subscribe
  • Media Kit
  • Contributors
  • About Us
  • Contact
Menu
  • The Four Hundred
  • Subscribe
  • Media Kit
  • Contributors
  • About Us
  • Contact
  • Johnny Can’t Read and RPG Can’t Do Math

    October 25, 2002 Timothy Prickett Morgan

    Hey, Ted:

    I ran into an anomaly and was wondering if you could shed some light on it.

    A subprocedure that I wrote doesn’t return the correct value. The problem is caused by an expression that doesn’t return the value that I think it should.

    Why does the expression, which returns the correct value (.3) in the Test4 evaluation, apparently return an incorrect value (.29999) when combined in another expression?

    H EXPROPTS(*MAXDIGITS)
    D X               S              5  0 Inz(1)
    D Y               S              5  0 Inz(6)
    D  ABC            S             15A   Inz('13077.39')
    D  NUM            S             15  5 Inz(0000013077.00000)
    D  Digits         S             10    Inz('0123456789')
    D   Test1         S             15  5
    D   Test2         S             15  5
    D   Test3         S             15  5
    D   Test4         S             15  5
    D   Rslt1         S             15  5
    D   Rslt2         S             15  5
    
     /Free
          Test1= (.1**X);
          Test2= (%Scan(%Subst(%Trim(ABC):X+Y:1):Digits)-1);
          Test3= (Test1)*(Test2);
          Test4= (.1**X)*(%Scan(%Subst(%Trim(ABC):X+Y:1):Digits)-1);
          Rslt1= NUM + (Test1)*(Test2);
          Rslt2= NUM + (.1**X)*(%Scan(%Subst(%Trim(ABC):X+Y:1):Digits)-1);
          *InLR= *On;
     /End-Free
    

    Running the program in debug produces the following results:

    TEST1 = 0000000000.10000
    TEST2 = 0000000003.00000
    TEST3 = 0000000000.30000
    TEST4 = 0000000000.30000
    RSLT1 = 0000013077.30000
    RSLT2 = 0000013077.29999
    

    RSLT1 and RSLT2 should yield the same value but don’t.

    The problem does not seem to be precision-related, since the incorrect value requires a precision of 5 decimal places to produce and the correct value is only 1 decimal place. I tried both EXPROPTS(*MAXDIGITS) and EXPROPTS(*RESDECPOS) with no effect.

    Any ideas?

    — Paul

    Welcome to the wonderful world of floating point arithmetic, Paul. Your problem reminds me of some of the war stories I used to hear from old timers when I was just getting into this business. Before COBOL was introduced, businesses were using FORTRAN. Rounding errors were an everyday occurrence.

    Here’s some information I received from Barbara Morris of the RPG compiler development team at IBM‘s labs in Toronto.

    Since 1**X is a floating point expression, the whole expression is done in floating point. When you use ** or any floating point value in an expression, you should consider whether you want the whole expression to be done in floating point. If not, use the %inth or %dech built-in functions to return to decimal arithmetic.

    Also, half adjust (eval(h) or %inth or %dech) should always be used with floating point, unless truncation is really wanted.  Using eval(h) for the expressions in this case gives .30000 all the time.

    — Ted

    Sponsored By
    RJS SOFTWARE SYSTEMS

    Implement Document Imaging on your iSeries-AS/400 in 30 minutes or less

    Image Server/400 is a Web browser-based document image management system for the iSeries.

    Documents can be quickly scanned and stored in IFS folders, and then located and retrieved for viewing via any Web browser. Integrate with other iSeries-AS/400 applications.

    Visit us at COMMON, Booth 418, call us at 888-RJS-SOFT, or download a FREE fully functional demo from our Web site at

    www.rjssoftware.com

    Share this:

    • Reddit
    • Facebook
    • LinkedIn
    • Twitter
    • Email

    Tags: Tags: mgo_rc, Volume 2, Number 82 -- October 25, 2002

    Sponsored by
    ARCAD Software

    Embrace VS Code for IBM i Development

    The IBM i development landscape is evolving with modern tools that enhance efficiency and collaboration. Ready to make the move to VS Code for IBM i?

    Watch this webinar where we showcase how VS Code can serve as a powerful editor for native IBM i code and explore the essential extensions that make it possible.

    In this session, you’ll discover:

    • How ARCAD’s integration with VS Code provides deep metadata insights, allowing developers to assess the impact of their changes upfront.
    • The role of Git in enabling seamless collaboration between developers using tools like SEU, RDi, and VS Code.
    • Powerful extensions for code quality, security, impact analysis, smart build, and automated RPG conversion to Free Form.
    • How non-IBM i developers can now contribute to IBM i projects without prior knowledge of its specifics, while ensuring full control over their changes.

    The future of IBM i development is here. Let ARCAD be your guide!

    Watch the replay now!

    Share this:

    • Reddit
    • Facebook
    • LinkedIn
    • Twitter
    • Email

    Type2 IFS Directories Reader Feedback and Insights: Thanks for the Information

    Leave a Reply Cancel reply

MGO Volume: 2 Issue: 82

This Issue Sponsored By

    Table of Contents

    • Reader Feedback and Insights: SQL to the Rescue!
    • Another Way to Dump an IFS File
    • Johnny Can’t Read and RPG Can’t Do Math

    Content archive

    • The Four Hundred
    • Four Hundred Stuff
    • Four Hundred Guru

    Recent Posts

    • POWERUp 2025 –Your Source For IBM i 7.6 Information
    • Maxava Consulting Services Does More Than HA/DR Project Management – A Lot More
    • Guru: Creating An SQL Stored Procedure That Returns A Result Set
    • As I See It: At Any Cost
    • IBM i PTF Guide, Volume 27, Number 19
    • IBM Unveils Manzan, A New Open Source Event Monitor For IBM i
    • Say Goodbye To Downtime: Update Your Database Without Taking Your Business Offline
    • i-Rays Brings Observability To IBM i Performance Problems
    • Another Non-TR “Technology Refresh” Happens With IBM i TR6
    • IBM i PTF Guide, Volume 27, Number 18

    Subscribe

    To get news from IT Jungle sent to your inbox every week, subscribe to our newsletter.

    Pages

    • About Us
    • Contact
    • Contributors
    • Four Hundred Monitor
    • IBM i PTF Guide
    • Media Kit
    • Subscribe

    Search

    Copyright © 2025 IT Jungle