The State Of The IBM i Base 2022: Third Party Software Conundrum
April 11, 2022 Timothy Prickett Morgan
Aside from death, most problems are not intractable. But people surely can be, and sometimes are. But luckily not often, and the thing about people is that, generally speaking, they can be reasonable when they are reasoned with. It is with all of this in mind that we come to the next in the State of IBM i Base stories for 2022, where we want to talk about the software trap that the remaining OS/400, i5/OS, and some IBM i shops have gotten themselves into and how we might help them get out of it to the mutual benefit of all.
As best we can figure, based on the data from the annual HelpSystems survey of the IBM i base, about two-third of the companies that take the survey have consistently said that they have homegrown applications running on their systems – something that was not asked in the original surveys from years gone by and a thing that I pointed out to HelpSystems and had the question changed because I simply did not believe that most of the base had third-party applications. Over the decades, the readers of The Four Hundred have consistently been do-it-yourself application shops and I just simply did not believe that somewhere along the way that changed so dramatically.
In theory, that means that two-thirds of the base is not facing what I will call the third-party software conundrum, where they are stuck on old releases without maintenance and no easy or cost-effective way to get those applications current. In practice, many IBM i shops are facing massive technical debt in their code, a lack of people with skills and insufficient funding to update their older RPG III, RPG IV, and ILE RPG applications to free form RPG, or worse yet, have lost the source code for their applications. And as a consequence they are just as stuck as anyone using an old suite of applications from a vendor that ended up inside of Infor or Oracle or one of the few remaining mid-sized ERP vendors catering to the IBM i crowd.
Part of the problem with regard to third-party application software, I think, is the fact that there is a long history of open source application code in the IBM midrange, and another part of the problem is the long practice of selling software with a perpetual use license that also has an annual software maintenance fee.
The fact that many of the thousands of application suites available for System/3X and AS/400 systems were available as source code meant that companies buying the software could indulge in customizing software at a level that we have generally not seen in the application space heretofore. There are plenty of IBM midrange shops that used a mix of custom code and heavily customized third-party code to create the systems that run their businesses, and at some point, the code has changed so much that there is no point in paying third-party maintenance on it. Companies could not upgrade to new application versions and suites form the vendor if they wanted because all of those customizations would have to be done again. So it is not just a matter of people not wanting to pay maintenance on application software – it would not get them anything if they had.
There are, of course, IBM i shops that have done a modest amount of customization on third-party code and when the budget gets tight, they stop paying for maintenance on it because they are not changing it, even if they do have the source code. And these days, with modern ERP, CRM, and SCM suites, they probably are not getting the source code for the new software unless it is grandfathered into their vendor contracts.
But even absent that, the way these licenses are sold were always a budgetary headache, and the problem is that people costs rise with gross domestic product and do not have Moore’s Law economic scaling, where things get cheaper per unit of capacity with each passing year, as happens with most elements of the system. This is why software maintenance really exists, and it is why it is set at 15 percent to 25 percent of the list price of the application software. That means every four to seven years, the maintenance fees are like buying the software all over again from an economic standpoint. And if the code is not changing – and because customers don’t want it to –and all the vendor is really doing is supplying security patches, then you can understand why IBM i customers might resent these fees.
Yes, it is unfair that some customers stayed on maintenance and paid and others did not, and that some IBM i shops expect a break on after license maintenance fees if they return to the fold and upgrade to software that is certified for modern IBM i operating systems and modern Power Systems hardware. But as I explained to a reader on LinkedIn last week in response to the software release problem with IBM i, where so many customers are on 7.1 or earlier releases, we can all dig our heels in and go straight to hell together, or we can figure out some way that everyone gives a little and we all benefit.
The application ISVs can dig their heels in and say they are entitled to all the back maintenance before getting customers current, and they will probably not get very far. If customers are going to spend huge amounts of money and have to massively customize a newer version of the code anyway, they will very likely just move to a different platform for political reasons more than technical ones. The economics will suck no matter what.
The customers who just sit there on older releases of operating systems and applications are sitting on a ticking timebomb, but sometimes this is in fact the least risky behavior as well as the least costly – right up to the catastrophe where all of this comes home to roost.
IBM has a hand in this, too, and has shown the way with amnesties on after license charges for IBM i Software Maintenance in 2015 and again in 2020, which you can see in the Related Stories section below.
All I know is that IBM i shops, application ISVs, and Big Blue have to all work together to solve this problem for those companies who rely on third party applications, and the fees that IBM i shops have to pay should be proportional to the amount of work it takes to get either old suites certified on new IBM hardware and operating systems or to get customizations ported to new suites that are already ported to them. (I think we all know which one is easier and cheaper.)
There is one more thing that we know. Real mitigation for Log4j security vulnerabilities has to be done, and that means IBM has to write new and secure logging software that snaps in place of Log4j and allows IBM i 7.1 releases and forward, including the Heritage Navigator as well as the new IBM i Navigator to work. Telling people with older releases to turn off Log4j and only turn it on to use Heritage Navigator at their own risk is not doing right by the customer, and IBM damned well knows it. When nearly half of the base is on IBM i 7.1, as I believe it is, and another fifth is on IBM i 6.1, and many of them are stuck, Big Blue simply cannot behave this way.