7.1 Flew Over The Cuckoo’s Nest
April 4, 2022 Timothy Prickett Morgan
We think there is a lot of Power7, Power7+, and Power8 iron out there in the Power Systems running IBM i base, and we think there is a lot of IBM i 6.1 and IBM i 7.1 running on that iron. Our assertion is based on years of anecdotal evidence from the resellers and business partners we talk to, the customers we talk to, and a whole lot of spreadsheet witchcraft that we do based on survey data we see.
The point is not just to come up with this data and then drop it and run, but to face it honestly and then do something about it. So here is the first suggestion that we are going to make: IBM should support nested virtualization on Power9 and Power10 so very old i5/OS and IBM i releases can be supported on new iron. Particularly IBM i 7.1.
In recent weeks, we have done our best to case the IBM i, i5/OS, and OS/400 installed base over time as well as the distribution of Power-based servers running these platforms based on the family of Power chips used in them. The base is a lot older than it looks, which we explained in detail and which we are not going to go through again here. We will remind you of the statistics we came up with. Based on some serious messaging of the data coming out of the annual IBM i Marketplace Survey done by HelpSystems, we reckon that among the primary operating systems installed at the 120,000 IBM i sites in the world, nearly half (49.1 percent) of them are running IBM i 7.1. And another 18 percent are running IBM i 6.1 as their primary operating system. The 6.1 release was at end of support as of September 30, 2015 – six and a half years ago – and the 7.1 release reached end of support on April 30, 2018 but has had its extended support pushed out until at least 2024.
Now why would IBM do that? Because so many customers are using it, we think. And to put that into perspective, IBM i 7.1 is a little bit younger than Windows Server 2008, which most definitely has not had regular support from Microsoft for years and which had its extended support removed on January 14, 2020.
As for hardware, we think that the Power7/7+ base of machines peaked in 2019 at just over 40 percent of the installed base (not primary machines, but the entire base) and has fallen to under 35 percent in the past year. The Power8 base has grown steadily and is now also nearly 35 percent of the installed base of iron. Power6/Power6+ and Power5/Power5+ machines each now represent about 10 percent of the base after steady declines that match the rise of the Power7 and Power8 pools, and the Power9 base has a less aggressive ramp than the Power8 and Power7/Power7+ part of the base and has only attained a 10 percent share of machines.
That brings us back to the idea of nested virtualization, which we actually discussed a bit with Ken King, general manager of the Power Systems business at IBM, back in February. It is not just an idea, it actually works, with some limitations that I cannot speak to and without official support from IBM, of course. But both of those conditions can change.
We know of a reseller who has an IBM i 7.1 customer who wanted to get on more modern iron, and just for fun, the reseller and the customer did an experiment. They took a machine running IBM i 7.3 and then did what is called “i hosting i,” which is a capability that came with IBM i 7.1 and which is explained pretty well here. Way back in the dawn of time, with V4R4, OS/400 itself was a hypervisor and it could host partitions itself – this was before the PowerVM hypervisor that supported OS/400, AIX, and Linux was even created. This capability is still buried in the microcode and accessible in the operating system. This is what is called a Type 2 hypervisor, with a primary partition and secondary guess partitions. If the primary goes down, they all go down, which is why companies created free-standing hypervisors. (Which are know as Type 1 hypervisors and which can also go down because they are just operating systems that have only the job of virtualizing compute, memory, storage, and networking.)
The point is, you can do an “i hosting i” partition without using the PowerVM server virtualization hypervisor or the Virtual I/O Server for virtualizing disk and networking.
And that is precisely what this reseller and customer did. They fired up IBM i 7.2 inside of an “i hosting i” partition, and then created an independent auxiliary storage pool (iASP) for IBM i 7.1 and did another “i hosting i” partition running IBM i 7.1. The IBM i 7.1 objects – the programs and databases – worked without any recompilation or anything.
So, in theory, this can work, and it doesn’t even require PowerVM, PowerHA, or VIOS. All of which for many IBM i customers would be a big plus.
We talked to the reseller, and they said there is no conceptual reason why they could not have started with IBM i 7.4 and went all the way back to IBM i 6.1, i5/OS 5.4, or even OS/400 V5R3. That would be seven layers of virtualization, and it could be quite a bit of overhead. The first you go back in time, the more overhead is what our guess was and what the resellers was as well, but we would have to really test this to be sure. At that many layers of virtualization, it would not be surprising to see 40 percent and maybe even 50 percent of the aggregate CPWs in a system go up the chimney running the virtualization layers and dealing with latency going through so many handoffs between layers of software.
It would be better if IBM i 7.4 could directly support “i hosting i” partitions all the way back to OS/400 V5R3, and something akin to the System/36 Emulation Environment (some documentation called it the Execution Environment) that has been embedded in all versions of the OS/400, i5/OS, and IBM i platform. There was a System/38 library for the AS/400, but not a full-blown emulation environment as such. The effect was the same – customers with RPG II and RPG III programs could run them in a kind of emulated mode, with a performance penalty of course compared to moving to RPG-IV and compiling it to run native on the AS/400.
The per-core performance of the Power10 chip in the “Denali” Power E1080 system is around 22,000 CPWs – a 240-core machine is rated at 5.27 million CPWs, and the cores run at between 3.8 GHz and 4 GHz. The low-end Power7 processor with four cores running at 3 GHz in the entry Power 710 and Power 720 servers from 2010 is rated at 23,800, which is just under 6,000 CPWs per core. That is another way of saying that a Power10 core is 3.7X more powerful than a Power7 core from a dozen years ago. A Power8 chip running at 3.4 GHz with four cores has a performance of 42,470 CPWs, or 10,618 CPWs per core. So even with a 50 percent overhead, a Power10 entry or midrange machine running dense, nested emulation might feel like a Power8 system to customers – but for those coming from Power6, Power6+, Power7, and Power7+ iron and stuck on IBM i 6.1 or IBM i 7.1, this would be a serious upgrade and would not necessitate recertifying their applications or changing anything else about them.
At least, that is the idea we keep coming back to. And if IBM does this, maybe it can sell a lot more Power10 machines than it otherwise might. And more customers than otherwise might be then able to start modernizing their applications on the latest-greatest iron and take advantage of all of the AI inference, encryption, and other kinds of acceleration that is embodied in the Power10 chip.
RELATED STORIES
The State Of The IBM Base 2022, Part Three: The Rusting Iron
The State Of The IBM i Base 2022, Part One: The Operating System
IBM i 7.1 Extended Out To 2024 And Up To The IBM Cloud
Big Blue Revives IBM i 7.1 With Power9 Support
IBM Further Extends Service Extension For IBM i 7.1
Service Extension Outlined For IBM i 7.1 And PowerHA 7.1
If they are running 7.1 they are not likely to do any form of modernization. I don’t see how you came up with the last sentence of your article.
Often the reason they are staying on older hardware and older releases is that they dropped support with their software vendor in a short sighted attempt to save money on maintenance costs. Now their vendor is charging astronomical fees to get back in support and this “customer” is upset because they felt all the other customers who kept up support should have been subsidizing them all these years. I put “customer” in quotes because I feel a customer is someone who exchanges money in return for goods or services and I suspect they haven’t wrote that vendor a check in years.
These companies need to get up to date on their vendor software, or replace them. They need to get up to a current version of the operating system which supports current ciphers and security requirements.
I fully understand the problem. And everyone can dig in their position and we can all go to hell together. Or, we can all make some concessions and move forward together. It is really that simple.
I wish it was Tim. But from my time working on the 6.1 release project I have to believe that going back earlier than 6.1 is problematic from a technical perspective. You’d need someone from Rochester to really discuss it but I think the notion that it is “simple” is a gross over simplification.
And given that IBM would have to do work to test (even if no code changes were needed) and subsequently support this approach – where does the revenue to support that come from?
Very innovative research and reporting, TPM.
Thanks to Timothy Pricket Morgan for another dazzling and perceptive and hugely important article.
in recent previous articles in IT Jungle TPM stated that IBM i customers and IBM itself are spending 40 billion dollars annually on people (384,000 people times $100,000 annual pay, at least, and they have been doing so for decades.
That 40-billion-dollar annual IBM i personnel cost is matched by the IBM System z (mainframe) annual personnel cost.
That 80 billion annual personnel cost by IBM customers dwarfs all of IBM’s revenue and profit from all of IBMs revenue sources.
Perhaps 50 % or much more of that 80-billion-dollar annual personnel cost of IBM programmers is currently totally wasted.
The real task is to modernize the people costing the 80 billion dollars annually, and that applies equally to all IBM customers regardless of the IBM operating system or release level the IBM customer uses.
IBM programmers (software developers) on all IBM platforms are using primitive and ancient and obsolete technology and TPM is certainly correct in stating that all should be able to fully utilize the most current IBM hardware technology.