Some Good Advice About Log4j Mitigation Gotchas
January 24, 2022 Timothy Prickett Morgan
The Apache Log4j logging utility written in Java and available since the end of the Dot Com Boom in early 2001, has been installed far and wide into many systems and systems software packages in the more than two decades it has been available. And that is why the zero-day security vulnerability discovered by Chinese computing giant Alibaba on November 24 last year and revealed on December 9 has caused so much concern.
Log4j is everywhere and that means the Log4Shell vulnerability that Alibaba described makes it particularly scary. But before we get into some of the mitigation advice that we have obtained courtesy of Doug Bidwell, systems engineering manager at DLB Associates and also the author of the IBM i PTF Guide, we have a few observations.
There is a spreadsheet that you can download as part of this story, which is available here.
First, the hyperscalers and cloud builders are looking for problems to better lock down their systems, but they cause as much trouble as they attempt to cure to one way of thinking. Take the side-channel speculative execution exploit that Google discovered a few years back. Google has some of the smartest techies on the planet, as does Alibaba, and of course they are able to find exploitable security vulnerabilities. I am all for openness, but look at the grief this caused. And speculative execution was buried into processors for decades and there was no easy way to get around these vulnerabilities. Log4j is similar in its scope. It is ironic that the world might be just as secure if Google and Alibaba just shut up as it is if they tell us. Thus far, there have not been, as far as we know, any exploits in the world from hackers that can use these side channel or Log4j vulnerabilities to hack into a system. Thank heavens.
What is obvious is that we owe the security officers and techies who keep our systems secure some gratitude. It is a never-ending slog.
Big Blue is not providing any patches for IBM i 7.1 and 7.2, even though both are on extended support, because security patches are done at IBM’s discretion on older releases that are not on normal support. You would think IBM would make an exception here and provide Log4j patches on IBM i 7.1 and 7.2 as it is doing for IBM i 7.3 and 7.4, particularly with so many customers stuck on older releases for a variety of reasons that have to do with their application incompatibility on newer iron. It sends a mixed message to put IBM i 7.1 on extended support and offer it on Power9 iron even but then not supply the Log4j patches. We suspect the reasons are more economical – meaning revenue driving and cost cutting for IBM – rather than technical.
Even if you do upgrade your IBM i operating system, you are going to have trouble and grief. So get used to this idea. Here are two points that Bidwell brought up when we talked to him about this:
- After going through the Log4j mitigation of putting on the Group PTFs, then modifying the script, the next Group PTF you put on the HTTP server will set everything back. So remember that you have to reapply the scripts mods.
- If the decision is made to remove heritage Navigator for i from the client PCs, and replace it with the new Navigator for i, you need to consider what kind of training will be required to educate the end users about how to use NewNav to do the same functionality they had with the heritage Navigator.
To help you organize your battle plan to do the mitigations on, Bidwell and his team has put together a spreadsheet that helps with the mitigations on IBM i 7.3 and 7.4, the integrated Web servers, Access Client Solutions, and Navigator for i heritage edition. There are three tabs in the sheet: one that discusses the various mitigations, one that has all the links to the Log4j security vulnerability and support pages relating to it from IBM, and another quick reference that shows the software and their release levels that are affected by the Log4j vulnerability.
Reach out to us if you need more help, and thanks to Brad Ford of the DLB Collective for testing all of the patches and seeing these issues pop up and warning us all.