IBM Brings Active-Active Mirroring Into Db2 For i Database
April 24, 2019 Timothy Prickett Morgan
As a platform that is approaching 40 years of deployment within enterprises that can’t afford downtime with their mission critical systems – that’s counting the System/38 as well as the AS/400 and its follow-ons as part of the same continuum – it is no surprise at all that IBM midrange systems running RPG and COBOL had some of the most sophisticated – and perhaps the only application-centric – clustering software ever developed.
Concurrent with the launch of IBM i 7.4 this week, Big Blue is rolling out a new kind of database clustering, which is called Db2 Mirror, that is intended for customers who cannot have any risk of downtime whatsoever in their transaction processing systems. Db2 Mirror is a supplemental feature to the Db2 for i database and is only available on Power8 and Power9 systems running IBM i 7.4. It is also a bit different from the high availability clustering tools sold by Maxava, Syncsort (which ate Vision Solutions, which acquired rivals iTera in 2006 and Lakeview Technology in 2007), DataMirror (which IBM bought in 2007 and then sold to Rocket Software in 2012), and Sheild Advanced Solutions, and we are frankly trying to figure out just how different.
What we can tell you is that Db2 Mirror creates what is called an active-active database cluster across two machines. Which is different from the active-passive setups that many enterprise-grade high availability clustering techniques employ. In the former case, a cluster of two machines are available to do work, and sometimes this work can be unique and sometimes the machines are configured in such a way that transactions commit across a pair of machines at something very closely approximating the same time.
This Db2 Mirror setup is a bit different from fault tolerant machines from days gone by from Tandem Computers and Stratus Technologies, but is clearly inspired by it, and it is also inspired by database clustering technologies created by IBM, such as Sysplex and Parallel Sysplex for the System z mainframe (which started back in 1990) as well as Oracle’s Real Application Clusters (or RAC), which was derived from the TruCluster software created by Digital Equipment for its Tru64 Unix platform and is itself based on the VAXCluster software for the venerable VMS operating system and the RDB database for the VAX platform. Some inspiration no doubt comes from IBM’s own Db2 PureScale distributed database, which was a distributed, parallel database extension of IBM’s Db2 database for AIX and Linux, which scaled up to 128 server nodes in a cluster, much further than the 32 mainframes in a Parallel Sysplex and the eight nodes in an Oracle RAC cluster. And let’s not forget that when IBM was having trouble scaling up its hardware with NUMA technology in the mid-1990s, it created DB2 Multisystem to created a shared nothing clustering setup similar in some ways (but certainly not all) to Tandem and Stratus and Oracle Parallel Server, the predecessor to Oracle RAC. IBM’s PowerHA does disk-based replication of data through mirrored storage arrays and high speed networks rather than database replication through remote journaling across a pair of servers, which has been part of the plumbing of OS/400, i5/OS, and IBM i since 1999.
There are many ways to skin high availability clustering at the system level and at the database level, and none of us has time to explore them all, no matter how fascinating it is. Suffice it to say, Db2 Mirror is a new way, which Steve Will, IBM i chief architect, tells us has been under development for the past three years and comes because some key IBM i customers have been pushing for active-active database clustering on the platform to allow them to have continuous availability – or as much as you can expect to have with two machines mirrored at the database level within a single datacenter.
“Db2 Mirror is not a journal-based thing,” Will explains. “In Db2 Mirror, when you pair two systems together, every database operation happens to the databases on both systems at exactly the same time. So if you do a table update and your application happens to be pointing at the database, that database operation does not complete that update until it has gotten to both systems. This is not a physical replication solution like with PowerHA. A similar thing would happen with PowerHA, but you would have to put that database into an independent auxiliary storage pool (iASP) that would have to replicate to the other side. But then on the other side, that database wasn’t available for doing active work against it. So there is a difference here with Db2 Mirror. We do not depend on journals. You can have independent ASPs, that’s fine, but we do not depend on independent ASPs. You don’t have to access the database in any particular way, old or new style or whatever.”
Given how much active-active clustering goes on out there in the world – you can get it for Windows Server and Linux systems running most modern relational databases, and there are some truly clever parallel, distributed databases inspired by work done by hyperscalers such as Google, Facebook, Microsoft, and Amazon that are available as open source as well – you might have thought that the Db2 for i database at the heart of the IBM i platform could have active-active clustering long since. But there was one real stickler in the way: the single-level storage architecture of the System/38, the AS/400, and their follow-ons.
With single level storage, the operating system treats the capacity of main memory, disk, and now flash storage as a single continuum of addressable memory space and abstracts the differences away as far as the database is concerned. This architecture is truly genius and unique in the world, and thus far only IBM has been able to make it work, and only in its midrange server line. This puts overhead on the system, but it has radically simplified programming, which is the hallmark of the AS/400 and its progeny. But single level storage gets in the way a bit with active-active mirroring, because it is like asking two different people share the same memories and not going insane.
Unix, Windows Server, and Linux active-active clustering technologies have a slightly easier time, according to Will.
“Part of the reason is the how they cluster and then scale. These technologies actually turn over the file system to the storage devices more than that we are able to. One of the reasons, architecturally, that IBM i has not had active-active before this is that we wanted to be able to preserve the single level store architecture but we couldn’t spread it across multiple systems. We had to find a way to do this, where each system still had its own address space because that single level store is how we do our file system. Nobody else has a file system like that. So they don’t have to worry about that. instead they have hierarchical storage systems, they have to spend time doing locking, and all sorts of other stuff that we don’t. There’s nothing wrong with that – it’s just that’s their architecture. But we needed at the database layer was to coordinate every operation so that every operation could be could be mirrored across the cluster. And the more nodes you had, the less likely it would be that we would be able to meet the performance objectives of many of our large clients.”
And that is why Db2 Mirror is restricted to a pair of systems, at least in its initial release. Which is fine. Customers who need more processing oomph can buy systems with more sockets and shared memory clustering using NUMA techniques at the hardware level. And those who need more than that can use Db2 Multisystem to partition their database tables and run them across a cluster of up 32 machines running in parallel in a shared-nothing setup similar to those Tandem and Stratus machines. Db2 Mirror is not about scale, but continuous availability, but it is fun to contemplate a massive IBM i cluster with up to 32 nodes running Db2 Multisystem to scale up the performance of machines by scaling out the hardware and then using Db2 Mirror to have another set of 32 nodes that makes it continuously available. The mind reels. . . .
Db2 Mirror requires the new IBM i 7.4 release, which we report on elsewhere in this issue, but customers are not required to buy it or install it any more than they have been required to buy Db2 Multisystem for more than a decade. Like Db2 Multisystem, Db2 Mirror is integrated into the platform and the database, but it is a separately licensed product – in this case, costing $20,000 per core. Both IBM i 7.4 and Db2 Mirror will be available on June 21, which is the AS/400’s 31st birthday.
We will be drilling down into the architecture and getting some feedback on Db2 Mirror from the folks selling HA software based on remote journaling techniques. What we can say right now is that Db2 Mirror requires a high speed Ethernet adapter linking the two machines that supports the RDMA over Converged Ethernet (RoCE) protocol, which is a clone of the Remote Direct Memory Access techniques built into InfiniBand switching two decades ago. This allows machines to access each other’s memories remotely, but it does have a distance limit of about 200 meters (we heard 1 kilometer as well, but the manual says differently from what we were told). What this means is that Db2 Mirror might be good for continuous availability within a datacenter, but it doesn’t do jack for disaster recovery across a remote datacenter, be it one running in the cloud by a service provider or one operated by a company itself.