Unisys First Up with TPC-E Benchmark Test

Published: August 8, 2007

by Timothy Prickett Morgan

After several years of development, the Transaction Processing Council, an industry consortium that creates and audits commercial transaction processing and data warehousing benchmarks, in March rolled out its long-awaited TPC-E benchmark test. TPC-E was designed to fix many of the criticisms of the popular TPC-C test, which has been used since 1992 to gauge the relative performance of servers on online transaction processing workloads.

Server maker Unisys, hoping to use the TPC-E test to its advantage to help it peddle its ES7000 Windows and Linux servers, was the first vendor to provide results for the TPC-E test.

The TPC-E benchmark simulates the data processing associated with running a stock brokerage house. It is based on a subset of real market trading data and provides all testers with a specific set of C++ code that implements the simulated brokerage. The TPC-C test, which was first used by IBM on its AS/400 proprietary minicomputers back in 1992 to test their mettle, simulates the data processing associated with a real warehouse--one with forklifts and boxes of stuff that need to be moved around. Back when TPC-C debuted, getting a box that could handle a few transactions per minute (TPM) was a big deal; today, big Unix boxes from IBM and Hewlett-Packard can push over 4 million TPM.

With the TPC-C test, vendors were provided specifications of the benchmark and wrote their own code; the data in the TPC-C test was also randomized, not real-world data, which in some ways made it easier for vendors to partition their databases and thereby improve performance. The TPC-C had ridiculously high storage requirements per user, which seemed reasonable back in 1992 but which raised system costs astronomically as the performance of systems grew very rapidly through the 1990s and early 2000s.

With TPC-E, customer accounts are based on a subset of census data from the United States, and stock values involved in trades are a subset of stock data from the New York Stock Exchange and Nasdaq stock markets. This makes it harder for vendors to game the benchmark, since this data is not random in nature. Whatever methods vendors come up with partitioning or caching, data will have to resemble what real companies have to do with real data. The TPC-E test also requires that the application check for constraints and provide referential integrity for transactions--something that businesses have done for years and that the TPC-C test did not require. RAID 1 (mirroring) or RAID 5 (parity with striping) are required on disk arrays used in the servers tested, too. The TPC-E test has 10 different transactions--looking up historical stock data, a stock feed, getting a customer position in a stock, and so forth--and counts the transaction called a trade result. This is the confirmation transaction that is received from the stock market that a trade was executed. To get smaller metric numbers--and because this is a denser benchmark--the TPC-E test is measured in transactions per second, or TPS.

Unisys ran the TPC-E test on its latest ES7000/one Enterprise 16s server. This machine was equipped with Intel's dual-core "Tulsa" Xeon 7100 processors (formerly known as the Xeon MP-class of chips because they are used in multiprocessor systems with more than two CPU sockets). Unisys plunked down 16 of these dual-core Xeon 7100 chips, running at 3.4 GHz and with 16 MB of L3 cache per chip, into the ES7000/one server. It then added 128 GB of main memory to this machine. External disk arrays were configured with RAID 10 data protection (which is cross-coupled, mirrored RAID 5 arrays) with a total capacity of 33.7 TB; remember, this is mirrored capacity and RAID 5, which means that less than half of that capacity is actually available for the workload. But, RAID 10 offers multiple paths to data, which enhances performance, which is why Unisys did it. Because the Tulsa chips have HyperThreading, the machine presented a total of 64 threads to the operating system and its applications.

To run its TPC-E test, Unisys went with the Windows stack from Microsoft, including the 64-bit versions of Windows Server 2003 R2 Datacenter Edition and SQL Server 2005 Enterprise Edition.

In terms of pricing, the ES7000/one server costs $230,876, plus another $13,680 for three years of maintenance. Windows Datacenter Edition costs $41,458, including an additional two years of maintenance (the first year comes free). SQL Server 2005 costs $374,912 on this machine, dwarfing the cost of the operating system and the base server together. As was the case with the TPC-C test, storage continues to be the biggest component of the system, and because Unisys chose RAID 10 data protection, the butcher's bill for disk arrays came to $632,458. With switching and client hardware for simulating TPC-E users thrown into the mix, the total price tag of the ES7000/one setup came to $1.17 million after a 14 percent discount from a Unisys reseller. The ES7000/one machine was able to process 660.85 TPS, and that works out to $1.777 per TPS.

Now, the rest of the server vendors have to get their acts together and get TPC-E results out, so we can make some comparisons.

RELATED STORIES

Transaction Processing Council Launches TPC-E Benchmark

New TPC Benchmarks Are on the Horizon

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot