• The Four Hundred
  • Subscribe
  • Media Kit
  • Contributors
  • About Us
  • Contact
Menu
  • The Four Hundred
  • Subscribe
  • Media Kit
  • Contributors
  • About Us
  • Contact
  • Where Is DB2 BLU Accelerator For IBM i?

    April 15, 2013 Timothy Prickett Morgan

    IBM has created a neat new database feature for its DB2 database for Linux, Unix, and Windows operating systems that will hopefully make its way into the integrated DB2 for i database that resides inside the IBM i operating system. For now, this BLU Accelerator feature, which can radically speed up the sifting through data, is only available for DB2 10.5 and only for reporting and analytics, but there is every reason to believe Big Blue will put it on the IBM i and mainframe versions of its DB2 database and use it to help goose transaction processing.

    Like other IT vendors, IBM wants companies to think that every bit of data that they generate or collect from their systems or buy from third parties in the course of running their business is valuable, and the reason is simple. This sells storage arrays, and if you can make CEOs think this data is potentially valuable, then they will fork out the money to keep it inside of various kinds of data warehouses or Hadoop clusters for data at rest or in InfoSphere Streams systems for data and telemetry in motion. There is big money in them there big data hills, and with server virtualization pulling the rug out from underneath the server business in the past decade, hindering revenue growth, the funny thing about these big data jobs is that none of them are virtualized and based on the massive amounts of data they need to absorb every day, they keep swelling like a batch of yeast.

    IBM is not making any promises about bringing BLE Accelerator, which can goose analytics queries by between a factor of eight and 25 times while at the same time reducing storage capacity needs for data sets thanks to columnar data compression, to other databases, but Tim Vincent, who is chief architect for DB2 on the Linux, Unix, and Windows platforms, who is an IBM Fellow, and who is chief technology officer for IBM’s Information Management division, hinted pretty strongly. “We do plan on extending this,” Vincent said at the BLU Accelerator launch in early April, “and we are going to bring the technology into new products going forward.”

    So what exactly is BLU Accelerator? Well, it is a lot of things. First, BLU implements a new runtime that is embedded inside of the DB2 database and a new table type that is used by that runtime. These BLU tables coexist with the traditional row tables in DB2, and have the same schema and use storage and memory the same way. The BLU tables orient data in columns instead of the classic row structured table used in relational databases, and this data is encoded in such a manner (using what Vincent called an approximate Huffman encoding algorithm) that has an extra feature whereby the data is kept in order so it can be searched even while it is compressed. The BLU Accelerator has a memory paging architecture so that an entire database table does not have to reside in main memory to be processed, but the goal is to use the columnar format to allow the database to be compressed enough so it can reside in main memory and be much more quickly searched. But again, it is not required, like some in-memory database management systems, and you can move chunks of a BLU database into main memory as you need to query it. The BLU Accelerator knows about multiple core processors and SIMD engines and vector coprocessors on chips, and it can take advantage of these units to compress and search data. The Actionable Compression algorithm, as IBM calls it, is patented and allows for data to be used without decompressing it, which is a neat trick. The accelerator feature also can do something called data skipping, which means it can avoid processing irrelevant data in a table to do a query.

    Here’s the compare and contrast between the way DB2 works now, with all of the snazzy features to improve its performance that have been added over the years, and the way the BLU Accelerator feature works:

    OK, I am not a database expert or a comedian, but that is funny. The freaky thing about BLU Accelerator is that it does have database indexes. You don’t have to do aggregates on the tables, you don’t have to tune your queries or the database, and you don’t have to make any changes to SQL or database schemes. “You just load the data and query it,” as Vincent said at the launch of the product.

    The reason that you don’t need a database index is that data is compressed so a BLU table can, generally speaking, reside in memory. Vincent said that 80 percent of the data warehouses in the world had 10 TB of capacity, so if you can use the Actionable Compression and get a 10X compression ratio, then you can fit the typical data warehouse in a 1 TB memory footprint. But there are more tricks that speed up those database queries, as you can see here:

    Once you have compressed the data so it all fits into main memory, you take advantage of the fact that you have organized the data in columnar format instead of row format. So, in this case, you put each of 10 years of data into 10 different columns each, for a total of 100 columns. And when you want to search in 2010 only for a set of the data, as the query above–find the number of sale deals that the company did in 2010–does, then you reduce that query down to 10 GB of the data in the entire set. The data skipping feature in this case knows to look for sales data, not other kinds of data, so that reduces the data set down to around 1 GB. The machine you are using to run this BLU Accelerator feature not only has 1 TB of main memory but 32 cores, so you parallelize the query and break it up so 32 MB chunks of the data are partitioned and parceled out to each of the 32 cores and their memory segments. Now, use the vector processing capability in an X86 or Power processor, and you get around a factor of four speedup in scanning the data for the sales data. And the result is that you can query a 10 TB table in a second or less.

    Sounds pretty useful, right? So when do the other DB2s get it? We’ll try to find out.

    RELATED STORIES

    TR6 Brings Assorted Tech Goodies To IBM i

    Applications Misfire When Database Integrity Ignored

    DB2 For i Modernization Gets Assist From RPG OA

    DB2 For i? This Is SQL Server Calling

    Business Strategy Bumps Into Database Deficiency

    DB2 for i: The Beating Heart of the IBM i Platform

    Get Database Skills for Career ROI

    DB2 on i: The Time, Money, and Risk of Modernization

    So Where Is PureXML for DB2/400?



                         Post this story to del.icio.us
                   Post this story to Digg
        Post this story to Slashdot

    Share this:

    • Reddit
    • Facebook
    • LinkedIn
    • Twitter
    • Email

    Tags:

    Sponsored by
    LaserVault

    Integrate Virtual Tape to Automate Your Backups And Strengthen Your Ability To Recover From Cyber Attacks And Disasters

    With most IT departments stretched thin, finding something that can quickly free up IT time is definitely a bonus. That’s why it’s important to stop and take a look at integrating virtual tape into your backup and recovery. Virtual tape is one of those technologies where once you have it, you’ll wonder why you didn’t do it sooner. See a demo and get a $50 gift card.

    But what is it about using virtual tape that makes it so worthwhile? Why is it that so many IBM i shops are already using or considering using virtual tape for all or part of their backup and recovery systems?

    Virtual tape and virtual tape libraries offer a way to both simplify and strengthen backup and recovery operations. By incorporating virtual tape technology, automation of backups becomes possible resulting in hundreds of hours saved annually for IT departments and personnel.

    “We needed to find a replacement that would lower the maintenance cost and reduce complexity of our backup and recovery functions without a major disruption to our operations.” David Fray, Director of Enterprise Systems, ABC Financial

    LaserVault ViTL is a virtual tape and tape library solution developed specifically for use with IBM Power Systems (from AS/400 to iSeries to Power 9s). With ViTL you can:

    • Replace physical tape and tape libraries and eliminate associated delays
    • Automate backup operations, including the ability to purge or archive backups
    • Remotely manage your backups – no need to be onsite with your server
    • Save backups to a dedupe appliance and the cloud
    • Recover your data at lightspeed greatly improving your ability to recover from cyberattacks
    • And so much more

    Sign-up now to see a ViTL online demo and get a $50 Amazon e-gift card when the demo is complete as our way of saying thanks for your time. Plus when you sign-up you’ll receive a free facts comparison sheet on using virtual tape vs tape so you can compare the functionality for yourself.

    Share this:

    • Reddit
    • Facebook
    • LinkedIn
    • Twitter
    • Email

    AURA Equipements Touts its SQLiPlug Zend Server 6 for IBM i Goes GA

    Leave a Reply Cancel reply

Volume 23, Number 15 -- April 15, 2013
THIS ISSUE SPONSORED BY:

ProData Computer Services
CCSS
Abacus Solutions
HiT Software
WorksRight Software

Table of Contents

  • i Witness Account: The Chief Architect’s View
  • COMMON Fights Off The Blues In Austin
  • Where Is DB2 BLU Accelerator For IBM i?
  • Mad Dog 21/21: Smith And Westin
  • IBM To Pump $1 Billion Into Flash Storage
  • Power Systems Marketing VP Sees Big Data Bulls Eye
  • CIOs Move With Caution On New IT Hiring
  • IBM Revises Another Power Systems Trade-In Deal
  • IBM Can’t Be Serious About Selling Used IBM i Systems
  • More Details On That PS702 Blade Deal For MSPs

Content archive

  • The Four Hundred
  • Four Hundred Stuff
  • Four Hundred Guru

Recent Posts

  • IBM Mulls Using DataMigrator as Cloud Warehouse Pipeline
  • PowerTech AV Automatically Detects Ransomware Activity
  • Infor Puts CM3 Project On Hold
  • Four Hundred Monitor, June 29
  • IBM i PTF Guide, Volume 24, Number 26
  • Guild Mortgage Takes The 20-Year Option For Modernization
  • IBM i Licensing, Part 3: Can The Hardware Bundle Be Cheaper Than A Smartphone?
  • Guru: The Finer Points of Exit Points
  • Big Blue Tweaks IBM i Pricing Ahead Of Subscription Model
  • We Still Want IBM i On The Impending Power E1050

Subscribe

To get news from IT Jungle sent to your inbox every week, subscribe to our newsletter.

Pages

  • About Us
  • Contact
  • Contributors
  • Four Hundred Monitor
  • IBM i PTF Guide
  • Media Kit
  • Subscribe

Search

Copyright © 2022 IT Jungle

loading Cancel
Post was not sent - check your email addresses!
Email check failed, please try again
Sorry, your blog cannot share posts by email.