One Way To Condense A Verbose Spooled File

November 29, 2016 Ted Holt

IBM i includes many reports that we can use to glean a lot of information. If you run packaged software, the vendor likely includes many other reports. The problem with such reports is that they often contain more information than what you need and want. After all, they’re written for the world at large, not tailored for you. It may be that a utility that resides on your system can transform reports into something more suitable for you and those who use your system.

First, let’s get a report.

DSPLNK OUTPUT(*PRINT) DETAIL(*EXTENDED) DSPOPT(*USER)

Running this command on one of the systems I use produced a 52-page report.

                       Display Attributes                           Page     2
5770SS1 V7R1M0  100423                        SYSTEMX   11/30/16  12:00:00 EST
 Object . . . . . . :   /home/tholt/archive.tar
 Type . . . . . . . . . . . . . . . . . :   STMF
 File ID  . . . . . . . . . . . . . . . :   X'00000000000000019748873800074532'
 Owner  . . . . . . . . . . . . . . . . :   THOLT
 System object is on  . . . . . . . . . :   Local
 Auxiliary storage pool . . . . . . . . :   1
   Object overflowed  . . . . . . . . . :   No
 Coded character set ID . . . . . . . . :   819
 Hidden file  . . . . . . . . . . . . . :   No
 PC system file . . . . . . . . . . . . :   No
 Read only  . . . . . . . . . . . . . . :   No
 Need to archive (PC) . . . . . . . . . :   Yes
 Need to archive (System) . . . . . . . :   Yes
 Creation date/time . . . . . . . . . . :   11/01/15  18:39:29
 Last access date/time  . . . . . . . . :   11/01/15  18:40:01
 Data change date/time  . . . . . . . . :   11/01/15  18:40:01
 Attribute change date/time . . . . . . :   11/01/15  18:40:01
 Size of object in bytes  . . . . . . . :   40960
 Allocated size of object . . . . . . . :   65536
 File format  . . . . . . . . . . . . . :   *TYPE2
 Size of extended attributes  . . . . . :   0
 Storage freed  . . . . . . . . . . . . :   No
 Temporary object . . . . . . . . . . . :   No
 Disk storage option  . . . . . . . . . :   *NORMAL
 Main storage option  . . . . . . . . . :   *NORMAL
 Auditing value . . . . . . . . . . . . :   *NONE
 Object domain  . . . . . . . . . . . . :   *SYSTEM
 Number of hard links . . . . . . . . . :   1
 Set effective user ID  . . . . . . . . :   No
 Set effective group ID . . . . . . . . :   No
 Restricted rename and unlink . . . . . :   No
 Last used date . . . . . . . . . . . . :   11/01/15
 Days used count  . . . . . . . . . . . :   1
   Reset date . . . . . . . . . . . . . :
 Allow write during save  . . . . . . . :   No
 Can be saved . . . . . . . . . . . . . :   Yes
 Digitally signed . . . . . . . . . . . :   No
 Object is currently journaled  . . . . :   No
 Object scanning  . . . . . . . . . . . :   *YES
   Scan status  . . . . . . . . . . . . :   *REQUIRED
                       Display Attributes                           Page     3
5770SS1 V7R1M0  100423                        IUSR011   11/28/16  01:00:49 EST
 System use . . . . . . . . . . . . . . :   *NONE
                       Display Hard Links                           Page     4
5770SS1 V7R1M0  100423                        IUSR011   11/28/16  01:00:49 EST

If I only want the object names, complete with path, I use this command within Qshell to reduce the report:

system 'DSPLNK OUTPUT(*PRINT) DETAIL(*EXTENDED) DSPOPT(*USER)' | 
sed -n '/Object . ./p' | 
rfile -wQ qsysprt

The system utility executes the DSPLNK command and sends the report to the standard output file, which is piped into sed, the stream editor. Sed selects the lines that contain the string “Object . .” and writes them to standard out, which is piped into the Rfile utility to produce a spooled file.

The -n switch tells sed not to write to standard output automatically. The p subcommand at the end of the pattern tells sed to print lines that match the pattern.

My report has shrunk to a single page!

  Object . . . . . . . :   *
  Object . . . . . . :   /home/tholt/archive.tar
  Object . . . . . . :   /home/tholt/archive.tar
  Object . . . . . . :   /home/tholt/custcdt1.txt
  Object . . . . . . :   /home/tholt/custcdt1.txt
  Object . . . . . . :   /home/tholt/custcdt2.txt
  Object . . . . . . :   /home/tholt/custcdt2.txt
  Object . . . . . . :   /home/tholt/custcdt3.txt
  Object . . . . . . :   /home/tholt/custcdt3.txt
  Object . . . . . . :   /home/tholt/customer-data.csv
  Object . . . . . . :   /home/tholt/customer-data.csv
  Object . . . . . . :   /home/tholt/customer-data.txt
  Object . . . . . . :   /home/tholt/customer-data.txt
  Object . . . . . . :   /home/tholt/data.csv
  Object . . . . . . :   /home/tholt/data.csv
  Object . . . . . . :   /home/tholt/data-script.txt
  Object . . . . . . :   /home/tholt/data-script.txt
  Object . . . . . . :   /home/tholt/data2.csv
  Object . . . . . . :   /home/tholt/data2.csv
  Object . . . . . . :   /home/tholt/pay-work.csv
  Object . . . . . . :   /home/tholt/pay-work.csv
  Object . . . . . . :   /home/tholt/pay-work-script.txt
  Object . . . . . . :   /home/tholt/pay-work-script.txt
  Object . . . . . . :   /home/tholt/src.tar
  Object . . . . . . :   /home/tholt/src.tar
  Object . . . . . . :   /home/tholt/yycomp
  Object . . . . . . :   /home/tholt/yycomp
  Object . . . . . . :   /home/tholt/AARDVARK.CSV
  Object . . . . . . :   /home/tholt/AARDVARK.CSV
  Object . . . . . . :   /home/tholt/AARDVARKTX.CSV
  Object . . . . . . :   /home/tholt/AARDVARKTX.CSV
  Object . . . . . . :   /home/tholt/CustCdt.coldef
  Object . . . . . . :   /home/tholt/CustCdt.coldef
  Object . . . . . . :   /home/tholt/CustCdt.csv
  Object . . . . . . :   /home/tholt/CustCdt.csv

Maybe I want a little more information, such as the creation date. Sed can handle that.

system 'DSPLNK OUTPUT(*PRINT) DETAIL(*EXTENDED) DSPOPT(*USER)' | 
   sed -n -e '/Object . ./p' -e '/Creation date/time/p' |
   rfile -wQ qsysprt

As before, the system utility runs the DSPLNK command and sends the report to sed. Since I want sed to look for two patterns, I must prefix each one with the -e switch. The report is longer, but still a single page. Here’s part of it.

  Object . . . . . . . :   *
  Object . . . . . . :   /home/tholt/archive.tar
  Creation date/time . . . . . . . . . . :   11/01/15  18:39:29
  Object . . . . . . :   /home/tholt/archive.tar
  Object . . . . . . :   /home/tholt/custcdt1.txt
  Creation date/time . . . . . . . . . . :   10/13/15  20:30:31
  Object . . . . . . :   /home/tholt/custcdt1.txt
  Object . . . . . . :   /home/tholt/custcdt2.txt
  Creation date/time . . . . . . . . . . :   10/13/15  19:54:52
  Object . . . . . . :   /home/tholt/custcdt2.txt
  Object . . . . . . :   /home/tholt/custcdt3.txt
  Creation date/time . . . . . . . . . . :   10/13/15  20:28:42
  Object . . . . . . :   /home/tholt/custcdt3.txt
  Object . . . . . . :   /home/tholt/customer-data.csv
  Creation date/time . . . . . . . . . . :   11/01/15  18:27:23
  Object . . . . . . :   /home/tholt/customer-data.csv
  Object . . . . . . :   /home/tholt/customer-data.txt
  Creation date/time . . . . . . . . . . :   10/13/15  23:15:13
  Object . . . . . . :   /home/tholt/customer-data.txt

It isn’t a beautiful report, but I can read it, and that’s all that matters to most users. I’d rather look at a short, ugly report with exactly what I need than a long, beautiful one with unnecessary information any day.

Tags:

Best Practices for Doing IBM i Cloud Backup & DRaaS Right

In the technology business for 30+ years including more than a decade in cloud backup and disaster recovery, we’ve learned a few things along the way. Here are best practices to follow – and land mines to avoid.

RELIABILITY. Up to 71% of restores from tape contain failures.

BEST PRACTICE: Use disk-to-disk technology for backups.

With disk-to-disk technology, your backup data resides on disk drives, proven to be far more reliable than tapes. When your backup completes, you know the data is secure and accessible on the disk drive. With tapes you never really know if your data is usable until you try to restore it, at which point it’s too late.

BREADTH OF OFFERING. Choice in product and service offerings meet your business’ needs.

BEST PRACTICE: Don’t settle for less than what you need.

Vendor offerings vary widely. Some are designed primarily for consumers and others for enterprise data centers. Choose a solution that scales and offers the features you need to provide the level of service you expect. De-duplication and delta-block technologies will improve performance, reduce your data footprint and save you money. Find out if their de-duplication offering is at the file level or the block level. Make sure the solution can back up servers, PCs, and laptops as well your applications.

SECURITY. 60% of organizations using tapes don’t encrypt their backups.

BEST PRACTICE: End-to-end encryption with no “back door.”

Using encryption with tape makes backups run slowly and often takes too long to fit within a backup window. As a result, most people simply turn encryption off, creating a security risk. Even with the physical safety of disk-to-disk backup, encryption is essential. Look for 256-bit AES. Find a solution that encrypts your data during transmission and storage. Make certain there isn’t a “back door” that would let someone else view your data.

ACCESSIBILITY. Companies waste thousands of hours waiting on tapes.

BEST PRACTICE: Ensure that you can get your data back with minimal delay.

You should have direct access to your backups, with no time spent on physical transport (no trucks, no warehouses). Restores should take minutes, not hours or days. Set yourself up to work with your data, not wait for it. Make sure your solution provider can meet your Return-to-Operations (RTO) and Recovery Point Objectives (RPO) which determine how quickly you can recover your data and maintain business continuity. Inquire about onsite and offsite replication that provide both improved performance and a solid disaster recovery strategy.

CYBER SECURITY. More than 80 percent of U.S. companies have been successfully hacked, according to a Duke University/CFO Magazine Global Business Outlook Survey.

BEST PRACTICE: Regular cyber security training and phishing tests for all employees using company email are essential to your organization.

Your end-users are the weak link in your network security. Today, your employees are frequently exposed to advanced phishing attacks. Trend Micro reported that 91% of successful data breaches started with a spear-phishing attack. Be sure your vendor of choice includes cyber security training as part of their backup and DR package.

SCALABILITY. Some backup systems can’t scale readily.

BEST PRACTICE: Invest in a data protection architecture that can grow with your business.

You should be able to back up your data no matter how large it grows. Starting small? Look for an option that handles your backups automatically. Then, as you grow, gives you tools to manage complex environments. Look for “changes-only” and compression technologies to speed backups and save space. And insist on bandwidth throttling to balance traffic and ensure network availability for your other business applications. Make sure that their solution offerings rely on common technology to scale easily as your business––and data––grow.

COST-EFFECTIVENESS

BEST PRACTICE: Calculate the true total cost of tape-based back up.

A 2019 Server OS Reliability Survey found that one hour of downtime costs at least $100K for 98% of companies to over $5 million for 34% of surveyed companies. When you do the math, the dollars make sense: Go with disk-to-disk. Unlike tape, there are close to zero handling costs—no rush deliveries, loading, accessing, locating, or repeated steps. And there’s one benefit you can’t factor directly: Reputation. Reliability and security can make an incalculable difference with just one avoided breach or failure.

COMPLIANCE. Most companies have problems satisfying privacy, security, and data retention regulations.

BEST PRACTICE: Choose a data protection partner who has deep know-how about compliance, and the technology to ensure it.

Disaster Recovery. Most companies lack a comprehensive, tested plan for disasters.

BEST PRACTICE: Find a vendor that delivers a complete DR solution.

You can’t say your data protection is complete until you have a disaster recovery plan that is itself complete and tested. Your backup vendor should have both the product mix and professional services team to help you prepare for a worst-case scenario. Make sure they can help configure your backups so you rebound quickly. Best bet: A vendor who can train you to deal with disasters confidently, based on your company’s actual configuration.

EASE OF USE. Some companies don’t —or can’t—manage their backups from one place.

BEST PRACTICE: Get control and reporting you can use anywhere, with ease. Managing your backup environment should be simple, and the software you use should eliminate any guesswork that could lead to lost data. You should know at all times if your data is protected across your entire network—including remote offices—by simply looking at a dashboard. The software should be simple to configure using wizards, yet powerful enough to meet your specific needs with customizable views, job propagation, and roles-based security.

OPERATING SYSTEM AND PLATFORM SUPPORT. Most backup vendors support a limited range of OS, server types, and applications.

BEST PRACTICE: Look for broad and deep technology that supports your complete environment. Your backup solution should accommodate your environment, not vice versa.

CUSTOMER SUPPORT. Backup vendors’ product support varies widely.

BEST PRACTICE: Find a vendor whose support is passionate, maybe even slightly obsessed. Customer support should be one of your vendor’s main selling points. You shouldn’t have to wonder if they’ll be there to help when you need them most. Do they offer phone support or email only, and who exactly are you talking to when you call that 800 number? Find a vendor that will treat your data as if it were their own.

REPUTATION. Does your backup vendor have a quality reputation and the financial resources to stay in business for the long haul?

BEST PRACTICE: Find a vendor with strong financial backing and customer references. There are a lot of vendors that have come and gone. When you consider a service provider, look for one that has strong financial backing, a solid business plan and the ability to be in business as long as your data needs to be stored. Ask for customer references and case studies as their customers are the best validation you can get.

Visit VAULT400.com/proposal to receive a FREE analysis and proposal

DOWNLOAD SOLUTIONS BRIEF:
Best Practices for IBM i Cloud Backup & Recovery

DURING THIS UNPRECEDENTED TIME, DON’T WAIT FOR YOUR BUSINESS
TO SUFFER A DISASTER TO TAKE ACTION.

Serving the US, Canada, & Latin America

VAULT400 Cloud Backup & DRaaS is an IBM Server Proven Solution.

800.211.8798 | info@ucgtechnologies.com| ucgtechnologies.com/cloud

To the First Responders serving on the front-lines during the COVID-19 pandemic,
we extend our heartfelt gratitude.