Untested Backup and Recovery Fools Midrange Shops
December 14, 2009 Dan Burger
The operations and day-to-day management of IT systems is a pain point that persistently stabs at small to mid-sized businesses. This may be particularly true for organizations with 100 people or less. In more ways than just the bottom line, these companies operate far differently than the world’s largest enterprises. Costs are examined much more closely. Hundreds of thousands of dollars don’t just fall through the cracks. The IBM AS/400–the Power Systems i–when Big Blue chooses to speak about platforms, has lived here for many years in obscurity.
Obscurity is in the eye of the beholder. As far as IBM is concerned, this territory is a little hazy. Big Blue is more at home in the executive offices of its customers. But the SMB is the land of bread and butter for many IBM business partners, who have trusted relationships with these businesses. The AS/400 may have the well-earned reputation for dependability, but on the service side the folks who work most closely with customers in the trenches with their business needs are the glue that connects IBM with the midrange in many instances.
Business needs in the midrange are changing as IT becomes more expensive and more complex. Integrating IT into a business requires a commitment that more small businesses would like to avoid. This fits with the idea that businesses want to focus on what they do best, which may be making and distributing widgets or providing financial services. When IT becomes a distraction to a business focus, there’s a problem.
Companies that are not putting an appropriate measure of focus and investment in existing IT run the risk of being fooled into thinking they have something that in reality is missing. Take, for example, backup and recovery systems.
I’m using this example because I’ve recently talked with Doug Fulmer, the consulting systems architect for Clear Technologies. The customers that rely on Clear Technologies are predominately midrange organizations with fewer than 100 employees. They also rely on IBM AS/400, iSeries, System i, and Power System i servers and applications that run on OS/400 or IBM i. At least 90 percent of the IBM i customers fit into or are close to this 100 employees and less midrange demographic.
From his view of the midrange, Fulmer sees a trend toward outsourcing aspects of IT that go beyond the skills and/or the time allowances (there are still only 24 hours in a day) of those running IT.
“We are seeing a lot more customers throwing operations and administration and day-to-day management of the system over the transom to somebody else as they try to cut costs,” Fulmer says. One area that Clear Technologies has honed in on is backup and recovery because, he says, “We think there is value to our customers of going in and making sure their backup and recovery strategy actually works.” Most companies don’t have anyone on staff who can do that or who has the time to do it.
Fulmer explains the reality of the Clear Technologies’ situation like this:
“Pretty much everyone we deal with has a backup strategy that involves full-system saves once every x period of time. Most are doing it only once a quarter and some haven’t done a full-system save in three years. They use Save While Active during the evenings or back up selected libraries, but the real danger is that because customers have not had their systems fail, they have not really tested their recovery side of their backup strategy. They think they have things backed up, but they don’t test. To some extent that is because they don’t have any extra hardware or a time window long enough to gamble on taking their own tapes and restore to their own hardware to make sure it works.”
It’s easy to agree that a backup and recovery plan that is untested is not a solid business plan. Very few companies have tried to do a restore to prove it can be done, Fulmer says. It’s a gamble that companies probably don’t even know they are taking. There’s an assumption that recovery is fairly easy and predictable. If you are going with that assumption, you should also assume things that sound too good to be true usually are.
During what Clear Technologies calls a restore verification, both the save side and the recover side go under the microscope. A problem can be as simple as changing the save procedure to make sure that all the objects are getting saved. In some cases, the correction might be more complex and could require a revision of the save and restore procedures and documentation that allows processes to be repeated on a consistent basis. Without a process that is understood and documented, periodic testing cannot and most likely will not happen.
“The idea,” Fulmer says, “is to have a backup and restore policy in place that the customer can do. They should be able to do this on a regular basis with job scheduler or with existing people as long as they can put tapes in the tape library and follow the instructions. They should be able to save and restore.”
A large majority of Clear Technology customers are willing to do this work themselves as part of their backup and recovery policies. Some will call in the experts for help in getting it right. Meanwhile, Fulmer says, the percentage of companies that want Clear Tech to take over full responsibility for backup and recovery “is small but it is growing.”
One reason is that companies are searching for ways to reduce employee-related costs.
“They know that an AS/400 runs without much intervention,” Fulmer says, “so if they can find someone like Clear who they can trust, who they know is competent, and who can be on site when needed, they are willing to pay for that.”
How much and how often the SMB will pay for IT services is still to be determined. It’s an inexact science and still early in the sorting out process.
Fulmer and Clear Technology’s CEO, Van Symons, as well as about a half dozen staff members are former IBMers with AS/400 experience. Although Clear is a hardware reseller, he says the company is positioning itself to be more services oriented because of the growing demand for skilled professionals who can handle things like backup and recovery, remote administration, and security. It has a business partner relationship with Vision Solutions for disaster recovery and high availability software.