Guild Companies

Are you testing your software before promoting it into production? More specifically, beyond unit or programmer testing, do you use an internal quality assurance department to test your software before it gets promoted into production? If so, you are probably wasting time and money. You are just as well off promoting your software into production right after a clean compile.

"Nonsense!" you say. Your test environment is top-notch, complete with proper test data, comprehensive test scenarios, and, most important, knowledgeable and motivated people, to ensure that the data is proper and that the test scenarios are comprehensive, right? Okay, then how large is your support staff? How many bug fixes do you have on your backlog? Does your IT department mildly resemble that of an emergency room after an English football match? If your answers to these questions are "small," "none," and "no" respectively, you can stop reading this article, pick up the phone, and call me--because I've got to see your environment--or call Liar's Anonymous, because you have other issues that I can't help you with right now.

Take the typical post-development routine. Instead of talking about software, let's talk about building bridges. Say we've just built this beautiful bridge, and to give it the IT spin we'll say that this bridge cost way more money and took much longer to build than originally estimated. Now that we have our bridge, we had better test to see if it works. The developer writes a test scenario and tests the bridge. That test goes something like this: Get an 80-pound girl to walk across the bridge and back (got to have a complete test), because she is the only person around. If she is able to perform that test satisfactorily, you've exhausted your test data and test scenarios and the bridge is now off to QA.

Now that QA has the bridge, they first check to see if you have a test plan. Without a test plan the bridge would quickly get slammed back to the bridge developer. But you've got a test plan: Have someone walk across the bridge and back. Sounds good. Well, who does QA get to walk across the bridge? QA doesn't have test people sitting around waiting to walk the bridge, so they use the bridge developer's test data: the 80-pound girl. QA then performs other checks to make sure the bridge is the right color and that all the right paper work has been turned in, and if all that is good, your bridge has passed QA and is off to production.

We all know what happens next. Hundreds of people cross the bridge at once, some weighing 380 pounds, and the bridge collapses, sending these people plunging to some unpleasing consequence below. The bridge developer's manager, the mayor of the town, and all the relatives of the people who fell off the bridge then pay a visit to--that's right, the bridge developer, not QA. All that, and the bridge still needs to be fixed.

Sounds ridiculous, right? Of course it does, but this kind of thing happens in IT shops all over the world, everyday. So what went wrong? There are many places to start, but let's just say that it's common practice for the QA department to use the developer's test plan. In that case, the developer should have written a more thorough test plan, which would have included more than one 80-pound girl crossing the bridge and back. Next, the developer should have had better test data at his disposal. Even if a better test plan were written, if there is only one 80-pound girl around to cross the bridge, that's all the testing you're going to get. So, in line with a proper test plan--which might include the minimum weight and number of people expected, the maximum weight and number of people, people running and jumping on the bridge, and so forth--there must be appropriate data. Specifically, we need a copy/subset of production data.

Regardless of what the bridge developer does, QA must understand what needs to be tested. If this is the first time they've ever tested a bridge, there must be some communication between the developer and QA to determine what really needs to be tested. If this is just one of many bridges, QA should have previously written test plans, which have been modified over the years as the process of bridge development has changed. Maybe bridges are getting smaller or larger, or are made from different materials. The point is, if QA has already tested bridges, and they should have documentation on what they tested, how the test performed, and any changes that were made based on the testing. QA should then have access to plenty of people to test the bridge. The 80-pound girl won't do. If that's all the developer had access to, well that's too bad; QA should go find the necessary people who fit the well-written test plan: a copy/subset of production data.

Once they have enough people to perform the test, QA should ensure that the test is performed in a safe environment, because it is the goal of QA to break the bridge. QA shouldn't test to pass something; they should test to fail something. QA has to find out how many people or how much weight it takes for the bridge to crumble. When it does crumble, the test data should only fall a couple of feet to a nice soft landing, as opposed to the production environment, which is hundreds of feet and not so soft.

It sounds like a lot of work to write a proper test plan, to ensure that the data is production-like, and to maintain a safe place to test, but when you step back and think about the overall picture, this up-front work is much less expensive, time consuming, and tragic than being visited by the manager, the mayor, and the victim's relatives, and rebuilding the bridge, and running the bridge through the same QA process that failed the first time.

Of course, responsibility falls on the developer, too. He is the one doing the work, but if you're going to have a QA department and incur all the time and cost associated with it, you owe it to yourself to take a good look at the process and see that it's not going to allow any collapsible bridges in your town.