• The Four Hundred
  • Subscribe
  • Media Kit
  • Contributors
  • About Us
  • Contact
Menu
  • The Four Hundred
  • Subscribe
  • Media Kit
  • Contributors
  • About Us
  • Contact
  • Thoughts on the Coexistence of Full Test Automation and Manual Testing

    April 3, 2007 Colin Armitage

    Automated software testing solutions have been around for more than 20 years but successful implementations are few and far between. Most organizations continue to execute the majority of their testing manually. Industry analysts peg market proliferation at just five percent, with 30 percent being the most extreme estimate. One reason for this damning failure is that users were promised that test automation could address 100 percent of their quality assurance requirements, but bitter experience has shown that total success was far more elusive.

    For these reasons, it can be said that anyone who tells you that software testing can be totally automated is lying.

    Expectations are a big part of the problem. The reality is that full automation is not always appropriate and the amount of effort necessary to achieve full automation should never be underestimated.

    There will always be a trade off between developing a fully automated process and just getting by with the manual testing. The key metric in the discussion of whether it is wiser to automate or not automate is re-use. It will always take longer to develop an automated test bench but there can be overwhelming payback if tests can be executed multiple times. If this “economy of scale” business case cannot be made, then organizations will always logically choose to execute their testing manually.

    In real life, the situation is very complicated since, for example, any third party application that has undergone extensive modification could be a moving target for testers. With new releases and bug fixes always in the queue, it’s nearly impossible to nail down a stable version to test. If an automated solution cannot adapt to a forever-changing application, then the potential for re-use simply disappears, along with the justification and user enthusiasm for automation.

    On the other hand, the great thing about automation is that it enables types of testing that are practically impossible to approach manually. Simulating extreme workloads and logging a program’s response can help you avoid large-scale problems once your application goes live. To attempt to test the breaking point of an application manually, dozens of developers or users would be needed for weeks or months and this is simply not practical.

    Regression testing is the process of comparing the results of initial baseline tests with those of current tests. The object of regression testing is to ensure that areas of an application are unchanged by a new release and continue to work as intended. Full regression testing can provide vital business stability throughout application changes but is incredibly difficult to perform manually due to the time, resource, and knowledge constraints.

    Life would be so much easier if the barriers to develop automated process could be lowered. All the traditional solutions require in depth programming skills that are rarely found among the quality assurance (QA) fraternity, whose strongest members come from the business side and are subject matter experts, rather than technical gurus.

    Even if testers can be persuaded to embrace such skills, it can be difficult to find time for the development of test scripts, test the scripts themselves, and train legions of testers in their use, while continuing to make progress with on-going test projects.

    With seemingly no clear or simple way to transition from a manual testing protocol to one that avails itself to the benefits of automation, people grow confused, lower their heads, and continue with the old way of doing things. In doing so, they lose out on automation’s important benefits: Principally the ability to address one of modern business’ key challenges, which is how do I reduce application time to market and thus deliver real world competitive advantage?

    You can solve most problems by first adopting a good strategy.

    To gain the commitment from the business and test teams to improve QA it is essential that the potential benefits from automation are aligned to effort and time expenditure, and can be incrementally delivered when the business requires. I call this approach Crawl, Walk Run.

    Crawl

    We all know the importance of an effective plan that enables you to take control and manage the whole testing process. The ability to manage and report progress against that plan is becoming a business imperative due to the ever increasing compliance and audit pressures modern businesses are facing today.

    As already discussed, a significant portion of testing is and always will be manual, so let’s start there. Hopefully you are already recording your manual testing along with any defects that are discovered. The challenges you face include reproducing defects, constructing test audit trails and verifying the results at visual and database levels.

    The last six months have seen some exciting developments in this area, not least the announcement of assisted manual testing solutions that can automatically document and track your steps while testing in a non-intrusive fashion, allowing you to correct defects much quicker, by re-creating the problem at a touch of a button.

    With the initial successes and quick ROI enjoyed in this Crawl phase, you will have more time to continue process improvements which will in turn allow you to benefit even more.

    Walk

    Once the testing fundamentals of planning and manual testing have been addressed you should be in a position to move to the Walk phase, which for many companies revolves around the subject of managing test data. What does managing test data involve? In a nutshell it is the creation, management, and re-use of data in a secure and protected environment. Test data management is a cornerstone to successful testing and is an area where technology can often delivery a quick payback.

    Effective test data creation provides more practical test verification and it also addresses issues relating to disk space, data verification, data confidentiality and protracted test cycles. Companies that do not manage their test data properly will always find any testing automation very painful, and verification almost impossible.

    Run

    With the fundamentals in place and a strong hold on your test data, you are in a powerful position to implement full test automation in every area it can be justified. We have already seen that effectively deployed software automation solutions can have a significantly positive effect on functional and regression testing timescales. Implemented correctly, automation can offer great benefits in areas such as time to market, risk reduction, cost savings, and resource efficiencies.

    Colin Armitage is CEO at Original Software, a company that specializes in the development of automated software testing solutions. Armitage is a renowned expert on software quality control.



                         Post this story to del.icio.us
                   Post this story to Digg
        Post this story to Slashdot

    Share this:

    • Reddit
    • Facebook
    • LinkedIn
    • Twitter
    • Email

    Tags:

    Sponsored by
    Racksquared

    It’s time for IBM Power in the Cloud!

    Stop buying hardware and make the move to the cloud. It’s easier and more cost effective than you might think.

    • IBM Power in the Cloud
    • IBM Power Backup Solutions
    • IBM Power High Availability and DR solutions
    • IBM Power Colocation with Management and Monitoring

    Let’s talk about your business needs.

    Call: 855-380-7225
    Email: Sales@racksquared.com

    Share this:

    • Reddit
    • Facebook
    • LinkedIn
    • Twitter
    • Email

    Sponsored Links

    VAULT400:  Securely archive data with Instant Back-Up & 24x7 Recovery
    Bsafe:  Enterprise Security for System i and Linux Systems
    COMMON:  Join us at the 2007 conference, April 29 – May 3, in Anaheim, California

    IT Jungle Store Top Book Picks

    The System i Pocket RPG & RPG IV Guide: List Price, $69.95
    The iSeries Pocket Database Guide: List Price, $59.00
    The iSeries Pocket Developers' Guide: List Price, $59.00
    The iSeries Pocket SQL Guide: List Price, $59.00
    The iSeries Pocket Query Guide: List Price, $49.00
    The iSeries Pocket WebFacing Primer: List Price, $39.00
    Migrating to WebSphere Express for iSeries: List Price, $49.00
    iSeries Express Web Implementer's Guide: List Price, $59.00
    Getting Started with WebSphere Development Studio for iSeries: List Price, $79.95
    Getting Started With WebSphere Development Studio Client for iSeries: List Price, $89.00
    Getting Started with WebSphere Express for iSeries: List Price, $49.00
    WebFacing Application Design and Development Guide: List Price, $55.00
    Can the AS/400 Survive IBM?: List Price, $49.00
    The All-Everything Machine: List Price, $29.95
    Chip Wars: List Price, $29.95

    BOS Shows Improved Financials as 2006 Comes to a Close FTP Means ‘First Try Pinging’

    Leave a Reply Cancel reply

Volume 7, Number 13 -- April 3, 2007
THIS ISSUE SPONSORED BY:

Vision Solutions
Aldon
COMMON
Patrick Townsend & Associates
RJS Software Systems

Table of Contents

  • CYBRA Finds the ‘Edge’ for Native i5/OS RFID Software
  • Lakeview Adds More Autonomics to MIMIX
  • Thoughts on the Coexistence of Full Test Automation and Manual Testing
  • Help/Systems Boosts Graphics with Robot/NETWORK V10
  • Infor Buys Workbrain for About $200 Million
  • Quadrant Addresses Canadian Check Requirements
  • IBM Delivers New Release of RFID Software
  • Esker Launches Fax Services for SAP
  • Make That Three 5250 Emulators Supporting Vista
  • ProData’s Hartley a Split Second from Milestone Victory

Content archive

  • The Four Hundred
  • Four Hundred Stuff
  • Four Hundred Guru

Recent Posts

  • IBM Tweaks Some Power Systems Prices Down, Others Up
  • Disaster Recovery: From OS/400 V5R3 To IBM i 7.4 In 36 Hours
  • The Disconnect In Modernization Planning And Execution
  • Superior Support: One Of The Reasons You Pay The Power Systems Premium
  • IBM i PTF Guide, Volume 25, Number 13
  • IBM i Has a Future ‘If Kept Up To Date,’ IDC Says
  • When You Need Us, We Are Ready To Do Grunt Work
  • Generative AI: Coming to an ERP Near You
  • Four Hundred Monitor, March 22
  • IBM i PTF Guide, Volume 25, Number 12

Subscribe

To get news from IT Jungle sent to your inbox every week, subscribe to our newsletter.

Pages

  • About Us
  • Contact
  • Contributors
  • Four Hundred Monitor
  • IBM i PTF Guide
  • Media Kit
  • Subscribe

Search

Copyright © 2023 IT Jungle