Custom Test Tool Development

JKVine

Nizam Abdallah, Senior Testing Specialist:

Custom test tool development can be used to speed up manual testing, creating a semi-automated approach to save time and improve quality. 

Nizam

Nizam Abdallah, Senior Testing Specialist, JKVine

This is possible without being overly expensive or creating too much overhead. So what is custom test tool development and why should I consider it?

Automated testing methodologies have been around for many years and are useful in improving the speed of test execution, test coverage and overall application quality. Many test automation tools available today such as HP UFT, Smart Bear TestComplete and the popular open source framework Selenium, focus on automation of front-end UI functionality.

The promise of these types of tools is they reduce the amount of manual testing required during the development process. If your goal for test automation is to automate a suite of UI-based regression tests to be executed periodically, to determine if existing functionality has been broken, then this approach may be suitable.

However, if your goal is to try and find new defects in the application under test, then a complete automated solution is not the answer. Rather, a semi-automated approach, where manual testers use tools to help them find and identify the root cause of defects, can help get more done.

While off-the-shelf tools can be used to assist in finding defects, there is no one-size-fits-all solution, because each application developed is different in complexity and architecture. A suitable approach is the development of multiple small, specific tools and utilities to assist the software test team to do their job.

An analogy I like to use is: If the goal of traditional test automation methodologies is to act like a robot attempting to perform a testers job, then the goal of semi-test automation is like placing a tester inside a mech robot. This can enable them to perform their jobs more thoroughly and accurately and, as a result, improve the overall quality of the software being developed.

The benefits of having a suite of custom built tools, which can be used by many testers across many projects, outweighs the initial cost of development. This also enables the test team to perform more thorough and accurate testing, giving more confidence in the quality of the software being developed.

Custom Test Tool Development

Without getting into the discussion of “Do testers need to know how to code?” (which I believe they do); custom test tool development is a software development task. Whoever is writing the test tool needs to not only understand how to code, but also what they are trying to achieve by developing the tool, in order to ensure the test results and conclusions they produce are accurate.

Tools can be written in any programming language and do not need to have a fancy UI. In fact, some of the most versatile tools I’ve created are command line tools. Custom tools I’ve created and repeatedly used include:

  • Test data generators, including the generation of names, addresses, post codes, phone numbers, barcodes, account numbers, GUIDs etc.
  • Malformed data generators, including the generation of malformed strings, malformed SQL injection strings, malformed code injection strings
  • Malformed file generators, for both common files (PDF, DOC, XLS etc.) and proprietary file types.
  • Fuzzers
  • Data Comparison Tools
  • System diagnostics reporting tools

Many custom tools can also be integrated with popular commercial off-the-shelf and open source test tools. Tools such as HP ALM, Rally, Jira, JMeter, SoapUI, UFT, TestComplete and Burp Suite Pro all have extensibility features, allowing custom features to be added and accessed from existing test projects.

How to Start?

A common question I’m asked is, “How do I start with custom test tool development?” If you are a manual tester and would like to build custom tools to use on your projects, the first step is to learn how to code. To start, pick an easy to learn language such as Python, Ruby, VBScript or even Excel or VBA.

The points below outline how to easily incorporate custom test tool development into the development process, while minimising the impact on project timelines:

  • Start simple
  • Build/extend existing tools throughout the project
  • Build a framework to allow for quick reuse of existing functionality
  • Create a project template in the language of your choice to allow for a rapid start to development of any tool
  • If building a framework in Python, add the framework directory to your PYTHONPATH to allow for module reuse
  • If building a framework in .NET, register the DLL as a COM object in Windows so you can use this functionality in both .NET and non .NET applications
  • Use extensibility functionality of commercial and open source tools already used by your team

Useful Links: