Software Testing Techniques

As I had informed in my first post that my area of interest being Software Engineering – Here is one post on Testing Techniques. In this post I will be only explaining about the Testing stratergies and in the next post, will explain about the various metrics for testing.

What is Testing:

Succeeding in the software industry takes more than just delivering a quality product to the marketplace in a timely and efficient manner. Today’s software products are increasingly complex, often operating within multi-layer, multi-platform environments. These products are being created in rapid development environments that drive them toward the marketplace at breakneck speed. Software testing is being fit into a rapid development cycle at a time when the test requirements are more extensive than ever. This dichotomy can affect the quality of the product and the reputation of your company.

 

Testing is checking the correctness, completeness, quality and security of the software developed. Testing is one of the important phase of software development which makes the software development team to improve the quality of the software. After each phase of the software development, testing is done to ensure the quality.

Testing Objectives:

• Testing is a process of executing a program with the intent of finding an error.
• A good test case is one that has a high probability of finding an as-yet-undiscovered error.
• A successful test is one that uncovers an as-yet-undiscovered error.

Testing Principles:

Davis suggests a set of testing principles. They are:

• All tests should be traceable to customer requirements
• Tests should be planned long before testing begins
• Apply Pareto principle to software testing. So wat is Pareto Principle – It states that 80 percent of the uncovered errors in testing is from the 20 percent of the software components.
• Testing should begin by focussing on testing the individual program components to the integrated clustered components.
• According to Davis, testing of all possible combinations is impossible as each combination leads to a different path, but it is possible to test all the logic statements and also check that all conditions in the component level design have been tested.
• The testing should be done by a third party. The software engineers who create the software is not the best person for testing the software. That is why we have testing teams to test the softwares.

Having known the what are the objectives and principles of testing, lets move on to know what are test cases.

Test Cases:

A test case is a set of conditions used by the tester to test if the requirements have been met by the software. There are various ways in which a test case can be generated. The testing tools helps the tester to generate the test cases.

If the application is created without formal requirements, then test cases are written based on the accepted normal operation of programs of a similar class.

What characterizes a formal, written test case is that there is a known input and an expected output, which is worked out before the test is executed. The known input should test a precondition and the expected output should test a postcondition.

Under special circumstances, there could be a need to run the test, produce results, and then a team of experts would evaluate if the results can be considered as a pass. This happens often on new products’ performance number determination. The first test is taken as the base line for subsequent test / product release cycles.

Written test cases include a description of the functionality to be tested taken from either the requirements or use cases, and the preparation required to ensure that the test can be conducted.

Written test cases are usually collected into Test suites.

Test Plan:

Software Testing Institute gives a clear idea of what is test plan and how the testing is to be planned. There are a various steps that are to be followed:

• Establish the scope and purpose of the test plan. Analyze the purpose, background, Technical architecture, specification, scope and project information.
• The next step is the requirement phase where the requirements are tested starting from testing the functions to testing the user-interface to testing the component integration is done.
• Next step is the Test Stratergy. This describes how the test objectives will be met for each type of testing that may be part of the test plan: unit, function, integration, system, volume, stress, performance, configuration and/or installation testing.
• Develop the project plan
• Document the schedules
• List all the deliverables
• Track the defects and report it to the software engineer
• Last get the approval and produce the result.

Test Case Design:

A rich variety of test case design methods have evolved for software, These methods provide the developer with a systematic approach to testing. Most important methods provide a mechanism that can help to ensure the completeness of tests and provide the heighest likelihood for uncovering errors in software.

White Box Testing:

White box testing is common in the quality assurance world. It is sometimes called clear box, open box, or simply informed testing. In white box testing, all information about the system under test is known to the tester. In the security world, this can also be thought of as an insider attack. The tester has access to the source code and design documentation. This allows the tester to be efficient. He can threat-model the system or do a line-by-line code review, looking for information to guide the selection of test data.

White box testing is the most efficient way to find security vulnerabilities. Why hide information from the security test team? More information allows quicker and more complete generation of interfaces to test. It also gives you an accurate picture of the system’s security because it doesn’t rely on security by obscurity, which is the hope that attackers will never discover information about how a system works. Security by obscurity is not real security. You should always assume that eventually all information about a system will be discovered or leaked. A well-designed and well-implemented system will still be secure. This is why good crypto algorithms can be published for review. They don’t rely on privacy for security.

Black Box Testing:

Black box testing involves examining the system as an outsider would, using tools to detect the attack surface and probe the system for internal information. With no internal knowledge of the system, the tester builds an understanding of the system. Information leakage is especially important to the black box tester because it helps him build more understanding than he would otherwise get by manipulating a leak-free program.

Many testers swear by black boxing techniques to complement white box testing. If too much emphasis is given to specifications and design documentation, the tester may miss parts of the system that were built incorrectly or were not included in the documentation. This out-of-spec functionality may harbor security flaws that must be discovered. Black box testing lets the tester probe all of the attack surface and generate test data for functionality that may not be in the design.

Gray Box Testing
Ideally both white box and black box techniques are used during security testing. White box testing is used to discover flaws in functionality that were specified in the design and development. Black box testing is used to discover flaws without having access to these application internals. Sometimes this combination is called gray box testing.

The application security tester typically performs gray box testing to find vulnerabilities in software. Flaws due to design and flaws due to unspecified functionality are equally important to discover. Because the source code is available to the security tester, it should be used to improve productivity.

Various Test Tools:

There are various software testing tools that help the tester to test the software. Some of them include:

Visual Test, SQA Suite, SQA Suite for PeopleSoft, PureCoverage, Purify, PreVue and others by Rational

There are various testing tools which you can read here..

Referrences:

1. For definition of whitebox, blackbox and graybox testing – computer world about finding software security flaws

2. Test Plan – From the article of Software Testing Institute

3. A book on Software Engineering by Roger Pressman

4. For definitions – Wikipedia

5. List of testing tools – http://www.cs.queensu.ca/~shepard/testing.dir/under.construction/tool_list.html