Regression Testing

What is Regression Software Testing?

Regression means retesting the unchanged parts of the application. Test cases are re-executed in order to check whether previous functionality of application is working fine and new changes have not introduced any new bugs.

This is the method of verification. Verifying that the bugs are fixed and the newly added feature have not created in problem in previous working version of software.

Why regression Testing?
Regression testing is initiated when programmer fix any bug or add new code for new functionality to the system. It is a quality measure to check that new code complies with old code and unmodified code is not getting affected.
Most of the time testing team has task to check the last minute changes in the system. In such situation testing only affected application area in necessary to complete the testing process in time with covering all major system aspects.

How much regression testing?
This depends on the scope of new added feature. If the scope of the fix or feature is large then the application area getting affected is quite large and testing should be thoroughly including all the application test cases. But this can be effectively decided when tester gets input from developer about the scope, nature and amount of change.

What we do in regression testing?

• Rerunning the previously conducted tests
• Comparing current results with previously executed test results.

Regression Testing Tools:
Automated Regression testing is the testing area where we can automate most of the testing efforts. We run all the previously executed test cases this means we have test case set available and running these test cases manually is time consuming. We know the expected results so automating these test cases is time saving and efficient regression testing method. Extent of automation depends on the number of test cases that are going to remain applicable over the time. If test cases are varying time to time as application scope goes on increasing then automation of regression procedure will be the waste of time.

Most of the regression testing tools are record and playback type. Means you will record the test cases by navigating through the AUT and verify whether expected results are coming or not.

Example regression testing tools are:

• Winrunner
• QTP
• AdventNet QEngine
• Regression Tester
• vTest
• Watir
• Selenium
• actiWate
• Rational Functional Tester
• SilkTest

Most of the tools are both Functional as well as regression testing tools.

Regression Testing Of GUI application:
It is difficult to perform GUI(Graphical User Interface) regression testing when GUI structure is modified. The test cases written on old GUI either becomes obsolete or need to reuse. Reusing the regression testing test cases means GUI test cases are modified according to new GUI. But this task becomes cumbersome if you have large set of GUI test cases.

Black Box Testing

DEFINITION

Black Box Testing, also known as Behavioral Testing, is a software testing method in which the internal structure/design/implementation of the item being tested is not known to the tester. These tests can be functional or non-functional, though usually functional.

This method is named so because the software program, in the eyes of the tester, is like a black box; inside which one cannot see.

Black Box Testing is contrasted with White Box Testing. 

Black Box Testing attempts to find errors in the following categories:

• Incorrect or missing functions
• Interface errors
• Errors in data structures or external database access
• Behavior or performance errors
• Initialization and termination errors

EXAMPLE

A tester, without knowledge of the internal structures of a website, tests the web pages by using a browser; providing inputs (clicks, keystrokes) and verifying the outputs against the expected outcome.

LEVELS APPLICABLE TO

Black Box Testing method is applicable to all levels of the software testing process:

• Unit Testing
• Integration Testing
• System Testing
• Acceptance Testing

The higher the level, and hence the bigger and more complex the box, the more black box testing method comes into use.

TEST DESIGN TECHNIQUES

Equivalence partitioning

Equivalence Partitioning is a software test design technique that involves dividing input values into valid and invalid partitions and selecting representative values from each partition as test data.

Boundary Value Analysis

Boundary Value Analysis is a software test design technique that involves determination of boundaries for input values and selecting values that are at the boundaries and just inside/outside of the boundaries as test data.

Cause Effect Graphing

Cause Effect Graphing is a software test design technique that involves identifying the cases (input conditions) and effects (output conditions), producing a Cause-Effect Graph, and generating test cases accordingly.

ADVANTAGES OF BLACK BOX TESTING

• Tests are done from a user’s point of view and will help in exposing discrepancies in the specifications
• Tester need not know programming languages or how the software has been implemented
• Tests can be conducted by a body independent from the developers, allowing for an objective perspective and the avoidance of developer-bias
• Test cases can be designed as soon as the specifications are complete

DISADVANTAGES OF BLACK BOX TESTING

• Only a small number of possible inputs can be tested and many program paths will be left untested
• Without clear specifications, which is the situation in many projects, test cases will be difficult to design
• Tests can be redundant if the software designer/ developer has already run a test case.
• Ever wondered why a soothsayer closes the eyes when foretelling events? So is almost the case in Black Box Testing.

Definition by ISTQB

Black box testing: Testing, either functional or non-functional, without reference to the internal structure of the component or system.  

Black box test design technique: Procedure to derive and/or select test cases based on an
analysis of the specification, either functional or non-functional, of a component or system
without reference to its internal structure.

Test Design Techniques

Test Design Techniques

Important Terms:

The test development process

Test case specification, test design, test execution schedule, test procedure specification, test script, traceability.

Categories of test design techniques

Black-box test design technique, specification-based test design technique, white-box test design technique, structure-based test design technique, experience-based test design technique.

Specification-based or black box techniques

Boundary value analysis, decision table testing, equivalence partitioning, state transition testing, use case testing.

Structure-based or white box techniques

Code coverage, decision coverage, statement coverage, structure-based testing.

Experience-based techniques

Exploratory testing, fault attack.

Choosing test techniques

No specific terms.                                

Test Design Techniques

o      Specification-based/Black-box techniques

o       Structure-based/White-box techniques

o       Experience-based techniques

I) Specification-based/Black-box techniques

§  Equivalence partitioning

§  Boundary value analysis

§  Decision table testing

§  State transition testing

§  Use case testing

Equivalence partitioning

o      Inputs to the software or system are divided in to groups that are expected to exhibit similar behavior

o      Equivalence partitions or classes can be found for both valid data and invalid data

o      Partitions can also be identified for outputs, internal values, time related values and for interface values.

o      Equivalence partitioning is applicable all levels of testing

Boundary value analysis

o      Behavior at the edge of each equivalence partition is more likely to be incorrect. The maximum and minimum values of a partition are its boundary values.

o      A boundary value for a valid partition is a valid boundary value; the boundary of an invalid partition is an invalid boundary value.

o      Boundary value analysis can be applied at all test levels

o      It is relatively easy to apply and its defect-finding capability is high

o      This technique is often considered as an extension of equivalence partitioning.

Decision table testing

o      In Decision table testing test cases are designed to execute the combination of inputs

o      Decision tables are good way to capture system requirements that contain logical conditions.

o      The decision table contains triggering conditions, often combinations of true and false for all input conditions

o      It maybe applied to all situations when the action of the software depends on several logical decisions

State transition testing

o      In state transition testing test cases are designed to execute  valid and invalid state transitions

o      A system may exhibit a deferent response on current conditions or previous history. In this case, that aspect of the system can be shown as a state transition diagram.

o      State transition testing is much used in embedded software and technical automation.

Use case testing

o      In use case testing test cases are designed to execute user scenarios

o      A use case describes interactions between actors, including users and the system

o      Each use case has preconditions, which need to be met for a use case to work successfully.

o      A use case usually has a mainstream scenario and some times alternative branches.

o      Use cases, often referred  to as scenarios, are very useful for designing acceptance tests with customer/user participation   

II) Structure-based/White-box techniques

o    Statement testing and coverage

o    Decision testing and coverage

o    Other structure-based techniques

§ condition coverage

§ multi condition coverage

Statement testing and coverage:

Statement

An entity in a programming language, which is typically the smallest indivisible unit of execution

Statement coverage

The percentage of executable statements that have been exercised by a test suite

Statement testing

A white box test design technique in which test cases are designed to execute statements

Decision testing and coverage

Decision

A program point at which the control flow has two or more alternative routes

A node with two or more links to separate branches

Decision Coverage

The percentage of decision outcomes that have been exercised by a test suite

100% decision coverage implies both 100% branches coverage and 100% statement coverage

Decision testing

A white box test design technique in which test cases are designed to execute decision outcomes.

Other structure-based techniques

Condition

A logical expression that can be evaluated as true or false

Condition coverage

The percentage of condition outcomes that have been exercised by a test suite

Condition testing

A white box test design technique in which test cases are designed to execute condition outcomes

Multiple condition testing

A white box test design technique in which test cases are designed to execute combinations of single condition outcomes

III) Experience-based techniques

o    Error guessing

o    Exploratory testing

Error guessing

o    Error guessing is a commonly used experience-based technique

o    Generally testers anticipate defects based on experience, these defects list can be built based on experience, available defect data, and from common knowledge about why software fails.

Exploratory testing

o    Exploratory testing is concurrent test design, test execution, test logging and learning , based on  test charter containing test objectives and carried out within time boxes

It is approach that is most useful where there are few or inadequate specifications and serve time pressure.