A Complete Guide for Beginners on Software Testing Tutorial

What is Testing?

Testing is the process of assessing a system or its component(s) with the intention of finding whether or not it meets the specified requirements. In simple words, testing is conducting a program to find any holes, bugs, or specifications that are lacking contrary to the actual requirements.

According to the ANSI / IEEE 1059 standard, testing can be defined as-A process to analyze a software item to detect the differences between existing and required conditions (that is, defects/errors/bugs) and to evaluate the software item’s features.

Why to Learn Software Testing?

Large companies in the IT industry have a team with responsibilities to evaluate the established software in the sense of the requirements provided. In addition, developers also carry out testing that is called Unit Testing. In most cases, the following experts use their respective capacities to test a device.

• Software Tester
• Software Developer
• Project Lead/Manager
• End-User

Different companies have different designations for people who use their experience and knowledge to test the software, such as Software Tester, Software Quality Assurance Engineer, QA Analyst, etc.

Applications of Software Testing

Cost-Effective Development – Early testing saves time and costs in many ways, but reducing costs without testing can result in a software application being improperly designed to make the product useless.

Product Improvement – Testing is never a time-consuming process over the SDLC phases. However, it is a time-consuming but productive activity to diagnose and fix the errors identified during the proper testing.

Test Automation – Test Automation reduces the testing time but during software development, it is not possible to start test automation at any time. Test automation should be started when the software has been tested manually and is somewhat stable. In addition, test automation can never be used if demands continue to change.

Quality Check – Software testing helps in determining the following set of properties of any software such as

• Functionality
• Reliability
• Usability
• Efficiency
• Maintainability
• Portability
• Audience

This tutorial is designed for professionals in software testing who want to understand the Testing Framework in detail, along with its types, methods, and levels. This tutorial provides enough ingredients from where you can take yourself to higher levels of expertise to start the software testing process.

Prerequisites

You will have a clear understanding of the life-cycle of software development (SDLC) before continuing with this tutorial. You should also have a clear understanding of computer software, using any programming language.

When to Start Testing?

An early start to testing reduces the expense and time needed to rework and generate error-free applications delivered to the consumer. However, testing can be started from the Requirements Gathering process in the Software Development Life Cycle ( SDLC), and continued before the program is deployed.

It also depends on the model which is being used for development. For instance, formal testing is performed in the test phase in the Waterfall model; but in the incremental model, testing is performed at the end of each increment/iteration, and the entire application is tested at the end.

Testing is done in different forms at every phase of SDLC −

• Analysis and review of specifications are also known as testing during the required selection process.
• Often known as testing is the review of the design during the design process with the aim to develop the design.
• Testing performed by a developer when the application is completed is often known as research.

When to Stop Testing?

Determining when to stop testing is difficult, as testing is a never-ending process and no one can claim that software is being tested at 100 percent. The following factors to interrupt the research process need to be considered −

• Testing Deadlines
• Completion of test case execution
• Completion of functional and code coverage to a certain point
• Bug rate falls below a certain level and no high-priority bugs are identified
• Management decision

Verification & Validation

For most people, who use them interchangeably, those two terms are very confusing. The table below highlights the difference between verification and validation.

Testing, Quality Assurance, and Quality Control

When it comes to pinning down the distinctions between quality assurance, quality control, and monitoring, most people get confused. Although interrelated and to some extent, they can be considered as the same activities, but there are points of distinction which distinguish them from each other. The table below lists the points which distinguish QA, QC, and Testing.

Audit and Inspection

Audit − Determining how the actual testing process is conducted within an organization or team is a systemic process. Generally speaking, it is an independent review of the processes involved during a software test. According to IEEE, this is a study of recorded procedures introduced and adopted by organizations. Audit forms include the Legal Audit, Organizational Audit, and Program Audit.

Inspection − This is a systematic procedure requiring, by finding some defect or void, formal or informal technical assessments of any product. As per IEEE94, inspection is a systematic assessment technique in which software specifications, designs, or codes are reviewed in detail by an individual or community other than the author to identify defects, breaches of quality standards, and other issues.

The following processes may include formal inspection meetings: planning, preparation for the overview, inspection meeting, rework, and follow-up.

Testing and Debugging

Testing − In software, it involves identifying bug/error/defect without correcting it. Normally professionals with a background in quality assurance are involved in identifying bugs. Testing is carried out during the test phase.

Debugging − This requires the detection, isolation, and fixing of problems/bugs. Developers who code the software perform debugging when an error has been found in the code. Debugging is part of Checking the White Box or Device. Debugging can be achieved during the development process when doing Unit Testing or in phases while fixing identified bugs.

Types of Testing

Manual Testing

Manual testing involves manually testing software, that is, without using any automated tool or script. In this case, the tester takes on an end-user function and checks the program to detect any unintended behavior or error. Manual testing stages, such as unit testing, integration testing, system testing, and user acceptance testing, are various.

Testers use test plans, test cases, or test scenarios to test software to make sure the test is complete. Exploratory testing also involves manual testing, as testers examine the program to find flaws in it.

Automation Testing

Automation testing, also known as Test Automation, is when the tester writes scripts and tests the product using another software. That process involves manual process automation. Automation software is used to re-run the test scenarios that were manually, easily, and repeatedly performed.

In addition to regression testing, automation testing is often used from a load, output, and stress point of view to evaluate the application. It increases the scope of the test, improves accuracy, and saves time and resources compared to manual tests.

What to Automate?

Automating everything in software isn’t possible. Any area where large numbers of users can access the software simultaneously should be automated in the areas where a user can make transactions such as the login form or registration forms.

Additionally, all Interface objects, database links, field validations, etc. can be checked efficiently by automating the manual process.

When to Automate?

Test Automation should be used when considering the following software aspects −

• Large and critical projects
• Projects that require testing the same areas frequently
• Requirements not changing frequently
• Accessing the application for load and performance with many virtual users
• Stable software with respect to manual testing
• Availability of time

How to Automate?

Automation is achieved with the use of a supporting programming language such as VB scripting and an automated software program. Many tools are available which can be used to write scripts for automation. Let’s identify the process that can be used to automate the testing process before mentioning the tools.

• Identifying areas within the software for automation
• Selection of appropriate tool for test automation
• Writing test scripts
• Development of test suits
• Execution of scripts
• Create result reports
• Identify any potential bug or performance issues

Software Testing Tools

The following tools can be used for automation testing −

• HP Quick Test Professional
• Selenium
• IBM Rational Functional Tester
• SilkTest
• TestComplete
• Testing Anywhere
• WinRunner
• LoadRunner
• Visual Studio Test Professional
• WATIR

Methods of Software Testing

There are various methods that can be used to evaluate applications. This chapter briefly outlines available methods.

Black-Box Testing

The testing technique without knowledge of the application’s interior workings is called black-box testing. The tester is unaware of the system architecture and has no access to the source code. Typically, a tester will interact with the user interface of the system while performing a black-box test, by providing inputs and examining outputs without knowing how and where the inputs are being processed.

The following table lists the advantages and disadvantages of black-box testing.

White-Box Testing

White-box research is a thorough analysis of the internal logic and the code structure. White-box testing is also called open-box testing or glass testing. To do white-box testing on an application, a tester needs to know the code’s internal workings.

The tester needs to look inside the source code to figure out which unit/chunk of code is inadequately behaving.

The following table lists the advantages and disadvantages of white-box testing.

Grey-Box Testing

Grey-box testing is a technique for testing the application with limited knowledge of an application’s internal workings. The more you know about the term in software engineering, the more it holds a lot of weight when evaluating an application.

Mastering a system’s domain always gives the tester some edge over someone with limited knowledge of the domain. Unlike black-box testing, where the tester only tests the user interface of the application; the tester has access to design documents and the database in the grey-box testing process. Having this information, when making a test plan, a tester may prepare better test data and test scenarios.

Software Testing – Levels

During the testing process, different levels exist. A brief overview of those rates is given in this chapter.

Test rates include different methodologies that can be used during software testing. The principal software testing levels are −

• Functional Testing
• Non-functional Testing

Functional Testing

This is a form of black-box testing, based on the software requirements to be evaluated. The application is checked by providing feedback, and the findings that need to adhere to the functionality for which it was intended are then examined. Functional software testing is carried out on a complete, integrated system to assess compliance of the system with its specified requirements.

There are five steps that are involved while testing an application for functionality.

• The determination of the functionality that the intended application is meant to perform.
• The creation of test data based on the specifications of the application.
• The output based on the test data and the specifications of the application.
• The writing of test scenarios and the execution of test cases.
• The comparison of actual and expected results based on the executed test cases.

A successful testing methodology will see the above measures applied to each organization’s testing policies and will also ensure that the company meets the most rigorous software quality standards.

Unit Testing

This type of testing is performed by developers to formally conduct the test cases before the system is handed over to the development team. Unit testing is carried out by the respective developers on the areas assigned to the individual units of the source code. The developers use test data which differs from the quality assurance team’s test data.

The aim of unit testing is to isolate each part of the program and show that the requirements and functionality of the individual parts are correct.

Limitations of Unit Testing

Testing can’t catch every single bug in an application. Every execution path can’t be tested in any software program. The same holds true for unit testing.

The number of scenarios and test data that a developer can use to validate a source code is limited. After all the options have been exhausted, there is no choice but to stop unit testing and merge the segment of code with other units.

Integration Testing

Integration testing is defined as testing combined parts of an application to determine whether they are working properly. Integration testing can be performed in two ways: integration testing from the bottom up, and integration testing from the top down.

Bottom-up testing is usually done initially in a comprehensive software development environment, followed by top-down testing. The process concludes with multiple full-application tests, preferably in scenarios designed to imitate actual situations.

System Testing

The system checks the machine as a whole. When all the components are incorporated, the application as a whole is rigorously reviewed to ensure that it meets the Quality Requirements defined. A specialized test team performs this kind of testing.

System testing is important because of the following reasons −

System testing is the first step in the life cycle of software development, where the application is tested in its entirety.
The application is thoroughly tested to verify that it meets both the technical and functional specifications.
The application is evaluated in a production setting that is very close to where the product will be deployed.
System testing helps one to check, verify, and validate both the business specifications and the architecture for the program.

Regression Testing

If a change is made to a software program it is also likely that this change has influenced other areas within the framework. Regression testing is performed to verify that a fixed bug has not resulted in another violation of functionality or the rule of business. The purpose of regression testing is to ensure that no change, such as a bug fix, should result in the application uncovering another fault.

Regression testing is important because of the following reasons −

• Minimize evaluation discrepancies when an application must be reviewed with changes made.
• Testing the new changes to verify that no other area of the application was affected by the changes.
• Mitigates risks on the application when regression testing is performed.
• Test coverage is increased without compromising timelines.
• Increase speed to market the product.

Acceptance Testing

It is probably the most important type of testing, as it is carried out by the Quality Assurance Team, which will determine whether the application meets the intended specifications and satisfies the requirements of the client. The QA team will have a set of pre-written scenarios and test cases used to test the app.

More ideas about the application will be shared and more tests can be carried out on it to gauge its accuracy and the reasons why the project was initiated. Acceptance tests are intended not only to highlight simple spelling errors, cosmetic errors, or interface gaps, but also to point out any bugs in the application that will result in system crashes or major application errors.

The test team will reduce how the application performs in production by performing acceptance tests on an application. There are also statutory and contractual requirements for system acceptance.

Alpha Testing

This test is the first test stage and will be carried out among the teams (developer and QA teams). Unit testing, integration testing, and device testing are called alpha testing when mixed together. The following aspects will be tested during this phase in the application −

• Spelling Mistakes
• Broken Links
• Cloudy Directions
• The Program will be checked for loading times and any latency issues on devices with the lowest specification.

Beta Testing

This test is conducted after positive alpha testing. In beta testing, the program is reviewed by a representative of the intended audience. Beta testing is also known as testing before release. Beta test software versions are ideally distributed to a broad audience on the Web, in part to give the program a “real-world” test and in part to provide a preview of the next release. The audience will test the following in this phase −

• Users must install, run the software, and submit feedback to the project team.
• Typographic errors, the application flow confusing, and even crashes.
• When the project team gets the feedback, they will correct the issues before delivering the app to the actual users.
• The higher the quality of your application, the more issues you fix that solve real user problems.
• Getting an application of higher quality when you report it to the general public can improve customer satisfaction.

Non-Functional Testing

Testing an application from its non-functional attributes is based on this section. Non-functional testing involves testing software from non-functional but important requirements such as performance, security, user interface, and so on.

Some of the major and commonly used types of non-functional testing are discussed below.

Performance Testing

It is mostly used to identify any problems with bottlenecks or performance, rather than to find bugs in software. There are various causes that contribute to software performance reduction −

• Network delay
• Client-side processing
• Database transaction processing
• Load balancing between servers
• Data rendering
• Load Testing

It is a process of testing a software ‘s behavior by applying a maximum load to access and manipulate large input data in terms of software. It can be done under both normal and peak charging conditions. This form of research determines the maximum machine capability and its peak-time behavior.

Load testing is most frequently achieved using automated tools including Load Runner, AppLoader, IBM Logical Performance Tester, Apache JMeter, Silk Artist, Visual Studio Load Check, etc.

In the automated testing tool, virtual users (users) are specified, and the script is executed to verify the program load test. Based on the requirements the number of users can be increased or decreased simultaneously or incrementally.

Software Testing – Documentation

Documentation testing involves documenting artifacts that should be developed before or during the Software testing.

Software testing documentation helps estimate the required testing effort, test coverage, requirement tracking/tracing, etc. This section describes some of the commonly used documented software testing artifacts, like −

• Test Plan
• Test Scenario
• Test Case
• Traceability Matrix

Test Plan

A research plan specifies the technique to be used to evaluate an application, the tools to be used, the testing environment in which testing will be conducted, and the testing limitations and scheduling of testing activities. The Quality Assurance Team Lead will typically be responsible for drafting a Test Plan.

A test plan includes the following −

• Introduction to the Test Plan document
• Assumptions while testing the application
• List of test cases included in testing the application
• List of features to be tested
• What sort of approach to use while testing the software
• List of deliverables that need to be tested
• The resources allocated for testing the application
• Any risks involved during the testing process
• A schedule of tasks and milestones to be achieved

Test Scenario

It is a one-line statement that notifies which area will be tested in the application. Test examples are used to ensure end-to-end validation of all process flows. Depending on the extent and nature of the application, a given region of an application can have as little as one test scenario to a few hundred scenarios.

The words ‘test scenario’ and ‘test case’ are used interchangeably, because there are several steps in a test scenario, while a test case has one stage. Test scenarios are test cases, viewed from this perspective, but they include several test cases and the sequence that they should be executed. Besides this, each test depends on the results from the previous test.

Test Case

Test cases involve a set of steps, conditions, and inputs that can be used to perform test tasks. The primary aim of this operation is to ensure that a program passes or fails with respect to its functionality and other aspects. There are many types of test cases like functional, negative, error, logical test cases, physical test cases, UI test cases, and so on.

In addition, test cases are written for keeping track of a software’s test coverage. There are usually no formal examples that can be used during the writing of test cases. The following components are however always available and are included in each test case

• Test case ID
• Product module
• Product version
• Revision history
• Purpose
• Assumptions
• Pre-conditions
• Steps
• Expected outcome
• Actual outcome
• Post-conditions

Many test cases may be based on a single test scenario. Furthermore, many test cases are often written for a single program that is commonly known as test suites.

Traceability Matrix

Traceability Matrix (also known as Requirement Traceability Matrix-RTM) is a table that is used during the Software Development Life Cycle to trace the requirements. For forward tracing (i.e. from Requirements to Design or Coding) or backward (i.e. from Coding to Requirements) it can be used. RTM contains several user-defined templates.

a requirement in the RTM document is connected to its corresponding test case so that testing can be carried out according to the requirements listed above. In addition, Bug ID is also included and linked to their associated test case and requirements. The primary goals of this matrix are −

• Make sure the software is developed as per the mentioned requirements.
• It helps in finding the root cause of any bug.
• Helps in tracking the developed documents during different phases of SDLC.

Test Point Analysis

This estimation process is used for the black-box or acceptance test function point analysis. Scale, Efficiency, Technique, Interfacing, Complexity, and Uniformity are the key elements of this system.

Also, See SAP Tutorial For Beginners – What is SAP? – Step by Step Guide