In software testing, a test plan is a must-have. A well-defined and comprehensive test plan provides all stakeholders involved with necessary information of the testing roadmap. Through it all members gain a shared vision for the testing approaches, strategies, objectives, resources, and timelines.
In this article, we will deep-dive into what a test plan is, why it is so important, and how to create a good test plan that truly improves the overall efficiency of your testing activities.
A test plan is a formal document that serves as a comprehensive and structured description of the testing activities and strategies that will be employed to check the quality of the software system under test. This document provides detailed information on the approach, methodology, scope, objectives, resources, timelines, and risks associated with the project.
The test planning process can be different depending on the methodology the team chose. For example, in Agile development, the test plan can be developed in smaller chunks during each separate iteration instead of at the very beginning of the entire project for other testing methodologies. This allows for better revision and adjustment of the test plan in case problems arise.
Everyone benefits in one way or another from a good test plan.
Without a good test plan, QA managers risk having uncoordinated testing effort, leading to inefficient allocation of resources. Test plan is not a fixed, one-off document. The initial metrics and timelines written on it can totally be adjusted along the way by QA managers to optimize resource use.
Team members would still need a test plan to have clear goals and objectives to follow during the sprint. From a technical perspective, a test plan typically includes detailed information on the testing methodologies, techniques, and tools that will be used to conduct the testing. It also includes information on the test environment, including hardware, software, and network configurations, as well as any external dependencies or services required for testing.
Read More: How To Build a Good Test Infrastructure?
For other stakeholders and the development team, the test plan is a crucial reference point, ensuring that everyone is aligned on the same vision. Information from the test plan can be used to keep track of the testing progress and make timely intervention in case a problem arises.
Another aspect of the test plan that many tend to gloss over: it identifies potential risks and issues that could impact the testing effort and outlines strategies and plans for mitigating these risks. Risk management boosts testing confidence, allowing teams to better prepare for unexpected scenarios.
Creating a test plan is a collaborative effort. The QA manager is usually the owner of the test plan, and it usually takes about 2 - 3 days to build and review a simple test plan, while complex test plans may take about 1 week or more to fully finish.
A test plan is essentially a project management plan, which should always include the major components of a normal plan for project management purposes, including an overview, scope, methodology, resources, schedule, risks, and communication plan.
Of course, a test plan should go beyond that and also include information that would be beneficial to the testing process. Here are 8 crucial items that should be included in a good test plan:
Having these items in a test plan makes it a "good" test plan and not just a normal test plan. It ensures that the testing effort is comprehensive, organized, focused on achieving the desired outcomes while minimizing the risks and challenges associated with testing. It also facilitates collaboration and communication among the testing team members, and it enables stakeholders to monitor and evaluate the effectiveness of the testing process.
Simply put, both the Test Plan and Test Strategy are essential documents in the testing process, but they serve different purposes and address different levels of detail. The Test Plan focuses on the specifics of testing for a particular project, while the Test Strategy sets the overarching approach and principles for testing across projects or the entire organization.
You can have a look at the table below to better understand the differences between the two
Aspect | Test Plan | Test Strategy |
Definition | A Test Plan outlines the approach, scope, objectives, resources, and schedule for testing a specific project or product. | A Test Strategy defines the high-level approach to testing for an organization or a project, guiding the overall testing process. |
Scope | Focuses on the details of how testing will be carried out for a specific project. | Encompasses a broader perspective, outlining principles, objectives, and methods to achieve consistent testing across projects. |
Purpose | Provides a detailed roadmap for the testing activities, including what, when, and how to test. | Sets the direction for testing efforts, aligning them with the organization's goals and helping in making important testing decisions. |
Audience | Primarily for the project team, including testers, developers, managers, and stakeholders. | Aimed at management, project leads, and high-level stakeholders involved in decision-making for testing strategies. |
Contents | Includes test scope, objectives, resources, schedule, test environment, test cases, entry/exit criteria, risks, and deliverables. | Covers testing approach, testing types (functional, performance, security, etc.), tools, defect management, risk assessment, and overall process. |
Level of Detail | Provides a detailed breakdown of test activities, including specific test cases, procedures, and schedules. | Offers a broader overview of the testing approach, principles, and guidelines, rather than focusing on specific details. |
Timeframe | Typically created for a specific project phase or release. | Generally applies to multiple projects or the entire organization and is less time-bound. |
Flexibility | More rigid and specific to the project, allowing less flexibility to adapt to changing circumstances. | Offers more flexibility in adapting to various project needs, as it doesn't delve into specifics. |
Dependencies | Based on the broader guidelines outlined in the Test Strategy. | Driven by the organization's policies, best practices, and project-specific requirements. |
Communication | Essential for aligning the project team and stakeholders on the testing approach and execution. | Ensures that testing efforts are aligned with the overall organizational goals and standards. |
Revision and Updates | Needs to be updated more frequently, especially with changes in project scope or requirements. | Changes less frequently and provides a stable framework for testing efforts across multiple projects. |
Example Use Case | Creating a detailed plan for testing a specific software release or product update. | Establishing guidelines for how various types of testing (functional, performance, etc.) will be conducted within the organization. |
Before creating a test plan, make sure that you have involved all the necessary stakeholders. The QAs should not be the only one doing this. Developers should provide technical input on the system architecture, software design, and coding standards to inform the testing approach. Similarly, the business analyst and domain-specific experts might also need to be involved to provide insights from the business side. Cross-team effort in the test planning should be encouraged.
After that, we can start test planning, which consists of 9 steps:
This is the preliminary stage when everyone gathers to develop a deep understanding of the product under test, most importantly its underlying architecture. Along the development roadmap, when testers have gained a firm understanding of the product, this stage gradually turns from product analysis to a review session of previous testing efforts.
Testers typically begin by reviewing the product specifications, documentation, and any other relevant information that is available to identify the features and functions of the product that need to be tested. Also, make sure to identify any dependencies or external systems that the product interacts with, since bugs may arise due to miscommunication between these modules.
Ask your stakeholders:
Based on the information gathered from the Product Analysis session, the QA manager starts to develop a Test Strategy document, which is a higher-level plan that outlines the project objectives. Usually it details the types of testing to be conducted so teams can prepare resources as well as technology to better meet these objectives.
There are plenty of testing types, but they usually are categorized into 2 major groups: functional tests (such as API testing) and non-functional tests (such as visual testing).
These objectives are not set in stone. Keep in mind that you can always shift your test objectives. However, at the earlier stages, a priority value should be assigned to each objective, so that teams know which objectives should be accomplished first. Usually those would be objectives with many dependencies. Over time, in face of requirement changes and evolving needs, teams may need to deprioritize certain objectives, and the test plan will be where we keep track of those changes.
Another important aspect of defining test objectives is the entry and exit criteria. Essentially they are sets of conditions that must be met to begin and end a particular phase of software development and testing.
Several entry criteria to consider:
Several exit criteria to consider:
At its core, defining entry and exit criteria is defining the scope of your test plan. These criteria are usually defined and agreed upon from the get-go so teams have a clear idea of where they are supposed to be heading.
Sometimes teams may even need to define a suspension criteria i.e. when should teams stop testing entirely and send the code back to the dev team to troubleshoot. This happens when there are too many bugs that it becomes counterproductive to test. To avoid this, teams usually conduct sanity checks beforehand (which are basically small-scale testing to gauge the overall quality of the code) so that they don’t waste resources doing unnecessary tests on a broken build.
This is when we get into the granularity and the specifics. Based on all of the information outlined before, the QA team members can now start brainstorming test scenarios. Think about how users interact with your system.
Let's say that the testing team is working on testing a new e-commerce website that allows users to purchase products online. The team has the following business logic, requirements, and test objectives:
Business Logic:
Requirements:
Test Objectives:
The testing team can brainstorm test scenarios based on the business logic, requirements, and test objectives. Here are some examples:
Scenario 1: Search by category
Objective: Verify the accuracy and speed of the search features
Test Steps: Search for products by category, such as electronics, apparel, or home goods. Verify that the search results are accurate and displayed quickly.
Scenario 2: Add products to shopping cart
Objective: Verify the accuracy of the pricing information in the shopping cart
Test Steps: Add one or more products to the shopping cart. Verify that the pricing information is accurate and reflects any discounts or promotions.
Scenario 3: Checkout process
Objective: Verify the security and ease of use of the checkout process.
Test Steps: Navigate through the checkout process, including entering shipping and billing information, selecting a shipping method, and reviewing the order summary. Verify that the process is secure, easy to use, and free of errors.
Scenario 4: Payment process
Objective: Verify the reliability and error-free operation of the payment process.
Test Steps: Choose a payment method, such as credit card, PayPal, or Apple Pay. Enter payment information and complete the payment process. Verify that the payment is processed correctly and that no errors occur.
Note that the examples above are must-haves, and QA teams must brainstorm even more scenarios depending on the specific design and context of the application under test. For example, a product that allows recording and voice search may require testing for the integration and interaction with external devices.
Another interesting question to be raised is “What defines a feature work or not work?”. To have a clear-cut plan, you need to include assertions, which are conditions that a detected bug is truly a bug. Here is a quick list of 10 assertions that you can use:
You should always categorize which tests should be performed manually or automatically.
With all of the important items listed out, both teams have a clear idea of what will be tested. The QA manager can now work on resource planning. Usually the following resources should be available.
The important element in resource planning is the testing tool. Not just a normal automation testing tool that allows you to create and execute tests. For the most value, you would want an automation testing platform that encompasses test planning, creation, execution, management, environment setup, and test reporting - for all of your testing types, in 1 place.
The content of a test case will be organized into different folders. This simplifies test planning when all of the test artifacts are displayed in 1 place, allowing your team to have a comprehensive, intuitive, and hierarchical view, no matter what testing stage you are in.
Object Repository is Katalon's centralized location for storing and managing your UI elements and properties such as buttons or text fields. They can be captured with Object Spy or while recording a test.
Deciding how to execute your tests is also easier. Instead of planning for investments into physical machines to run different types of tests, or considering local/remote execution, with Katalon, you’re just a command-line away from running Katalon tests in parallel with other automated test suites on:
For test planning specifically, Katalon and Jira integration helps you report, track, and manage defects. Testers can filter tickets, find root causes with linked requirements, and view defect trends to streamline the tracking process and improve day-to-day jobs.
Not just test planning, with the Katalon Platform, teams can script and debug automation tests, build their own keywords library or record test steps from UI interactions. With record-and-playback, Katalon allows testers to auto-capture test objects, properties, and locators for faster test authoring.
For those who have coding expertise and are looking for customization, Katalon has a full-code scripting interface in Java/Groovy. Code-support features, including syntax highlighting, code suggestion, and debugging are also available.
Test deliverables are the artifacts that are produced during the testing process and that provide evidence of the testing activities that were performed. These deliverables may vary depending on the type of testing being conducted and the specific requirements of the project. Examples of test deliverables include:
If you don't know which test cases to start with, here are the list of popular test cases for you. They should give you a good foundation of how to approach a system as a tester.
After that, based on the list of deliverables, you can start thinking about the test schedule. Estimate the approximate time it takes for each stage of the testing activities based on their complexity, their dependencies, resources required, and many other factors.
Set milestones along the way. They are the signals and temporary checkpoints to tell you and your team where your team is heading. Have clear delivery dates.
If you notice, we are essentially following the SMART rules. Sticking to the 5 letters of the SMART rules and you should have a well-defined test plan.
This is the final QA step for your QA plan. Review the items you laid out. Ask yourself several questions to check your previous decisions:
Once it’s all good, you have a clear test plan ready to be used.
A test plan is a document that describes the scope, approach, resources, and schedule of intended testing activities. It identifies the items to be tested, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning.
A test plan should include objectives, scope, approach, resources, schedule, test deliverables, dependencies, test environment, risk management, roles and responsibilities, and a communication plan.
In Agile development, test planning is iterative and done in smaller chunks during each sprint. This allows for continuous revision and adjustment based on feedback and changing requirements.
Typically, the QA manager is responsible for creating the test plan, but it should be a collaborative effort involving input from developers, business analysts, and other stakeholders to ensure comprehensive coverage and accuracy.