When should you use managed testing?
Managed Testing is an ideal choice when the testing topic is complex and involves coordinating multiple parties, which requires resources and expertise about interdependent technical requirements. When used properly, Managed Testing saves teams time, money and resources while generating high-quality results without the need for in-house test engineers.
Although automating QA activities reduces the time and resources involved in manual testing, purchasing software licenses and testing in-house isn’t always the right option, especially in cases such as:
Lack of in-house expertise and knowledge
Limited personnel and resources
Budget constraints
Managed Testing solves these issues by providing software, test engineers and automation know-how for the duration of a test project. This leaves customers with more time to focus on their core business, while a third-party team of experts look after your test project from start to finish while collaborating closely with all stakeholders.
What does managed testing include?
Managed Testing Services cover all activities in a test project, including the entire testing process. At the beginning, the customer and service provider discuss the “what”, “when” and “how” of the project as well as the “who”. Managed Testing is recommended when the testing topic is complex and requires coordinating several parties, all of which requires resources and expertise that the customer may not have.
Therefore, Managed Testing requires defining.
What needs to be tested?
This defines the system under test, as well as requirements or features that need to be tested. Eventually this should be formalized as a list of test cases.
How it should be tested?
This defines the depth and detail of the tests. For example:
- When testing phone networks, does it just check that call was made, or does it also check that the customer was charged correctly?
- Should system records or logs be collected and verified?
When testing should occur and when the testing should be complete?
Does testing take place at certain times? Are there expected outages or potential issues that could delay completion?
Who are the key stakeholders?
Who needs to be included in different communication and decision-making processes?
QiTASC’s customers often use our Managed Testing Services for long-running projects that include multiple phases and take an incremental approach to test automation. With every new project phase, a subset of existing test cases might be re-used in regression tests or incorporated into a larger build. This ensures that the existing solution implementation is unaffected by newly introduced configurations or code changes.
Managed Testing may also include periodically testing and retesting new versions of a software, firmware, app newly-introduced features on different devices. In such cases, we may also develop newly-defined and written test cases to cover any new configurations, features or other implementations.
What are provider and customer responsible for?
No two Managed Testing projects are the same, and the combination of services varies from project to project and customer to customer. High-quality testing services involve strong communication between the service provider and the customer that ensure worry-free projects, avoid delays and clearly define both each party’s responsibilities.
Responsibilities of the service provider
The service provider is typically responsible for the test case tool landscape
- Test case development
- Running and maintaining test cases
- Report and defect resolution support
- Retesting following but fixing
Managed Testing may also involve uploading results to an external quality management tool, analyzing the software or product’s requirements and creating the appropriate test scope to cover these requirements. For example, here at QiTASC, we may open defects in an issue-tracking tool and managing these issues. Managed Testing Services may also include finding and coordinating activities internally, with the customer and with third parties.
Responsibilities of the customer
The customer is typically responsible for the provision of a working and complete test environment maintenance of that environment. Depending on the scope of the Managed Testing, it may also cover defect management and tracking.
For example, QiTASC’s customers usually provide us with the following:
- Access to the test lab
- The test bed (such as systems, SIM cards)
- Contribute and develop test case list and requirements
- Create communication channels to supporting customer departments
- Define escalation processes
What are the key benefits of using managed testing instead of in-house testing?
Organizations choose Managed Testing because of the flexibility, predictability and reduced costs it entails. By consolidating all testing activities, licenses, services and points of contact to one point, projects become easier to track and forecast. This improves time-to-market and facilitates a calculable cost and timeframe for any project. Furthermore, having an objective, outside partner who runs test projects also helps maintain an impartial and neutral approach to identifying and fixing defects.
When is managed testing not the right choice?
Although Managed Testing is an excellent option for large, on-going and complex projects, it is not suited it for all cases. For example:
- A project is very small, and all resources are available
- Project/technology know-how is very specific
- Intellectual property rights must be kept secure
- The used test data must stay secure, such as customer data
- Short running, one-off projects
Conclusio
Managed Testing is an efficient and cost-effective solution for complex, large and on-going test projects that require a great deal of automated testing expertise. This is especially true when limited time and resources mean that purchasing new automation software licenses and training employees in-house is more costly than outsourcing it to experts. The greatest benefits of Managed Testing are that it improves quality and time-to-market by consolidating all testing activities to an outside, expert partner with experience managing far-ranging and complex tests. However, Managed Testing is not always the right choice: When a project is very small, know-how is specific or when certain artifacts must be kept secure, testing in-house is usually the best choice. The same is true for short-running or one-off projects.
By partnering with QiTASC, customers can continue to focus on internal activities, fixing defects and preparing a product for the market while we handle all QA aspects. This includes creating test scopes, running, executing and analyzing defects as well as communicating with different stakeholders. Because communication between QiTASC and the customer is essential to a successful testing project, Managed Testing defines clear roles and points of contact from the very beginning.
When we discuss Managed Testing with our customers, we discuss all the requirements, expectations, and timelines in-depth and reach a conclusion about how best to proceed. In cases where Managed Testing might not be appropriate, we propose alternatives, which may include in-house training and testing or a different type of testing service that is less comprehensive but provides the customer with exactly the right tools, know-how and predictability that they need.
