CASE STUDY
Optimized QA Practices Lead to Productivity and Quality Improvements
Background
The company is a leading worldwide business software provider and has a SaaS application which provides next-generation student information system for colleges and universities.

The client was experiencing a large volume of high severity defects on its SaaS application resulting in post-deployment escalations and firefighting to resolve quality issues. Crosslake became a partner in the client's efforts to improve its overall quality processes, tools and metrics with the goal of reducing the volume of high severity defects.

Situation and Business Challenge
The client's SaaS application was constantly growing (more features were added) and we had to find a way to increase the amount of test cases, reduce testing times and improve the overall quality without introducing additional testing time. Crosslake proposed two phases to improve the overall quality of the product:

  1. Assess existing test process and tests cases.
  2. Evaluate the current test automation approach.
The first step was assessing the existing test process and test coverage. Lack of complete test coverage and incomplete execution and analysis of the regression test bed caused risks to product quality. Test case standards and management was inconsistent across locations. New feature testing and identification of new functional test scenarios for regression was not defined as a repeatable process.

The client had begun a test automation project, focusing exclusively on UI test automation using Selenium WebDriver. Crosslake evaluated the automated tests and found that the majority of tests were CRUD tests and over time had not been valuable in finding defects.

Crosslake has demonstrated expertise in implementing agile development and QA practices and was instrumental in rolling out a new, enterprise class development model for us.
Jerry L., Director of QA
Solution
Our approach to advance testing focused on increasing test coverage and reducing testing times.

Increasing test coverage

By identifying key use cases and creating a traceability matrix, Crosslake worked with the client to address gaps in the regression test suite and increase coverage. To help the Company understand and increase test coverage and report on test execution progress, Crosslake standardized test case management. Crosslake helped the client establish a dedicated regression sprint with change controls and quality metrics. To ensure a repeatable process for new features testing, Crosslake implemented user story based acceptance tests and a process at the end of every sprint to identify new functional testing for regression. Crosslake also engaged Customer Support and Professional Services to test the product and feed high severity escaped defects back into the testing process.

These improvements contributed substantially to the overall procedures and helped the testing process to be effective and sustainable.

Reducing testing times

Crosslake worked with the client to define a multi-layered test automation strategy. To augment the UI automated testing Crosslake recommended test automation at the API/Services layers. Starting with automating a high ROI smoke test, Crosslake helped accelerate test automation and also identified best practice patterns to be applied to the client's test framework. We set up an offshore test automation delivery team and are in the process of helping the client execute on the multi-layered test automation strategy.

Results
  • Within a 3-month timeframe increased test cases 15x resulting in the prevention of 115 high severity defects being released to customers resulting in fewer escalations and less firefighting post deployment.
  • Increased regression test coverage without introducing additional testing time.
  • Automating a high value smoke test led to higher percentage of defects identified in the early stage of testing and improving test efficiencies.
  • Introducing a multi-layered test automation strategy positions the Company for further reduction in test cycle times and reduced maintenance.
  • Lower total cost of ownership through introduction of offshore test automation delivery.
  • Developed enhanced status reporting and bi-weekly metrics to dashboard to track progress.

Modernize=Survive.