The client is a major UK Facilities Management Company operating in the Facilities Management industry. The engagement centred on supporting their Microsoft Dynamics 365 (D365) implementation, which was experiencing significant delays in the testing phase. The company was seeking expert support to stabilise and accelerate their quality assurance process.
The Challenge - Delays, High Rework and Manual Overhead
The client was facing significant issues that delayed their Microsoft Dynamics 365 (D365) implementation and sought expert support to stabilise and accelerate the quality assurance process. The core challenges included:
- Absence of D365 Expertise: The project lacked a dedicated Quality Assurance (QA) function with essential knowledge and experience in Dynamics 365 (D365).
- Inefficient Regression Testing: The regression testing process was entirely manual, resource-heavy, and time-consuming, requiring eight testers and four weeks to complete.
- High Defect Leakage: The manual process led to a notable defect leakage, with 14% of defects reaching User Acceptance Testing (UAT) and 7% leaking to production.
- Insufficient Coverage: There was insufficient test coverage at integration points between modules and the process of role-based testing was manually intensive and time-consuming.
The Solution - Establishing a Federated QE and Automation Framework
Testhouse was invited to join the project mid-implementation due to testing delays. The solution focused on introducing structure, automation and specialised expertise:
- QE Structure: A federated Quality Engineering (QE) structure was established, incorporating functional and technical consultants linked directly to specialised Centres of Excellence (CoEs).
- Targeted Test Design: The team engaged with Business Stakeholders to identify everyday life scenarios for effective testing and classified test cases into various packs for optimised regression testing. Services included Regression Testing, Automation Testing and User Acceptance Testing Support.
- Automation Framework: Testhouse adapted its open-source automation framework, Frameium, to validate end-to-end business processes.
- Continuous Automation: The team successfully achieved 80% regression test automation coverage and developed functions that enabled in-sprint automation levels of 40%.
- Tools: The process was managed using Jira for Project Management, Xray in Jira for Test Management and Azure DevOps for integration.
The Results - Metrics of Savings and Speed
The intervention successfully stabilised the D365 implementation and delivered significant business benefits, converting testing from a bottleneck into an accelerator:
| Metric |
Outcome |
Detail |
| Annual Savings |
$2 million USD |
Achieved in annual operational savings (FY24-25) by optimising the testing lifecycle and reducing the test effort. |
| Defect Leakage Reduction |
80% |
Achieved an 80% reduction in defect leakage to User Acceptance Testing (UAT) from 14% to less than 4% and 0% leakage to production. |
| Release Frequency |
Fortnightly |
Release frequency was improved from quarterly schedules to fortnightly schedules |
| Regression Test Coverage |
80% |
Automated 80% of the regression suite with full coverage on PI and P2 regression tests. |
| In-Sprint Automation |
40% |
Developed reusable functions, reducing automation design effort by 40% and enabling in-sprint test automation for functional BAU changes. |
