Microsoft Dynamics CRM & GP End-to-End Testing for a Global Software Company

Background

Our client is a global software company based in New Jersey. Their products are used by thousands of companies worldwide, with over 12,000 deployments in 70 countries. 

As part of their business process and Customer service enhancement, they implemented Microsoft Dynamics 365 (Sales, Marketing and Customer service), SSO, Customer portals and Microsoft Dynamics GP.

Challenges

With hundreds of manual test cases, the quality assurance process for the client’s Dynamics 365 CRM application was quickly becoming a challenge to maintain. The application was constantly growing (as new features were added) and the client had to find ways to reduce the testing time, increase the number of test cases and improve the overall system quality without introducing any additional testing time. The challenges include –

• Several customisation and integrations to suit business needs caused significant quality challenges
• The release of application changes did not match the business demands
• Limited knowledge of Quality Engineering Best Practices and principles- Shift Left, Fail-fast, Risk-based, etc.
• Limited test resources & lack of test assets lead to key business processes being untested
• The constant acquisition of new organisations required ongoing changes to Dynamics CRM
• Dynamics knowledge gap and limited Dynamics testing experience within the internal team

Solution

Utilising our 10- vector QE framework we assessed the landscape and prepared a plan of action against each of the identified areas of improvement. To ensure maximum collaboration we educated the client team on the benefits of the 4 QE principles we administer:

1. Automation first,
2. Shifting left,
3. Eliminate waste and
4. Transparent and consistent visualisation of progress/results.

Our Cost to value delivered matrix identified the low hanging fruit with the maximum gains and in collaboration with the client, we embarked on a journey to transform their QE landscape.

Four of the major solutions that scripted our success:

• Bridging the knowledge and skill gap in the challenging IT landscape via our D365 established practice that maintains a federated structure. Our Functional consultants and D365 IP helped to improve the quality of requirements without having to hire additional consultants.
• Optimising the unwieldy test pack: We used our THREG-D365 framework to categorise the existing test suite, merge and purge the test cases led to the formation of functionality based modular regression pack that facilitates risk-based regression.
• Improving automation efficiency: Analysis of the automation suite revealed limited coverage and scalability hence we upgrade the framework with our technology-agnostic framework- Frameium. Our existing open-source framework was customised to enable the automation of the different integration points in the client system. The reusable D365 automation IP helped to improve test coverage with minimal design effort.
• Shift left and Transparency: these 2 principles were enabled with no additional effort for the customer as they came as existing capabilities within Frameium. We were able to integrate with the client ALM system – JIRA and have a single source or reporting

Results

The business started to see an acceleration in the frequency of releases with a significant reduction in quality-related issues.

We helped the client to reduce the test execution time by 40% and added additional test cases without affecting the schedules. All test cycles were kept under control. The full test suite that previously required weeks of effort by more than four engineers was reduced to a tester with few hours of effort.

. An extensive library of keyword actions and reusable functions were created specifically for Dynamic 365 CRM, allowing even non-technical testers to create new test suites with no programming support. This approach enabled 80% of the tests to be automated.

Customer Benefits –

1. Gatekeeper: Value of Quality was realised as an established decision point with a centralised and value-adding service for the customer
2. Time to Market: Customer moved from Quarterly releases to fortnightly releases with overnight daily automated regression cycles
3. Quality: Defect leakage to UAT was brought down from 22 % to just 4 % in every release
4. Preparedness – Significant improvement in client’s ability to manage features & updates and ability to implement & react to change
5. Agility: Customer became truly agile and Ad-hoc release could be scheduled after risk-based regression cycles proved effective
6. Customer Work Force Optimisation:  helped to boost employee morale by ensuring employees got a chance to upskill and cross skill
7. Shifting Left: Developers’ unit test quality improved by adopting an automation framework to build BDD based unit tests at the API layer
8. Holistic Approach: In-house Non -functional capabilities helped the customer to address concerns concerning security and performance features.
9. Evergreen Readiness – We enabled preparedness for Evergreen using automated assets & procedures
10. Coverage – Significant improvement in test coverage providing business confidence in application quality & consistency
11. Efficiency – Increase in test efficiency through automation improved speed of delivery
12. Strategy – Our new test strategy brought structure and stability to quality
13. Preparedness – Significant improvement in client’s ability to manage features & updates and ability to implement & react to change