Client Context
Scale Factory was engaged to provide testing services for an International Law firm, who was in the process of implementing a heavily customised and integrated Practice Management System.
The Challenge
Scale Factory had a number of challenges to overcome as part of this engagement.
There was no formal development roadmap in place which made it difficult for Scale Factory to plan in terms of resourcing requirements.
The integration team had little existing architectural and infrastructure documentation to work with.
The Approach
Scale Factory was joined by multiple providers on the programme including the client, the vendor, application experts, data analysts, training providers and other testing providers. Scale Factory’s responsibility included governing the overall testing effort, providing quality assurance of the other test providers, implementing test processes (defect, review, rework) and tools (TFS), providing regular reporting (passive and active) to various levels of stakeholders as well as executing the testing and defect management.
The development of customisations was to be delivered in a manner leaning heavily on Agile principals however the wider programme and integrations were following a more traditional Waterfall methodology.
The primary role of functional testing involved Scale Factory working with a third party test supplier who were application experts. The third party was responsible for writing test scripts. Scale Factory implemented a review process for these scripts using the third party’s Leads to perform an initial peer review.
This was subsequently passed to client Subject Matter Experts to review from a business perspective. Finally, Scale Factory reviewed to ensure they were written to standards before execution and uploading into Microsoft’s Team Foundation Server.
The team was also responsible for planning and coordinating a UAT phase involving over 300 scenarios, executed by 150 business users, spread over 5 different global locations.
Scale Factory’s total team onsite flexed throughout the duration of the programme up to a maximum of 15 onsite concurrently. The team’s structure included; Test Director, Test Management, Test Leads and Test Analysts. The role of the Test Director was to set strategy, manage stakeholders, control budget and set resource models depending on demand and constraints.
To help ensure efficient delivery of the testing, an onboarding process was set up that involved a mixture of application and process documentation, shadowing and mentoring as well as using logical functional areas of the system for initial execution.
A rotation system was implemented to remove ‘key man’ dependencies.
An integration team was established to focus on the 30+ direct and indirect integrations in the overall landscape that relied on the core product being implemented. This required definition of requirements and a formal Integration Test Plan to be defined.
The plan utilised the priority of each integration, as well as the complexity to order the testing in a risk-based approach. It also provided a resource model and forecast. One critical component of the end-to-end solution involved a US-based 3rd party testing provider. Scale Factory was asked to coordinate with them to set expectations on progress and ingest their metrics to provider wider reporting.
As well as functional testing Scale Factory also implemented a full automated framework using Eggplant which fully automates the 3E regression pack. This has developed to an automation first approach where 80% of all manual 3E scripts are now automated.
To support the onsite team and the client, Scale Factory delivered through an account management service which involved regular account review meetings with the programme director and the client head of QA. A mid-point service review was also conducted. This involved a test expert not involved in the programme coming onsite reviewing processes and collateral, interviewing Scale Factory resources and stakeholders, and delivering a health report to both the internal team and the client.
Finally, regular and ad-hoc meetings were organised for the programme director and various other stakeholders to meet Scale Factory’s Client Engagement Director to discuss tactical and strategic consulting needs, providing valuable expert guidance on best practice.
Scale Factory worked with the development partners to identify forecasted development delivery of new code or fixes and based on this was able to identify any constraints to increasing productivity. When this constraint was resource-related, Scale Factory was able to plan onboarding of additional personnel in a timely and planned fashion.
This was resolved through a discovery phase to document the entire application landscape with integration points and data flows. The team worked with solution architects, enterprise architects, application owners and the client infrastructure team to put this in place. The resulting documentation was then used to create a test plan to cover; data migration testing, point-to-point integration testing, end-to-end system integration testing and user acceptance testing.
Outcomes & Impact
Through strict test governance and Scale Factory’s ability to flexibly scale up and down testing resources in line with development forecasts, we were able to successfully support the delivery of a major PMS transformation project on time and on budget.
This has led to Scale Factory being continually utilised for ongoing PMS enhancements as well as a further major upgrade which is planned through 2020.
In addition, the automation-first framework that is now in place had delivered significant timesaving in terms of 3E execution. This allows the client better and more regular regression coverage, improving test quality and identifying defects early.
Client: Hogan LovellsIndustry: LegalTechnologies: Elite 3E Enterprise Business ManagementScale Factory Capabilities: Quality and Test Strategy Consultancy, Managed Test Services, Performance Test and EngineeringScale Factory was engaged to provide testing services for an International Law firm, who was in the process of implementing a heavily customised and integrated Practice Management System.
The Project
Scale Factory was joined by multiple providers on the programme including the client, the vendor, application experts, data analysts, training providers and other testing providers. Scale Factory’s responsibility included governing the overall testing effort, providing quality assurance of the other test providers, implementing test processes (defect, review, rework) and tools (TFS), providing regular reporting (passive and active) to various levels of stakeholders as well as executing the testing and defect management. The development of customisations was to be delivered in a manner leaning heavily on Agile principals however the wider programme and integrations were following a more traditional Waterfall methodology.
What we did
The primary role of functional testing involved Scale Factory working with a third party test supplier who were application experts. The third party was responsible for writing test scripts. Scale Factory implemented a review process for these scripts using the third party’s Leads to perform an initial peer review. This was subsequently passed to client Subject Matter Experts to review from a business perspective. Finally, Scale Factory reviewed to ensure they were written to standards before execution and uploading into Microsoft’s Team Foundation Server. The team was also responsible for planning and coordinating a UAT phase involving over 300 scenarios, executed by 150 business users, spread over 5 different global locations. Scale Factory’s total team onsite flexed throughout the duration of the programme up to a maximum of 15 onsite concurrently. The team’s structure included; Test Director, Test Management, Test Leads and Test Analysts. The role of the Test Director was to set strategy, manage stakeholders, control budget and set resource models depending on demand and constraints. To help ensure efficient delivery of the testing, an onboarding process was set up that involved a mixture of application and process documentation, shadowing and mentoring as well as using logical functional areas of the system for initial execution. A rotation system was implemented to remove ‘key man’ dependencies. An integration team was established to focus on the 30+ direct and indirect integrations in the overall landscape that relied on the core product being implemented. This required definition of requirements and a formal Integration Test Plan to be defined. The plan utilised the priority of each integration, as well as the complexity to order the testing in a risk-based approach. It also provided a resource model and forecast. One critical component of the end-to-end solution involved a US-based 3rd party testing provider. Scale Factory was asked to coordinate with them to set expectations on progress and ingest their metrics to provider wider reporting. As well as functional testing Scale Factory also implemented a full automated framework using Eggplant which fully automates the 3E regression pack. This has developed to an automation first approach where 80% of all manual 3E scripts are now automated. To support the onsite team and the client, Scale Factory delivered through an account management service which involved regular account review meetings with the programme director and the client head of QA. A mid-point service review was also conducted. This involved a test expert not involved in the programme coming onsite reviewing processes and collateral, interviewing Scale Factory resources and stakeholders, and delivering a health report to both the internal team and the client. Finally, regular and ad-hoc meetings were organised for the programme director and various other stakeholders to meet Scale Factory’s Client Engagement Director to discuss tactical and strategic consulting needs, providing valuable expert guidance on best practice.
Challenges
Scale Factory had a number of challenges to overcome as part of this engagement. There was no formal development roadmap in place which made it difficult for Scale Factory to plan in terms of resourcing requirements. Scale Factory worked with the development partners to identify forecasted development delivery of new code or fixes and based on this was able to identify any constraints to increasing productivity. When this constraint was resource-related, Scale Factory was able to plan onboarding of additional personnel in a timely and planned fashion. The integration team had little existing architectural and infrastructure documentation to work with. This was resolved through a discovery phase to document the entire application landscape with integration points and data flows. The team worked with solution architects, enterprise architects, application owners and the client infrastructure team to put this in place. The resulting documentation was then used to create a test plan to cover; data migration testing, point-to-point integration testing, end-to-end system integration testing and user acceptance testing.
“Partnering with Scale Factory to develop a highly effective and professional testing approach has enabled us to pursue the implementation with pace and confidence.” Steve Capon, Head of Global Finance Projects, Hogan Lovells
Benefits
Through strict test governance and Scale Factorys ability to flexibly scale up and down testing resources in line with development forecasts, we were able to successfully support the delivery of a major PMS transformation project on time and on budget. This has led to Scale Factory being continually utilised for ongoing PMS enhancements as well as a further major upgrade which is planned through 2020. In addition, the automation first framework that is now in place had delivered significant timesaving in terms of 3E execution. As of April 2020, 800 3E scenarios are automated which take 1 day to execute. If these were to be executed manually it would take 2-3 weeks for a single resource. This allows the client better and more regular regression coverage, improving test quality and identifying defects early.
How you can achieve the same results
Deliver programmes, projects, small changes and improvement initiatives with Scale Factory’s test and quality consultants who bring a wealth of experience to support our clients to realise change and transformation with confidence. Read more about our Quality Engineering solutions and speak to a member of our team to learn how we can help you.
“Partnering with Scale Factory to develop a highly effective and professional testing approach has enabled us to pursue the implementation with pace and confidence.”
