+ 
From Limited Coverage to Dynamic, Environment-Aware Test Automation
Test Cases Automated
Core Areas Covered
Environments Automated
About Fivable
Fivable is the development company behind LEAD, a learning platform built around guided learning experiences that helps users navigate and complete structured educational content. As LEAD grew and the number of unique instances expanded, so did the need for a reliable and scalable quality assurance process to match their pace of development.
"Every deployment meant our team spending about 5 hours on manual smoke and regression testing that could be better spent elsewhere. Now we've got automation covering everything we used to do by hand, plus a full test suite for things we hadn't previously been testing on a recurring cadence. QLTY Automation has given us our time back, and that's invaluable."
Urban Jaklin, Solutions Architect, Fivable
The Challenge
When Fivable approached QLTY Automation, they had an existing test automation setup, but it had outgrown its original scope and wasn't scaling with LEAD. Coverage was limited to just one of their three core functional areas, leaving critical parts of the platform unverified before each release. When issues came up, the team had to manually verify across environments to isolate and confirm failures, a time-consuming process that didn't scale with LEAD's growing number of instances.
The goal was clear: move beyond the current setup and build a foundation designed to scale.
The Approach
Rather than attempting to salvage the existing automation, QLTY Automation deployed the QLTY Framework, a structured test automation framework built on Python and Selenium, tailored to Fivable's platform and workflows.
Over the course of the engagement, the focus was on:
- Mapping and automating coverage across all three core functional areas of LEAD
- Building a full regression test suite designed to run on demand against their staging environment on every code change
- Ensuring the suite was maintainable and understandable for the internal team going forward
Intelligent Test Selection for Production Deployments
One of the more technically sophisticated pieces of this engagement was solving a real-world deployment challenge: LEAD runs on 50+ server instances, each with its own configuration, meaning not every test is relevant to every environment.
To handle this, QLTY Automation developed a dynamic test selection system. In collaboration with Fivable's development team, a dedicated API was enabled to expose configuration flags per instance. At runtime, the system reads those flags, determines which test cases are applicable, and executes only the relevant subset, generating a report scoped specifically to that instance.
The result: no wasted test execution, no irrelevant failures, and no manual intervention to figure out what to run where. As long as the configuration is kept current, the system adapts automatically.
The Results
100 test cases captured and automated
Across all three core functional areas, up from minimal coverage in just one area
Hours of manual verification eliminated per deployment
With 50+ environments, tracking down and verifying failures manually could take hours. Automation now catches issues across all instances automatically
A full regression suite the team can trust
Run on demand to catch regressions before they reach production
Full traceability through TestRail integration
Every test execution is reported to TestRail, giving developers clear visibility into what was tested, what passed, and what failed, allowing them to swiftly track down issues
Increased confidence deploying at scale
Every sprint, across all environments
"QLTY Automation set up a future-oriented and highly effective automated testing framework for our exact needs that has had an immediate impact on our confidence running deployments at scale every sprint. It is a testament to the value delivered that we're still likely going to engage them again, in spite of the fact that they set us up so well we could continue on our own."
David Cooley, Solutions Architect, Fivable
Ready to Build Automation That Actually Works?
Tell us about your testing challenges and we will show you what reliable automation looks like for your team.
Get in Touch