The road to CI/CD: Scaling automated testing at Point

In a previous blog, we shared how Point's Quality Engineering (QE) team has been on a mission to enhance product quality and team efficiency. We’ve taken strategic steps to optimize our testing processes, including building a robust regression test suite, shifting left with automation, and incorporating data-driven insights to improve decision-making. 

These investments have led to significant improvements in key engineering metrics—faster test cycles, higher-quality products, and a smoother development cycle. As a result, our engineering team can deliver more business value with shorter turnaround times while simultaneously improving developer experience.

Our journey to CI/CD

Every engineering team strives for greater efficiency, faster releases, and higher-quality software. While the goal of continuous integration and deployment (CI/CD) is common, the path to achieving it is unique for each organization. 

At Point, we have three distinct teams, each with distinct challenges and requirements: one focusing on customer-facing applications and two focusing on complex internal applications to support critical business processes. A single, uniform approach wouldn’t work for all teams, as each has its own dependencies and constraints.

In this post, we’ll walk through our journey toward CI/CD—the challenges we faced, the solutions that worked, and the lessons we learned.

Laying the groundwork

Before diving into implementation, we focused on establishing a strong foundation for a smooth transition:

Strategic alignment for CI/CD success:

• Aligning with engineering leaders: We worked closely with engineering leadership to foster a culture supportive of CI/CD.
• Empowering CI/CD through QE-DevOps collaboration:  We collaborated with the DevOps team on CI/CD improvements, ensuring close cooperation between QE and DevOps for successful implementation.
• Identifying champions: We appointed key Quality and DevOps champions within each team to drive the transition and act as points of contact.
• Enabling feature flags: We worked with developers and DevOps to implement feature flags, allowing incremental deployments and controlled feature rollouts.

Process optimization for seamless CI/CD:

• Embedded test automation: We made it easier for developers to contribute to automated tests by ensuring test code is placed in the most suitable location—either within the same repo or a separate repo—based on each team's needs.
• Tracking key metrics: We monitored test coverage, certification time, deployment time, and release frequency to continuously refine our CI/CD process.
• Managing merge conflicts and test failures: We collaborated with DevOps to develop processes for handling test failures and merge conflicts efficiently.
• Implementing a rollback strategy: We built a robust rollback mechanism to ensure quick recovery in case of deployment failures.

With this groundwork in place, we adjusted our release frequency for each team. Initially, releases were scheduled at fixed times, but by increasing automation and reducing dependencies, teams were able to double their release cadence.

Key requirements and milestones

To achieve CI/CD, we identified key technical requirements and broke our journey into a series of milestones that addressed those needs incrementally. 

Key requirements

Ephemeral environments

To prevent conflicts between concurrent development efforts, we implemented ephemeral test environments—temporary, self-contained test setups that could be spun up and torn down on demand. This allowed developers to run tests in isolation without interfering with others’ work.

To streamline setup, we created a pull request (PR) template that automatically configured ephemeral environments with necessary integrations, ensuring consistent test conditions.

On-demand testing with GitHub Actions

With ephemeral environments in place, we needed a seamless way to trigger tests. By integrating GitHub Actions (GHA), we allowed developers to start tests by adding predefined labels to PRs. This automated process provided flexibility for developers to validate changes at any point in their workflow.

Optimizing testing for speed and reliability

• Prioritized test flow: We categorized tests by priority, ensuring critical tests ran first to provide rapid feedback.
• Addressing flaky tests: We identified and resolved inconsistent tests caused by race conditions, network latency, and external dependencies, improving CI/CD reliability.
• Test evolution: As features changed, we refined our test suite through feature-driven test creation, continuous test coverage reviews, and optimized test selection.
• Parallel test execution: We reduced test run times by running tests in parallel across multiple agents, speeding up feedback loops.

Major milestones achieved

With foundational improvements in place, we tackled two key milestones to accelerate our CI/CD adoption:

Instant feedback with PR-level tests

We automated tests for every PR, providing developers with immediate feedback on their code changes. This ensured early bug detection and prevented regressions before merging.

"With our streamlined release pipelines, we’ve replaced late-stage manual QA reviews that slow down feature delivery and introduce process bottlenecks with automated tests that are faster and more reliable. This makes us more efficient and allows us to respond to high-priority asks in hours rather than weeks."

Neal Audenaert, Engineering Manager

Seamless integration with merge queues

We ran tests in a merge queue after PRs passed their initial validation. This process ensured smooth integration with the main branch and blocked merges if tests failed, preserving code quality.

These milestones not only enhanced our ability to catch issues early – but also paved the way for more frequent and reliable releases. By automating feedback and ensuring seamless integration, we were able to significantly reduce manual intervention, leading to faster, higher-quality deployments.

“Faster release cycles free up head space for the current task. No more worrying about code I wrote two weeks ago.”

Kerstin Kohout, Senior Software Engineer

What’s next?

As we continue refining our CI/CD pipeline, our future milestones include:

• Enhancing test speed and stability: Transitioning some UI tests to API tests for faster, more reliable validation.
• In-sprint automation: Embedding test automation directly within sprint cycles to ensure continuous validation.
• Test caching: Implementing caching mechanisms to skip tests on unchanged code, reducing test execution time while maintaining coverage.
• Continuous monitoring and feedback: Tracking how new code affects production and resolving issues proactively.

Final thoughts

Achieving CI/CD is an ongoing journey, and our progress so far has reinforced our commitment to quality, efficiency, and continuous improvement. By breaking down challenges into achievable milestones and fostering collaboration across teams, we’ve made meaningful strides toward our goal.

If you want to join us in making homeownership more valuable and accessible, keep an eye on our careers page and follow us on LinkedIn.