pull request · Shared CI/CD library
Pipeline platform
Mergedplatform/shared-pipeline-librarymain
AuthorJack Devlinplatform engineerLoweconex, a UK IoT platform business
A change to the build pattern used to mean a pull request to twenty repos, so it didn't get made. It now ships once: one Bitbucket pipeline library, imported by every Java and Node service. Tests live in their own repo. Promotion and reporting belong to ArgoCD.
- bitbucket-pipelines.yml × 20~500 lines → a six-line import
- pipeline reporter (bash)−1,071
- orchestrator, five modules+142
The −1,071 is the bash reporter that posted to Teams at every stage. It worked, and it had grown to the point where changing it safely was hard. The +142 is its replacement: an orchestrator in five modules, each tested.
Every service had its own CI
Loweconex runs a UK IoT platform. Behind it are twenty Java and Node services, deployed across four environments: dev, qa, preprod and prod.
Each of those twenty shipped its own bitbucket-pipelines.yml, around five hundred lines apiece. Same rough shape every time — build, test, scan, push, deploy — but no two of them the same, so a change to the build pattern meant a PR to twenty repos.
A 1,071-line bash pipeline reporter lived in the base image and posted to Teams at every stage. It worked, and nobody wanted to touch it.
Tests ran inside the pipeline, before pods were healthy. They were flaky and most failures weren't real.
Jira gates, Veracode and SourceClear were copy-pasted into every yaml.
One library, imported by every service
I split the pipeline into two repos: java-shared-pipeline and node-shared-pipeline. Each exports a set of Bitbucket selectors using Bitbucket's Shared Pipelines Configuration. Service repos import them by tag.
Per-service config is one file. Name, runtime, dockerfile, image repo, build commands. That's all a service author has to know about CI.
Three gates are optional: Veracode SAST, SourceClear SCA, and Jira Fix Version validation. All three live in the same library and are switched on by env var. One library handles the services that need them and the services that don't. The difference is an env var on the import, not a fork of the pipeline.
The library is semver-tagged. Services adopt a new version on their own schedule by bumping the tag. Old tags stay around as long as anyone is still on them.
Tests aren't pipeline steps
I pulled test infra out into its own repo. The pipeline builds and pushes the image, then stops. A separate ArgoCD PostSync hook runs the test job after the deploy is actually healthy, so the tests run against the real running thing rather than half a pod.
Every run produces an Allure report, and pass/fail goes into a result store. I built a dashboard on top of that store. I called it Sentry, which was a mistake given the error-tracking product of the same name, but it's what everyone calls it now. It answers one question: is the fleet green?
It scores the platform separately from the services on it. Cluster, Kafka, databases and secrets sit across the top as Platform Foundation; per-service test health sits below. "The cluster is broken" and "Data Flow has a flaky test" are different conversations, and they usually need different people, so they get scored separately.
ArgoCD took over the rest
The bash reporter is gone. I moved the bits worth keeping into a shared-scripts repo and retired the rest when ArgoCD's Notifications controller took over deploy reporting.
Promotion is decoupled from build. The pipeline emits .ci/out/build.json: commit, image, digest, tags, build url, and stops. ArgoCD Image Updater watches ECR and opens the GitOps bump itself when it sees a new tag.
The pipeline doesn't know which environment its image will land in. Rollback is a revert, and the audit log is git log: I've rolled a release back at 2am with git revert and gone back to sleep.
How it fits together
Four layers, top to bottom. A service repo and the shared libraries it imports. A Bitbucket run that produces an image and a metadata file. Image Updater promotes it into a Kubernetes deploy, and a PostSync hook writes the test result into Sentry.
Pipeline platform · system overview
Select any node for a one-line explanation of what it does.
Nothing selected. Every box below is a real repo, controller or service; pick one and its description appears here.
The three questions it had to answer
A migration like this has to survive twenty people with twenty services to protect, each of whom can veto it by simply not adopting it. These are the three objections the design had to answer.
review · objection
Some of us need the Veracode and Jira gates and some of us don't. Why aren't there two templates?
↳ reply
Because two templates become five. Both kinds use the same shared pipeline tag and the difference is an env var, not a different selector. The library stays a singleton, and the diff between any two services' CI is something you can read in their builds.yaml instead of tracing through forks.
review · objection
The old pipeline deployed. This one stops after the build. Isn't that a step backwards?
↳ reply
It is, deliberately. The pipeline stops at an image and a build.json, and Image Updater watches ECR and opens the GitOps commit itself. The old arrangement had the deploy credentials and the promotion rules sitting inside every service's pipeline, so a service could break its own deploy with a typo in its own yaml, and any change to how promotion worked had to ship as a pipeline release and then be adopted, repo by repo, before it was actually in effect. Promotion is one controller's job now.
review · objection
Why are the tests outside the pipeline? I want the build to go red when they fail.
↳ reply
A pipeline that finishes before the pods are healthy is telling you about the build, not the deploy. PostSync runs the tests against what's actually running, and Sentry surfaces the result whether or not anyone was watching. Most of the failures we'd been calling flaky were tests hitting a pod that had started but wasn't serving yet. Nobody had written down that the tests assumed readiness.
What the diff came to
Onboarding a new service used to mean copying someone else's yaml and editing it until it built. It now takes one builds.yaml and one import line, and the difference between any two services' CI fits on a screen.