Building only changed services in a monorepo pipeline is surprisingly simple once you understand how Tekton’s PipelineRun can be parameterized and how git commands can be leveraged.
Let’s see this in action. Imagine a monorepo with two services, service-a and service-b. We want a Tekton PipelineRun that only builds service-a if its code has changed since the last successful build.
Here’s a simplified Pipeline definition:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: monorepo-build-changed
spec:
params:
- name: git-url
description: The Git repository URL
type: string
- name: revision
description: The Git revision to build
type: string
- name: changed-services
description: A comma-separated list of services that have changed
type: string
tasks:
- name: detect-changes
taskRef:
name: git-detect-changes
params:
- name: url
value: $(params.git-url)
- name: revision
value: $(params.revision)
- name: base-revision
value: "main" # Or a previous successful commit hash
# This task will output a list of changed services
# We'll use this in subsequent tasks
- name: build-service-a
taskRef:
name: build-service
params:
- name: service-name
value: "service-a"
- name: build-command
value: "make build-a" # Example build command
runIf: "$(tasks.detect-changes.results.changed-services contains 'service-a')"
# This task will only run if 'service-a' is in the changed-services list
- name: build-service-b
taskRef:
name: build-service
params:
- name: service-name
value: "service-b"
- name: build-command
value: "make build-b" # Example build command
runIf: "$(tasks.detect-changes.results.changed-services contains 'service-b')"
# This task will only run if 'service-b' is in the changed-services list
And here are the Tasks:
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: git-detect-changes
spec:
params:
- name: url
type: string
- name: revision
type: string
- name: base-revision
type: string
results:
- name: changed-services
description: Comma-separated list of changed services
steps:
- name: detect
image: alpine/git
script: |
#!/bin/sh
mkdir -p /workspace/source
git clone $(params.url) /workspace/source
cd /workspace/source
# Fetch the base revision (e.g., main branch or a specific commit)
git fetch origin $(params.base-revision)
git fetch origin $(params.revision)
# Get the list of changed files between the current revision and the base revision
# We're looking for files that start with a service name followed by a slash
# For example, 'service-a/main.go' or 'service-b/Dockerfile'
changed_files=$(git diff --name-only $(params.base-revision)..$(params.revision) -- '*/')
# Extract unique service names from the changed files
# This assumes a directory structure like 'service-name/...'
changed_services=$(echo "$changed_files" | sed -E 's/^([^/]+)\/.*/\1/' | sort -u | paste -sd,)
# If no files changed, or no services identified, output an empty string
if [ -z "$changed_services" ]; then
echo "" > $(results.changed-services.path)
else
echo "$changed_services" > $(results.changed-services.path)
fi
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: build-service
spec:
params:
- name: service-name
type: string
- name: build-command
type: string
steps:
- name: build
image: ubuntu:latest # Replace with your actual build image
script: |
#!/bin/bash
echo "Building $(params.service-name)..."
# In a real scenario, you'd checkout the code here,
# navigate to the service directory, and execute the build command.
# For simplicity, we just echo the command.
echo "Executing: $(params.build-command)"
# Example:
# cd /workspace/source/$(params.service-name)
# $(params.build-command)
echo "$(params.service-name) built successfully."
The core idea is to use Tekton’s runIf condition. This condition allows a Task to execute only if a certain expression evaluates to true. In our case, we’re checking if the output from the git-detect-changes task (which is a comma-separated string of changed services) contains the name of the service we intend to build.
The git-detect-changes task itself is crucial. It clones the repository, fetches both the current revision and a base revision (often main or a previous successful commit hash), and then uses git diff --name-only to list all files that have changed between these two points. The sed and sort -u commands then extract the unique top-level directory names, assuming your services are organized in such a way (e.g., service-a/, service-b/). The output is a comma-separated string like "service-a,service-b".
When you create a PipelineRun, you’ll pass the git-url, revision, and crucially, you can either determine the changed-services dynamically (e.g., in a CI trigger) or let the git-detect-changes task figure it out. If you let the task figure it out, you’d omit the changed-services parameter from the PipelineRun and adjust the runIf conditions to directly reference the result of the git-detect-changes task.
For example, if service-a changed, the git-detect-changes task would output "service-a". The build-service-a task’s runIf condition $(tasks.detect-changes.results.changed-services contains 'service-a') would evaluate to true, and the task would run. For build-service-b, the condition $(tasks.detect-changes.results.changed-services contains 'service-b') would evaluate to false, and the task would be skipped.
The runIf condition is Tekton’s way of implementing conditional execution. It uses CEL (Common Expression Language) for evaluation. The contains operator is perfect for checking if a substring exists within a larger string.
The git diff --name-only $(params.base-revision)..$(params.revision) command is the linchpin for detecting changes. By comparing the current commit ($(params.revision)) against a known good state ($(params.base-revision)), you get a precise list of modified files. The subsequent shell scripting then translates this file list into a list of affected services.
The most surprising aspect here is how a simple git diff and string manipulation, combined with Tekton’s runIf condition, can dramatically optimize your CI/CD pipeline by avoiding unnecessary builds.
The next logical step is to integrate this with a GitOps workflow, where changes to specific service directories trigger this pipeline.