Shift Left – CI/CD Scanning & Signing Pipeline

Table of Contents

Overview
Why it matters
What does it solve
Your Mission
- Prerequisites
Failure Simulation
Debrief
Cleanup

Duration: ~25 minutes

Overview

Integrate security scanning and policy checks into pipelines.

Build a Tekton pipeline that starts FROM the workshop golden Python base (python-alpine-golden:0.1 in Quay — already pulled and pushed by setup/lab-environment.sh), then scans, gates, and signs the result and deploys by digest.

Why it matters

Unsigned, unscanned images reach production when security sits after deploy. Pipelines that build, scan, gate, and sign shrink the window where attackers swap in a poisoned tag. Starting FROM an approved golden base — not docker.io/python at build time — is the other half of that story: the cluster only consumes a base you already scanned and parked in Quay.

What does it solve

Silent promotion of unscanned images
Builds that pull a mutable public base instead of your golden image
Deployments that float on mutable tags instead of digests
Missing signature evidence for admission and audit

Your Mission

Build a Tekton path FROM python-alpine-golden that refuses unscanned or unsigned images—then deploy only by digest so a supply-chain swap cannot silently replace what you ran.

Prerequisites

OpenShift Pipelines (Tekton) Operator installed
Cosign installed locally, or simulate signing in the pipeline tasks
Lab setup already ran (source ~/.bashrc or source ~/.acs-roadshow/env): QUAY_URL, QUAY_USER, and the golden image ${QUAY_URL}/$admin/python-alpine-golden:0.1
The python-alpine-golden repository is public in Quay (same Settings path as frontend in acs-00) so the cluster can import it

Click each step only if you need a hint.

Create project 201-04-s-pipeline

Stay in this project for Tasks, Pipeline, and the digest deploy.

oc new-project 201-04-s-pipeline

Now using project "201-04-s-pipeline".

Confirm the golden Python image

This is the approved base from setup — not Docker Hub at pipeline time. Pull proves the tag exists before you import it.

source ~/.acs-roadshow/env 2>/dev/null || source ~/.bashrc
echo "${QUAY_URL}/$admin/python-alpine-golden:0.1"
podman pull "${QUAY_URL}/$admin/python-alpine-golden:0.1"

The pull should complete without unauthorized or manifest unknown. If QUAY_URL is empty, re-run lab setup or source ~/.acs-roadshow/env.

Import python-alpine-golden into this project

Copy the golden image into the project ImageStream so the pipeline FROM line stays on-cluster.

source ~/.acs-roadshow/env 2>/dev/null || source ~/.bashrc
oc import-image python-alpine-golden:0.1 \
  --from="${QUAY_URL}/$admin/python-alpine-golden:0.1" \
  --confirm --insecure -n 201-04-s-pipeline

imagestreamtag.image.openshift.io/python-alpine-golden:0.1 imported. If this fails with unauthorized, make the Quay repository public and retry.

Create ServiceAccount pipeline

Tekton runs as this SA. Do not reuse default.

oc create sa pipeline -n 201-04-s-pipeline

serviceaccount/pipeline created.

Grant anyuid and image-builder to SA pipeline

Buildah often needs anyuid. system:image-builder lets the Task push demo to this project’s internal registry. Treat both as pipeline-only debt, not an app SCC.

oc adm policy add-scc-to-user anyuid -z pipeline -n 201-04-s-pipeline
oc policy add-role-to-user system:image-builder -z pipeline -n 201-04-s-pipeline

Skip the SCC grant if your cluster already lets the pipeline SA build without it.

Apply Task build-image

The Task builds FROM the imported golden ImageStream, adds a non-root Python HTTP server, pushes, and writes an image digest result.

oc apply -n 201-04-s-pipeline -f - <<'EOF'
apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: build-image
spec:
  results:
  - name: image-digest
  params:
  - name: IMAGE
  - name: BASE_IMAGE
    description: Golden Python base already in this project
  steps:
  - name: build
    image: registry.access.redhat.com/ubi9/buildah
    script: |
      #!/usr/bin/env bash
      set -euo pipefail
      cat > Dockerfile <<DOCKER
      FROM $(params.BASE_IMAGE)
      WORKDIR /opt/app
      RUN mkdir -p /opt/app && echo 'shift-left' > /opt/app/index.html && chown -R 1001:0 /opt/app
      USER 1001
      EXPOSE 8080
      CMD ["python", "-m", "http.server", "8080"]
      DOCKER
      buildah bud --tls-verify=false -f Dockerfile -t $(params.IMAGE) .
      buildah push --tls-verify=false $(params.IMAGE)
      DIGEST=$(skopeo inspect --tls-verify=false docker://$(params.IMAGE) | jq -r .Digest)
      echo -n "$DIGEST" > $(results.image-digest.path)
EOF

task.tekton.dev/build-image created.

Apply Task scan-image

The scan is simulated. FAIL=1 in the script is how you will fail the gate later.

oc apply -n 201-04-s-pipeline -f - <<'EOF'
apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: scan-image
spec:
  params:
  - name: IMAGE
  results:
  - name: scan-status
  steps:
  - name: scan
    image: registry.access.redhat.com/ubi9/ubi
    script: |
      #!/usr/bin/env bash
      echo "Simulating vulnerability scan"
      # set FAIL=1 to simulate failure
      if [ "$FAIL" = "1" ]; then echo -n FAIL > $(results.scan-status.path); exit 1; fi
      echo -n PASS > $(results.scan-status.path)
EOF

task.tekton.dev/scan-image created.

Apply Task sign-image

Signing is simulated. Real clusters replace this step with cosign.

oc apply -n 201-04-s-pipeline -f - <<'EOF'
apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: sign-image
spec:
  params:
  - name: IMAGE
  - name: DIGEST
  steps:
  - name: sign
    image: registry.access.redhat.com/ubi9/ubi
    script: |
      #!/usr/bin/env bash
      echo "Simulating cosign sign $(params.IMAGE)@$(params.DIGEST)"
      echo "signature-ok" > /tekton/home/signature.txt
EOF

task.tekton.dev/sign-image created.

Apply Pipeline secure-build

when on the sign task is the gate: scan result must be PASS. BASE_IMAGE is the golden Python ImageStream in this project.

oc apply -n 201-04-s-pipeline -f - <<'EOF'
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: secure-build
spec:
  params:
  - name: IMAGE
  - name: BASE_IMAGE
  tasks:
  - name: build
    taskRef:
      name: build-image
    params:
    - name: IMAGE
      value: $(params.IMAGE)
    - name: BASE_IMAGE
      value: $(params.BASE_IMAGE)
  - name: scan
    runAfter: [build]
    taskRef:
      name: scan-image
    params:
    - name: IMAGE
      value: $(params.IMAGE)
  - name: sign
    runAfter: [scan]
    when:
    - input: "$(tasks.scan.results.scan-status)"
      operator: In
      values: ["PASS"]
    taskRef:
      name: sign-image
    params:
    - name: IMAGE
      value: $(params.IMAGE)
    - name: DIGEST
      value: $(tasks.build.results.image-digest)
EOF

pipeline.tekton.dev/secure-build created.

Apply PipelineRun secure-build-run

Build FROM the imported golden tag. Push the app image to this project’s internal registry.

oc apply -n 201-04-s-pipeline -f - <<'EOF'
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: secure-build-run
spec:
  serviceAccountName: pipeline
  pipelineRef:
    name: secure-build
  params:
  - name: IMAGE
    value: image-registry.openshift-image-registry.svc:5000/201-04-s-pipeline/demo:latest
  - name: BASE_IMAGE
    value: image-registry.openshift-image-registry.svc:5000/201-04-s-pipeline/python-alpine-golden:0.1
EOF

pipelinerun.tekton.dev/secure-build-run created.

Get PipelineRuns

Wait until the run is not still Unknown / Running.

oc get pipelineruns -n 201-04-s-pipeline

secure-build-run should move to True / Succeeded if the scan passed.

Describe PipelineRun secure-build-run

Confirm each task (build, scan, sign) completed.

oc describe pipelinerun secure-build-run -n 201-04-s-pipeline | grep -i status

Look for Succeeded on the pipeline and on the sign task.

Get pipelineResults from secure-build-run

Copy the digest. You will pin the Deployment to it.

oc get pipelinerun secure-build-run -n 201-04-s-pipeline -o jsonpath='{.status.pipelineResults}'

You should see an image-digest value. If empty, check Task results on the build task.

Apply Deployment app pinned to the digest

Replace <DIGEST> with the sha256 from the previous step. Do not deploy :latest.

oc apply -n 201-04-s-pipeline -f - <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
spec:
  replicas: 1
  selector:
    matchLabels: {app: demo}
  template:
    metadata:
      labels: {app: demo}
    spec:
      containers:
      - name: web
        image: image-registry.openshift-image-registry.svc:5000/201-04-s-pipeline/demo@sha256:<DIGEST>
EOF

deployment.apps/app created. Image must contain @sha256:.

Failure Simulation

Re-run with FAIL=1 in the scan Task script and confirm the sign task is skipped.

Debrief

Your pipeline built FROM the Quay golden Python base, enforced scan-before-sign, skipped signing on failure, and deployed by digest so mutable tags cannot sneak into production.

What breaks without this:

Unscanned or unsigned images → known-bad artifacts reach the cluster
Tag-based deploys → non-reproducible, retaggable runtime
FROM python:3.12-alpine (floating tag) at build time → Alpine minor (and Clair matching) drift under you; use the pinned golden digest instead

Tekton (or equivalent) gates, conditional sign tasks, digest pins, and admission that extends trust (signatures/SBOM) at deploy time—see 201-10 for RHACM Policy that configures restricted PSA (and fleet-wide baselines) so tag-based privileged workloads still fail at the API.

the when-clause is the gate; admission and SBOM checks are the natural next layers.

Cleanup

Before moving to the next module, run the lab cleanup script to reset transient resources from this module.

cd ~/ocp5-rhacs-showroom
bash setup/lab-cleanup.sh --module 201-04