Secure by Design – Rebuilding Images for OpenShift

Duration: ~25 minutes

Overview

101-01 showed that a public httpd image fails under OpenShift defaults, then swapped in a pre-built UBI image. This module is the rebuild: you own the Dockerfile so the image itself is OpenShift-ready.

Five focus areas, each in its own part: UBI base images, non-root, high ports, multi-stage builds, and container build best practices. Complete Part A through Part E. 101-12 deploys its own app for TLS.

Why it matters

Shipping an image that needs root pushes teams toward anyuid and other SCC exceptions. Rebuild instead: trusted UBI base, non-root, high port, a runtime image that does not ship the toolchain, and a Dockerfile that other people can review. The same image then runs on restricted clusters without special cases.

What does it solve

  • Unknown public bases → hidden CVEs and no support story

  • Root-only services → privilege-escalation path after compromise

  • Privileged port 80 → pressure to run as root

  • Compilers and package managers in the runtime image → extra CVEs and shells

  • Ad-hoc Dockerfiles → every cluster becomes an SCC exception negotiation

Your Mission

The public httpd image just crashed because it wants root and port 80 — the same privilege attackers hope you unlock with anyuid. Rebuild on UBI so the app serves under restricted SCC: non-root, high port, multi-stage, and the rest of the OpenShift-ready build checklist. Prove the UID stays non-root, and expose it without widening the blast radius.

Click each step only if you need a hint.

Create project 101-11-r-rebuild

Stay in this project for every command in this module.

oc new-project 101-11-r-rebuild || oc project 101-11-r-rebuild
oc project should print Now using project "101-11-r-rebuild".
Create the build directory ~/101-11-rebuild

All Dockerfiles in this lab are written here. cd here before every cat > Dockerfile.

mkdir -p ~/101-11-rebuild
cd ~/101-11-rebuild
pwd should end with 101-11-rebuild.
Deploy the public httpd image

This is the image you are about to replace. Admission may accept it; restricted SCC will not let it run as it did on a laptop.

oc create deployment webapp --image=httpd
oc rollout status deployment/webapp --timeout=90s || true
The rollout will time out. That is expected — the pod will not become Available.
Read the httpd failure logs

Look for a bind to port 80 and "Permission denied".

oc logs -l app=webapp --tail=40 --all-containers=true || true
Expected failure resembles the following. 
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.128.2.191. Set the 'ServerName' directive globally to suppress this message
(13)Permission denied: AH00072: make_sock: could not bind to address [::]:80
(13)Permission denied: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs

Why this image is a defender’s problem:

  • Apache httpd on port 80 (privileged; pushes teams toward root)

  • Writes under root-owned paths (fails for random UIDs)

  • Not built for OpenShift restricted SCC

Crash-loop — do not "fix" with anyuid. Rebuild instead. Parts A through E each close one of those gaps.

Part A: UBI base images

A public tag is an unknown builder, an unknown patch level, and often a root-friendly default. Red Hat Universal Base Images (UBI) are redistributable RHEL userspace, scanned and supported, and published on registry.access.redhat.com / registry.redhat.io. Start every Dockerfile from a UBI image that matches what you need: general userspace, minimal runtime, or a language/runtime image that is already OpenShift-ready.

UBI image Use when

ubi9/ubi

Full RHEL userspace (dnf) for tooling-heavy stages

ubi9/ubi-minimal

Smaller runtime; microdnf if you must install packages

ubi9/ubi-micro

Smallest footprint; no package manager in the image

ubi9/httpd-24

Apache already built for non-root on 8080

ubi9/python-311, go-toolset, and similar

Language toolsets (typical builder stage)
Write a Dockerfile FROM ubi9/httpd-24

httpd-24 is the runtime; ubi-minimal will be the builder in Part D. Both come from a registry you already trust (see 101-07).

cd ~/101-11-rebuild
cat > Dockerfile <<'EOF'
# Runtime base: supported UBI Apache, not docker.io/httpd
FROM registry.access.redhat.com/ubi9/httpd-24:latest
EOF
cat Dockerfile
The file should show one FROM line on registry.access.redhat.com. Language images (python-311, go-toolset) are UBI too — use those when you are compiling, not a random golang:latest.

Part B: Non-root

OpenShift restricted SCC injects a random non-root UID and sets runAsNonRoot: true. Images that assume UID 0 fail. A USER instruction documents non-root intent (and helps on clusters that do not randomize UID). On OpenShift the numeric USER is not the runtime UID — the process still gets a project-range UID that is a member of group 0. Writable paths must be group-owned by 0 with group write (chgrp -R 0 and chmod -R g=u), not hard-coded to a single UID.

Read runAsUser from the failing webapp pod

The public httpd image expected UID 0. OpenShift did not grant it.

oc get pod -l app=webapp -o jsonpath='runAsUser={.items[0].spec.containers[0].securityContext.runAsUser}{"\n"}runAsNonRoot={.items[0].spec.securityContext.runAsNonRoot}{"\n"}'
Expect a large non-zero UID (for example 1000690000) and runAsNonRoot=true. That mismatch is the guardrail, not a cluster bug.
Add USER 1001 to the Dockerfile

USER 1001 is the conventional non-root user in UBI application images. Combined with root-group-writable directories, it survives OpenShift’s random UID.

cd ~/101-11-rebuild
cat > Dockerfile <<'EOF'
FROM registry.access.redhat.com/ubi9/httpd-24:latest
USER 1001
EOF
cat Dockerfile
Do not chown application directories to a single UID such as 1001 and stop there. OpenShift will still run as a different UID in group 0. Prefer chgrp -R 0 <dir> && chmod -R g=u <dir> when you create writable paths. ubi9/httpd-24 already does this for /var/www and its log/run dirs.

Part C: High ports

Ports 0-1023 are privileged. Only root can bind them. A non-root process that listens on 80 will fail — that is the log you already saw. Listen on 8080 (or another port >= 1024). The OpenShift Route can still present 80/443 outside the cluster; the container never needs the privileged bind.

Grep the logs for the port 80 bind failure

oc create deployment does not set containerPort. The proof is the log line bind to address 0.0.0.0:80, not a port in the pod spec.

oc logs -l app=webapp --tail=50 --all-containers=true | grep -E 'bind to address|:80|Permission denied' || true
You should see Permission denied and :80.
Add EXPOSE 8080 to the Dockerfile

EXPOSE 8080 documents the contract for humans, oc expose, and image metadata. ubi9/httpd-24 already listens on 8080; declaring it makes the rebuild explicit for any base you use later.

cd ~/101-11-rebuild
cat > Dockerfile <<'EOF'
FROM registry.access.redhat.com/ubi9/httpd-24:latest
USER 1001
EXPOSE 8080
EOF
cat Dockerfile
External users still reach the app on 80/443 through a Route. High ports are the container listen address, not the public URL.

Part D: Multi-stage builds

A single-stage image that dnf install gcc, compiles, and then ships the compiler is a larger CVE surface and a nicer shell for whoever lands in the container. Multi-stage builds use one FROM to build artifacts and a second FROM as the runtime. Only the last stage is the image you deploy; earlier stages are discarded.

Write index.html in ~/101-11-rebuild

This is the site file the builder stage will copy. In a real app this stage would compile, pip install, or npm run build.

cd ~/101-11-rebuild
cat > index.html <<'EOF'
<html><body><h1>Secure UBI httpd App</h1></body></html>
EOF
ls index.html should succeed.
Write a two-stage Dockerfile

The builder copies index.html and emits the file the runtime will serve. The runtime stage is UBI httpd only — no microdnf, no compiler.

cat > Dockerfile <<'EOF'
# Builder: assemble the site. This stage never ships to the cluster.
FROM registry.access.redhat.com/ubi9/ubi-minimal:latest AS builder
USER 0
WORKDIR /src
COPY index.html /src/index.html
RUN mkdir -p /out && cp /src/index.html /out/index.html

# Runtime: UBI httpd only -- non-root, high port, no build toolchain
FROM registry.access.redhat.com/ubi9/httpd-24:latest
COPY --from=builder /out/index.html /var/www/html/index.html
USER 1001
EXPOSE 8080
EOF
cat Dockerfile
Name the first stage (AS builder) so later stages can COPY --from=builder. Anything you RUN in the builder — compilers, npm, go build — stays out of the running container.

Part E: Container build best practices

Parts A-D are the controls that make the image OpenShift-ready. This part is the rest of the checklist, then you build once on the cluster (no laptop root shortcuts) and prove restricted SCC still holds.

Practice Why it matters

UBI (or equivalent maintained) base

Known provenance; see Part A

Non-root USER + group-0-writable dirs

Survives random UID; see Part B

Listen on >= 1024

No privileged bind; see Part C

Multi-stage

Runtime does not ship gcc/npm/pip; see Part D

Do not COPY secrets

.env, keys, and tokens in the image are recoverable from any pull

One process per container

Clearer signals, smaller blast radius

Clean package caches if you RUN a package manager

Smaller image, fewer leftover tools

Labels for name and exposed port

Humans and automation can see the contract

Prefer digest pins in production

101-07; tags move
Write the production Dockerfile with labels

Add labels. Do not copy credentials. Recreate index.html so the build context is complete even if you skipped Part D.

cd ~/101-11-rebuild
cat > index.html <<'EOF'
<html><body><h1>Secure UBI httpd App</h1></body></html>
EOF
cat > Dockerfile <<'EOF'
# Builder: assemble the site. This stage never ships to the cluster.
FROM registry.access.redhat.com/ubi9/ubi-minimal:latest AS builder
USER 0
WORKDIR /src
COPY index.html /src/index.html
RUN mkdir -p /out && cp /src/index.html /out/index.html

# Runtime: UBI httpd only -- non-root, high port, no build toolchain
FROM registry.access.redhat.com/ubi9/httpd-24:latest
COPY --from=builder /out/index.html /var/www/html/index.html

# OpenShift assigns a random UID in group 0. USER documents non-root
# intent; do not assume 1001 is the runtime UID.
USER 1001
EXPOSE 8080

LABEL name="webapp" \
      summary="UBI httpd rebuilt for OpenShift restricted SCC" \
      io.openshift.expose-services="8080:http"

# Do not COPY .env, SSH keys, or cloud credentials into any stage.
EOF
grep LABEL Dockerfile should show name="webapp".
Create a Docker-strategy binary BuildConfig named webapp

Skip creation if the BuildConfig already exists from a retry.

cd ~/101-11-rebuild
oc get bc webapp >/dev/null 2>&1 || oc new-build --name webapp --binary --strategy=docker
oc get bc webapp should list a Docker-strategy build.
Start the binary build from ~/101-11-rebuild

Stream the Dockerfile context into OpenShift. Wait until the build logs finish.

cd ~/101-11-rebuild
oc start-build webapp --from-dir=. --follow
The build should end with Push successful. If it fails with "no such file", cd ~/101-11-rebuild and retry.
Confirm ImageStream webapp has a tag 
oc get is webapp
You should see a latest tag (or a new digest) on ImageStream webapp.
Point deployment/webapp at the ImageStream you built

Replace the public httpd image with the in-cluster ImageStream.

oc set image deployment/webapp *=image-registry.openshift-image-registry.svc:5000/101-11-r-rebuild/webapp:latest
oc rollout status deployment/webapp --timeout=120s
Rollout should complete successfully this time.
Read the new webapp logs 
oc logs -l app=webapp --tail=20 --all-containers=true || true
You should not see a bind failure on port 80.
Prove the process UID is not 0 
oc exec deploy/webapp -- id -u
The number is non-zero (NOT 0).
Confirm the pod SCC is restricted-v2 
oc get pod -l app=webapp -o jsonpath='{.items[0].metadata.annotations.openshift\.io/scc}{"\n"}'
Output is restricted-v2. That SCC denies UID 0, privilege escalation, hostPID/IPC/Network, extra capabilities, and uses default seccomp plus SELinux MCS isolation.
Expose Service and Route on port 8080

The container listens on 8080. The Route is the public door — still no anyuid.

oc expose deployment webapp --port=8080 --target-port=8080
oc expose service webapp
ROUTE=$(oc get route webapp -o jsonpath='{.spec.host}')
curl -s http://$ROUTE | head -3
curl should print the Secure UBI httpd App heading. 101-12 uses its own project (101-12-tls); you can clean this project up when you are done.

Debrief

You rebuilt for restricted SCC across five focus areas: UBI base, non-root, high port, multi-stage runtime, and the remaining Dockerfile hygiene — then deployed without negotiating anyuid.

What breaks without this:

  • USER root / port 80 / fixed ownership → every cluster becomes an SCC exception negotiation

  • Unknown bases → hidden CVEs and drift from what you tested

  • Build tools in the runtime image → extra CVEs and a working compiler after compromise

UBI (or maintained) bases, non-root plus group-0-writable dirs, high ports, multi-stage, no secrets in the image, RHACS root policies, and Dockerfile history in GitOps.
fix the image once across environments; anyuid is long-term debt. Smaller runtime images usually mean fewer CVEs.
giphy

Cleanup

Before moving to the next module, run the lab cleanup script to reset transient resources from this module.

101-12 does not need this project. You can run cleanup now. 
cd ~/ocp5-rhacs-showroom
bash setup/lab-cleanup.sh --module 101-11