Logs Don’t Lie – Tracing Who Did What (Audit Forensics)
Duration: ~15 minutes
Overview
Read API audit logs for risky verbs such as exec, port-forward, Secret get, and RBAC changes, then write a short report that names who did what and when.
Find exec and related high-risk API audit events, attribute them to identities, and turn the details into a short incident summary.
Why it matters
When something goes wrong, audit logs show the API calls: user, verb, object, time, source. That is how you reconstruct an incident instead of arguing from memory.
What does it solve
-
Reduces downtime in incident reconstruction
-
Prevents disputes over actions
-
Supports least privilege validation
Your Mission
Someone may already have exec’d into a pod or opened a port-forward tunnel. Your mission: pull the audit trail fast enough to name the actor, command, and timing—before the story becomes guesswork.
Assuming audit.json file (JSON lines).
Click each step only if you need a hint.
Confirm the evidence file is present
ls -l audit.jsonHunt every pod/exec — classic post-compromise foothold
jq 'select(.objectRef.subresource=="exec") | {time:.requestReceivedTimestamp,user:.user.username,ns:.objectRef.namespace,pod:.objectRef.name,cmd:.requestObject.command}' audit.jsonFallback:
grep -F '"exec"' audit.json | headHunt port-forward tunnels — internal services exposed out
jq 'select(.objectRef.subresource=="portforward") | {time:.requestReceivedTimestamp,user:.user.username,ns:.objectRef.namespace,pod:.objectRef.name}' audit.jsonFocus the investigation on a suspected identity
USER=suspect@example.com
jq --arg U "$USER" 'select(.user.username==$U and (.objectRef.subresource=="exec" or .objectRef.subresource=="portforward")) | {time:.requestReceivedTimestamp,sub:.objectRef.subresource,pod:.objectRef.name,ns:.objectRef.namespace,sourceIPs:.sourceIPs}' audit.jsonStage distribution — did stages complete or abort?
jq 'select(.objectRef.subresource=="exec") | .stage' audit.json | sort | uniq -cUnique exec actors — who is pivoting live?
jq 'select(.objectRef.subresource=="exec") | .user.username' audit.json | sort | uniq -cCommands captured — what did they run?
jq 'select(.objectRef.subresource=="exec") | .requestObject.command' audit.json | sort | uniq -cDebrief
You queried audit evidence for pod/exec, port-forward, actors, and captured commands—the forensics baseline when someone pivots inside a namespace.
What breaks without this:
-
No retention or shipping → you cannot answer who/what/when after an incident
-
Ignoring exec/port-forward → classic post-compromise footholds stay invisible
Controls that matter: keep OpenShift audit on, aggregate immutably, correlate with RHACS/runtime signals, and alert on anomalous verbs for sensitive namespaces.
Quick facts: audit shows API intent; it complements (does not replace) runtime detection. Confirm retention and SIEM forwarding before you need them.
