Cloud Forensics Guide Walks Through a Simulated Container Compromise Investigation
Summary
A cloud forensics guide outlines key evidence sources and walks through a simulated container-compromise investigation, tracing access from weak WordPress credentials through suspected container escape and lateral movement to inferred S3 data theft.
Key points
- The guide recommends preparing evidence collection in advance and using five questions to investigate access, spread, persistence, impact, and prevention.
- Key sources include cloud-provider audit logs, network flow logs, container-orchestration audit logs, workload snapshots, and runtime events; Auditd and eBPF are cited for Linux runtime monitoring.
- In the simulated scenario, a reverse-shell alert and Apache logs linked activity to a WordPress admin login and upload of a web shell, with weak credentials identified as the initial access issue.
- A suspicious mount in a privileged pod suggested a possible container escape. Kubernetes audit logs showed a blocked pod-creation attempt, while investigation found a privileged container configured to run a backdoor.
- Network-flow and CloudTrail evidence led the authors to infer that the attacker reached a data-fetcher pod and stole client records from S3; the article presents this as a simulated investigation, not a confirmed real-world breach.
- The scenario highlights risks from exposed admin panels, internet-facing privileged containers, and sensitive data stored in cleartext.
- The guide recommends automatic workload-file collection triggers for suspicious events, noting that cloud-provider workload snapshots are not enabled by default.
Article Details
- Defense Focus
- Prepare cloud environments for incident response by collecting and correlating evidence to reconstruct initial access, privilege escalation, persistence, lateral movement, and data impact.
- Detection Methods
- Correlate cloud-provider audit logs, network flow logs, Kubernetes audit logs, workload snapshots, and runtime events across the suspected incident window.
- Review Apache access logs for attacker-IP activity, admin logins, and plugin-upload requests; inspect uploaded plugin files for web shells.
- Investigate runtime alerts for reverse-shell execution and suspicious mount commands, including the executing user and command line.
- Query Kubernetes audit events for pod creation and related API requests by the node identity; check whether the requests succeeded.
- Inspect node containers with `docker ps` and review `/var/lib/docker/containers` for unexpected privileged containers or startup backdoors.
- Use network flow logs to identify suspicious traffic between workloads and the attacker IP, including possible movement to other pods.
- Review CloudTrail and S3 access events for anomalous user agents and unexpected `GetObject` requests; compare event timing and transferred-data volume.
- Data Sources
- Cloud-provider audit logs, including identity, resource, and API activity
- VPC or cloud network flow logs
- Kubernetes and other container-orchestration audit logs
- Workload disk snapshots, including shell history, web-service access logs, system logs, and collected payloads
- Workload runtime events from containers, nodes, and virtual machines
- Apache access logs
- CloudTrail and S3 object-access events
- Rule Types
- Kubernetes audit-log JSON-field filters on user identity, verb, and request URI
- Predefined Auditd rules for tracking suspicious resource events
- Platforms
- Cloud environments
- Kubernetes-orchestrated container workloads
- Linux workloads
- Defensive Actions
- Configure and retain cloud-provider, network-flow, and orchestration audit logs before an incident; enable Kubernetes audit logging where it is not enabled by default.
- Set automatic workload file-collection triggers for suspicious events and preserve snapshots for investigation.
- Collect runtime telemetry with Auditd or eBPF to capture activity that may leave little disk evidence.
- Investigate suspicious container mounts and privileged containers, and restrict container privileges and access to the container runtime socket.
- Strengthen WordPress administrator credentials and avoid exposing the administration panel unnecessarily.
- Avoid running internet-facing services in privileged containers.
- Protect sensitive S3 data and avoid storing it in cleartext.
MITRE ATT&CK
T1059.004 · Unix ShellThe investigation found a reverse shell and command execution on the Linux container.T1078 · Valid AccountsThe simulated attacker logged in to the WordPress admin interface using weak credentials.T1505.003 · Web ShellThe attacker uploaded a WordPress plugin containing a PHP web shell at `/wp-content/plugins/wp-shell/rshell.php`.T1530 · Data from Cloud StorageThe attacker accessed the sensitive S3 bucket and issued a `GetObject` request for client records.T1610 · Deploy ContainerThe attacker created a privileged container on the node that ran a backdoor at startup.T1611 · Escape to HostThe attacker used a suspicious mount command from a privileged pod in an attempt to escape to the host; the article’s investigation concludes the attacker escaped the pod.T1613 · Container and Resource DiscoveryAfter the suspected container escape, the attacker listed pods in the cluster.
Vendors
Products
CloudTrailAWS – CloudTrail CloudWatchIn EKS you need to enable these logs (logs will be streamed to CloudWatch) Containerdthreat actor was able to create a container via the Docker socket. Behind the scenes, Kubernetes uses containerd, which supports the Docker API by exposing `docker.sock`. Attackers that can interact with this socketDockerKubernetes audit logs (taken from AWS CloudWatch)In light of the pod creation attempt, let’s also inspect Docker images and containers to see if the threat actor was able to create a container via the Docker socket.EKSIn EKS you need to enable these logs (logs will be streamed to CloudWatch) Kubernetesare composed of microservices managed by a container orchestration tool. We will focus on Kubernetes as it is the predominant container orchestration tool. Like cloud provider audit logs, containerS3Impact: The `data-fetcher` pod has the `s3-get-object-role` IAM role attached to its service account. The role has all S3 permissions granted for the “super sensitive” S3 bucket. When searching CloudTrail logs or
Tools
auditdThe top technologies used for tracking runtime events in Linux are Auditd and eBPF. eBPFThe top technologies used for tracking runtime events in Linux are Auditd and eBPF. fanotifyOther useful tools are fanotify and inotify. These tools allow you to monitor events on the filesystem such as file creation, file modification, and file opening. inotifyOther useful tools are fanotify and inotify. These tools allow you to monitor events on the filesystem such as file creation, file modification, and file opening.