BrokenSesame: Alibaba Cloud Registry Write Permissions Enabled Potential Database Supply-Chain Attacks
Summary
Wiz researchers chained container isolation flaws and excessive permissions in two Alibaba Cloud PostgreSQL services, exposing potential cross-tenant access and supply-chain attacks. Alibaba says all issues were mitigated and no customer data was compromised.
Key points
- BrokenSesame affected ApsaraDB RDS for PostgreSQL and AnalyticDB for PostgreSQL; researchers began with database code execution through previously discovered PostgreSQL vulnerabilities.
- In AnalyticDB, a writable library loaded by a root cron job enabled privilege escalation; shared container resources and an exposed Docker socket then enabled escape to the Kubernetes node.
- In ApsaraDB RDS, a symlink-based file disclosure exposed management-container source code. Researchers found command injection in an upgrade check and escaped the privileged container through core_pattern.
- Overprivileged kubelet credentials exposed cluster resources, while databases from other tenants were mounted on the compromised ApsaraDB node. Researchers stopped without accessing other customers' data.
- Registry credentials permitted image pushes, not just pulls, potentially allowing attackers to overwrite images used by both database services and achieve supply-chain code execution.
- Alibaba restricted registry and kubelet permissions, hardened containers, and removed and rotated exposed secrets. All mitigations were deployed by April 12, 2023.
- Alibaba reported no customer data compromise or evidence of exploitation by other parties; customers were not required to take action.
Article Details
- Attack Vectors
- Wiz Research used previously discovered PostgreSQL vulnerabilities to execute code within its own database instances before investigating tenant-isolation weaknesses.
- In AnalyticDB for PostgreSQL, a root cron task loaded a user-writable shared library. Modifying that library's dependencies caused researcher-controlled code to execute as root and create a SUID shell.
- Shared PID namespaces and home-directory mounts allowed lateral movement into an operational container by modifying the SSH client's LocalCommand configuration. An exposed Docker socket then enabled creation of a persistent privileged container with host namespaces and the host filesystem mounted.
- In ApsaraDB RDS for PostgreSQL, replacing a shared configuration file with a symbolic link exploited an operational container's sed processing to disclose its Python source code.
- An unsanitized PostgreSQL superuser role name was incorporated into a root-executed upgrade-check command. Researchers used this command injection to reach a privileged container, then modified core_pattern and triggered a crash to execute a reverse shell on the host.
- Overprivileged kubelet credentials exposed cluster resources and registry credentials. The registry credentials unexpectedly permitted image pushes, creating the potential to overwrite images used by both database services; a malicious supply-chain deployment was not reported.
- Defensive Notes
- Avoid unnecessary sharing of PID namespaces and filesystem mounts between database containers and operational containers.
- Do not rely solely on ordinary Linux containers to isolate mutually untrusted tenants. The article recommends considering Gvisor or Kata containers; Alibaba Cloud implemented its internal container-hardening solution.
- Apply least privilege to kubelet identities. Alibaba Cloud reduced kubelet permissions to better isolate nodes from other cluster resources.
- Scope image-pull credentials to pull operations only. Alibaba Cloud removed registry push permissions from the affected credentials.
- Remove build-time secrets from node images and logs, rotate exposed credentials, and restrict their permissions. Alibaba Cloud reported taking these measures.
- Alibaba Cloud confirmed full mitigation and stated that no customer data was compromised and no customer action was required. Its investigation found no evidence of exploitation by other parties; relevant logged activity was attributed to the researchers.
MITRE ATT&CK
T1053.003 · CronAn existing cron task executed /usr/bin/tsar as root every minute, providing the execution trigger for the modified shared library.T1057 · Process DiscoveryA Python script enumerated processes and their command lines to detect an SCP process running in a neighboring container sharing the PID namespace.T1059.004 · Unix ShellResearchers injected shell syntax through a PostgreSQL role name and configured core_pattern to execute a bash reverse shell on the host.T1083 · File and Directory DiscoveryResearchers inspected the neighboring container through /proc/{pid}/root/ and examined host filesystem contents after mounting the host root directory.T1548.001 · Setuid and SetgidThe injected library copied /bin/bash to /bin/dash and made it SUID; researchers subsequently used the SUID binary to execute as root in a neighboring container.T1552.001 · Credentials In FilesResearchers retrieved registry credentials from docker-image-secret and found additional access keys in node configuration and log files.T1552.003 · Shell HistoryA secret scan found an access key in the node's /root/.bash_history file.T1574 · Hijack Execution FlowResearchers altered the dependencies of a user-writable libgcc_s.so.1 so that a root-executed tsar process loaded their shared library.T1610 · Deploy ContainerResearchers used an exposed Docker Unix socket to create a persistent privileged container sharing host namespaces and mounting the host root filesystem at /mnt.T1611 · Escape to HostResearchers escaped containers through a privileged Docker-created container in AnalyticDB and through writable host-shared core_pattern settings in ApsaraDB RDS.T1613 · Container and Resource DiscoveryUsing node kubelet credentials, researchers enumerated Kubernetes pods and other cluster resources, identifying pods belonging to other tenants.
People
Hillai Ben-SassonNamed member of the Wiz Research Team presenting the cloud tenant-isolation research.Nir OhfeldNamed member of the Wiz Research Team presenting the cloud tenant-isolation research.Ronen ShustinNamed member of the Wiz Research Team presenting the cloud tenant-isolation research.Sagi TzadikNamed member of the Wiz Research Team presenting the cloud tenant-isolation research.Shir TamariNamed member of the Wiz Research Team presenting the cloud tenant-isolation research.
Vendors
Products
AnalyticDB for PostgreSQLvulnerabilities in two of Alibaba Cloud’s popular services, ApsaraDB RDS for PostgreSQL and AnalyticDB for PostgreSQL. Dubbed #BrokenSesame, the vulnerabilities potentially allowed unauthorized access to AlibabaApsaraDB RDS for PostgreSQLhas discovered a chain of critical vulnerabilities in two of Alibaba Cloud’s popular services, ApsaraDB RDS for PostgreSQL and AnalyticDB for PostgreSQL. Dubbed #BrokenSesame, the vulnerabilities potentially allowedDockercapabilities of the second container, we realized that it was privileged. Additionally, we found that the Docker Unix socket (/run/docker.sock) was accessible from the container, a known element used inKubernetesMultitenancy in KubernetesPostgreSQLa chain of critical vulnerabilities in two of Alibaba Cloud’s popular services, ApsaraDB RDS for PostgreSQL and AnalyticDB for PostgreSQL. Dubbed #BrokenSesame, the vulnerabilities potentially allowed