OpenSSL Patches HollowByte DoS That Can Exhaust Server Memory

Summary
Okta researchers found that an 11-byte payload could trigger excessive memory allocation in OpenSSL. Patches are included in versions 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21.
Key points
- Okta's red team discovered HollowByte, a denial-of-service vulnerability in older OpenSSL versions.
- An 11-byte payload can declare a larger handshake message, prompting OpenSSL to pre-allocate up to 131 KB before receiving the message body.
- Repeated connections with randomized claimed sizes can fragment and retain memory through glibc, potentially exhausting server memory even after the connections close.
- In testing, a 1 GB system became unresponsive after 547 MB of memory was fragmented and frozen; a 16 GB system had 25% of its memory locked.
- Applications and services using OpenSSL may be affected; patches are included in versions 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21.
- The patched library grows buffers as bytes arrive instead of trusting the handshake header; the article says connection limits do not stop the attack in the reported testing.
Article Details
- Vulnerability Types
- Denial-of-service (DoS)
- Memory exhaustion
- Exploitation Status
- unknown
- Exploit Availability
- unknown
- Patch Status
- available
