OpenSSL HollowByte Flaw Can Lock Up Server Memory With 11-Byte TLS Requests

· Original article ↗

Summary

Okta’s Red Team disclosed HollowByte, an OpenSSL TLS denial-of-service flaw that can allocate up to 131 KB per connection from an 11-byte request; tests found persistent glibc heap fragmentation. June fixes cover TLS, but DTLS remains unchanged, and no CVE was issued.

Key points

  • An attacker can trigger an OpenSSL TLS handshake to allocate up to 131 KB before sending the declared message body, then leave the connection waiting.
  • Okta reported that varied request sizes fragmented memory on tested glibc systems, where freed buffers were not returned for reuse; restarting the process cleared the buildup.
  • In Okta’s NGINX tests, a 1 GB server was OOM-killed with 547 MB in fragments; a 16 GB server had 25% of memory locked up. These are Okta’s test results.
  • OpenSSL fixed the TLS issue in releases 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21, dated June 9; earlier releases on those branches are affected.
  • The fix has no CVE or advisory, complicating vulnerability scanning and downstream patch tracking. The article says maintainers may need to verify whether they applied the relevant patch.
  • The fix does not cover DTLS; the article reports that the DTLS handshake path still sizes its buffer using the peer-declared length. Upgrade and restart processes using the old OpenSSL build.

Article Details

Vulnerability Types
  • Denial of service through memory exhaustion
  • Heap fragmentation
Severity
Not classified by OpenSSL; handled as a bug or hardening fix
Affected Versions
  • OpenSSL 4.0.x before 4.0.1
  • OpenSSL 3.6.x before 3.6.3
  • OpenSSL 3.5.x before 3.5.7
  • OpenSSL 3.4.x before 3.4.6
  • OpenSSL 3.0.x before 3.0.21
Exploitation Status
unknown
Exploit Availability
none_reported
Patch Status
partial
Workarounds
  • Upgrade to a listed fixed OpenSSL release and restart processes that loaded the old version. The fix covers TLS; the DTLS path remains unchanged.

MITRE ATT&CK

CVE

People

Products

Related Articles