Integer underflow in emsa_pkcs1_v1_5_encode() lets an attacker with a small-modulus RSA key corrupt the entire OP-TEE secure heap via an unbounded memset(), crashing the TEE.
Imagine a bank uses a special locked vault to protect its most sensitive operations — checking account transfers, approving loans, that sort of thing. This vault is supposed to only open with the bank manager's unique handwritten signature. Now imagine someone discovered they could forge that signature by exploiting a flaw in how the bank checks whether the signature is real.
That's essentially what this vulnerability does. OP-TEE is a heavily armored section of computer chips found in smartphones, tablets, and IoT devices. Its whole job is to protect sensitive operations that regular software can't be trusted with — things like unlocking your phone, processing payments, or verifying your identity.
The problem is in how this protected area verifies digital signatures, which are like cryptographic seals proving a piece of software or data actually came from who it claims to. When using smaller encryption keys, attackers can craft fake signatures that this system incorrectly accepts as genuine.
Who should worry? Mostly companies and governments relying on these chips for security-critical work. This includes financial institutions, healthcare providers, and defense contractors. Regular smartphone users are probably safe for now — this requires specific conditions that most consumer devices don't meet.
The good news: there's no evidence anyone has actually exploited this in the wild yet, and patches are available.
What to do: If you work in cybersecurity or operate critical infrastructure, update OP-TEE immediately to version 4.11 or later. If you're responsible for any IoT devices in your organization, contact your vendor about patches. For everyone else, just keep your devices updated — a habit worth maintaining regardless.
Want the full technical analysis? Click "Technical" above.
CVE-2026-33662 is a heap-smashing integer underflow in OP-TEE OS versions 3.8.0 through 4.10. The vulnerable path lives exclusively in the hardware-accelerated RSA signing code path — specifically the EMSA-PKCS1-v1_5 padding encoder. An attacker who can invoke an RSA signing operation with an adversarially-small modulus (easily done from the Normal World via the GlobalPlatform TEE Client API) can cause memset() to be called with a size_t derived from an unsigned integer underflow, effectively 0xFFFFFFFF...FF - N bytes. The memset runs until the secure world faults, producing a denial-of-service against every TA in the system. On platforms without hardware-enforced secure memory bounds, pre-crash overwrite of adjacent heap objects may yield secondary primitives.
CVSS 7.5 (HIGH) — AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H. The attack surface is any OP-TEE deployment that registers a platform-specific RSA acceleration driver (e.g., NXP CAAM, STM32 PKA, Marvell OcteonTX).
Affected Component
File: core/drivers/crypto/crypto_api/acipher/rsassa.c
Function: emsa_pkcs1_v1_5_encode()
Versions: OP-TEE OS 3.8.0 – 4.10 (inclusive)
Prerequisite: Platform must register an RSA acceleration driver via drvcrypt_register_rsa(). Softcrypto fallback is not affected.
Root Cause Analysis
The EMSA-PKCS1-v1_5 encoding format is: 0x00 0x01 [PS] 0x00 [DigestInfo]. The spec mandates the padded message length equals the modulus size in bytes (em_len). The minimum overhead for the fixed fields is:
overhead = 3 bytes (0x00 0x01 0x00)
+ DER AlgorithmIdentifier prefix (varies by digest, e.g. 19 bytes for SHA-256)
+ digest output length (e.g. 32 bytes for SHA-256)
= ~54 bytes for SHA-256
The vulnerable arithmetic computes the padding string size ps_size as:
/*
* emsa_pkcs1_v1_5_encode()
* core/drivers/crypto/crypto_api/acipher/rsassa.c
* OP-TEE OS 3.8.0 – 4.10
*/
static TEE_Result emsa_pkcs1_v1_5_encode(struct rsa_encrypt_state *state,
uint8_t *em, size_t em_len)
{
const struct digest_info *di = state->di; /* AlgorithmIdentifier DER */
size_t di_len = di->der_prefix_len + state->hash_size;
size_t ps_size;
/*
* BUG: pure subtraction — no check that em_len > (di_len + 3).
* If the caller supplies a modulus small enough that em_len <= di_len + 3,
* ps_size wraps around to a huge unsigned value.
*/
ps_size = em_len - di_len - 3; // BUG: unsigned underflow when em_len is too small
/* EM = 0x00 || 0x01 || PS || 0x00 || DigestInfo */
em[0] = 0x00;
em[1] = 0x01;
/*
* memset with underflowed ps_size — effectively memset(em+2, 0xff, SIZE_MAX - N).
* Overwrites the entire secure heap until a page fault terminates the TEE.
*/
memset(em + 2, 0xff, ps_size); // BUG: ps_size is ~SIZE_MAX, unbounded overwrite
em[2 + ps_size] = 0x00;
memcpy(em + 3 + ps_size, di->der_prefix, di->der_prefix_len);
memcpy(em + 3 + ps_size + di->der_prefix_len,
state->hash, state->hash_size);
return TEE_SUCCESS;
}
em_len is derived from the RSA key modulus length in bytes. For SHA-256, di_len is approximately 51 bytes. If an attacker registers or presents a key whose modulus is, say, 32 bytes (256-bit — still a valid RSA modulus structurally), ps_size underflows:
Root cause:emsa_pkcs1_v1_5_encode() computes the PKCS#1 padding length as an unchecked unsigned subtraction, allowing an attacker-controlled small modulus to produce a SIZE_MAX-scale ps_size value that is fed directly to memset(), obliterating the secure heap.
Memory Layout
The encoding buffer em is allocated on the OP-TEE secure heap immediately before the call. Adjacent heap objects are overwritten as the memset marches forward.
SECURE HEAP — BEFORE memset() CALL:
[em buffer, 32 bytes @ tee_mm_heap+0x0000] ← memset starts here +2
[TA session context struct @ tee_mm_heap+0x0020]
[mobj (shared memory mapping) @ tee_mm_heap+0x0080]
[thread_ctx for current thread @ tee_mm_heap+0x0140]
[TA binary mapping metadata @ tee_mm_heap+0x0400]
... (rest of secure heap) ...
[tee_mm_pool metadata @ tee_mm_heap+0x7F00]
AFTER memset(em+2, 0xff, 0xFFFFFFFFFFFFFFEA):
[em[0..1] @ tee_mm_heap+0x0000] = 0x00 0x01 (intact)
[em[2..end of address space] @ tee_mm_heap+0x0002] = 0xFF x ~SIZE_MAX
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ALL secure heap objects overwritten with 0xFF.
thread_ctx->flags, mobj->paddr, session->ref_count — all 0xFFFFFFFF...
Execution continues until MMU fault or watchdog fires.
Exploitation Mechanics
EXPLOIT CHAIN (DoS / potential secondary primitives):
1. Normal World process opens a session to any TA that exposes RSA signing
(TEE_ALG_RSASSA_PKCS1_V1_5_SHA256 or similar).
2. Attacker generates a structurally valid but tiny RSA key:
openssl genrsa -out tiny.pem 512 (or even smaller synthetic key object)
A 256-bit modulus (32 bytes) is sufficient to trigger underflow for SHA-256.
Any modulus where em_len < (hash_len + der_prefix_len + 3 + 8) underflows.
3. Attacker invokes TEE_AsymmetricSignDigest() via TEEC_InvokeCommand(),
passing the adversarial key handle and any 32-byte hash value as input.
4. OP-TEE secure kernel dispatches to the platform RSA driver which calls
emsa_pkcs1_v1_5_encode() to format the padded message.
5. ps_size underflows to ~0xFFFFFFFFFFFFFFEA.
6. memset(em + 2, 0xff, 0xFFFFFFFFFFFFFFEA) executes, overwriting:
- All remaining TA heap objects
- tee_mm pool headers (destroying heap metadata)
- thread stacks for other secure-world threads
- Potentially: crypto key material for other sessions
7. Secure world takes a data abort / permission fault from the MMU
(secure DRAM has a finite mapped region), panicking the TEE.
8. Normal World receives TEEC_ERROR_COMMUNICATION or similar abort.
All subsequent TEE calls fail — full TEE denial of service.
SECONDARY PRIMITIVE HYPOTHESIS:
On platforms where multiple large contiguous secure DRAM regions are
mapped (e.g., some automotive SoCs), an attacker may control heap layout
via preceding allocation requests, placing a sensitive struct (e.g.,
a key object with a function pointer) immediately after em[], then
overwriting it with 0xFF before the fault fires — producing a controlled
call to 0xFFFFFFFFFFFFFFFF. This requires heap grooming and is
platform-specific; no public PoC demonstrates code execution.
Patch Analysis
The fix introduces an explicit bounds check before the subtraction, returning TEE_ERROR_BAD_PARAMETERS if the modulus is too small to encode the requested digest algorithm. This mirrors the requirement stated in RFC 8017 §9.2 that emLen >= tLen + 11.
The patch constant + 8 enforces the RFC 8017 requirement that the padding string be at least 8 bytes long. This both prevents the underflow and rejects keys that are technically too small to produce a valid PKCS#1 v1.5 signature for the chosen digest algorithm.
Detection and Indicators
At runtime (Normal World): Any TEEC_InvokeCommand() returning TEEC_ERROR_COMMUNICATION (0xFFFF000E) during an RSA signing operation on a session that was previously healthy indicates a potential TEE panic. Repeated occurrences without system reboots are suspicious.
Kernel logs:
optee: SMC call returned unexpected value: 0xffffffff
optee: tee-supplicant not responding, SMC error
arm-smccc: secure monitor returned error after tee call
OP-TEE secure console (UART if enabled):
E/TC: 0 tee_abort: Data Abort at 0xXXXXXXXX
E/TC: 0 tee_abort: Unhandled Exception in Supervisor Mode
E/TC: 0 panic at core/kernel/panic.c:XX <tee_abort>
Static analysis: Grep for calls to memset where the size argument is derived from subtraction involving attacker-influenced fields without an intervening range check. The pattern size_t x = a - b - c; memset(..., x); with no guard on a > b + c is the exact anti-pattern here.
Remediation
Update OP-TEE OS to commit post-4.10 patch (GHSA-4cf8-v5g3-73gr). The fix is a one-line bounds check.
If patching is not immediately possible, audit platform drivers calling drvcrypt_register_rsa() and consider temporarily disabling hardware RSA acceleration to fall back to the softcrypto path, which uses a different (unaffected) encoding routine.
Input validation at the TA layer: TAs that accept externally-supplied key material should validate RSA modulus size against the digest algorithm being used before passing to TEE_AsymmetricSignDigest(). Minimum modulus: 1024 bits for SHA-256 in practice; RFC 8017 requires at least tLen + 11 bytes where tLen is the DigestInfo length.
Fuzzing: Add a fuzz target for emsa_pkcs1_v1_5_encode() with boundary key sizes (e.g., 1-byte through 64-byte moduli) to catch similar arithmetic issues in adjacent encode/decode paths.