The Regex Character Class Trap: Why Only Half an Emoji Gets Removed

Guide · Last verified Aug 27, 2026

Tools that "remove specific characters" almost always use a regex character class [...] internally. But when that character class is built without the u (unicode) flag, characters represented by two UTF-16 code units — like most emoji — can produce unexpected results. We ran the actual code behind the Special Character Remover in Node.js to verify exactly what form this problem takes.

1. What is a surrogate pair

Unicode defines over a million characters, from U+0000 to U+10FFFF, but UTF-16 — the internal encoding of JavaScript strings — can only directly represent U+0000 through U+FFFF with a single code unit (16 bits). This range is called the Basic Multilingual Plane (BMP), and most emoji (U+1F300 and above) fall outside it. Characters beyond the BMP are represented using a "surrogate pair": a high surrogate and a low surrogate code unit joined together to form one character. For example, 😀 (U+1F600) is internally two code units in sequence: \ud83d followed by \ude00.

2. How a regex without the u flag handles surrogates

The problem is that when the JS regex engine runs without the u flag, it reads a string as a sequence of UTF-16 code units, not Unicode code points. That means building a character class [😀] without the u flag doesn't make it match "the single character 😀" — instead it's interpreted as two separate elements meaning "either the code unit \ud83d or the code unit \ude00." Looking at the actual source of the Special Character Remover (remove-special-characters.html), this is exactly how it builds its character class.

3. What actually happens when you run it

We reproduced this tool's regex directly in Node.js and tested two cases. First, the user enters the emoji they want removed, 😀 (U+1F600), in the extra-removal field, and the same 😀 also appears in the body text.

Here's the interesting part: the emoji you actually meant to remove happens to get removed "completely." Since both its high and low surrogate code units are individual elements in the class, replace() matches twice and the whole thing disappears. The real problem shows up elsewhere. If the body text contains a different emoji that shares the same high surrogate (\ud83d) — which covers the U+1F300–U+1F5FF and U+1F600–U+1F64F ranges, meaning a huge number of common emoji like 😁, 😂, 🙂, and more — only that emoji's high surrogate gets individually matched and removed, leaving its low surrogate stranded alone. Since a lone, invalid surrogate can't be rendered, the browser displays a replacement character (□, U+FFFD, etc.) instead. In short: the emoji you removed disappears cleanly, but other emoji you never intended to touch get corrupted.

4. Why adding the u flag fixes it

A regex with the u flag interprets a string as a sequence of Unicode code points, not code units. With the same [😀] character class but the u flag added, it only matches "the entire surrogate pair as a single code point, 😀" — it no longer treats the high and low surrogates as separate elements. So even if a different emoji happens to share half of that surrogate pair, it can no longer be mismatched. If you need Unicode-aware character handling, an alternative is iterating by code point using codePointAt() or the spread operator ([...text]), as the Unicode Converter does.

5. How to use this safely in practice

• Instead of specifying a single emoji in the extra-removal field, use the predefined categories offered as checkboxes (punctuation, symbols, etc.) first. That path never touches characters with a surrogate-pair issue.
• If you want to strip out emoji entirely, a purpose-built tool like the Emoji Remover is safer than the special character remover.
• If you're writing your own regex, the surest habit is to add the u flag as soon as a non-ASCII character enters a character class.

Frequently Asked Questions