Why the Text Summarizer Doesn't Work on Korean
Plenty of people have pasted Korean text into a "paste text, get the key sentences back" summarizer and found the results strange. Sentences do get picked, but it's unclear why those particular ones were chosen, and the keyword box is sometimes empty too. This guide opens the actual JavaScript source of the MODOO HUB Text Summarizer and checks, from the code itself rather than a guess, what actually happens with Korean input.
1. Extractive summarization was built on the assumption of "word boundaries"
The method this tool uses isn't generative AI summarization — it's frequency-based extractive summarization. It counts how many times each word appears across the whole text, scores sentences higher the more they contain frequently-occurring words, and then pulls out the highest-scoring sentences verbatim. For this to work at all, the code has to be able to cut precisely where a word begins and ends. English has spaces between words and relatively simple inflection, so a single regex can capture that boundary fairly reliably.
2. Why the same approach struggles to cut words in Korean
Korean is an agglutinative language: particles keep attaching after a noun, and endings keep attaching after a verb or adjective stem. The same word "회사" (company) keeps changing shape depending on the sentence — "회사는", "회사가", "회사를", "회사에서" — and counting these as a single word requires morphological analysis that strips the particles and endings. Splitting purely on whitespace and punctuation the way English does means "회사는" and "회사가" get treated as different words and their frequency gets scattered — or, as the next section shows, if the regex itself was never designed with Hangul in mind, the words disappear from the count entirely.
3. Opening the actual code — the regex inside tokenize()
The tokenize() function in text-summarizer.html is defined like this:
function tokenize(text){return text.toLowerCase().replace(/[^\w\s]/g,' ').split(/\s+/).filter(w=>w.length>1&&!STOP.has(w));}
The key part is replace(/[^\w\s]/g,' '). This regex replaces every character that is neither a word character (\w) nor whitespace (\s) with a space. But in JavaScript's regex engine, \w means [A-Za-z0-9_] — ASCII letters, digits, and underscore only — and does not include Hangul syllables (가-힣). That means feeding in something like "한국어" (Korean) results in every single character being judged as neither \w nor \s, so it gets replaced with a space. The following split(/\s+/) then splits the string on those spaces, so a purely Korean sentence ends this step with not a single recognizable word token left — effectively an empty array.
4. What the actual summary output looks like as a result
Following the summarize() function further, each sentence's score is calculated as words.reduce((acc,w)=>acc+(freqMap[w]||0),0)/Math.max(words.length,1). For a Korean sentence, words (the tokenize result) is an empty array, so the numerator is 0 and the denominator is Math.max(0,1)=1, making the score exactly 0. Even if you switch to position-based mode, that mode only multiplies this 0 score by a first-sentence/last-sentence weight — and 0 times anything is still 0. The upshot is that every sentence ties at a score of 0, and which sentences actually get selected as the "summary" comes down to JavaScript's Array.sort tie-breaking behavior (sort stability) rather than any real judgment about which sentences matter in that text. The keyword tag box appearing empty for Korean input has the same root cause — since no Korean word ever makes it into the frequency count (freqMap), there's nothing to surface as a top keyword.
5. So what should you do with Korean text?
This doesn't mean the tool is "broken." Feed it English text and \w correctly captures letters and digits, so frequency counting and sentence selection work exactly as designed. But for Korean input, you should know that even though a summary is produced, there's no guarantee those are actually the key sentences. If you need to summarize a Korean document, it's safer — in terms of actual result quality — to mark the key sentences yourself paragraph by paragraph, or to use a separate Korean-specific summarization service built on morphological analysis and language understanding.
Frequently Asked Questions
Q. Why do all Korean sentences get the same score in the summarizer?
A. The tokenize() function that counts word frequency replaces non-word characters with spaces using the regex [^\w\s]. But JavaScript's \w only matches ASCII letters, digits, and underscore — it doesn't include Hangul (Korean characters). As a result, feeding it a Korean sentence leaves no recognized word tokens at all, so every sentence's score comes out to exactly 0.
Q. Does switching to position-based mode improve results for Korean?
A. No. Position-based mode just multiplies a sentence's base frequency score by a first-sentence/last-sentence weight. Since the base score for Korean sentences is already 0, multiplying it by any factor still leaves 0.
Q. So is this tool completely unusable for Korean text?
A. A summary output does appear, but because every score ties at 0, which sentences get picked is essentially down to the coincidence of JavaScript's Array.sort stability rather than any real judgment of importance. In other words, getting a result doesn't mean it actually identified the key sentences. If you need to summarize Korean text, you're better off reading and selecting sentences yourself, or using a dedicated language-model-based summarization service.
Q. Does it work correctly on English text?
A. Yes. The \w regex inside tokenize() correctly recognizes ASCII letters and digits as word characters, so for English text, word frequencies are actually tallied, sentences are scored based on that frequency, and the key sentences get selected as intended.