Learn / Performance / Lesson 13

Memory in Node.js

How V8 spends RAM, why short-lived garbage still costs you, and how to build caches that cannot grow forever on a small server.

Last updated: 2026-09-19

What it is

A Node.js process keeps its JavaScript objects in the V8 heap, which is split in two:

The young generation: small and fast. Almost every new object starts here and most die here within milliseconds.
The old generation: large. Objects that survive a few collections are moved here, and collecting it is slower.

Memory outside the heap, mostly Buffer data, is counted as external. The garbage collector does not have to walk through it.

The collector pauses your code while it works. So memory is not only "how much RAM", it is also "how often, and how long, everything stops".

Why it is a rule

Discord, 2020. One of Discord's busiest services, written in Go, showed a latency spike every two minutes, like clockwork. The cause was the garbage collector, which was forced to run at least every two minutes and scan a large in-memory cache each time. They rewrote the service in a language without a garbage collector, and the spikes went away.

The same shape happens in Node: a big, long-lived structure plus a lot of short-lived garbage turns into regular pauses that every user feels.

How to do it

Set the ceiling on purpose

node --max-old-space-size=256 server.js

On a small container, give V8 a limit below the container's memory. Without it, V8 may grow past what the box has and the process is killed with no warning.

Do not redo work whose inputs have not changed

A page built from files on disk and constants is the same for every request until the next deploy. Build it once and reuse the result.

const cache = new Map()
async function page(key, build) {
  let buf = cache.get(key)
  if (!buf) cache.set(key, (buf = Buffer.from(await build(), 'utf-8')))
  return buf
}

Rebuilding it per request reads the file, copies the string several times and encodes it again, which for a large page is over a megabyte of garbage per view.

Store bytes, not strings

A cache of Buffers lives in external memory, so the garbage collector does not trace it on every major collection. A cache of large strings sits in the old generation and makes every pause longer. Hand the same Buffer to every request, and never modify it.

Bound every cache, by bytes

A cache counted in entries is not bounded if the entries can be any size. Cap the total bytes and the bytes per entry, and evict the least recently used entry when full.

if (buf.length > MAX_ENTRY) return stream(file)
while (total + buf.length > MAX_TOTAL) evictOldest()

Anything too large for one entry is streamed from disk instead of held.

Look for the usual leaks

Maps and arrays that are only ever added to.
Event listeners and timers that are never removed.
Closures that keep a whole request alive.
Loading a whole table into memory where paging or streaming would do.

Measure

process.memoryUsage() reports rss, heapUsed and external. Log them at boot and on an interval. A number that climbs and never falls after a collection is a leak.

How we do it here

The production process runs with an old-space ceiling set well below its container's memory. Pages are composed once per deploy and cached as Buffers in a byte-bounded LRU, and anything too big is streamed. A memory watcher logs heap and external memory and flags steady growth. A change that raises steady-state heap is treated as a regression, even when it is faster.

Benefits

Fewer and shorter garbage collection pauses, so smoother response times.
Predictable memory on cheap, small containers.
Reused work frees CPU for the requests that actually need it.

Disadvantages

Every cache trades memory for CPU, and a cache is a place stale or wrong data can hide.
A shared Buffer handed to every request is a trap if anyone ever modifies it.
A tight heap ceiling crashes the process sooner when a real leak appears. That is a feature in production, but it is still a crash.

Checklist

--max-old-space-size set below the container limit.
Work whose inputs cannot change is done once.
Every cache is bounded by bytes, with LRU eviction.
Memory is logged and watched for steady growth.

Sources

Read this lesson as Markdown