---
title: Memory in Node.js
summary: How V8 spends RAM, why short-lived garbage still costs you, and how to build caches that cannot grow forever on a small server.
topic: performance
order: 1
updated: 2026-09-19
---

## What it is

A Node.js process keeps its JavaScript objects in the **V8 heap**, which is split in two:

- The **young generation**: small and fast. Almost every new object starts here and most die
  here within milliseconds.
- The **old generation**: large. Objects that survive a few collections are moved here, and
  collecting it is slower.

Memory outside the heap, mostly `Buffer` data, is counted as **external**. The garbage
collector does not have to walk through it.

The collector pauses your code while it works. So memory is not only "how much RAM", it is
also "how often, and how long, everything stops".

## Why it is a rule

- **Discord, 2020.** One of Discord's busiest services, written in Go, showed a latency spike
  every two minutes, like clockwork. The cause was the garbage collector, which was forced to
  run at least every two minutes and scan a large in-memory cache each time. They rewrote the
  service in a language without a garbage collector, and the spikes went away.

The same shape happens in Node: a big, long-lived structure plus a lot of short-lived garbage
turns into regular pauses that every user feels.

## How to do it

### Set the ceiling on purpose

```sh
node --max-old-space-size=256 server.js
```

On a small container, give V8 a limit below the container's memory. Without it, V8 may grow
past what the box has and the process is killed with no warning.

### Do not redo work whose inputs have not changed

A page built from files on disk and constants is the same for every request until the next
deploy. Build it **once** and reuse the result.

```js
const cache = new Map()
async function page(key, build) {
  let buf = cache.get(key)
  if (!buf) cache.set(key, (buf = Buffer.from(await build(), 'utf-8')))
  return buf
}
```

Rebuilding it per request reads the file, copies the string several times and encodes it
again, which for a large page is over a megabyte of garbage per view.

### Store bytes, not strings

A cache of `Buffer`s lives in external memory, so the garbage collector does not trace it on
every major collection. A cache of large strings sits in the old generation and makes every
pause longer. Hand the same `Buffer` to every request, and never modify it.

### Bound every cache, by bytes

A cache counted in entries is not bounded if the entries can be any size. Cap the total bytes
and the bytes per entry, and evict the least recently used entry when full.

```js
if (buf.length > MAX_ENTRY) return stream(file)
while (total + buf.length > MAX_TOTAL) evictOldest()
```

Anything too large for one entry is **streamed** from disk instead of held.

### Look for the usual leaks

- Maps and arrays that are only ever added to.
- Event listeners and timers that are never removed.
- Closures that keep a whole request alive.
- Loading a whole table into memory where paging or streaming would do.

### Measure

`process.memoryUsage()` reports `rss`, `heapUsed` and `external`. Log them at boot and on an
interval. A number that climbs and never falls after a collection is a leak.

## How we do it here

The production process runs with an old-space ceiling set well below its container's memory. Pages are composed once per
deploy and cached as `Buffer`s in a byte-bounded LRU, and anything too big is streamed. A
memory watcher logs heap and external memory and flags steady growth. A change that raises
steady-state heap is treated as a regression, even when it is faster.

## Benefits

- Fewer and shorter garbage collection pauses, so smoother response times.
- Predictable memory on cheap, small containers.
- Reused work frees CPU for the requests that actually need it.

## Disadvantages

- Every cache trades memory for CPU, and a cache is a place stale or wrong data can hide.
- A shared `Buffer` handed to every request is a trap if anyone ever modifies it.
- A tight heap ceiling crashes the process sooner when a real leak appears. That is a feature
  in production, but it is still a crash.

## Checklist

- `--max-old-space-size` set below the container limit.
- Work whose inputs cannot change is done once.
- Every cache is bounded by bytes, with LRU eviction.
- Memory is logged and watched for steady growth.

## Sources

- [V8 — Trash talk: the Orinoco garbage collector](https://v8.dev/blog/trash-talk)
- [Node.js — process.memoryUsage()](https://nodejs.org/api/process.html#processmemoryusage)
- [Discord — Why Discord is switching from Go to Rust](https://discord.com/blog/why-discord-is-switching-from-go-to-rust)