Notes on a Three-Day Memory Leak

A background worker started restarting every six hours or so under load. Nothing crashed loudly — memory just climbed until the process manager killed it and started a fresh one. Easy to ignore, which is exactly why it took three days.

The wrong assumption

I assumed the leak was in the queue library, because that’s where all the “long-lived object” folklore points. I spent a day adding instrumentation around job handlers, watching queue depth, checking for unresolved promises. Nothing.

The actual leak was in a logging wrapper that cached a formatter per request ID, and never evicted anything:

const formatters = new Map();

function getFormatter(requestId) {
  if (!formatters.has(requestId)) {
    formatters.set(requestId, buildFormatter(requestId));
  }
  return formatters.get(requestId);
}

Every request ID is unique. Every entry stays forever. A cache with no eviction policy isn’t a cache — it’s just a leak with a friendlier name.

What actually found it

Heap snapshots, taken an hour apart under load, compared with --diff. The retained-size delta pointed straight at formatters, which I’d walked past in code review twice because it “looked like a cache.”

The fix was a five-line LRU wrapper. The lesson took longer to learn: when memory grows monotonically, stop guessing and diff two heap snapshots before you touch any code.