Learn / Performance / Lesson 16
Measuring performance
A change is faster only when a measurement says so, beyond the noise. How to run an optimisation loop that keeps real wins and never trades away correctness or security.
Last updated: 2026-09-19
What it is
Performance work has two kinds of numbers:
- Objectives, which you try to push down: response time, CPU, RAM, allocations, query count.
- Constraints, which must never move: correct output, security, validation, access control, data integrity.
An objective can be traded for another objective. A constraint is never traded for anything.
Why it is a rule
Speed changes the world outside the server too.
- Google's Core Web Vitals judge a page as good only when the main content appears within 2.5 seconds (LCP), the page reacts to input within 200 milliseconds (INP), and the layout barely jumps (CLS below 0.1). They feed into search ranking.
- Many "optimisations" in the wild have quietly dropped a permission check or widened a cache key. The page got faster and started showing one customer's data to another. That is why security is written down as a constraint, not left to good intentions.
How to do it
The loop
inspect → find the bottleneck → measure a baseline → form a hypothesis
→ make ONE change → run the tests → check security
→ benchmark → compare with the best known → keep or revert → repeatOne change at a time
Two changes measured together cannot be told apart, and a regression hides behind a win.
Know your noise
Run the same benchmark several times without changing anything. The spread you see is the noise. A "win" smaller than the noise is not a win. Keep the change only when it beats the best known result by more than that.
Look at the tail, not only the average
The average hides the slow requests people actually complain about. Report the median (p50), p95 and p99. A change that improves p50 but worsens p99 made things worse for your unluckiest users.
Measure memory with speed
Record steady and peak RAM with every run. A faster change that raises steady-state memory toward the ceiling is a regression on a small server.
Work from the top down
- Algorithms: O(n²) to O(n log n), repeated passes to one.
- Database: N+1 queries, repeated identical queries, missing indexes, columns nobody reads.
- I/O: repeated file reads, network calls, parsing.
- Computation repeated with the same inputs.
- Memory: copies, allocations, object lifetimes.
- Micro-optimisations: last, and only with a measured win.
Keep a log
Every attempt, kept or reverted, with its numbers. The reverted ones stop the next person trying the same idea.
How we do it here
Optimisation passes follow this loop exactly: one change per iteration, the full test suite, a security read of the diff, then a benchmark compared against the best known result, beyond a measured noise threshold. Benchmark results are machine-specific and never committed.
Benefits
- Only real improvements survive, and every one has a number behind it.
- Security and correctness cannot be traded away by accident.
- The log turns guesses into a record the whole team can build on.
Disadvantages
- It is slow. One change per iteration, fully measured, takes discipline and time.
- Benchmarks on a laptop do not match production; they show direction, not exact numbers.
- Chasing the tail (p99) often needs deeper changes than chasing the average.
Checklist
- A baseline exists before the change.
- Exactly one change was measured.
- The win is bigger than the noise.
- Tests and security checks pass.
- RAM did not get meaningfully worse.