The request lifecycle, one layer at a time
Walking a single request from the browser to the database and back, and what each layer is actually responsible for.
A request looks instantaneous from the outside. You click, something happens, the screen updates. Underneath, it crossed six or seven boundaries, and every one of them had a chance to be the reason your p99 looks the way it does.
This is the mental model I keep coming back to when I'm designing or debugging a backend. It's deliberately boring. The value isn't in the diagram, it's in being clear about what each layer owns — and, more importantly, what it refuses to own.
1. The user
Requests start with a focused interface and clear intent.
That sounds obvious, but a surprising number of backend problems are really product problems wearing a costume. If the UI lets someone trigger an expensive operation by accident, or fires the same request three times because the button doesn't disable, no amount of caching downstream will save you.
The first place to reduce load is to not generate it.
2. The API gateway
Routes, validates, and protects the backend surface.
This is your bouncer. It should do a small number of things and do them before anything expensive happens:
- Authenticate and authorize. Reject early, reject cheaply.
- Validate the shape of the request. A malformed payload should never reach your business logic.
- Rate limit. This is where a token bucket lives. There's a live one in the playground if you want to see how capacity and refill rate pull in different directions.
- Route. Nothing more.
The failure mode here is scope creep. Once a gateway starts making business decisions, you have a distributed monolith with extra latency.
3. Queue and workers
Moves expensive work off the request path.
The single most useful question in backend design: does the user need the result of this to continue?
If the answer is no — sending the email, generating the report, re-indexing the
document, calling the third-party API that times out twice a week — it does not belong
in the request. Write an intent to a queue, return a 202, and let a worker deal with it.
What you get:
- The request path stays short and predictable.
- Bursts become backlog instead of timeouts.
- Retries become a property of the system rather than something the user does by refreshing.
What you take on:
- Your work must be idempotent. Queues deliver at-least-once and eventually you will process the same message twice.
- You now have a queue depth to monitor. An unwatched queue is just a slower outage.
4. The database
Stores source-of-truth data with indexing and access discipline.
Most "we need to scale the database" conversations are actually "we never looked at the query plan" conversations. Before sharding anything:
| Check | Why it matters |
|---|---|
| Is the query using an index? | A sequential scan that's fine at 10k rows is an outage at 10M. |
| Are you selecting columns you don't use? | Wide rows cost I/O and network on every single read. |
| Is this N+1? | One query per item in a loop is the most common self-inflicted wound. |
| Is the transaction longer than it needs to be? | Long transactions hold locks and block everyone else. |
Access discipline matters as much as schema design. If six services write to the same table, you don't have a database, you have a shared mutable global variable with a network hop.
5. Response flow
Returns consistent payloads and observability signals.
Two things travel back out, and most teams only think about one.
The payload. Consistent envelopes, consistent error shapes, consistent pagination. A good API contract should feel obvious enough that clients stop reading the docs.
The signals. Every request should leave a trace behind: latency, status, the route, a correlation ID that survives across service boundaries. You cannot fix what you cannot see, and you will not add instrumentation during an incident.
6. Scaling out
Multiple workers and replicas absorb growth cleanly.
Scaling out only works if the layers above were honest about state. Adding a second instance is trivial when your service is stateless and your work is idempotent, and nearly impossible when it's holding in-memory session data and assuming it's the only one running.
So the order matters:
- Make the request path short.
- Move the expensive work off it.
- Make that work idempotent.
- Then add instances.
Doing step four first is how you get a system that's twice as expensive and exactly as slow.
None of this is novel. But having a shared vocabulary for which layer a problem belongs to turns "the app is slow" into a question someone can actually answer.