The cover illustrates a fictional store; it does not show a deployed system or real test results.
A customer waits five seconds on the order page while the CPU graph still has room. If we start by adding servers, we may pay for more machines that are still waiting on the same database. I would first identify what the customer is waiting for and what “handling many users” means for a particular kind of work at a particular time.
This series uses Ready Parts, a fictional parts store. It has public product pages, customer-specific prices, quote requests, and order confirmation. Every number in the series is a decision-making example, not a real store test or a provider guarantee. A system can start on one machine; an intention to grow does not itself require Kubernetes.
Registered users, people online, and requests are different counts
The cumulative number of accounts tells us the size of the customer base, not the load this afternoon. Someone may leave a page open without sending any work, while another person searches repeatedly and generates several requests each second. One page can also fetch several images, scripts, and API responses. We cannot convert a user count directly into a machine count.
An API lets programs send requests to one another. A CDN is a content delivery network that may answer some requests before they reach the origin. Requests per second (RPS) measures incoming request rate, but we must say whether it is counted at the CDN, origin, or a particular API. Throughput measures completed work. If incoming work continually exceeds completions, it accumulates or gets rejected. Showing only arrivals can make a system with a growing queue look healthy.
Concurrency is the number of jobs inside the same measured boundary at once. It may include active and waiting jobs, depending on where we measure. Always state that boundary. It is not a substitute for registered accounts or people leaving a page open.
RPS measures incoming requests
Concurrency includes running and waiting work within its boundary
Throughput must define what counts as complete
A short equation for asking questions, not guaranteeing system size
Little’s Law relates average work in a system, average throughput, and average time in the system when their boundaries and measurement periods align. It is written L = λW, with L as average jobs in the system, λ as average rate, and W as average time. Do not substitute p95 for a mean and call the result maximum capacity.
Suppose an API receives and completes an average of 100 requests per second, each spending an average of 0.25 second inside it, with no continually growing backlog. The average number of requests in that boundary is 25. If average time rises to one second at the same rate, the average rises to 100 requests. The system needs more room for work in progress even though the customer count is unchanged.
This is a hypothetical calculation, not a measurement or a reason to set exactly 25 workers. Real work varies, arrives in bursts, hits connection limits, and may not be in a steady state. Use the relationship to check whether measurements make sense, then test usable capacity with room for variation.
Define “done” differently for a product page and an order
Someone viewing a part needs to see the information and be able to continue. A completed order means the system has recorded it correctly under the agreed conditions. An HTTP 200 response with an incomplete record or duplicate order should not count as a business success. Sales and operations need to agree on that boundary before building charts.
An SLI is a service-level indicator while an SLO is an agreed target. One example is the share of eligible orders confirmed correctly within the agreed time over a specified window. The target should reflect customer needs and what the team can control. Do not choose 99.99% just because it looks serious before understanding the cost.
| Task | What the customer needs | What the team should collect |
|---|---|---|
| View a product | See the main content and be able to continue | Real experience by page, device, and network |
| Search | Get authorized results within the agreed time | Search rate, response time, and errors |
| Confirm an order | Correct, nonduplicate record and a clear result | Actually completed orders, pending work, and reconciliation |
| Send a document | Receive it within the background-job deadline | Time from receipt to successful delivery and oldest pending job age |
An average can hide the people who wait longest
If most requests are fast but a few are very slow, a single average can hide the problem. Percentiles describe the distribution: p95 is the value at or below which roughly 95% of observations in the measured set fall. It does not promise every customer that time. Averaging percentiles from several machines also does not automatically produce the correct combined percentile.
Read percentiles alongside request counts, errors, and the measurement boundary. If slow requests time out and disappear from the timing sample, the graph for the remaining requests may improve while more customers fail. Count successes, failures, and throttled work; do not exclude unwanted outcomes.
For web pages, Core Web Vitals help measure real experience around main-content rendering, interaction responsiveness, and visual stability. Use real user monitoring (RUM) alongside lab tests. A single Lighthouse run on a powerful machine does not show that customers’ phones in every location feel fast. Check current definitions and assessment methods in web.dev.
The web.dev documentation consulted for this series defined “good” as LCP, main-content rendering, at most 2.5 seconds; INP, interaction responsiveness, at most 200 milliseconds; and CLS, unexpected layout movement, at most 0.1. These are assessed at p75 separately for mobile and desktop. They are metric thresholds, not measurements of the fictional store or an agreed order-success target.
Time to First Byte (TTFB) helps diagnose response delivery, but it is not another Core Web Vital. Total Blocking Time (TBT) is a Lighthouse diagnostic, not INP measured from real user interactions. Know what each tool reports before comparing unlike numbers or using them to accept work.
Trace time from the screen to the waiting job
Start with journeys a business person can describe: open a product, search a part number, add an item, and confirm an order. Then separate DNS, connection, Time to First Byte, download, JavaScript work, and server time. If using traces, pass a trace identifier safely across services to see where one request waits on a database or external service.
Logs should not contain passwords, tokens, or unnecessary customer data. Keep order references separate from sensitive details and set access rights and retention periods. Observability has its own cost, both paid log volume and the risk of collecting too much data. We will return to that in the per-order cost in Episode 7.
Download and browser work
Processing and waiting
Queries, partner services, and queues
Test realistic arrivals without letting the load generator hide a queue
A simulation in which each user waits for a response before sending the next job is useful for some questions. But as the site slows, the arrival rate may also fall. If we need to test work arriving at an external rate despite the slowdown, use a matching arrival model and verify that the load generator reaches its target. A generator that saturates first can mislead us about system capacity in either direction.
Mix the right work too. Cacheable product pages and order writes have different costs. Use realistically sized data and test accounts isolated from real customers. Test a cold cache, bursts, a run long enough to reveal resource leaks, and component failures within authorized bounds. Do not load-test production or outside providers without an agreement.
Do not let an aggregate graph erase a customer group’s problem
The store may serve mobile customers on slow networks and staff on an office connection. If everyone is combined, a large, fast group can mask another group’s poor experience. Segment by meaningful journey and conditions, but avoid drawing conclusions from a group with only a few requests. Show sample counts and periods with no data.
For resources, read work rate, utilization, errors, and queue growth together. Low CPU may mean idle or waiting. High CPU may reflect useful work or needless repetition. Tie it back to completed outcomes rather than judging the service by one green metric.
Record test conditions with each result: software version, data size, time window, and simulated external services. If the next run changes several things at once, say that the cause of any difference cannot yet be separated. That limit helps people use the result within its proper scope and choose the next test better than a precise-looking speed number with no audit trail.
Hypothetical average rate
Average time in the system
Average in the system, not server count
As average time rises, average requests in the system rise
Average requests in the system (requests)
Average time in the system (seconds)
- Constant average throughput of 100 requests/second
- 0.25 seconds / 25 requests
- 1 second / 100 requests
These hypothetical numbers use means and one consistent boundary under Little’s Law. They are not p95, a maximum-capacity test, or a server-purchase count.
Before approving the first server purchase, I want a one-page record of the key journeys, service target, simulated load, point where the target fails, and limits of the test. If we cannot yet answer those questions, the first money should buy enough understanding to choose what to fix.
Questions to ask the team before ending the meeting
If someone asks “How many people can it handle?” ask what those people are doing. For “How fast?”, specify the page, device, and percentile. For “Did the test pass?”, include errors and transaction correctness. With a clear boundary, we can compare before and after without moving the goalposts to suit the result.
The next episode starts with work that should not be created in the first place. Images larger than the screen, scripts that block interaction, and public content repeatedly rebuilt at the origin make users wait and raise costs. Reducing that work may be a better first move than adding servers, while keeping personal-data boundaries right.
Read the sources and limits of the example
The following sources explain the concepts and limitations and were checked on October 18, 2026. The fictional store numbers and decision suggestions are my synthesis, not provider prices, test results, or guarantees. Before using them in a real configuration, check the current version, plan, and terms.
