The cover illustrates a fictional store, not a deployed system or real test result.
Ready Parts’ marketing team is considering a campaign. The technical team proposes a CDN, more servers, a database replica, and a queue. Should the owner approve all of them at once? I would start with a plan that tests one claim at a time: what must pass, what level is acceptable, who can stop the trial, and how we return to the earlier state.
This series does not prescribe one architecture for every business. A store with a few operators may do best with a comprehensible app and managed services; work with special requirements may need more separation. Component count is not a quality score. The goal is to deliver correct work as promised within cost and team capacity.
Write the service agreement before the purchase list
Start with customer journeys: view products, search with authorization, confirm orders, and receive documents. Define checkable success and separate what users see immediately from later background work. Then set SLIs and SLOs with the window, eligible requests, and error counting. Timeouts and throttled requests should not disappear from a report merely because they make the average look worse.
Set hard correctness conditions apart from temporary slowness we may tolerate: no cross-account prices, no duplicate order outcomes, and no stock confirmation outside the rules. These conditions come before speed comparisons; many fast requests cannot average away a breach of them.
An SLO also requires a conversation about business impact and operating effort. A stricter one may require more reserve capacity, detection, backups, and on-call coverage. An error budget, the allowed shortfall within a defined period, can inform risk decisions. It is not permission to corrupt data or postpone legal or contractual duties.
Build a baseline from the current system
Collect only necessary real-user data without violating privacy, and record software version, data size, work mix, user regions, and collection period. If there is no prior data, do not claim a percentage speedup. Measure a baseline first, or compare options in one controlled test and state the limit.
Connect tests to a comparable month’s cost and actual labor. A one-day campaign cannot reveal the whole year’s bill, though it can help estimate costs that change with work. Separate recurring, one-time, and uncertainty allowances. State how a short test is projected into monthly spending; do not call the projection an incurred bill.
Define completed work and load
Test and remove unnecessary work
Add only what passes quality, cost, and team-capacity checks
Test a reasoned change first
If the browser is overloaded, begin with assets and rendering. If reads repeat and are shareable, cache them with the right access rules. If a heavy query creates a queue, fix the question and concurrent access before adding app instances. Choose a change with a clear hypothesis about what it reduces, and compare before and after under similar conditions.
Not every commit needs to be tiny, but each change set should have a result we can assess and a path back. Changing the framework, database, region, and cache policy at once may produce a new graph with no clear cause, good or bad, and a rollback harder than the team can manage.
| Decision | Evidence to pass | If it fails |
|---|---|---|
| Measure and define | Know journeys, success unit, load, and service target | Collect evidence before approving capacity that cannot yet be estimated |
| Reduce work | Less browser and origin work with access rights and quality intact | Fix assets, caching, or queries at the measured point |
| Divide and add capacity | Meet target load with headroom and without saturating shared services | Fix bottleneck, limit intake, or change the scope before adding |
| Withstand failure | Correct status, working recovery, and people able to follow the plan | Pause expansion; fix recovery and ownership |
| Worthwhile and operable | Cost per outcome and team load within agreement | Adjust service, remove excess, or choose a simpler option |
A load plan must say what is sent and what actually finishes
Build a business-representative mix, not only warmed-cache product pages. Separate public, account-specific, write, and background work. State arrival rate, duration, burst periods, and data distribution. Test normal cases and a hot key or product so an average does not hide work that cannot be divided.
Choose an open or closed load model to match the question from Episode 1. Check whether the load generator reaches its target and whether iterations were dropped. Include relevant timeouts and rejected work. Do not add p95 values from each layer and call them customer p95; measure the end-to-end journey too.
Start in a controlled environment where side effects can be contained. Isolate test accounts and data, and set stop criteria for impacts on data, budget, or others’ services. Testing production needs a suitable plan and authorization. A heavy load test is not a proof of courage that we should run and hope survives.
Reserve capacity from test evidence, not a users-to-servers formula
Per-instance test results may help estimate an initial count when the work mix matches and jobs truly divide. Allow for variation, startup delay, and the failures the design is supposed to tolerate. Multiplying one-instance results by ten does not prove ten instances yield ten times the work: they share a database, network, and coordination duties.
Set a maximum count and signal for hitting it. At that limit, which work is throttled, what does the customer see, and who can raise the budget? For autoscaling, test from trigger until new capacity actually serves work, not only after every instance is ready. Scaling down must not cut off pending jobs or discard needed state.
Practice failures within a chosen boundary
Test components that disappear, slow down, or return uncertain results according to risk: one app instance stops, a replica lags, a document sender times out, or AI is unavailable. Watch user-visible status, final transaction outcomes, and time for operators to identify the problem. There is no need to trigger every failure together and lose the cause.
For queues, test redelivery and a stop during processing. For the database, restore a backup and check record completeness. For caches, switch accounts and edit data. A spare machine or backup file does not show when the store can accept orders again or how much data may be lost.
Stops, slows, returns uncertainty, or hits quota
Bounds work, checks status, and prevents duplicates
Restore data, reconcile, and verify the outcome
Set business recovery objectives for how much recent data may be lost and how long the service may stop, then test the designed method. Buying a service package does not certify those objectives. If the test fails, report the gap and remediation plan before claiming readiness.
Rollback must include data, not just code
A new version should have a way to limit impact and inspect signals before wider release, perhaps with a controlled group or routed path. Preserve access rights and transaction completeness. A test group is not an excuse to skip real outcome checks. When signals cross a limit, know who can stop the rollout and what evidence they will use.
For schema or data-format changes, check whether old and new versions can read and write together. Rolling code back does not roll data back automatically. If work already happened, such as sending a document or charging a customer, repair it through authorized compensation or reconciliation. Do not erase history to make a screen look as though it never happened.
Document the exit from any service-specific dependency: how data leaves, the time and people required, and the cost of running both paths during transition. We need not design for a weekly provider switch, but a budget approver should know what commitment is being made and its approximate exit cost.
Give approvers and operators the same evidence
A good decision note is short enough for a meeting, with links to test results, query plans, cost inputs, and measurement limits. Name assumptions that trigger review when they change, such as request mix, data size, completion rate, outside wait time, or operator count. Do not review only by calendar date while ignoring actual signals.
Mis-scoped data access, duplicate transaction outcomes, and important risks need immediate action under incident rules, not a monthly review. Routine capacity increases can follow agreed thresholds and budgets. A plan should not slow the team’s response to known customer harm.
Old/new code and schema must coexist
Preserve history and check changed records
Compensate or reconcile with authority
I would approve the next expansion when the team can show the bottleneck, what the option buys, the conditions under which it passed testing, and how to reverse it. If the evidence says to reduce work or stop adding components, that is a sound decision for a growing system too.
The plan ends with accountable people who can say whether customers still complete important work as volume grows, not a checklist of every technology layer. With enough evidence, the team can choose the next step without always starting at “add servers.”
Read the sources and limits of the example
The following sources explain the concepts and limitations and were checked on October 18, 2026. The fictional store numbers and choices are my synthesis, not provider prices, test results, or guarantees. Check the current version, plan, and terms before applying them.