p50, p95, p99: What Averages Hide

Questions

1. 1
   Not answered. If a service's p99 latency for the last hour is 250 ms, what does that number mean, precisely?
2. 2
   Not answered. Server A and Server B each report a p99 latency of 100 ms over the same hour. If you merge their traffic into one pool, what is the combined p99?
3. 3
   Not answered. A dashboard shows a p99 of 300 ms for each of the last 24 hours. Is the daily p99 also 300 ms?
4. 4
   Not answered. Which of these statistics can be correctly combined across two data sources — for example, two hosts, or two time buckets — using simple arithmetic?
5. 5
   Not answered. A metrics backend stores latency as histogram buckets (for example, 0–10 ms, 10–25 ms, 25–50 ms, 50–100 ms, …) instead of raw samples. What's true about a p99 computed from those buckets?
6. 6
   Not answered. Why is the maximum observed latency a real, meaningful signal, but usually a bad choice for a service-level objective (SLO) target?
7. 7
   Not answered. You want an accurate p99 for your service across all 200 hosts in the fleet, not just one host. Which approaches actually produce a correct fleet-wide p99?
8. 8
   Not answered. Latency distributions are typically right-skewed: a hard floor near zero, and a long tail of slow requests stretching out to the right. What does that imply about mean vs. median latency?
9. 9
   Not answered. A service's latency has two clear humps: most requests hit a warm cache and finish in about 5 ms, and the rest miss the cache and take about 200 ms. Why does a single percentile like p90 poorly describe this distribution?
10. 10
    Not answered. This week, p50 latency rose from 40 ms to 90 ms, but p99 latency stayed flat at 800 ms. What kind of problem does that pattern usually point to?
11. 11
    Not answered. p50 latency stays flat at 40 ms, but p99 latency climbs from 300 ms to 1200 ms. Which statements about this pattern are true?
12. 12
    Not answered. The chart below shows a CDF: cumulative fraction of requests on the y-axis against latency on the x-axis, with arrows showing how to read off p50, p95, and p99. What does a CDF make easier to read directly than a raw histogram does?
13. 13
    Not answered. Which of these are realistic, commonly seen causes of bimodal — two-humped — latency distributions in a web service?
14. 14
    Not answered. A page load makes 100 independent backend calls in parallel. Each individual call has a p99 latency threshold — meaning 1% of calls, independently, exceed it and count as "slow." Assuming independence, what percentage of page loads see at least one slow call?
15. 15
    Not answered. When a page waits on N backend calls issued in parallel, the page's total latency is the maximum of the N call latencies, not their average. What does that imply as N grows?
16. 16
    Not answered. Why is a system's overall tail latency generally determined by its slowest dependency, not by its average dependency?
17. 17
    Not answered. Which of these are true about hedged requests — sending a duplicate request after a short delay and taking whichever response comes back first — or retries, as a way to reduce tail latency?
18. 18
    Not answered. Same setup, but the page only fans out to 20 independent backend calls instead of 100, still each with a 1% independent chance of being the slow one. What percentage of page loads now see at least one slow call?
19. 19
    Not answered. With the same 100 independent backend calls, hedging cuts each call's effective chance of being "slow" from 1% down to 0.1% — the hedge catches almost all of the independent slowness. What percentage of page loads now see at least one slow call?
20. 20
    Not answered. Little's Law relates the average number of requests in a system (L), the average arrival or throughput rate (λ), and the average time each request spends in the system (W). Which equation is it?
21. 21
    Not answered. A service handles 200 requests per second on average (λ), and on average has 50 requests concurrently in flight (L). Using Little's Law, what is the average time each request spends in the system (W), in milliseconds?
22. 22
    Not answered. A service handles 500 requests per second on average (λ), and each request takes an average of 100 ms to complete (W). Using Little's Law, how many requests are concurrently in flight in the system on average (L)?
23. 23
    Not answered. The chart below plots average queueing wait time against server utilization (ρ), annotated at 50%, 80%, 90%, and 95% busy. Based on the shape of that curve, what happens to wait time as utilization rises from 80% to 90% to 95%?
24. 24
    Not answered. Using ρ / (1 − ρ) as a relative queueing-delay factor, where higher means more wait, what is its value at ρ = 0.95?
25. 25
    Not answered. A system is provisioned to handle exactly its average load, with no spare capacity. Why does it tend to fall over — not just slow down a bit — once real traffic hits roughly 2× that average?
26. 26
    Not answered. Two systems have the exact same average utilization, say 80%, and the exact same average service time per request. One has highly variable service times — some requests take 1 ms, others take 500 ms. The other has nearly constant service times — every request takes about the same time. Which statements are true?
27. 27
    Not answered. Ten checkout servers each get their own separate queue of waiting requests, versus the same ten servers fed by one shared queue, where the next free server pulls the next waiting request. Assuming the same total arrival rate and the same per-server capacity, which arrangement gives lower average wait time, and why?
28. 28
    Not answered. A write path can batch several items into one database call. Larger batches mean fewer, more efficient calls — higher throughput — but each item waits longer for its batch to fill, meaning higher latency. Where does this trade-off's practical crossover point usually sit?
29. 29
    Not answered. The timeline below shows a load generator's intended request schedule, evenly spaced, against when it actually issues requests, during a stall where the target service stops responding for a while. Based on that gap, what is "coordinated omission"?
30. 30
    Not answered. What effect does coordinated omission have on latency percentiles measured by a closed-loop load generator — one that waits for each response before sending the next request?
31. 31
    Not answered. A request's server-side handler logs 40 ms of processing time, but the client's own timer, from clicking a button to receiving the response, measured 180 ms for the same request. What does that 140 ms gap most likely contain?
32. 32
    Not answered. Why does a well-formed latency SLO need both a measurement window, for example "over any rolling 30 days," and a target, for example "p99 under 500 ms," rather than just a bare number?
33. 33
    Not answered. An SLO targets 99.9% availability, measured over a rolling 30-day window. How many minutes of downtime does that allow within those 30 days?