Methodology
How every number is computed.
Each figure on this site comes from a measurement we took or a document a vendor published. This page names which, for all of them — and says what we will not compute.
1. Measured availability
Our own probes, sent from our infrastructure to the vendor's data-plane endpoint — the API your code calls, not the status page. Availability is the share of conclusive samples that returned what the endpoint is supposed to return, to three decimal places.
A sample is inconclusive, and excluded from both sides of that fraction, when the failure was ours rather than the vendor's: marker, prober_network, stale_contract, throttled, uncorroborated. Counting our own throttling against a vendor would be measuring us.
Where we run no probe, we publish no availability figure. That is the honest answer for most of the catalog and it is what makes the published ones worth reading.
2. Disclosure record — and why it is not availability
Separately we record what a vendor said about itself: the share of a window in which they declared no customer impact, computed from their own status posts.
This is not availability, and we label it so everywhere it appears. It moves the wrong way — a vendor that posts a notice for every datacentre re-route scores below one that stays silent through an outage, and any vendor carrying a single open "degraded" notice pins to exactly 70.000%. We published Cloudflare at 74.587% under that reading once. It is a record of what a vendor communicates, which is separately interesting and is never a reliability score.
3. Time to resolve
Median and 90th-percentile minutes from the vendor's own incident start to their own resolution — their record of their own handling. Publishing more incidents cannot move it in the flattering direction, which is why we publish it and not an incident count.
Two populations are excluded, and both are stated on the page beside the figure:
- Incidents whose start time we stamped, because the vendor never published one. Their duration is a lower bound, and mixing bounded values with unbounded ones produces a median that means nothing.
- Maintenance announcements, which are not outages.
Below 5 resolved incidents we state no median at all. A median and a 90th percentile that are the same number is one data point wearing a distribution.
4. Downtime weighting
Where a figure has to fold different severities into one number, intervals are weighted by customer-perceived impact. Two mappings do this, and which one applies depends on whether the incident has been classified yet.
When a classifier has read the vendor's own write-up and produced a verdict, that verdict is weighted:
| Impact | Weight |
|---|---|
| down | 1.0 |
| degraded | 0.3 |
| none | 0.0 |
Otherwise the vendor's own severity label is weighted directly, in whichever vocabulary their status page uses. This is the mapping that fires most often, and it is not a rounding of the one above — a status-page major is neither a full outage nor a blip:
| Impact | Weight |
|---|---|
| critical | 1.0 |
| major_outage | 1.0 |
| major | 0.7 |
| partial_outage | 0.7 |
| minor | 0.3 |
| degraded | 0.3 |
| degraded_performance | 0.3 |
| maintenance | 0.0 |
| under_maintenance | 0.0 |
| none | 0.0 |
| operational | 0.0 |
A label appearing in neither table weighs nothing. Understating downtime costs us a claim; overstating it files a false one against a vendor.
Overlapping intervals are merged within each weight before being summed, so two component-level notices about one outage count once. Scheduled maintenance is subtracted — every major SLA excludes it, and leaving it in inflates the breach and gets a claim rejected on that ground alone. An unresolved incident is capped at the present moment, never run to the end of the window.
5. SLA credits
The money path is pure arithmetic over measured data and the vendor's published terms. No model is consulted at any point in it.
A credit is computed only from claim-grade evidence: the vendor's own status-page admission, and the customer's own SDK telemetry. Every SLA we have read puts the evidentiary burden on the account holder and names what it will accept — none of them accepts a third party's synthetic check. That is not a criticism of our probes; it is a statement about what a claim can be built from.
Within that evidence, only grades meaning unavailable count — critical, down, major_outage. A partially degraded minute earns no partial credit, because no SLA we have read grants any: a vendor defines a minute as unavailable or not.
The resulting uptime is resolved against the vendor's own published credit ladder. The dollar figure is that percentage applied to the spend you have told us you pay. With no spend on file we state the percentage and no dollar amount.
6. Customer telemetry
Error rates from SDK telemetry are read as a statement about a vendor only above 20 requests in the five-minute window. Below that a single caller is the whole window, and its timeouts, its DNS and its expired token all read as the vendor being down.
Aggregate telemetry is published only once 3 organisations are reporting. Below that it is not aggregate data — it is one identifiable customer's request volumes on a public page about their vendor.
7. What we will not compute
The list below is the useful half of any methodology. Each entry is a number this system could produce and deliberately does not.
- An availability figure with no probe. Absence is published as absence, not filled from the vendor's status posts.
- A dollar amount without your spend. The credit percentage stands alone; we never infer a bill.
- An uptime percentage from how often a vendor posts. The disclosure record is published as what it is, and never as availability.
- A credit from evidence a vendor rejects. A window whose only support is our own synthetic sampling states no credit.
- A claim while the outage is still running. Total downtime is not final, so any tier computed now would change.
- A probability that a claim will be approved. Nothing has ever been fitted to vendor outcomes. We show which preconditions are met instead.
- An AI-generated figure anywhere money is involved. Models summarise and classify; they never compute an entitlement.
8. Corrections
Every threshold quoted above is read from the code that enforces it, so this page cannot drift from the behaviour it describes.
If a number here looks wrong, it is worth more to us than it is to you — [email protected].