Methodology
What our numbers can tell you.
Before you use a figure to investigate an outage, check its source, its coverage and what is missing.
The starting point
Different sources answer different questions.
CheckUpstream brings these records together. Each keeps its own meaning, so you can decide what to investigate next.
- Vendor reportsTheir status page
The vendor’s incident posts, with the start and closure times in our record.
A quiet status page does not establish that every customer was unaffected.
- Independent checksOur monitoring infrastructure
How a configured vendor endpoint responded to our checks.
Coverage is limited to the endpoints, regions and samples we collected.
- Application telemetryYour instrumented requests
The request failures and latency recorded by the optional SDK.
A request failure still needs attribution; it may originate in your app or network.
Independent checks
Counts within a stated scope.
Our check sends one request to a configured vendor endpoint, its front door, and compares the answer with what that endpoint is documented to send. We run it every 15 minutes from a single CheckUpstream location and publish the results as counts, never as a percentage.
- Host
- The endpoint the checks were sent to.
- Reachable
- Checks whose answer matched the documented response.
- With a clear answer
- Checks that answered clearly, either way.
- Window
- The last 30 days.
- Cadence
- Every 15 minutes.
Checks without a clear answer are counted separately and added to neither number.
Checks 15 minutes apart cannot see disruptions shorter than that. A ratio of these counts would claim more than checks this far apart can see, so we do not publish one.
A reachable front door does not establish that another product, account or region worked. No configured check means no count. When no clear answer has arrived for 24 hours, the page says so instead of presenting older counts as current.
Which checks have no clear answer?
- markerThe response did not match our configured marker.
- prober_networkThe failure was attributed to our monitoring network.
- stale_contractThe check’s expected response needs review.
- throttledOur check was rate-limited.
- uncorroboratedAn initial failure has not been corroborated.
A failed check with an HTTP 4xx response has no clear answer either, including request and authentication errors.
Vendor reports
What the vendor has said.
The disclosure record comes from the vendor’s status posts: reported incidents, their severity, reported starts and the closure times stored in our record.
A period with no reported incident tells you about those reports. It does not prove continuous availability or confirm that your app was unaffected. Missing resolution information also does not prove an outage is still happening.
Use this record to find the vendor’s account of an event, then compare it with your own logs and request data.
Incident history
Incident durations need enough history.
Median and 90th-percentile recorded incident durations run from a vendor-supplied start to the closure time stored by CheckUpstream. They describe qualifying records that started in the selected window.
A closure time may come from a vendor update, from when we observe that a post has disappeared, or from expiry of our tracking period. These records do not reliably distinguish the cause. A closed record does not confirm vendor or application recovery.
qualifying closed records before we publish a median or 90th percentile.
Records without a closure time, maintenance records and incidents without a vendor-supplied start time do not enter those duration statistics. Invalid timestamp pairs are excluded too.
A first-observed start can miss the beginning of an incident. Excluding these records avoids mixing observed starts with vendor-supplied starts.
Application requests
Evidence from the app that made the call.
The optional SDK records aggregate request failures and latency by vendor or host. This adds context from instrumented traffic; a public vendor report alone cannot establish your app’s impact.
- SDK failure alert
- 5%of at least 20 calls to one vendor in a 5-minute window must fail before your organisation is alerted.
- Public aggregation floor
- 3reporting organisations before aggregate telemetry is published for a vendor.
Only network errors, timeouts and 5xx responses count as failures. A 4xx response is the vendor refusing the request, so it never counts. The alert is a fixed threshold, not a statistical confidence, and a counted failure can start in your own network. The aggregation floor governs publication only.
Reading the gaps
What we will not compute.
A missing figure is useful information. It tells you what the available records cannot support.
- No configured check
- No front door counts. Vendor status posts do not fill the gap.
- Too little history
- No resolution-time median or percentile below the sample floor.
- A vendor’s SLA
- No credit percentage, dollar estimate or breach verdict. The vendor’s published terms and its own decision govern a claim.
- An unresolved incident
- Its duration is unfinished. Treat any current estimate as provisional.
Help us correct the record.
The thresholds here are read from the code that applies them. That keeps their values in sync; it does not make the data or this explanation infallible.
If a result looks wrong, send us its page URL, the vendor, the time window and the source you are comparing it with.
Report a data issueRead the record for your vendors.
Explore their reports, measurements and coverage with these limits in mind.