Change risk scoring

A 0–100 risk score for every change, from your own CI/CD history.

Settings → Change risk

A 0–100 score for every Change Request, computed from your own CI/CD history rather than from someone's guess. It answers one question for the CAB: how likely is this change to go wrong, given how this code has actually been behaving?

Where the evidence comes from

Your connected GitHub App already sends pull-request events. Change risk additionally ingests workflow_run (GitHub Actions builds) and deployment_status events for any repository mapped to a project, under Settings → Source control. GitLab (pipeline and deployment hooks) and Bitbucket (commit statuses) feed the same history through their webhooks. Nothing is polled and no extra credential is needed.

On GitHub, both events must be ticked in the GitHub App's own webhook settings on github.com — a one-time step outside this app. Until they are, changes score on scope and schedule alone and the CI/CD visibility factor flags the gap.

The nine factors

Pipeline failure rate — how often CI has been red on the mapped repositories in the lookback window.
This change's checks — the runs for this change's own commits, matched through its linked PRs. A still-running check counts as half a failure: unverified is not the same as passing.
Deployment failure rate — counted separately from builds, because green builds that still fail on the way out are a different problem.
Deploy path freshness — a release path nobody has walked in weeks is a release path nobody knows still works.
Change size — linked pull requests, with unmerged ones weighted double.
Blast radius — linked configuration items plus everything one hop downstream of them in the CMDB.
Change type — Emergency changes skip the usual lead time and review. An unclassified change is scored as Normal, never as zero.
Schedule collisions — other changes planned in the same window, which make a failure impossible to attribute.
CI/CD visibility — the blind-spot factor, described below.

Why “no data” is not the same as “green”

A factor with no evidence behind it is excluded from the score entirely — from the total and from the divisor both — and shown as no data rather than as a passing row. Without that rule, a change nobody had any pipeline data for would collect eight reassuring zeroes and come out looking safer than a change with a slightly flaky build.

The CI/CD visibility factor exists for the same reason, from the other direction: it scores full weight when no repository is mapped at all, so “we know nothing about this change” reads as risk instead of as safety.

Weights, bands and the CAB gate

Everything is configurable under Settings → Change risk. Weights are relative, not percentages: the score is the weight earned divided by the weight in play, so setting a factor to zero removes it cleanly rather than deflating every score.

Off — scores are shown, nothing is ever flagged.
Warn (the default) — changes at or above the gate band are flagged on the ticket and here, but submission still goes through.
Block — submitting such a change for CAB approval is refused until its risk drops.

The gate always recomputes at the moment of submission rather than reading the cached score, so it reflects the pipeline as it is right then — a score from before this morning's red build would defeat the point.

The score at submission time is written to the audit log alongside the CAB request, so re-tuning the weights later never rewrites what the board was actually shown when it was asked to vote.