Tools alone do not show whether a team is improving. The DORA research programme (DevOps Research and Assessment, now part of Google Cloud), summarised in the book Accelerate and the yearly State of DevOps reports, found four metrics that predict software delivery performance and, in the research, organisational outcomes too. They are simple enough to measure in a week and useful enough to review every month.
The four metrics
| Metric | Question it answers | Measures |
|---|---|---|
| Deployment frequency | How often do we release to production? | Throughput |
| Lead time for changes | How long from commit to running in production? | Throughput |
| Change failure rate | What share of releases cause an incident, rollback or hotfix? | Stability |
| Time to restore service | How quickly do we recover from a failure in production? | Stability |
The first two measure speed, the last two stability. The headline finding of the research is that the best teams do well on both; speed and stability are not a trade-off, because small, frequent, well-tested changes are easier to review, to roll back and to diagnose. Later reports add reliability (meeting your SLOs) as a fifth measure of operational performance, which pairs well with these four.
Where each number comes from
flowchart TB
GIT[Git history<br/>commit and merge timestamps] --> LEAD[Lead time<br/>first commit or merge to production deploy]
CI[CI/CD pipeline events<br/>deployment succeeded to production] --> FREQ[Deployment frequency<br/>deploys per day or week]
CI --> LEAD
CI --> CFR
INC[Incident tracker and alerts<br/>opened, mitigated, resolved] --> CFR[Change failure rate<br/>failed changes divided by all changes]
INC --> MTTR[Time to restore<br/>detection to recovery]
ROLL[Rollbacks and hotfix deployments] --> CFR
LEAD --> DASH[Dashboard and monthly review]
FREQ --> DASH
CFR --> DASH
MTTR --> DASH
DASH --> ACT[Pick one bottleneck and one experiment]
Define each term in writing before you collect anything:
- A deployment is a release of a change to production, counted once per service. Decide how to count configuration-only changes and feature flags (usually count them if they can break users).
- Lead time is easiest from the first commit on a branch, or merge to main, to the moment the code serves production traffic. Use the same start point always.
- A failed change is one that needs remediation: rollback, hotfix, patch or a customer-visible incident linked to the release. Link incidents to the deployment that caused them.
- Time to restore runs from the start of customer impact (or detection) to recovery, not to the final root-cause write-up.
Collect them with what you already have
A pipeline step can record every production deployment as a small event:
# last step of the production deploy job
curl -sS -X POST "$METRICS_URL/deployments" -H "Authorization: Bearer $METRICS_TOKEN" \
-d "{\"service\":\"shop-api\",\"sha\":\"$GIT_COMMIT\",\"environment\":\"production\",\"deployed_at\":\"$(date -u +%FT%TZ)\"}"Lead time then joins the deployment’s SHA with commit timestamps:
# commits that went out between two production tags
git log --format='%H %ct' v1.41.0..v1.42.0 | awk -v d="$(date -d '2023-01-09T10:00:00Z' +%s)" '{print $1, (d-$2)/3600 " hours"}'For a quick start, GitHub’s deployments API or release tags give frequency, pull request merge times give lead time, and a labelled incident list gives failure rate and restore time. Purpose-built tools (open-source Four Keys, or commercial engineering-analytics products) do the joins for you; a spreadsheet is fine at the beginning. Publish the numbers to CloudWatch or Grafana only after the definitions are stable.
Reading the numbers
The State of DevOps reports group teams into performance bands. Treat them as a rough guide, because bands change from year to year and depend on context:
| Band | Deployment frequency | Lead time | Change failure rate | Time to restore |
|---|---|---|---|---|
| High performers | On demand, several per day | Under a day to a week | Low (a minority of changes) | Under a day |
| Medium | Weekly to monthly | A week to a month | Moderate | Under a day to a week |
| Low | Monthly or rarer | Months | High | Days to weeks |
Look at trends over months, at the median and the slow tail (the 85th percentile), not a single average. One huge deployment freeze, a holiday week or a migration will skew a month.
Typical bottlenecks and fixes
| What the data shows | Likely cause | Try |
|---|---|---|
| Long lead time, short build | Reviews waiting | Smaller pull requests, review-time agreement, pairing |
| Long lead time, long pipeline | Slow or flaky tests, manual approvals | Parallelise, quarantine flaky tests, automate checks |
| Few deployments, big batches | Fear of releasing | Feature flags, trunk-based development, automated rollback |
| High failure rate | Weak tests, no staging parity | Better integration tests, canary releases, progressive delivery |
| Slow restore | Poor observability, manual rollback | Dashboards and alerts per service, one-command rollback, runbooks |
Do not misuse them
- Do not rank people or teams against each other: it invites gaming (splitting deployments, hiding incidents) and destroys the honesty the numbers depend on. Compare a team with its own past.
- Do not set the metric itself as the goal. Goodhart’s law applies: “10 deployments a day” achieved by deploying empty changes proves nothing.
- Combine the numbers with qualitative signals: team surveys about burnout, deployment pain and tooling satisfaction.
- Pair stability data with SLOs and error budgets, so reliability work has a place on the roadmap.
Start this month
- Pick one service and agree the four definitions with the team.
- Record deployments and incidents in one place for four weeks.
- Review the numbers in the retrospective, choose one bottleneck and one experiment, and measure again.