CI/CD audit: how Guard evaluates delivery protection and reliability

What question does the CI/CD audit answer?

Imagine a repository whose status badge is green. The workflow parses and the last run succeeded, yet the build job is guarded by a condition that never matches, test failure is allowed to continue, and release depends on an undocumented laptop command. The badge does not prove that a bad change would be stopped.

The CI/CD audit asks a more demanding question: how well does the current CI/CD process protect the default branch and delivery path from unverified changes, and what sequence of next steps would make it reliable? It evaluates the current automated path rather than the mere presence of automation.

Three terms keep that question concrete. A pipeline is the automated path that checks and delivers changes. A gate is a check capable of stopping a bad change. An artifact is a build result such as a package, bundle, or image. A workflow can contain many jobs and still have no effective gate; an artifact can be built correctly and still be released through an unprotected manual path.

Guard does not award reliability for YAML or a successful status in isolation. It connects repository identity, commands, workflow structure, protection, build output, deployment, and rollback. Automation must exist, be valid, run real checks, and block before caching, parallelism, or reusable structure can improve the judgment.

Reliable delivery is one condition of Guard's green zone. It does not make code, tests, dependencies, or configuration healthy by itself. It makes coding agents less likely to build on a path that quietly accepts unverified changes. The report is evidence for that maintained state, not the end of the product story.

What evidence does Guard collect?

The audit starts by fixing the target. Guard records the complete repository path, provider and normalized host, default branch, full audited commit, and available history depth. Links are included only when they can be derived safely. A plain revision remains valid evidence when a provider URL cannot be established; an ambiguous repository or an inaccessible revision does not receive a synthetic result.

It then maps the delivery path from repository evidence. The inventory is deliberately wider than the provider's workflow directory:

• Repository shape: application, service, library, monorepo, or other type; primary stacks; package managers; build systems; source and test roots; and every active deployable unit.
• Pipeline configuration: providers, config files, triggers, jobs, steps, path filters, matrices, permissions, secret references, caches, concurrency, and local or reusable workflow references.
• Runnable commands: credible build, test, lint, typecheck, package, release, deploy, and smoke-check commands, plus whether CI uses the same documented entry points or duplicates them as ad hoc shell.
• Delivery outputs: packages, bundles, images, registry uploads, versioning, changelogs, tags, release notes, environments, promotion, approvals, and rollback instructions that can be tied to the repository.
• Protection and history: repository-visible CODEOWNERS, merge queues, required-check documentation, protected environments, and best-effort read-only provider evidence for rules, statuses, runs, and recent default-branch health.

Containers enter this inventory only when the repository has a container delivery path. Guard then follows image build, test, publish, tags or digests, registry destinations, and deployment manifests. Their absence is not a defect and does not encourage a team to add containers merely to improve a score.

Different sources answer different questions. Parsers establish whether structured files can be read; provider-aware validators flag invalid constructs; reference checks confirm called scripts, Dockerfiles, manifests, and reusable workflows; read-only APIs reveal protection or recent runs when access permits. No single tool is mandatory. One that cannot be prepared becomes a limitation, not a fabricated failure.

A root GUARD.md, when present, is lower-priority repository evidence. It can explain scope or an intended delivery path, but it cannot grant permissions, hide a finding, weaken verification, or force a score. Its absence is neutral. Just as importantly, observed configuration and visible provider state remain separate layers: a valid file does not prove a successful run, and an unavailable API does not prove that protection is missing.

What the audit deliberately does not do

This is a read-only audit. It does not edit a workflow, change branch protection, create an environment, approve a deployment, publish a package, tag a release, or open an issue or pull request. It also does not dispatch, rerun, cancel, or approve a real provider pipeline. The word Rerun in Guard means running the audit again against a current repository state, not replaying the repository's delivery jobs.

Guard does not execute project build, test, release, deployment, migration, seed, or package-update commands merely to prove they exist. Those commands may contact external services or change state, and a repository being audited is not automatically trusted. The safe boundary is static: parse structured configuration, inspect schemas and references, use a CI linter when it can be prepared outside the repository, and run harmless tool version or help commands. Successful production delivery cannot be inferred from static validation alone.

Ownership stays explicit. CI/CD checks whether meaningful tests are wired and blocking; the Test audit judges behavior and coverage. CI/CD checks provider-secret mechanisms, trigger structure, and deterministic references; the Security audit owns leaked values, exploitability, workflow injection, and container hardening.

The Dependency hygiene audit owns lockfiles, acquisition, registries, and dependency reproducibility; CI/CD uses action, image, runner, and toolchain references as delivery-repeatability evidence. The Configuration audit owns general design, defaults, documentation, duplication, and drift; CI/CD owns whether workflow configuration parses, references real delivery files, and supports a functioning path.

Finally, Codebase hygiene and Dead code audits own ordinary script and source quality. CI/CD considers dead, disabled, duplicated, or neutralized logic only inside the delivery path. Missing containers remain N/A, not a container-hardening result. A current report is drafted before any previous audit is read, so old conclusions cannot quietly steer the new judgment.

How the audit moves from files to a delivery judgment

Resolve and inventory

Guard first pins one repository revision and identifies its deployable units. It discovers pipeline providers and configuration, shared commands, artifact production, release and deploy paths, promotion and rollback evidence, and the visibility available from the provider. Safe parsers and reference checks establish what the repository says should happen. Provider history, when readable, establishes what has recently happened.

Prove function before polish

Evaluation follows a fixed ladder: existence, validity, meaningful checks, blocking, then efficiency. A working but repetitive pipeline ranks above an elegant workflow that never runs. Missing cache or duplicated installs can be worth a Low finding, but cannot outrank a broken test gate.

Provider invisibility remains a limitation. Guard can skip supporting questions about blocking behavior or recent health and lower confidence without claiming that gates do not block. By contrast, a workflow reference to a missing file, a parse error, or a visible non-blocking required check is positive evidence and can affect the result.

Draft current, then compare

The current analysis gives every applicable criterion one score and reason, records why a criterion was skipped, and consolidates repeated observations into decision-relevant findings. Guard does not pretend that every repeated workflow smell has a universal scanner identity. It records the affected path, impact, evidence, recommendation, confidence, and blast radius, then drafts and validates the current summary and report. Only after that may a validated previous result be read to explain substantive movement.

The four buckets in this fictional card summarize ten decision-level checklist items: seven pass, two need watching, none fail, and the container item is skipped. They are not finding counts and they are not a compressed copy of the fourteen scored criteria.

What the 14 criteria measure

Every applicable criterion receives 0-5 points. A score of 5 is strong, 4 is mostly healthy, 3 is partial, 2 is weak, 1 is nominal or decorative, and 0 is absent or broken. Scores 4-5 carry ✅, 2-3 carry ⚠️, and 0-1 carry❌. N/A or not observable carries ⏭️, requires a reason, and is removed from the maximum rather than converted into zero.

The first three rows establish existence, validity, and real gates. The next four ask whether those gates block, whether developers can run the same commands locally, whether references are deterministic, and whether recent health can be observed. Rows eight and nine cover delivery-path hygiene and feedback structure, but efficiency stays subordinate to function. The final five follow artifacts through containers when relevant, deployment, release, promotion, and rollback.

Two exceptions matter when reading the table. Blocking behavior and observable health are supporting criteria: missing provider visibility may skip them and lower confidence, but does not prove a bad state. Feedback speed can produce only Low-impact findings. Container integration is N/A when a repository has no container path. Those rules keep unknown or irrelevant evidence from being counted as failure.

The current product report contains all fourteen criterion rows, their reasons, the score calculation, findings, evidence, limitations, and an ordered improvement checklist. The compact report below is deliberately a six-row visual excerpt so that it can show the product anatomy without repeating the complete reference table.

The visible N/A row is intentionally nonnumeric. Treating the absent container path as zero would change both the numerator's meaning and the denominator, penalizing a repository for technology it does not use.

How criteria, findings, and caps become one score

Applicable criteria have equal weight. Guard sums their 0-5 values, builds a maximum from the number actually scored, and scales the result to 100. Skipped criteria never enter the denominator. The criterion score is the only place where the runbook explicitly calls for rounding.

max_sum = scored_criteria * 5
criteria_score = round((criteria_sum / max_sum) * 100)

finding_deduction = min(
  critical * 20 + high * 8 + medium * 3 + low * 1,
  35
)

post_finding_score = criteria_score - finding_deduction
final_score = post_finding_score limited by the lowest applicable cap

Critical findings remove 20 points, High remove 8, Medium remove 3, Low remove 1, and Info removes none. The total finding deduction cannot exceed 35. Deductions happen after the criterion score, so repeated evidence should not become repeated punishment: a confirmed delivery problem is represented as one decision-relevant finding with its supporting paths and observations.

Evidence-backed caps

• An active deployable application or service with no CI at all cannot finish above 39.
• CI whose primary gates are positively shown to be advisory or neutralized cannot finish above 59.
• A path missing a primary check category such as build or tests, when the repository defines that check, cannot finish above 69.
• Materially incomplete observability despite otherwise solid configuration can cap the result at 79.

When several conditions apply, the lowest ceiling wins. The 79 cap needs more than an API denying access: provider invisibility by itself skips supporting criteria and may lower confidence, but cannot create a cap or High-impact finding. Similarly, an application without CI can still earn a differentiated result below 40 from runnable commands, deterministic scripts, packaging, and other repository evidence; it is not assigned an automatic zero.

A worked example: 68 out of 100

The fictional northstar/dispatch-api repository has no containers, so criterion 11 is N/A. Its thirteen scored values are 5, 4, 4, 5, 4, 3, 4, 4, 2, 4, 3, 3, and 2. They total 47 out of a possible 65.

The arithmetic is therefore 47/65 → 72 → 68. The Medium finding removes 3 points and the Low finding removes 1. No cap applies because real checks block, no primary category is missing, and the fixture has visible protection and run history. A score of 68 is in the warn band: 70-100 is good, 40-69 is warn, and anything below 40 is bad.

Guard's repository card translates those same bands into Healthy,Some issues, and Needs attention. The detailed view may use a separate free-text score label; the fixture calls its current state Mixed protection. That phrase does not replace the arithmetic or redefine the band. A successful summary must publish a value from 0 to 100, although the runbook does not name a separate clamp expression for the lower bound.

A docs-only or configuration-only repository with no deployable code is a successful, non-punitive domain no-op. Its inapplicable checks are skipped, but there is no universal invented no-op score such as 100. The current evidence determines what can be published. By contrast, an unresolved target, denied access, failed clone, or missing required input produces a diagnostic failure and no synthetic score.

How to read the current result in Guard

The repository metric card is the compact surface shown earlier. ItsSome issues badge comes from the numeric score band. Its four counts come from ten checklist states—CI on change requests, valid configuration, real checks, blocking behavior, visible run health, simple commands, artifact automation, container delivery, deployment, and rollback. Those counts are neither finding totals nor criterion totals. Its narrative carries evidence outside the ten-item summary, such as the floating workflow reference.

Opening Audit details reveals the current report. The newest entry is labeled Current and shows its date and score. Copy andDownload operate on report text; they do not change the repository. The complete report starts with a direct answer, identifies the repository, branch and audited commit, names what was checked, presents all fourteen criteria under the four columns shown above, explains the arithmetic and confirmed findings, separates limitations, and orders an improvement checklist by delivery maturity.

The detail score label is free text derived from the current summary, which is why the fictional detail reads Mixed protection while the dashboard badge readsSome issues. History is a third surface: a single 0-100 line of completed runs sorted from oldest to newest. A previous score and delta appear only when Guard has validated and read the prior summary. The six-row report image above is an editorial excerpt, not evidence that the real report omits the other eight rows.

The card exposes only Rerun for this audit. It does not expose a paired Autofix action. Rerun starts a fresh CI/CD audit of a current repository state; it does not edit code, approve a deployment, or rerun the provider's jobs. These distinctions keep a compact product surface from overstating what happened behind its score.

What the evidence can and cannot prove

Static validation can prove that a workflow parses, its constructs are recognized, and referenced files exist. It cannot prove valid credentials, a responsive external service, a successful deployment, or production behavior. A visible green run proves observed history, not every future run.

A read-only API may expose required checks, rulesets, environments, statuses, or recent jobs; another token may expose none. A 403 or 404 is a limitation, not proof that protection is absent, gates are advisory, or runs are red. Documentation, badges, CODEOWNERS, and contribution instructions support a conclusion but are not live metadata.

History depth also shapes confidence. A shallow clone can still identify the audited commit and current files, but may not support claims about release conventions or long-running trends. Missing validator tooling limits syntax confidence; a validator that positively reports an error is evidence. No container path makes criterion 11 N/A. Deployment outside the repository can be scoped out only when the available evidence supports that boundary rather than merely failing to reveal a file.

Confidence is reported as high, medium, or low with a reason; there is no hidden numeric confidence formula. Strong repository evidence plus visible provider state can support high confidence. Solid static evidence with missing provider history may support a lower confidence result and skipped supporting criteria. A failed target resolution, access denial, or clone failure prevents a score entirely. A missing previous audit does not: the independent current result still publishes and becomes the next baseline.

Root guidance can explain scope, but it cannot authorize execution or compel a conclusion. The audit also cannot certify test quality, dependency safety, secure workflow code, general configuration quality, or container hardening through CI/CD evidence alone. Those remain separate audit questions even when they touch the same files.

The practical next verification depends on the actual gap: obtain read-only protection metadata before calling gates non-blocking; inspect a failed reference before blaming the provider; confirm who owns an external deployment before scoring it absent; and rerun after a relevant revision changes. That is more useful than filling uncertainty with generic best practices.

Where the audit stops and improvement begins

The audit ends with an ordered Improvement Checklist and, when evidence is strong enough, bounded candidates for future work. It does not carry out that work. Each candidate needs a concrete scope, supporting evidence, priority, maturity step, reason, acceptance signal, confidence, blast radius, and a safe first action. Pure visibility gaps and low-confidence guesses stay in Limitations rather than becoming changes to make.

Ordering matters because delivery improvements depend on one another:

1. Establish simple, runnable repository commands.
2. Create or repair syntactically and referentially valid CI configuration.
3. Wire real build, lint, typecheck, test, and repository-specific checks.
4. Make the primary gate block merge or release.
5. Complete the gate composition for every active deployable unit.
6. Stabilize references, remove dead paths, and then optimize feedback.
7. Automate the relevant artifact, image, deploy, release, promotion, and rollback path.

A repository that lacks a blocking test gate should not begin with cache tuning. One with no credible command should not begin by adding an elaborate provider matrix. The audit can recommend creating a pipeline, adding or blocking a check, pinning references, repairing a broken job, reducing redundant work, wiring an existing container path, or automating a release or deployment—but the recommendation stays proportional to evidence.

Branch protection, release, deployment, promotion, and rollback are permissioned, high-blast-radius surfaces. They usually need issue-only or high-review handling, provider access, an accountable owner, and an explicit rollback plan. Even a narrow workflow edit can change what reaches production. Human review and approval are therefore part of the boundary, not ceremony added after the fact.

The current CI/CD metric has no paired Autofix action and the article shows no Improvement Steps control. Guard may preserve bounded context for a future CI/CD improvement consumer, but a handoff is not write permission. Any later executor must revalidate the repository identity, current revision, permissions, evidence, and blast radius before it creates an issue, branch, commit, pull request, protection rule, release, or deployment. The audit itself never merges or deploys.

How reruns support maintained delivery health

A CI/CD score belongs to one audited commit. A new workflow, changed package script, provider rule, release path, or rollback procedure can make yesterday's evidence stale. Rerun evaluates the changed or current repository state and publishes a new completed result. It does not overwrite an old report or replay a historical provider pipeline. Automatic audit runs may react to changed commits on a cadence, but that recurrence is still an audit—not an automatic code or delivery change.

Guard drafts and validates the current result before reading previous data. If no valid previous locator exists or no prior result can be read, the current result remains unchanged, no delta is invented, and the run becomes the next baseline. If only a prior report is readable, Guard may describe supported substantive differences but cannot populate the detail-card delta. That delta requires a validated prior summary with its score. When both current and usable prior evidence exist, the report can distinguish unchanged, new, and resolved findings and explain the confidence of the comparison.

The previous score is an anchor, not truth. Repository changes, corrected evidence, changed scope or guidance, and material methodology changes can justify movement; reworded prose or unrelated commits cannot. In the fictional history below, 64 becomes 68 after one release check becomes required. The floating reference and rollback gap remain, so the label stays Mixed protection.

Repetition is what turns the audit from a report into a health mechanism. Each relevant revision re-establishes whether real checks still protect the branch and delivery path, whether a former limitation is now observable, and whether an accepted improvement had the intended effect. That evidence helps coding agents build new features on a path whose ability to reject unverified changes is checked again as the project evolves.