Security audit: what Guard checks and how risk affects the score

What question does the security audit answer?

The audit asks a revision-bound question: which reachable trust-boundary failures or unsafe patterns are supported by concrete repository and explicitly linked website evidence, how severe and well-supported are the deduplicated root causes, and what should be verified or improved next?

“Reachable” matters. A suspicious helper, permissive option, or package declaration is a lead until the audit connects it to a request, user, privileged process, data path, or delivery environment. Five symptoms produced by one missing authorization check become one root cause; one quiet configuration line can matter when it exposes a privileged install path.

The result belongs to the exact commit Guard inspected. It neither predicts the next commit nor certifies the product. It shows an owner where the repository left the green zone, the evidence behind that conclusion, and the first bounded change to consider.

What Guard checks

Guard begins with repository identity and architecture, not a generic vulnerability checklist. It records the provider-neutral repository path, default branch, full commit, available history, technologies, and boundaries between clients, servers, data stores, queues, files, outbound requests, workflows, and deployment infrastructure. Manifests, lockfiles, configuration, tests, local history, reports, and documentation can contribute evidence.

The coverage matrix below has twenty areas. They are not twenty scoring criteria. Each is recorded as checked, not applicable, or needing verification. “Not applicable” requires evidence—for example, a service with no browser session surface—not the absence of an obvious file name.

1. Secrets and environment handling: committed values, unsafe defaults, examples, logs, workflow inputs, and the paths by which credentials reach a runtime or delivery job.
2. Authentication: sign-in, session, token, invite, recovery, and logout behavior, including at least one representative read and write path.
3. Authorization: permission checks at handlers and data access, with attention to identifiers supplied by an authenticated but unauthorized user.
4. Tenant isolation: owner, organization, project, or account filters carried from the request boundary into reads and writes.
5. Input validation: server-side schemas, parsing, size and type limits, and places where client validation is trusted without an equivalent server check.
6. Injection: user-controlled values reaching SQL, shell, templates, interpreters, search syntax, or other executable query boundaries.
7. Cross-site scripting: HTML, markdown, Mermaid, rich-text, link, and client rendering paths, including raw-HTML escape hatches.
8. CSRF and CORS: cookie-backed state changes, origin policy, credentialed requests, preflight behavior, and the assumptions connecting them.
9. Live web security headers: passively observable CSP, HSTS, frame, referrer, permissions, content-type, and related response controls on an explicitly linked site.
10. Live web exposure: public status, redirect, title, cookie, debug, version, stack-trace, and same-origin asset signals that can be checked without probing.
11. Server-side request forgery: user-controlled URLs, hosts, callbacks, import sources, and redirects crossing an outbound network boundary.
12. Files and storage: upload, import, export, download, path construction, object keys, content types, access control, and retention-sensitive storage behavior.
13. Dependencies: visible versions, lockfiles, security-related scripts, install hooks, and repository-backed risk without unsupported advisory claims.
14. Supply chain: CI actions, images, remote scripts, provenance and integrity controls, and third-party code entering build or delivery paths.
15. Unsafe package acquisition paths: the exact install, update, build, test, or bootstrap definition that can resolve and execute public package code.
16. Cryptography: key use, randomness, password hashing, signature verification, encryption modes, and custom cryptographic constructions.
17. Logging: credentials, tokens, personal data, request bodies, error details, and security-relevant events that are missing or overexposed.
18. Repository/application header configuration: the tracked application or infrastructure source that is meant to set security headers, distinct from what a linked site currently returns.
19. Rate limits: abuse-sensitive authentication, recovery, invite, export, webhook, and expensive request paths, including where limits are enforced.
20. Security tests: tracked tests for authentication, permissions, validation, tenant isolation, and known security boundaries; definitions are read, not executed.

Coverage has to follow the path

Labels alone do not establish coverage. For authentication, authorization, and tenant isolation, Guard traces at least one representative read and one write from the external input through the handler and into the data boundary. For validation, it compares server enforcement with client assumptions. For XSS, it follows markdown, Mermaid, rich-text, links, and raw HTML into the renderer. For SSRF and file access, it follows controlled strings until they cross a network or disk boundary.

Secret review includes examples, documentation, CI, and committed configuration, but evidence names the path and key rather than reproducing a value. Package acquisition is similarly concrete: Guard reads the exact install or update command, frozen-lockfile and integrity behavior, whether fresh public versions can be selected, any age gate or quarantine, lifecycle-script controls, and whether the job can access repository, release, publishing, cloud, or deployment credentials. The package manager's name is context, never the finding by itself.

What the audit deliberately does not do

The Security audit is analysis-only. It does not edit files or settings, install dependencies, run package-manager resolution, execute lifecycle hooks, build the application, run tests, invoke repository-provided scanners, create an account, open an issue or pull request, or produce an exploit artifact. A tracked command can be inspected as text without being trusted enough to execute.

An explicitly linked website permits a small passive observation, not a pentest. Guard may send safe HEAD or GET requests to the exact supplied URL, follow normal same-origin redirects, and record HTTPS behavior, status, final URL, response headers, cookie properties, page title, and directly loaded same-origin public assets. It does not submit forms, try login or reset flows, fuzz inputs, spray payloads, brute-force paths, crawl unrelated pages, follow third-party origins, or increase request volume to prove impact.

The repository boundary is equally specific. node_modules, vendored trees, virtual environments, target, dist, build, .next, .nuxt, cache directories, coverage, .git, logs, and temporary files are excluded unless one is directly material. Public advisories are not queried unless that evidence was explicitly authorized, and issue or review-request APIs are not silently used as history.

Security-relevant workflow facts remain in scope: unsafe permissions, secret exposure, workflow injection, unpinned actions, and privileged dependency acquisition can create a concrete risk. General pipeline existence, branch protection, release blocking, and delivery automation belong to the CI/CD audit. Likewise, Security owns leaked values and unsafe secret defaults; ordinary configuration duplication, option design, and unwired templates belong to Configuration hygiene.

How the audit works

1. Fix the target and map the trust boundaries

Guard first proves repository access, resolves the full commit, records history limits, and inventories the security surface. It locates authentication and session code, authorization and tenant filters, input and upload paths, manifests and lockfiles, security-sensitive workflows, deployment files, infrastructure, and only the linked website origins supplied with the task. Optional existing project reconnaissance can help orientation when it matches the same repository, but current files remain the evidence. Missing or mismatched context is disclosed as a limit, not converted into a finding.

2. Trace current evidence and calculate the independent result

The analysis maps sources to security-sensitive sinks. It reads source, manifests, lockfiles, configuration, workflows, documentation, tests, and available local history; performs only the bounded exact-origin web observation described above; and uses external advisory evidence only when explicitly authorized. No branded scanner is assumed. A reconnaissance hint or file-name match can direct attention, but cannot confirm a finding without current reachability and impact evidence.

Findings are classified, confidence-gated, and deduplicated before severity counts enter the score. The complete current report and compact summary are drafted from this revision before any previous audit is read, so an old number cannot steer the current judgment.

3. Publish the report, then reconcile a usable previous result

The saved report gives a plain-language current answer, audit target, checked surfaces, findings, severity arithmetic, evidence, limitations, and an ordered improvement checklist. Only after that independent draft exists may Guard validate and read one previous result. If the prior source, target, or revision cannot be established, the current result becomes the baseline. If only part of the prior result is usable, the comparison is partial and says so.

How evidence becomes a finding

A finding begins with a security-sensitive surface and a trace, not a verdict attached to a keyword. Guard records the source an attacker or ordinary user can control, the sink or privilege boundary it reaches, the trigger, why the current code makes that route reachable, its impact, and the exact path, line, or stable symbol needed to revalidate it.

The normalized finding also carries a short title, affected area, severity, confidence, safer direction, and a concrete recommendation. Evidence snippets stay short and redact secret-looking values. A missing header seen on a linked site is connected back to tracked application or infrastructure configuration when possible; if the repository connection cannot be established safely, that uncertainty remains visible.

Confirmed risk, hardening, and open questions are different outputs

A confirmed finding needs evidence of both risky behavior and reachability. A hardening observation may describe a worthwhile defense-in-depth change without establishing immediate impact. An open question names the verification that safe static or passive work could not complete. Low-confidence observations remain in limitations or verification work and do not quietly inflate the severity chart.

One root cause is counted once

If the same missing project-membership check appears in an export route, a background job, and a shared helper, the report may name representative locations, but the score counts the underlying authorization root cause once at the highest supported severity. Guard does not invent an advisory-alias system or a waiver mechanism for this audit, and an accepted-risk note cannot suppress a currently reachable issue by itself.

The improvement view is not a second finding model

A useful finding can also produce an ordered candidate with a stable identifier, bounded paths or components, weakness, evidence, and change blast radius. Candidate order favors user value, confidence, evidence strength, and likely safe fixability—not merely the smallest edit. Blast radius describes the risk of changing the code; it is not another name for finding severity. The candidate records audit-time evidence and must be checked again before any write.

Severity, confidence, and the absence of a criteria scorecard

Security uses four categorical severities. They describe supported impact and reachability, not the number of matching files:

• Critical means likely direct compromise, exposure, destructive action, or broad tenant failure. For package acquisition it requires a concrete privileged CI, build, release, deploy, or bootstrap path where freshly resolved public package code can execute while core controls are absent: deterministic locking or equivalent pinning, an age gate or registry quarantine, and lifecycle-script control. A privileged path that can select a newly published package and run its lifecycle code without an age gate also qualifies unless another evidenced control blocks the fresh version.
• High means a credibly exploitable weakness with meaningful impact. It can also describe a deployable path that resolves floating public versions before release without deterministic pinning when secret access or unreviewed lifecycle execution is not established, or a privileged frozen-lockfile update path that can accept a newly published version without quarantine before review or deployment.
• Medium means a real weakness that needs conditions or has limited impact. Range-based dependencies, a missing exact-version policy or committed lockfile, a non-frozen install, or incomplete package hardening stay here when evidence does not establish a release-impacting privileged path that executes fresh unreviewed code.
• Low means hardening or defense-in-depth weakness with limited immediate impact.

Package-manager choice, a semver range, a missing lockfile, non-frozen installation, or a missing age-gate setting is never Critical by itself. Global lifecycle scripts matter to the Critical rule only when they participate in the evidenced privileged fresh-code path. Missing trust-policy or exotic-subdependency hardening is ordinarily a warning unless current evidence connects it to an untrusted downgrade, exotic source, or that privileged acquisition path.

Confidence controls what enters the arithmetic

Confidence is high, medium, or low. The runbook does not attach numeric thresholds to those words. Only high- and medium-confidence root causes enter the severity counts and score. The gate applies to every row, including Low: the Low count means sufficiently supported Low findings, not every hardening note in the prose.

There is no 0–5 criteria rubric, weighted scorecard, earned-points model, pass percentage, or numeric confidence formula. The twenty areas define coverage. Deduplicated severity counts that clear the confidence gate define the score.

How the final score is calculated

Let C, H, M, and L be the numbers of deduplicated Critical, High, Medium, and Low findings with at least medium confidence. Guard starts at 100 and subtracts a fixed amount for each root cause.

raw score = 100 - (C × 45) - (H × 16) - (M × 6) - (L × 2)

if C > 0:
  score = min(raw score, 39)
otherwise:
  score = raw score

score = clamp(score, 0, 100)

The inputs and deductions are integers, and the result is an integer. The runbook defines no rounding step. One or more Critical findings impose the explicit ceiling of 39. A High finding has no automatic cap; its 16-point deduction already affects the score. The final clamp keeps any result between 0 and 100.

A genuinely severe case may receive a lower evidence-backed result, but the runbook does not define a second trigger or arithmetic rule for that exceptional judgment. Guard must disclose such a decision in the report. The example below uses no exceptional adjustment, and the article does not invent one.

Display bands and unavailable results

• 70–100: good; the product label is Healthy.
• 40–69: warn; the product label is Some issues.
• 0–39: bad; the product label is Needs attention.

A score is unavailable when the repository input is missing or ambiguous, access or clone fails, the exact revision cannot be resolved, or the evidence required for the final result cannot be produced. Guard reports a diagnostic instead of manufacturing a successful summary. Missing history, a missing prior audit, an unavailable linked site, or optional context that does not match the repository can remain disclosed limitations without erasing a repository-backed current score.

A high score without confirmed Medium-or-higher findings still does not certify the repository as secure. It states only what the bounded audit supported at that revision.

A worked example: 68 out of 100

The fictional northstar/payments-api run contains no Critical finding, one High, two Medium, and two Low deduplicated findings. Every counted item has at least medium confidence.

100 - (0 × 45) - (1 × 16) - (2 × 6) - (2 × 2)
= 100 - 0 - 16 - 12 - 4
= 68

No Critical ceiling applies, the clamp changes nothing, there is no fractional value to round, and no exceptional adjustment is used. The same arithmetic explains the fictional history: 84 is one High finding; 76 is one High, one Medium, and one Low; 68 is one High, two Medium, and two Low. Time did not cause the decline—the underlying evidence changed.

How to read the result in Guard

The Security card is the compact orientation surface. It shows the 0–100 ring, one of the exact labels Healthy, Some issues, or Needs attention, and four severity chips named Critical, High, Medium, and Low. A zero count is an em dash, as in the fictional Critical chip above, rather than a numeral zero. The card can also expose the distinct actions Autofix, Auto-pentest, and Rerun.

The red edge on the fictional card represents its worst supported signal—the High finding—while the yellow ring and Some issues badge represent the final score band at 68. Those views are not contradictory. One answers “what is the most serious evidence?”, and the other answers “where did the total score land?”

Opening the audit reveals the exact branch and commit, current report, evidence, limitations, arithmetic, and improvement checklist. Reports marks the report list, Current identifies the newest report in that list, and Copy and Download act on that saved report. The Audit history chart runs oldest to newest on a 0–100 axis. Each point describes its own revision, not a score recalculated from today's date.

Comparison begins only after Guard independently drafts the current result. A validated prior report may support unchanged, new, or resolved root causes and an explanation for movement from repository change, corrected evidence, changed scope or limitations, material guidance, or methodology—not cosmetic rewriting. Without usable prior data, the current audit is the baseline.

The product may add Out of date after seven days, and it may disable a rerun when the selected repository version has already been audited. Freshness and action availability are interface rules. Neither silently changes the historical score or the revision it describes.

Where the audit stops, autofix begins, and Auto-pentest differs

The Security audit ends with read-only evidence, a score, limitations, and ordered improvement candidates. It does not create an issue, branch, commit, pull request, or fix. Current product configuration exposes vulnerability Autofix because the published vulnerability-improvement action points back to Security; the audit itself does not declare a paired writer. Auto-pentest is published separately and is not that paired improvement.

Vulnerability Autofix is issue-first and human-gated

Before writing, the follow-on work revalidates the candidate against the current default branch, current issues and pull requests, duplicates, scope, safety, and available verification. An ambiguous, stale, broad, unsafe, or unverifiable candidate can stop for human review or end with a report and no repository write. When write access is allowed and a current candidate passes the duplicate, safety, and verification checks, both supported modes are issue-first: they create or reuse exactly one issue. Issue-only mode ends there. Issue-and-PR mode may add one unmerged review request only when the change is current, bounded, useful, duplicate-free, safe, and verifiable. Failed verification means no pull request; direct default-branch writes, auto-merge, and automatic acceptance remain forbidden.

Selecting a row in the interface expresses a preference, not a promise that the same root cause will receive a patch. The follow-on run rechecks the ordered current queue and may select a different safe candidate or produce no write. A person remains the acceptance gate. Even a created pull request does not change the displayed Security score; the change must be reviewed, merged, and audited at a new revision.

Auto-pentest is active, consent-gated work against one site

Auto-pentest performs attack emulation against exactly one explicitly linked site per run. Before requests begin, a person selects the target and explicitly accepts disruption, ownership, and retention terms. It may run once or on a schedule. When an issue would add value, issue mode creates, reuses, or comments on the primary-risk issue and opens no pull request. A duplicate, already resolved, non-actionable, or access-limited result—or one with no useful new information—can instead finish without an issue, comment, or pull request. PR mode may use a different bounded safe-fix candidate from the primary risk and may open one unmerged review request. This is a separate security-group action—not a hidden phase of the read-only repository audit and not another name for vulnerability Autofix.

Limits, reruns, and maintained security health

A defensible Security result states what could still change the conclusion:

• Static repository evidence can prove a reachable code path, but the audit does not execute the application, exploit the path, or certify production behavior.
• Linked-site evidence is limited to safe observation of exact supplied URLs and normal same-origin behavior. Authenticated areas, form submissions, active probes, and unrelated origins remain untested.
• Public advisory evidence is absent unless explicitly authorized. A visible version may support acquisition and hardening analysis without supporting a CVE claim.
• Low-confidence observations remain verification work and do not enter the severity arithmetic. A clean count is therefore bounded by available evidence and confidence.
• Repository guidance can clarify paths and constraints, but it cannot expand live scope, grant writes, force a score, or hide a reachable risk. Its absence is neutral.
• A successful end-to-end run requires both a usable current result and a valid improvement handoff tied to the same project. If Guard cannot preserve that boundary safely, it reports failure rather than pretending the complete handoff succeeded.

Rerun when the evidence changes

The result describes one exact commit. Rerun after relevant code, configuration, dependency, infrastructure, linked-site scope, or verification changes. A prior result is used only when its source, target, and revision context can be validated; no previous result is a normal baseline, not a failure. A partial prior result supports only a partial comparison with the missing evidence disclosed.

An issue, branch, or unmerged review request is evidence that work started, not evidence that risk changed. A fix affects the score only after merge and a new audit. A disclosed correction to evidence, scope, guidance, or methodology can also move an assessment without pretending the repository changed. Later code can introduce a new root cause or widen an old path, so a score may improve, stay flat, or decline.

Green is a maintained state

The useful outcome is not a report filed away after one run. Recurring audits detect when a project leaves the green zone; focused, human-approved improvements move it back; and a rerun verifies the state that coding agents will inherit. Keeping that loop active means new agent-written features are less likely to build on an authorization gap, unsafe acquisition path, exposed secret, or unverified boundary that the repository already contained.