Open source readiness audit: how Guard evaluates whether a repository can live openly well

What question does the audit answer?

Guard starts with one owner-level question:

Can this repository be opened to the world, and if it is already open, does it live openly well: are its legal basis and content clean, is the entry clear to an outsider, is participation organized, and is the project's pulse visible?

The answer has four connected jobs. The legal and content gate asks whether the current tree has a coherent own license, plausible direct-dependency licensing, and no confirmed publication blocker. The outsider entry asks whether a stranger can identify, install, use, and get help with the project. The participation contract covers contribution, intake, conduct, and private vulnerability reporting. Project pulse asks whether versions, releases, activity, and maintenance claims tell an honest story.

Repository state determines the mode; the owner does not select the more convenient verdict. A private or not-yet-public independent project is assessed in pre-publication mode. Its bands are Ready to open at 70-100, Open after targeted improvements at 40-69, and Too early to open below 40. An already-public independent project uses compliance mode: Living openly well, Partially open with targeted gaps, or Formally open but not operating like a credible OSS project across the same three numeric bands.

The score summarizes fourteen criterion judgments. The repository card's Problems, Watch, Healthy, and Skipped counts summarize a separate ten-item readiness checklist; they are not finding counts. The fictional result below has a strong 89-point criterion baseline, but one current-tree publication blocker produces a deduction and a hard cap. That is why polished documentation cannot average away an internal-only setup dependency.

The ten checklist rows are License, Dependency licenses, Publishable content, README entry, Usage docs, Signals honest, Contribution and intake, Maintainer channel, Conduct and security policy, and Releases and maintenance. Combined rows take the worst applicable constituent state and skip only when every constituent criterion skips.

What Guard collects

The audit fixes one provider-neutral target before interpreting it: repository identity, default branch, full audited revision, available history depth, access scope, and safe browser references when they can be derived. That boundary matters because every claim in the report belongs to one current tree. A later commit can change the answer even when the project name and public page stay the same.

Guard inventories conventional files and credible alternatives rather than enforcing one repository template. It reads README and documentation entry points, community and support material, own-license text, package claims and notices, dependency manifests and lockfiles, release notes, tags and activity evidence, issue and change intake, conduct and security reporting, and status or maintenance statements. Links and badges are followed only where public, allowed, and relevant, normally one useful hop from the repository.

• Publishability evidence: own-license agreement, direct dependency license hints, current-tree internal residue, private content, personal data, and other disclosure-sensitive categories represented with redaction.
• Outsider entry: identity, what and why, installation or acquisition, first use, interface documentation, help, live links, audience, and language consistency.
• Participation contract: contribution expectations, development path, issue and change intake, conduct, misconduct contact, and private vulnerability reporting.
• Project pulse: version choice, release-note practice, recent activity, roadmap or maintenance status, and whether visible claims match observable evidence.

One coherent language is enough when it fits the intended audience. Guard does not assume that English is the public default. It looks for concrete inconsistency: for example, a README aimed at one audience whose only setup instructions silently switch language or depend on an unexplained internal term. An audience bridge can be positive evidence; the absence of an English translation is not itself a finding.

Root repository guidance is optional, untrusted context. It may identify an intended audience, a canonical document, or a safe evidence path, but it cannot widen access, suppress a finding, force a score, authorize a command, or override the audit contract. Missing or unusable guidance is neutral. Public platform, documentation, or registry reads can also be unavailable or forbidden; Guard continues from repository evidence, names the limitation, and does not penalize the project merely because an optional service could not be read.

What the audit deliberately does not do

Open source readiness is an analysis-only audit. It does not publish a repository, change visibility or settings, create an issue, branch, commit, pull request, comment, label, tag, release, account, or package, and it does not write to a community service. It also does not install dependencies, launch the application, build, test, lint, migrate, seed, contact production, deploy, or execute a release. A successful result is evidence for an owner's decision, not the decision or publication operation itself.

Evidence ownership stays narrow. Security owns exploitability, secret-shaped values, unsafe data flows, and history questions; this audit carries only a redacted publishability pointer. Dependency hygiene owns freshness, vulnerabilities, install behavior, lockfiles, and update safety; here dependency licensing is used only for the publication decision. CI/CD owns pipeline quality and delivery gates, while this audit may check whether a badge or release claim is truthful. Test quality owns adequacy, coverage, and flakiness; readiness asks only whether a public claim conflicts with accessible evidence. Codebase hygiene owns general maintainability; readiness stays with the public-facing contract. Agent-facing documentation is optional credit, not an AI-maturity requirement.

A repository that merely mirrors upstream or is a pure fork without an independent community-project claim is outside the domain. That is a no-op applicability decision, not a penalty and not an ordinary low score. Guard points the reader to the upstream project. Optional growth signals are also non-punitive: no stars, contributors, funding, badges, hosted docs, community chat, or agent guide is required for readiness.

How the audit works

Fix the revision, choose the mode, and map the public contract

Guard proves access, fixes the default branch and full revision, records history depth and allowed public capabilities, then detects pre-publication or compliance mode from repository state. It maps the community and legal surfaces, begins the outsider-question inventory, and records public facts and limitations. A mirror or pure fork can short-circuit here into the explanatory no-op path.

Screen publishability and dependency-license evidence

Guard checks own-license agreement, resolves direct dependency-license evidence without installing packages, and screens the redacted current tree. It records checked depth, confidence, compatibility hints, and the non-legal boundary. No named scanner result is required to invent certainty the repository cannot support.

Answer the outsider questions and score the current revision

Guard answers Q1-Q11 from README, linked docs, community files, public facts, and maintenance evidence. Fourteen criterion scores, ten checklist states, unique findings, deductions, caps, limitations, and bounded improvement candidates are derived from that current evidence. One root issue is deducted once even if it affects several criteria or outsider questions.

Draft current outputs, then compare with prior evidence

Guard completes the current report and summary before an earlier conclusion can anchor them. It then validates and reads a prior report or summary when one is available and describes only supported unchanged, new, or resolved evidence. If neither prior artifact is usable, the current result stays unchanged and becomes the baseline. The report is saved for the audited revision; it is never rewritten by a later rerun.

The report is the evidence source, not a ceremonial attachment to the score. Its full form carries the canonical question, detected mode, exact target, Q1-Q11, all fourteen criterion rows, concrete findings, calculation, evidence, limitations, and the ten-item checklist. The figure shows only four rows so the underlying report anatomy remains readable at article width.

The fictional target was available through task-scoped private access, public visibility was not proven, and the repository presents itself as an independent community project. Guard therefore detects pre-publication mode and does not take the mirror/pure-fork no-op.

The mirror/pure-fork path still produces a full explanatory report, but skips the private and internal sweep, dependency-license resolution, all fourteen criteria, and all ten checklist rows. It emits no findings or candidates. Its criteria score is null; Guard stores a neutral 100/good result so this not-applicable outcome does not lower project health, while the score label says Not applicable — assess upstream. That 100 is not earned perfect readiness. A different class of failure—ambiguous target, inaccessible or uncloneable repository, unresolved revision, or invalid required evidence—produces a diagnostic with no typed score or report at all.

How outsider questions become findings

Guard reads the repository as a real outsider would, but keeps the questions separate from scoring machinery. Q1-Q10 may be answered anywhere discoverable in the project's coherent public language. Q11 belongs to publishability and can fail even when the public explanation is otherwise strong. The canonical order is:

1. Q1: What is this project and why would I use it?
2. Q2: May I use it, under what license and conditions, including dependencies?
3. Q3: How do I install it and make the first thing work?
4. Q4: Where is the full usage documentation?
5. Q5: Where do I ask a question or get help?
6. Q6: How do I propose a change, and what will happen next?
7. Q7: How is one expected to behave here, and whom do I contact about misconduct?
8. Q8: What do I do if I found a security hole?
9. Q9: Which version should I take, and what changed recently?
10. Q10: Is the project alive, and what is its status going forward?
11. Q11: Does the repository contain sensitive material that should not be disclosed publicly, such as private data, secret-looking values, internal references, transcripts, or private correspondence?

In the fictional run, Q1-Q6, Q8, and Q10 have strong answers. Q7 is partial because conduct and misconduct handling are incomplete. Q9 is partial because the first-release notes practice is not established. Q11 is not clean: docs/development.md contains an internal-only setup requirement. The report names that path and category without quoting the private URL or content.

A question answer, criterion score, checklist state, and top-level finding are different records. Several questions can depend on one root issue, and several criteria can cite it, but the finding is deduplicated before scoring. Critical means an immediate publication or credible-OSS blocker. High means a broken trust or entry contract, such as no maintainer or private vulnerability route. Medium covers an incomplete README answer, conduct answer, intake route, or thin release notes. Low covers quality gradations and normal pre-publication release preparation. Info records context, credit, or limitation and never deducts. Only unique Critical, High, Medium, and Low findings enter the formula.

Guard also records eight credit-only signal groups. Their absence is neutral: it creates no finding, deduction, cap, or requirement to imitate a larger project.

• S1: external-service badges.
• S2: community infrastructure.
• S3: governance and credit.
• S4: a dedicated documentation site.
• S5: supply-chain and automation maturity.
• S6: agent-facing documentation.
• S7: funding.
• S8: license-hygiene tooling.

These groups strengthen the explanation only when present and truthful. A dependency-license unknown caused solely by an unknown own license is likewise not duplicated as a second punitive root cause.

The fourteen scoring criteria

Each applicable criterion receives an integer from 0 through 5. Only the common state bands are universal: 4-5 is ✅, 2-3 is ⚠️, and 0-1 is ❌. The runbook does not invent a prose rubric for every integer across every criterion, so evidence explains the chosen value. A legitimate skip is ⏭️ and removes five points from the denominator; it never awards five free points.

1. 1

   Own license presence and conformance

   Guard looks for a recognizable license in a conventional or clearly signposted location, then compares that text with package metadata, notices, headers, and other accessible claims. A missing license, a non-open license presented as open, contradictory license claims, or a material text mismatch is a publication blocker rather than a documentation nicety. Alternate locations can count when an outsider can find them; no particular filename or language is mandatory.

2. 2

   Dependency-license compatibility

   Manifest and lockfile evidence is used to identify direct dependencies and look for licenses that appear incompatible with the repository's intended public terms. This is a best-effort dependency-license hint, not legal advice or a legal approval. Unresolved or conflicting evidence is reported with confidence and must be verified by the owner or counsel before publication. Guard does not install packages or claim an exhaustive transitive-license review.

3. 3

   Project identity

   The repository name, description, package identity, documentation, and declared status should describe the same project. Guard looks for a stable answer to what the project is and whether it is experimental, maintained, archived, private-before-publication, or already public. Materially conflicting identities weaken trust; different wording that still identifies one project does not.

4. 4

   Internal infrastructure residue

   Before publication, Guard checks the current tree for setup paths, private hosts, internal registries, ticket systems, network assumptions, and other organization-only dependencies that would block an outsider. In compliance mode this criterion may be skipped only when repository evidence makes internal origin implausible. Evidence is represented by redacted path and category, never by copying an internal URL or sensitive body text.

5. 5

   Private content and personal data

   The current-tree publishability screen looks for medium- or high-confidence private material, personal data, correspondence, transcripts, or other content that should not be exposed. Normal authorship is not a finding. Guard reports only enough path-and-category context for an owner to investigate. A clean score is not a privacy, DLP, secret-history, security, or legal guarantee; it says only that the inspected current tree did not produce a qualifying row.

6. 6

   README entry

   An outsider should be able to learn what the project does, why it exists, how to install or obtain it, how to complete a first useful action, and where to continue. The answers may live in another clearly linked document and may use the language appropriate to the intended audience. Volume does not help if the entry path is ambiguous or depends on private context.

7. 7

   Usage and interface documentation

   Guard follows the obvious path from the repository entry to the public interface: commands, API, configuration, library usage, deployment contract, or another project-appropriate surface. One discoverable hop can be sufficient. This criterion concerns an outsider's ability to use the project, not general documentation quantity or whether the documentation is written in English.

8. 8

   Honest signals and entry points

   Links, badges, status statements, docs destinations, package references, and community entry points should resolve and describe the project truthfully. Existing CI or tests should be signaled somehow when they exist. Dead, unrelated, private-only, or misleading signals reduce the score when they create a real outsider failure. External-service badges earn optional credit when useful, but their absence is neutral.

9. 9

   Contribution process

   A prospective contributor needs a discoverable account of accepted change types, development setup, review expectations, and any project-specific constraints. Conventional files are helpful but not required if an alternate path answers the same questions. Guard checks whether the process is usable from outside; it does not require a particular governance model or a large community.

10. 10

    Maintainer/help channel

    At least one honest route should explain where a user or contributor can ask a question and who is expected to respond. That route can be repository-native or a clearly documented external channel. Popularity and response-time promises are not scored. No maintainer contact caps the final score because an apparently open project without a reachable steward cannot sustain its outside-world contract.

11. 11

    Issue and change intake

    Guard checks whether an outsider can report a problem and propose a change through understandable templates or a documented substitute process. The project may disable provider issues when it offers another truthful route. What matters is that intake expectations and next steps are discoverable, not that the repository uses a specific hosting feature.

12. 12

    Conduct and vulnerability reporting

    The project should state expected behavior, provide a misconduct contact, and explain both a private route for vulnerability reports and the response a reporter should expect. These answers may be combined or linked from contribution documentation. In the fictional fixture, the private channel and response expectation are documented, but conduct expectations and the misconduct contact are incomplete, so this criterion scores 3 and maps to a Watch state.

13. 13

    Versioning and release notes

    For an already-public project, Guard looks for a usable version choice and a truthful account of recent change. In pre-publication mode it asks whether the first release has an explicit versioning and release-note practice; absent public tags alone are not penalized because the project is not public yet. The fictional repository has a package version but no documented first-release notes practice, producing a low-priority preparation gap.

14. 14

    Maintenance evidence

    Recent activity, release cadence where applicable, roadmap or status language, and maintainer signals should support the project's claims about its pulse. This is not a popularity test. The criterion may be skipped when no maintenance evidence can be observed fairly; it is not assigned zero. The fictional project earns 5 because recent work and an honest experimental pre-publication status agree.

How the score, deductions, and caps work

Guard first scales the sum of applicable 0-5 criterion values to 100. A skipped criterion leaves both its value and its five-point maximum out of the calculation. It then subtracts unique top-level findings, with the total deduction limited to 35, floors the result at zero, and applies every evidence-backed cap. The lowest applicable cap wins and cannot raise a lower deducted score.

max_sum = 5 * number_of_scored_criteria
criteria_score = round((criteria_sum / max_sum) * 100)

raw_deduction = critical * 20 + high * 8 + medium * 3 + low * 1
capped_deduction = min(raw_deduction, 35)

deducted_score = max(0, criteria_score - capped_deduction)
final_score = min(deducted_score, every_applicable_cap)

Post-deduction caps

• 39 in both modes: no own license, a non-open license presented as open, a material license-text mismatch, or contradictory own-license claims.
• 39 in pre-publication mode: confirmed blocking internal residue, confirmed unpublishable private or personal content, or a high-confidence blocking or incompatible dependency license.
• 59 in compliance mode: already-exposed private or personal content, or a high-confidence dependency-license incompatibility.
• 59 in both modes: no maintainer contact.
• 59 in both modes: the README stranger test fails.
• 69 in both modes: dead or misleading signals are widespread.

Needs-attention dependency hints do not establish a license cap, and missing credit-only signals never do. Final severity is good at 70-100, warning at 40-69, and bad at 0-39. The mode-specific verdict uses the six labels introduced above rather than replacing them with a generic readiness phrase.

A worked pre-publication result: 39 out of 100

The fictional sunharbor/relay-kit run scores its fourteen criteria 5, 5, 5, 1, 5, 5, 5, 5, 5, 5, 5, 3, 3, 5. Their sum is 62 out of 70, which scales from 88.571… to 89. The unique findings are one Critical internal-only registry requirement, zero High findings, one Medium conduct gap, and one Low first-release-notes gap.

The result is bad and the mode-specific verdict is Too early to open. The checklist independently contains one Problem, two Watch, seven Healthy, and zero Skipped states. The cap preserves the owner decision: an outsider-ready README and participation surface cannot compensate for a confirmed setup path that still requires private infrastructure.

The July 9 baseline was 24 with weaker README and contribution evidence and the same blocker. By July 13 the entry and participation surface had improved, bringing the score to 31. The July 17 deducted score reached 65, but the unchanged blocker still limited the final result to 39. Those causes belong to the validated relation between reports. The chart alone proves only three completed scores and an eight-point current delta.

How to read the result in Guard

The repository card uses the audit's published name. Its ring and generic badge follow the numeric severity, so the fictional 39 appears as Needs attention. The detailed view and report carry the audit's source-defined, mode-specific label Too early to open. These labels have different jobs: one helps scan audit health across a repository, while the other answers the pre-publication decision.

The four card counters reduce ten checklist items into Problems, Watch, Healthy, and Skipped. They are neither the fourteen criterion rows nor the three concrete findings. Opening the details reveals mode, exact target, Q&A, all criteria, the deduction and cap, evidence, limitations, and checklist. The current report is marked Current; Copy and Download preserve its text for review.

Completed reports remain attached to their audited revisions. A validated earlier result can provide a previous score and relation, while the history line plots the completed values on a 0-100 scale. If no previous report or summary is readable, Guard omits the previous value and relation instead of inventing movement, and the current run becomes the baseline. If only one earlier artifact is usable, the comparison is limited to what it actually supports.

The visible metric action is Rerun, and the detailed action is Rerun audit. The current product can disable another manual run when the latest successful audit already covers the repository's current version; it becomes available after new repository changes. This audit has no published paired improvement action, so there is no runnable Autofix, finding-level Improvement steps panel, or Autofix report to interpret.

A mirror/pure-fork no-op needs special care in the same UI. Guard stores a neutral 100/good result, but its criteria score is null, every criterion and checklist item is skipped, and its detailed verdict is Not applicable — assess upstream. The green value avoids punishing a repository for an inapplicable audit; it does not certify the fork or award perfect readiness.

Where the audit stops and improvement begins

A finding can become a bounded improvement candidate, but candidate context is not write permission. The exact types preserve different decision boundaries: remove_private_content requires an owner to decide on deletion and any history rewrite; clean_internal_reference can describe a mechanical replacement after the public setup approach is approved; resolve_dependency_license remains an owner or counsel decision; and fix_license_metadata is mechanical only when a license already exists—choosing one is not.

Documentation candidates are answer_outsider_question for Q1-Q10, fix_entry_point for a dead or misleading route, add_intake_templates for issue/change intake, establish_release_notes for the next release, and declare_maintenance_status when claims and activity disagree. optional_growth covers non-required suggestions and can never outrank a legal, content, entry, or participation gap.

Pre-publication ordering has seven ranks: clean publishability; make the own-license and dependency decision; make the README stranger-ready; establish the participation contract; repair intake and truthful signaling; establish release discipline; then consider optional growth. Compliance uses six: remove exposed or illegal blockers; repair the trust surface; fill the community baseline; strengthen the newcomer path; restore release and maintenance discipline; then optional growth. Compliance never invents a seventh rank.

For sunharbor/relay-kit, rank 1 is clean_internal_reference: replace the private-registry dependency with a public, reproducible outsider setup path and recheck Q11. Rank 4 is answer_outsider_question for Q7's conduct expectations and misconduct contact. Rank 6 is establish_release_notes for the first public release. No current paired action can execute them.

A human owner confirms the public setup, any removal of private material, any history rewrite, the license choice, and the handling of a license conflict. An authorized route can then implement and review the work. Any downstream executor must revalidate repository identity, access, current revision, and evidence before creating an issue, branch, commit, pull request, publication, deletion, or external-service change. Guard audits the new revision afterward rather than assuming the recommendation remained correct or caused a higher score.

Limits, reruns, and maintained health

A typed result describes the accessible current tree at one full commit. It cannot prove that history contains no secret, perform exhaustive privacy/DLP or legal due diligence, establish exploitability, clear every transitive license, or substitute for the adjacent Security, Dependency hygiene, CI/CD, Test quality, or Codebase hygiene audits. Public platform, registry, or documentation reads may be forbidden or unavailable. Guard uses repository evidence where it remains sufficient, names the limitation, and does not turn optional-service failure into an automatic penalty.

Criterion-level unavailability has a narrower meaning. Criterion 4 can skip in compliance mode only when internal-origin traces are not plausible; criterion 14 can skip when no maintenance evidence is observable. Each skip removes five from the maximum instead of adding five to the sum. A mirror/pure-fork no-op skips the whole substantive domain. By contrast, invalid target identity, access, clone, revision, or required evidence prevents the typed result entirely.

Shallow history can limit maintenance judgment and comparison. An unavailable previous result is normal: Guard publishes the independently derived current baseline without a previous field or invented delta. Root guidance can also be absent or unusable without penalty. When comparison is possible, only repository change, corrected evidence, a material scope or access change, applicable guidance change, or methodology change can support movement; cosmetic report wording cannot.

Rerun evaluates a new repository revision and archives another report. It can rise after a blocked setup path is removed, remain flat when optional polish changes, or fall when a new dependency, dead link, private reference, release claim, or maintenance signal breaks the outside-world contract. The current UI waits for a new repository version after a successful run rather than repeatedly scoring the same head.

Maintained openness is the useful outcome. Publication is one event; repository identity, docs, participation routes, dependencies, releases, and internal residue continue to change. Recurring readiness checks keep that public contract visible so coding agents can add features to a project whose outside entry remains healthy instead of building on accumulated disclosure and participation risk.