Skip to content

Intake bench writes check lists IssueFacts fields by hand, so a new field escapes it #4402

Closed as not planned

Description

@Wirasm

Problem

The intake bench's writes check compares two repository snapshots field by field, but the list of fields it compares is written out by hand and is not tied to the type it compares.

At c47d37a976586e6923862e2b7b0c4aaac2a3e509, in .archon/workflows/sdlc/intake/scripts/bench/runner.ts:

  • IssueFacts (runner.ts:330-336) declares labels, state, title, body_digest and comments.
  • repositoryFacts (runner.ts:362-388) fills those five fields.
  • snapshotChanges (runner.ts:422-444) compares them through the literal ['labels', 'state', 'title', 'body_digest', 'comments'] as const at runner.ts:431.

The two lists agree today. If someone adds a field to IssueFacts (for example assignees or milestone, to catch more agent writes) and fills it in repositoryFacts, type-check still passes, and snapshotChanges silently ignores that field. A batch that changed it would still report writes as clean.

Why

The writes check is the bench's guarantee that a batch did not modify the set's own repository. A check that can silently stop covering a field is the kind of weak guard that hides the very failure it exists to catch. AGENTS.md treats two declarations kept in agreement by discipline as a present defect.

Why now

The bench is being actively extended for live batches (#4298, #4103), so IssueFacts is the type most likely to grow next.

Desired outcome

The set of compared fields is derived from IssueFacts itself, or the compiler rejects a field list that does not cover every key of IssueFacts. Adding a field to IssueFacts either gets compared automatically or fails type-check until the comparison is updated.

Invariants

  • snapshotChanges output for today's five fields is unchanged, including the comments changed wording and the from ... to ... wording for the others.
  • The posted_text check (postedComments, runner.ts:446-466) is unaffected.
  • No new runtime dependency.

Acceptance

  • Adding a field to IssueFacts without comparing it fails type-check (or is compared without further edits). Show this once, then revert the probe.
  • Existing snapshotChanges cases in .archon/scripts/__tests__/intake-bench.test.ts (around lines 802-815) still pass.
  • bun run check:bundled is clean, since the pack bundle includes this file. This is a pack change, so the maintainer merges.

Found by Archon run 1cfa585c-9725-4b1a-a476-cdddc8bb8547 and verified at c47d37a by the archon-discoveries workflow.

Activity

  1. Wirasm commented on Oct 19, 2026

    @Wirasm
    CollaboratorAuthor

    Closing as superseded. the intake bench it describes was deleted by #4574 (cut 7)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      Sponsor
      SponsoredKunjungi sekarang
      Promo