Skip to content

1140 D4–D7: D4 References and persisted node output; D5 Run control and retry; D6 Checkout guard and trigger rules; D7 Provider resolution and capability preflight; one provider-invoking classifier #4505

Closed

Description

@Wirasm

Problem

packages/workflows/src/dag-executor.ts is one of the large files #1140 splits. Steps D4–D7 of lane D, done together in one run: D4 References and persisted node output; D5 Run control and retry; D6 Checkout guard and trigger rules; D7 Provider resolution and capability preflight; one provider-invoking classifier.

Why

#1140's goal is a clean, maintainable core: modules with one owner each, tests that move with their code, and no dead code. The lane plan is #1140 (comment).

Why now

The steps before these in lane D have merged. They're batched because the overnight one-step-per-run pace was too small (maintainer, 2026-10-18).

Desired outcome

Steps D4–D7 as the plan describes them below, landed as one PR, in order, one commit per step. Re-read the code first, because earlier steps have changed it. A later step may adjust to what an earlier one found.

Invariants

  • No behavior change. No engine functionality that works today is removed. The public entry stays where it is, and the internals move.
  • Each deleted test names the test that still covers its behavior, in the PR body, or is a test that only pinned structure.
  • The A complexity ratchet whose baseline the change cannot edit #3020 complexity ratchet holds or goes down.
  • No new dependency.

Acceptance

  • The moved suites' pass count equals their count before the move, minus the deletions the PR body names.
  • Type-check and lint pass in the touched packages, and CI is green.

The steps (from the lane plan)

D4. References and persisted node output

  • Boundary: move verbatim to
    • dag/references.ts: resolveRunInputs, ShellInputContext, inputEnvVars, buildNodeShellEnvironment, wholeRefLogicalValue, resolveWorkflowValue, resolveValueTemplate, resolveNodeBindings, resolveBindingDirective, shellQuote, writeSpillFile, shellQuoteOrFile, RequiredOutputRefContext, requiredOutputRefError, nodeOutputRefToken, TemplateSources, TextSurface, nodeTemplateReferences, substituteNodeOutputRefs, substituteWaitMessage, collectLoopBodyNodeIds, loopPrevRefToken, loopScopeTokens, LoopBodyScope, loopBodyScope, substituteLoopBodyText, resolveAttentionOptions. (Bindings and templates call each other, so they are one module.)
    • dag/node-output.ts: NODE_OUTPUT_FILE_THRESHOLD, PERSISTED_NODE_OUTPUT_MAX_BYTES, utf8SequenceLength, formatPersistedNodeOutput, persistedExecutionOutput, rejectedArtifactPointer.
  • Files: dag-executor.ts, the two new files, dry-run.ts (import from ./dag/references; dry-run stops loading the whole executor for pure text functions).
  • Tests that move to dag/references.test.ts: describe('output reference errors name the reference as written'), describe('substituteNodeOutputRefs'), describe('substituteNodeOutputRefs -- shell escaping'), describe('substituteNodeOutputRefs -- large output file substitution'), describe('substituteNodeOutputRefs -- structuredOutput preference'), and the unit-level resolveNodeBindings cases inside describe('value transport (#2637): persistence, resume, and node-local bindings') that call resolveNodeBindings directly (the executeDagWorkflow cases there stay for D20).
  • Size: about 900 moved lines; no logic change.
  • Conflicts: dag-executor.ts, dag-executor.test.ts, dry-run.ts.

D5. Run control and retry

  • Boundary: move verbatim to
    • dag/run-control.ts: lastNodeCancelCheck, CANCEL_CHECK_INTERVAL_MS, shouldContinueStreamingForStatus, waitForNodeRetry, SUBPROCESS_STOP_CHECK_INTERVAL_MS, runSubprocessUntilRunStops, lastNodeActivityUpdate, ACTIVITY_HEARTBEAT_INTERVAL_MS.
    • dag/retry.ts: DEFAULT_NODE_MAX_RETRIES, DEFAULT_NODE_RETRY_DELAY_MS, getEffectiveNodeRetryConfig, getExplicitNodeRetryConfig, retryableFailureClass, runBudgetedAttempt, runNodeRetryLoop, runDeterministicNodeWithRetry.
  • Files: dag-executor.ts, two new files. schemas/workflow-run.ts mentions shouldContinueStreamingForStatus in a comment only; update the location if it names the file.
  • Tests that move: describe('shouldContinueStreamingForStatus') → dag/run-control.test.ts; describe('executeDagWorkflow -- node-level retry for transient errors'), describe('executeDagWorkflow -- retry on deterministic (bash/script) nodes (#2088)') → dag/retry.test.ts; describe('executeDagWorkflow -- a run stopped while its subprocess runs') → dag/run-control.test.ts.
  • Size: about 380 moved lines.
  • Conflicts: dag-executor.ts, dag-executor.test.ts.

D6. Checkout guard and trigger rules

  • Boundary: move verbatim to
    • dag/checkout-guard.ts: checkoutSnapshotExcludes, isInsideAny, porcelainPaths, snapshotCheckout, snapshotGuardedNode, isCheckoutGuarded, assertCheckoutUntouched.
    • dag/trigger-rule.ts: TriggerRuleDecision, TriggerRuleUpstream, triggerSkipCause, checkTriggerRule, checkTriggerRuleForDependencies, checkComposedBlockBoundaries.
  • Files: dag-executor.ts, two new files, dry-run.ts (trigger-rule imports).
  • Tests that move: describe('checkTriggerRule'), describe('checkTriggerRule -- classify-gated pipeline behavior on failure'), describe('checkTriggerRule -- missing upstream treated as failed') → dag/trigger-rule.test.ts; describe('executeDagWorkflow -- node-level mutates_checkout: false (#2771)') → dag/checkout-guard.test.ts.
  • Size: about 290 moved lines.
  • Conflicts: dag-executor.ts, dag-executor.test.ts, dry-run.ts.

D7. Provider resolution and capability preflight; one provider-invoking classifier

  • Boundary:
    • dag/provider-resolution.ts: applyPresetOptions, parseMcpServerNames, NodeMcpResolution, resolveNodeMcp, resolveNodeProviderAndModel, SCOPED_CAPABILITIES, ScopedCapability, namesScopedCapability, unsupportedScopedCapabilities, scopedCapabilityMismatchMessage.
    • dag/capability-preflight.ts: visitProviderInvokingNodes, collectContainerIncompatibleProviders, ScopedCapabilityMismatch, collectScopedCapabilityMismatches, StrictSchemaViolation, collectStrictSchemaViolations.
    • resolveNodeProviderAndModel takes (ctx, node, { resolveAiText, warnedProviderConflicts }) instead of 15 positionals; _cwd is deleted. The approval rework call keeps passing text => text and undefined for conflicts so its behavior is unchanged.
    • Fix the hand-synced pair: one exported, exhaustive switch (node.kind) classifier (for example providerInvocation(node): 'invokes' | 'group' | 'gate_rework' | 'none', name provisional) in dag/capability-preflight.ts, used by both visitProviderInvokingNodes and collectRunCredentialRequirements in run-preflight.ts. A new node kind then fails to compile until someone classifies it. The selected set must stay {agent, loop, loop_group, gate with rework}; each walker keeps its own resolution scope (the credential walker resolves models; the capability walker only providers).
    • Fix the stale resolveNodeModel docstring in node-model-resolution.ts ("Mirrors dag-executor's chain exactly" → it is the one chain shared by execution, dry run and run preflight).
  • Files: dag-executor.ts (4 call sites in runLayers, 1 in executeApprovalNode, preflight calls in executeDagWorkflow), two new files, run-preflight.ts, node-model-resolution.ts, defaults/bundled-defaults.test.ts (imports visitProviderInvokingNodes).
  • Tests that move to dag/provider-resolution.test.ts: describe('resolveNodeMcp'), describe('executeDagWorkflow -- home MCP paths'), describe('executeDagWorkflow -- MCP failure filtering'), describe('executeDagWorkflow -- skills options'), describe('executeDagWorkflow -- Claude SDK advanced options'), describe('provider resolution -- regression for #1610'), describe('executeDagWorkflow -- a workflow-level provider/model conflict is reported once'), describe('executor capability ownership') (its behavioral cases). To dag/capability-preflight.test.ts: describe('collectContainerIncompatibleProviders'), describe('container preflight provider equivalence with dispatch'), describe('collectStrictSchemaViolations'), describe('executeDagWorkflow -- strict-schema preflight').
  • New test: one case in run-preflight.test.ts or capability-preflight.test.ts that feeds every DagNode['kind'] through the classifier and asserts the selected set, so both walkers are proven to agree.
  • Size: about 620 moved lines plus about 120 changed (call sites, classifier).
  • Conflicts: dag-executor.ts, dag-executor.test.ts, run-preflight.ts, node-model-resolution.ts.

Part of #1140.

Current scope (maintainer ruling on the merge queue's question on PR 4516, 2026-10-18)

The maintainer's words: Merge. The #1140 lane plan's step D7 asks for this one provider-invoking classifier, to fix the hand-synced pair, and the review found nothing open. Orchestrator ruling, 2026-10-18.

Activity

  1. added
    archon-readyTriage: ready to deliver
    archon-size-lTriage: several packages, schemas, or surfaces
    on Oct 18, 2026
  2. Wirasm commented on Oct 18, 2026

    @Wirasm
    CollaboratorAuthor

    Delivered by #4516 (merged). The PR linked this issue with Refs, so GitHub didn't close it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    archon-readyTriage: ready to deliverarchon-size-lTriage: several packages, schemas, or surfacesarchon-type-refactorTriage: same behavior, better structure

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      Sponsor
      SponsoredKunjungi sekarang
      Promo