Skip to content

Split references, run control, checkout guard and provider resolution out of dag-executor.ts - #4516

Merged
Wirasm merged 6 commits into
devfrom
build/4505-1140-d4-d7-d4-references-and-persisted-n-56a2454e
Oct 18, 2026
Merged

Wirasm merged 6 commits into
devfrom
build/4505-1140-d4-d7-d4-references-and-persisted-n-56a2454e

Conversation

@Wirasm

@Wirasm Wirasm commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Problem and outcome

packages/workflows/src/dag-executor.ts is still a 12,000-line file that mixes reference resolution, retry, checkout guarding, trigger rules and provider resolution with the executor itself. This is steps D4–D7 of lane D of #1140, landed together, one commit per step.

  • Outcome: each concern lives in a module under packages/workflows/src/dag/ with its tests beside it. dag-executor.ts shrinks from 12,370 to 10,115 lines and dag-executor.test.ts from 43,258 to 38,453.
  • Invariant: no behavior change. The public entry (executeDagWorkflow) stays where it is. Importers point at the new owners; dag-executor.ts keeps no re-exports.
  • Scope boundary: later lane D steps are not touched. Neither is anything outside packages/workflows.

Review guidance

  • Feedback requested: the D7 classifier and the resolveNodeProviderAndModel signature. D4–D6 are mechanical.
  • Start here: packages/workflows/src/dag/capability-preflight.ts (providerInvocation). It is the only non-verbatim design change.
  • Review order: commit a7d4fb8fd (D7), then skim D4 35edb2c7d, D5 ee41f14d2 and D6 7087f7911.
  • Lower-attention areas: D4–D6 moved declarations with their JSDoc, byte-for-byte apart from an added export, re-wrapped signatures, and one import('../schemas/node-execution') type path. Tests moved the same way.
  • Known risk or uncertainty: none known. Run test files one at a time (bun run test <file>), as the package runner does. The run-control stopped-subprocess test times out if retry.test.ts shares its process.

Solution

Each step moves its declarations into the modules the plan names:

  • D4: dag/references.ts and dag/node-output.ts. dry-run.ts now imports the pure text functions from references and no longer loads them through the executor.
  • D5: dag/run-control.ts and dag/retry.ts.
  • D6: dag/checkout-guard.ts and dag/trigger-rule.ts.
  • D7: dag/provider-resolution.ts and dag/capability-preflight.ts. Two things change beyond the move:
    • resolveNodeProviderAndModel(ctx, node, { resolveAiText, warnedProviderConflicts }) replaces fifteen positional arguments, and the unused _cwd is gone. The approval rework call still passes text => text and no conflict set.
    • The capability walker and the credential walker in run-preflight.ts each listed the node kinds that reach a provider, kept in sync by hand. Both now select nodes through one exhaustive switch (node.kind) classifier, providerInvocation. A new node kind no longer compiles until it is classified. The selected set is unchanged: agent, loop, loop_group and a gate with rework. Each walker keeps its own traversal and resolution scope.
    • The stale resolveNodeModel docstring now says it is the one chain that execution, dry run and run preflight share.

Changed seams

Boundary or contract Change Evidence
capability-preflight / run-preflight → node-kind selection Two hand-synced lists become one exhaustive classifier dag/capability-preflight.ts providerInvocation; new providerInvocation suite feeds every node kind through it
runLayers / executeApprovalNode → resolveNodeProviderAndModel Fifteen positionals become (ctx, node, { resolveAiText, warnedProviderConflicts }) the four call sites in dag-executor.ts

Validation

  • bun x tsc --noEmit -p . in packages/workflows — clean.
  • bun x biome lint --error-on-warnings packages/workflows/src — clean.
  • bun run check:ratchet — passed, with no Ratchet-Allow trailer.
  • bun run test <file> per file in packages/workflows — 0 failures: dag-executor.test.ts (817), dag/references (63), dag/run-control (9), dag/retry (33), dag/trigger-rule (18), dag/checkout-guard (13), dag/provider-resolution (62), dag/capability-preflight (37), run-preflight (29), dry-run (115).
  • packages/core process-preflight.test.ts (1) and process-registration.test.ts (85) — pass; they exercise credential requirements.
  • Test counts: dag-executor.test.ts went from 1,050 to 817 tests, and 233 tests moved unchanged into the new files (817 + 233 = 1,050). No test was deleted. capability-preflight.test.ts adds 2 new providerInvocation tests: one feeds every DagNode['kind'] through the classifier and asserts the selected set, the other covers a gate without rework.
  • I confirmed the classifier guards fail when broken. Dropping a case is a compile error. Classifying workflow as invokes fails the new test.
  • Not verified: the full repository suite and CI. They run on the pull request.

Links

Summary by CodeRabbit

  • New Features

    • Workflow runs now support configurable retries for eligible failures, with retry progress and accumulated usage reflected in run results.
    • Runs stop active subprocesses when cancelled or otherwise no longer eligible to continue.
    • Workflow inputs and node outputs can be referenced in prompts, scripts, and shell commands, with safer handling for large values.
    • Large node outputs are retained in full when possible, while persisted previews are capped and indicate when content was truncated.
    • Provider capability checks identify incompatible workflow requirements before execution.
    • Nodes marked as not modifying the checkout are checked for unexpected repository changes.
  • Bug Fixes

    • Improved handling and reporting of missing, unavailable, or failed output references and skipped workflow dependencies.

Step D4 of the dag-executor split (part of #1140). The declarations move verbatim into
two owners under packages/workflows/src/dag/:

- references.ts: run inputs, node bindings, value templates, shell quoting and spill
  files, output-reference errors, template substitution, loop-body scopes and attention
  options. Bindings and templates call each other, so they are one module.
- node-output.ts: the persisted-output byte cap and spill, the execution output a node
  persists, and the artifact-pointer rejection.

dry-run.ts now imports the text functions from dag/references.ts. The five reference
suites move to dag/references.test.ts with their 63 tests unchanged. No behavior change.
Step D5 of the dag-executor split (part of #1140). The declarations move verbatim into
two owners under packages/workflows/src/dag/:

- run-control.ts: the cancel-check and activity-heartbeat throttles, the streaming
  status policy, the retry wait that watches for a stopped run, and the subprocess
  runner that ends a node's process tree when its run stops.
- retry.ts: the node retry defaults and config, retryable-failure classification, the
  budgeted attempt, and the retry loops for AI and deterministic nodes.

The streaming-policy and stopped-subprocess suites move to dag/run-control.test.ts
(9 tests) and the two retry suites to dag/retry.test.ts (33 tests), unchanged apart from
the streaming suite's dynamic import now naming run-control. No behavior change.
Step D6 of the dag-executor split (part of #1140). The declarations move verbatim into
two owners under packages/workflows/src/dag/:

- checkout-guard.ts: the checkout snapshot and its engine-write exclusions, and the
  mutates_checkout: false assertion.
- trigger-rule.ts: trigger-rule decisions for a node's upstreams and the composed-block
  boundary check.

dry-run.ts now imports checkTriggerRule and checkComposedBlockBoundaries from
dag/trigger-rule.ts. The three checkTriggerRule suites move to dag/trigger-rule.test.ts
(18 tests) and the mutates_checkout suite to dag/checkout-guard.test.ts (13 tests),
unchanged. No behavior change.
….ts; one provider-invocation classifier

Step D7 of the dag-executor split (part of #1140). The declarations move into two
owners under packages/workflows/src/dag/:

- provider-resolution.ts: preset options, node MCP resolution, per-node provider and
  model resolution, and the scoped-capability checks.
- capability-preflight.ts: the provider-invoking node walk and the container,
  scoped-capability and strict-schema preflights.

resolveNodeProviderAndModel now takes (ctx, node, { resolveAiText,
warnedProviderConflicts }) instead of fifteen positionals; the unused cwd is gone. The
approval rework call still passes text => text and no conflict set.

The capability walker and the credential walker in run-preflight.ts each kept their own
list of node kinds that reach a provider, in sync by hand. Both now select nodes through
providerInvocation, an exhaustive switch on node.kind, so a new node kind does not
compile until it is classified. The selected set is unchanged: agent, loop, loop_group,
and a gate with rework. Each walker keeps its own resolution scope.

Twelve suites move to dag/provider-resolution.test.ts (62 tests) and
dag/capability-preflight.test.ts (35 tests) unchanged apart from resolveNodeMcp's
dynamic import; a new providerInvocation suite feeds every node kind through the
classifier. The stale resolveNodeModel docstring now says it is the one chain
execution, dry run and run preflight share. No behavior change.
@coderabbitai

coderabbitai Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7d7e055a-5d4f-40e7-b01e-186595d632a2
📥 Commits

Reviewing files that changed from the base of the PR and between 23a5565 and 366d5dd.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fcf338e1-0754-4341-a650-255dafcf3e63
📥 Commits

Reviewing files that changed from the base of the PR and between 07ee6c2 and 23a5565.

📒 Files selected for processing: 26
  • packages/workflows/src/condition-evaluator.ts
  • packages/workflows/src/dag-executor.test.ts
  • packages/workflows/src/dag-executor.ts
  • packages/workflows/src/dag/capability-preflight.test.ts
  • packages/workflows/src/dag/capability-preflight.ts
  • packages/workflows/src/dag/checkout-guard.test.ts
  • packages/workflows/src/dag/checkout-guard.ts
  • packages/workflows/src/dag/node-output.ts
  • packages/workflows/src/dag/provider-resolution.test.ts
  • packages/workflows/src/dag/provider-resolution.ts
  • packages/workflows/src/dag/references.test.ts
  • packages/workflows/src/dag/references.ts
  • packages/workflows/src/dag/retry.test.ts
  • packages/workflows/src/dag/retry.ts
  • packages/workflows/src/dag/run-context.ts
  • packages/workflows/src/dag/run-control.test.ts
  • packages/workflows/src/dag/run-control.ts
  • packages/workflows/src/dag/test-utils.ts
  • packages/workflows/src/dag/trigger-rule.test.ts
  • packages/workflows/src/dag/trigger-rule.ts
  • packages/workflows/src/dry-run.ts
  • packages/workflows/src/node-model-resolution.ts
  • packages/workflows/src/output-ref.ts
  • packages/workflows/src/run-preflight.ts
  • packages/workflows/src/schemas/workflow-run.ts
  • packages/workflows/src/utils/output-truncation.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review.

📝 Walkthrough

The change adds workflow DAG modules for reference handling, provider resolution, checkout mutation checks, output persistence, retries, run control, and trigger-rule evaluation. It also updates imports and comments to point to the new module locations.

Changes: DAG references

Layer / File(s) Summary
Input and binding resolution
packages/workflows/src/dag/references.ts, packages/workflows/src/dag/references.test.ts
Adds input precedence, node-local binding resolution, and output references with producer-state and field validation. Tests cover reference errors and output cases.
Template substitution and loop scopes
packages/workflows/src/dag/references.ts, packages/workflows/src/dag/references.test.ts, packages/workflows/src/dag/run-context.ts, packages/workflows/src/dag/test-utils.ts, packages/workflows/src/condition-evaluator.ts, packages/workflows/src/output-ref.ts
Adds template and loop-scope substitution, shell quoting, and spill handling. Tests cover escaping, spill behavior, and structured output. Related imports and comments point to the new module.

Provider resolution and preflight

Layer / File(s) Summary
Provider, model, and MCP resolution
packages/workflows/src/dag/provider-resolution.ts, packages/workflows/src/dag/provider-resolution.test.ts, packages/workflows/src/node-model-resolution.ts
Adds node provider and model resolution, MCP handling, option construction, and scoped-capability checks. Tests cover provider routing, MCP configuration, capability handling, and execution cases.
Provider invocation and capability preflight
packages/workflows/src/dag/capability-preflight.ts, packages/workflows/src/dag/capability-preflight.test.ts, packages/workflows/src/run-preflight.ts
Adds provider-invocation classification and collectors for container compatibility, scoped capability mismatches, and strict-schema violations. Run credential collection uses the classification.

Checkout mutation guard

Layer / File(s) Summary
Snapshot and mutation enforcement
packages/workflows/src/dag/checkout-guard.ts, packages/workflows/src/dag/checkout-guard.test.ts
Adds Git snapshots for eligible nodes and converts detected checkout changes into output-contract failures. Tests cover exclusions, concurrency, layers, and resumed workflows.

Node output persistence

Layer / File(s) Summary
Bounded output and artifact validation
packages/workflows/src/dag/node-output.ts, packages/workflows/src/utils/output-truncation.ts
Adds bounded persisted output with truncation metadata and spill paths, plus artifact-pointer validation against the producing run.

Retry and run control

Layer / File(s) Summary
Retry policy and attempt accounting
packages/workflows/src/dag/retry.ts, packages/workflows/src/dag/retry.test.ts
Adds retry configuration and failure classification for agent and deterministic nodes. The shared loop records retries and accumulates attempt usage.
Status-aware waits and subprocesses
packages/workflows/src/dag/run-control.ts, packages/workflows/src/dag/run-control.test.ts, packages/workflows/src/schemas/workflow-run.ts
Adds status checks during retry waits and subprocess execution. Tests cover cancellation and streaming-status decisions.

DAG trigger rules

Layer / File(s) Summary
Dependency and boundary evaluation
packages/workflows/src/dag/trigger-rule.ts, packages/workflows/src/dag/trigger-rule.test.ts, packages/workflows/src/dry-run.ts
Adds dependency and composed-boundary trigger evaluation with skip-cause selection. Tests cover trigger outcomes and cause provenance.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Refactor

Merge Risk: ⚪ Minimal · up to 23a55

This change reorganizes the DAG executor into smaller modules and shares one classification of provider-invoking nodes between run preflight and capability checks. The intended behavior is unchanged, and no concrete regression was found. Confirming that CI passes once the PR leaves draft is the remaining normal step before merge.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 76.47% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 68 functions across 24 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly summarizes the main change: moving references, run control, checkout guarding, and provider resolution out of dag-executor.ts.
Description check Passed The description includes the required Problem and outcome, Review guidance, Solution, and Validation sections. It also states scope, invariants, changed seams, test results, and remaining verification…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description includes the required Problem and outcome, Review guidance, Solution, and Validation sections. It also states scope, invariants, changed seams, test results, and remaining verification limits.

✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Wirasm

Wirasm commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator Author

Reviewed commit: a7d4fb8fd97884102a791d4faa008bc0edab9893
Verdict: ready

Review report

Verdict

Ready: true. Action: none. All four selected lenses completed with no findings, and my check of the classifier and its two consumers agrees; the one open acceptance item (CI green) needs a step that comes after review.

Accepted contract

Source: issue #4505 (part of #1140), PR #4516, base dev. Required outcome: Lane D steps D4–D7 landed as one PR, one commit per step: D4 references + node-output, D5 run-control + retry, D6 checkout-guard + trigger-rule, D7 provider-resolution + capability-preflight with one exhaustive provider-invoking classifier.
Acceptance: (1) moved suites' pass count equals the count before the move, minus named deletions; (2) type-check and lint pass in touched packages, and CI is green.
Invariants: no behavior change; each deleted test is named or pinned only structure; #3020 ratchet holds; no new dependency.
Steering: verbatim moves into the named dag/ files; importers updated (dry-run.ts, schemas/workflow-run.ts comment); D7 signature (ctx, node, { resolveAiText, warnedProviderConflicts }) with _cwd deleted; one exhaustive switch (node.kind) classifier used by both walkers with selected set {agent, loop, loop_group, gate with rework}; test feeding every kind; no re-export path from dag-executor.ts.
Boundaries: nothing outside packages/workflows; later steps (D20) untouched. No part assignment.

Reviewed head SHA

a7d4fb8

Contract coverage

Item Verdict Evidence
Required outcome (D4–D7, one commit per step) met Commits 35edb2c, ee41f14, 7087f79, a7d4fb8; eight dag/ modules exist; code lens diffed moved bodies as verbatim.
Acceptance 1: test counts met Code lens: 898 declarations at base, 710 + 190 = 900 after (two new classifier tests). Tests lens: 878 → 695 + 185 = 880. Implementation notes: 1050 = 817 + 233, no deletions. Counting methods differ, all agree on no loss. Package bun run test exit 0.
Acceptance 2: type-check, lint met Code lens: tsc --noEmit -p packages/workflows exit 0, bun run lint exit 0.
Acceptance 2: CI green deferred PR is a draft, CI jobs skipped. Evidence exists only at a later step of this delivery (CI after leaving draft).
Invariant 1: no behavior change met Code lens line-set comparison, call-site tracing of resolveNodeProviderAndModel, walker equivalence against dev.
Invariant 2: deleted tests met No tests deleted.
Invariant 3: ratchet met bun run check:ratchet passed (code lens).
Invariant 4: no new dependency met No manifest changes in the changed-file list.
Steering 1–3, 5, 7 met Code lens traced signature, importers, no re-export; bundled-defaults.test.ts never imported the symbol on dev, so nothing to update.
Steering 4: one exhaustive classifier met capability-preflight.ts:37-58 has no default; used at capability-preflight.ts:87 and run-preflight.ts:305; each walker keeps its own scope.
Steering 6: every-kind test met capability-preflight.test.ts:510-558 uses a Record<Kind, …> and asserts the selected set plus gate-without-rework.

Findings

None. No blocking findings, no notes.

Rejected findings

None raised.

Discoveries

  1. See /Users/rasmus/.archon/workspaces/coleam00/Archon/artifacts/runs/56a2454e-580b-40e6-a0f7-642cc39a2070/discoveries.md. Unrelated discoveries do not affect readiness.

Review coverage

Route reason: first round; the diff moves retry, checkout-guard, trigger-rule and provider-resolution code and adds a shared classifier, so seams, tests and errors were selected alongside code; docs was off because there is no user-facing change. Selected: seams, code, tests, errors. All four ran, each has a completed review-lens artifact for this round's loop iteration, and each reported no findings. Docs not selected, not a finding. Not obtainable here: CI (draft PR).

@Wirasm
Wirasm marked this pull request as ready for review October 18, 2026 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Sponsor
SponsoredKunjungi sekarang
Promo