ADR-0014: Manual-Effect Scope — Reactive Cascade Deferral

• Status: Accepted
• Date: 2026-06-29

Context

Bug 1 (2026-06-29, fixed in commit 7d9293a): a manual action whose effect is

(begin
  (set-field "状态" "text" "done")
  (transition "done"))

would, after the set-field commits but before transition ran, fire any reactive rule watching (field "状态"). Such a rule — modelled in domains/memory.scm as correct_task_status — compares the field to __state__<Machine> and rewrites the field to match. Mid-effect, it saw pre-transition state (__state__Task = "in_progress") and “corrected” 状态 back from "done" to "in_progress". After transition ran, the node had 状态 = "in_progress" and __state__Task = "done" — inconsistent.

This broke tests/test_sprint_rollover_e2e.py (3 cases): the rollover handler filters tasks by 状态 in ("open", "in_progress"); done tasks whose 状态 had been clobbered were wrongly migrated.

Why the bug existed

The reactive bus subscribed to Store.on_commit and dispatched on every commit. The store commits per write (each _txn is its own SQLite SAVEPOINT that releases and fires on_commit immediately). A single effect block is therefore N commits, N cascade firings — not one atomic boundary.

The system had three layers of “atomicity” but none of them answered the question that mattered:

No document stated what a reactive rule author could assume about the state of the store when their trigger fired. Every reactive rule was implicitly gambling that its dependencies didn’t include a sibling write inside the same effect block.

The structural defenses that existed were coincidences, not contracts:

• _depth caps recursive reactive cascade (depth ≤ 10)
• Most reactive rules don’t read __state__, so the bug stayed latent
• In-stack recursion means cascades complete synchronously

These did not protect the bug’s surface. The fix had to be a contract, not a coincidence.

Decision

We introduce a manual-effect scope — an atomicity boundary maintained by Engine.dispatch around the evaluation of a manual action’s effect block. Reactive cascades do not fire inside the scope; they fire once when the scope exits.

Approach: deferred notification, not MVCC

The contract M3 promises — “reactive sees the complete post-effect snapshot OR nothing” — admits two fundamentally different implementations. We chose one explicitly; recording the choice here prevents a future reader from “fixing” the design by switching to the other.

方案 A — Deferred notification (chosen). SQLite writes commit per _txn() exactly as today; only the on_commit callback firing is deferred until store.atomic() exits. Two layers compose: store.atomic() holds the data-commit notification boundary (M5), ReactiveBus.begin/end_manual_effect holds the reactive- cascade timing boundary (M1). Implementation footprint: ~80 lines in store.py (one new context manager + _atomic_depth counter) + ~50 lines in reactive.py (scope API + queue).

方案 B — Memory shadow snapshot / MVCC (rejected for now). At scope entry, clone the relevant store state into a shadow buffer; effect writes go to shadow only; external observers read from the canonical main store; on success, atomically swap shadow into main.

Comparison:

Why A’s commonly-cited weaknesses are not blockers in this domain:

• “Effect-internal self-read sees new values while external readers see old” is the standard SQL READ COMMITTED semantics that every DSL author already understands from BEGIN ... COMMIT. Not a confusion vector.
• “Event flood at flush” — event count equals write count per effect, not write count squared. Subscribers that need dedup or last-write-wins already declare conflict_policy (ADR-0005 lineage). Pinning this in the contract would be over-fitting.
• “Partial rollback data leakage” — closed by M5. The outermost SAVEPOINT _atomic_root collapses every inner SAVEPOINT, so data written before a raise is undone. The store-level primitive does what shadow snapshots would otherwise do, but without the copy.

When B would become the right choice — documented trigger conditions so future readers know when to revisit:

• Multi-effect concurrency: two manual effects must read each other’s stable mid-effect state (A cannot guarantee this — each effect sees the other’s reactive-flushed final state, never mid-effect)
• Cross-row constraints that cannot be validated per-write (e.g. “only one Task in state X at a time”) — A would surface the violation too late for the effect to handle gracefully; B’s commit-time validation aligns better
• Ontology size or write-rate that makes deferred-notification cost dominate over snapshot-copy cost (unlikely at current scale but not architecturally ruled out)

Until one of these trigger conditions appears, A is the right call.

M1 — Scope lifecycle

Engine.dispatch(action_name, node_id, actor=...) wraps its evaluate(effect_ast, effect_env) call in bus.effect_scope():

bus = self.bus
if bus is not None:
    with bus.effect_scope():
        if _atomic:
            with self.store.atomic():
                evaluate(effect_ast, effect_env)
        else:
            evaluate(effect_ast, effect_env)
else:
    ...  # no bus wired — plain evaluation

effect_scope() is a @contextmanager that delegates to the internal begin_manual_effect / end_manual_effect counter. Nested scopes are supported; events flush only when the outermost scope exits. On exception, only events added during the failing scope are discarded — events from enclosing scopes survive (ADR-0015 M6).

ADR-0015 extends this with a dual-entry design:

• Scheme A (primary): Engine.dispatch opens effect_scope() at the action boundary — every action gets it automatically.
• Scheme B (safety net): _eval_begin in the DSL evaluator detects __reactive_bus__ in env and transparently opens effect_scope() on any bare (begin ...) block — covers code paths that don’t go through Engine.dispatch.

M2 — Cascade trigger matrix

M3 — Reactive rule visibility contract

A reactive rule, when its trigger fires, can assume:

1. The store reflects all writes from the just-closed consistency point. A consistency point is one of:
   • external single store write (immediate cascade)
   • manual effect scope exit (deferred cascade)
2. Therefore __state__<M> reflects the final transition of that scope, not an intermediate one.
3. Multiple events queued during one scope are dispatched in FIFO commit order to the same depth-handling cascade.

This means the canonical “field must mirror state” pattern (correct_task_status) is now correct by construction: when its trigger fires after a manual complete_task, both 状态 = "done" and __state__Task = "done" are already committed.

M3.flap — Forced last-write-wins + net-zero short-circuit. The queued events are additionally deduped at flush time:

• Per-field LWW: for every (target, field_name) group, only the last set_field event survives. Intermediate writes to the same field never reach the cascade — observers see exactly one event per field, carrying the post-effect value. Eliminates the “event-payload-≠-DB-value” Bug-1 shape at its root.
• Net-zero short-circuit: if the surviving event’s values equal the FIRST event’s prev_value (carried in payload since M3.flap), the entire group is dropped — the sequence was a flapping no-op from the observer’s point of view (e.g. A→B→C→B→A on a field already holding A). Zero dispatches for the field.

Implementation requires set_field to carry prev_value (the field’s pre-image) in its event payload — see “Wiring” below. Other ops (create_node, create_link, tag, untag, audit events) pass through dedup unchanged. tx_log keeps every write; dedup is purely at the observer boundary.

M4 — Failure isolation

• Pending events that fail to flush (any exception inside _on_commit during deferred dispatch) are caught and audited as reactive_flush_error. The original manual dispatch’s return value is not affected.
• A failed cascade inside a scope does not abort the scope — it audits and continues.
• Partial-effect rollback (M5 below) is the deeper guarantee: an exception inside the effect block rolls back ALL writes in the scope AND drops ALL reactive notifications. This is the strongest form of “either the rule sees the complete post-effect snapshot, or it sees nothing.”

M5 — Atomic boundary on the store

Engine.dispatch wraps the evaluate(effect_ast, effect_env) call in self.store.atomic() — a new store-level primitive that groups every write inside the manual effect into one logical transaction.

• store.atomic() opens an outer SAVEPOINT _atomic_root. Every _txn() call inside the scope uses a nested _sp savepoint.
• On clean exit: outermost atomic fires on_commit once with all accumulated events; inner _txn releases do NOT fire on_commit.
• On raise: outermost atomic executes ROLLBACK TO SAVEPOINT _atomic_root, drops all pending events, and re-raises. Inner _txn rolls back at its own level first (dropping events it had appended), then the root catches anything outside an inner _txn.
• atomic() calls nest; only the outermost owns commit/rollback.

Why this is M5 and not part of M3: M3 is the reactive rule’s visibility contract; M5 is the store-level machinery that makes M3 hold under failure. Without M5, a partial-effect raise would leave half-committed data in the DB AND let the inner SAVEPOINT’s on_commit callback leak the partial writes to the bus — the bus’s deferral would then carry those writes into the flush, and reactive observers would see “ghost” notifications for data that no longer exists. M5 closes that seam.

The reactive cascade’s recursive dispatches pass _atomic=False so their legacy best-effort semantics survive — a reactive effect that commits one write then raises keeps that write, matching TestReactiveFailureSwallowed. Only the outermost manual dispatch gets the atomic guarantee.

Wiring summary

• Store.__init__ adds _atomic_depth: int = 0
• Store.atomic() — new context manager; see M5
• Store._txn() — modified:
  • Snapshot events_before on entry; del self._pending_events[events_before:] on rollback so inner-rollback doesn’t leak events to outer scope
  • Fire on_commit only when _atomic_depth == 0 (otherwise the parent atomic() owns the firing)
• Engine.dispatch(..., _atomic=True) — wraps effect eval in store.atomic(); reactive cascade calls pass _atomic=False
• Engine.dispatch wraps effect eval in bus.effect_scope() (Scheme A of ADR-0015) — this composes with store.atomic(): bus defers reactive cascade timing, store provides data+notification atomicity
• ReactiveBus.add_observer(obs) — test-only hook for the tests/_spy.py::CascadeSpy infrastructure. Phase tag is forced to "post-effect" during end_manual_effect’s flush so the spy can distinguish “flushed at manual scope exit” from external writes (where it auto-detects "external" vs "reentrant").

Unchanged invariants

• Recursion depth limit (_depth ≤ max_depth, default 10) — unchanged
• Audit event types (reactive_dispatch_plan / reactive_guard_rejected / reactive_illegal_transition / reactive_dispatch_error) — semantics unchanged; only their firing-time is now scope-bound
• Standalone Engine(store) (no bus wired) — behaviorally unchanged; scope gating is a no-op when self.bus is None
• Engine.dispatch return value (True / False / IllegalTransitionError) — unchanged

Wiring

• ReactiveBus owns _in_manual_effect: int and _pending_events: list[dict]
• ReactiveBus.effect_scope() is the public context-manager API (ADR-0015 Scheme A). begin_manual_effect / end_manual_effect remain as internal primitives for backward compatibility.
• Engine._make_env injects __reactive_bus__ into the DSL env so _eval_begin can transparently wrap bare (begin ...) blocks (ADR-0015 Scheme B).
• Engine.__init__ accepts an optional bus (typed Any to avoid the Engine ↔ ReactiveBus import cycle; reactive.py already imports Engine)
• Ontology facade wires self.engine.bus = self.bus after constructing both, so dispatch can find the bus
• Standalone test code wires the same way: engine.bus = ReactiveBus(engine); engine.bus.install()
• Store.set_field carries the field’s pre-image as payload.prev_value (read BEFORE the DELETE inside the same SAVEPOINT). ReactiveBus._dedup_field_writes uses this to detect net-zero flapping (M3.flap). Implementation lives in store.py::_decode_row (the inverse of _coerce_value).

Consequences

Test coverage

tests/test_reactive_bus.py::TestManualEffectDeferral — 7 cases (pinning every clause of M1–M5):

Basic deferral contract (M1, M2):

1. test_set_field_then_transition_does_not_clobber_by_reactive — canonical Bug 1 reproducer; would fail loudly if deferral reverted
2. test_reactive_after_effect_uses_final_state — proves cascade fires exactly once post-effect (not mid + post)
3. test_nested_manual_effects_flush_only_at_outermost_exit — nested manual dispatch; flush only at outermost exit
4. test_reactive_still_fires_on_plain_store_writes — guards against the over-fix of turning deferral into a global kill-switch

Hard guarantees — the three “刁钻” probes (M3, M5):

5. test_interceptor_sees_only_complete_post_effect_snapshot — a spy reactive rule records what __state__Task is at the moment its trigger fires; dead-line: must equal the post-transition "done", never the pre-transition "in_progress". Pre-fix run records "in_progress" because the cascade fired mid-effect.
6. test_cross_field_correctness_rule_clobber_count_is_zero — a production-shaped correct_task_status reactive rule; clobber count must be exactly 0 because when the cascade fires post-effect the field and state are equal and the guard rejects. Pre-fix run records clobber_count = 1.
7. test_partial_effect_failure_invisible_to_reactive_observers — effect writes label then transitions to an invalid state (raises EvalError). Three layered assertions:
   • DB rollback: label field is gone
   • Reactive spy: zero notifications on (field "label")
   • Direct on_commit hook: zero set_field events for label All three must hold simultaneously — M5’s atomic boundary is what makes M3’s visibility contract survive partial failures.

SpyObserver-driven assertions — call-count / phase / consistency-point (M1, M2, M3, M5):

Tests 8–11 use the tests/_spy.py::CascadeSpy infrastructure (see “Wiring summary” below). These pin the contracts that no rule-side effect can reach: the exact number of cascade dispatches, the phase at which each fires, and the store snapshot captured at the moment of dispatch.

8. test_spy_call_count_is_exactly_one_post_effect — a manual effect with one set-field + transition fires each matching reactive rule exactly once at phase "post-effect". Uses spy.assert_count("tap", 1) and spy.assert_phase("tap", "post-effect"). A pre-fix mid+post double-fire shows up as count == 2; a buggy future implementation that retries shows up the same way.
9. test_spy_snapshot_consistency_at_fire_time — when correct_label_to_state fires, __state__F in the captured snapshot MUST be "done", never "open". Uses spy.assert_snapshot_state("correct_label_to_state", field="__state__F", equals="done"). This is the literal “if state changes, the marker must change at the same instant” dead-line, mechanically enforced at the snapshot boundary.
10. test_spy_records_zero_calls_on_partial_effect_failure — partial-effect failure (M5 atomic rollback) leaves the spy with zero records. Uses spy.assert_count("spy_on_label", 0). A half-fix (bus deferral without store.atomic) would leave the events in _pending_events and end_manual_effect would still flush them — count would be 1.
11. test_spy_temporal_sequence_across_rules — when one manual effect produces multiple events matching different rules, the cascade fires them in commit order at phase "post-effect" and each call’s snapshot is the same consistency point. Uses seq ordering and explicit __state__<M> equality between snapshots. Catches both ordering bugs and consistency-point divergence.

Backward compatibility

• No DSL change. Existing .scm ontology files work unchanged.
• No store schema change.
• No CLI surface change.
• Behaviour change is observable only via:
  • Corrected field/state consistency after manual actions
  • Audit event timing (events now appear at scope-exit time, not mid-effect) — observable in tx_log inspection

Performance

• One additional counter increment/decrement per dispatch
• Pending-event queue is bounded by writes per scope (small in practice; rarely > 10 in current ontologies)
• No recursion change; _depth semantics unchanged
• Negligible overhead measured in test_sprint_rollover_e2e.py: full suite time unchanged

Documentation debt

• reactive.py top-of-file docstring (“Cascade & cycle protection” section) will be updated to point at this ADR and state M1–M4
• engine.py::dispatch docstring will reference this ADR for the begin/end block
• facade.py will reference this ADR at the back-reference wiring

Rejected alternatives

• 方案 B — Memory shadow snapshot / MVCC: full comparison in the “Approach” section above. Briefly: cost model mismatch with current ontology sizes, late-conflict retry layer not justified by today’s workloads, and the self-read semantic diverges from SQL transaction familiarity. Revisit if any trigger condition documented in “Approach → When B would become the right choice” appears.
• Defer all writes (not just manual effects): breaks the reactive-from-external-write contract — e.g. CLI users expect a set_field to immediately trigger an audit / writeback cascade
• Snapshot-and-rollback on conflict: too heavy; the cascade is best-effort audit, not transactional rollback
• Per-field last-write-wins reordering: silently hides the inconsistency; makes correct_task_status-style invariants impossible to express
• 方案一 (FIFO 原批发射) with opt-in LWW via conflict_policy: initially proposed as a softer alternative. Rejected by the 2026-06-29 architecture review on two grounds:
  1. Visibility gap is structurally inevitable, not a config knob. A reactive rule that reads the same field it just rewrote (e.g. (set-field x (+ (field x) 1))) reads from a snapshot whose freshness depends on the FIFO queue’s processing order; off-by-one bugs trace out as a Stale Read Under Mutation (the “幽灵 bug” pinned by the dispatch-count tests). Folding the payload vs DB-value gap with reactive cascade rules only papers over the architecture.
  2. Complexity does not move up to DSL authors. Any per-rule or per-effect opt-in forces every .scm author to reason about “will this effect’s repeated writes dedup or not?” That is the exact cognitive load ADR-0014 was supposed to remove. The reviewer-mandated fix: forced LWW dedup at flush time, no opt-in. Tests under TestFlushDedupForced pin the contract.
• Tag-triggered scope (per machine): too coarse; machine scope covers all writes to that supertag, not one effect block

Cross-references

• ADR-0005: Prefer Reactive Semantics Over Imperative Orchestration — this ADR refines the cascade timing without touching the reactive-over-imperative decision
• ADR-0010: Durable Agent Work Outside Reactive Bus — confirms the reactive bus is for synchronous audit/derivation only, not durable work; deferral fits this scope
• ADR-0015: Implicit Effect Scope — Dual-Entry Design — extends ADR-0014 with the effect_scope() context-manager API and the Scheme B safety net in _eval_begin
• Bug 1 / 2026-06-29 — original report in runex-bugs-discovered-2026-06-29.md