Roadmap

Where the architecture is going — future direction only. This file does not track what shipped or restate what the system is; it points at the docs that own those:

Planning detail (phase task breakdown, requirement IDs, acceptance criteria) lives in tasks/, not here. When a milestone ships, its tasks/ PRD + checklist become its archived record, the changelog gets the release entry, and this file’s pointer advances to the next milestone.

Shipped: v0.2.0 — Proactive Agent Runtime

Delivered the first production-shaped reflex arc: ontology state durably dispatches slow Agent/tool work, survives restart, requests human approval, and continues through ordinary reactive transitions —

Obsidian Thought / Session → extraction WorkItem signaled → worker runs it
  off-bus → Task proposal written back → human approves → Task materialized,
  every step visible in graph state and event history

Record: changelog.md [0.2.0]; spec + acceptance: ../tasks/prd-state-machine-proactive-agent-framework.md (§10) and its checklist ../tasks/tasks-state-machine-proactive-agent-framework.md.

Next: post-v0.2 direction

The phases below are planned, not yet specced — each needs a tasks/ PRD + checklist when it is picked up. Numbering follows the canonical tasks/ plan (PRD §9, §11): Phase 7 and 8 are the original post-v0.2 phases; Phase 9 is the agent-mesh capability ADR-0012 added (next free number — see its note).

Phase 7 — Assertion / Conflict / Current Truth

Outcome: the system can answer “what is currently true, on what evidence, and who adjudicated it.”

• define Assertion / Conflict / Decision as an ontology pack (Supertags + Machines), extending the minimal sync-only Conflict from Phase 1 into a full fact-conflict model
• produce provenance-rich semantic-extraction proposals (source, valid time, confidence, authority)
• implement deterministic current-truth projection with human adjudication
• add two-layer context: raw-material/vector retrieval at extraction time, graph retrieval at execution time
• dogfood a “project delivery date contradicts itself over time” scenario

Exit gate: a contradictory pair of assertions yields one current-truth projection plus a visible Conflict, and an adjudication decision is recorded with its provenance.

Phase 8 — Artifact & external collaboration

Outcome: humans, Agents, Obsidian, and one external Channel collaborate without silent loops or split-brain.

• Artifact/Blob references and a Bookmark-intelligence scenario (artifact nodes and links, not binary payloads in events)
• extend field-ownership to a Feishu Command ingest path
• reuse the Conflict/Approval model for external-channel write conflicts
• a schema-migration plan plus a scan-driven batch upgrade

Exit gate: a Feishu-originated command and an Obsidian edit to the same node’s machine-owned field produce a Conflict (never an overwrite), and a schema migration upgrades existing nodes in place via scan.

Phase 9 — Agent mesh: dispatch as DSL primitive, Agents as graph nodes

Phase 9, not a renumber of the reserved 7/8: agent-mesh is a capability ADR-0012 added during v0.2.0 acceptance, so it takes the next free number. Its keystone was pulled forward (the two Done items below already shipped).

Outcome: the dispatch decision (when, which Agent, what context, where the result routes) becomes evolvable .scm data rather than Python plumbing, so the Phase 6 gate can eventually let an Agent reprogram routing itself. Multi-Agent collaboration composes from existing reactive cascades — no new orchestration mechanism. Decided in adr/adr-0012-agent-dispatch-via-dsl-primitive-agents-as-graph-nodes.md.

• Done — add a (signal-work …) effect-only primitive: a thin shell over the blessed signal_workitem SoT (no WorkItem-contract re-encoding in the DSL); guard-disallowed; surfaced in ontology primitives. (signal-agent follows with the Agent bundle below.)
• Done — supersede the Phase 5 ExtractionScheduler polling peripheral: its dispatch decision now lives in the reactive request_extraction action, which calls (signal-work …) inline. The action moved out of core-memory into a new proactive extraction.scm pack (loaded with workitem.scm), so core-memory stays self-contained — a Thought is just not auto-dispatched when the proactive layer is absent (graceful no-op). The Python scheduler, the host’s signal stage, and the intermediate extraction_requested state are deleted.
• model Agent / AgentRun / Session as Supertags + Machines (e.g. dispatched → running → done), exactly like WorkItem/Proposal/Approval
• add one generic dispatcher leaf (kernel) that reads the target Agent node (CLI command, allowlisted tools, output schema) and subprocesses to the named CLI — no Agent-specific Python orchestration
• extend the Phase 6 draft → verify → approve → deploy gate to schema / machine / guard drafts under the same gate (the non-negotiable guardrail: an Agent may never load evolved brain structure directly into production)

Exit gate: a reactive action dispatches an Agent via (signal-agent …) with no Python scheduler in the path; the dispatched work runs off-bus under a worker lease; a second reactive action routes the result to a QA Agent; and the dispatch policy is a .scm ActionDefinition that the Phase 6 pipeline can draft and re-deploy.

Operability & onboarding backlog (2026-06-27 review)

Opportunistic, IP-flywheel-scoped (not committed phases): pick up when a story needs them, keep each within the ≤maintenance budget. Each is read-time assembly or a thin skill — no new machines/kernels unless noted.

• runex ps — read-time operator console. ✅ Shipped (2026-06-27). One read-only screen assembled from existing state: the WorkItem queue by state + backlog total (surfaces piles like the 1600-item extract backlog), an In-flight panel (which agent/handler is doing what, its goal, attempt progress n/max, and owning worker — from WorkItem fields; deeper per-run history on AgentRun), inbound IngestJobs with connector→recipe/source/mode/ cursor, outbound projection policies per supertag (human-owned vs ontology-owned = the bidirectional direction), and the last N tx. --json for agents. The monitor skill closes the conversational loop (“现在在跑什么” / “what’s running” → runex ps). No control-plane; pure read-time composition, zero new machines/kernels. (Q7)

• runex top — live operator TUI. ✅ Shipped (2026-06-27). The top to ps’s snapshot: a Textual app (src/runex/tui.py) auto-refreshing every --interval seconds over the same _gather_ps data, surfacing what only motion reveals — a daemon heartbeat (worker-host-cursor.json mtime → live/STALE; the “is my engine actually running my work” signal a snapshot can’t give), backlog trend + throughput deltas, in-flight agents with a live lease countdown (red when expired = stuck/crashed worker), and a scrolling event tail. Read-only polling client over data.db — never mutates, never connects to the daemon process (the dockerd/many-clients model). When a socket runexd lands it can switch poll→push without changing what it shows. Deliberately not a full btop (no mouse/graphs/multi-screen). (Q7)

• Vault onboarding skill — two modes, never mutate a user’s existing vault. new vault — ✅ Shipped (2026-06-27) as the vault-setup skill: scaffolds a fresh Obsidian dir (one folder per built-in type), loads the packs those types need, and saves an inbound IngestJob so human notes flow in and machine-owned fields project back. Reuses built-in supertags; invents nothing. existing vault — ✅ Shipped (2026-06-27) as the vault-adopt skill: an orchestrator that surveys the user’s vault read-only, hands the patterns to scm-authoring to design fitting types, generates a fresh go-forward vault via vault-setup, and optionally does a one-shot read-only import (listen --once --vault, no saved job, source files unchanged — verified). The original stays a read-only archive: it never carries the round-trip contract, so it is never broken by a schema/engine upgrade — one vault owns the contract, not two. One dir = one object, same as the engine’s source-of-truth. Aligns with ADR-0011’s ownership model. (Q4)

• scm-authoring debug skill — ✅ Shipped (2026-06-27). Lets an agent build new business .scm without reading engine source: a triggered skill that routes to ontology primitives --json + the DSL/pattern references, drafts to ~/.runex/ontology/ (never the wheel’s site-packages copy — survives upgrades), and runs the load → check → verify loop. Closes the “agents need source to debug” gap without open-sourcing the engine. (Q3/Q9)

Broader product surface & non-goals

The full capability map (semantic compilation types, current-truth projection, two-layer retrieval, blob/artifact handling, Feishu/Obsidian collaboration, proactive scenarios) and the explicit non-goals (autonomous open-ended self-scheduling, a general-purpose workflow/DAG product, pretending SQLite gives multi-machine strong consistency) are specified once, in the PRD — not restated here: ../tasks/prd-state-machine-proactive-agent-framework.md §6 (capability map), §4 (non-goals), §11 (follow-up candidates).

Open architectural positions (undecided)

Forks recorded so a future decision has a baseline — not yet ADRs:

• runexd — headless daemon vs in-process monitor. Current position: the MonitorOrchestrator (Obsidian file-watcher, Claude session poller, OpenCode DB poller) runs inside whatever long-lived process the user has open (currently the tray app). The right end-state is a standalone runexd process managed by launchd / systemd / Windows Service, with the tray app and CLI connecting to it over a Unix socket or localhost HTTP. This decouples sync liveness from UI liveness — sync continues even if the tray crashes, and future GUI clients (browser, Obsidian plugin, etc.) all share one daemon without re-implementing the orchestrator. Not yet specced; pick up when the tray packaging story stabilises.

• Multi-machine / cross-agent consistency. Current position: the vault is the sync layer — each machine runs its own runex instance ingesting the same synced files and converges; eventual consistency comes from the file-sync layer, not runex. Reactive effects must be idempotent because two machines may fire the same action on the same event. A genuine multi-writer strongly- consistent store would be a major architectural shift; not the right problem until real multi-machine dogfood surfaces concrete conflicts.

• GUI / runex-product = read-only peripheral, not a second engine. The listen host is already the headless daemon the runexd note describes: ingest+work+materialize+project run with no UI attached, so a GUI dying cannot stop sync. Position: any GUI (incl. runex-product) is another peripheral reading the same Store — it never re-implements the orchestrator and is not a separate distributable engine. Time-triggered events stay single-point (one host cron fires (scheduled) actions); many CLI instances may read/write the graph, but only the cron-holder fires schedules. Open part: whether runex-product collapses into “opinionated starter vault + schema pack + tutorial” layered on the one engine, vs. staying a separate repo. Lean: collapse it — the engine ships generic, the product ships content. (Q6/Q8)

• Extension discovery is not test-isolated. discover_sources/sinks/… resolve ~/.runex/extensions from Path.home(), not RUNEX_HOME. A user who drops in a real source extension (the sanctioned “no core edit” path, Invariant 12) breaks tests/test_datasource_registry.py, which asserts a fixed datasource catalog. Either tests isolate the extension base, or extensions anchor to RUNEX_HOME — the latter touches the trust boundary, so it needs an ADR, not a drive-by. Surfaced 2026-06-27 by an opencode-transcripts source extension. (Q2)

• Ontology metadata links (MachineDefinition / ActionDefinition / Supertag). Current position: these definitions reference each other by name (text fields), not by typed edges. ActionDefinition.machine → "Task" is a string, not a belongs_to link to the MachineDefinition node. This is fine at ~100 defs (name resolution at startup costs ~2ms), but becomes a gap when ontology definitions themselves need reference integrity, graph-topology diff/merge, or cross-instance migration. The fix is a data-action (link-metadata-defs) that walks all def nodes and creates the corresponding links — no Python migration framework, no Alembic. Deferred until ontology versioning or multi-instance diff surfaces concrete pain.

• Recipe ↔ Supertag-schema coupling — intentional, but counts. ADR-0008 carves the ingest pipeline as Capability → Connector → RawRecord → Recipe → CanonicalItem → Store. Recipe (Python, in-process) cross-reads the loaded .scm manifest (manifest()) to type-correct its CanonicalItem emission; Store (SQLite) accepts whatever field names Recipe hands it, typed by the value_type Recipe declares. The coupling is fixed and small today (recipe count = connector × projection pair count, currently 6: obsidian-mirror / -frontmatter / three session mirrors / proposal-project). Each recipe module documents which supertag + fields it writes.

  When it could bite:

  • Recipe count grows > 10 and recipes start sharing projection patterns (mirroring identity fields, body → longtext, frontmatter-projection rules) — each Python module re-implements the same scaffolding, with drift (see obsidian_mirror_recipe.py:167 comment claiming “the engine will reject unknown fields at set_field time” — factually wrong; Store accepts dynamic fields today. The only strict path is the BusinessRule compiler at worker/rule.py:14, which validates reactive-rule field references against the declared schema).
  • The reactive-rule compiler blind-spot becomes load-bearing: a rule that wants to observe frontmatter.title from a typeless Obsidian note (a field Recipe just wrote dynamically) cannot, because the field isn’t in the supertag schema, so the verifier rejects the rule.
  • Long-tail ingest on arbitrarily-shaped sources (CSV exports, third-party APIs returning JSON with surprise keys) demands dynamic field writes; today the path works at runtime but is invisible to rule / FTS layers (declared longtext fields get FTS indexing; dynamic longtext fields do not).

  Migration shape if revisited (sketch only — not a plan):

  • new .scm form (ingest-target <supertag> (mirror-fields ...) (body-field ...) (frontmatter-projection dynamic|strict)) co-located with (supertag ...) / (machine ...) / (action ...) in the DSL;
  • one general Python recipe reads the declarations and routes accordingly;
  • migrate the existing 6 recipes by deleting Python and appending .scm blocks — the recipe-scm coupling collapses from “every move is a Python edit” to “schema change = edit the matching .scm block”.

  Decision until then: stay. The coupling is bounded and documented. Each recipe module names its supertag and field set in its own docstring. The plan is not to migrate; only to watch for the trigger conditions above.

• DSL evolution — user-defined functions in .scm, never business combinators in primitives. Current position: the DSL primitive catalog (src/runex/ontology/primitives.py) stays domain-neutral (set-field, transition, create-link, call-kernel, signal-work, etc.) and the DSL evaluation surface (src/runex/dsl/eval.py) is intentionally minimal — docs/dsl-spec.md forbids define, lambda, and macros. Business logic is therefore written in .scm as imperative sequences of those primitives, with let / begin / nested if for control flow. The intent — per AGENT.md §11.2 (“Logic in .scm, not Python”) and ADR-0018 (engine keeps minimal substrate) — is that the engine stays a substrate that knows nothing about “revenue”, “quarter”, “proposal”, or any other business concept; the SCM author composes whatever domain abstractions the ontology needs.

  Where this starts to hurt: as SCM authors push past the trivial-mutation shape (the dogfood case is tests/fixtures/ontology/goal_handoff.scm’s count_won_deal, which nests two lets and a begin of seven primitives to express “count this won deal against the linked quarter goal and roll up completion rate”), the effects read like an accounting script rather than a business rule — the structure of the intent is hidden inside the procedure. Symptom, not cause: the DSL lacks any mechanism for the author to factor a multi-step computation into a named helper they can reuse across actions. Today their only options are (a) inline the steps everywhere they are needed, or (b) promote the helper to a full action with its own machine / from-states / trigger — overhead that doesn’t match the intent of “a 4-line recipe for how to compute 完成率”.

  Hard constraint (do not regress): even as DSL expressiveness grows, primitives never absorb business concepts. Adding (revenue-add q amt), (update-quarterly ...), (process-crm-deal ...) to primitives.py is wrong on principle — the engine would know about revenue / quarter / deal, violating ADR-0018 and §11.2. The right shape is always: SCM authors compose their own combinators from generic primitives.

  Why Scheme: the language was chosen explicitly for extensibility — homoiconicity (code-as-data), closure capture, and a macro system that can rewrite the AST before evaluation. ADR-0007 (“Python-as-host-and-extension-interface”) treats .scm as the user-extensible language surface, not as a closed vocabulary. The absence of define / lambda in the current DSL is therefore a deliberate “no yet” rather than a “no forever” — the substrate was kept simple to ship the reactive spine first; language features belong to a later milestone.

  Candidate extensions, in order of likely return:

  1. User-defined functions in .scm (define + lambda), no macro. Adds closure capture to the evaluator; SCM authors write (define (revenue-add q amt) (set-field-on q "当前营收" "number" (+ (field-on q "当前营收") amt))) and reuse. Engine stays zero-business-concept; combinators live where they belong. Cost: parser + evaluator (closure env, tail-call-or-not — keep if/begin/let semantics), backward-compat verification across all bundled SCM, a focused closure-correctness test suite, and a re-run of the dogfood scenarios to make sure no existing action silently changes meaning.

  2. Macro for the implicit effect_scope wrap (Option B from the prior internal review: rewrite _SPECIAL["begin"] so every begin block transparently opens a manual effect scope). This is already partially satisfied — engine.dispatch opens bus.effect_scope() at the action root, and the bus supports re-entrant nesting (reactive.py:179 effect_scope). What’s missing is the same guarantee for begin blocks invoked outside a manual dispatch (rare today; matters if user-defined helpers run raw). Cost: lower than (1) — evaluator change only. Risk: surprising semantics if a begin the author thought was local ends up buffering events. Worth doing only if (1) lands and authors start writing helper functions that call begin / multi-statement bodies outside an action context.

  3. Business combinators in primitives — explicitly REJECTED. Not “deferred”, rejected: violates ADR-0018 and §11.2. If anyone is tempted, the answer is to write the combinator in .scm once (after (1) lands) and call it.

  What’s already settled and need not be revisited:

  • Reactive-over-imperative (ADR-0005) stays. Even with user-defined functions, action effects are sequences of mutations, and the ReactiveBus remains the only coordinator; functions are scope helpers, not control-flow primitives.
  • Engine minimal substrate (ADR-0018) stays. Adding define/ lambda to the DSL is on the user-facing language side and does not change what the engine knows; it stays domain-neutral.
  • The Python evaluator (src/runex/dsl/eval.py) stays <300 LOC and pure tree-walking; no new runtime types, no new dataflow machinery.

  ADRs that may need to change (or a new one written):

  • docs/dsl-spec.md currently states “No macros, no define, no lambda, no tail-call optimization.” Lifting that one sentence (define + lambda only; keep macro out until a concrete use case) is the bulk of the work. If we want to codify the principle (“primitives never absorb business concepts; user-defined functions are the canonical extension path”), that deserves its own ADR so it survives contributor turnover.
  • ADR-0018 does not need editing — it already says “the engine stays domain-neutral”; adding user functions to the DSL doesn’t change that. A short pointer in ADR-0018 to the new “primitives stay business-concept-free” ADR would help future readers connect the dots.

  Status / restart condition: not now. The 0.2.0 release line ships as-is; goal_handoff.scm is the only .scm that crosses the “nesting hurts” line and even there the imperative shape is still readable. Revisit when any one of these becomes true:

  • a single action’s effect body exceeds ~30 effective lines (counting begin / let / nested control, not blank lines);
  • the same 5+-line step pattern appears verbatim in three or more distinct actions across bundled SCM;
  • a third .scm author (human or AI agent) is observed hand-inlining what should obviously be a named helper. When any of those fires, draft adr-XXXX-dsl-user-defined-functions.md with §11.2 / ADR-0018 cross-references and a concrete migration plan for the affected bundled SCM, before touching the evaluator.

Resolved forks now live in their ADRs: the bidirectional-sync source-of-truth question is settled by adr/adr-0011-obsidian-ontology-human-sovereign-mirror.md (has been in effect), the derived-vs-native node duality is documented in architecture.md.