agent-core-system

agent-core-system is the host-control layer for an agent runtime. It sits above the proven substrate and beside provider domains such as agent-memory; providers are self-contained, while this repository owns admission, authority, audit, execution envelopes, workflow gates, and the narrow lifecycle seam that can start/connect to one configured rpc-plugin-system substrate daemon.

Purpose

This repo defines and tests the core contracts that keep provider execution from becoming a permissions shortcut:

• capability registry normalization and provider descriptor admission
• invocation candidate validation before execution
• approval grants, revocation, freshness, and audit correlation
• sealed execution envelopes owned by the host
• provider execution/runtime normalization
• host state-machine transition contracts
• instruction, tool-skill, trigger, and Lua advisory boundaries
• provider handoff/evidence contracts
• workflow manifest enforcement for branch, source, and evidence layout
• generic cmd/tui operator surface over host-normalized descriptors, admission, execution, approval, events, and audit

Non-goals

• Providers do not get to own host admission, retry, approval, audit, or revocation semantics.
• Memory is not architecturally special; it is one durable-write provider domain example.
• Lua, tool-skills, and deterministic triggers may propose candidates only. They do not mint executable authority.
• Workflow state does not live under src/.
• This repo does not use a dependency grab bag. Go stdlib plus golang.org/x/* only.

Layout

• src/ — Go source, tests, and examples only.
• workflow/ — workflow records, feature evidence, reviews, postmortems, and generated coverage artifacts.
• workflow.toml — canonical workflow manifest.
• tool-skills.md — top-level core tool-skill asset document.
• docs/PRD.md — product requirements and acceptance criteria for the client/broker/provider core.
• docs/architecture.md — accepted architecture baseline.
• docs/implementation-spec.md — implementation decomposition contract.
• src/cmd/agent-core-host — host-control entrypoint that owns one configured local rpcplugind process and reads redacted substrate status/routes as inert facts.
• src/cmd/tui — first-party operator TUI entrypoint; fullscreen on interactive terminals, line mode for scripted/non-TTY use, and explicit --fullscreen / --line overrides.
• src/internal/substrate — narrow lifecycle/admin seam for starting rpcplugind, passing explicit provider specs, reading admin status/routes, and stopping only the owned process.

How to use agent-core-system

Use this repository as the host-control core for an agent runtime. It is not a provider framework, not a plugin's private state store, and not a shortcut around approval. The core owns normalized capability admission, approval, execution envelopes, audit, host state, operator surfaces, and configured substrate lifecycle ownership. Providers own their own feature semantics.

1. Verify the checkout

Before trusting any checkout or branch, run the full gate:

make verify
git diff --check

make verify runs the workflow checker, all Go tests, build, and coverage artifact generation. A green build without workflow evidence is not enough; canonical implementation state lives in workflow.toml.

For faster local checks while editing:

make doc-check   # workflow manifest/layout check
make test        # all tests
make build       # workflow check + go build ./...
make coverage    # refresh workflow/artifacts/coverage.*

2. Read the boundary before adding anything

The design rule is simple:

provider/plugin proposes or implements feature behavior
agent-core-system admits, denies, executes, audits, and controls authority

Read these before changing contracts:

• docs/PRD.md — product requirements, scope, non-goals, and acceptance criteria.
• docs/architecture.md — authority, trust, provider, TUI, Lua, and production-host boundaries.
• docs/implementation-spec.md — feature decomposition, schemas, required tests, and done gates.
• workflow.toml — canonical state for root features and feature slices.
• workflow/artifacts/global-coverage-map.md — accepted coverage posture and known gaps.

Do not put provider feature semantics into core. If a provider needs domain behavior, put that behavior in the provider and expose normalized descriptors/capabilities to core.

3. Use the workflow manifest

Check the current canonical state:

go run ./src/cmd/workflow-check
sed -n '1,140p' workflow.toml

When adding work:

1. Add or update a root/feature entry in workflow.toml.
2. Put workflow records under workflow/root/... or workflow/features/....
3. Put source under src/..., never workflow paperwork under src/.
4. Record evidence under the feature's evidence/ directory.
5. Run make verify before calling the slice done.

The checker rejects workflow records under src/ and unknown canonical manifest fields. That is intentional. Do not route around it.

4. Integrate a provider or plugin

A provider/plugin should be self-contained. Core should see only host-normalized records and admitted assets:

• provider identity and availability
• capability descriptors
• command/target/flag descriptors when operator commands exist
• tool-skill or tool-skill-equivalent assets after host admission
• Lua hook or Lua machine descriptors after host admission
• generation stamps and provenance

The provider must not get direct ownership of host policy, approval grants, retry rules, audit facts, or execution authority. Invocation candidates from providers, tool-skills, Lua, triggers, clients, or LLM planning are only candidates. Host admission creates executable envelopes.

Provider diagnostics are structured audit facts, not payload logs and not authority transfer. Core may ingest or correlate provider diagnostic facts only as redacted, non-authoritative observability records inherited from the agent-provider-bundle-contracts / rpc-plugin-system provider observability contract. Diagnostic join keys such as provider, capability, operation, generation, status, counts, durations, and correlation IDs do not grant permission, satisfy broker admission, create authority refs, expose provider-private material, or become client-visible surfaces.

Final provider diagnostics status: accepted with constraints in the cross-repo judge sweep. Core may correlate redacted provider diagnostic facts for observability, but diagnostics never become descriptor admission, usage admission, executable authority, route selection, owner selection, approval evidence, or a client-visible provider API.

Client/broker/provider authority model

The core system is the broker between clients and providers. The provider advertises to the broker; the client requests from the broker; the broker decides what exists and what may run.

Canonical flow:

provider submits tool-skill descriptors and Lua state-machine scripts to the broker
  -> broker admits and emits the advertised client surface
  -> client requests broker-emitted operations
  -> broker applies advertisement authority and usage authority
  -> broker admits/denies and emits the executable use contract
  -> authority owner issues/resolves opaque authority_use_ref when external authority is needed
  -> provider executes only through admitted mediated authority
  -> audit links provider advertisement, broker emission, client request, admission, issuance, use, and result

A tool-skill descriptor says what the provider can do. A Lua state-machine script says how that advertised operation is shaped or driven. Neither is permission, and neither is client-visible until the broker admits and emits it.

Clients may request only broker-emitted operations. A client request is not executable authority. Provider "I can" and client "I want" become "I may" only when the broker admits usage and seals an execution envelope.

Provider diagnostic records never shortcut that flow. They may help audit and troubleshooting after redaction, but the broker must not treat diagnostics as advertisement admission, usage admission, permission evidence, executable authority, or provider-to-client API surface.

5. Submit work through host-owned paths

The intended flow is:

candidate/request
  -> host registry and generation validation
  -> host policy and side-effect classification
  -> approval/grant check when required
  -> sealed execution envelope
  -> provider execution through normalized capability
  -> normalized result and audit/event records

Never build a direct candidate-to-provider execution path. Never let the TUI, Lua, a tool-skill, or provider metadata mint approval authority.

6. Start the core host with a provider binary directory

agent-core-system is the host-control owner above rpc-plugin-system; it is not a provider and not a substrate plugin. The local startup shape is explicit and directory based:

provider repos -> executable provider files -> one provider directory -> agent-core-host -plugin-dir <dir>

agent-core-host starts exactly one configured rpcplugind, passes the desired plugin set to it, waits for the admin socket, reads redacted substrate status/routes, prints one JSON status snapshot, and owns shutdown of that substrate process. Those substrate facts are observability only. Plugin-directory presence, process health, capabilities, routes, PIDs, and generations do not become broker admission, authority ownership, executable handles, or client-visible tools.

Directory contract

-plugin-dir DIR is intentionally dumb:

• only direct entries in DIR are considered;
• dotfiles are ignored;
• directories and non-executable files are ignored;
• executable regular files are provider plugins;
• executable symlinks are accepted only when their target stats as a regular executable file;
• the plugin id is the executable basename;
• an empty configured plugin directory is denied;
• explicit -plugins id=path,... and -plugin-dir DIR may be combined, but duplicate plugin ids are denied by the substrate config validator.

Do not put rpcplugind, test helpers, shell scripts, or unrelated executables in the provider directory. If it is executable and directly inside that directory, core will hand it to rpcplugind as a desired plugin. That is the mechanism. No discovery theater.

Tutorial: configure and start a local host

This tutorial follows the current repo evidence for the plugin-directory slice. It builds rpcplugind, rpcpluginctl, and substrate-compatible provider binaries; puts only provider binaries in the provider directory; then starts the core host.

1. Pick paths. Keep runtime state and build output outside the repo:

   ROOT=/tank/development/linus
   RUN=/tmp/agent-core-provider-run
   BIN="$RUN/bin"
   PLUGINS="$RUN/plugins"
   RUNTIME="$RUN/runtime"
   rm -rf "$RUN"
   mkdir -p "$BIN" "$PLUGINS" "$RUNTIME"
   

2. Build the substrate daemon and admin CLI:

   (cd "$ROOT/rpc-plugin-system/src" && \
     go build -buildvcs=false -o "$BIN/rpcplugind" ./cmd/rpcplugind && \
     go build -buildvcs=false -o "$BIN/rpcpluginctl" ./cmd/rpcpluginctl)
   

3. Build provider executables into the provider directory. The output filename is the plugin id, so choose the basename deliberately:

   (cd "$ROOT/agent-filesystem" && \
     go build -buildvcs=false -o "$PLUGINS/filesystem" ./src/cmd/agent-filesystem)
   (cd "$ROOT/agent-exec" && \
     go build -buildvcs=false -o "$PLUGINS/exec" ./src/cmd/agent-exec)
   (cd "$ROOT/agent-memory" && \
     go build -buildvcs=false -o "$PLUGINS/memory" ./src/cmd/agent-memory)
   (cd "$ROOT/agent-network" && \
     go build -buildvcs=false -o "$PLUGINS/network" ./cmd/agent-network)
   

4. Check the provider directory before startup:

   find "$PLUGINS" -maxdepth 1 -type f -perm -111 -printf '%f -> %p\n' | sort
   

   The printed basenames are the plugin ids core will request. If rpcplugind, rpcpluginctl, shell scripts, test helpers, or random executables show up here, fix the directory before starting. This is not a suggestion.

5. Run a one-shot smoke start:

   go run ./src/cmd/agent-core-host \
     -rpcplugind "$BIN/rpcplugind" \
     -runtime-dir "$RUNTIME" \
     -plugin-dir "$PLUGINS" \
     -startup-timeout 10s \
     -dial-timeout 2s \
     -once
   

   -once starts the substrate, verifies admin status, prints one JSON status snapshot, then stops the owned daemon. Use this before trying a long-running host.

6. Start the local host for an operator session by omitting -once:

   go run ./src/cmd/agent-core-host \
     -rpcplugind "$BIN/rpcplugind" \
     -runtime-dir "$RUNTIME" \
     -plugin-dir "$PLUGINS" \
     -startup-timeout 10s \
     -dial-timeout 2s
   

   Keep that process in the foreground. Stop it with Ctrl-C; core will stop only the rpcplugind process it owns.

7. From another terminal, inspect the substrate admin socket if needed:

   "$BIN/rpcpluginctl" -runtime-dir "$RUNTIME" status
   "$BIN/rpcpluginctl" -runtime-dir "$RUNTIME" plugins
   "$BIN/rpcpluginctl" -runtime-dir "$RUNTIME" routes
   

   These commands expose substrate health and route facts only. They are not broker admission, not authority grants, and not client-visible capability publication.

Equivalent explicit specs are supported when ids should not match basenames or the directory contains files that must not be treated as providers:

go run ./src/cmd/agent-core-host \
  -rpcplugind "$BIN/rpcplugind" \
  -runtime-dir "$RUNTIME" \
  -plugins "filesystem=$PLUGINS/filesystem,exec=$PLUGINS/exec"

A TOML -config may also be passed through to rpcplugind, but agent-core-host still requires a -runtime-dir so it can own and read the admin socket. Do not use config indirection to hide which providers are being started.

7. Use the operator TUI

cmd/tui is the generic operator client. It renders host-normalized descriptors, snapshots, approvals, results, and bounded event/audit streams. It is not a provider-specific console and it does not bypass host admission, approval, execution, retry, cancellation, revocation, or audit.

The TUI is meant to attach after agent-core-system is already running. Starting the TUI must not bootstrap the core runtime, become the host control loop, or require providers to expose private handles.

Try line mode without a running host:

go run ./src/cmd/tui --line /snapshot
go run ./src/cmd/tui --line "summarize current agent status"

This intentionally fails closed for host-backed data because no running host service is attached. That is correct behavior, not a setup failure.

Attach to a running host through a local process hook when one exists:

go run ./src/cmd/tui --line --process-hook /usr/lib/agent-core-system/operator-hook /snapshot
AGENT_CORE_TUI_PROCESS_HOOK=/usr/lib/agent-core-system/operator-hook go run ./src/cmd/tui --line /snapshot

The process hook receives one JSON request on stdin and returns one JSON response on stdout. Supported read requests are snapshot and events; action/write/control hooks are not part of this read-only attach path.

Use fullscreen only from an interactive terminal:

go run ./src/cmd/tui --fullscreen --process-hook /usr/lib/agent-core-system/operator-hook

Inside the prompt:

• core> /snapshot refreshes the host snapshot pane.
• core> /help toggles prompt help.
• core> /clear clears snapshot/result/event panes.
• core> /focus cycles pane focus.
• core> /quit exits and restores the terminal.

For scripted checks, stay in line mode. Do not scrape fullscreen output in automation.

8. Understand TUI input rules

• /plugin-name command target --flags routes through deterministic operator command handling.
• non-slash input routes to the normal task/LLM path.
• command, target, flag, Lua-machine, and Lua-hook facts must come from admitted host-normalized descriptors.
• missing, unavailable, stale, revoked, ambiguous, or denied descriptors fail closed.

Optional local TUI-side stores:

AGENT_CORE_TUI_PROCESS_HOOK=/usr/lib/agent-core-system/operator-hook \
AGENT_CORE_TUI_PENDING_APPROVAL_STORE=/tmp/agent-core-tui-pending.json \
AGENT_CORE_TUI_COMMAND_SESSION_STORE=/tmp/agent-core-tui-sessions.json \
  go run ./src/cmd/tui --line /snapshot

Those files persist local TUI-side pending/session state only. They do not issue grants, expose provider internals, or create execution authority.

9. Known current limits

Current production-host integration is deliberately conservative:

• process-hook attach is local and bounded; no TCP, HTTP, websocket, bearer-token network path, or shared provider handle exists here
• event/audit storage is currently in-memory in the host-integration slice
• agent-core-host owns a local child rpcplugind process, but there is no installed system service manager, durable daemon registry, network listener, or multi-host control plane here
• shutdown of the owned substrate process is implemented; richer provider drain/retry/cancellation policy remains feature-scoped work, not implicit runtime magic
• fullscreen sizing is deterministic in the first shell slice; live resize/raw-mode polish is later work

Do not hide those limits behind documentation theater. If you need durable storage, daemon lifecycle, network transport, or richer fullscreen behavior, add a feature slice with explicit acceptance criteria, tests, evidence, and coverage gaps.

Branch ladder

Promotion is fast-forward only through:

1. design-architecture
2. feature/*
3. development
4. test
5. main

design-architecture is allowed as the long-lived architecture-baseline branch. workflow-check validates actual local and gitea/* refs against settings.branch_ladder when .git is present.

Gates

Run the full gate before admitting changes:

make verify
git diff --check

make verify runs:

• workflow manifest/layout checks
• all Go tests
• Go build
• coverage artifact generation under workflow/artifacts/

Current coverage evidence is intentionally behavior-boundary based, not fake 100% line-coverage theater. Remaining accepted gaps must be recorded in workflow/artifacts/global-coverage-map.md.

Current status

The implementation slices currently declared in workflow.toml are done and promoted through gitea/main. The latest admitted state includes:

• full green make verify gates
• branch-ladder and workflow-layout enforcement
• capability registry, candidate/admission, approval, execution, host-state-machine, instruction/tool-skill/Lua, and provider-boundary contracts
• first fullscreen-shell operator TUI slice with host service seams, provider command-surface bundle admission, canonical Lua-machine specs, command-surface generation stamping, fake provider onboarding fixture, host execution-control channel seam, late-attach process-hook read support, and --line fallback
• Architecture A substrate lifecycle: src/cmd/agent-core-host starts one configured rpcplugind, reads redacted inert admin status/routes, and stops only the owned process
• plugin-directory startup: agent-core-host -plugin-dir DIR maps executable direct entries to provider ids by basename, with deterministic scan order and fail-closed empty/duplicate behavior

No provider-specific TUI, provider-private control path, or admission bypass is part of the current core.