fullsend run

Execute an agent locally in a sandbox. fullsend run resolves the agent harness, provisions a sandbox container, and runs the agent to completion.

Usage

fullsend run <agent-name> [flags]

Flags

Flag	Description
--fullsend-dir	Path to the .fullsend configuration directory
--runtime	Override the agent runtime from config.yaml for this run (claude, pi, codex, dummy or dummy-playback); also FULLSEND_RUNTIME
--model	Override the harness/agent model for this run (alias, model id, or provider/id on pi and codex — codex takes OpenAI ids only); also FULLSEND_MODEL
--effort	Override the harness effort level for this run (low…max); also FULLSEND_EFFORT
--output-dir	Base directory for run output (default: /tmp/fullsend)
--target-repo	Path to the target repository
--fullsend-binary	Path to a Linux fullsend binary to copy into the sandbox
--env-file	Load environment variables from a dotenv file (repeatable)
--no-post-script	Skip post-script execution
--keep-sandbox	Skip sandbox deletion after the run
--debug [filter]	Enable agent runtime debug logging with optional category filter (e.g. "api,hooks")
--forge	Forge platform to use (github, gitlab). When omitted, resolved from forge: in the config.yaml at the root of --fullsend-dir, then GITHUB_ACTIONS/GITLAB_CI
--offline	Reject network fetches; only use cached remote resources
--max-depth	Maximum dependency depth for transitive resolution (0 disables)

Plan block

At startup, fullsend run prints a plan block summarizing the resolved configuration:

Agent:     code
Role:      code
Model:     sonnet
Effort:    high
Runtime:   claude (from /path/to/.fullsend/config.yaml)
Image:     fullsend-sandbox:latest

The Runtime line shows which runtime was selected and the config source it was read from. When no config.yaml exists, the source reads default (config not found).

Runtime selection

The runtime for a run is resolved once, in this order: --runtime flag, FULLSEND_RUNTIME, runtime: on the agent's agents: entry in config.yaml / .fullsend/config.yaml, the repo-wide runtime: there, then the built-in claude. The same order applies to the model (--model, FULLSEND_MODEL, model: on the agent's agents: entry, harness model:, agent frontmatter; FULLSEND_PI_MODEL on pi and FULLSEND_CODEX_MODEL on codex are lower-precedence aliases, each read only when that runtime is the one selected) and to effort (--effort, FULLSEND_EFFORT, effort: on the agent's agents: entry, harness effort:). <agent> is the name given to fullsend run (triage, code, …); see Runtimes — per-agent settings. For Codex native children, --effort sets the root only; child roles retain their native model defaults (see Codex native children). FULLSEND_FALLBACK_MODELS=a,b becomes Claude Code's --fallback-model; pi uses it for aliased models on the top-level run when Vertex does not serve the model (two specific 404/403 messages, same provider only; sub-agent children get none); codex ignores it with a warning.

The plan block prints Runtime: <name> (from <source>) and, when an override applied, Model: <value> (from <source>); stderr carries runtime: selected "<name>" from <source> (and model: requested "<value>" from <source>) for scripts. A value from the config file is labelled with the file path, suffixed agents.<name> when the agent's entry decided. When models.aliases in .fullsend/config.yaml remaps the alias, the line keeps the alias and its source and adds the remap: Model: sonnet (from <source>) → claude-sonnet-5 (from <config path> models.aliases), with model: alias "sonnet" remapped to "claude-sonnet-5" from <config path> models.aliases on stderr. Aliased entries in the Fallback models line show as alias → id (sonnet → claude-sonnet-5, claude-opus-4-6 (from FULLSEND_FALLBACK_MODELS)); literal ids print as written; pi uses the chain for aliased models when Vertex does not serve the model and ignores it for pinned ids. An invalid override — unknown runtime, unknown effort level, an agents: entry that names no agent, or a models.aliases key or value the block does not accept — fails before the sandbox is created. Codex still requires an explicit OpenAI model ID: models.aliases does not make Claude aliases valid for Codex, even if the shared plan display shows a remap.

# try a repo's triage on pi with Gemini Flash, without touching its config
fullsend run triage --fullsend-dir . --target-repo ../repo \
  --runtime pi --model google-vertex/gemini-2.5-flash --effort medium

On pi, when the agent's skills ship sub-agent personas, Bootstrap prints the resolved per-persona table just after this block — one line per persona with the model it resolved to and which entry decided, plus the blanket default used by children that name no persona:

subagents: challenger → xai-vertex/xai/grok-4.6 (from subagents.challenger)
subagents: correctness → anthropic-vertex/claude-opus-4-6 (from frontmatter)
subagents: docs-currency → google-vertex/gemini-3.8-flash (from subagents.docs-currency)

A malformed subagents key or model reference is rejected by config validation, before the sandbox is created, like the other invalid overrides above. A key that names no discovered persona, or a model this run cannot serve, is caught slightly later — at Bootstrap, once the harness's skills have been read — so the sandbox exists but the agent has not started. See pi § Per-persona model configuration.

Security escalation

With security enabled, Codex's trusted audit collector retains adapter security decisions outside the agent's write permissions. Critical findings halt at the iteration boundary, before validation, retries and post-script execution. Collected output, transcripts and usage are retained. This boundary requires the runtime's trusted-audit capability; agent-writable workspace logs are not a substitute. Other runtimes retain their existing hook and output-scan behavior. For Codex, host output findings also gate the post-script. The default is security.escalation.on_critical: halt; review warns without automatically labeling the forge item. Scanner errors follow security.fail_mode, while missing, incomplete or invalid trusted audit evidence is fatal in either mode. This does not promise immediate cancellation during an active model turn. See harness security settings.

Finalized Codex audit snapshots are stored in iteration-N/security/findings.jsonl; host output scan findings remain in security/findings.jsonl. A verified collector result with no findings writes an explicit empty log. A missing sandbox file is never treated as a clean scan. Other runtimes' workspace audit logs remain available as diagnostics under security/sandbox/findings.jsonl; they cannot overwrite host findings or establish a trusted audit.

Output artifacts

Each run produces artifacts in the output directory. Relative --output-dir values are resolved to an absolute path before the run directory is created, so FULLSEND_VALIDATED_ITERATION_DIR (injected into the post-script) does not depend on the process cwd.

File	Description
metrics.json	Behavioral metrics: tokens, cost, model, runtime, iterations
transcripts/	Agent conversation transcripts
claude-debug.log, pi-debug.log or codex-debug.log	Debug log (when --debug is set)

metrics.json fields

Field	Description
runtime	Runtime that executed the run (e.g. claude, pi, codex)
model	Model the provider reported using
requested_runtime	Runtime selected for the run (config file, or a --runtime/FULLSEND_RUNTIME override)
requested_model	Model the harness/agent requested
override_source	Where requested_model came from (--model flag, FULLSEND_MODEL, FULLSEND_PI_MODEL, FULLSEND_CODEX_MODEL, <config path> agents.<name>, harness, default), suffixed , remapped by <config path> models.aliases when a per-repo alias override applied to it
runtime_source	Where requested_runtime came from (--runtime flag, FULLSEND_RUNTIME, the config file path — suffixed agents.<name> when the agent's entry decided — or default (config not found))
total_cost_usd	Total inference cost in USD, as reported by the runtime (raw floating-point aggregate across all iterations; no fullsend-side pricing-table fallback). See Cost data contract
cost_unavailable	When true, the runtime did not report complete dollar cost. Codex sets this; zero in total_cost_usd does not mean free inference. Omitted for runtimes with known cost
num_turns	Number of conversation turns
iterations	Number of agent iterations run; an iteration killed at the budget is not retried (see Budget and deadline)
per_model_usage	Per-model breakdown when supplied by the runtime (pi with the Agent tool enabled, and Codex). See below

Per-model usage

A map from model spec (anthropic-vertex/claude-opus-4-6 on pi, gpt-5.6-luna on Codex) to {requests, input_tokens, output_tokens, reasoning_tokens, cache_creation_input_tokens, cache_read_input_tokens, cost_usd, cost_unavailable}. Optional zero/false fields may be omitted. It attributes usage from both the parent and children; child usage does not appear in the parent's ordinary stream counters.

• What folds. Tokens and cost, from both the parent and every child, summed across retry iterations. Pi counts one requests for the parent plus one per sub-agent call. Codex counts unique native thread/model invocations with recorded usage. Neither counts HTTP requests.
• The invariant. The breakdown sums to the run's input, output, cache-creation and cache-read token totals. Codex also attributes reasoning tokens per model; pi child records do not yet provide that split. Cost sums are meaningful only when cost is available. An incomplete Codex collection preserves known usage and fails the run; it must not be read as complete accounting.
• What stays parent-only. num_turns and tool_calls are read from the parent's stream and are not broken down or added to per model — a child's turns and tool calls are recorded in its own session transcript (pi's <agent>-sub<seq>-*.jsonl, Codex's native thread-ID filename) instead.
• A model spec of unknown is a usage record that carries no model spec at all; its cost is bucketed there rather than dropped. A dispatch rejected before a model was resolved writes no record, so it never reaches the breakdown.

Budget and deadline

Each agent iteration gets the harness's timeout_minutes (30 when it sets none). When the budget is spent the runner ends the iteration and sweeps the processes the agent left running in the sandbox, best effort. Before every iteration the runner rewrites .fullsend/iteration.env (sourced last by the sandbox .env) with the budget, the kill time, and this iteration's W3C trace context, on every runtime (claude, pi, codex):

Variable	Value
FULLSEND_TIMEOUT_MINUTES	The budget: the harness's timeout_minutes, or 30 when it sets none
FULLSEND_ITERATION_DEADLINE	Unix time (seconds) at which the running iteration is killed
TRACEPARENT	W3C trace context of this iteration's agent span (not the run-root span that pre/post scripts receive). Runtimes use it to join Fullsend traces. An inbound unsampled parent (-00) is preserved so runtime export stays suppressed. Empty when telemetry produced no valid span context. Reserved: an env.sandbox entry with this name is dropped.

Example. A probe that shows both, and what a kill looks like. The agent prints the variables with its own clock, then sleeps past the budget:

# Budget probe (timeout)

Do exactly these two steps with the Bash tool, nothing else. Do not write any output file.

1. Run: `env | grep -E '^FULLSEND_(TIMEOUT_MINUTES|ITERATION_DEADLINE)=' | sort; date +%s`
2. Run: `sleep 600`

Say nothing before step 1. After step 2 finishes, reply with the single word DONE.

Its harness is a copy of the review harness with these fields changed. Keep role as it is: the mint issues the agent's token by role, which must be one it serves, and never reads slug (an install-time hint for fullsend github setup). The probe ran locally without a mint.

agent: agents/probe-timeout.md
role: review            # unchanged
slug: probe-timeout
validation_loop:
  script: scripts/validate-output-schema.sh
  schema: schemas/review-result.schema.json
  max_iterations: 2
timeout_minutes: 1

$ fullsend run probe-timeout --fullsend-dir "$PROBE" --target-repo "$TARGET" \
    --env-file "$PROBE_ENV" --forge github --no-post-script

What you see. The agent's first step printed:

FULLSEND_ITERATION_DEADLINE=1788632648
FULLSEND_TIMEOUT_MINUTES=1
1788632601

The last line is the agent's date +%s: the deadline is 47 seconds ahead of it. The runner's heartbeat counts down to the same instant. At the budget the runner ends the exec (it records exit code -1), sweeps the processes the agent left running in the sandbox (best effort), extracts what the agent wrote, and the loop stops after the first iteration:

  ⏳ Agent running (30s elapsed, 30s remaining)
  ⏳ Agent running (1m0s elapsed, 0s remaining)
  ! Agent exited with code -1
    Agent exceeded its budget; terminating its processes in the sandbox
  Terminated 4 stray process(es) left running by the timed-out iteration (200ms)
  ...
  ✗ Validation failed: FAIL: output/agent-result.json not found
  ! Agent timed out (used 1m0s of 1m0s budget) — not retrying
  ...
    Agent runs: 1
    Validation: failed
  ...
Error: agent timed out after 1m0s without completing (timeout: 1m0s)

For comparison:

• Before this rule, the same harness ran Iteration 2 of 2, was killed again, and ended with Agent runs: 2 and Error: validation failed after 2 iteration(s).
• On --runtime pi the run ends the same way, including the sweep. The pi agent handed the probe's commands to a sub-agent, and that child printed both variables with a deadline 45 seconds ahead of its clock: children inherit the values.
• On --runtime codex (--model openai/gpt-5.6-luna) the same: both variables printed with the deadline 50 seconds ahead, Terminated 5 stray process(es) left running by the timed-out iteration, one iteration, the same error.
• A validation failure still retries. A control agent that exits at once without a usable result ran both iterations under timeout_minutes: 5; the deadline it printed moved from 1788630695 in the first iteration to 1788630719 in the second, five minutes after each start.
• On the issue or pull request, the completion status comment shows the run's error text as its failure detail, so the timeout is visible without a post-script. The post-script itself is skipped on a failed run, as for any other failure.

Rules.

• A retry (validation_loop.max_iterations) is for output the agent finished but validation rejected. An iteration the runner killed is not retried, because the next one would replay the same run with the same budget. The run ends with agent timed out after <elapsed> without completing (timeout: <budget>).
• "Killed" means: the iteration failed — non-zero exit, or exit 0 with an error reported in its transcript — after 90 % or more of the budget. This is the same test the no-loop path has used since #5075. An agent that fails deliberately in the last tenth of its budget is reported as timed out too, and a late API error does not get a retry.
• A valid result wins. The killed iteration's output is still validated; if the agent wrote a result that passes before the kill, the run succeeds.
• An agent that exits early with invalid output is a validation failure and keeps its retry. When every iteration fails that way the run ends with validation failed after N iteration(s).
• The kill lands at the deadline or a few seconds after it. Treat the deadline as the hard stop and write your result before it.
• The budget ends the agent's exec, not its processes: OpenShell has no per-exec kill by design (an exec's processes are not expected to exit with the caller, NVIDIA/OpenShell#3159), so the runner sweeps the processes the agent left running before it extracts the output, under --keep-sandbox too. The sweep is best effort, like the one between iterations: a sweep that fails prints Warning: could not terminate stray sandbox processes and the run continues.
• Child processes inherit both variables. In a pi sub-agent read the deadline, not the budget: the deadline is the same wall-clock limit the parent has, while FULLSEND_TIMEOUT_MINUTES is the parent iteration's whole budget, not the time the child has left.
• Both names are reserved: an env.sandbox entry with either name is dropped. Pre-, post- and validation scripts run on the host and do not see them.

Pacing from a skill. Read the deadline, not the budget, and keep the arithmetic in two steps — the sandbox's command scanner blocks a $( ) substitution nested inside $(( )):

now=$(date +%s)
remaining=$(( FULLSEND_ITERATION_DEADLINE - now ))
echo "remaining=$remaining"

Run under a three-minute budget by a probe agent, about twenty seconds after its start:

remaining=161

Skills in the agents repository that mirror the budget through a harness TIMEOUT_SECONDS variable and their own start time can read FULLSEND_ITERATION_DEADLINE instead; the mirror becomes redundant once a runner with these variables is deployed.

If it goes wrong.

You see	Meaning	Do
agent timed out after 20m0s without completing (timeout: 20m0s)	The agent was killed at the budget and no iteration produced a valid result.	Raise timeout_minutes, or make the agent write its result before FULLSEND_ITERATION_DEADLINE.
validation failed after 2 iteration(s) with Agent exited with code 0 above it	The agent finished in time; its output failed validation on every iteration.	Read the Validation failed: lines and fix the output, not the budget.
Could not export the iteration deadline: exit 1: ...	The runner could not write the variables into the sandbox. The iteration runs without them; the runner still kills it at the budget.	Usually a non-writable sandbox workspace; a sandbox that died mid-run looks the same. Check the sandbox logs and the image.
clearing stale iteration deadline (iteration N): ...	The write failed and the previous iteration's file could not be removed either. The run stops rather than let the agent read a stale deadline.	Same as above.
FULLSEND_ITERATION_DEADLINE unset inside the agent	The agent's shell was started without sourcing /sandbox/workspace/.env.	Runtimes that fullsend ships always source it; a custom command must do the same.

OpenAI credentials on pi and codex

A fullsend-openai provider (providers: [openai] on the harness, openai/<id> models on pi or codex) gets its credential from the runner, never from the harness or the sandbox:

Runner environment	Effect
FULLSEND_OPENAI_AUDIENCE, FULLSEND_OPENAI_IDENTITY_PROVIDER_ID, FULLSEND_OPENAI_SERVICE_ACCOUNT_ID	Workload Identity Federation (GitHub Actions only): the run exchanges the job's OIDC token for a short-lived OpenAI token, refreshes it before expiry, and refuses a token whose mapping grants more than model access. All three must be set together; when unset, the inference.openai block of config.yaml (written by fullsend github setup --openai-*) supplies them — except on a machine without a GitHub OIDC endpoint where OPENAI_API_KEY is set, which then wins.
OPENAI_API_KEY	Static key for local runs, or CI when supplied by the FULLSEND_OPENAI_API_KEY repository secret (used only when the three above are unset). In harness YAML, env.sandbox and provider definitions ${OPENAI_API_KEY} expands to the empty string (like the other runner-only variables), and it is never passed to pre/post scripts; the sandbox sees only the gateway placeholder. In CI the runner warns and WIF remains preferred.

In CI the run prepares .fullsend/providers/ from the upstream defaults, so a file there with a scaffold-shipped name (openai.yaml, github-ro.yaml, vertex-ai.yaml, …) is replaced by the upstream copy; give repository-specific providers their own file name. A harness that declares the bare name openai with no providers/openai.yaml on disk gets the definition built into fullsend; other bare names still need a file.

Both paths create a provider named after the run and remove it when the run ends. Setup and troubleshooting: OpenAI Workload Identity.

GitLab role identity

On --forge gitlab (or when GITLAB_CI=true), fullsend run does not mint a GitHub App token. It selects a registered GitLab role credential and exports GITLAB_TOKEN from that CI/CD variable:

• Gate leftover unset/disabled or explicit rollback: shared FULLSEND_FORGE_TOKEN (leftover shared-token runtime, or emergency recovery — ordinary unflagged repos install converges existing shared-token installs to enforced once roles are ready). If FULLSEND_FORGE_TOKEN is absent, a directly-set GITLAB_TOKEN is still used as a fallback (a warning is logged). --gitlab-role-migration no longer accepts disabled.
• migrating/enforced: Poller/Analyst/Coder (or a registered custom role) via gitlabroles.SelectAgent. A missing role secret fails closed; there is no shared-token fallback. Unregistered custom agents fail closed. Analyst jobs do not receive PUSH_TOKEN. A Coder identity cannot approve a merge request. migrating is an internal install intermediate, not an operator-settable flag.

See GitLab Role-Credential Contract.

GitHub Packages credentials

On GitHub Actions, fullsend run copies the pre-mint GH_TOKEN (the job's workflow token) to GH_WORKFLOW_TOKEN before replacing GH_TOKEN with the minted App token. Provider credentials may expand ${GH_WORKFLOW_TOKEN}; every other harness ${} site refuses it, and pre/post/validation child environments never receive it. Outside Actions the variable is not derived from a local PAT — a caller may set it explicitly. GH_TOKEN / PUSH_TOKEN still receive the minted App identity.

Repo-level setup (provider, profile, ~/.npmrc, code/fix overlays): Private registries and GitHub Packages.

Related

• Running Agents Locally for a step-by-step walkthrough
• Runtimes for runtime selection and capabilities
• CLI internals for the full command tree