Skip to main content

​
Development

This is the operating contract for changing Ara. It combines implementation, testing, environments, delivery, observability, and code style in one file. The method is a thin operating contract with strong instructions. Put product behavior in the domain that owns it. Root scripts may coordinate CI, releases, migrations, canaries, rollback, observability, and cross-boundary verification, but they must call the owning product modules instead of duplicating product behavior. Put repeatable judgment, acceptance contracts, and failure handling here. Do not build a second framework to explain the first one.

​
Index: ownership and source map

Use this map to find the owner before adding a new root file, framework, or cross-boundary shortcut. The root contracts divide decisions deliberately:
  • GOAL.md owns product direction and load-bearing product scope.
  • AGENTS.md is the short repository entrypoint and invariant list.
  • DESIGN.md owns every rendered and user-facing design decision.
  • This file owns implementation, testing, environments, delivery, and operations.
The product has three route families. /app is the first-party web API, /v3 is the versioned public REST API represented by openapi/, and /mcp/ara is a compatibility transport for short-lived run principals. The public API-key and OAuth MCP adapter is retired. The web app calls /app; external clients use /v3; do not use /v3 as internal web plumbing. Deployment follows ownership: frontend/ ships to Vercel, backend-watch inputs ship the API to Railway, migrations ship through the database release path, and client/package release workflows own their artifacts. Verify the surface that the diff can actually deploy.

​
The four-stage path

Every change moves through the same path. A stage can be shortened when it does not apply, but it cannot be silently skipped. State which evidence was produced and which evidence was not available. HARD RULE: Localization is marketing-only. Product and authenticated app surfaces use English copy and en-US formatting. Keep locale selection, translated catalogs, and automatic translation inside marketing shells. Do not add product messages to the localization catalogs or restore a product language setting.

​
Stage 1: Implement and fast-test

Start with a narrow contract:
  1. Write the expected user-visible or system-visible outcome.
  2. Identify the owning source, boundary, data model, and telemetry event.
  3. Any UI change or UI implementation must follow DESIGN.md.
  4. Define the smallest deterministic check that can fail before the change and pass after it.
  5. Make the smallest coherent implementation.
During the edit loop:
  • Read current source before changing it. Code and observed behavior outrank old prose.
  • Keep the pinned @earendil-works/pi-agent-core, pi-ai, and pi-coding-agent versions aligned across the root and workspace packages. A Pi upgrade is one runtime change, not three independently versioned changes.
  • Keep product behavior beside its owning domain. Root scripts/ coordinate operations and verification across domains.
  • Treat executable contracts as the source of truth for mechanical interfaces, not agent reasoning. Command, route, capability, artifact-input, and release catalogs live with one owner. Smoke tests, help output, workflow filters, and release automation derive from that contract or equality-test their mirror; they do not maintain another literal list.
  • Define agent behavior through clear intent, capable tools, and observable outcomes. Constrain effects at boundaries and let the model choose the path; prescribe steps or policy structures only when the protocol itself or a safety invariant is the product contract. Prefer plain-language prompts and system instructions over schemas that encode judgment, prescribed tool plans, or precomputed context; add structure only when ordinary code must validate, route, persist, or enforce a boundary.
  • Parse untrusted data at API, network, CLI, and persistence boundaries. Internal logic receives validated types.
  • Model discrete states with discriminated unions and make invalid states hard to construct.
  • Add structured Axiom telemetry for every state mutation. Add capture() for a user-meaningful product action. These are separate evidence channels.
  • Ship backward-compatible migrations with the code that uses them.
  • Remove dead code, obsolete workarounds, suppressed warnings, and narrative comments in the touched path.
Run the smallest useful checks first:
Pure helpers and behavior changes need colocated *.test.ts files. Keep unit tests near the behavior. Use root tests/ only for integration, end-to-end, browser, edge, and cross-package contracts that have no single source owner. Tests observe state and emitted effects, not scheduler luck. Use fake clocks or wait for the owned state transition; a polling assertion must not mutate the state it observes. Every concurrent runner declares a worker budget that leaves room for sibling lanes. Do not default to every available CPU when typechecks, builds, or other test pools run beside it.

​
Testing and debugging ladder

Use the lowest level that reproduces the failure, then climb only when the boundary under test requires it:
  1. Focused bun test <file> for the owning invariant.
  2. bun run test:int, or bun run test:integration only with CI’s disposable Postgres recipe, for HTTP and persistence boundaries.
  3. bun run test:e2e for the composed API and web path.
  4. Faux-provider and package mechanics tests, then an explicitly authorized judge-only Live Pi eval for model behavior.
  5. Preview and staging acceptance using source, deployment, telemetry, and behavior proof.
  6. Risk-based human QA for meaning, usability, and high-risk journeys.
A regression test must fail for the reconstructed invariant before it passes for the fix. Do not edit a red test merely to match the broken behavior. For an eval assertion, keep a positive case and its closest negative twin so success cannot come from vacuous output or an over-broad matcher.

​
Architecture boundaries

Simple internals, strict boundaries, isolated volatility.
  • Give each behavior and business invariant one domain owner.
  • Package source does not import application source. Cross-boundary behavior uses an owned contract installed at a composition root.
  • Do not introduce a runtime import cycle or grow an existing one.
  • Parse external data once. Application logic receives closed domain types.
  • Inject I/O, clocks, randomness, and mutable configuration at real orchestration seams. Do not create one-use interfaces for pure local code.
  • Keep side effects at named boundaries. Pure transforms do not read clocks, environment, network, storage, or mutable globals behind a value-returning API.
  • Do not add another responsibility to a multi-thousand-line owner. Extract only a cohesive domain slice with typed inputs, outputs, and focused tests.
  • Keep harmless duplication until a stable semantic owner emerges. Centralize business knowledge and invariants once they do.
  • Reuse an existing owner and pattern when its semantics match. A genuinely different invariant gets a new explicit owner; superficial consistency is not a reason to couple unrelated behavior.
bun run check:architecture enforces dependency direction, static runtime cycles, scan coverage, and the module-mock ratchet. Existing debt lives in scripts/architecture-baseline.ts. The baseline must decrease in the same change as the debt; never regenerate or raise it to make a new violation pass. Fan-out reporting is advisory and must not become an arbitrary file-size gate.

​
API, auth, and MCP boundaries

  • /app is the WorkOS/JWT-authenticated first-party BFF. Reject public API keys; new web capabilities belong here.
  • /v3 is the scoped, versioned public REST contract. Change its generated OpenAPI surface with the implementation.
  • /mcp/ara admits only short-lived run principals. Resolve the principal and current organization membership before discovery or dispatch. Run tokens do not receive a long-lived public bearer or the generic /v3 bridge.
  • Resolve organization scope once with requireOrgFromPath for org routes. Every mutating Session route must apply writeGate and return cloud_agents_disabled with 503 when closed. Add new route files to scripts/check-write-gate-coverage.ts; bun run check:write-gate fails a vacuous scan.
The advertised Ara tool catalog is re-exported from backend/src/ara-mcp/catalog.ts; the shared text is in cli/agent-skill.ts and two published Aradotso/ara-mcp plugin skills. Update all copies together and run bun run check:agent-skills. An unadvertised escape hatch needs a documented allowlist entry; a dispatchable legacy alias must be named as deprecated.

​
Observability and request debugging

Backend state mutations use logInfo, logWarn, or logError with a dotted lowercase event name and a flat field object. Do not use console.*, and do not pass caller fields named event, level, or service. Log each external I/O boundary with outcome and duration, never payload secrets. The request middleware owns X-Request-Id, request_id, trace context, and the single api.request event. Add route context through addRequestContext so a request can be traced without emitting a second request event. Sanitize browser telemetry before ingestion. Production writes the engineer Axiom dataset; staging writes engineer-v2-staging. Keep arbitrary caller properties inside the bounded props map so new fields do not exhaust the dataset column budget.

​
Database migrations

Every schema change is a new YYYYMMDDHHMMSS_slug.sql file under db/migrations/ and ships in the same pull request as the code that uses it. The migration ledger rejects a reused timestamp with different content; choose a new timestamp instead of overwriting history. Migrations are append-only after merge: never edit, delete, rename, or reverse one recorded in an environment ledger. Repair it with another forward migration. Use expand/contract so old and new application versions both work whether code or schema arrives first. Avoid destructive or locking DDL in the release path. The lint owns the narrow markers: -- migrate:no-transaction for a single statement such as a concurrent index, and -- migrate:lock table_a, table_b for an intentionally pre-locked transaction. Migrations use the direct DATABASE_DIRECT_URL connection, normally port 5432, never the pooled runtime URL. Before handoff, run:
CI loads the production baseline into disposable Postgres, applies the pull request migrations, and requires an empty pending set. A migration is ready only when lint and baseline apply pass, the mixed-version code path is safe, and the recovery plan is a forward fix. Staging and production apply through their release workflows; do not mutate a remote schema by hand.

​
Session sandbox standby

A visible transcript does not imply physical work. With tool-scoped standby enabled, chat and side-chat views remain Brain-only between tool calls. The runtime kill switch may retain already-acquired Hands through an active run, but it must not let visibility provision Hands. A mounted hidden Files, Browser, Terminal, or Desktop surface must preserve local UI state without issuing a guest-backed request. An active physical surface may hold Hands only for the user’s explicit operation. HARD RULE: Session visibility never creates a keepAlive process, provisions Hands, or wakes Hands. Only an active physical tool or selected machine surface may do that. The standby kill switch controls retention after that acquisition, not acquisition itself.

​
Structured capability replies

A structured repl direct reply is a protocol error, not a successful user answer. Keep raw JSON out of the transcript and provider context, leave the Turn open, and give the model one bounded instruction to retry with a concise prose string. Do not publish a generic read receipt as the answer. Preserve structured output only when the user explicitly requests exact output.

​
Inference retry safety

A provider completion that contains only private reasoning has produced no actionable output or external side effect. Retry it with a fresh physical request identity, but preserve one logical assistant message and record whether the completion was empty or reasoning-only. A physical provider request that emits no event may likewise time out below the run watchdog and retry with a fresh identity. Once any visible text or tool call exists, never replay the attempt. Before leaving Stage 1:
  • The acceptance contract has deterministic proof.
  • The scoped typecheck passes.
  • The diff has no unrelated edits, dead branches, secret values, or unexplained generated output.
  • External payloads, org scope, write gates, and telemetry are covered where the change crosses those boundaries.

​
Stage 2: Prove the behavior locally

Stage 1 proves code. Stage 2 proves the product path with the local API and web app, a real identity when needed, real model behavior when authorized, and correlated telemetry.

​
Start and stop the local stack

The launcher starts at web port 3001 and API port 4001, walks to free ports, and prints the authoritative URLs. Use those reported ports. It wraps both processes with the dev Infisical folder when the CLI is available and otherwise uses the ambient environment or .env; --no-infisical makes that fallback explicit. Verify both reported surfaces before QA. The split commands are bun run dev:api and bun run dev:web. Vite strips the web’s /api prefix and proxies it to the local API, so the normal browser value is VITE_CLOUD_API_URL=/api. To isolate a pull request backend, run only the local web with VITE_CLOUD_API_URL=https://<railway-pr-api>.up.railway.app. API boot requires CLOUD_AGENT_SECRET_ENC_KEY, WORKOS_CLIENT_ID, WORKOS_API_KEY, and SESSION_JWT_SECRET; browser auth also needs VITE_WORKOS_CLIENT_ID. Database paths use pooled DATABASE_URL (normally 6432), while migration DDL uses direct DATABASE_DIRECT_URL (normally 5432).
Stop the foreground stack with Ctrl-C or a background stack with bun run dev:down, then run bun run dev:check. Do not trust teardown text without probing the recorded ports.

​
Local identity and routes

Signed-out localhost visits to /org/acme-ara use the local guest seam. That is enough for Sessions, Session detail, the composer, and Customize. Do not extend that exception to any other Workspace or any hosted environment. Settings routes need a real local session. Start with the explicit dev identity:
Then open:
The /api prefix is required through Vite. If hosted WorkOS login returns an internal error locally, stop old processes, confirm both ports are free, restart with fresh environment values, and retry in a clean browser context before changing auth code. For live Pi, memory, skill, or repository tests, do not use the shared Acme Workspace. Create a disposable PostgreSQL container and a dedicated local identity instead:
prepare accepts only a loopback PostgreSQL URL whose database name starts with ara_qa_. It loads the checked-in baseline and migrations, creates an isolated local record for qa-runner@ara.so and a unique non-Acme Workspace through the product stores, verifies the existing GitHub App installation for qa-runner-web/vibecoding-company with read-only provider calls, and records that binding only in the disposable database. It stores the user id as the memory owner used by both the first-party dev session and evaluator Sessions. It creates the Free plan billing account with the normal first-Workspace grant and arms only the disposable database’s inference control. dev loads model, Device, encryption, WorkOS, and GitHub credentials from the Infisical dev environment. An env command after the Infisical boundary forces both DATABASE_URL and DATABASE_DIRECT_URL to the recorded loopback URL. It also enables the dedicated first-party dev session and adds only that local identity to OPERATOR_EMAILS for this process. The isolated stack uses API port 4301 and web port 3301 so it does not collide with the normal local stack. With the stack running, create the local bootstrap key in another terminal and run a bounded evaluator case:
The bootstrap key never leaves .dev/. The evaluator uses it to mint a scoped public /v3 key and deletes that key in finally. Stop the foreground dev stack, then remove the exact labeled container and local key files:

​
Browser QA

Use the in-app Browser for all application QA. Use Aside when its existing authenticated session, cookies, or device control are required. Do not switch to a lower-fidelity test because the first browser lacks authentication. Stable UI anchors include:
  • Sidebar footer: Settings for <name>.
  • Session composer: Message this session.
  • New-task composer: Start a new task.
  • Model picker: Select model, effort, and speed.
  • Slash menu: Slash search results.
  • Settings navigation: aside[aria-label="Settings sidebar"].
Opening routes and controls is safe. Sending a composer message creates a real run and spends credits. Do not send one during smoke testing unless the requested test requires it. Known local noise is not proof of a new regression:
  • Usage metrics can fail when the local database role lacks a migration-applied function grant.
  • Billing summary can return 403 under the anonymous guest seam.
  • Vite reconnect messages, React DevTools hints, and local analytics debug lines are expected.
Investigate the actual request, response, UI state, and Axiom rows before classifying any of these as product failures.

​
Mobile QA through a temporary tunnel

The localhost guest seam must remain localhost-only. For phone testing, use a temporary Cloudflare tunnel and real WorkOS staging authentication:
Add the exact temporary origin and <origin>/auth/return to the WorkOS staging allow-lists. Confirm <origin>/org/acme-ara returns 200 before sharing it. The tunnel URL is temporary and changes after restart. Test the same path in the in-app Browser and on the phone. A rendered login page is not auth proof. Require an auth.session_resolved Axiom event with provider: workos and ok: true. If that event exists but the UI says no browser session was created, inspect frontend token hydration before changing WorkOS configuration.

​
Ara agent runtime operating model

The product is the Ara agent runtime. Pi is its executor and runtime implementation; use Pi runtime as the short implementation name and Live Pi eval only for evaluation. Do not describe the product by a rollout codename or as an evaluation framework. An ordinary Conversation has the conversation profile and adapts within one thread. Brain and Hands are execution lanes selected for the current work, not permanent modes a user chooses. Bounded subagents have exactly four profile names: code_explorer, context_explorer, default, and fork_self. These names are the persisted policy, prompt-manifest, telemetry, and evaluator identifiers; titles and prose are never profile identity. The complete provider vocabulary is repl, bash, read_file, write_file, edit_file, websearch, webfetch, get_time, write_todos, memory_search, routine_update, subagent, subagent_wait, and ask_user_question. A request receives only the contextual subset permitted by its profile and granted capabilities. code_explorer is limited to repl, read_file, and get_time; context_explorer can additionally receive web read and personal-memory search; default and fork_self omit further delegation and questions. Physical availability and connected-resource readiness can narrow any declared surface further. The declaration recorded for that provider request is authoritative. Where Pi owns a provider-tool contract, use its exact field names; for example, bash accepts command and an optional timeout in seconds. repl is one provider tool and one mental model. It exposes the typed, scope-filtered ara.* namespaces and virtual files needed for Workspace skills, personal memory, and connected resources. Do not advertise raw ara_* tools, a second MCP carrier, or a CLI/HTTP fallback in model-visible instructions. Prompt ownership is also singular. Stable profile text lives in packages/pi-cloud/src/roles/; the profile registry and cache identity live in packages/pi-cloud/src/runtime/agent-profile.ts; dynamic context is composed as ordered stable_base, agent_rules, control_guidance, workspace_skills, personal_memory, connected_mcp, hands_guidance, and session_goal layers. The provider task message carries only the current task and typed repository, branch, Workspace-source, and trigger facts. Workspace skills are Workspace-scoped and revisioned. Personal memory is owner-scoped, opt-in, revisioned, and never inherited through an API-key principal. Subagents receive mechanically bounded scopes and cannot infer broader tenant or Device authority from their prompt. Compaction preserves the logical Conversation across physical Attempts. Recovery reclaims the same durable Brain, Session resource, and eligible Hands workspace instead of creating an untracked parallel execution. Axiom must record prompt/profile identity, the exact provider surface, tool start/completion, compaction and recovery, model/usage/cost, terminal outcome, and pre-dispatch rejections for every live proof.

​
Live Pi eval

Use the live Pi evaluator for real-model prompt, tool, continuity, and workflow checks. Route and package tests remain the contract guard; the live evaluator answers whether the deployed Pi path behaves correctly through the supported public /v3 Sessions API. Every ordinary user Session must report the conversation agent profile. A bounded subagent reports one of four profile names: code_explorer, context_explorer, default, or fork_self. Match the selected profile to the requested work: repository-only investigation uses code_explorer; memory, Workspace skills, web, and classified read-only integrations use context_explorer; a general bounded worker uses default; and a bounded projection of the source prompt and transcript uses fork_self. Do not infer a profile from prose. Verify the recorded name, revision, and digest. The required QA matrix is behavioral, not a set of marker replies:
  • a 20-plus-turn Conversation proves continuity, a settled compaction boundary, later constraint changes, and evidence-preserving synthesis;
  • two independent Sessions prove Workspace skill creation, natural discovery, full read/use, and exact cleanup;
  • personal memory proves teaching, extraction, teacher-Session deletion, fresh Session recall through search/read, and exact memory cleanup;
  • repository work proves inspection, a useful bounded change, tests, commit, push, draft pull request, independent GitHub verification, and branch/PR cleanup;
  • one coordinating Conversation proves all four bounded subagent profiles from exact profile projections, contextual provider surfaces, terminal outcomes, and zero unexplained pre-dispatch rejections.
Each live journey uses the dedicated QA identity, a unique execution id, fixed time and spend brakes, immediate clickable Session URLs, an atomic local receipt, exact disposable-resource cleanup, and post-run Axiom correlation. Preserve the receipts before cleanup. Keep proof layers separate: None of these substitutes for another. State unavailable proof explicitly. Real QA creates persisted Sessions, remote Device work, inference cost, and telemetry. Run it only with explicit authorization. Use qa-runner@ara.so and a dedicated isolated QA Workspace. Never use Acme, a personal Workspace, a customer Workspace, or an implicit default.

​
Hosted QA runner profile

The canonical real-test identity is Blake (qa-runner@ara.so), a target Ara customer persona who works as a founder and AI full-stack engineer. Use GitHub as qa-runner-web, Git as Blake (QA Runner) <qa-runner@ara.so>, and the Vercel team qa-runner with team slug qa-runner-2654. Blake’s workstation root is /Users/adi/qa-runner@ara.so. The local profile source of truth is /Users/adi/.aside/u/1/skills/user/qa-runner-workflow/SKILL.md. Required environment variable names and values may be loaded at execution time from /Users/adi/qa-runner@ara.so/.env. Never print, log, copy into prompts, commit, or document plaintext values from that file. Available working copies include:
  • /Users/adi/qa-runner@ara.so/Code/vibecoding-company, backed by qa-runner-web/vibecoding-company and https://vibecoding-company.vercel.app.
  • /Users/adi/qa-runner@ara.so/Code/personal-site, backed by qa-runner-web/personal-site and https://personal-site-seven-inky.vercel.app.
  • ara-web-frontend, ara-inference-api, and ara-queue-worker under the same Code directory.
A Supabase project is available for realistic database work, and Gemini inference is configured locally. Access both through the environment only. Do not copy project credentials or the Gemini key into code, prompts, logs, or receipts. The Slack test persona is @blake in the Ara Workspace. Relevant test channels are #vibecoding-lounge, #agentic-builds, #ship-log, and #all-ara. Do not commit raw Workspace, user, or channel IDs. Do not send Slack messages without explicit authorization. Real evaluation uses open-ended requests that fit Blake’s work. Exercise repo discovery, implementation judgment, tests, commits, draft PRs, deployment inspection, Supabase-backed behavior, memory, skills, and multi-turn continuity. Exact-string replies and marker-only cases are preflights, never agent-quality proof. Operate only on the dedicated QA account and its resources. Use unique branches and draft PRs, preserve receipts before cleanup, avoid destructive production changes, and never use Acme, customer, or personal memory. Confirm the supported entrypoint before spending traffic:
Run one bounded live case with either an existing scoped API key or an operator key that can mint and delete a disposable key:
Use repeated --message arguments with a complete --rubric for one multi-turn check, or --journey for a schema-validated JSON manifest. Every live journey is judged. The JSON report contains Session and observed run IDs, reported cost and usage, judge output, evidence, and exact runtime or measurement failures. The evaluator reconstructs the full user and assistant transcript from public messages plus the safe sessions:debug event projection. This matters because assistant prose is stored as Session events rather than message rows. Use the checked-in 24-turn adaptation journey and the first-party compaction checkpoint for an open-ended continuity review against the disposable local QA Workspace:
Its rubric grades repository grounding, adaptation, safety boundaries, and final synthesis across the full transcript. It does not require an exact reply marker. The runner requests compaction after turn 12, records the concrete compaction event or persisted summary, and sends turn 13 only after that proof. It prints the clickable Session URL and every exact Attempt identity as soon as each becomes observable. To verify durable Workspace skills across independent Sessions, run the fixed cross-session journey against the isolated QA Workspace:
Session A saves a useful release-readiness workflow without being told which tool call to make. Session B receives a normal release-readiness request without the skill name or id, discovers relevant Workspace guidance, reads it, and uses its scope, validation, risk, and approval rules. The evaluator checks the public org-scoped skill resource and observable search/read behavior, then deletes only that unique authored skill in finally, including when Session B fails. A disposable evaluator key adds skills:read and skills:write for this journey and is deleted after the skill. Run --workspace-skill-journey --dry-run to inspect the two-Session plan without creating a Session, skill, or key. Use the same disposable QA stack for the other required live proofs:
The memory journey uses the localhost first-party session cookie, deletes the teaching Session before fresh recall, and requires extraction, revision, search, read, semantic, Axiom, and cleanup receipts. The repository journey independently verifies the draft pull request, commit, changed files, checks, and exact branch before cleanup. The profile journey proves exactly code_explorer, context_explorer, default, and fork_self from member projections and correlated Axiom rows rather than titles or prose.
After the terminal report, query each returned run ID in Axiom. Verify the agent_profile name, revision, and digest; tool surface; Device lifecycle; final output; model; cost; warnings; errors; tool failures; latency; and terminal status.

​
Interactive latency critical path

Interactive Sessions optimize user-perceived send-to-first-answer latency and cold start. Do not insert setup, network calls, telemetry flushes, provisioning, or optional context work between send and the first assistant answer. Defer work until after the first answer when correctness allows it, and do not make cold boot slower to improve a later step. Any change that can affect admission, inference, prompt assembly, Session boot, Hands acquisition, or first output requires comparable before-and-after send_to_first_assistant_ms evidence for warm and cold scenarios plus the affected runtime suite. Keep inputs, environment, model, and sample shape fixed; report first-answer latency separately from settlement and next-turn readiness. Read the cohort with bun run logs:axiom -- --preset pi-latency --since 24h. Only when production traffic is explicitly authorized, run:

​
Live Pi eval and local exit gate

Pi eval outcomes are judge-only. The isolated judge grades correctness, autonomy, continuity, safe behavior, tool choice, and artifact quality from the collected Session, Axiom, pull-request, attachment, and runtime evidence. HARD RULE: Never add a deterministic Pi-eval outcome grader, expected-tool array, regex pass condition, per-case hard assertion, marker, or required-evidence rule outside judge instructions. The runner may emit INVALID only when measurement or evidence infrastructure is untrustworthy, and STOP only when continuing is unsafe. It must never turn those observations into semantic PASS or FAIL. Keep prompts natural and open-ended. Put precise success criteria and evidence expectations in the judge rubric. Exact reply markers and fixed-output prompts are preflights, not agent-quality proof. Operational availability canaries are a separate health instrument. Their mechanical assertions must never be imported into or treated as a Pi-eval verdict. Run the package tests to verify catalog shape, selection, evidence collection, and runner mechanics without creating or grading a Session:
Record case inputs, expected behavior, actual behavior, model identity, timing, cost, and evidence. Keep user prompts natural and concise when the behavior allows it; put the precise success criteria and evidence expectations in the judge instructions. Judge changes also ship a balanced adversarial calibration fixture: genuine success, contradicted or missing evidence, and a semantically equivalent valid path where relevant. Run at least three independent judge samples; any mixed verdict is FLAKY, never an accepted one-off:
The live suites are explicit and require the intended environment credentials:
Before leaving Stage 2:
  • The local stack path works in the in-app Browser.
  • Authenticated paths use the intended identity seam.
  • Axiom contains the expected events and no unexplained errors.
  • Prompt or runtime changes have judge-backed Pi evidence and, when authorized, isolated real-model proof.
  • The result is labeled precisely using the evidence names below.

​
Stage 3: CI, preview, and staging

Stage 3 proves the integrated change outside the local checkout.

​
Pull request and CI

Start ordinary work from current origin/staging-dev on a short-lived codex/<task> branch. Open a focused pull request with gh pr create --base staging-dev; production main is not the ordinary feature target. Put worktrees under .worktrees/. For a stack, target the first pull request at staging-dev and each dependent pull request at its predecessor. The auto-merge label arms GitHub native squash auto-merge after protected checks; removing it cancels the request. Never merge a staging pull request by hand. Do not arm auto-merge when review was requested first, the change is a draft, or CI cannot prove a risky migration, secret, auth/org-scope, billing, or rollout change. Changes to AGENTS or DEVELOPMENT also wait for final-revision human approval. A production hotfix requires current explicit human direction, a current-main hotfix/* or codex/hotfix-* branch, the hotfix-main label, and the complete Production hotfix authorization PR-body contract enforced by scripts/staging-release-policy.ts. Otherwise production receives only the protected staging-dev promotion. Run the complete local gate before handoff:
CI is authoritative for the repository-wide matrix. Do not suppress, retry into green, or delete a failing test to obtain a merge. Fix the cause or document a verified infrastructure failure with its evidence.

​
CI topology and performance budget

HARD RULE: Every deployable artifact whose declared inputs changed must clean-build in a required pre-merge check. Docker inputs, release classifiers, and workflow path filters derive from one owned manifest or have an executable equality test. A post-merge build is verification, not the first place missing or stale build inputs may be discovered. Validate is the single required application CI check. Its one 16-vCPU job runs the full pipeline with bounded ownership: tests use 8 workers, static checks use 4, and build work uses the remaining CPU. Keep a robust p50 below 60 seconds without weakening coverage, hiding failures, or oversubscribing the runner. The separate required database check always reports and performs its baseline apply only for migration-related changes. Add a test to the existing owner lane; do not add a new shard for one suite. Wall-clock is the slowest lane, billing is the sum of every lane, and each shard repeats fixed setup. Before adding a large or slow test, measure the owning lane’s before-and-after wall time and keep it inside the 60-second gate target. Split fixtures or move expensive non-gating breadth to a scheduled suite only when the required gate retains the smallest deterministic contract that would catch the regression. scripts/check-ci-budget.ts monitors the p50 target on a schedule; a vacuous or missing sample is a failure, never a pass. For a pull request environment:
  • Verify the preview deployment identity belongs to the pull request.
  • Treat the Railway service as HTTP-only: it does not run autonomous workers, agent execution, inference, the scheduler, or MCP dispatch.
  • Use the in-app Browser against the preview for user-visible changes.
  • Point a local frontend at the pull request API only when isolating backend behavior.
  • Correlate the preview action with the preview or staging Axiom dataset.

​
Staging deployment proof

After the change reaches staging-dev, establish four separate facts:
  1. Source proof: the change is in origin/staging-dev ancestry.
  2. Deployment proof: the correct Vercel or Railway service reports the deployed identity that contains the change.
  3. Telemetry proof: staging Axiom rows show the expected path and no unexplained failures.
  4. Behavior proof: the real staging UI or API produces the expected result.
Frontend-only changes require Vercel and rendered staging proof. Backend-only changes require Railway, API, and Axiom proof. A change touching both needs both deployment identities. Staging is https://staging.ara.so plus https://api-staging.ara.so, the synthetic ara-staging Workspace, and Axiom dataset engineer-v2-staging. railway.json and scripts/runtime-release-policy.ts own the backend image inputs; a frontend-only diff does not redeploy Railway. Use /deployment-check to compare the web and API identities. The normal production path is the gated 9 AM Pacific daily promotion. Sandbox-boundary releases also require an immutable provider image whose name encodes the complete source revision. Point staging at that exact candidate, restart its API, and verify the provider image identity plus engine readiness before accepting a security canary whose egress boundary reports PASS with confined=yes; the confined attack replay must target the same image. Promote only through the staging-dev to main release train; wait for the corresponding production image import before changing its Infisical pointer, then restart the API so its boot-cached image selection takes effect. If engine readiness or the runtime canary fails after Bash confinement is enabled, turn global confinement off, clear the affected canary-org IDs, restart the API, and prove health before continuing. Run staging Pi evals for any change that can affect model behavior, prompt composition, tool admission, continuation, delegation, memory, or Device use. Use the same case manifest and acceptance thresholds used locally. A local Pi result is not staging proof. If the current eval runner has no live staging target, report that evidence as unavailable instead of relabeling local output.

​
Human QA on staging

Human QA is risk-based, not ceremonial. Require it for:
  • New or materially changed user journeys.
  • Auth, org scope, permissions, secrets, billing, or destructive actions.
  • Mobile, browser, Device, or interaction behavior that automation cannot judge.
  • Prompt behavior where correctness depends on meaning, usefulness, or a multi-turn interaction.
The human tester records the exact environment, account or QA profile, steps, expected outcome, actual outcome, and screenshots or run IDs. Human review does not replace automated proof. Before leaving Stage 3:
  • Required CI is green.
  • The preview and staging deployment identities are known.
  • Staging Axiom, Pi evals, API checks, browser checks, and human QA are complete in proportion to risk.
  • No staging-only exception is being mistaken for production readiness.

​
Stage 4: Production and post-release proof

Production receives the protected staging-dev promotion. A direct hotfix to main requires explicit current authorization. Merge status alone is not production proof. The trusted main release verifier authenticates deployment identity through /v3/deployment with ARA_AUTOMATION_API_KEY; it must not depend on a retired operator-only endpoint. When a promotion changes a fundamental document, trusted automation requires a human approval for that exact revision, verified against repository collaborator permissions before native auto-merge is armed. Match verification to the changed surface:
  • Frontend: confirm a Ready Vercel production deployment newer than the merge, then verify the rendered path on ara.so in the in-app Browser. Vercel queues nearby production deploys; do not require the first build to have the exact merge SHA.
  • Backend: confirm the Railway deployment identity for api.ara.so, then verify the API response and corresponding Axiom path.
  • Database: confirm the migration ledger and the new code path. Never infer migration success from application deployment alone.
  • Agent runtime: use an isolated production QA Workspace only after explicit authorization, then verify model, profile, tools, Device lifecycle, terminal state, latency, and cost.
Use Axiom for operational truth:
Check PostHog for curated product truth. Confirm the intended event name, distinct identity, organization, properties, and stable $insert_id when a webhook or job can redeliver. Axiom rows do not prove the PostHog funnel, and a PostHog event does not prove the full runtime trace. For onboarding or billing changes, verify the complete state transition:
  • User-visible entitlement or checkout result.
  • Durable account, credit, or ledger state.
  • Axiom mutation events.
  • PostHog business event.
  • Idempotency under retry.
Watch the immediate release path and a short post-release window for errors, latency changes, duplicate events, stuck Sessions, billing anomalies, and user feedback.

​
Rollback operations

Prefer a forward fix. When containment may require rollback, begin with the read-only recommendation:
It combines structural compatibility, current Vercel and Railway identities, retained known-good candidates, and production canary evidence. Add --post-slack only when the bounded report should be posted to #alerts. The command never changes traffic, Git, configuration, secrets, or data. Act only when the report proves the exact compatible candidate and a human approves the action. Restore provider traffic before source recovery when that is the proven containment path. The workflow may create a draft source-revert pull request, but never merges or deploys it. Never reset main, manually force a deployment identity, or reverse/delete an applied migration. If compatibility is uncertain, contain the affected feature and roll forward. Stage 4 is complete only when the deployed identity, live behavior, Axiom trace, and relevant PostHog or data effects agree.

​
Evidence names

Use these labels exactly: Do not compress several labels into “tested” or “live”.

​
Testing instruments

Use the instrument that owns the question. Start here is the lookup route, not an exhaustive manual: follow the command help, closest focused test, linked self-documenting owner, or named skill before improvising a new path. A source file is linked only when it carries the relevant command, schema, or contract. Do not advertise an active instrument without a traceable entrypoint. Hypercheck is omitted until the repository owns an executable contract for it. Generic desktop computer use is exceptional rather than a normal Ara verification layer; use Aside for authenticated browser state and document any native-only proof explicitly. Do not add a platform-wide admin identity, impersonation header, hidden cross-workspace fallback, or direct-database mutation CLI. Direct database mutations are not a testing shortcut.

​
CLI installation telemetry and legacy Device migration

The CLI has two temporary observability cohorts that must stay distinct:
  • Current installer/updater observations emit cli.installation_observed and cli.update_checked. A random installation_id supports unique-install and retention counts. Version, platform, architecture, background mode, update outcome, legacy-state presence, and coarse ingress country support rollout diagnosis.
  • Pre-update physical installations continue emitting local_runtime.device_heartbeat under their existing authenticated device_id. Physical rows carry migration_state=legacy_device_drained and cannot claim work.
Define an active current installation as a distinct installation_id with a successful ingest in the trailing 24 hours. Define an active legacy installation as a distinct drained physical device_id heartbeating in the same window. Never sum the two cohorts without deduplicating the migration overlap. Version adoption uses installed_version for the updater cohort and ara_runtime_version for the legacy cohort; country is an ingress-derived, two-letter code and can be absent. Because the current installer endpoint is intentionally credential-free, these counts are directional operational telemetry, not a billing or security source of record. Do not join the random installation identifier to an Ara account. The no-reinstall migration and reactivation sequence is:
  1. Keep the shared PHYSICAL_DEVICE_EXECUTION_ENABLED release switch false. The API drains physical heartbeats and rejects enrollment, targeting, enqueue, and claim paths.
  2. Use the already-running signed Device updater to deliver the dormant controller. It keeps the service, Device identity, heartbeats, updater, and execution plumbing, but independently refuses to claim work while the same switch is false.
  3. Measure controller-version adoption before reactivation. An offline legacy computer receives the dormant controller when its service or CLI next runs; no reinstall is required.
  4. Before changing the switch, run bun run rehearse:physical-device-rollout. It exercises the real consent/grant decision, local controller eligibility, API admission, MCP target gate, public CLI route, and web target selection with switch=true while asserting that the committed switch remains false.
  5. Deploy the API that understands execution_eligible before publishing the controller release. Missing eligibility fails closed, and ineligible controllers heartbeat with no targetable capabilities.
  6. Change the one shared release switch in a normal release. The updater may activate only future_device installations whose versioned consent matches a separately recorded private ~/.ara/workspace grant. updates_only, cli_only, missing-grant, and legacy full-home installations stay drained until a fresh explicit folder action makes them eligible.
  7. For artifacts whose ownership cannot be proven from the signed runtime, known service identity, or installer receipt, preserve both execution locks and ask the user to run a bounded repair command. Reinstallation is not required.
A computer on which neither the background service nor the CLI ever runs again is unreachable by definition. It remains denied server-side, but no updater can modify or report dormant local software without a future process launch. HARD RULE: Never make physical Device rollout an environment-variable matrix. The shared source switch is the release boundary; consent eligibility and the local claim lock remain mandatory even when it is enabled. HARD RULE: CLI telemetry has a strict field allowlist. It never accepts local paths, filenames, hostnames, environment names, file contents, or secret values.

​
Environments and secrets

  • Infisical project 6d518288-7854-49d2-aa42-8ffd285dafa1, path /github-native-engineer, is the source of truth. Local uses environment dev; hosted targets select their named environment.
  • Pull request environments are disposable integration targets.
  • staging-dev owns staging.
  • main owns production.
  • Existing process or platform variables override fetched Infisical values. Treat an unexpected Railway value as shadowing, not as proof Infisical failed. Keep Railway to the Infisical pointer and sanctioned platform variables; do not duplicate ordinary secret plaintext there.
  • Use target-locked wrappers: bun run admin:infisical for production and bun run admin:staging for staging. Change a secret in the intended Infisical environment, redeploy the consumers, and verify names, source, and behavior without printing values. Remove the previous value only after overlap is safe.
  • Retired direct-provider and inference-broker credentials are forbidden; the boot loader strips or rejects them and the boundary checks must stay green.
  • Release and auto-merge workflows mint repository-scoped installation tokens for ara-swe-staging[bot] from the existing dev Infisical App credentials. Request workflows: write only for workflow-bearing promotion and sync paths.
  • AUTO_MERGE_TOKEN remains stored and is the availability fallback. Do not remove or rotate it as part of App-token adoption.
  • Any pull request workflow that reads App credentials runs as pull_request_target, checks out trusted main, and never checks out or runs pull request code.
  • Never commit plaintext secrets, log secret values, print bearer tokens, or store QA credentials in these files.
  • Secret plaintext crosses the product boundary only through the shared secret contract and write-only injection paths.

​
Public docs and OpenAPI

Every product-affecting change updates a published Mintlify source in the same pull request: docs/public/*.mdx, docs.json, GOAL, DEVELOPMENT, or generated openapi/openapi.json. Internal engineering prose does not satisfy this gate. Run bun run check:mintlify and bun run check:docs-coverage. For a public /v3 change, update the route-owned schema, run bun run openapi:gen, commit the generated contract, then run bun run openapi:gen:check and bun run check:openapi-parity. Do not hand-edit generated operations.

​
Self-update protocol

This file is writable operating memory. Its changelog is only a bounded record; the update protocol is what makes it self-correcting. The no-regression law applies to every update: preserve every still-valid hard rule, acceptance threshold, and verified proof path. If a timing, cost, or error baseline moves by more than 20 percent across comparable runs, record the old and new measurements before changing the expectation. Never invent a baseline from one noisy run. Enter update mode when a documented command fails because its interface changed, an API or deployment contract changed, a repeated failure exposes a missing hard rule, or measured behavior invalidates a stated baseline. HARD RULE: When verified code, command help, schema, deployment metadata, telemetry, or rendered behavior contradicts this document, treat the current executable evidence as truth and update this document in the same change. Do not leave a known false instruction for a later documentation cleanup.
  1. Observe the ground truth. Inspect source, --help, the response, deployment metadata, and telemetry. Do not patch from a guess.
  2. Isolate the smallest stale instruction.
  3. Make a surgical replacement. Do not rewrite unrelated working sections.
  4. Promote a recurring failure into a HARD RULE near the affected workflow.
  5. Delete the superseded command, exception, or workaround. Never keep contradictory instructions.
  6. Re-run the affected command or proof path immediately.
  7. Bump the version: patch for a corrected fact, minor for a workflow or interface change, major for a changed engineering model.
  8. Add one changelog line with the verified result.
HARD RULE: Never update this file from an unverified chat claim. HARD RULE: Never append a workaround without removing the obsolete instruction that it replaces. HARD RULE: Never lower an established test or evidence bar to make a change pass. Keep the latest ten changelog entries. Git retains older history. Update AGENTS.md, DESIGN.md, or GOAL.md in the same change only when their own contracts also changed.

​
Anti-slop and deslop

Run this pass on the changed diff before committing. Keep behavior unchanged unless the task explicitly removes behavior or fixes a proven bug. Existing slop is not precedent. New and touched code must follow these rules. These rules combine the TypeScript evidence rules from dmmulroy/anti-slop with the deletion and simplification pass from pro-workflow/deslop. They are review rules until a current lint command enforces them. Never claim CI enforcement without naming the command and check.

​
TypeScript evidence rules

  • no-chained-type-assertions: Never fabricate a type through chained casts. Replace as unknown as T with parsing, narrowing, or a typed owner contract.
  • no-conditional-empty-object-spread: Do not use {} as the false branch of a conditional spread. Construct the optional field explicitly.
  • no-known-value-widening: Do not erase known keys or literals with a broad annotation. Preserve inference or use satisfies.
  • no-module-mocking: Do not mock modules. Pass a dependency through a real interface, service, or faithful implementation and test through that seam.
  • no-object-parameters: Do not accept the broad object type. Accept the exact contract the function uses.
  • no-reflect-apply: Call typed functions directly. Do not use Reflect.apply to bypass a call signature.
  • no-reflect-get: Use typed property access or parse a dynamic object at its boundary. Do not use Reflect.get to hide an unknown shape.
  • no-runtime-typeof: Parse at the boundary instead of scattering ad hoc typeof checks through business logic. A named type guard or assertion function may use them.
  • no-shape-in-symbol-names: Do not name types or values *Shape. Name the domain contract they represent.
  • no-unknown-parameters: Do not push unknown into application logic. Parse it at the boundary. The caught-error cause convention is the narrow exception.
  • no-unknown-returns: A function must return a usable contract, not unknown or Promise<unknown>.
  • no-unknown-type-aliases: Do not conceal unknown behind an alias.
  • no-unsafe-dictionary-type: Do not use dictionaries whose values are any, unknown, object, {}, or an alias for one of them. Parse entries into a closed value type.
  • no-widen-then-assert: Do not widen a known value and cast it back later.
  • require-safety-comment-for-type-assertion: Prefer inference, as const, satisfies, parsers, and guards. A necessary non-const assertion needs an immediately preceding SAFETY: comment naming the check that proves it.
Use discriminated unions for discrete states. Use constructive types when they make invalid states impossible, such as non-empty tuples and explicit duration fields. Ban @ts-ignore, @ts-expect-error, as any, and blanket linter disables.

​
Delete the whole obsolete path

Deletion is the default for dead internal behavior. Do not hide it, comment it out, rename it with an underscore, preserve a pass-through wrapper, add a deprecated re-export, or leave a fallback that can never run. When a feature, path, or workaround is removed:
  1. Prove the producer and every consumer with repository search and, when relevant, runtime evidence.
  2. Delete the implementation, imports, exports, types, routes, flags, branches, compatibility aliases, tests, CSS, translations, fixtures, generated output, assets, comments, package commands, workflows, and documentation that exist only for it.
  3. Search again for its names, paths, event names, storage keys, selectors, and old command strings. The expected result is no unexplained matches.
  4. Run the lowest test that proves the surviving behavior, then its owning typecheck or build.
Do not keep an internal compatibility shim without a current consumer. Public /v3 contracts, stored data, applied migrations, customer integrations, and published SDK behavior are not internal debris. Change those through an explicit versioned or expand-contract migration. Delete tests that cover deleted behavior, implementation strings, duplicated cases, or module-mock scaffolding after equal or stronger behavioral evidence exists. Do not delete a failing test to make CI green. Do not delete tests to shrink a deployment artifact; exclude them from the deployment context. For UI deletion, remove the full slice. Static unused-selector scans only produce candidates because class names can be constructed dynamically. Verify the owning routes in both themes and representative desktop and mobile widths. Keep global frontend/src/styles.css rules only when they are genuinely shared; colocate the rest with the route or component.

​
Simplify the changed diff

Remove:
  • Comments and docstrings that state the obvious, narrate history, apologize, or describe code outside the changed behavior.
  • Defensive try/catch, repeated null checks, fallback branches, and error states for cases the trusted internal contract makes impossible.
  • One-use factories, helpers, registries, configuration layers, and interfaces that make a direct operation harder to read.
  • Deep nesting that early returns or a smaller direct function can remove.
  • Internal backwards-compatibility hacks, renamed _vars, forwarding exports, tombstone comments, and dead feature flags.
  • Features, refactors, telemetry, types, or “improvements” beyond the requested outcome.
  • Large captures and generated media that object storage can own.
Three clear repeated lines are better than a premature abstraction. Do not split a file merely to lower its line count, but do not add another responsibility to a multi-thousand-line owner. Extract only a real domain boundary with typed inputs and outputs.

​
Writing pass

Use active voice, straight quotes, concrete nouns, and measured claims. Cut a sentence if it contains no fact, instruction, decision, or evidence. Do not use em dashes, bold inline-header lists, chatbot filler, sycophancy, puffery, or vague metaphors. Prefer “is” and “has” to “serves as”, “stands as”, “boasts”, and “features”. Delete “Certainly”, “Of course”, “Great question”, “delve”, “enhance”, “evolving landscape”, “groundbreaking”, “testament to”, “tapestry”, “interplay”, “intricate”, “beacon”, “bedrock”, “scaffolding”, “flywheel”, and “north star” unless the literal word is required.

​
Finish condition

Review git diff origin/staging-dev...HEAD and the working-tree diff. Remove scope creep and dead lines. Re-run the smallest relevant tests and typecheck. The final summary names what was deleted and stays concise.

​
Human approval

HARD RULE: A change to AGENTS.md or DEVELOPMENT.md requires explicit human approval on the final pull request revision. Surface the exact diff and do not enable auto-merge before approval. Required CI enforces this rule.

​
Changelog

  • v2.15.1 (2026-08-24): Added a bounded fresh-identity retry for physical provider requests that emit no event before the run watchdog.
  • v2.15.0 (2026-08-24): Made Pi eval outcomes judge-only and limited runner verdicts to measurement INVALID or safety STOP.
  • v2.14.0 (2026-08-24): Integrated the canonical Ara agent runtime vocabulary, adaptive Conversation model, prompt ownership, and full Live Pi eval proof matrix with current staging delivery and Device contracts.
  • v2.13.0 (2026-08-23): Required sampled adversarial judge calibration and canonical case-owned suite, risk, cost, scheduling, tag, and path metadata.
  • v2.12.0 (2026-08-23): Reconciled the testing, API/MCP, observability, local development, delivery, migration, secrets, and public-doc contracts with their current executable owners.
  • v2.11.0 (2026-08-23): Required the CI gate to verify final-revision human approval for AGENTS or DEVELOPMENT edits.
  • v2.10.1 (2026-08-23): Allowed one bounded reasoning-only provider retry while prohibiting replay after visible text or tool output and requiring the retry class in telemetry.
  • v2.10.0 (2026-08-23): Defined the Ara agent runtime operating model, exact profiles and contextual tool vocabulary, canonical prompt ownership, and the complete Live Pi eval proof matrix.
  • v2.9.1 (2026-08-23): Required the three pinned Pi runtime packages to move in lockstep across the root and workspace manifests.
  • v2.9.0 (2026-08-23): Made executable contracts authoritative, required scheduler-independent tests and clean affected-artifact builds, and added actionable ownership and boundary guidance to architecture failures.