Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ dist/
*.log
coverage/

# Internal/private review + design notes — kept in private KB, never in the public repo
# Local review notes. Do not commit.
CODEX_REVIEW.md
docs/design/
85 changes: 50 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,14 @@ Token budget governor. The substrate every other system runs on.
- `TokenTracker` persists actual spend to disk
- `CostCheckpoint` is the runtime API: every LLM call passes through it, and it can refuse the call when budget is exhausted
- `KillSwitch` enforces hard daily caps
- `UsageReporter` formats spend rollups for inbox / Slack delivery
- `UsageReporter` formats spend rollups as Markdown
- `decimal.js` for every cost calculation — no float drift on currency

### System B — Thought Multiplier

Strategic thinking layer. Use this when one model giving one answer isn't good enough.

- A **router** decides which specialized thinkers a request needs (architecture, planning, test strategy, security, API design, spec writing)
- A **router** decides which specialized thinkers a request needs (planning, test strategy, security, API design, spec writing)
- Each **thinker** runs independently with its own system prompt and JSON output envelope
- A **synthesizer** merges their recommendations, surfaces conflicts, and computes a confidence score
- The **papa agent** (Opus-tier) resolves conflicts and produces a final handoff brief
Expand Down Expand Up @@ -92,10 +92,17 @@ ASIL can run against any HTTP server speaking the OpenAI-compatible `/v1/chat/co

ASIL executes target-repo code (`pnpm install`, build, tests) inside disposable worktrees. To keep that safe:

- **Secrets are stripped from the execution environment.** The runner that executes target-repo commands gets a scrubbed env — `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, and git/GitHub tokens are never in scope, so a malicious `postinstall`/build/test script can't exfiltrate them. The git/`gh` operations that genuinely need a token use a separate, full-env runner (credential separation).
- **Secrets are stripped from the execution environment.** The runner that executes target-repo commands gets a scrubbed env — `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, and git/GitHub tokens are never in scope, so a malicious `postinstall`/build/test script can't exfiltrate them. Git and `gh` use a separate allowlisted environment that keeps only path, locale, git identity, and the GitHub token those commands need.
- **Install lifecycle scripts are disabled by default** (`pnpm install --ignore-scripts`). Opt back in with `ASIL_ALLOW_INSTALL_SCRIPTS=1` only for repos you trust.

This is process-level hardening (Level 1). It removes the credential-exfiltration vector but does **not** sandbox the filesystem or network — a hostile repo's build/test can still read local files or make network calls. For genuinely adversarial input, run ASIL inside a container with `--network=none` (containerized exec is a planned opt-in). See the design notes in the private KB.
This is process-level hardening. It removes the credential-exfiltration vector for install, build, and test, and it adds the following limits:

- Git and `gh` do not inherit model API keys. Commits set `core.hooksPath` to `/dev/null` and pass `--no-verify`, so hooks in the target repository do not run.
- Worktrees live under the system temp directory and include a hash of the repository path, so two repositories cannot delete each other's worktrees. Cleanup refuses to delete anything else.
- Patch file reads stay inside the target repository. Relative paths configured for usage data, the queue, skills, and domain answers cannot climb out of the repository. An absolute path is used as given, unless it contains `..` and resolves outside the repository.
- `auto grind` does not commit, push, or open a pull request unless you pass `--apply`. Without that flag a task that passes the gates stays queued for a later run.

It does **not** sandbox the filesystem or network. A hostile repository's build or test can still read local files or make network calls. For an untrusted repository, run ASIL inside a container that does not share your home directory or network.

### Languages — Python profile

Expand All @@ -120,15 +127,15 @@ pnpm --filter asil-runners run:b "add rate limiting to the connect API endpoint"

# 4. Run the autonomous loop (System A)
pnpm --filter asil-runners auto grind --dry-run
pnpm --filter asil-runners auto grind --max-tasks 3
pnpm --filter asil-runners auto grind --max-tasks 3 --apply
```

See [`examples/quickstart.md`](examples/quickstart.md) for a five-minute integration walkthrough.

## What it looks like

```
$ pnpm --filter asil-runners auto grind --max-tasks 5
$ pnpm --filter asil-runners auto grind --max-tasks 5 --apply

🤖 ASIL — Autonomous Improvement Loop
Repo: /workspace/your-app
Expand Down Expand Up @@ -211,11 +218,11 @@ Every LLM call in B and A flows through a `CostCheckpoint`. Each checkpoint is b
|---|---|---|---|
| `ANTHROPIC_API_KEY` | env | required | Primary LLM |
| `OPENAI_API_KEY` | env | required | Adversarial gate (different provider) |
| `REPO_ROOT` | env | required | Absolute path to the target repo |
| `ASIL_USAGE_DATA_DIR` | env | `<REPO_ROOT>/.asil/usage-data` | Cost tracker state |
| `ASIL_QUEUE_PATH` | env | `<REPO_ROOT>/.asil/usage-data/queue.json` | Task queue |
| `ASIL_SKILLS_PATH` | env | `<REPO_ROOT>/.asil/skills` | Markdown skills the thinkers load |
| `ASIL_DOMAIN_ANSWERS_PATH` | env | `<REPO_ROOT>/.asil/domain-answers.json` | Resolved domain answers |
| `REPO_ROOT` | env | current working directory | Absolute path to the target repo. Set this explicitly. |
| `ASIL_USAGE_DATA_DIR` | env | `<REPO_ROOT>/.asil/usage-data` | Cost tracker state. Relative values must stay inside the repo. An absolute value is used as given unless `..` leaves the repo. |
| `ASIL_QUEUE_PATH` | env | `<REPO_ROOT>/.asil/usage-data/queue.json` | Task queue. Same path rules as `ASIL_USAGE_DATA_DIR`. |
| `ASIL_SKILLS_PATH` | env | `<REPO_ROOT>/.asil/skills` | Markdown skills. Files are `{name}.md` or `{name}/SKILL.md` directly under this directory. Same path rules as `ASIL_USAGE_DATA_DIR`. |
| `ASIL_DOMAIN_ANSWERS_PATH` | env | `<REPO_ROOT>/.asil/domain-answers.json` | Resolved domain answers. Same path rules as `ASIL_USAGE_DATA_DIR`. |
| `papaModel` / `thinkerModel` | `ThoughtMultiplierConfig` | `opus` / `sonnet` | Model tiers per role |
| `executionModel` / `reviewModel` | `ImprovementLoopConfig` | `sonnet` / `sonnet` | Model tiers in System A |
| `maxTasksPerRun` | `ImprovementLoopConfig` | CLI flag | Hard cap per `auto grind` |
Expand All @@ -230,49 +237,57 @@ Every LLM call in B and A flows through a `CostCheckpoint`. Each checkpoint is b

## Integration

ASIL is a TypeScript monorepo. Add the four packages to your workspace and wire them up via the `LoopDeps` and `LLMCaller` interfaces.
ASIL is a TypeScript monorepo. The four library packages are not published to npm yet — vendor this repository and add `asil/packages/*` to your workspace. `asil-runners` is a private reference CLI. Import its wiring from the vendored source (`packages/asil-runners/src/wiring.ts`); it has no package entry point.

`runLoop` takes the config first and the dependencies second:

```ts
import { runLoop } from 'asil-improvement-loop';
import {
createAnthropicCaller,
createCodexCaller,
createGitOps,
createCommandRunner,
createCostInfra,
createDiffApplier,
createFileFetcher,
} from 'asil-runners';

const llm = createAnthropicCaller(process.env.ANTHROPIC_API_KEY!);
const codex = createCodexCaller(process.env.OPENAI_API_KEY!);
const git = createGitOps(process.env.REPO_ROOT!);
const cost = createCostInfra(process.env.REPO_ROOT!);
const diff = createDiffApplier();
const files = createFileFetcher();

await runLoop({
llm,
codex,
git,
diff,
fileFetcher: files,
costInfra: cost,
config: {
createFileReader,
createGitOps,
scrubbedEnv,
} from './wiring.js';

const repoRoot = process.env.REPO_ROOT!;
const cost = createCostInfra(repoRoot);

await runLoop(
{
executionModel: 'sonnet',
reviewModel: 'sonnet',
maxTasksPerRun: 5,
maxAttempts: 2,
taskCooldownMs: 5000,
markdownSkillsPath: '.asil/skills',
repoRoot: process.env.REPO_ROOT!,
queuePath: '.asil/usage-data/queue.json',
markdownSkillsPath: `${repoRoot}/.asil/skills`,
repoRoot,
queuePath: `${repoRoot}/.asil/usage-data/queue.json`,
skipCategories: [],
// Set true only when this process should push and open a pull request.
openPullRequests: false,
codexConfig: { apiKey: 'OPENAI_API_KEY', model: 'gpt-4o' },
},
});
{
llm: createAnthropicCaller(process.env.ANTHROPIC_API_KEY!),
codex: createCodexCaller(process.env.OPENAI_API_KEY!),
git: createGitOps(repoRoot),
tracker: cost.tracker,
budgetManager: cost.budgetManager,
runner: createCommandRunner({ env: scrubbedEnv() }),
fileReader: createFileReader(),
fileFetcher: createFileFetcher(),
diff: createDiffApplier(),
},
);
```

`LLMCaller` is a tiny mock-friendly interface (`call(systemPrompt, userPrompt, model) → Promise<{content, inputTokens, outputTokens}>`). The shipped wirings cover Anthropic and OpenAI; swapping in a local model, a self-hosted relay, or a different provider is a one-file change.
`LLMCaller` is a tiny mock-friendly interface (`call(systemPrompt, userPrompt, model) → Promise<{content, inputTokens, outputTokens}>`). The shipped wirings cover Anthropic, OpenAI, and any OpenAI-compatible HTTP server.

## Key innovations

Expand Down
34 changes: 19 additions & 15 deletions examples/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,13 @@ packages:
- "asil/packages/*" # add this line
```

`pnpm install` will link them in. Or, once published, install the library packages from npm:
`pnpm install` will link them in. The library packages are not on npm yet, so install them from this git repository rather than with `pnpm add`.

```bash
pnpm add asil-cost-controller asil-thought-multiplier asil-improvement-loop asil-analyzer
```

(`asil-runners` is the reference CLI/app, not a published library — vendor it or copy its wiring. The four libraries above are what you compose into your own runner.)
(`asil-runners` is the reference CLI, not a library — vendor it or copy its wiring. The four libraries above are what you compose into your own runner.)

## 2. Set up environment

Create `.env` in your repo root:
Create `.env` next to `pnpm-workspace.yaml` in the ASIL checkout. The CLI loads that file and no other:

```
ANTHROPIC_API_KEY=sk-ant-…
Expand All @@ -39,16 +35,24 @@ REPO_ROOT=/absolute/path/to/your/repo

## 3. Drop in the runner skills

ASIL's thinkers expect Markdown skill files at `<REPO_ROOT>/.asil/skills/`. The simplest setup:
Thinkers and the executor load Markdown from `ASIL_SKILLS_PATH` (default `<REPO_ROOT>/.asil/skills`). Both of these layouts work, with the flat file tried first:

```bash
# Flat file
.asil/skills/planning-and-task-breakdown.md

# Directory layout used by many public skill libraries
.asil/skills/planning-and-task-breakdown/SKILL.md
```

Historical names such as `security-review` also resolve to `security-and-hardening`. A missing file falls back to the inline instructions.

```bash
mkdir -p .asil/skills
# Copy or symlink any Markdown skills (e.g. from a public skill library) you want to use.
# A minimal install is fine — the thinkers fall back to inline defaults
# if a skill file is missing.
# Copy or symlink the skill files you want. A minimal install is fine.
```

Override the location with `ASIL_SKILLS_PATH` if you want them somewhere else.
Override the location with `ASIL_SKILLS_PATH`. A relative value must stay inside the repository. An absolute value is used as given, unless it contains `..` and resolves outside the repository.

## 4. Run System B (Thought Multiplier) standalone

Expand All @@ -66,19 +70,19 @@ You'll get a Markdown brief with merged recommendations, conflict resolutions, a
pnpm --filter asil-runners auto grind --dry-run
```

This scans, queues tasks, and reports — without touching git or making any LLM calls past the scan. Use it to verify the scanner picks up what you expect.
This scans and reports without executing tasks, calling a model, or changing git. The scanner does run the target repository's test, typecheck, and search commands, so it is not a no-op. Use it to verify the scanner picks up what you expect.

## 6. Run a real grind (small budget)

```bash
pnpm --filter asil-runners auto grind --max-tasks 3 --skip dependency-update,documentation
pnpm --filter asil-runners auto grind --max-tasks 3 --skip dependency-update,documentation --apply
```

The runner will:

1. Scan and queue tasks
2. Pause on any unresolved `// DOMAIN_QUESTION:` markers and offer triage
3. For each task: isolate, execute, typecheck, test, self-review × 3, adversarial gate, domain guard, open PR
3. For each task: isolate, execute, typecheck, test, self-review × 3, adversarial gate, domain guard. `--apply` then commits, pushes, and opens a pull request. Without `--apply` the task stays queued.
4. Print a summary with cost spent and PRs opened

## 7. Inspect the spend
Expand Down
2 changes: 1 addition & 1 deletion packages/asil-analyzer/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"files": ["dist", "README.md"],
"repository": {
"type": "git",
"url": "git+https://github.com/telivity-otaip/asil.git",
"url": "git+https://github.com/TelivityAI/asil.git",
"directory": "packages/asil-analyzer"
},
"publishConfig": { "access": "public" },
Expand Down
4 changes: 2 additions & 2 deletions packages/asil-cost-controller/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# asil-cost-controller

Token-budget governor for autonomous LLM agents — per-task budgets, checkpoints, kill switch, daily caps. Part of [ASIL](https://github.com/telivity-otaip/asil).
Token-budget governor for autonomous LLM agents — per-task budgets, checkpoints, kill switch, daily caps. Part of [ASIL](https://github.com/TelivityAI/asil).

See the [ASIL README](https://github.com/telivity-otaip/asil#readme) for the full architecture.
See the [ASIL README](https://github.com/TelivityAI/asil#readme) for the full architecture.
2 changes: 1 addition & 1 deletion packages/asil-cost-controller/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"files": ["dist", "README.md"],
"repository": {
"type": "git",
"url": "git+https://github.com/telivity-otaip/asil.git",
"url": "git+https://github.com/TelivityAI/asil.git",
"directory": "packages/asil-cost-controller"
},
"publishConfig": { "access": "public" },
Expand Down
4 changes: 2 additions & 2 deletions packages/asil-improvement-loop/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# asil-improvement-loop

The autonomous code-improvement pipeline — scan, isolate, execute, 3-persona self-review, adversarial gate, domain guard, PR. Part of [ASIL](https://github.com/telivity-otaip/asil).
The autonomous code-improvement pipeline — scan, isolate, execute, 3-persona self-review, adversarial gate, domain guard, PR. Part of [ASIL](https://github.com/TelivityAI/asil).

See the [ASIL README](https://github.com/telivity-otaip/asil#readme) for the full architecture.
See the [ASIL README](https://github.com/TelivityAI/asil#readme) for the full architecture.
2 changes: 1 addition & 1 deletion packages/asil-improvement-loop/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"files": ["dist", "README.md"],
"repository": {
"type": "git",
"url": "git+https://github.com/telivity-otaip/asil.git",
"url": "git+https://github.com/TelivityAI/asil.git",
"directory": "packages/asil-improvement-loop"
},
"publishConfig": { "access": "public" },
Expand Down
53 changes: 53 additions & 0 deletions packages/asil-improvement-loop/src/__tests__/loop.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,59 @@ describe('runLoop — integration', () => {
expect(result.outcomes[0]?.totalTokenUsage.outputTokens).toBeGreaterThan(100);
});

it('does not push or open a pull request when openPullRequests is false', async () => {
const queue = new TaskQueue(queuePath);
queue.enqueue(mkTask({ id: 't-local' }));

const git = mockGit('https://example.com/pr/9');
const result = await runLoop(cfg({ openPullRequests: false }), {
llm: goodLLM(),
codex: mockCodex(JSON.stringify({ approved: true, severity: 'pass' })),
git,
tracker,
budgetManager,
runner: goodRunner(),
fileReader: mockFileReader(),
fileFetcher: mockFileFetcher(),
diff: mockDiffApplier(),
readCurrent: fakeReadCurrent,
queue,
});

expect(result.prsOpened).toBe(0);
expect(result.outcomes[0]?.status).toBe('publish-skipped');
expect(git.operations.some((op) => op.startsWith('commit:'))).toBe(false);
expect(git.operations.some((op) => op.startsWith('push:'))).toBe(false);
expect(git.operations.some((op) => op.startsWith('createPR:'))).toBe(false);
const again = queue.dequeue();
expect(again?.task.id).toBe('t-local');
expect(again?.attempts).toBe(1);
});

it('pushes and opens a pull request when openPullRequests is true', async () => {
const queue = new TaskQueue(queuePath);
queue.enqueue(mkTask({ id: 't-publish' }));

const git = mockGit('https://example.com/pr/10');
const result = await runLoop(cfg({ openPullRequests: true }), {
llm: goodLLM(),
codex: mockCodex(JSON.stringify({ approved: true, severity: 'pass' })),
git,
tracker,
budgetManager,
runner: goodRunner(),
fileReader: mockFileReader(),
fileFetcher: mockFileFetcher(),
diff: mockDiffApplier(),
readCurrent: fakeReadCurrent,
queue,
});

expect(result.outcomes[0]?.status).toBe('pr-opened');
expect(git.operations.some((op) => op.startsWith('push:'))).toBe(true);
expect(git.operations.some((op) => op.startsWith('createPR:'))).toBe(true);
});

it('installs target-repo deps with --ignore-scripts by default (Codex #1 — no lifecycle-script RCE)', async () => {
const queue = new TaskQueue(queuePath);
queue.enqueue(mkTask({ id: 't-ignore-scripts' }));
Expand Down
24 changes: 24 additions & 0 deletions packages/asil-improvement-loop/src/__tests__/paths.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import { describe, expect, it } from 'vitest';
import { resolveConfiguredPath } from '../paths.js';

describe('resolveConfiguredPath', () => {
const root = '/repo';

it('resolves a relative path under the repository', () => {
expect(resolveConfiguredPath(root, '.asil/skills')).toBe('/repo/.asil/skills');
});

it('rejects a relative path that leaves the repository', () => {
expect(() => resolveConfiguredPath(root, '../other')).toThrow(/outside the repository/);
});

it('keeps an absolute path the operator named explicitly', () => {
expect(resolveConfiguredPath(root, '/var/asil-data')).toBe('/var/asil-data');
});

it('rejects an absolute path whose .. segment leaves the repository', () => {
expect(() => resolveConfiguredPath(root, '/repo/../other')).toThrow(
/outside the repository/,
);
});
});
Loading
Loading