Skip to content

feat(learning): settlement, reflect Role, lessons Source and per-seat scorecard (#143) - #151

Merged
sudoTomas merged 2 commits into
mainfrom
feat/learning-loop
Nov 4, 2026
Merged

sudoTomas merged 2 commits into
mainfrom
feat/learning-loop

Conversation

@sudoTomas

Copy link
Copy Markdown
Owner

What changed

  • Settlement (trader.learning.settlement, migration 0018_learning.sql): every journaled decision with a decision payload (open, close, hold, shadow included) settles 5 of its symbol's sessions after the session it was made in. Outcomes come from daily bars fetched on use; nothing is stored except the outcome: forward return, benchmark return (SPY for equities/ETFs, BTC/USD for crypto), excess return, direction (+1/-1/0), and correct (NULL for a hold). settled_at is the end of the exit bar. decision_settlements and lessons are append-only (trigger).
  • reflect Role: in ROLE_KINDS (chat), registry ROLES, Spend-cap ROLES, the Probe contracts and the desk Models page. Its .env chain follows the research chain: TRADER_RESEARCH_* if set, else a non-reasoning Grok (TRADER_REFLECT_XAI_MODEL, default grok-4.20-0309-non-reasoning, sharing the process's one xAI client) with the decide client as backstop. Metered under reflect, so a Spend cap applies. It writes a 2-4 sentence Lesson per settled decision, newest first, at most TRADER_REFLECT_MAX_PER_PASS per pass. A capped or down Role ends the pass, which resumes on the next one.
  • lessons Source (loop/sources.py): a Bot's latest 3 Lessons for the symbol (whole entries, 1200-char budget). It only reads Lessons whose available_at (the Settlement's settled_at) is at or before the read's instant. Live reads use now and replay reads use the step's as_of. Source version = the Lesson ids read. Off per Bot: Bot Flip briefs into the decide lane (was EAI-192) #1 turns it on with TRADER_LESSONS_ENABLED, other Bots by adding lessons to their spec sources. Blind replay drops it, because Lessons are free text that can name the asset.
  • Scorecard (trader.learning.scorecard): for each seat (decide = the Draft from raw_output before any critic, each analyst:<lane>, critic as a call on the Draft's direction: approve +1, shrink +0.5, veto -1), it reports hit rate and slugging on excess return, and pooled rank IC of signed conviction vs forward return, at the 5-day horizon.
  • Surfaces: service queries scorecard and lessons and command settle_decisions; API GET /scorecard, GET /lessons, POST /settlement/run; MCP get_scorecard, get_lessons, settle_decisions; CLI trader scorecard, trader lessons, trader settle; the desk shows a Scorecard card on the Decisions page. Replay gains lessons (service/API/CLI --lessons) and config.bot_id.
  • CONTEXT.md: Settlement, Lesson, Scorecard; reflect added to Role. Settings documented in docs/configuration.md / .env.example.

Acceptance

  • Settlement tests on fixture bars: test_a_decision_settles_five_sessions_after_the_session_it_was_made_in, the pending cases, crypto vs BTC, and the pass end to end (test_a_pass_settles_every_journaled_decision_and_writes_lessons).
  • Replay test that a Lesson settled after the bar is never read: test_a_replay_never_reads_a_lesson_settled_after_the_bar.
  • Scorecard numbers checked against a hand-worked example (worked in the test docstring): test_the_scorecard_matches_a_hand_worked_example.
  • CONTEXT.md terms: Settlement, Lesson, Scorecard.

How to switch it on

  1. TRADER_SETTLEMENT_ENABLED=true starts a background pass every TRADER_SETTLEMENT_INTERVAL_SECONDS (default 3600). It needs market-data credentials. trader settle runs one pass on demand either way.
  2. Optional: set a reflect Spend cap or route reflect in a Profile.
  3. Feed Lessons into a prompt (this changes decide behaviour, so it goes through the decide-change gate): replay with --lessons first, then TRADER_LESSONS_ENABLED=true for Bot Flip briefs into the decide lane (was EAI-192) #1, or add lessons to another Bot's sources.

Everything defaults off. Deploying this changes no live behaviour.

Notes

Closes #143

https://claude.ai/code/session_01UDNLNu854ULHbHSL59UJKu

… scorecard (#143)

Every journaled decision settles 5 of its symbol's sessions on, from daily bars fetched
on use (forward and excess return vs SPY or BTC/USD, direction right or wrong). A new
`reflect` Role (Profile-routable, .env on a non-reasoning Grok over the research chain)
writes a short Lesson per settled decision, metered and capped like every Role. The
`lessons` Source feeds a Bot's latest Lessons on the symbol into its decide prompt, off
until the Bot subscribes (TRADER_LESSONS_ENABLED for Bot #1), and reads only Lessons
whose Settlement had settled by the read's instant, so replay stays point-in-time. The
Scorecard grades the decide Draft, each analyst lane and the critic on hit rate,
slugging and rank IC, on the API, MCP, CLI and the desk.

Migration 0018: decision_settlements and lessons, both append-only.

Claude-Session: https://claude.ai/code/session_01UDNLNu854ULHbHSL59UJKu
The lessons Source now stamps each slice with an availability time (the newest served
Lesson's settlement) and has its look-ahead case in tests/test_point_in_time.py. The
reflect Role's default xAI model is grok-4-1-fast-non-reasoning, now priced.

Claude-Session: https://claude.ai/code/session_01UDNLNu854ULHbHSL59UJKu
@sudoTomas
sudoTomas merged commit e2cea69 into main Nov 4, 2026
1 check passed
@sudoTomas
sudoTomas deleted the feat/learning-loop branch November 4, 2026 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Settlement, reflect Role, lessons Source and per-seat scorecard

1 participant