Repository navigation
feat(learning): settlement, reflect Role, lessons Source and per-seat scorecard (#143) - #151
Merged
Merged
Conversation
… scorecard (#143) Every journaled decision settles 5 of its symbol's sessions on, from daily bars fetched on use (forward and excess return vs SPY or BTC/USD, direction right or wrong). A new `reflect` Role (Profile-routable, .env on a non-reasoning Grok over the research chain) writes a short Lesson per settled decision, metered and capped like every Role. The `lessons` Source feeds a Bot's latest Lessons on the symbol into its decide prompt, off until the Bot subscribes (TRADER_LESSONS_ENABLED for Bot #1), and reads only Lessons whose Settlement had settled by the read's instant, so replay stays point-in-time. The Scorecard grades the decide Draft, each analyst lane and the critic on hit rate, slugging and rank IC, on the API, MCP, CLI and the desk. Migration 0018: decision_settlements and lessons, both append-only. Claude-Session: https://claude.ai/code/session_01UDNLNu854ULHbHSL59UJKu
The lessons Source now stamps each slice with an availability time (the newest served Lesson's settlement) and has its look-ahead case in tests/test_point_in_time.py. The reflect Role's default xAI model is grok-4-1-fast-non-reasoning, now priced. Claude-Session: https://claude.ai/code/session_01UDNLNu854ULHbHSL59UJKu
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
trader.learning.settlement, migration0018_learning.sql): every journaled decision with a decision payload (open, close, hold, shadow included) settles 5 of its symbol's sessions after the session it was made in. Outcomes come from daily bars fetched on use; nothing is stored except the outcome: forward return, benchmark return (SPY for equities/ETFs, BTC/USD for crypto), excess return, direction (+1/-1/0), andcorrect(NULL for a hold).settled_atis the end of the exit bar.decision_settlementsandlessonsare append-only (trigger).ROLE_KINDS(chat), registryROLES, Spend-capROLES, the Probe contracts and the desk Models page. Its.envchain follows the research chain:TRADER_RESEARCH_*if set, else a non-reasoning Grok (TRADER_REFLECT_XAI_MODEL, defaultgrok-4.20-0309-non-reasoning, sharing the process's one xAI client) with the decide client as backstop. Metered underreflect, so a Spend cap applies. It writes a 2-4 sentence Lesson per settled decision, newest first, at mostTRADER_REFLECT_MAX_PER_PASSper pass. A capped or down Role ends the pass, which resumes on the next one.loop/sources.py): a Bot's latest 3 Lessons for the symbol (whole entries, 1200-char budget). It only reads Lessons whoseavailable_at(the Settlement'ssettled_at) is at or before the read's instant. Live reads use now and replay reads use the step'sas_of. Source version = the Lesson ids read. Off per Bot: Bot Flip briefs into the decide lane (was EAI-192) #1 turns it on withTRADER_LESSONS_ENABLED, other Bots by addinglessonsto their specsources. Blind replay drops it, because Lessons are free text that can name the asset.trader.learning.scorecard): for each seat (decide= the Draft fromraw_outputbefore any critic, eachanalyst:<lane>,criticas a call on the Draft's direction: approve +1, shrink +0.5, veto -1), it reports hit rate and slugging on excess return, and pooled rank IC of signed conviction vs forward return, at the 5-day horizon.scorecardandlessonsand commandsettle_decisions; APIGET /scorecard,GET /lessons,POST /settlement/run; MCPget_scorecard,get_lessons,settle_decisions; CLItrader scorecard,trader lessons,trader settle; the desk shows a Scorecard card on the Decisions page. Replay gainslessons(service/API/CLI--lessons) andconfig.bot_id.docs/configuration.md/.env.example.Acceptance
test_a_decision_settles_five_sessions_after_the_session_it_was_made_in, the pending cases, crypto vs BTC, and the pass end to end (test_a_pass_settles_every_journaled_decision_and_writes_lessons).test_a_replay_never_reads_a_lesson_settled_after_the_bar.test_the_scorecard_matches_a_hand_worked_example.How to switch it on
TRADER_SETTLEMENT_ENABLED=truestarts a background pass everyTRADER_SETTLEMENT_INTERVAL_SECONDS(default 3600). It needs market-data credentials.trader settleruns one pass on demand either way.reflectSpend cap or routereflectin a Profile.--lessonsfirst, thenTRADER_LESSONS_ENABLED=truefor Bot Flip briefs into the decide lane (was EAI-192) #1, or addlessonsto another Bot's sources.Everything defaults off. Deploying this changes no live behaviour.
Notes
available_at(repo query plus a defence-in-depth filter in the Source). It is not a general clamp; Point-in-time clamp + SEC EDGAR, Polymarket, StockTwits/Reddit Sources #142's clamp can subsume it.tests/test_learning.py); ruff clean; config reference in sync.Closes #143
https://claude.ai/code/session_01UDNLNu854ULHbHSL59UJKu