Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ next-env.d.ts
*.db
*.db-journal
migrations/
!packages/judging-db/migrations/
!packages/judging-db/migrations/**

# Editor
.vscode/*
Expand Down
444 changes: 444 additions & 0 deletions PLAN.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@

## Two products, one database

The schema is split on purpose. Mixing them previously made membership vanish when a new hackathon edition was drafted.
The schema is split on purpose. Mixing them previously made membership vanish when a new hackathon edition was drafted. Panel judging tables live in this same Postgres database.

### Club

Expand Down
72 changes: 72 additions & 0 deletions docs/judging/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Panel

Hackathon judging inside the portal. Rubrics, tracks, timers, rooms, and
dispatch strategy are rows, not code. It runs in the mainweb process on the
club Postgres, and the portal sign-in decides who is judging or organizing.

## Layout

| Path | Package | What it is |
| --- | --- | --- |
| `packages/judging-core/` | `@query/judging-core` | Pure functions. No I/O, no clock, no runtime dependencies. |
| `packages/judging-db/` | `@query/judging-db` | Schema, committed SQL migrations, demo seed. |
| `packages/judging-server/` | `@query/judging-server` | The judging API (Hono + tRPC) as a library. |
| `packages/judging-cli/` | `@query/judging-cli` | `panel migrate`, `panel seed demo`, `panel doctor`, `panel outbox drain`. |
| `packages/api/src/services/panel.ts` | `@query/api/panel` | One API instance per process, and the portal user as a judging actor. |

## In the portal

Judging is per hackathon edition. On `/admin/judging`, staff pick the edition
and press "Switch to panel judging". That creates the edition's judging event
(organization `hacklytics`, event slug = the hackathon id) in `setup`, with
default timers and the club's five criteria out of ten, then copies in the
submitted projects and the portal's judges. Switching back to classic leaves
the event in place.

| Route | Who | What |
| --- | --- | --- |
| `/api/panel/*` | the pages below | The judging API. Same routes as `docs/judging/api.md`, without the prefix. |
| `/judge/panel/[hackathonId]` | approved judges | Judge desk: next table, QR or table number, scores, pairwise, offline hold. |
| `/admin/judging/panel/[hackathonId]` | staff | Organizer console: phases, floor, tables, rubric, tracks, judges, results. |
| `/judging/[hackathonId]` | anyone | Public board while judging, placements after publish. |
| `/judging/[hackathonId]/feedback/[token]` | a team | That team's feedback card after publish. |

Roles come from the `admins` row: `super_admin` is owner, other staff are
admin, a volunteer row is volunteer, a bug tester has none. A judge is matched
by the email on their portal judge row. Approving a judge in the portal
approves them in judging, and deactivating suspends them. Promoting
submissions copies projects in, and pulling results writes published
placements back into `hackathon_result`.

Live views poll, and only while they can change. The board polls at the
event's `board_poll_seconds` and the console polls the floor every five
seconds, both only during `judging_live` and only while the tab is visible. The judge
desk asks `/v1/session/status` every 15 seconds while a visit is open and the
screen is on, which is how a recall or a voided visit reaches the phone.
Neon suspends after five idle minutes and the free plan has 100 compute hours
a month, so a forgotten board in any other phase must not keep it awake.
`/api/panel/readyz` and `/api/panel/metrics` are not exposed for the same
reason. Judging uses the club's connection pool rather than a second one.

## Checks

`pnpm lint`, `pnpm typecheck`, and `pnpm test` at the root cover these
packages with the rest of the repository.

## Database

Postgres is the one database the rest of the monorepo uses (`DATABASE_URL`).
Panel migrations add its tables there.

```
docker compose up -d
$env:DATABASE_URL = "postgresql://postgres:postgres@localhost:5433/neondb"
pnpm --filter @query/judging-cli panel migrate
pnpm --filter @query/judging-cli panel seed demo
pnpm --filter @query/judging-cli panel doctor
```

Webhooks queue in `outbox`. Publishing results drains it; delivered rows are
deleted, since `event_log` keeps the history. A delivery that failed stays
queued for `panel outbox drain`. A Hacklytics-size event is on the order of
20 MB across the judging tables.
26 changes: 26 additions & 0 deletions docs/judging/adr/001-standalone.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# ADR-001: Standalone product

## Status

Superseded. Panel's packages are ordinary monorepo workspaces
(`packages/judging-*`) under the `@query/*` scope, served by the portal. The import
boundary and the plan to split it out were dropped.

## Decision

Panel lives under `judging/` with its own packages and its own deploy. It does
not share a module graph with the application that hosts the first event.
ADR-003 puts the tables in that application's Postgres instead of a second
database.

Internal imports are `@panel/*` only. A lint rule rejects imports from the
host. The host is allowed to import `@panel/*`. That one-way edge is what
makes `git subtree split --prefix=judging` a later extraction rather than a
rewrite.

## Why

Judging welded into one event's tables cannot be given to the next event, or
to anyone else, without carrying the rest of that application with it.
The panel tables are prefixed so they can share that database until an
extraction.
22 changes: 22 additions & 0 deletions docs/judging/adr/002-pure-core.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# ADR-002: Pure core

## Status

Accepted

## Decision

Every decision Panel makes — the next table, whether a visit is still live,
whether a score is valid, how a set of votes becomes a ranking — is a function
in `@query/judging-core`. Those functions take plain data, including `now` and a
`random` source, and return plain data. They do not read the clock, generate
randomness, or touch a database, the network, or the filesystem.

The server is the only place that performs I/O. It loads rows, calls the
core, and writes the result in the same transaction as the event log.

## Why

A ranking you cannot recompute from the votes is not a result you can defend.
Tests of the core need no database, so the scoring bugs surface without
standing up Postgres. The same functions can later run offline on an export.
26 changes: 26 additions & 0 deletions docs/judging/adr/003-one-database.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# ADR-003: One database, one server

## Status

Accepted. Supersedes the separate-database sentence in ADR-001.

## Decision

Panel tables live in the same Postgres as the club app. The connection is
`DATABASE_URL`. `PANEL_DATABASE_URL` is read only when that is unset. The
tables that would collide with the club schema are `panel_user`,
`panel_event`, and `panel_judge`.

There is one API process. Redis is optional. When `REDIS_URL` is unset or
Redis fails, that process keeps delivering live updates in memory.

`judging/` stays in this monorepo. Splitting it out, and deleting the legacy
judge UI, wait until an event has run on panel. Sign-in mail waits on
`PANEL_SMTP_URL`. Neither connection is made yet.

## Why

The host is the existing app. A second database and a second API fleet were
the extraction plan, and they are not how this checkout runs. The replay of
the published score fixture still matches `@query/judging-core` `rank()` at pairwise
weight 0, so the scoring check does not need that event.
17 changes: 17 additions & 0 deletions docs/judging/api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# API

`GET /openapi.json` is the route list (OpenAPI 3.0.3). The same procedures are also mounted at `/trpc`.

Send `Authorization: Bearer` with a JWT from `POST /v1/auth/verify`, or with an API key from `catalog.issueApiKey`. A key whose scopes include `import`, `export`, or `webhooks` acts as an organizer. A revoked key is rejected.

These routes do not require a token:

- `GET /healthz`, `GET /readyz`, `GET /metrics`, `GET /openapi.json`
- `POST /v1/auth/magic-link` and `POST /v1/auth/verify`
- `GET /v1/public/{orgSlug}/{eventSlug}`
- `GET /v1/live/{orgSlug}/{eventSlug}`
- `GET /v1/feedback/{token}`

`GET /v1/results/{eventId}` requires a bearer token and returns 403 until results are published. Feedback returns 403 for an unknown token and before publish.

Live clients connect to `/ws?channel=event:{eventId}:board`. The channels are `event:{id}:board`, `event:{id}:leaderboard`, and `judge:{id}`.
28 changes: 28 additions & 0 deletions docs/judging/contributing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Contributing

Panel is `packages/judging-*` plus its portal routes in `sites/mainweb` (listed in `README.md`). `pnpm lint`, `pnpm typecheck`, and `pnpm test` at the root cover it with the rest of the repository.

## Checks

From the repository root:

```
pnpm --filter @query/judging-core test
pnpm --filter @query/judging-server test
pnpm --filter @query/judging-cli test
pnpm --filter @query/judging-core lint
pnpm --filter @query/judging-server lint
pnpm --filter web lint
pnpm --filter @query/judging-db typecheck
pnpm --filter @query/judging-server typecheck
pnpm --filter web typecheck
pnpm --filter @query/judging-cli typecheck
```

## Migrations

Schema changes are SQL files in `packages/judging-db/migrations`, applied with `panel migrate`. Do not rename or drop a column in the same change that ships code reading the new shape.

## Pull requests

Describe the behaviour a judge or an organizer will see. Include the command you ran.
23 changes: 23 additions & 0 deletions docs/judging/deploy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Deploy

Judging ships with mainweb. There is no separate service or image: the API is
`/api/panel` in the portal, and the pages are portal routes (see `README.md`).

Before the first event on panel, against the production database:

```
DATABASE_URL=<the club database>
pnpm --filter @query/judging-cli panel migrate
pnpm --filter @query/judging-cli panel doctor
```

Then switch the edition to panel on `/admin/judging`. Nothing goes in the
environment: the event is created from the hackathon row.

`packages/db/drizzle.config.ts` hides the judging tables from the deploy's
`drizzle-kit push` and declares the judging enum types
(`packages/judging-db/src/enums.ts`), so push neither tries to drop them nor
stalls on a prompt.

`PANEL_JWT_SECRET` is only needed for callers outside the portal that send a
bearer token; API keys from `catalog.issueApiKey` work without it.
7 changes: 7 additions & 0 deletions docs/judging/rubric.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Rubric

A rubric is a named list of criteria. Each criterion has a key, a label, a minimum, a maximum, and a weight. The default rubric is what judges see. A track can point at a different rubric, which is how a sponsor prize asks different questions.

Scores outside a criterion's range are rejected. The visit total is the weighted mean of the criteria. Comments are stored with the vote and shown on the team card only after results are published.

Create one from the organizer console with the track form for a new prize group, or through `catalog.saveRubric` with the criteria in order. Anchors are the numbers a judge can tap; they are the integers from min to max.
9 changes: 9 additions & 0 deletions docs/judging/scoring.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Scoring

A project's rubric score is the weighted mean of its criterion values. Each judge's scores are then z-scored against that judge's own mean and spread and placed on the field's mean and spread, so a harsh judge and a lenient judge who ranked the same projects agree.

Projects with few votes are shrunk toward the field average. The config value `bayesian_c` is how many average votes that prior is worth. At 2, two real votes and the prior weigh the same.

Pairwise comparisons are a separate component. `pairwise_weight` is the share of the final score that comes from them. At 0 the ranking is the rubric score only, which matches a results run that never asked judges to compare tables.

Publishing freezes one result run. A later run can be diffed against it before it is published.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"lint": "turbo run lint",
"format": "prettier --write .",
"typecheck": "turbo run typecheck",
"test": "vitest run packages/api packages/auth packages/db sites/mainweb/lib"
"test": "vitest run packages/api packages/auth packages/db sites/mainweb/lib packages/judging-core packages/judging-server packages/judging-cli"
},
"dependencies": {
"next": "16.3.7",
Expand Down
6 changes: 5 additions & 1 deletion packages/api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
"./eastern-time": "./src/services/eastern-time.ts",
"./metrics": "./src/services/metrics.ts",
"./portal-context": "./src/types/portal-context.ts",
"./resume-list": "./src/services/resume-list.ts"
"./resume-list": "./src/services/resume-list.ts",
"./panel": "./src/services/panel.ts"
},
"scripts": {
"lint": "eslint . --max-warnings 0",
Expand All @@ -28,6 +29,8 @@
"dependencies": {
"@query/auth": "workspace:*",
"@query/db": "workspace:*",
"@query/judging-db": "workspace:*",
"@query/judging-server": "workspace:*",
"@tanstack/react-query": "5.103.2",
"@trpc/client": "11.19.0",
"@trpc/next": "11.19.0",
Expand All @@ -42,6 +45,7 @@
"zod": "4.6.5"
},
"devDependencies": {
"@query/judging-core": "workspace:*",
"@query/tsconfig": "workspace:*",
"@types/sanitize-html": "^2.16.0",
"typescript": "7.0.2",
Expand Down
10 changes: 9 additions & 1 deletion packages/api/src/routers/judge/admin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import {
import { CacheKeys, invalidatePortalContext } from "../../middleware/cache";
import type { DrizzleDB } from "@query/db";
import { isLive, loadGroups, loadPool } from "./dispatch";
import { syncJudgeToPanel, syncProjectsToPanel } from "../../services/panel-sync";

// Judges draw tables from a shared pool (dispatch.ts) instead of holding a
// list built in advance. Clears rows an earlier version built that the judge
Expand Down Expand Up @@ -263,7 +264,7 @@ export const judgeAdminRouter = createTRPCRouter({
promoteSubmissions: isAdmin
.input(z.object({ hackathonId: z.string().uuid() }))
.mutation(async ({ ctx, input }) => {
return await (ctx.db as DrizzleDB).transaction(async (tx) => {
const promoted = await (ctx.db as DrizzleDB).transaction(async (tx) => {
// Serializes concurrent promotions, so two organisers pressing the button
// together cannot both read the same max table number and duplicate it.
await tx
Expand Down Expand Up @@ -350,6 +351,8 @@ export const judgeAdminRouter = createTRPCRouter({
total: submissions.length,
};
});
await syncProjectsToPanel(ctx.db as DrizzleDB, input.hackathonId);
return promoted;
}),

setActive: isAdmin
Expand Down Expand Up @@ -467,6 +470,11 @@ export const judgeAdminRouter = createTRPCRouter({
}
}

await syncJudgeToPanel(ctx.db as DrizzleDB, {
hackathonId: updated.hackathonId,
email: updated.email,
isActive: input.isActive,
});
return { success: true, isActive: input.isActive, queuedProjects };
}),

Expand Down
26 changes: 26 additions & 0 deletions packages/api/src/routers/judge/portal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import {
lockDispatch,
} from "./dispatch";
import type { DrizzleDB } from "@query/db";
import { panelConsoleUrl, panelDeskUrl, setJudgingBackend } from "../../services/panel-sync";


export const judgePortalRouter = createTRPCRouter({
Expand Down Expand Up @@ -68,6 +69,31 @@ export const judgePortalRouter = createTRPCRouter({
return result;
}),

/** The panel judge desk in the portal, only for an edition that opted in. */
panelDesk: protectedProcedure.query(async ({ ctx }) => {
return panelDeskUrl(ctx.db as DrizzleDB, ctx.userId as string);
}),

/** The panel organizer console in the portal when this edition uses panel. */
panelConsole: isAdmin
.input(z.object({ hackathonId: z.string().uuid() }))
.query(async ({ ctx, input }) => {
return panelConsoleUrl(ctx.db as DrizzleDB, input.hackathonId);
}),

/** Moves an edition between classic judging and panel. Panel gets its event and a first sync. */
setJudgingBackend: isAdmin
.input(
z.object({
hackathonId: z.string().uuid(),
backend: z.enum(["legacy", "panel"]),
}),
)
.mutation(async ({ ctx, input }) => {
await setJudgingBackend(ctx.db as DrizzleDB, input.hackathonId, input.backend);
return panelConsoleUrl(ctx.db as DrizzleDB, input.hackathonId);
}),
Comment thread
cursor[bot] marked this conversation as resolved.

getMyAssignments: protectedProcedure.query(async ({ ctx }) => {
const db = ctx.db as DrizzleDB;

Expand Down
Loading
Loading