⚡️ZigQuery - A BigQuery Emulator

A BigQuery emulator server, built on Zig, GoogleSQL, and SQLite.
Focused on query engine correctness, performance, throughput, and stability

Usage

docker pull ghcr.io/recidiviz/zigquery:latest

Usage:
  zigquery-server --project=<name> [OPTIONS]

A local BigQuery emulator serving the REST API and the Storage Read API.

Flags take the form --name=value.

Options:
  --project=<name>        project ID to serve (required)
  --dataset=<name>        create an empty dataset with this ID at startup
  --host=<addr>           address both servers bind (default: 0.0.0.0)
  --port=<port>           REST API port (default: 9050)
  --grpc-port=<port>      Storage Read API gRPC port; 0 picks a free port (default: 9060)
  --database=<path>       SQLite file to store data in; :memory: or empty keeps data in memory (default: :memory:)
  --data-from-json=<path> load datasets and tables from a JSON file at startup; repeatable, files load in order
  --storage-emulator-host=<addr>
                          Cloud Storage emulator that load jobs read gs:// sources from, as host:port or
                          http://host:port (default: $STORAGE_EMULATOR_HOST)
  --log-level=<level>     accepted for compatibility; currently ignored
  --log-format=<format>   accepted for compatibility; currently ignored
  -h, --help              show this help and exit

To keep data across restarts, mount a volume at /data, which the image's nonroot user owns:

docker run -p 9050:9050 -v zigquery-data:/data ghcr.io/recidiviz/zigquery:latest \
  --project=my-project --database=/data/bigquery.db

Loading from Cloud Storage

Load jobs can specify sourceUris with gs:// objects to be read from a Cloud Storage emulator such as fake-gcs-server, never from real Cloud Storage. Point the emulator at it with STORAGE_EMULATOR_HOST or with --storage-emulator-host.

• A URI may hold one * wildcard, and each matching object is read in name order.
• CSV and newline-delimited JSON are supported, up to
• Up to 256 MiB per load job.
• Each storage emulator request has a 30-second timeout, and all of a load job's sources has 120;
• The server will not respond to other requests while a load is reading from GCS.

services:
  gcs:
    image: fsouza/fake-gcs-server:1.56.1
    command: ["-scheme", "http", "-port", "4443"]
  bigquery:
    image: ghcr.io/recidiviz/zigquery:latest
    command: ["--project=local-project", "--port=9050"]
    environment:
      STORAGE_EMULATOR_HOST: http://gcs:4443
    ports: ["9050:9050"]

Development

See docs/BUILD.md for environment setup and build commands.

Why Zig?

GoogleSQL's query analyzer powers the core of the emulator. It is written in C++ and requires a complex build system configured in Bazel. Zig can link C natively, so interop becomes nearly free.

Intentional, explicit control over memory allocation improves process stability.

CI & releases

Workflow	Trigger	What it does
ci.yml	PR, push	ephor + zig fmt, unit tests, the spec corpus under --strict, the concurrency harnesses, and a gen-proto drift check
bench.yml	push, bench label, manual	gsql-bench --json charted by github-action-benchmark on the benchmarks branch
release.yml	v*.*.* tag	four platform tarballs, a multi-arch GHCR image, and a GitHub release

Runners are powered by RunsOn on EC2 (see .github/runs-on.yml). Release builds for darwin run on GitHub-hosted larger runners: arm64 and x64 built and smoke-tested natively rather than cross-compiled.

License

zigquery is licensed under the Apache License 2.0; see NOTICE. Release binaries statically link third-party libraries whose licenses are collected in THIRD_PARTY_NOTICES. zigquery is not affiliated with or endorsed by Google. BigQuery is a trademark of Google LLC.

Addendum

The architecture comes from our work on the fork of @goccy's original implementation. We are very grateful for his contributions to open source software.