Skip to content
databrickslabsPublic

About

GeoBrix is a high-performance spatial processing library.

Topics

Resources

Contributing

Security policy

Stars

36 stars

Watchers

2 watching

Forks

Latest commit

 

History

6,361 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GeoBriX

build documentation scala python license

Functions RasterX GridX VectorX PMTiles General

VizX

GeoBrix is a high-performance Databricks Labs spatial library that delivers the next generation of product-augmenting capabilities — raster, discrete global grids, and vector format I/O — and is built to drive you deeper into Databricks-native GEOMETRY/GEOGRAPHY and ST/H3 functions, not replace them. It is the modern successor to DBLabs Mosaic (now in maintenance).

Full docs: https://databrickslabs.github.io/geobrix/ — this README is the 2-minute tour.

GeoBrix on Databricks: results land in native GEOMETRY, H3 and FILE columns

Implementations

GeoBrix has one function set. You call it from SQL, Python, Unity Catalog SQL or Scala, and gbx.register(spark) picks the level that runs each function on the compute. It says why when it falls back. Lightweight and heavyweight are short names for the Python and JVM levels.

  • Python level (lightweight) — runs every function in Python on rasterio/pyogrio/shapely, no JAR and no init script. The env wheel carries its own GDAL, PROJ and GEOS. Runs on Serverless, standard (shared), Lakeflow pipelines, and ARM.
  • JVM level (heavyweight) — Scala + GDAL on dedicated clusters, where it is faster. Add the GeoBrix JAR (gbx.stage_jar stages it from the env wheel) as a cluster library, or use the GeoBrix DCS image; see Choose an install path. The code does not change between the two.
  • Native kernels where measured faster — GDAL, plus Rust for BNG cell encoding and TIN triangulation, both in the env wheel. We continue to assess where Rust can help further.

Packages

  • RasterX — full-spectrum raster I/O and analytics (gap-filling; the platform has no built-in raster): reprojection, terrain, spectral indices, XYZ/PMTiles tiling, and H3/quadbin/BNG aggregation, plus virtual tiles and a COG-preparation lane for memory-safe processing of multi-gigabyte rasters. Both tiers — lightweight pyrx and heavyweight Scala.
  • GridX — discrete global grids: British National Grid, CARTO quadbin, and custom user-defined grids — cell math, k-ring/k-loop, polyfill, tessellation, and grid-aware aggregation (pairs with native H3 for global hex). Both tiers — lightweight pygx and heavyweight Scala.
  • VectorX — vector operations that augment the native ST functions: MVT tile encoding, TIN elevation surfaces, authority-string CRS transforms, antimeridian handling, and distributed-shapely geometry validity, cleaning & coverage-validity, plus legacy-geometry migration. Both tiers — lightweight pyvx and heavyweight Scala.
  • VizX — notebook visualization: render raster tiles/files, whole VRT mosaics (plot_mosaic), and PMTiles/COG archives, and turn Spark geometry/H3-cell/grid DataFrames into GeoPandas for static & interactive maps. Python-only (no SQL); works with either tier.

All SQL functions register with a gbx_ prefix (e.g. gbx_rst_clip, gbx_bng_cellarea, gbx_st_asmvt) so usage is clearly attributable to GeoBrix on classic compute. Python/Scala bindings mirror the names. See benchmarks for light-vs-heavy timings.

Supported Databricks Runtimes

GeoBrix supports the following Databricks Runtime releases:

DBR Ubuntu Spark Python Scala Java GeoBrix
17.3 24.04 4.0.0 3.12.3 2.13.16 17 ✅ Supported
18 24.04 4.1.0 3.12.3 2.13.16 21 ✅ Supported
19 24.04 4.2.0 3.12.3 2.13.18 21 ✅ Supported

A single JAR runs on 17.3, 18, and 19: Scala 2.13 minor versions are binary-compatible, the JAR is compiled to Java-17 bytecode so it loads on all three JVMs, and Spark is a provided dependency.

On Databricks, GeoBrix installs as the env wheel for the runtime, with no light extra to pick: geobrix-<version>+env6-py3-none-any.whl for Serverless environment 6 (recommended), DBR 19, SQL warehouses and UC functions; geobrix-<version>+env5-py3-none-any.whl for Serverless environment 5, DBR 17.3 / 18 and Lakeflow pipelines. Each carries its own GDAL, PROJ and GEOS. See Installation for the wheels and their extras.

Quick start

Stage the env wheel for your runtime (a Releases artifact, not on PyPI) in a Unity Catalog Volume, then install it (Serverless env 6, recommended; add feature extras such as [vizx] on the same line):

%pip install "geobrix @ file:///Volumes/<catalog>/<schema>/<volume>/geobrix-<version>+env6-py3-none-any.whl"

Use the quoted geobrix[<extras>] @ file://… form (PEP 508, one argument). Don't put an extra on the path ('/Volumes/…/…whl[vizx]') — on Serverless, %pip keeps the surrounding quotes and pip reads [vizx] as part of the filename, failing with "Expected package name at the start of dependency specifier." The named form installs cleanly on Serverless, standard/shared, and ARM.

from databricks.labs import gbx
from databricks.labs.gbx.rasterx import functions as rx   # gbx_rst_* functions

gbx.register(spark)   # gbx_* functions plus the *_gbx readers and writers

# Read a GeoTIFF and compute with RasterX
rasters = spark.read.format("gtiff_gbx").load("/Volumes/<catalog>/<schema>/<volume>/*.tif")
rasters.select(rx.rst_width("tile"), rx.rst_srid("tile")).show()

# Vector read -> write (round-trips with the matching reader)
boroughs = spark.read.format("geojson_gbx").load("/Volumes/.../boroughs.geojson")
boroughs.write.format("geojson_gbx").mode("overwrite").save("/Volumes/.../out.geojson")

On a dedicated cluster, the same code also uses the JVM implementation once you add the JAR (gbx.stage_jar) or use the GeoBrix DCS image. See Surfaces & Levels. Moving from 0.5.x? Read the migration guide.

Readers & writers

Readers and writers (*_gbx) are pure Python and need no JAR. Full options, output schemas, and examples: Readers · Writers.

Readers

Format Reader
Raster (generic) raster_gbx
GeoTIFF gtiff_gbx
COG cog_gbx
NetCDF netcdf_gbx
Zarr zarr_gbx (plain stores; gbx_zarr_read adds versions)
PMTiles pmtiles_gbx
File lister file_gbx
LiDAR (LAS/LAZ) lidar_gbx
EXIF/GPS (images) exif_gbx
Vector (generic) vector_gbx
Shapefile shapefile_gbx
GeoJSON geojson_gbx
GeoJSONL geojsonl_gbx
GeoPackage gpkg_gbx
File Geodatabase file_gdb_gbx

Writers

Format Reader
Raster (generic) raster_gbx
GeoTIFF gtiff_gbx
COG cog_gbx
Zarr gbx_zarr_write (a function, not a DataSource)
PMTiles pmtiles_gbx
NetCDF netcdf_gbx
LiDAR (LAS/LAZ) lidar_gbx
Vector (generic) vector_gbx
Shapefile shapefile_gbx
GeoJSON geojson_gbx
GeoJSONL — sharded, multi-file geojsonl_gbx
GeoPackage gpkg_gbx
File Geodatabase file_gdb_gbx

The sharded GeoJSONL writer writes one shard per partition, with no driver merge. It is the recommended vector writer at any scale.

The vector readers/writers exchange geometry as WKB/WKT with companion *_srid columns — convert to/from Databricks GEOMETRY with st_geomfromwkb / st_aswkb (see Databricks Spatial).

Examples

End-to-end worked examples — runnable notebook series and app examples. See the Examples overview for the full catalog.

Notebooks

End-to-end worked examples live in examples/notebooks/, with a docs page for each one.

Notebook Series What it shows
eo-series Earth Observation (STAC) A four-notebook Earth Observation series over Sentinel-2 L2A (Alaska), moving from a vector area-of-interest → STAC discovery (Microsoft Planetary Computer) → band download → gridded (H3) raster tables → multi-band stacking and clipping, on the lightweight tier (Serverless).
xview Per-object clipping Loads high-resolution aerial GeoTIFFs from the xView Detection Challenge into Lakehouse tables and clips rasters to labeled GeoJSON objects: raw TGZ → gtiff raster table → GeoJSON object table (EWKT + SRID) → per-object clipped tiles written back to a Volume, using RasterX with the built-in Spatial SQL functions.
wireless-coverage 01 → 02a / 02b → 03a → 03b → 04 · LiDAR surfaces → H3 gridding + raster-pixel contrast → tower siting A six-notebook series over San Francisco. Parts 1 and 2a build the LiDAR surface stack (lidar_gbx, rst_binpoints_agg, rst_chm, rst_dtmfromgeoms_agg) — DSM, bare-earth DTM, CHM — as permanent Delta tables; Part 2b is the DSM-raster alternative. Part 3a H3-grids all three surfaces (rst_isoband, gbx_rst_h3_rastertogridmax, h3_cellfill). Part 3b is the raster-pixel counterpoint: native 1 m pixels vs H3 cells, with both a raster viewshed (rst_viewshed_towers) and an inline H3 line-of-sight pass run independently — the two engines compared head-to-head on the same towers. Part 4 places one candidate tower per H3 res-9 centroid, rules out ~65% with a cheap coarse quick-pass, then runs an exact H3 line-of-sight viewshed via h3_los_visible for each survivor, scoring sites on two independent factors (required mast height and viewshed cell count) and writing per-cell coverage-depth tables.
h3-rasterize DEM extra (Wireless Coverage series) Downloads a USGS 3DEP seamless (10 m) DEM for the San Francisco Bay Area via DemDownloader(), tiles it into distributed virtual tiles with the raster_gbx reader, clips each tile to a land mask from Overture Maps (rst_clip), extracts twelve 60 m elevation bands with rst_isoband, indexes them with Databricks product H3 (h3_try_coverash3) at resolution 9, assembles a multi-band raster stack, and dissolves per-group polygons on Spark via the product st_union_agg in cells_as_gdf.
helios Tiling to PMTiles A four-notebook solar site-selection series that turns one San Francisco bounding box into self-contained PMTiles archives — candidate rooftops (vector), aerial basemap (raster), and terrain slope/aspect/hillshade scoring (elevation) — then reshards a layer into a distributed multi-archive PMTiles mosaic + catalog for web-scale delivery, viewable in a single browser tab with no tile server.
vapor-eyes Methane monitoring Works a satellite methane signal over the Permian Basin — from a wide-area screen down to the operator whose well pad is leaking — reading Sentinel-5P (TROPOMI) swaths as points with the netcdf_gbx reader alongside EMIT plume data. Ships in two flavors: a production-ready Lakeflow pipeline and an interactive notebook series.
orthomosaic Photogrammetry Extracts per-image EXIF/GPS telemetry with exif_gbx, computes GSD, and runs distributed sparse SfM with pycolmap on Serverless env 5 — with an optional GPU dense MVS pass — to produce a georeferenced orthomosaic; publishes it via color-correction → COG → PMTiles, then (optionally) derives GeoSAM instance-object polygons and an rst_land_cover region map, serving the object model behind a Unity Gateway GPU Model Serving endpoint.
zarr-weather-features Zarr weather features Registers an ERA5-like weather cube in place as a transactional Zarr store (gbx_zarr_register), reads a box as blocks and as a table, samples 100 sites into a features table (gbx_zarr_sample), writes a derived wind speed back as a new variable in one transaction (gbx_zarr_write), reads the version before it, maps it as raster tiles clipped with rst_clip, and tags the version for training (gbx_zarr_tag), on the lightweight tier (Serverless environment 6 or Databricks Runtime 19).

Apps

App What it does
Genie Map Databricks App (React + kepler.gl) that turns Vapor-Eyes methane gold data into an interactive map you can query by natural language.

Known limitations

  • Native Databricks GEOMETRY/GEOGRAPHY are not produced directly yet — geometries are exchanged as WKB/WKT (+ *_srid); convert with the native ST functions (Databricks Spatial).
  • Spatial KNN is not yet ported.

Building, deploying, releasing

See the scripts folder and the docs.

Support

Databricks Labs projects are provided AS-IS, for exploration only, and are not covered by Databricks SLAs. Please file issues as GitHub Issues; they are reviewed as time permits. Do not file Databricks support tickets for these projects.

About

GeoBrix is a high-performance spatial processing library.

Topics

Resources

Contributing

Security policy

Stars

36 stars

Watchers

2 watching

Forks

Releases

Packages

Used by

Contributors

Languages