Skip to content

Write the GDS file with KLayout's Python module; the installer drops the KLayout application - #4757

Draft
oharboe wants to merge 3 commits into
The-OpenROAD-Project:masterfrom
oharboe:gds-pypi-klayout
Draft

oharboe wants to merge 3 commits into
The-OpenROAD-Project:masterfrom
oharboe:gds-pypi-klayout

Conversation

@oharboe

@oharboe oharboe commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Follow-up to #4712, and a draft until it merges: the first commit here is #4712's.

KLayout's own Python module, pip install klayout, writes the GDS file. It is a prebuilt wheel, with no Qt and nothing to compile, for Python 3.10 and up on Linux (x86_64 and aarch64, glibc 2.27 and up) and macOS. With it, the installer no longer installs the KLayout application, and the flow writes a real GDS file wherever its Python packages are installed.

sky130hd lb_32x128, make do-gds KLayout Python module 0.30.12 KLayout application 0.30.0
layout identical (LayoutDiff, 12 cells) identical
bytes that differ 26, the GDS timestamps
wall time, peak memory (one run) 0.34 s, 101 MB 1.36 s, 403 MB

Installation, simpler

etc/DependencyInstaller.sh loses its KLayout install for every OS, and the module comes in with the other Python packages from etc/requirements-common_lock.txt. The lock already needs Python 3.10, so this adds no new requirement on the Python version. Deleted:

  • klayoutVersion and the version-check-and-reinstall logic, written three times (CentOS 7, CentOS 8, Rocky Linux 9) around klayout.org RPMs.
  • On Ubuntu x86_64: the klayout.org .deb download, its md5 table per release, and the "Unsupported Ubuntu version" exit, so KLayout's packaging no longer limits which Ubuntu releases the installer supports.
  • On Ubuntu aarch64: the build of KLayout from source, with its Qt, Ruby and libgit2 build dependencies, the slowest step of a fresh aarch64 install.
  • klayout from apt on rodete, and the Homebrew cask on macOS.
  • env.sh: the KLayout.app PATH entry on macOS.
  • flow/scripts/variables.mk: the lookup of a KLayout in tools/install/klayout, which nothing installs.
  • docs/user/BuildWithPrebuilt.md: the KLayout version note and the LD_LIBRARY_PATH line for a KLayout built from source.

127 lines deleted from the installer, env.sh, variables.mk and the docs, 13 added. Added: klayout in etc/requirements-common.in, pinned at 0.30.12 in the lock, the version the installer installed until now.

What still needs the KLayout application

make drc, make lvs, the gallery and the viewers. The DRC and LVS rule decks are scripts in KLayout's Ruby-based DSL, which the Python module does not include. None of them runs in the default make, nor in CI's finish metadata. Those who run them install KLayout themselves; the docs say where.

Commits

  1. klayout.sh: without KLayout, stream out an empty GDS file and warn #4712: without KLayout, the GDS file is written empty with a warning.
  2. def2stream.py has a command line and is run by PYTHON_EXE. It takes the first KLayout it finds: the Python module in-process, then a KLayout executable (KLAYOUT_CMD or klayout on PATH) run as before, then neither, and the file is written empty with klayout.sh: without KLayout, stream out an empty GDS file and warn #4712's warning. The warning moves from klayout.sh into def2stream.py, and klayout.sh goes back to running the KLayout application only.
  3. The installer, env.sh, variables.mk, the requirements and the docs, as above.

Tests

  • flow/test/test_def2stream.py checks each of the three choices: the module gets the arguments; an executable on PATH gets them as -rd variables, with the environment, and its exit status is returned; without either, the file is written empty with the warning. Only an absent KLayout is let through: a KLAYOUT_CMD naming no file, or an executable that fails -v, is an error and writes no file.
  • flow/test/test_klayout_sh.py keeps klayout.sh: without KLayout, stream out an empty GDS file and warn #4712's checks that DRC and the viewer stop without KLayout, and that an installed KLayout gets the call unchanged.
  • End to end, make do-gds on sky130hd lb_32x128 once for each choice: the module and the application wrote the layouts in the table above, and with neither the file was written empty with the warning.
  • Not run: the wrapped-macro merge (WRAP_LEFS), which only gf12 designs use. It calls the same script with the same arguments as before.

🤖 Generated with Claude Code

Quality of life for those who don't need or want a GDS file. KLayout is
a big application, packaged differently on every distribution, and the
flow needs it for the small task of writing out a GDS file at the very
end. Without it, `make` stopped there:

  Error: KLayout not found. Install KLayout or set KLAYOUT_CMD.

Now klayout.sh, when KLAYOUT_CMD is empty and the call is the stream-out
(def2stream.py with an out_file), writes an empty file and says so:

  WARNING: klayout not found, mocking klayout by writing empty file <out_file>

and the flow runs to the end. Installing KLayout opts out. Nobody
mistakes an empty GDS file for a valid one, and the warning says why it
is empty. Every other KLayout call, DRC, LVS, the gallery and the
viewers, still stops with the error.

No Makefile conditionals: the decision is made in klayout.sh, at the
one call that writes the GDS file. The GDS merge no longer depends on
check-klayout. DRC and LVS were guarded by check-klayout only through
the merge they depend on, so they get it directly: `make drc` without
KLayout stops with the same error as before. The hint under that error
names what still needs KLayout. A hierarchical design
works the same way: each block's GDS is written empty, and so is the
parent's merge of them.

flow/test/test_klayout_sh.py checks the stream-out writes an empty file
and warns, DRC and the viewer still stop without KLayout, and an
installed KLayout gets the call unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the flow to make KLayout an optional dependency by utilizing the klayout Python module for GDS/OAS stream generation, falling back to mocking (writing an empty file with a warning) when KLayout is entirely missing. It updates dependency installers, Makefiles, and documentation to reflect this change, and adds corresponding unit tests. The review feedback correctly identifies a critical issue in flow/util/def2stream.py where a subprocess version check could cause an unnecessary hard failure, and points out that the custom environment dictionary is not being forwarded to the spawned subprocesses.

Comment thread flow/util/def2stream.py Outdated
oharboe and others added 2 commits October 28, 2026 08:45
KLayout is a big application, packaged differently on every
distribution, and the flow needs it for the small task of writing the
GDS file at the end. KLayout's own Python module, `pip install klayout`,
does that task: it is a prebuilt wheel with no Qt and nothing to
compile, for Linux (x86_64 and aarch64, glibc 2.27 and up) and macOS.

def2stream.py now has a command line and is run by PYTHON_EXE. It takes
the first KLayout it finds:

1. the klayout Python module, in-process;
2. a KLayout executable, KLAYOUT_CMD or klayout on PATH, run as before
   (klayout -zz -rd ... -r def2stream.py);
3. neither: the stream file is written empty with the warning
   klayout.sh gave for it until now.

Only an absent KLayout is let through. A KLayout that is there but
misconfigured stops the flow: KLAYOUT_CMD naming no file, an executable
that fails `-v` (klayout.sh ran it under `set -e` too), or a failing
merge. The executable gets the environment the choice was made with.

klayout.sh goes back to running a KLayout executable only, for DRC, LVS,
the gallery and the viewers, which still need one: the DRC and LVS decks
are scripts in KLayout's Ruby-based DSL, which the Python module does
not include.

The wrapped-macro merge passed `-rd config_file=$(FILL_CONFIG)`, which
def2stream.py never read; it is not passed any more.

flow/test/test_def2stream.py checks each of the three choices: the
module gets the arguments, an executable on PATH gets them as -rd
variables with the environment and its exit status is returned, and
without either the file is written empty with the warning. It checks
that a misconfigured KLayout is an error and writes no file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
The flow writes its GDS file with KLayout's Python module, so the
installer no longer installs the KLayout application. The module comes
with the other Python packages, from etc/requirements-common_lock.txt:
a prebuilt wheel for Python 3.10 and up on Linux (x86_64 and aarch64,
glibc 2.27 and up) and macOS. The lock already needs Python 3.10, so
this adds no new requirement on the Python version.

Deleted:

- etc/DependencyInstaller.sh: klayoutVersion, and the KLayout install
  for each OS: the CentOS 7, CentOS 8 and Rocky Linux 9 RPMs with their
  version check and reinstall, the Ubuntu .deb with its md5 table per
  release and the "Unsupported Ubuntu version" exit, the build from
  source on Ubuntu aarch64 with its Qt, Ruby and libgit2 build
  dependencies, `klayout` from apt on rodete and the Homebrew cask on
  macOS.
- env.sh: the KLayout.app PATH entry on macOS.
- flow/scripts/variables.mk: the lookup of a KLayout built into
  tools/install/klayout, which nothing installs. KLAYOUT_CMD is the
  klayout on PATH unless set, and is exported bare.
- docs/user/BuildWithPrebuilt.md: the KLayout version note and the
  LD_LIBRARY_PATH line for a KLayout built from source.

DRC, LVS, the gallery and the viewers still need the KLayout
application, installed by those who run them; the docs say where to get
it. None of them runs in the default flow or in CI's `finish metadata`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
@oharboe

oharboe commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

@maliberty @eder-matheus @vvbandeira TL;DR klayout has been a thorny dependency and it confuses users

This is intended to prune as much as possible from ORFS install scripts, etc. while giving each use-case the least chafing experience.

We try PyPi klayout first, then local and finally we write an empty .gds file and a warning.

The full klayout install is still needed for make lvs/drc and gallery, but I think those are out of the way for most users and PyPi klayout doesn't have ruby, so there's nothing for it, full klayout install is needed. This would be solved by documentation, this is an out of the way use-case you have to install klayout yourself.

@openroad-ci

openroad-ci commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

🔍 QoR check

Metrics reflect the PR merge build — i.e. what will land on the target branch.

Commit e0a9659 · Jenkins build #2 · Baseline: build · View build on dashboard

62 design(s) checked — 0 with regression(s), 1 without a comparable baseline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants