Repository navigation
Conversation
Quality of life for those who don't need or want a GDS file. KLayout is a big application, packaged differently on every distribution, and the flow needs it for the small task of writing out a GDS file at the very end. Without it, `make` stopped there: Error: KLayout not found. Install KLayout or set KLAYOUT_CMD. Now klayout.sh, when KLAYOUT_CMD is empty and the call is the stream-out (def2stream.py with an out_file), writes an empty file and says so: WARNING: klayout not found, mocking klayout by writing empty file <out_file> and the flow runs to the end. Installing KLayout opts out. Nobody mistakes an empty GDS file for a valid one, and the warning says why it is empty. Every other KLayout call, DRC, LVS, the gallery and the viewers, still stops with the error. No Makefile conditionals: the decision is made in klayout.sh, at the one call that writes the GDS file. The GDS merge no longer depends on check-klayout. DRC and LVS were guarded by check-klayout only through the merge they depend on, so they get it directly: `make drc` without KLayout stops with the same error as before. The hint under that error names what still needs KLayout. A hierarchical design works the same way: each block's GDS is written empty, and so is the parent's merge of them. flow/test/test_klayout_sh.py checks the stream-out writes an empty file and warns, DRC and the viewer still stop without KLayout, and an installed KLayout gets the call unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
There was a problem hiding this comment.
Code Review
This pull request refactors the flow to make KLayout an optional dependency by utilizing the klayout Python module for GDS/OAS stream generation, falling back to mocking (writing an empty file with a warning) when KLayout is entirely missing. It updates dependency installers, Makefiles, and documentation to reflect this change, and adds corresponding unit tests. The review feedback correctly identifies a critical issue in flow/util/def2stream.py where a subprocess version check could cause an unnecessary hard failure, and points out that the custom environment dictionary is not being forwarded to the spawned subprocesses.
KLayout is a big application, packaged differently on every distribution, and the flow needs it for the small task of writing the GDS file at the end. KLayout's own Python module, `pip install klayout`, does that task: it is a prebuilt wheel with no Qt and nothing to compile, for Linux (x86_64 and aarch64, glibc 2.27 and up) and macOS. def2stream.py now has a command line and is run by PYTHON_EXE. It takes the first KLayout it finds: 1. the klayout Python module, in-process; 2. a KLayout executable, KLAYOUT_CMD or klayout on PATH, run as before (klayout -zz -rd ... -r def2stream.py); 3. neither: the stream file is written empty with the warning klayout.sh gave for it until now. Only an absent KLayout is let through. A KLayout that is there but misconfigured stops the flow: KLAYOUT_CMD naming no file, an executable that fails `-v` (klayout.sh ran it under `set -e` too), or a failing merge. The executable gets the environment the choice was made with. klayout.sh goes back to running a KLayout executable only, for DRC, LVS, the gallery and the viewers, which still need one: the DRC and LVS decks are scripts in KLayout's Ruby-based DSL, which the Python module does not include. The wrapped-macro merge passed `-rd config_file=$(FILL_CONFIG)`, which def2stream.py never read; it is not passed any more. flow/test/test_def2stream.py checks each of the three choices: the module gets the arguments, an executable on PATH gets them as -rd variables with the environment and its exit status is returned, and without either the file is written empty with the warning. It checks that a misconfigured KLayout is an error and writes no file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
The flow writes its GDS file with KLayout's Python module, so the installer no longer installs the KLayout application. The module comes with the other Python packages, from etc/requirements-common_lock.txt: a prebuilt wheel for Python 3.10 and up on Linux (x86_64 and aarch64, glibc 2.27 and up) and macOS. The lock already needs Python 3.10, so this adds no new requirement on the Python version. Deleted: - etc/DependencyInstaller.sh: klayoutVersion, and the KLayout install for each OS: the CentOS 7, CentOS 8 and Rocky Linux 9 RPMs with their version check and reinstall, the Ubuntu .deb with its md5 table per release and the "Unsupported Ubuntu version" exit, the build from source on Ubuntu aarch64 with its Qt, Ruby and libgit2 build dependencies, `klayout` from apt on rodete and the Homebrew cask on macOS. - env.sh: the KLayout.app PATH entry on macOS. - flow/scripts/variables.mk: the lookup of a KLayout built into tools/install/klayout, which nothing installs. KLAYOUT_CMD is the klayout on PATH unless set, and is exported bare. - docs/user/BuildWithPrebuilt.md: the KLayout version note and the LD_LIBRARY_PATH line for a KLayout built from source. DRC, LVS, the gallery and the viewers still need the KLayout application, installed by those who run them; the docs say where to get it. None of them runs in the default flow or in CI's `finish metadata`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
1ccf595 to
1971407
Compare
|
@maliberty @eder-matheus @vvbandeira TL;DR klayout has been a thorny dependency and it confuses users This is intended to prune as much as possible from ORFS install scripts, etc. while giving each use-case the least chafing experience. We try PyPi klayout first, then local and finally we write an empty .gds file and a warning. The full klayout install is still needed for make lvs/drc and gallery, but I think those are out of the way for most users and PyPi klayout doesn't have ruby, so there's nothing for it, full klayout install is needed. This would be solved by documentation, this is an out of the way use-case you have to install klayout yourself. |
🔍 QoR checkMetrics reflect the PR merge build — i.e. what will land on the target branch. Commit 62 design(s) checked — 0 with regression(s), 1 without a comparable baseline. |
Follow-up to #4712, and a draft until it merges: the first commit here is #4712's.
KLayout's own Python module,
pip install klayout, writes the GDS file. It is a prebuilt wheel, with no Qt and nothing to compile, for Python 3.10 and up on Linux (x86_64 and aarch64, glibc 2.27 and up) and macOS. With it, the installer no longer installs the KLayout application, and the flow writes a real GDS file wherever its Python packages are installed.lb_32x128,make do-gdsLayoutDiff, 12 cells)Installation, simpler
etc/DependencyInstaller.shloses its KLayout install for every OS, and the module comes in with the other Python packages frometc/requirements-common_lock.txt. The lock already needs Python 3.10, so this adds no new requirement on the Python version. Deleted:klayoutVersionand the version-check-and-reinstall logic, written three times (CentOS 7, CentOS 8, Rocky Linux 9) around klayout.org RPMs..debdownload, its md5 table per release, and the "Unsupported Ubuntu version" exit, so KLayout's packaging no longer limits which Ubuntu releases the installer supports.klayoutfrom apt on rodete, and the Homebrew cask on macOS.env.sh: the KLayout.appPATHentry on macOS.flow/scripts/variables.mk: the lookup of a KLayout intools/install/klayout, which nothing installs.docs/user/BuildWithPrebuilt.md: the KLayout version note and theLD_LIBRARY_PATHline for a KLayout built from source.127 lines deleted from the installer,
env.sh,variables.mkand the docs, 13 added. Added:klayoutinetc/requirements-common.in, pinned at 0.30.12 in the lock, the version the installer installed until now.What still needs the KLayout application
make drc,make lvs, the gallery and the viewers. The DRC and LVS rule decks are scripts in KLayout's Ruby-based DSL, which the Python module does not include. None of them runs in the defaultmake, nor in CI'sfinish metadata. Those who run them install KLayout themselves; the docs say where.Commits
def2stream.pyhas a command line and is run byPYTHON_EXE. It takes the first KLayout it finds: the Python module in-process, then a KLayout executable (KLAYOUT_CMDorklayoutonPATH) run as before, then neither, and the file is written empty with klayout.sh: without KLayout, stream out an empty GDS file and warn #4712's warning. The warning moves fromklayout.shintodef2stream.py, andklayout.shgoes back to running the KLayout application only.env.sh,variables.mk, the requirements and the docs, as above.Tests
flow/test/test_def2stream.pychecks each of the three choices: the module gets the arguments; an executable onPATHgets them as-rdvariables, with the environment, and its exit status is returned; without either, the file is written empty with the warning. Only an absent KLayout is let through: aKLAYOUT_CMDnaming no file, or an executable that fails-v, is an error and writes no file.flow/test/test_klayout_sh.pykeeps klayout.sh: without KLayout, stream out an empty GDS file and warn #4712's checks that DRC and the viewer stop without KLayout, and that an installed KLayout gets the call unchanged.make do-gdson sky130hdlb_32x128once for each choice: the module and the application wrote the layouts in the table above, and with neither the file was written empty with the warning.WRAP_LEFS), which only gf12 designs use. It calls the same script with the same arguments as before.🤖 Generated with Claude Code