Skip to content

feat: Modernize WebVirtCloud, add Python 3.12+ & openSUSE/SLES 15 support, and enhance test coverage - #688

Merged
catborise merged 10 commits into
retspen:masterfrom
catborise:master
Sep 25, 2026
Merged

catborise merged 10 commits into
retspen:masterfrom
catborise:master

Conversation

@catborise

Copy link
Copy Markdown
Collaborator

Summary

This PR provides a comprehensive modernization of WebVirtCloud:

  • Full support for Python 3.10 – 3.12 (with preparation for 3.13)
  • Multi-distribution installation and compute node support (openSUSE Leap 15.x / Tumbleweed, SLES 15, Ubuntu 24.04 LTS, Debian 12, Rocky Linux 9/10)
  • Containerization upgrade with reproducible Docker builds and a ready-to-use docker-compose.yml
  • Migration of REST API documentation from deprecated drf-yasg to drf-spectacular (OpenAPI 3.0)
  • Decoupled testing suite and full GitHub Actions CI matrix workflow for Python 3.10, 3.11, and 3.12
  • Security, reverse proxy, and permissions fixes

Key Changes

1. Multi-Distribution Support (openSUSE / SLES 15, Ubuntu 24.04, Debian 12, RHEL/Rocky)

  • openSUSE & SLES 15:
    • Added native Zypper package management, dedicated SUSE Nginx templates (conf/nginx/suse_nginx.conf), and Supervisor service handling in webvirtcloud.sh.
    • Added dynamic Python >= 3.10 binary auto-detection (e.g. python311) without modifying or overwriting host system symlinks.
    • Added openSUSE/SLES support to compute node bootstrapping (dev/libvirt-bootstrap.sh).
    • Adjusted minimum constraints (libvirt-python>=9.0.0, lxml>=4.9.0) so distro-provided binary packages work out of the box without recompilation errors.
    • Tested and verified end-to-end inside clean opensuse/leap:15.6 container.
  • Ubuntu 24.04 & Debian 12: Modernized package lists and service hooks in installer scripts.
  • RHEL / Rocky Linux: Improved compatibility for local development and compute setups.

2. Modern Containerization & Docker Compose

  • Updated Dockerfile with --system-site-packages and prebuilt distro bindings for faster, zero-compilation image builds.
  • Added /etc/my_init.d/10_webvirtcloud_init.sh container initialization script for runtime migrations, dynamic SECRET_KEY generation, and volume persistence.
  • Prevented baking temporary build secrets or local settings into images (.dockerignore and build cleanup).
  • Added modern docker-compose.yml with persistent SQLite and SSH volumes.

3. Python 3.12+ & Core Modernization

  • Locking & Networking: Modernized vrtManager/rwlock.py (threading._RLock → threading.RLock) for Python 3.12+ and replaced vendored IPy with stdlib ipaddress.
  • Console: Replaced deprecated optparse with argparse and fixed invalid regex escape sequences.
  • API Documentation: Migrated from deprecated drf-yasg to drf-spectacular for OpenAPI 3.0 support (/swagger/, /redoc/, /api/schema/).
  • Static Assets: Cleaned up stale drf-yasg references from static/staticfiles.json and recalculated manifest hash.

4. Security & Reverse Proxy Fixes

  • Fixed CSRF 403 errors when DEBUG=False by properly populating CSRF_TRUSTED_ORIGINS and adding reverse proxy forwarding headers (SECURE_PROXY_SSL_HEADER, USE_X_FORWARDED_HOST, USE_X_FORWARDED_PORT).
  • Fixed Nginx proxy protocol forwarding header documentation ($remote_addr → $scheme).
  • Hardened installer bootstrap (install.sh) with privilege escalation checks, fallback downloads, and ShellCheck compliance.
  • Removed opaque shortened URLs (bit.ly, clck.ru) in favor of direct repository URLs.

5. CI/CD & Testing

  • Added GitHub Actions Test Suite workflow (.github/workflows/test.yml) running matrix tests on Python 3.10, 3.11, and 3.12.
  • Decoupled test suite from bare-metal libvirt daemons via configurable test host (WEBVIRTCLOUD_TEST_LIBVIRT_URI) and standalone mocks.
  • All 96 Django tests and 38 vrtManager unit tests pass cleanly in automated CI.
  • Removed obsolete .travis.yml.

Verification

  • GitHub Actions CI: All matrix tests passed for Python 3.10, 3.11, and 3.12.
  • openSUSE Leap 15.6 (Docker): Full webvirtcloud.sh run (exit 0), manage.py test passed (96 passed, 15 skipped), vrtManager tests passed.
  • Docker Compose: Built and validated container startup, migrations, Nginx reverse proxy, and Sign-In page response (HTTP 200/302).

…ler execution

- Add Python 3.11 package detection and alternatives configuration for openSUSE/SLES
- Relax libvirt-python and lxml minimum versions in requirements to match distro-provided packages
- Gracefully handle missing hostname command in minimal/containerized SUSE environments
- Prevent redundant git clone if workspace already exists in target path
…ulti-distro setup

- Replace shortened URL links (bit.ly, clck.ru) with direct upstream/repository paths
- Modernize and reorganize manual production installation guides for Ubuntu/Debian, RHEL/Rocky, and openSUSE/SLES
- Fix Nginx proxy header ($remote_addr -> $scheme) and streamline Supervisor configs
- Unify Compute Node (Hypervisor) setup into a dedicated section
- Update test running documentation reflecting isolated mocks and WEBVIRTCLOUD_TEST_LIBVIRT_URI
- Document drf-spectacular OpenAPI 3.0 schema endpoints and fix typos
- Convert screenshot gallery to clean Markdown table formatting
…taticfiles.json

- Remove legacy .travis.yml since continuous integration runs on GitHub Actions
- Remove 34 obsolete drf-yasg file mappings from static/staticfiles.json
- Recompute staticfiles manifest hash to match current static asset inventory
… 3.12

- Configure matrix testing across Python 3.10, 3.11, and 3.12 on ubuntu-latest
- Install system libvirt, ldap, and C build dependencies with pip cache
- Run Django database migrations and test suite
- Run vrtManager unit test discovery with test_*.py pattern
- Fix vrtManager unittest discovery command in README.md
…e docker-compose setup

- Add python3-libvirt to apt and configure --system-site-packages for zero-compilation build
- Introduce 10_webvirtcloud_init.sh to handle container startup, migrations, and SQLite persistence
- Exclude host settings.py in .dockerignore for clean builds from settings.py.template
- Add docker-compose.yml with persistent data and SSH volume mounts
- Document Docker Compose deployment in README.md
…ymlinks, and injection safety

- Prevent baking SECRET_KEY into Docker image during collectstatic
- Persist runtime SECRET_KEY in data/secret_key volume or read from environment
- Remove sed-based injection of CSRF_TRUSTED_ORIGINS in favor of native os.getenv
- Avoid overwriting system python3 symlink on SUSE; auto-detect python 3.10+ binaries
- Make /etc/hosts entry in installer idempotent
- Add security notice regarding compute node VNC/SPICE port firewall isolation
@catborise
catborise merged commit bb920b0 into retspen:master Sep 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant