diff --git a/.claude/skills/plugin-review/SKILL.md b/.claude/skills/plugin-review/SKILL.md index c274e1529..315b9eda2 100644 --- a/.claude/skills/plugin-review/SKILL.md +++ b/.claude/skills/plugin-review/SKILL.md @@ -61,4 +61,4 @@ A worked example: a WiFi-scanning plugin PR added `iw` plus its `setcap` grant t `scanPresence` defaults to `1` (`server/db/schema/app.sql`) when a plugin's `config.json` never maps it - every row that plugin reports asserts "currently online." That's correct for a plugin that only reports a device when it actually observed it *this cycle* (arp_scan, icmp_scan). It's wrong for a plugin whose data source is a **roster of known/historical entries** the upstream service remembers regardless of current connectivity (a client list built from DNS query history, a full device inventory export, anything that persists once discovered rather than expiring when the device goes quiet). Such a plugin re-reports the same entries every run whether or not the device is actually reachable, so the default `1` silently asserts permanent presence - read the actual data source (the API/file the script pulls from), not just the plugin's name, to tell which shape it is. `docs/PLUGINS_IMPORT_BEHAVIOR.md` already documents the fix (`scanPresence = 0` for "a reservation, a lease record, or a static IPAM entry"); this check exists because that guidance wasn't being verified against actual plugin behavior at review time. -If the plugin's own data does contain *some* real recency signal (an active DHCP lease, a last-seen timestamp within a threshold), prefer computing `scanPresence` per-row from that over a blanket static value - see `pihole_api_scan`'s `lastSeen`-vs-threshold check and `adguard_import`'s active-dynamic-lease check (`server/plugins/adguard_import/adguard_import.py`) for two different real implementations of the same idea. A `scanPresence = 0` row is an *abstention*, not a vote for offline - `update_presence_from_CurrentScan()`/`current_scan_presence_condition()` (`server/scan/presence.py`) only ever require *some* row asserting `1` to mark a device present, so a plugin correctly reporting "no evidence of presence" never suppresses a real scanner's positive signal for the same device. When writing the comment explaining this on a plugin's own `scanPresence` logic, state the plugin's own contract ("this value means no active lease was found, not that the device is offline") rather than asserting how the core presence-aggregation code behaves internally - that couples a leaf plugin file to framework internals it doesn't own and won't be updated if that logic ever changes. +If the plugin's own data does contain *some* real recency signal (an active DHCP lease, a last-activity timestamp within a threshold), prefer computing `scanPresence` per-row from that over a blanket static value - see `pihole_api_scan`'s `lastQuery`-vs-threshold check (`server/plugins/pihole_api_scan/pihole_api_scan.py`) and `adguard_import`'s active-dynamic-lease check (`server/plugins/adguard_import/adguard_import.py`) for two different real implementations of the same idea. A timestamp the upstream keeps refreshing for every client it merely remembers is not a recency signal: Pi-hole FTL refreshes `ips[].lastSeen` for every client it holds in memory, so `pihole_api_scan` reads the device's `lastQuery` (time of its last DNS query). A `scanPresence = 0` row is an *abstention*, not a vote for offline - `update_presence_from_CurrentScan()`/`current_scan_presence_condition()` (`server/scan/presence.py`) only ever require *some* row asserting `1` to mark a device present, so a plugin correctly reporting "no evidence of presence" never suppresses a real scanner's positive signal for the same device. When writing the comment explaining this on a plugin's own `scanPresence` logic, state the plugin's own contract ("this value means no active lease was found, not that the device is offline") rather than asserting how the core presence-aggregation code behaves internally - that couples a leaf plugin file to framework internals it doesn't own and won't be updated if that logic ever changes. diff --git a/.gemini/skills/plugin-review/SKILL.md b/.gemini/skills/plugin-review/SKILL.md index c274e1529..315b9eda2 100644 --- a/.gemini/skills/plugin-review/SKILL.md +++ b/.gemini/skills/plugin-review/SKILL.md @@ -61,4 +61,4 @@ A worked example: a WiFi-scanning plugin PR added `iw` plus its `setcap` grant t `scanPresence` defaults to `1` (`server/db/schema/app.sql`) when a plugin's `config.json` never maps it - every row that plugin reports asserts "currently online." That's correct for a plugin that only reports a device when it actually observed it *this cycle* (arp_scan, icmp_scan). It's wrong for a plugin whose data source is a **roster of known/historical entries** the upstream service remembers regardless of current connectivity (a client list built from DNS query history, a full device inventory export, anything that persists once discovered rather than expiring when the device goes quiet). Such a plugin re-reports the same entries every run whether or not the device is actually reachable, so the default `1` silently asserts permanent presence - read the actual data source (the API/file the script pulls from), not just the plugin's name, to tell which shape it is. `docs/PLUGINS_IMPORT_BEHAVIOR.md` already documents the fix (`scanPresence = 0` for "a reservation, a lease record, or a static IPAM entry"); this check exists because that guidance wasn't being verified against actual plugin behavior at review time. -If the plugin's own data does contain *some* real recency signal (an active DHCP lease, a last-seen timestamp within a threshold), prefer computing `scanPresence` per-row from that over a blanket static value - see `pihole_api_scan`'s `lastSeen`-vs-threshold check and `adguard_import`'s active-dynamic-lease check (`server/plugins/adguard_import/adguard_import.py`) for two different real implementations of the same idea. A `scanPresence = 0` row is an *abstention*, not a vote for offline - `update_presence_from_CurrentScan()`/`current_scan_presence_condition()` (`server/scan/presence.py`) only ever require *some* row asserting `1` to mark a device present, so a plugin correctly reporting "no evidence of presence" never suppresses a real scanner's positive signal for the same device. When writing the comment explaining this on a plugin's own `scanPresence` logic, state the plugin's own contract ("this value means no active lease was found, not that the device is offline") rather than asserting how the core presence-aggregation code behaves internally - that couples a leaf plugin file to framework internals it doesn't own and won't be updated if that logic ever changes. +If the plugin's own data does contain *some* real recency signal (an active DHCP lease, a last-activity timestamp within a threshold), prefer computing `scanPresence` per-row from that over a blanket static value - see `pihole_api_scan`'s `lastQuery`-vs-threshold check (`server/plugins/pihole_api_scan/pihole_api_scan.py`) and `adguard_import`'s active-dynamic-lease check (`server/plugins/adguard_import/adguard_import.py`) for two different real implementations of the same idea. A timestamp the upstream keeps refreshing for every client it merely remembers is not a recency signal: Pi-hole FTL refreshes `ips[].lastSeen` for every client it holds in memory, so `pihole_api_scan` reads the device's `lastQuery` (time of its last DNS query). A `scanPresence = 0` row is an *abstention*, not a vote for offline - `update_presence_from_CurrentScan()`/`current_scan_presence_condition()` (`server/scan/presence.py`) only ever require *some* row asserting `1` to mark a device present, so a plugin correctly reporting "no evidence of presence" never suppresses a real scanner's positive signal for the same device. When writing the comment explaining this on a plugin's own `scanPresence` logic, state the plugin's own contract ("this value means no active lease was found, not that the device is offline") rather than asserting how the core presence-aggregation code behaves internally - that couples a leaf plugin file to framework internals it doesn't own and won't be updated if that logic ever changes. diff --git a/.github/skills/plugin-review/SKILL.md b/.github/skills/plugin-review/SKILL.md index 59ae30823..9b548c803 100644 --- a/.github/skills/plugin-review/SKILL.md +++ b/.github/skills/plugin-review/SKILL.md @@ -61,4 +61,4 @@ A worked example: a WiFi-scanning plugin PR added `iw` plus its `setcap` grant t `scanPresence` defaults to `1` (`server/db/schema/app.sql`) when a plugin's `config.json` never maps it - every row that plugin reports asserts "currently online." That's correct for a plugin that only reports a device when it actually observed it *this cycle* (arp_scan, icmp_scan). It's wrong for a plugin whose data source is a **roster of known/historical entries** the upstream service remembers regardless of current connectivity (a client list built from DNS query history, a full device inventory export, anything that persists once discovered rather than expiring when the device goes quiet). Such a plugin re-reports the same entries every run whether or not the device is actually reachable, so the default `1` silently asserts permanent presence - read the actual data source (the API/file the script pulls from), not just the plugin's name, to tell which shape it is. `docs/PLUGINS_IMPORT_BEHAVIOR.md` already documents the fix (`scanPresence = 0` for "a reservation, a lease record, or a static IPAM entry"); this check exists because that guidance wasn't being verified against actual plugin behavior at review time. -If the plugin's own data does contain *some* real recency signal (an active DHCP lease, a last-seen timestamp within a threshold), prefer computing `scanPresence` per-row from that over a blanket static value - see `pihole_api_scan`'s `lastSeen`-vs-threshold check and `adguard_import`'s active-dynamic-lease check (`server/plugins/adguard_import/adguard_import.py`) for two different real implementations of the same idea. A `scanPresence = 0` row is an *abstention*, not a vote for offline - `update_presence_from_CurrentScan()`/`current_scan_presence_condition()` (`server/scan/presence.py`) only ever require *some* row asserting `1` to mark a device present, so a plugin correctly reporting "no evidence of presence" never suppresses a real scanner's positive signal for the same device. When writing the comment explaining this on a plugin's own `scanPresence` logic, state the plugin's own contract ("this value means no active lease was found, not that the device is offline") rather than asserting how the core presence-aggregation code behaves internally - that couples a leaf plugin file to framework internals it doesn't own and won't be updated if that logic ever changes. +If the plugin's own data does contain *some* real recency signal (an active DHCP lease, a last-activity timestamp within a threshold), prefer computing `scanPresence` per-row from that over a blanket static value - see `pihole_api_scan`'s `lastQuery`-vs-threshold check (`server/plugins/pihole_api_scan/pihole_api_scan.py`) and `adguard_import`'s active-dynamic-lease check (`server/plugins/adguard_import/adguard_import.py`) for two different real implementations of the same idea. A timestamp the upstream keeps refreshing for every client it merely remembers is not a recency signal: Pi-hole FTL refreshes `ips[].lastSeen` for every client it holds in memory, so `pihole_api_scan` reads the device's `lastQuery` (time of its last DNS query). A `scanPresence = 0` row is an *abstention*, not a vote for offline - `update_presence_from_CurrentScan()`/`current_scan_presence_condition()` (`server/scan/presence.py`) only ever require *some* row asserting `1` to mark a device present, so a plugin correctly reporting "no evidence of presence" never suppresses a real scanner's positive signal for the same device. When writing the comment explaining this on a plugin's own `scanPresence` logic, state the plugin's own contract ("this value means no active lease was found, not that the device is offline") rather than asserting how the core presence-aggregation code behaves internally - that couples a leaf plugin file to framework internals it doesn't own and won't be updated if that logic ever changes. diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md index a3008f70d..f9c77531c 100755 --- a/docs/PERFORMANCE.md +++ b/docs/PERFORMANCE.md @@ -46,6 +46,9 @@ The **System Info → Performance** page graphs NetAlertX's own process CPU%, me > [!NOTE] > The IO numbers come from the kernel's own per-process I/O accounting (`/proc//io`), which the kernel itself documents as an approximation. On some Docker storage drivers this can be badly inaccurate - confirmed on the deprecated `aufs` driver (common on older Synology NAS Docker setups), where union-filesystem copy-up writes get mis-attributed and can report write volumes many times larger than what's actually reaching disk (`docker info | grep -i "storage driver"` shows which driver you're on; `aufs` is a known-bad case, alongside its own `blkio throttle` warnings confirming the kernel's block-I/O accounting isn't properly wired up for it). If the IO chart shows large, consistent write numbers that don't match actual database growth (check the DB/WAL size on **System Info → Storage**), that's this known driver limitation, not a real cost. Newer Docker storage drivers (`overlay2`) don't have this problem. +> [!NOTE] +> The IO charts can read `0` when NetAlertX runs as a non-root user without elevated privileges (the default). The Python interpreter carries file capabilities (`NET_RAW`/`NET_ADMIN`), so the server process runs non-dumpable and cannot read its own `/proc/self/io`. CPU, memory, and scan duration are unaffected. Running as root or with `--privileged` does not have this limitation. See [Supported Capabilities](./docker-troubleshooting/missing-capabilities.md#supported-capabilities). + * Controlled by the `MAINT_PERF_DAYS` setting (part of the `MAINT` plugin) - the number of days of history to retain, `0` disables collection. Defaults to `30`. * History cannot be backfilled - data starts accumulating from whenever collection is enabled. * `GET /health` also exposes live `process_cpu_pct`/`process_rss_mb` fields for external monitoring, independent of this history. diff --git a/docs/PIHOLE_GUIDE.md b/docs/PIHOLE_GUIDE.md index ecfcda6a1..c3302e851 100755 --- a/docs/PIHOLE_GUIDE.md +++ b/docs/PIHOLE_GUIDE.md @@ -15,7 +15,7 @@ To use this approach, make sure a Web UI password is configured in **Pi-hole**. | `PIHOLEAPI_URL` | Your Pi-hole base URL, including the port. | `http://192.168.1.82:9880/` | | `PIHOLEAPI_RUN_SCHD` | If you run multiple device scanner plugins, configure them to use the same schedule. | `*/5 * * * *` | | `PIHOLEAPI_PASSWORD` | The Pi-hole Web UI admin password. NetAlertX automatically handles Base64 encoding and decoding. | `passw0rd` | -| `PIHOLEAPI_SSL_VERIFY` | Whether to verify HTTPS certificates. Disable only when using self-signed certificates. | `False` | +| `PIHOLEAPI_VERIFY_SSL` | Whether to verify HTTPS certificates. Disable only when using self-signed certificates. | `False` | | `PIHOLEAPI_API_MAXCLIENTS` | Maximum number of devices to request from Pi-hole. The default value is usually sufficient. | `500` | | `PIHOLEAPI_FAKE_MAC` | Generate a deterministic fake MAC address from the IP address. | `False` | diff --git a/docs/PLUGINS_DEV.md b/docs/PLUGINS_DEV.md index cfce62b93..eeca8dda8 100755 --- a/docs/PLUGINS_DEV.md +++ b/docs/PLUGINS_DEV.md @@ -246,6 +246,7 @@ Check your plugin against these repo-wide conventions before opening a PR (verif - **Persist plugin state under `dbFolderPath`, config artifacts under `configPath`** — see [Persisting Plugin Data](#persisting-plugin-data-state--config-files) below. - **A plugin mapped to `mapped_to_table: "CurrentScan"` must also map `scanSourcePlugin`** (a static value via `mapped_to_column_data`, see [Static Value Mapping](#static-value-mapping) below), **and that value must equal the plugin's own `unique_prefix` exactly.** Whether the mapping is present at all isn't mechanically enforced by `test_plugin_conventions.py`, so review that by eye - but `test_scan_source_plugin_matches_unique_prefix` does mechanically check the value matches `unique_prefix` whenever the mapping exists, since `update_devices_data_from_scan()` (`server/scan/device_handling.py`) reads it verbatim to build the `_SET_ALWAYS`/`_SET_EMPTY` settings keys (`server/db/authoritative_handler.py`) - any mismatch (wrong case, a friendly display name, punctuation) means those settings silently never resolve, so the plugin's SET_ALWAYS/SET_EMPTY overrides get quietly ignored (this shipped for real: `freebox` used `"Freebox"` against `unique_prefix: "FREEBOX"`, `fritzbox` used `"Fritz!Box"` against `"FRITZBOX"`). Omitting the mapping entirely leaves `scanSourcePlugin` `NULL` on every row this plugin inserts, which silently breaks two other things: `create_new_devices()`'s `plugin_prefix = str(scanSourcePlugin).strip() if scanSourcePlugin else "NEWDEV"` mislabels devices this plugin creates as source `NEWDEV`; and `update_devices_data_from_scan()`'s `SELECT DISTINCT scanSourcePlugin FROM CurrentScan` + `[row[0] for row in plugin_rows if row[0]] or [None]` drops the `NULL` rows entirely (the `or [None]` fallback never triggers once any other plugin contributes a non-null prefix), so this plugin's `CurrentScan` rows never get picked up by the per-plugin device-update loop at all - the plugin can *insert* into `CurrentScan` but never actually confirm/update a device's presence. - **A setting's `dataType` and `default_value` must actually agree.** `dataType: "array"` (or `"object"`) means `default_value` must be a real JSON literal for that shape — `'["default"]'`, not the bare string `"default"`. `setting_value_to_python_type()` (`server/helper.py`) `json.loads()`s the default at runtime; a bare string fails that parse, silently logs a decode error, and returns `[]` instead of your intended default — this shipped for real in `devParentRelType`/`UI_theme`/`UI_TOPOLOGY_ORDER` before being caught. If `elementOptions` already sets `multiple`/`orderable: "false"`, that's a strong signal the setting is actually scalar and `dataType` should be `"string"`, not `"array"`, regardless of what UI widget (`select`, etc.) renders it. +- **Read a setting under the exact key `config.json` defines** (`_`, e.g. `PIHOLEAPI_VERIFY_SSL` for the `VERIFY_SSL` setting of `PIHOLEAPI`). `get_setting_value()` returns `""` for an unknown key, so a misspelled key silently turns the setting off. `test_own_setting_keys_exist_in_config` (`test/plugins/test_plugin_conventions.py`) checks every literal key that starts with the plugin's own prefix. - **Only set `"allow_raw_text": true` on a display-only column type (`textarea_readonly`).** Every plugin-sourced field is HTML/control-char-stripped by default before it reaches the DB; see [Field Sanitization](#field-sanitization) below. `test_allow_raw_text_only_on_safe_types` (`test/plugins/test_plugin_conventions.py`) enforces the type restriction; it can't catch a column that legitimately needs the opt-out but is rendered somewhere unsafe, so use it only for values that are never interpreted as HTML. - **A new system package or Python dependency needs mirroring across every build target it should work on, not just the root `Dockerfile`.** `.devcontainer/Dockerfile` is auto-generated (its own header says so) by `.devcontainer/scripts/generate-configs.sh`, which concatenates the root `Dockerfile` with `.devcontainer/resources/devcontainer-Dockerfile`. It's a committed, git-tracked file, not something read fresh at build time - a package added to the root `Dockerfile` without re-running that script leaves the devcontainer's checked-in Dockerfile stale, so the package is silently missing there until someone regenerates it. `Dockerfile.debian` (`docs/BUILDS.md`) is a second, separately-maintained build target with its own `apt-get install` package list and `setcap` calls - a system package (and its `setcap` grant, if it needs one) has to be added there too, not just to the Alpine `Dockerfile`. For a new Python dependency, the root `requirements.txt` is not the only one either: `install/ubuntu24/requirements.txt` and `install/proxmox/requirements.txt` are separate lists for their respective non-Docker install methods, not mechanically kept in sync with the root file (no CI check covers this) - check whether the new dependency is actually needed by those install methods too rather than assuming the root file alone is enough. diff --git a/docs/docker-troubleshooting/missing-capabilities.md b/docs/docker-troubleshooting/missing-capabilities.md index 74b476282..23ac98230 100644 --- a/docs/docker-troubleshooting/missing-capabilities.md +++ b/docs/docker-troubleshooting/missing-capabilities.md @@ -12,6 +12,23 @@ Network scanning and monitoring requires low-level network access that these cap This occurs when the container doesn't have the necessary Linux capabilities granted. Docker containers run with limited capabilities by default, and network monitoring tools need elevated network privileges. +## Supported Capabilities + +NetAlertX supports exactly these capabilities; anything beyond them triggers the [excessive capabilities](excessive-capabilities.md) warning. + +| Capability | Needed for | Required | +|------------|-----------|----------| +| `NET_RAW` | Raw socket operations by the scanners | Yes - the image's Python binary fails to start without it | +| `NET_ADMIN` | ARP scanning, and applying [ARP sysctls](arp-flux-sysctls.md) at runtime with host networking | Yes - the image's Python binary fails to start without it | +| `NET_BIND_SERVICE` | `nbtscan` binding to privileged UDP port 137 | Yes, for `nbtscan` - its binary carries this as a file capability, so without it `nbtscan` can't execute (NetBIOS discovery of legacy devices stops working) and a startup warning is logged | +| `CHOWN` | Root priming of `/data` and `/tmp` ownership at startup | Only with `cap_drop: [ALL]` - without it ownership is not changed and a warning is logged (see below) | +| `SETUID`, `SETGID` | Dropping from root to `PUID:PGID` at startup | Only when starting as root with `cap_drop: [ALL]` - without them the container keeps running as root ([details](PUID_PGID_SECURITY.md)) | + +The image sets `+eip` file capabilities on `nmap`, `arp-scan`, `iw`, `traceroute` and the Python interpreter (`NET_RAW`, `NET_ADMIN`), and on `nbtscan` (also `NET_BIND_SERVICE`). A binary with file capabilities fails with "Operation not permitted" if those capabilities are not in the container's bounding set, which is why the `cap_add` entries below are required. + +> [!NOTE] +> Because the Python interpreter carries file capabilities, the server process runs non-dumpable, and the kernel makes its `/proc/self/io` readable only by root. When NetAlertX runs as a non-root user without elevated privileges (the default), the **System Info → Performance** IO charts can therefore read `0`, while CPU, memory, and scan duration stay accurate. Running as root or with `--privileged` does not have this limitation. See [Performance](../PERFORMANCE.md#tracking-netalertxs-own-resource-usage-over-time). + ## How to Correct the Issue Add the required capabilities to your container: diff --git a/server/plugins/nbtscan_scan/nbtscan.py b/server/plugins/nbtscan_scan/nbtscan.py index cc7781326..525df00c7 100755 --- a/server/plugins/nbtscan_scan/nbtscan.py +++ b/server/plugins/nbtscan_scan/nbtscan.py @@ -82,6 +82,14 @@ def main(): return 0 +def is_valid_netbios_name(name): + """ + Return True if name looks like a real NetBIOS name (1-15 characters, not + an nbtscan placeholder such as or ). + """ + return 0 < len(name) <= 15 and not name.startswith('<') + + # =============================================================================== # Execute scan # =============================================================================== @@ -102,7 +110,7 @@ def execute_name_lookup(ip, timeout): output = subprocess.check_output( args, universal_newlines=True, - stderr=subprocess.STDOUT, + stderr=subprocess.PIPE, timeout=(timeout), text=True ) @@ -115,15 +123,21 @@ def execute_name_lookup(ip, timeout): # Split the output into lines lines = output.splitlines() - # Look for the first line containing a valid NetBIOS name entry + # Look for the first result row whose first field is exactly the queried IP for line in lines: - if 'Doing NBT name scan' not in line and ip in line: - # Split the line and extract the primary NetBIOS name - parts = line.split() - if len(parts) > 1: - domain_name = parts[1] - else: - mylog('verbose', [f'[{pluginName}] ⚠ ERROR - Unexpected output format: {line}']) + parts = line.split() + if not parts or parts[0] != ip: + continue + + if len(parts) > 2 and parts[1].lower() == 'sendto' and parts[2].lower().startswith('failed'): + mylog('verbose', [f'[{pluginName}] ⚠ ERROR - nbtscan reported a failure: {line}']) + continue + + if len(parts) > 1 and is_valid_netbios_name(parts[1]): + domain_name = parts[1] + break + + mylog('verbose', [f'[{pluginName}] ⚠ ERROR - Unexpected output format: {line}']) mylog('verbose', [f'[{pluginName}] Domain Name: {domain_name}']) @@ -134,7 +148,7 @@ def execute_name_lookup(ip, timeout): # if "NXDOMAIN" in e.output: # mylog('verbose', [f'[{pluginName}]', f"No PTR record found for IP: {ip}"]) # else: - mylog('verbose', [f'[{pluginName}] ⚠ ERROR - {e.output}']) + mylog('verbose', [f'[{pluginName}] ⚠ ERROR - {e.output} | stderr: {e.stderr}']) except subprocess.TimeoutExpired: mylog('verbose', [f'[{pluginName}] TIMEOUT - the process forcefully terminated as timeout reached']) diff --git a/server/plugins/pihole_api_scan/README.md b/server/plugins/pihole_api_scan/README.md index df6f92cef..464a37d24 100644 --- a/server/plugins/pihole_api_scan/README.md +++ b/server/plugins/pihole_api_scan/README.md @@ -9,7 +9,7 @@ The plugin connects to your Pi-hole’s API and retrieves: * IP addresses * Hostnames (if available) * Vendor info -* Last-seen timestamps +* Time of the last DNS query NetAlertX then uses this information to match or create devices in your system. @@ -31,7 +31,7 @@ No additional Pi-hole configuration is required. | ---------------------------- | -------------------------------------------------------------------------------- | | **PIHOLEAPI_URL** | Your Pi-hole base URL. | | **PIHOLEAPI_PASSWORD** | The Web UI base64 encoded (en-/decoding handled by the app) admin password. | -| **PIHOLEAPI_SSL_VERIFY** | Whether to verify HTTPS certificates. Disable only for self-signed certificates. | +| **PIHOLEAPI_VERIFY_SSL** | Whether to verify HTTPS certificates. Disable only for self-signed certificates. | | **PIHOLEAPI_RUN_TIMEOUT** | Request timeout in seconds. | | **PIHOLEAPI_API_MAXCLIENTS** | Maximum number of devices to request from Pi-hole. Defaults are usually fine. | | **PIHOLEAPI_FAKE_MAC** | Generate FAKE MAC from IP. | @@ -139,7 +139,9 @@ If names are missing, confirm they appear in Pi-hole’s own UI first. ### Notes -- Additional notes, limitations, Author info. +- **Online status comes from DNS activity.** A device counts as online when its last DNS query to Pi-hole is at most `PIHOLEAPI_CONSIDER_ONLINE` seconds old. Pi-hole's per-IP "last seen" value is not used, because Pi-hole keeps refreshing it for as long as it holds the client in memory (about 24 hours in practice), also after the device has left. +- Devices that send a DNS query only every few hours, such as smart plugs, powerline adapters or speakers, can show as offline although they are on. Run a presence scanner such as [ARPSCAN](https://docs.netalertx.com/plugins/arp_scan) on the same schedule. If PIHOLEAPI is your only presence source, raise `PIHOLEAPI_CONSIDER_ONLINE` instead (for example to `3600`). A device that has left the network then also counts as online for that long. +- With `PIHOLEAPI_GET_OFFLINE` enabled, offline devices are still imported (name, vendor, IP), but they are not marked as online. - Version: 1.0.0 - Author: `jokob-sk`, `leiweibau` diff --git a/server/plugins/pihole_api_scan/config.json b/server/plugins/pihole_api_scan/config.json index c0f7259a4..fa0039534 100644 --- a/server/plugins/pihole_api_scan/config.json +++ b/server/plugins/pihole_api_scan/config.json @@ -259,7 +259,7 @@ "description": [ { "language_code": "en_us", - "string": "Import offline devices. By default only online devices are retrieved." + "string": "Also import offline devices, without marking them online. By default only online devices are imported." } ] }, @@ -281,13 +281,13 @@ "name": [ { "language_code": "en_us", - "string": "Online if last seen" + "string": "Online if last query" } ], "description": [ { "language_code": "en_us", - "string": "Time in seconds to consider a device online." + "string": "A device is online if its last DNS query is at most this many seconds old." } ] }, @@ -549,6 +549,22 @@ } ] }, + { + "column": "helpVal4", + "mapped_to_column": "scanPresence", + "css_classes": "col-sm-2", + "show": false, + "type": "none", + "default_value": "", + "options": [], + "localized": ["name"], + "name": [ + { + "language_code": "en_us", + "string": "N/A" + } + ] + }, { "column": "Dummy", "mapped_to_column": "scanSourcePlugin", diff --git a/server/plugins/pihole_api_scan/pihole_api_scan.py b/server/plugins/pihole_api_scan/pihole_api_scan.py index 2afcd6df6..d1905dc55 100644 --- a/server/plugins/pihole_api_scan/pihole_api_scan.py +++ b/server/plugins/pihole_api_scan/pihole_api_scan.py @@ -43,6 +43,7 @@ PIHOLEAPI_API_MAXCLIENTS = None PIHOLEAPI_VERIFY_SSL = True PIHOLEAPI_GET_OFFLINE = False +PIHOLEAPI_CONSIDER_ONLINE = 300 PIHOLEAPI_RUN_TIMEOUT = 10 PIHOLEAPI_FAKE_MAC = get_setting_value('PIHOLEAPI_FAKE_MAC') VERSION_DATE = "NAX-PIHOLEAPI-1.0" @@ -75,6 +76,11 @@ def pihole_api_auth(): except requests.exceptions.Timeout: mylog('none', [f'[{pluginName}] Pi-hole auth request timed out. Try increasing PIHOLEAPI_RUN_TIMEOUT.']) return False + except requests.exceptions.SSLError as e: + # SSLError is a subclass of ConnectionError, so it must be caught first + mylog('none', [f'[{pluginName}] TLS certificate check failed during Pi-hole auth: {e}. ' + 'Turn off PIHOLEAPI_VERIFY_SSL if Pi-hole uses a self-signed certificate.']) + return False except requests.exceptions.ConnectionError: mylog('none', [f'[{pluginName}] Connection error during Pi-hole auth. Check PIHOLEAPI_URL and PIHOLEAPI_PASSWORD']) return False @@ -193,11 +199,17 @@ def get_pihole_network_devices(): return data.get('devices', []) +# ------------------------------------------------------------------ +def is_device_online(last_query, now_ts, consider_online): + """Return True if last_query (Unix seconds of the device's last DNS query, 0 = never) is at most consider_online seconds old.""" + return last_query > 0 and (now_ts - last_query) <= consider_online + + # ------------------------------------------------------------------ def gather_device_entries(): """ - Build a list of device entries. - Online status is determined by comparing lastSeen (in seconds) vs the current time. + Build a list of device entries, one per device IP. + A device is online if its last DNS query is at most PIHOLEAPI_CONSIDER_ONLINE seconds old. """ entries = [] devices = get_pihole_network_devices() @@ -214,29 +226,22 @@ def gather_device_entries(): if not device_ips: continue - # 1. Find the freshest timestamp across all IPs for this MAC - # This ensures if the device is active on ANY IP, the MAC is considered online. - max_last_seen = 0 - for ip_info in device_ips: - ls = ip_info.get('lastSeen', 0) - if ls > max_last_seen: - max_last_seen = ls - - # 2. Determine online status: (Current Time - Last Seen) <= PIHOLEAPI_CONSIDER_ONLINE - # Math is in seconds. - if (now_ts - max_last_seen) <= PIHOLEAPI_CONSIDER_ONLINE: - is_online = True - else: - is_online = False + # 1. Determine online status from the device's last DNS query. ips[].lastSeen is not + # used: FTL refreshes it for every client it still holds in memory, even when the + # client is not in the ARP cache. + last_query = device.get('lastQuery', 0) + is_online = is_device_online(last_query, now_ts, PIHOLEAPI_CONSIDER_ONLINE) - # 3. Skip if offline (and user doesn't want offline devices) + # 2. Skip if offline (and user doesn't want offline devices) if not is_online and not PIHOLEAPI_GET_OFFLINE: - mylog('verbose', [f'[{pluginName}] Not online in the last {PIHOLEAPI_CONSIDER_ONLINE}s, import of offline disabled (PIHOLEAPI_GET_OFFLINE) skipping device: {device}.']) + query_age = f'{now_ts - last_query}s ago' if last_query > 0 else 'never' + mylog('verbose', [f'[{pluginName}] Skipping offline device {hwaddr}: last DNS query {query_age}, ' + f'PIHOLEAPI_CONSIDER_ONLINE={PIHOLEAPI_CONSIDER_ONLINE}s, PIHOLEAPI_GET_OFFLINE disabled.']) continue mac_vendor = device.get('macVendor', '') - # 4. Process each valid IP for the device + # 3. Process each valid IP for the device for ip_info in device_ips: ip = ip_info.get('ip') @@ -257,8 +262,9 @@ def gather_device_entries(): 'ip': ip, 'name': name, 'macVendor': mac_vendor, - # Pass the Unix timestamp as a string for NAX tracking - 'lastQuery': str(max_last_seen) if max_last_seen > 0 else "" + # Unix timestamp of the device's last DNS query, as a string for NAX tracking + 'lastQuery': str(last_query) if last_query > 0 else "", + 'isOnline': is_online }) return entries @@ -281,7 +287,7 @@ def main(): PIHOLEAPI_PASSWORD = get_setting_value('PIHOLEAPI_PASSWORD') PIHOLEAPI_API_MAXCLIENTS = get_setting_value('PIHOLEAPI_API_MAXCLIENTS') # Accept boolean or string "True"/"False" - PIHOLEAPI_VERIFY_SSL = get_setting_value('PIHOLEAPI_SSL_VERIFY') + PIHOLEAPI_VERIFY_SSL = get_setting_value('PIHOLEAPI_VERIFY_SSL') PIHOLEAPI_RUN_TIMEOUT = get_setting_value('PIHOLEAPI_RUN_TIMEOUT') PIHOLEAPI_GET_OFFLINE = get_setting_value('PIHOLEAPI_GET_OFFLINE') PIHOLEAPI_CONSIDER_ONLINE = get_setting_value('PIHOLEAPI_CONSIDER_ONLINE') @@ -315,7 +321,10 @@ def main(): watched3=str(entry['lastQuery']), watched4="", extra=pluginName, - foreignKey=str(entry['mac']) + foreignKey=str(entry['mac']), + # Mapped to CurrentScan.scanPresence: an offline device imported via + # PIHOLEAPI_GET_OFFLINE must not be marked as present + helpVal4='1' if entry['isOnline'] else '0' ) else: mylog('verbose', [f"[{pluginName}] Skipping invalid MAC (see PIHOLEAPI_FAKE_MAC setting): {entry['name']}|{entry['mac']}|{entry['ip']}"]) diff --git a/server/scan/name_resolution.py b/server/scan/name_resolution.py index d3cf654d7..dd7e475ac 100755 --- a/server/scan/name_resolution.py +++ b/server/scan/name_resolution.py @@ -19,14 +19,22 @@ def __init__(self, db): self.db = db def resolve_from_plugin(self, plugin: str, pMAC: str, pIP: str) -> ResolvedName: + """Return the name a plugin resolved for this MAC at this IP, newest row first. + + Plugins store one Plugins_Objects row per (MAC, IP) queried and old-IP rows + are kept, so the MAC lookup is bound to the current IP. If none matches and + NEWDEV_IP_MATCH_NAME is on, falls back to any row for the IP. + """ sql = self.db.sql nameNotFound = ResolvedName() - # Check by MAC + # Check by MAC and current IP sql.execute(""" SELECT watchedValue2 FROM Plugins_Objects - WHERE plugin = ? AND objectPrimaryId = ? - """, (plugin, pMAC)) + WHERE plugin = ? AND objectPrimaryId = ? AND objectSecondaryId = ? + ORDER BY dateTimeChanged DESC, "index" DESC + LIMIT 1 + """, (plugin, pMAC, pIP)) result = sql.fetchall() # self.db.commitDB() # Issue #1251: Optimize name resolution lookup if result: @@ -39,6 +47,8 @@ def resolve_from_plugin(self, plugin: str, pMAC: str, pIP: str) -> ResolvedName: sql.execute(""" SELECT watchedValue2 FROM Plugins_Objects WHERE plugin = ? AND objectSecondaryId = ? + ORDER BY dateTimeChanged DESC, "index" DESC + LIMIT 1 """, (plugin, pIP)) result = sql.fetchall() # self.db.commitDB() # Issue #1251: Optimize name resolution lookup diff --git a/server/scan/resource_history.py b/server/scan/resource_history.py index 4df0e33ff..2f8ddfa2e 100644 --- a/server/scan/resource_history.py +++ b/server/scan/resource_history.py @@ -25,20 +25,31 @@ def get_process_cpu_times_and_io(): children field, so IO here only ever reflects NetAlertX's own direct I/O, never a plugin subprocess's. - Defensive: some container security profiles (custom seccomp, restricted - runtimes) can make io_counters()/cpu_times() raise psutil.AccessDenied or - AttributeError - sampling must never be able to break the actual scan - tick, so failures fall back to zeros rather than propagating. + CPU and IO are sampled independently, each falling back to zeros on + psutil.Error/AttributeError, so sampling never breaks the scan tick. A + non-dumpable process (e.g. python3 carrying file capabilities) can't read + /proc/self/io, so IO may be zero while CPU, read from /proc/self/stat, + stays accurate. """ + cpu_time_s = 0.0 + read_bytes = 0 + write_bytes = 0 + try: p = psutil.Process() t = p.cpu_times() - io = p.io_counters() cpu_time_s = t.user + t.system + t.children_user + t.children_system - return cpu_time_s, io.read_bytes, io.write_bytes except (psutil.Error, AttributeError) as e: - mylog("verbose", [f"[resource_history] psutil sampling failed, using zeros: {e}"]) - return 0.0, 0, 0 + mylog("verbose", [f"[resource_history] CPU sampling failed, using zero: {e!r}"]) + + try: + io = psutil.Process().io_counters() + read_bytes = io.read_bytes + write_bytes = io.write_bytes + except (psutil.Error, AttributeError) as e: + mylog("verbose", [f"[resource_history] IO sampling failed, using zeros: {e!r}"]) + + return cpu_time_s, read_bytes, write_bytes def insert_resource_history(db, pre, post, duration_ms, tick_failed=False): diff --git a/server/scan/session_events.py b/server/scan/session_events.py index 321ae7a27..7b3fad563 100755 --- a/server/scan/session_events.py +++ b/server/scan/session_events.py @@ -50,9 +50,9 @@ def _known_plugin_ip_addresses_sql(all_plugins): """Build a SQL derived-table body (scanMac, knownAddr) unioning every (plugin, column_role) pair get_plugin_columns_for_field() names for the 'ip' field - the Plugins_Objects-derived half of insert_events()'s IP - Changed query's additive "known" test. An address already reported (and - not missing-in-last-scan) by any - eligible plugin for a MAC counts as known, on top of the existing + Changed query's additive "known" test. An address already reported by any + eligible plugin for a MAC, unchanged since the plugin's previous run + (status watched-not-changed), counts as known, on top of the existing devPrimaryIPv4/devPrimaryIPv6/devLastIP three-slot check - never instead of it, so a device whose only contributing plugins can't participate (dockerdisc, wificanary) gets an always-empty set here and the overall @@ -74,7 +74,7 @@ def _known_plugin_ip_addresses_sql(all_plugins): f"""SELECT objectPrimaryId AS scanMac, {column} AS knownAddr FROM Plugins_Objects WHERE plugin = '{plugin}' - AND status != 'missing-in-last-scan' + AND status = 'watched-not-changed' AND {column} IS NOT NULL""" for plugin, column in pairs ) diff --git a/test/db_test_helpers.py b/test/db_test_helpers.py index d49c18fc0..aa29ca018 100644 --- a/test/db_test_helpers.py +++ b/test/db_test_helpers.py @@ -110,7 +110,7 @@ scanType TEXT, scanCreatesDevice INTEGER NOT NULL DEFAULT 1, scanNotificationMode TEXT NOT NULL DEFAULT 'normal', - scanPresence INTEGER NOT NULL DEFAULT 1 + scanPresence INTEGER NOT NULL DEFAULT 1 CHECK (scanPresence IN (0, 1)) ) """ @@ -664,7 +664,7 @@ def make_plugin_event_row(prefix: str, primary_id: str, secondary_id="sec", watched1="val1", watched2="", watched3="", watched4="", changed="2026-01-01 00:00:00", extra="", user_data="", foreign_key="", - status="not-processed", help_val1=None): + status="not-processed", help_val1=None, help_val4=None): """Build a tuple mimicking a raw plugin output row (19 columns + index).""" return ( 0, # index (placeholder, not used for events) @@ -685,22 +685,23 @@ def make_plugin_event_row(prefix: str, primary_id: str, secondary_id="sec", help_val1, None, # helpVal2 None, # helpVal3 - None, # helpVal4 + help_val4, ) def seed_plugin_object(cur, prefix: str, primary_id: str, secondary_id="sec", watched1="val1", status="watched-not-changed", - changed="2026-01-01 00:00:00"): + changed="2026-01-01 00:00:00", + watched2=""): """Insert a row into Plugins_Objects to simulate a pre-existing object.""" cur.execute( """INSERT INTO Plugins_Objects (plugin, objectPrimaryId, objectSecondaryId, dateTimeCreated, dateTimeChanged, watchedValue1, watchedValue2, watchedValue3, watchedValue4, status, extra, userData, foreignKey) - VALUES (?, ?, ?, ?, ?, ?, '', '', '', ?, '', '', '')""", - (prefix, primary_id, secondary_id, changed, changed, watched1, status), + VALUES (?, ?, ?, ?, ?, ?, ?, '', '', ?, '', '', '')""", + (prefix, primary_id, secondary_id, changed, changed, watched1, watched2, status), ) diff --git a/test/plugins/test_nbtscan_parsing.py b/test/plugins/test_nbtscan_parsing.py new file mode 100644 index 000000000..798d086bc --- /dev/null +++ b/test/plugins/test_nbtscan_parsing.py @@ -0,0 +1,104 @@ +""" +Tests for nbtscan_scan/nbtscan.py output parsing in execute_name_lookup(). + + pytest test/plugins/test_nbtscan_parsing.py -v +""" + +import os +import sys +import tempfile +import types +from unittest.mock import MagicMock, patch + +_tmp_log = tempfile.mkdtemp() + +_stubbed_module_names = [] + + +def _stub(name: str, **attrs): + """Register a placeholder module in sys.modules if the real one is absent.""" + if name not in sys.modules: + mod = types.ModuleType(name) + for k, v in attrs.items(): + setattr(mod, k, v) + sys.modules[name] = mod + _stubbed_module_names.append(name) + + +_stub("pytz", timezone=lambda tz: tz) +_stub("conf", tz=None) +_stub("const", logPath=_tmp_log) +_stub("plugin_helper", Plugin_Objects=MagicMock) +_stub("logger", mylog=lambda *a: None, Logger=MagicMock) +_stub("helper", get_setting_value=lambda k, default="": "") +_stub("models") +_stub("models.device_instance", DeviceInstance=MagicMock) + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "server", "plugins", "nbtscan_scan")) + +import nbtscan # noqa: E402 + +for _name in _stubbed_module_names: + sys.modules.pop(_name, None) + +IP = "192.168.129.11" + +HEADER = "Doing NBT name scan for addresses from 192.168.129.11\n\n" +TABLE_HEADER = ( + "IP address NetBIOS Name Server User MAC address \n" + "------------------------------------------------------------------------------\n" +) + + +def _lookup(output, ip=IP): + """Run execute_name_lookup with nbtscan's output mocked.""" + with patch.object(nbtscan.subprocess, "check_output", return_value=output): + return nbtscan.execute_name_lookup(ip, 10) + + +def test_valid_result_row_returns_name(): + output = HEADER + TABLE_HEADER + f"{IP} MYPC aa:bb:cc:dd:ee:ff\n" + assert _lookup(output) == ("MYPC", "") + + +def test_sendto_failed_line_is_not_a_name(): + output = HEADER + f"{IP}\tSendto failed\n" + assert _lookup(output) == ("", "") + + +def test_sendto_failed_line_with_spaces_is_not_a_name(): + output = HEADER + f"{IP} Sendto failed\n" + assert _lookup(output) == ("", "") + + +def test_name_containing_failed_is_kept(): + output = HEADER + TABLE_HEADER + f"{IP} FAILED-SRV aa:bb:cc:dd:ee:ff\n" + assert _lookup(output) == ("FAILED-SRV", "") + + +def test_no_result_row_returns_empty(): + output = HEADER + TABLE_HEADER + assert _lookup(output) == ("", "") + + +def test_ip_must_be_first_field_not_substring(): + output = HEADER + TABLE_HEADER + "192.168.129.110 OTHERPC aa:bb:cc:dd:ee:ff\n" + assert _lookup(output) == ("", "") + + +def test_unknown_placeholder_name_rejected(): + output = HEADER + TABLE_HEADER + f"{IP} aa:bb:cc:dd:ee:ff\n" + assert _lookup(output) == ("", "") + + +def test_overlong_name_rejected(): + output = HEADER + TABLE_HEADER + f"{IP} {'A' * 20} aa:bb:cc:dd:ee:ff\n" + assert _lookup(output) == ("", "") + + +def test_called_process_error_logs_stderr(): + error = nbtscan.subprocess.CalledProcessError(1, "nbtscan", output="out", stderr="boom-stderr") + with patch.object(nbtscan.subprocess, "check_output", side_effect=error), \ + patch.object(nbtscan, "mylog") as mock_log: + assert nbtscan.execute_name_lookup(IP, 10) == ("", "") + assert any("boom-stderr" in str(call.args) for call in mock_log.call_args_list) diff --git a/test/plugins/test_pihole_api_scan.py b/test/plugins/test_pihole_api_scan.py new file mode 100644 index 000000000..448c9fd73 --- /dev/null +++ b/test/plugins/test_pihole_api_scan.py @@ -0,0 +1,216 @@ +"""Tests for the pihole_api_scan (PIHOLEAPI) plugin's online detection. + +pihole_api_scan.py is loaded with its NetAlertX-internal dependencies stubbed +out (same approach as test_adguard_import.py), so these tests run without a +Pi-hole instance. get_pihole_network_devices() is patched per test. + +Pi-hole FTL refreshes ips[].lastSeen for every client it still holds in +memory, even when the client is not in the ARP cache, so lastSeen stays fresh +for about 24 hours after a device left. The device's lastQuery (time of its +last DNS query) is what tells the plugin whether a device is still active. +""" + +import datetime +import importlib.util +import os +import re +import sys +import types +from pathlib import Path +from unittest.mock import MagicMock, patch + +import pytest + +INSTALL_PATH = os.getenv("NETALERTX_APP", "/app") +sys.path.extend([f"{INSTALL_PATH}/server/plugins", f"{INSTALL_PATH}/server"]) + +# Unix time of the debug run the bug report is based on (2026-10-31 20:05:10 UTC). +NOW_TS = 1793477110 + +_MAC_PATTERN = re.compile(r"^([0-9a-f]{2}:){5}[0-9a-f]{2}$") + + +def _fixed_now(as_string=True): + """Stand-in for timeNowUTC() that always returns NOW_TS.""" + now = datetime.datetime.fromtimestamp(NOW_TS, tz=datetime.timezone.utc) + return now.strftime("%Y-%m-%d %H:%M:%S") if as_string else now + + +def _load_pihole_api_scan_module(): + """Import pihole_api_scan.py with NetAlertX modules stubbed, then restore sys.modules.""" + missing_module = object() + previous_modules = {} + + def stub(name, **attributes): + """Put a fake module with the given attributes into sys.modules, remembering the previous entry.""" + previous_modules[name] = sys.modules.get(name, missing_module) + module = types.ModuleType(name) + for attribute, value in attributes.items(): + setattr(module, attribute, value) + sys.modules[name] = module + + stub("plugin_helper", Plugin_Objects=MagicMock, is_mac=lambda value: bool(_MAC_PATTERN.match(value))) + stub("utils.datetime_utils", timeNowUTC=_fixed_now) + stub("logger", mylog=MagicMock(), Logger=MagicMock()) + stub("helper", get_setting_value=MagicMock(return_value="UTC")) + stub("const", logPath="/tmp") + stub("conf", tz=None) + stub("pytz", timezone=MagicMock(return_value="UTC")) + stub("utils.crypto_utils", string_to_fake_mac=MagicMock(return_value="fa:ce:00:00:00:01")) + + module_path = Path(__file__).resolve().parents[2] / "server" / "plugins" / "pihole_api_scan" / "pihole_api_scan.py" + spec = importlib.util.spec_from_file_location("pihole_api_scan_script", module_path) + module = importlib.util.module_from_spec(spec) + try: + spec.loader.exec_module(module) + finally: + for name, previous_module in previous_modules.items(): + if previous_module is missing_module: + sys.modules.pop(name, None) + else: + sys.modules[name] = previous_module + + return module + + +pihole_api_scan = _load_pihole_api_scan_module() + + +def _api_device(hwaddr, ips, last_query, last_seen): + """Build one /api/network/devices entry with the fields Pi-hole FTL v6 returns.""" + return { + "id": 1, + "hwaddr": hwaddr, + "interface": "eth0", + "firstSeen": NOW_TS - 30 * 86400, + "lastQuery": last_query, + "numQueries": 42, + "macVendor": "Test Vendor", + "ips": [ + {"ip": ip, "name": f"host{index}.lan", "lastSeen": last_seen, "nameUpdated": last_seen} + for index, ip in enumerate(ips) + ], + } + + +def _gather(devices, get_offline=False, consider_online=300): + """Run gather_device_entries() against the given API devices and plugin settings.""" + with patch.object(pihole_api_scan, "get_pihole_network_devices", return_value=devices), \ + patch.object(pihole_api_scan, "PIHOLEAPI_GET_OFFLINE", get_offline), \ + patch.object(pihole_api_scan, "PIHOLEAPI_CONSIDER_ONLINE", consider_online), \ + patch.object(pihole_api_scan, "PIHOLEAPI_FAKE_MAC", False): + return pihole_api_scan.gather_device_entries() + + +class TestOnlineFromLastQuery: + def test_stale_last_query_with_fresh_last_seen_is_skipped(self): + """The reported bug: the phone's last DNS query is 5 h 47 min old, but + Pi-hole still refreshes its ips[].lastSeen (10 s old).""" + phone = _api_device("0a:60:b8:ad:b6:9d", ["192.168.178.64"], last_query=1793456269, last_seen=1793477100) + + assert _gather([phone]) == [] + + def test_entry_reports_device_last_query(self): + """The 'Last Query' column (watched3) shows the last DNS query, not ips[].lastSeen.""" + laptop = _api_device("f8:01:b4:b4:67:94", ["192.168.178.52"], last_query=1793477050, last_seen=1793477105) + + entries = _gather([laptop]) + + assert [entry["lastQuery"] for entry in entries] == ["1793477050"] + + def test_recent_last_query_is_online(self): + """A device whose last DNS query is 60 s old is imported as online.""" + laptop = _api_device("f8:01:b4:b4:67:94", ["192.168.178.52"], last_query=1793477050, last_seen=1793477105) + + entries = _gather([laptop]) + + assert [entry["isOnline"] for entry in entries] == [True] + + @pytest.mark.parametrize( + "last_query, expected_online", + [ + (1793476810, [True]), # exactly 300 s old: still online + (1793476809, []), # 301 s old: offline, so skipped + (1793477140, [True]), # 30 s in the future (Pi-hole clock ahead): online + ], + ) + def test_consider_online_window(self, last_query, expected_online): + """The PIHOLEAPI_CONSIDER_ONLINE window is inclusive, and a lastQuery slightly in the future counts as online.""" + laptop = _api_device("f8:01:b4:b4:67:94", ["192.168.178.52"], last_query=last_query, last_seen=1793477105) + + entries = _gather([laptop], consider_online=300) + + assert [entry["isOnline"] for entry in entries] == expected_online + + @pytest.mark.parametrize( + "last_query, consider_online", + [ + (None, 300), # field missing in the API response + (0, 300), # Pi-hole never saw a DNS query from this device + (0, 10_000_000_000), # a huge window must not turn "never" into online + ], + ) + def test_never_queried_device_is_offline(self, last_query, consider_online): + """A device without a lastQuery (missing or 0) is offline, whatever the window.""" + scale = _api_device("00:24:e4:d0:82:7c", ["192.168.178.50"], last_query=last_query, last_seen=1793477100) + if last_query is None: + del scale["lastQuery"] + + entries = _gather([scale], get_offline=True, consider_online=consider_online) + + assert [entry["isOnline"] for entry in entries] == [False] + + def test_multi_ip_device_rows_share_online_state(self): + """lastQuery is per device, so every IP row of a device gets the same state, + however fresh each IP's own lastSeen is.""" + tv = _api_device("f8:01:b4:b4:67:94", ["192.168.178.52", "192.168.178.53"], last_query=1793453291, last_seen=1793477100) + + entries = _gather([tv], get_offline=True) + + assert [(entry["ip"], entry["isOnline"]) for entry in entries] == [ + ("192.168.178.52", False), + ("192.168.178.53", False), + ] + + +def _use_settings(monkeypatch, settings): + """Serve settings to main() via get_setting_value, and restore the module globals main() rebinds after the test.""" + for name in ("PIHOLEAPI_URL", "PIHOLEAPI_PASSWORD", "PIHOLEAPI_API_MAXCLIENTS", "PIHOLEAPI_VERIFY_SSL", + "PIHOLEAPI_RUN_TIMEOUT", "PIHOLEAPI_GET_OFFLINE", "PIHOLEAPI_CONSIDER_ONLINE"): + monkeypatch.setattr(pihole_api_scan, name, getattr(pihole_api_scan, name)) + monkeypatch.setattr(pihole_api_scan, "get_setting_value", settings.get) + + +def _run_main(monkeypatch, devices): + """Run main() with PIHOLEAPI_GET_OFFLINE on against the given API devices; return add_object() kwargs keyed by (MAC, IP).""" + # Auth and the devices request are patched below, so main() needs no other settings. + _use_settings(monkeypatch, { + "PIHOLEAPI_URL": "http://pi.hole/", + "PIHOLEAPI_GET_OFFLINE": True, + "PIHOLEAPI_CONSIDER_ONLINE": 300, + }) + plugin_objects = MagicMock() + monkeypatch.setattr(pihole_api_scan, "pihole_api_auth", lambda: True) + monkeypatch.setattr(pihole_api_scan, "pihole_api_deauth", lambda: None) + monkeypatch.setattr(pihole_api_scan, "get_pihole_network_devices", lambda: devices) + monkeypatch.setattr(pihole_api_scan, "PIHOLEAPI_FAKE_MAC", False) + monkeypatch.setattr(pihole_api_scan, "plugin_objects", plugin_objects) + + pihole_api_scan.main() + return {(call.kwargs["primaryId"], call.kwargs["secondaryId"]): call.kwargs for call in plugin_objects.add_object.call_args_list} + + +class TestPresenceFromOnlineState: + def test_offline_import_is_not_present(self, monkeypatch): + """With PIHOLEAPI_GET_OFFLINE, an offline device is still imported, but its + rows must not assert presence (helpVal4 maps to CurrentScan.scanPresence).""" + phone = _api_device("0a:60:b8:ad:b6:9d", ["192.168.178.64"], last_query=1793456269, last_seen=1793477100) + tv = _api_device("f8:01:b4:b4:67:94", ["192.168.178.52", "192.168.178.53"], last_query=1793477050, last_seen=1793477100) + + calls = _run_main(monkeypatch, [phone, tv]) + + assert {key: kwargs.get("helpVal4") for key, kwargs in calls.items()} == { + ("0a:60:b8:ad:b6:9d", "192.168.178.64"): "0", + ("f8:01:b4:b4:67:94", "192.168.178.52"): "1", + ("f8:01:b4:b4:67:94", "192.168.178.53"): "1", + } diff --git a/test/plugins/test_pihole_api_scan_verify_ssl.py b/test/plugins/test_pihole_api_scan_verify_ssl.py new file mode 100644 index 000000000..c4d3898c7 --- /dev/null +++ b/test/plugins/test_pihole_api_scan_verify_ssl.py @@ -0,0 +1,117 @@ +"""Tests for the "Verify SSL" setting of the pihole_api_scan (PIHOLEAPI) plugin. + +pihole_api_scan.py is loaded with its NetAlertX-internal dependencies stubbed +out (same approach as test_adguard_import.py), so these tests run without a +Pi-hole instance. Only the HTTP call to Pi-hole is mocked; main() and +pihole_api_auth() run for real. +""" + +import importlib.util +import os +import sys +import types +from pathlib import Path +from unittest.mock import MagicMock + +import pytest + +INSTALL_PATH = os.getenv("NETALERTX_APP", "/app") +sys.path.extend([f"{INSTALL_PATH}/server/plugins", f"{INSTALL_PATH}/server"]) + + +def _load_pihole_api_scan_module(): + """Import pihole_api_scan.py with NetAlertX modules stubbed, then restore sys.modules.""" + missing_module = object() + previous_modules = {} + + def stub(name, **attributes): + """Put a fake module with the given attributes into sys.modules, remembering the previous entry.""" + previous_modules[name] = sys.modules.get(name, missing_module) + module = types.ModuleType(name) + for attribute, value in attributes.items(): + setattr(module, attribute, value) + sys.modules[name] = module + + stub("plugin_helper", Plugin_Objects=MagicMock, is_mac=MagicMock(return_value=True)) + stub("utils.datetime_utils", timeNowUTC=MagicMock()) + stub("logger", mylog=MagicMock(), Logger=MagicMock()) + stub("helper", get_setting_value=MagicMock(return_value="UTC")) + stub("const", logPath="/tmp") + stub("conf", tz=None) + stub("pytz", timezone=MagicMock(return_value="UTC")) + stub("utils.crypto_utils", string_to_fake_mac=MagicMock(return_value="fa:ce:00:00:00:01")) + + module_path = Path(__file__).resolve().parents[2] / "server" / "plugins" / "pihole_api_scan" / "pihole_api_scan.py" + spec = importlib.util.spec_from_file_location("pihole_api_scan_verify_ssl_script", module_path) + module = importlib.util.module_from_spec(spec) + try: + spec.loader.exec_module(module) + finally: + for name, previous_module in previous_modules.items(): + if previous_module is missing_module: + sys.modules.pop(name, None) + else: + sys.modules[name] = previous_module + + return module + + +pihole_api_scan = _load_pihole_api_scan_module() + + +def _use_settings(monkeypatch, settings): + """Serve settings to main() via get_setting_value, and restore the module globals main() rebinds after the test.""" + # raising=False: a global may only come into existence when main() runs. + for name in ("PIHOLEAPI_URL", "PIHOLEAPI_PASSWORD", "PIHOLEAPI_API_MAXCLIENTS", "PIHOLEAPI_VERIFY_SSL", + "PIHOLEAPI_RUN_TIMEOUT", "PIHOLEAPI_GET_OFFLINE", "PIHOLEAPI_CONSIDER_ONLINE"): + monkeypatch.setattr(pihole_api_scan, name, getattr(pihole_api_scan, name, None), raising=False) + monkeypatch.setattr(pihole_api_scan, "get_setting_value", settings.get) + + +def _login_failure(monkeypatch, error): + """Run pihole_api_auth() with the login request raising error; return its result and everything it logged as one string.""" + log = MagicMock() + monkeypatch.setattr(pihole_api_scan, "mylog", log) + monkeypatch.setattr(pihole_api_scan, "PIHOLEAPI_URL", "https://pi.hole/") + monkeypatch.setattr(pihole_api_scan, "PIHOLEAPI_VERIFY_SSL", True) + monkeypatch.setattr(pihole_api_scan.requests, "post", MagicMock(side_effect=error)) + + result = pihole_api_scan.pihole_api_auth() + return result, " ".join(str(call.args[1]) for call in log.call_args_list) + + +class TestVerifySsl: + def test_certificate_error_points_to_verify_ssl_setting(self, monkeypatch): + """A failed TLS certificate check at login names PIHOLEAPI_VERIFY_SSL instead of sending the user to URL and password.""" + result, logged = _login_failure(monkeypatch, pihole_api_scan.requests.exceptions.SSLError("certificate verify failed")) + + assert result is False + assert "PIHOLEAPI_VERIFY_SSL" in logged + assert "Check PIHOLEAPI_URL and PIHOLEAPI_PASSWORD" not in logged + + def test_plain_connection_error_still_points_to_url_and_password(self, monkeypatch): + """A connection error that is not a certificate problem keeps pointing to PIHOLEAPI_URL and PIHOLEAPI_PASSWORD.""" + result, logged = _login_failure(monkeypatch, pihole_api_scan.requests.exceptions.ConnectionError("connection refused")) + + assert result is False + assert "Check PIHOLEAPI_URL and PIHOLEAPI_PASSWORD" in logged + assert "PIHOLEAPI_VERIFY_SSL" not in logged + + @pytest.mark.parametrize("verify_ssl", [True, False]) + def test_verify_ssl_setting_reaches_pihole_request(self, monkeypatch, verify_ssl): + """The VERIFY_SSL setting (key PIHOLEAPI_VERIFY_SSL) is what main() passes as verify= to the Pi-hole API.""" + _use_settings(monkeypatch, { + "PIHOLEAPI_URL": "https://pi.hole/", + "PIHOLEAPI_PASSWORD": "secret", + "PIHOLEAPI_RUN_TIMEOUT": 30, + "PIHOLEAPI_VERIFY_SSL": verify_ssl, + }) + auth_response = MagicMock() + auth_response.json.return_value = {"session": {"valid": False}} + post = MagicMock(return_value=auth_response) + monkeypatch.setattr(pihole_api_scan.requests, "post", post) + monkeypatch.setattr(pihole_api_scan.requests.packages.urllib3, "disable_warnings", MagicMock()) + + pihole_api_scan.main() + + assert post.call_args.kwargs["verify"] is verify_ssl diff --git a/test/plugins/test_plugin_conventions.py b/test/plugins/test_plugin_conventions.py index 939e6deab..1753ee77e 100644 --- a/test/plugins/test_plugin_conventions.py +++ b/test/plugins/test_plugin_conventions.py @@ -235,6 +235,38 @@ def test_hardcoded_default_matches_config(plugin_name): ) +# --------------------------------------------------------------------------- +# A plugin reading one of its own settings under a key config.json never defines +# +# get_setting_value() returns "" for a key nobody defined, so a misspelled key +# silently turns the setting off. Only literal keys that start with the +# plugin's own unique_prefix are checked; core settings and other plugins' +# keys are out of scope. +# --------------------------------------------------------------------------- +_SETTING_KEY_RE = re.compile(r'''get_setting_value\(\s*["']([A-Za-z0-9_]+)["']''') + + +@pytest.mark.parametrize('plugin_name', _PLUGIN_NAMES) +def test_own_setting_keys_exist_in_config(plugin_name): + """Every literal get_setting_value('_X') in a plugin's scripts names a setting its config.json defines.""" + config = _load_config(plugin_name) + own_prefix = config.get('unique_prefix', '') + '_' + defined = set(_setting_defaults(config)) + + unknown = [] + for py_file in _plugin_py_files(plugin_name): + with open(py_file) as f: + source = f.read() + for key in _SETTING_KEY_RE.findall(source): + if key.startswith(own_prefix) and key not in defined: + unknown.append(f"{os.path.basename(py_file)}: get_setting_value('{key}')") + + assert not unknown, ( + f"{plugin_name}: setting key(s) not defined in config.json - get_setting_value() returns '' for them, " + "so the setting is silently ignored:\n" + "\n".join(unknown) + ) + + # --------------------------------------------------------------------------- # RUN_TIMEOUT reused as a per-call timeout inside a loop # diff --git a/test/scan/test_ip_changed_multi_address.py b/test/scan/test_ip_changed_multi_address.py index de4286019..1854bca4a 100644 --- a/test/scan/test_ip_changed_multi_address.py +++ b/test/scan/test_ip_changed_multi_address.py @@ -221,3 +221,64 @@ def test_two_new_addresses_one_cycle_one_row(self): info = rows[0]["eveAdditionalInfo"] assert "10.0.0.50" in info assert "IPv6-new" in info + + +class TestSingleAddressPluginOwnRowDoesNotSuppress: + """A plugin that reports one address per MAC (the internet-IP plugin's + shape) writes its current address into Plugins_Objects before + insert_events() runs. A row that is new or changed this cycle must not + make that address look already known.""" + + def _run(self, current_status): + conn = _make_db_with_plugins_objects() + cur = conn.cursor() + insert_device(cur, "internet", alert_down=1, present_last_scan=1, last_ip="203.0.113.1") + conn.commit() + + insert_current_scan_row_from_dict(conn, make_current_scan_dict("internet", scanLastIP="203.0.113.2")) + seed_plugin_object(cur, "INTRNT", "internet", secondary_id="203.0.113.2", status=current_status) + seed_plugin_object(cur, "INTRNT", "internet", secondary_id="203.0.113.1", status="missing-in-last-scan") + conn.commit() + + insert_events(DummyDB(conn), all_plugins=[_ip_eligible_plugin("INTRNT")]) + return conn.execute( + "SELECT eveIp FROM Events WHERE eveMac = 'internet' AND eveEventType = 'IP Changed'" + ).fetchall() + + def test_brand_new_address_fires(self): + rows = self._run("new") + assert [r["eveIp"] for r in rows] == ["203.0.113.2"] + + def test_returning_address_with_changed_watched_value_fires(self): + rows = self._run("watched-changed") + assert [r["eveIp"] for r in rows] == ["203.0.113.2"] + + +class TestVolatileWatchedValueTradeoff: + """Pins a known trade-off of counting only watched-not-changed plugin rows + as known: on a cycle where an eligible plugin's watched value for an + already-seen secondary address changed, that address is not treated as + known, so a multi-address device can get an IP Changed event for it.""" + + def _run(self, status): + conn = _make_db_with_plugins_objects() + cur = conn.cursor() + insert_device(cur, MAC, alert_down=1, present_last_scan=1, last_ip="IPv6-4") + cur.execute("UPDATE Devices SET devPrimaryIPv6 = 'IPv6-4' WHERE devMac = ?", (MAC,)) + conn.commit() + + insert_current_scan_row_from_dict(conn, make_current_scan_dict(MAC, scanLastIP="IPv6-4")) + insert_current_scan_row_from_dict(conn, make_current_scan_dict(MAC, scanLastIP="IPv6-2")) + seed_plugin_object(cur, "FREEBOX", MAC, secondary_id="IPv6-2", status=status) + conn.commit() + + insert_events(DummyDB(conn), all_plugins=[_ip_eligible_plugin()]) + return conn.execute( + "SELECT eveIp FROM Events WHERE eveMac = ? AND eveEventType = 'IP Changed'", (MAC,) + ).fetchall() + + def test_unchanged_known_address_does_not_fire(self): + assert self._run("watched-not-changed") == [] + + def test_known_address_with_changed_watched_value_fires(self): + assert [r["eveIp"] for r in self._run("watched-changed")] == ["IPv6-2"] diff --git a/test/scan/test_name_resolution.py b/test/scan/test_name_resolution.py new file mode 100644 index 000000000..5c7368e4f --- /dev/null +++ b/test/scan/test_name_resolution.py @@ -0,0 +1,169 @@ +""" +Tests for NameResolver.resolve_from_plugin (server/scan/name_resolution.py). + +A name-resolution plugin (DIGSCAN, NSLOOKUP, AVAHISCAN, NBTSCAN) stores one +Plugins_Objects row per (MAC, IP) it queried. When a device changes IP the old +row is never deleted, so the resolver must only trust the row for the device's +current IP instead of any row sharing its MAC (issue #1731). +""" + +import os +import sys + +import pytest + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) +from db_test_helpers import DummyDB, make_plugin_db, seed_plugin_object # noqa: E402 + +import scan.name_resolution as name_resolution # noqa: E402 +from scan.name_resolution import NameResolver # noqa: E402 + +MAC = "aa:bb:cc:dd:ee:01" +OTHER_MAC = "aa:bb:cc:dd:ee:02" +OLD_IP = "10.0.1.11" +NEW_IP = "10.5.1.40" +OLD_NAME = "nb-pre3581-03.example.local." +NEW_NAME = "nb-3520-20.example.local." +NOT_FOUND = "(name not found)" + +PLUGIN_CASES = [ + ("DIGSCAN", "resolve_dig"), + ("NSLOOKUP", "resolve_nslookup"), + ("AVAHISCAN", "resolve_mdns"), + ("NBTSCAN", "resolve_nbtlookup"), +] + + +@pytest.fixture +def plugin_conn(): + """In-memory DB with the plugin tables; yields (DummyDB, cursor, conn).""" + _, conn = make_plugin_db() + yield DummyDB(conn), conn.cursor(), conn + conn.close() + + +@pytest.fixture +def ip_match_setting(monkeypatch): + """Patch NEWDEV_IP_MATCH_NAME / NEWDEV_NAME_CLEANUP_REGEX for the resolver.""" + def _set(ip_match): + def _get(key, default=""): + if key == "NEWDEV_IP_MATCH_NAME": + return ip_match + if key == "NEWDEV_NAME_CLEANUP_REGEX": + return [] + return default + monkeypatch.setattr(name_resolution, "get_setting_value", _get) + return _set + + +@pytest.mark.parametrize("plugin,method", PLUGIN_CASES) +def test_current_ip_row_wins_over_stale_old_ip_row(plugin_conn, ip_match_setting, plugin, method): + """Two rows for one MAC: the row for the device's current IP is returned.""" + db, cur, _ = plugin_conn + ip_match_setting(False) + seed_plugin_object(cur, plugin, MAC, OLD_IP, watched2=OLD_NAME, + status="missing-in-last-scan", changed="2026-01-01 00:00:00") + seed_plugin_object(cur, plugin, MAC, NEW_IP, watched2=NEW_NAME, + changed="2026-02-01 00:00:00") + + resolved = getattr(NameResolver(db), method)(MAC, NEW_IP) + + assert resolved.raw == NEW_NAME + + +@pytest.mark.parametrize("plugin,method", PLUGIN_CASES) +def test_current_ip_row_wins_regardless_of_insertion_order(plugin_conn, ip_match_setting, plugin, method): + """The result must not depend on which row happens to be stored first.""" + db, cur, _ = plugin_conn + ip_match_setting(False) + seed_plugin_object(cur, plugin, MAC, NEW_IP, watched2=NEW_NAME, + changed="2026-02-01 00:00:00") + seed_plugin_object(cur, plugin, MAC, OLD_IP, watched2=OLD_NAME, + changed="2026-03-01 00:00:00") + + resolved = getattr(NameResolver(db), method)(MAC, NEW_IP) + + assert resolved.raw == NEW_NAME + + +@pytest.mark.parametrize("plugin,method", PLUGIN_CASES) +def test_stale_old_ip_row_is_not_used_for_new_ip(plugin_conn, ip_match_setting, plugin, method): + """Only an old-IP row exists and the device moved: no name, not the stale one.""" + db, cur, _ = plugin_conn + ip_match_setting(False) + seed_plugin_object(cur, plugin, MAC, OLD_IP, watched2=OLD_NAME, + status="missing-in-last-scan") + + resolved = getattr(NameResolver(db), method)(MAC, NEW_IP) + + assert resolved.raw == NOT_FOUND + assert resolved.cleaned == NOT_FOUND + + +@pytest.mark.parametrize("plugin,method", PLUGIN_CASES) +def test_missing_in_last_scan_current_ip_row_still_matches(plugin_conn, ip_match_setting, plugin, method): + """Status is not a filter: a resolved device's row goes missing-in-last-scan + once the plugin stops querying it, but is still the valid answer.""" + db, cur, _ = plugin_conn + ip_match_setting(False) + seed_plugin_object(cur, plugin, MAC, NEW_IP, watched2=NEW_NAME, + status="missing-in-last-scan") + + resolved = getattr(NameResolver(db), method)(MAC, NEW_IP) + + assert resolved.raw == NEW_NAME + + +def test_same_mac_and_ip_returns_newest_row(plugin_conn, ip_match_setting): + """Duplicate (MAC, IP) rows resolve deterministically to the most recently changed.""" + db, cur, _ = plugin_conn + ip_match_setting(False) + seed_plugin_object(cur, "DIGSCAN", MAC, NEW_IP, watched2=OLD_NAME, + changed="2026-01-01 00:00:00") + seed_plugin_object(cur, "DIGSCAN", MAC, NEW_IP, watched2=NEW_NAME, + changed="2026-02-01 00:00:00") + + assert NameResolver(db).resolve_dig(MAC, NEW_IP).raw == NEW_NAME + + +def test_other_plugin_rows_are_ignored(plugin_conn, ip_match_setting): + """A row from a different plugin never answers a DIGSCAN lookup.""" + db, cur, _ = plugin_conn + ip_match_setting(False) + seed_plugin_object(cur, "NSLOOKUP", MAC, NEW_IP, watched2=NEW_NAME) + + assert NameResolver(db).resolve_dig(MAC, NEW_IP).raw == NOT_FOUND + + +def test_ip_match_fallback_still_matches_other_mac_by_ip(plugin_conn, ip_match_setting): + """NEWDEV_IP_MATCH_NAME keeps matching another MAC's row by IP, tagged "(IP match)".""" + db, cur, _ = plugin_conn + ip_match_setting(True) + seed_plugin_object(cur, "DIGSCAN", OTHER_MAC, NEW_IP, watched2=NEW_NAME) + + resolved = NameResolver(db).resolve_dig(MAC, NEW_IP) + + assert resolved.raw == NEW_NAME + assert resolved.cleaned.endswith("(IP match)") + + +def test_ip_match_fallback_disabled_returns_not_found(plugin_conn, ip_match_setting): + """With NEWDEV_IP_MATCH_NAME off, another MAC's row is never used.""" + db, cur, _ = plugin_conn + ip_match_setting(False) + seed_plugin_object(cur, "DIGSCAN", OTHER_MAC, NEW_IP, watched2=NEW_NAME) + + assert NameResolver(db).resolve_dig(MAC, NEW_IP).raw == NOT_FOUND + + +def test_mac_row_takes_priority_over_ip_match_fallback(plugin_conn, ip_match_setting): + """A matching (MAC, IP) row wins over another MAC's row for the same IP.""" + db, cur, _ = plugin_conn + ip_match_setting(True) + seed_plugin_object(cur, "DIGSCAN", OTHER_MAC, NEW_IP, watched2=OLD_NAME) + seed_plugin_object(cur, "DIGSCAN", MAC, NEW_IP, watched2=NEW_NAME) + + resolved = NameResolver(db).resolve_dig(MAC, NEW_IP) + + assert resolved.raw == NEW_NAME + assert "(IP match)" not in resolved.cleaned diff --git a/test/scan/test_piholeapi_scan_presence_mapping.py b/test/scan/test_piholeapi_scan_presence_mapping.py new file mode 100644 index 000000000..aa1edfa78 --- /dev/null +++ b/test/scan/test_piholeapi_scan_presence_mapping.py @@ -0,0 +1,48 @@ +""" +PIHOLEAPI's config.json must route the plugin's helpVal4 to +CurrentScan.scanPresence (server/plugin.py:process_plugin_events()). + +pihole_api_scan.py writes helpVal4 '0' for a device it imports only because +PIHOLEAPI_GET_OFFLINE is on. Without the mapping, scanPresence falls back to +its default 1 and that offline device is marked as present. +""" + +import json +import os +import sys +from pathlib import Path + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) +from db_test_helpers import ( # noqa: E402 + make_plugin_db, + make_plugin_dict, + make_plugin_event_row, + CREATE_CURRENT_SCAN, +) + +from plugin import process_plugin_events # noqa: E402 + +CONFIG_PATH = Path(__file__).resolve().parents[2] / "server" / "plugins" / "pihole_api_scan" / "config.json" + + +def _piholeapi_plugin() -> dict: + """Return a plugin dict with PIHOLEAPI's real CurrentScan mapping from its config.json.""" + config = json.loads(CONFIG_PATH.read_text()) + plugin = make_plugin_dict(config["unique_prefix"]) + plugin["mapped_to_table"] = config["mapped_to_table"] + plugin["database_column_definitions"] = config["database_column_definitions"] + return plugin + + +def test_offline_row_does_not_assert_presence(): + """A PIHOLEAPI row with helpVal4 '0' lands in CurrentScan with scanPresence 0.""" + db, conn = make_plugin_db() + conn.execute(CREATE_CURRENT_SCAN) + conn.commit() + try: + row = make_plugin_event_row("PIHOLEAPI", "0a:60:b8:ad:b6:9d", secondary_id="192.168.178.64", help_val4="0") + process_plugin_events(db, _piholeapi_plugin(), [row]) + + assert conn.execute("SELECT scanMac, scanPresence FROM CurrentScan").fetchall() == [("0a:60:b8:ad:b6:9d", 0)] + finally: + conn.close() diff --git a/test/scan/test_resource_history.py b/test/scan/test_resource_history.py index e52ab2776..ab7bebf4f 100644 --- a/test/scan/test_resource_history.py +++ b/test/scan/test_resource_history.py @@ -55,9 +55,10 @@ def raising_process(): def test_cpu_times_falls_back_on_attribute_error(monkeypatch): - """A mocked cpu_times() missing children_* fields must fall back to zeros.""" + """A mocked cpu_times() missing children_* fields must fall back to zero CPU.""" fake_process = MagicMock() fake_process.cpu_times.return_value = SimpleNamespace(user=1.0, system=1.0) # no children_* + fake_process.io_counters.side_effect = psutil.AccessDenied() monkeypatch.setattr(rh.psutil, "Process", lambda: fake_process) result = rh.get_process_cpu_times_and_io() @@ -65,6 +66,38 @@ def test_cpu_times_falls_back_on_attribute_error(monkeypatch): assert result == (0.0, 0, 0) +def test_cpu_still_sampled_when_io_counters_access_denied(monkeypatch): + """ + A non-dumpable process (python3 with file caps) can't read /proc/self/io, + so io_counters() raises AccessDenied - CPU, read from the still-readable + /proc/self/stat, must not be zeroed along with it. + """ + fake_process = MagicMock() + fake_process.cpu_times.return_value = SimpleNamespace(user=1.0, system=2.0, children_user=3.0, children_system=4.0) + fake_process.io_counters.side_effect = psutil.AccessDenied() + monkeypatch.setattr(rh.psutil, "Process", lambda: fake_process) + + cpu_time_s, read_bytes, write_bytes = rh.get_process_cpu_times_and_io() + + assert cpu_time_s == pytest.approx(10.0) + assert read_bytes == 0 + assert write_bytes == 0 + + +def test_io_still_sampled_when_cpu_times_fails(monkeypatch): + """The reverse isolation: a failing cpu_times() must not zero a readable IO sample.""" + fake_process = MagicMock() + fake_process.cpu_times.side_effect = psutil.AccessDenied() + fake_process.io_counters.return_value = SimpleNamespace(read_bytes=100, write_bytes=200) + monkeypatch.setattr(rh.psutil, "Process", lambda: fake_process) + + cpu_time_s, read_bytes, write_bytes = rh.get_process_cpu_times_and_io() + + assert cpu_time_s == 0.0 + assert read_bytes == 100 + assert write_bytes == 200 + + # --------------------------------------------------------------------------- # insert_resource_history() # ---------------------------------------------------------------------------