Skip to content

Implement APRS decoding and encoding across the vector corpus - #5

Merged
afourney merged 11 commits into
scaffold/aprs-vectorsfrom
implement/aprs-codecs
Oct 17, 2026
Merged

afourney merged 11 commits into
scaffold/aprs-vectorsfrom
implement/aprs-codecs

Conversation

@afourney

@afourney afourney commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #4: targets scaffold/aprs-vectors at b97beb7970c6b6fec2c0b6e30200f348bb87d96c.

Implement APRS decoding and encoding across the vector corpus, including compressed positions, Mic-E, objects/items, weather, telemetry, messages, queries, status, and NMEA. Preserve packet models and their JSON schema. Replace the envelope profile strings with the agreed configurable policy API.

  • Preserve every original TNC2 byte, source expectation, option, and provenance field. A frozen integrity test verifies preservation. Three later amendments (WX-067 to 069) are frozen separately.
  • Exercise all 1,564 packet occurrences, including stateful sequences. Every one is parsed by the public parse_packet and compared with its stored expectation; there is no receiver-emulation layer. Receiver state remains in the test harness.
  • Encode from semantic fields, and verify semantic round trips and canonical byte stability without replaying original wire evidence.

Envelope policies

All six envelope entry points now accept policy= instead of profile=: parsing and serialization for packets and TNC2 lines, check_packet, and validate_packet. Public exports include the immutable EnvelopePolicy, RF, and APRS_IS; EnvelopeProfile is removed.

  • Independent source, destination, and path-address regular expressions use fullmatch; path addresses exclude their repeated marker.
  • Optional limits cover path entries, information bytes, and complete TNC2 line bytes including CRLF. None means unlimited; zero is a real limit.
  • RF uses AX.25 address syntax, eight path entries, and 256 information bytes. APRS_IS is the default. It enforces internet address formats, including that SSID -0 is never sent, and a 512-byte line limit.
  • Parsing keeps decoded fields and adds diagnostics. Validation and serialization recompute conformance from current fields.
  • Policies constrain the outer envelope consistently across parsing and serialization. Third-party headers remain encapsulated text; ordinary syntax and payload checks still apply to them.
  • Invalid policy types, patterns, and limit values fail explicitly. No named-policy branches are needed for custom formats.

The new default is stricter than the former monitor profile. An explicit permissive policy, EnvelopePolicy(".*", ".*", ".*"), preserves representational round trips for deliberately invalid addresses.

  • Round trips: 1,305 corpus round trips succeed under that policy. Of those, 1,290 also pass default-policy validation, and the other 15 must be refused for their addresses under APRS_IS.
  • Refusals: 259 representation or payload refusals are checked with their exact reasons: 176 malformed packets and 83 payload-conformance violations.
  • NMEA checksums: three intentionally bad checksums are refused until the caller repairs them explicitly, by clearing checksum_valid. Sending never re-signs a sentence known to be corrupt.

Parsing and composing

Embedded ! positions are opt-in.

  • Background: APRS101 p.18 (X1J) allowed a ! position within the first 40 bytes of other text, and the APRS 1.1 addendum deprecates that search.
  • New parameter: parse_packet, parse_payload and parse_tnc2_line take legacy_positions: bool = False. By default such text is a Beacon; True returns a LegacyPosition.
  • Corpus: historical-1.0 and FAP rows parse with legacy_positions=True.

Mic-E

  • Type codes: status text follows the APRS 1.2 type codes (mic-e-types.txt). device_code holds the type byte plus any declared suffix: `_% (Yaesu FTM-400DR), '|3 (Byonics TinyTrack3), ]= (TM-D710), >^ (TH-D74).
  • Legacy hex telemetry: APRS101's hex telemetry is read only in its complete form, when no declared suffix ends the text. No type byte or suffix makes a packet Malformed.
  • Checks before sending: check_packet reports an undeclared type code, a type code combined with telemetry, and a comment that would be read back as a different type code.
  • Longitude: 100–109° are encoded as l–u, following APRS101's table, never as control bytes.

Weather

  • Comment budget: structured weather data is exempt from the 43-byte position-comment budget. The uncompressed wind slot still counts, and envelope limits still apply.
  • Software/unit codes are read only when they end the report. Sending refuses a comment after codes, and a comment that would be read back as codes.
  • APRS 1.2 fields (weather-new.txt): Weather.radiation (Xnnn, nSv/h) and Weather.battery_voltage (Vnnn, volts), with ... meaning not available. These are no longer misread as software/unit codes.
  • Water level: a decimal F20.1 keeps the APRS 1.1 meaning (feet). Four bytes without a decimal point, such as F0123, are the 1.2 proposal's tenths of a foot. Unencodable radiation or battery values are reported before sending.

Telemetry

  • Short reports: classic telemetry with missing channels decodes as a nonconforming Packet holding only the channels sent, so sending refuses it. Eight binary digits at the end are read as the digital bits.
  • Plain decimals: channel values must be plain decimals. inf, nan, 1e5 and -1. are Malformed.
  • Encoding value removed: the unverified "compatibility" telemetry encoding is gone.

Other

  • Query responses: text that only looks like a response (for example Directs= are fine…) stays a Message instead of becoming Malformed.
  • Object recovery: Malformed.recovered now covers objects with an invalid timestamp, as it already did for positions and status reports.

PHG/PHGR

  • Rate alphabet: parse the documented PHGabcdr/ rate alphabet (0–9, A–Z) and consume its required separator. A digit or letter without the slash remains comment text. Encode the rate and separator before other extensions.
  • Comment delimiters: handle ordinary PHG comment delimiters and surrounding ASCII whitespace in the public decoder. Preserve slashes in messages and non-PHG comments, and protect literal comment prefixes during encoding.
  • Corpus: all 29 PHG/PHGR projections, including 15 FAP vectors, go through the public API. FAP-415 and FAP-466 round-trip without refusal records.

Stored vector schema and FAP divergences

The finalized public schema is stored in the JSONL expectations themselves. The offline migration rewrites expectations for equivalent weather units, compressed wind nesting, legacy-position and vendor-weather wrappers, typed NMEA observations, and Mic-E device codes. It also maps FAP's radiation, battery and water-level readings into the new weather fields.

  • Inputs: every raw TNC2 string and original source snapshot is preserved. Only captured assertions are converted; generation never consults decoder output.
  • Divergences: tests/fap_divergences.json records the 52 FAP rows where FAP's interpretation differs from the specification-following public decoder, each with its reasons. Examples: comment whitespace kept as sent, disputed wind units, NMEA default symbols and precision, Mic-E type codes, invalid timestamps and incomplete GLL or telemetry that FAP accepts, and packets FAP refuses but the spec allows. The generator applies them and records the reasons in each row's divergence field; outcome and source_expected stay as FAP captured them.
  • Not asserted: FAP's own extra output fields (display resolution, raw timestamps, warnings, indoor sensors) remain provenance in source_expected, without assertions. PHG rows still check theirs from public models.
  • Verification: tests check inverse unit conversions, source-value preservation, Mic-E comment reconstruction, and deterministic regeneration, and confirm that mutating a stored assertion makes a test fail.

Validation

On Python 3.11, 3.12 and 3.15:

  • 3,543 tests passed, 883 subtests passed, with no skips or expected failures.
  • All 1,564 vector occurrences compare public parser output directly. There are 1,305 semantic round trips under the explicit permissive envelope policy, and 259 checked refusals.
  • Ruff, mypy, the repository-pinned Black and Prettier, and the vector regeneration check all pass.

See the envelope policy documentation for migration and customization, and the compatibility-adapter removal comment for details of the latest round.

🤖 Generated with Claude Code

@afourney afourney left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of #5 at 977a523

This is substantial, careful work. On the core profiles the public decoder is close to the original corpus. I checked it independently, and the problems I found are mostly outside what the corpus covers: everyday Mic-E traffic, re-encoding, and a few send-time rules. Two findings are P1. The test harness also hides two divergences from the source vectors.

All reproductions use from hamkit import aprs as a on Python 3.12.

How I checked

  • Provenance. Every row's packet, source_expected, title, profile, source, outcome, note, scope_note, extension_tags and options is byte-identical to the base (b97beb7). Only expected (105 rows), unmapped, gaps and sequence state changed. I reviewed all 105 expected changes; apart from the items below they are schema mappings consistent with their sources.
  • Public decoder vs. original assertions. I wrote a checker that does not use _compatibility.view() or the stored projections. It parses each non-FAP row with a.parse_packet (RF for strict-envelope/ax25, otherwise APRS_IS) and compares the result directly with source_expected. About 11,200 comparisons run, covering header, position, ambiguity cells, symbols, comments, timestamps, motion, PHG/RNG/DFS/DF bearing, altitude, compression csT, Mic-E message/fix, objects/items, status grid/beam, messages, telemetry, weather (in wire units), storms and areas. Everything matches except OTH-033/035 (finding 3).
  • Re-encoding. Of 890 accepted non-FAP vectors re-serialized under a permissive policy, 811 reproduce the original bytes exactly and 6 are refused (the deliberately invalid BAD-033…040 extension text). I reviewed all 73 differences. Most are legitimate normalizations: placeholders, ignored-csT padding, /A= placement, latitude-imposed ambiguity, beta DTIs. The exceptions are findings 4, 10 and 11.
  • I spot-checked base91 comment telemetry against the specification's |ss11| example; it is correct.

P1

1. Mic-E manufacturer prefixes make ordinary Yaesu and Byonics packets Malformed.
_mic_e.read treats a leading ` or ' in the status text as APRS101's original hex telemetry (_mic_e.py:111). APRS 1.2 deprecates that format ("Original Mic-E Telemetry Format is depricated. Overtaken by MFR TYPE codes"). Today those characters mainly mark the manufacturer: Yaesu sends `…_" or `…_% and Byonics sends '…|3. As a result:

base = b"N0CALL>S32UVT:`(_f\x1e\x1eOj/"
a.parse_packet(base + b'`"4T}Hi_%')  # Yaesu with altitude -> Malformed: invalid Mic-E hex telemetry
a.parse_packet(base + b"'Hello|3")   # Byonics TinyTrack3   -> Malformed
a.parse_packet(base + b"'1020 x")    # 4 hex after ' (FAP-348) -> Malformed
  • When the comment happens to start with hex digits, it is silently read as telemetry instead.
  • FAP-430 (a real Byonics packet) is Malformed from the public parser and only passes through the compatibility adapter.
  • The encoder rejects any device marker other than ]/> (_mic_e.py:208), so these packets can't be composed either.
  • The corpus has no Yaesu or Byonics core vectors, so nothing caught this.

Suggestion: recognize the APRS 1.2 prefix/suffix type codes, interpret hex telemetry only when the complete legacy form is present, and never let an unrecognized status prefix make the packet Malformed; fall back to comment text. Please add core vectors for Yaesu, Byonics and other common markers.

2. A corrupted NMEA sentence is re-sent with a valid checksum.
A sentence received with a bad checksum decodes as checksum_valid=False, position_usable=False with a "checksum mismatch" diagnostic. Yet check_packet returns (), and serialize_packet recomputes the checksum:

p = a.parse_packet(b"N0CALL>APRS:$GPRMC,180000,A,3350.000,S,15112.000,E,0.0,0.0,151026,,*00")
a.check_packet(p)      # ()
a.serialize_packet(p)  # b'...$GPRMC,180000,A,3350,S,15112,E,0,0,151026,,*0B'

Downstream receivers will trust data this library knew was corrupt. The PR description mentions this ("three intentionally bad NMEA checksums are repaired during encoding"), but it should not be the default. Suggestion: make checksum_valid is False a conformance finding, so sending needs an explicit repair, for example clearing the flag.

P2

3. The core profile searches for an embedded ! (X1J), and the harness hides it.
OTH-033/035 (core-1.1, outcome opaque, kind non-aprs-beacon) decode as LegacyPosition from the public parser. The corpus records that the APRS 1.1 addendum deprecated this search in 2012; only the historical profile should apply it. _compatibility.view() then swaps the payload for a Beacon whenever the profile isn't historical (_compatibility.py, the LegacyPosition branch), so the test passes while the public parser disagrees with the vector. Suggestion: don't search in the public parser (or make it an explicit option), and remove the rewrite so the test compares real output.

4. Mic-E longitude 100–109° is encoded with control characters.
MIC-116…125 are the corpus's ordinary vectors for these degrees, using l…u (bytes 108–117) as in APRS101's longitude table. The encoder emits bytes 28–37 instead, the FS…US control characters followed by !…%:

MIC-116  b'`l_f\x1e\x1eOj/'  ->  b'`\x1c_f\x1e\x1eOj/'

This decodes arithmetically, but it isn't the specified form and puts control bytes on the air unnecessarily. Please emit l–u for 100–109°.

5. Standard weather reports are refused for exceeding the 43-byte comment budget.
Positioned weather is now a structured extension, so _conformance._position counts it against "extensions and comment exceed 43 bytes". An ordinary report is refused:

p = a.parse_packet(b"N0CALL>APRS:@092345z4903.50N/07201.75W_220/004g005t077r000p000P000h50b09900L123wRSW")
a.check_packet(p)  # ['extensions and comment exceed 43 bytes'] -> cannot be sent

APRS101's weather chapter doesn't apply the 43-character limit to weather data, and the APRS 1.1 fields (L/l, s) push ordinary reports past it. Suggestion: exclude weather data from the comment budget, or cite the rule if the strictness is intended.

6. check_packet doesn't predict encode-time errors, and Mic-E needs a hand-computed destination.
check_packet promises "every conformance finding serialize_packet would raise". Several encoders raise FieldError only during serialization: Mic-E destination mismatch, unsupported Mic-E device markers, and Mic-E altitude reference. Composing a Mic-E packet also requires the caller to supply a destination that already encodes latitude, hemisphere, ambiguity and message, and no public helper derives it:

m = a.MicE(33.4, -112.1, a.Symbol("/", ">"), message="en_route",
           extensions=a.Extensions(motion=a.Motion(90, 10)))
p = a.Packet(a.Header("N0CALL", "S32UVT"), m)
a.check_packet(p)      # ()
a.serialize_packet(p)  # FieldError: Mic-E destination does not match latitude, ...

Suggestion: derive the destination in serialize_packet (or expose a helper), and report encode-time problems through check_packet.

7. Ordinary messages that start like a query response become Malformed.
QRY-015…017 are classified by their sources as ordinary messages (kind: message). The PR now recognizes response grammars in message text. That is reasonable as an addition, but when the grammar doesn't fit the parser fails instead of falling back:

a.parse_packet(b"N0CALL>APRS::W1AW     :Directs= are fine, see you at 7")
# Malformed: requires a station token of 1..9 characters

The same response also decodes as Response without a message ID but as Message with one, so consumers must handle both types. Suggestion: fall back to Message whenever the response grammar doesn't match exactly.

8. APRS_IS accepts SSID -0.
Both cited pages say so: "SSID -0 is dropped as conflicts with not present SSID" (ServerDesign) and "never explicitly define -0 as a SSID" (Connecting). _APRS_IS_SOURCE/_APRS_IS_ADDRESS accept N0CALL-0. Please exclude -0 from the APRS-IS patterns. The RF/AX.25 policy is unaffected.

9. The Mic-E grid is stored inside device_code.
MIC-265's source asserts type_marker: ">", grid_locator: IO91SX and grid_symbol: /G as separate facts. The public output is device_code=">IO91SX/G", and the test passes only because device_code isn't in that row's projection. A device marker field shouldn't carry a locator and symbol. Suggestion: add a Mic-E grid field (with its symbol), or leave the grid in the comment.

P3

10. The NMEA encoder rewrites numeric fields. It strips decimal points from coordinates (3350.000 becomes 3350, 01131.000 becomes 01131), drops leading zeros (satellites 08 becomes 8) and precision (47.0 becomes 47). The checksum is recomputed, so the sentence is valid, but it differs from what the GPS produced, and many NMEA consumers expect ddmm.mm with a decimal point. Consider keeping the received field text (fields_raw exists) or fixed NMEA formats.

11. Other canonical-form changes worth confirming against the spec:

  • PARM/UNIT are padded to 13 fields (PARM.Temp,,,,,,,,,,,,), which wastes message bytes and can push a valid definition past 67 bytes.
  • Query footprint radius 0200 becomes 200.
  • ?APRSH N0CALL loses the callsign padding.

12. 000000z turns the whole packet Malformed. TIM-011 allows this ("retaining other fields with a warning is allowed"). But FAP-454…456 show it is common in real traffic, the compatibility view rescues those packets for FAP, and the position is otherwise intact. Consider returning a Packet without the timestamp, plus a diagnostic.

13. Question: telemetry decimals. T#005,1.5,… is decoded but marked nonconforming ("requires an integer in 0..255"), and FAP-432/433 accept decimals. If the intended baseline is APRS101 only, that's fine; if APRS 1.2 telemetry is in scope, please confirm.

Test architecture

  • Most rows don't compare parse_packet directly. Only PHG rows and the 19 SCHEMA_NATIVE_IDS do; every other row goes through _compatibility.view(). For core profiles that is nearly a passthrough: it selects RF for strict-envelope rows, plus the rewrite in finding 3. For FAP it is effectively a second decoder: it strips comments, uppercases calls, re-parses weather tails, rounds course, maps 0° to 360°, decodes relaxed telemetry and Mic-E, and replaces the rejection decision with _receiver_rejects. The description qualifies this, but "all 1,561 vectors pass" shouldn't be read as coverage of the public parser for FAP rows.
  • Test-only code ships in the package. _compatibility.py (719 lines) is only used by tests but lives in src/ and ships in the wheel. Consider moving it under tests/.
  • Hard-coded IDs. SCHEMA_NATIVE_IDS is a hand-maintained list. Selecting rows by a property of the row would be more robust.

Minor

  • The Position docstring in _models.py still says recognized-but-undecodable extensions make the payload Opaque, but DAO and base91 telemetry are now decoded.

Verified correct (no action needed)

Compressed positions and csT, ambiguity cells including latitude-imposed ambiguity, Mic-E message bits including custom/unknown, standard and alternate speed/course, area offsets and corridors, storm fields, status beam (^B7 gives 110° and 490 W), base91 comment telemetry, reply-ack, and the 000/000 disputed-zero policy (WX-060).

@afourney

Copy link
Copy Markdown
Owner Author

Removing _compatibility.py: every vector now tests the public parser

This covers commits 038603b, d8cd5ea and e2c709b. Before them, about 1,500 of the 1,564 corpus rows went through _compatibility.view(), not straight through parse_packet.

  • FAP rows: the adapter was a second decoder. It trimmed comments, uppercased calls, re-decoded Mic-E and telemetry, re-parsed weather tails, rescued 000000z packets, and replaced the library's decision to reject a packet with its own copy of FAP's rules.
  • Core rows: it also hid a real bug, by rewriting the result for one pair of rows (OTH-033/035).

So "all vectors pass" partly measured the adapter rather than the library.

Now every row is parsed with parse_packet and compared with its stored expectation. The adapter is deleted. The only thing that differs by profile is the legacy_positions option, described below.

Where we deliberately differ from FAP

The FAP rows record what the FAP library reported. Where FAP's interpretation differs from ours and ours follows the spec, the difference is recorded in a new file, tests/fap_divergences.json, rather than recreated at test time.

  • The file: 52 rows, each naming its reasons.
  • The generator: applies these overrides and writes the reasons into the row's new divergence field.
  • What stays as captured: the row's outcome and FAP's own values in source_expected, so the frozen provenance digest is unchanged.
Reason Rows
Comment whitespace kept as sent (FAP trims it) 22
Uncompressed wind unit kept as disputed_knots_or_mph (FAP assumes mph) 4
NMEA default symbol from the source SSID, APRS101 (-11 is the balloon, /O) 3
000000z timestamp is Malformed, with fields kept in Malformed.recovered (TIM-011 allows either) 3
Mic-E type codes, APRS 1.2: ]= (TM-D710) suffix; ' with incomplete hex is a type byte; Byonics '|3 4
Callsign case kept as sent 2
Compressed course byte 0 kept as 0° (FAP rewrites it to 360) 2
Text with no position in the first 40 bytes is a non-APRS beacon (FAP rejects it) 2
O is not a weather field in any spec, so the rest stays in the comment 2
FAP rejects, the spec allows: {{ user-defined data, unknown $ sentence (kept as opaque), $GPVTG 3
Corrupted Mic-E packet is Malformed (FAP's accept_broken_mice guesses a repair) 1
Mic-E hex telemetry has no sequence number (FAP reports 0) 1
NMEA course keeps its precision, 27.9 (FAP rounds to 28) 1
GLL with no time or fix status is Malformed; nothing is invented 1
Telemetry: comment kept / short report keeps the channels sent / non-numeric channel is Malformed 3
Lowercase parm. is an ordinary message (APRS101 keywords are upper case) 1
Weather symbol with no weather fields is a position with a comment 1

Some rows have more than one reason, so the column adds up to more than 52.

No longer asserted: FAP's own extra fields (display resolution, raw timestamps, warnings, indoor sensors) for about 430 rows. They describe FAP's output, not our schema, and stay in source_expected as provenance. The 15 FAP PHG rows still check theirs from public models.

Parser and API changes

1. The embedded ! search is now opt-in (new parameter)

  • APRS101 p.18 (X1J) allowed a ! position within the first 40 bytes of other text, and the APRS 1.1 addendum deprecated that search. Until now the parser always did it, and the adapter hid that from the core vectors OTH-033/035.
  • New: parse_packet, parse_payload and parse_tnc2_line take legacy_positions: bool = False. By default such text is a Beacon; True restores the LegacyPosition search. A non-bool value raises TypeError.
  • Corpus: historical-1.0 and FAP rows parse with legacy_positions=True, since those receivers do this search.

2. Mic-E type codes follow APRS 1.2 (fixes review finding #1)

  • The rules come from aprs.org/aprs12/mic-e-types.txt. In 2008 it redefined a leading ` or ' as a device type byte, replacing APRS101's hex telemetry. A declared suffix at the end of the status text names the model.
  • device_code now holds the type byte plus any declared suffix, for example `_% (Yaesu FTM-400DR), '|3 (Byonics TinyTrack3), ]= (Kenwood TM-D710) or >^ (TH-D74). The comment is the text between them, and the encoder writes the suffix back at the end.
  • Legacy hex telemetry is read only in its complete APRS101 form (4 hex digits after `, 10 after ') and only when no declared suffix ends the text. The core vectors MIC-261/262 are unchanged.
  • No more Malformed from markers: ordinary Yaesu and Byonics packets used to be Malformed ("invalid Mic-E hex telemetry"). No type byte or suffix can do that now.
  • Only declared suffixes are recognized. The "awaiting confirmation" entries are left out, so ordinary text such as "text-3" isn't stripped. A leading space, also listed as a type byte, stays comment text.
  • Sending: check_packet now reports what used to fail only inside the encoder:
    • an undeclared type code;
    • a type code combined with hex or binary telemetry, since both use the first status byte;
    • a comment that would be read back as a different type code, for example "Hi_%" or a comment starting with ]. This uses a new encode-and-read-back check of the status text.
  • Encoder byte order: hex or binary telemetry is now written immediately after the symbol, then altitude, then the grid, matching the decoder.

3. Short telemetry is a nonconforming packet, not Malformed

  • Missing channels: T#001,42 and T#1,1,,3,,5 decode with only the channels sent. They are marked nonconforming against APRS101, so sending is refused. This follows the parser's own rule: a Packet whenever every field decodes, with departures from APRS rules reported as nonconforming.
  • Digital bits: when channels are missing, eight binary digits at the end are read as the digital bits. They can never be a valid APRS101 analog value (0..255). Before, they were silently read as a fifth analog value of 0 (BAD-102).
  • Stricter numbers: channel values must be plain decimals. The decoder used float(), which accepted inf, nan, 1e5 and -1.; those are now Malformed ("invalid telemetry channel").
  • Encoder: it writes only the parts present, so a short report round-trips through serialize_payload internals without gaining a trailing comma.

4. APRS 1.2 weather fields (schema addition)

  • These come from aprs.org/aprs12/weather-new.txt.
  • New Weather fields:
    • radiation, in the new unit nSv/h. Xnnn is two digits times a power of ten: X123 is 12 × 10³ nSv/h.
    • battery_voltage, in the new unit volts. Vnnn is tenths of a volt: V128 is 12.8 V.
    • ... means "not available" for both.
  • Bug fixed: X123 and V128 used to be read as weather software/unit codes (software X, unit 123). The adapter hid this by clearing the codes. Real codes after these fields, as in X123V128XRSW, still decode.
  • Water level, two spellings told apart by syntax:
    • A decimal value (F20.1, f-1.2) is the APRS 1.1 note's feet or meters, unchanged (core vectors WX-063 to 066).
    • Four bytes without a decimal point (F0123, F-042, F....) are the 1.2 proposal's tenths of a foot. F0123 used to decode as 123 ft; it is now 12.3 ft.
    • On re-send, a value is written in the decimal form (F12.3). An unavailable feet reading is written as F.....
  • Sending: check_packet reports radiation that can't be written as two digits times a power of ten, and battery voltage too large for its three-digit field. For positionless reports, it also encodes and reads back the report, and reports a comment that would be read back as weather fields (for example "X123 hot").
  • FAP rows: the generator now maps FAP's Radiation, BatteryVoltage and WaterLevel into these fields. FAP reports water level in meters, which is converted to feet.

5. Malformed.recovered now covers objects

  • An object whose timestamp is invalid (for example ;TestObj *000000z…) now recovers its name, alive flag, timestamp fields, position, symbol and comment.
  • Recovery only runs when bytes 10–16 look like a timestamp, so an object with no timestamp (BAD-085) isn't misread.

6. Removed: the "compatibility" telemetry encoding

  • The adapter was the only thing that produced this value, and no vector expected it. It is removed from TelemetryEncoding, the message encoder, the conformance rules and the docs.

Tests

  • Corpus test (tests/packets/test_corpus.py): one code path, parse then compare. It uses RF for strict-envelope and AX.25 rows, APRS-IS otherwise, and legacy_positions for historical and FAP rows. SCHEMA_NATIVE_IDS and the temporary pending list are gone.
  • Round-trip test: parses with the same legacy_positions setting. Its exceptions were updated with the exact current reasons. FAP-348 and FAP-430 now round-trip and were removed.
  • Schema-migration tests: these asserted FAP's values (mph, no NMEA symbol, course 28). They now accept a recorded divergence, and assert the newly mapped radiation, battery and water-level values.
  • Contract tests: they patch the public parse_packet instead of the adapter.
  • New regression tests:
    • Mic-E type codes (9 cases) and the checks before sending them;
    • the legacy_positions option, including the boundary at byte 40;
    • the APRS 1.2 weather fields (10 cases) and the checks before sending them;
    • short telemetry, and plain-decimal channel values;
    • object recovery after an invalid timestamp.

Counts

Before Now
Rows compared through the adapter ~1,500 0
Successful round trips (permissive envelope policy) 1,303 1,305
Refusals with recorded reasons (malformed / conformance) 261 (181 / 80) 259 (176 / 83)
Valid under the default APRS_IS policy 1,288 1,290

3,543 tests pass on Python 3.11, 3.12 and 3.15. ruff, mypy, black and prettier are clean, and generate_vectors.py reproduces the JSONL exactly.

Public API changes in this round

  • parse_packet, parse_payload and parse_tnc2_line take legacy_positions=False. Embedded ! positions are no longer found by default.
  • Mic-E device_code is now the APRS 1.2 type code (type byte plus declared suffix). ` and ' markers no longer produce Malformed.
  • Weather.radiation and Weather.battery_voltage are new, with the new units nSv/h and volts.
  • TelemetryEncoding no longer includes "compatibility".
  • Classic telemetry with missing channels is a nonconforming Packet. Non-decimal channel values (inf, nan, 1e5, -1.) are Malformed.
  • F water level with four bytes and no decimal point is read in tenths of a foot.

Still open

  • Mic-E grid (review finding #9): the grid still lives in device_code, for example >IO91SX/G.
  • Encode-time checks (finding Move APRS tests to JSONL case vectors, compared against FAP and live traffic #6): composing Mic-E still requires a destination computed by hand.
  • Corpus vectors: there are no core vectors for Yaesu, Byonics or Kenwood-suffix Mic-E, or for the APRS 1.2 weather fields. They could be added as amended rows, as with WX-067 to 069.
  • PR description: it still cites the old counts.

@afourney
afourney merged commit 9141a9d into scaffold/aprs-vectors Oct 17, 2026
3 checks passed
@afourney
afourney deleted the implement/aprs-codecs branch October 19, 2026 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants