[bruno-f-cruz]

Summary

Replaces harp-python with harp-data (the harp.data namespace). Harp data is now read through the package's public API (harp.device.schema.create_device_module + harp.data.open_dataset) instead of harp-python private internals (harp.reader._ReaderParams, harp.reader._create_register_handler).

• contract/harp.py: rewritten on top of harp.data. The only custom logic left is locating the device.yml (WhoAmI registry, URL, register 0), which the new package does not provide, and schema metadata the device module does not carry (register descriptions, firmwareVersion).
• device.yml caching: a schema fetched by WhoAmI or URL is now downloaded once per process instead of once per device load. Failed fetches are not cached.
• QC suites (qc/harp/*): updated to the new DataFrame layout. Bit-mask checks use bitwise ops on the integer value column, with flags taken from the package (harp.device.core.ClockConfigurationFlags, the generated module's LickChannels).
• Bug fixes: the firmware/core version tests no longer crash (TypeError/AttributeError) when a version register has no Read message or the schema has no firmwareVersion. They fail the test instead.
• Typing: DataStreamCollectionBase.__getitem__ now returns the collection's item type, so device["X"] is typed as HarpRegister. Runtime behavior is unchanged.
• Dependencies: harp-python → harp-data>=0.5.0. aind_behavior_services>=0.13.9 is the first release that no longer depends on harp-python, so harp-python is no longer installed.
• Tests: new tests/test_contract/test_harp.py writes real Harp .bin files and loads them through HarpDevice. Previously all harp tests used mocks, so a change in the data format would not have been caught.

Breaking changes for downstream code

1. DataFrame layout of every register

	Before (harp-python)	After (harp-data)
Message type column	MessageType, last column	message_type, first column
Message type values	"READ", "WRITE", "EVENT"	"Read", "Write", "Event" (pd.Categorical)
Single-value register	column named after the register (df["WhoAmI"], df.RawVoltage)	df["value"]
Register with payloadSpec	PascalCase members (Temperature, Encoder)	snake_case members (temperature, encoder, torque_load_current)
Bit mask register	always expanded, one bool column per flag (df["Channel0"])	not expanded by default: one integer value column
Group mask register	str values	pd.Categorical values (string comparisons still work)
Time index	Time	Time (unchanged)

Warning

Positional access fails silently. Because message_type moved from last to first, code like df.iloc[-1].values[0] or row.iloc[0] now returns "Read"/"Event" instead of the register value. It raises no error and returns the wrong value. Always select by column name (df["value"].iloc[-1]).

2. API changes in contraqctor.contract.harp

Before	After
HarpDevice.device_reader (harp.reader.DeviceReader)	removed. Use HarpDevice.dataset_reader (harp.data.DatasetReader) and HarpDevice.device_schema (parsed device.yml)
device.device_reader.device.whoAmI / .firmwareVersion	device.device_schema.whoAmI / .firmwareVersion
device.device_reader.registers[...]	device.dataset_reader.device_module.REGISTER_MAP (by address), or register classes by name on device.dataset_reader.device_module
HarpRegisterParams (alias of harp.reader._ReaderParams, fields base_path, epoch, keep_type)	contraqctor's own dataclass with fields keep_type, decode_enums, demux_bit_masks. The time reference (epoch) is set once per device on HarpDeviceParams, not per register
HarpRegister.from_register_reader(name, reg_reader, params)	HarpRegister.from_dataset_reader(name, dataset_reader, register_class, *, description=None, params=...)
n/a	HarpRegister.register_class: the harp.protocol register class (.address, .payload_type, ...)
HarpDeviceParams(show_private=...)	removed (passing it raises TypeError). Private registers are never exposed
n/a	HarpDeviceParams(decode_enums=True, demux_bit_masks=False)
import harp, harp.read, harp.read_schema, harp.reader.*	no longer installed. Use harp.data.read, harp.data.open_dataset, harp.device.schema.parse_device_schema / create_device_module

3. Behavior changes

• WhoAmI mismatch fails the load. harp.data.DatasetReader compares the schema's whoAmI against the device.yml in the data folder. A mismatch (e.g. a DeviceYmlByWhoAmI hint pointing at the wrong device) now raises ValueError: WhoAmI mismatch when the device loads. Previously this was only reported by the QC check.
• Registers with a custom interfaceType are decoded as their native numeric type (require_converters=False), matching what harp-python effectively did.
• Register order: registers are listed in address order instead of YAML order.
• Missing register files still fail to load with FileNotFoundError, as before. Plain harp.data would return an empty DataFrame instead; contraqctor keeps the error so has_data and the read-dump QC keep their meaning.
• device.yml cache: a schema fetched from a repository's main branch is not re-fetched until the Python process restarts.
• QC suites need unexpanded bit masks: HarpHubTestSuite (clock generator checks) and HarpLicketySplitTestSuite read the integer value column. Loading those devices with demux_bit_masks=True makes those checks raise KeyError: 'value'.

Fix reference

# Message type filter
df[df["MessageType"] == "EVENT"]              # before
df[df["message_type"] == "Event"]             # after

# Single-value register
device["WhoAmI"].data.WhoAmI.iloc[-1]         # before
device["WhoAmI"].data["value"].iloc[-1]       # after

# Positional access (silently wrong after this PR)
device["SensorDataDispatchRate"].data.iloc[-1].values[0]   # before
device["SensorDataDispatchRate"].data["value"].iloc[-1]    # after

# payloadSpec members are snake_case
device["SensorData"].data["Temperature"]      # before
device["SensorData"].data["temperature"]      # after

# Bit mask flags: use the flag from the device module (or harp.device.core for core registers)
device["LickState"].data["Channel0"]          # before
channel0 = int(device.dataset_reader.device_module.LickChannels.CHANNEL0)
(device["LickState"].data["value"] & channel0) != 0         # after (bool Series)
# int(...) is required: IntFlag members are iterable, and pandas rejects `series & flag`

from harp.device.core import ClockConfigurationFlags
row["ClockGenerator"]                         # before
row["value"] & ClockConfigurationFlags.CLOCK_GENERATOR       # after

# Or opt back in to per-flag columns (names are SCREAMING_CASE)
HarpDeviceParams(path=..., demux_bit_masks=True)
device["LickState"].data["CHANNEL0"]

# Schema metadata
device.device_reader.device.firmwareVersion   # before
device.device_schema.firmwareVersion          # after

Name conversion rules come from harp-device, and its runtime modules match the static generated device packages: payload fields → snake_case (DutyCycle → duty_cycle), enum/flag members → SCREAMING_CASE (DIPort0 → DI_PORT0). Register, enum and payload class names are kept verbatim from the YAML.

Known issues (pre-existing, not addressed here)

• HarpLicketySplitTestSuite.test_minimum_lick_rate and test_refractory_period_violations count lick offsets as licks: .where(lambda x: x == 1).index keeps every row, so the lick count and rate come out roughly doubled. test_lick_duration counts correctly (lick[lick == 1].index).
• HarpDeviceTestSuite.test_core_version calls self.fail_test("Core version is not a valid semver version."), which passes the message as the result value, so the failure has no message.

Testing

• pytest: 183 passed, including the new integration tests in tests/test_contract/test_harp.py.
• End-to-end checks against the real device.yml of the Lickety Split (WhoAmI 1400) and Environment Sensor (1405), with synthetic frames written to disk and the QC suites run on them.
• Not yet run against recorded session data: running examples/qc.py on a real dataset is recommended before release.