feat(dr-egeria): Reference Data authoring + Valid Metadata family; fix ISSUE-142

[dwolfson]

Summary

Adds authoring of business value lists to Dr.Egeria's Reference Data family, and splits the type-system commands out into a new Valid Metadata family. Also fixes ISSUE-142.

Two terms used in this PR:

• Reference data: business value lists, such as order statuses or country codes, stored as ReferenceDataSet / ReferenceDataValue.
• Valid metadata values: the allowed values of Egeria's own properties, e.g. fileType (valid_metadata.py).

Reference Data: new commands

• Create Reference Data Set. A set's members come from three attributes:

  • Members: values that already exist, kept in sync on each run.
  • Values: created inline, one per line, with commas kept as part of the value.
  • Value Table: a markdown table with Value, Display Name, Description, Usage, Data Type, Is Case Sensitive, Default and Ordinal columns.

  Inline values are named <set qualified name>::<value>, so re-running a document updates them in place. A member no longer listed is unlinked, never deleted.

• Create Reference Data Value, optionally joining a set (In Reference Data Set, Is Default Member, Member Ordinal).

• Link/Detach for Valid Value Member, Consistent, Associated and Mapped Valid Values, and Valid Value Implementation. One table in the code (VALID_VALUE_LINKS) drives all five, and the processor branches on the verb.

• Relationship properties the existing commands used to drop: Link Element to Valid Values now sends Strict Requirement; Link Reference Value Assignment now sends Attribute Name, Confidence, Steward and Notes.

Valid Metadata: new family

• 9 commands move here unchanged: Setup/Clear Valid Metadata Value, Map Name and Map Value; Set Consistent Metadata Values; Link/Detach Specification Property Assignment. Existing documents keep working.
• Setup Valid Metadata Value accepts a Metadata Value Table (Preferred Value, Display Name, Description).

Parser and specs

• Two new attribute styles: Table (each row becomes a dict keyed by the header) and Line List (split on new lines only). Both are also in the Spec Editor's style list.
• All spec edits went through the Spec Editor API. Shared attribute and bundle names were copied from the definitions currently in effect, not from another family's local copy.
• I compared every expanded command spec before and after: the only changes are the family label on the 9 moved commands, the 12 new commands, and the 5 commands meant to change.
• refresh_specs added 4 report specs and changed nothing else. Only Reference Data and Valid Metadata templates changed.
• Preferred Value is no longer marked required; the processors check for it where it's needed.

Design notes

• Only the ReferenceDataSet / ReferenceDataValue subtypes are created. The Reference Data view's finds default to type ReferenceDataValue (ReferenceDataInstance.java:65-69), so a plain ValidValueDefinition is invisible to them.
• No Delete commands, matching every other Dr.Egeria family.

ISSUE-142

Any Egeria-wrapped relatedHTTPCode 401/403 used to be raised as "User not authorized … for user - ``". Egeria also uses 403 for errors that aren't about authorization, e.g. OMAG-GENERIC-HANDLERS-403-005 when deleting an element that still has dependents.

Now an error is treated as an authorization failure only for UserNotAuthorizedException, or a 401 with no exception class. Any other 403 keeps Egeria's own message, and the authorization message now names the user.

Test plan

• pytest tests/micro-tests: 1090 passed, 1 skipped.
  • test_reference_data_family.py: 16 new tests covering parsing, specs, set and member sync, the links and valid metadata.
  • test_issue_142_403_not_always_unauthorized.py: 3 new tests.
• validate_compact_specs: 0 errors. The 28 warnings were there before.
• Dr.Egeria --validate on the new regression doc tests/dr-egeria-command-tests/dr_test_reference_data.md: all commands pass.
• Live --process on qs-view-server, under a throwaway prefix:
  • Sets, inline and table values, member ordinals and defaults, and the mapping and association properties all checked by a different route. The Detach worked.
  • The same document processed a second time: every Create became an Update, with the same 8 GUIDs and identical members and links.
  • Metadata Value Table set up 2 values, and re-running it stayed at 2.
  • Everything deleted by GUID afterwards and confirmed gone.

Follow-ups

• Refresh the template copies in trellis and egeria-workspaces-fs after merge.
• Clearing valid metadata values returned an error on the first call, although the values were gone afterwards. Setup also leaves a per-property container element that clearing doesn't remove. Not investigated yet.

🤖 Generated with Claude Code