diff --git a/PYEGERIA_ISSUES.md b/PYEGERIA_ISSUES.md index a402d324..6db51209 100644 --- a/PYEGERIA_ISSUES.md +++ b/PYEGERIA_ISSUES.md @@ -128,105 +128,112 @@ Supply Chain`'s `Purposes`/`Scope` silently dropped — renumbered from a same-day `ISSUE-62` collision with the `DeleteElementRequestBody` entry above) after root-causing it live. See each entry for full detail. +**Reorganized 2026-10-12**: every entry's status line was re-read and each +entry moved to the section its status actually says. The open part now has +three subsections — **Open pyegeria items** first (actionable here), then +**Awaiting re-test** (an Egeria-side fix has merged, or the original cause +was fixed on our side, and what's left is a live re-check against a current +platform), then **Open Egeria Server issues** (still needing an Egeria +change). Within every section, entries are sorted newest-first by +`ISSUE-#`. Moved to the Appendix as closed: ISSUE-135, 133, 131, 126, 122, +121 and 110 (fixed in pyegeria) and ISSUE-102 (no longer reproduces on the +2026-10-05 platform). ISSUE-136, 134, 132 and 129 moved from the Egeria +Server section to the pyegeria one; ISSUE-124 and 125 the other way. A +2026-10-05 Egeria-team note about ISSUE-108, which had been stranded in a +section preamble, now sits at the end of ISSUE-108's own entry. No entry's +content was changed. The numbered section list above describes the +2026-08-15 layout; this note supersedes it. + --- # Open Issues -## Open Egeria Server issues (not fixable in pyegeria) +**At a glance** — 6 open pyegeria items, 7 awaiting re-test, 6 open Egeria Server issues. -These need an Egeria Server-side fix, an upstream bug report, or a -deployment/config change — nothing here can be resolved by editing this -repo. Where an entry has a known pyegeria/Dr.Egeria follow-on once the -Egeria side lands, it's noted inline (also cross-referenced from the -"Open pyegeria items" section below where the follow-on is substantial -enough to track there too). +**Open pyegeria items** +- ISSUE-140: The executor charts any dict with two or more top-level numbers, regardless of the result's state +- ISSUE-139: The executor renders no result envelope, and the Vega bar chart has no subtitle, so a not-established result has nowhere to say so on a chart +- ISSUE-138: Analytic params are JSON-only, and only parameters named `mgr`/`ce`/`expert`/`client` are bound as clients, so a routine taking a reader object can't be named by its own dotted path +- ISSUE-132: `DeleteElementRequestBody` serializes the cascade flag as `cascadeDelete`; Egeria's field is `cascadedDelete` and unknown fields are ignored, so no pyegeria cascade delete has ever cascaded +- ISSUE-127: `CollectionManager.delete_collection(cascade=True)` silently never sends `cascadeDelete` -- same shape as ISSUE-62 +- ISSUE-123: `overview_metrics.count_elements` (and `_find`/`_element_count`) return `0`/`[]` on any failure — a failed measurement is indistinguishable from a real zero + +**Awaiting re-test** +- ISSUE-125: `deleteEnumDef` answers 500 "unknown TypeDef" for an enum the server lists -- an enum added through the API cannot be removed +- ISSUE-124: `updateTypeDef` (Metadata Expert `open-metadata-types/update`) can never succeed -- the server builds the patch without `updatedBy`, which the repository services require +- ISSUE-117: Cascade delete that takes the soft-delete (Memento) path fails partway with `OMAG-REPOSITORY-HANDLER-400-010` unless `forLineage=true` — leaves the asset live and an anchored element already soft-deleted +- ISSUE-112: `AutomatedCurationRESTServices.saveClientSideSecret`/`deleteClientSideSecret` return a plain success `VoidResponse` when the resolved connector isn't a `YAMLSecretsFileConnector` — no error, no write, no indication anything was skipped +- ISSUE-90: [fixed?] `qs-engine-host` retries `startMissedEngineActions` forever when one incomplete engine action's anchor is unreadable by the engine-host user +- ISSUE-79: native survey against a template-created `FileFolder` asset fails server-side — `assetConnector` is null in `BasicFolderConnector.getFile()` +- ISSUE-38 (PY-18): `count_relationships_between_elements("Exception")` (276) disagrees with `ClassificationExplorer.get_relationships("Exception")` (55) + +**Open Egeria Server issues** +- ISSUE-130: The YAML secrets store's user directory caches a user forever, so editing an existing user needs a platform restart +- ISSUE-128: Retention cannot be set on Egeria 6.2: the server registers the Jackson subtype as `RetentionProperties.class` (typo), and pyegeria only allows `RetentionClassificationProperties` +- ISSUE-108: newly created relationships stay invisible to related-element queries for up to ~20 minutes after creation +- ISSUE-95: No catalog template registered for the "Apache Kafka Server" technology type — `Create Kafka Server Element` (Asset Maker) fails with a 400 on `qs-view-server` +- ISSUE-89: No configurable bearer-token lifetime in `application.properties` — platform tokens are fixed at 3600 s, and the signing key is random per restart +- ISSUE-85: No REST endpoint exists to create a `SolutionPort` element — only attach/detach/delegation relationships are exposed --- -### ISSUE-135: 14 `*_by_name` wrappers required `name` even though they accept `body`, so a body-only call raised `TypeError` +## Open pyegeria items (including follow-ons blocked on an Egeria fix) -**Layer:** pyegeria · **Status:** fixed in #445 (2026-10-10), pending merge · **Found:** -2026-10-10, reported by the Resource Explorer session (`ActorManager.get_actor_profiles_by_name` on 6.1.15; still -present in 6.2.1). +Actionable in this repo. Some of these are fully blocked today — waiting +on an Egeria Server capability that doesn't exist yet — but the pyegeria/ +Dr.Egeria-side work each will need once that capability ships is written +into the entry now, so it isn't rediscovered from scratch later. -**What:** 14 of the 47 `*_by_name` wrappers declared their first argument (`name` or `filter_string`) with no -default, while also accepting `body`. A body-only call such as `get_actor_profiles_by_name(body={...})` raised -`TypeError`, while siblings such as `get_user_identities_by_name` default `name` to `None`. Affected: -`actor_manager` (actor profiles, actor roles), `asset_maker` (assets, software capabilities), `connection_maker` -(connections, connector types, endpoints), `data_designer` (data fields, data value specifications), -`data_discovery` (analysis reports), `product_manager` (governance action processes), `schema_maker` (schema types, -schema attributes) and `solution_architect` (concept model elements). +--- -**Workaround (before the fix):** pass a placeholder name with the body, e.g. `get_actor_profiles_by_name("*", body=...)`. -An explicit body is sent as given, and the name and the wrapper's other arguments are ignored. +### ISSUE-140: The executor charts any dict with two or more top-level numbers, regardless of the result's state -**Fix:** the first argument defaults to `None` in both the async and sync versions. `None` already meant "match -everything", like `"*"`, so name-only and name+body calls send exactly what they did before. -`_async_get_data_value_specifications_by_name` also gets the `classification_names=None` default its sync version -already had. Tests: `tests/micro-tests/test_issue_135_by_name_body_only.py`. +**Layer:** pyegeria (format_set_executor) · **Status:** open, logged only (no fix without the owner's explicit go-ahead) · **Found:** 2026-10-12, reported by the Resource Explorer PR/CI session from trellis-analytics (trellis PR #622, `trellis_analytics/portal.py`), which works around it today. -**Behaviour to know (unchanged):** when a body is given, it is the whole request. The wrapper's default type filter -(e.g. `metadataElementTypeName: "ActorProfile"`) is not added to it, so a body without `metadataElementTypeName` -searches all types. +**What:** BAR/SERIES output is drawn whenever the result dict has two or +more numeric top-level keys, whether or not the result says it was +measured. A `not_established` result that happens to carry numbers would be +charted as real zeros/values. -**Also fixed in #445 (found while testing):** five `*_by_name` wrappers that already defaulted `name` still -mishandled a body-only call: -- `SolutionArchitect.get_info_supply_chain_by_name`, `get_solution_blueprints_by_name`, - `get_solution_components_by_name` and `get_solution_roles_by_name` did `body["filter"] = name`, which overwrote - the caller's filter with `None` on a body-only call and mutated the caller's dict. They now copy the body and set - `filter` only when a name is given (a given name still wins). They also sent paging as URL query parameters, which - Egeria ignores: in Egeria source (`SolutionArchitectResource.java`, `ActorManagerResource.java`) the only query - parameter on these endpoints is `addImplementation`, on information supply chains. Paging now goes in the body, - without overriding values the body sets. -- `AutomatedCuration.get_engine_actions_by_name` rejected a missing `name` (`VALIDATION_ERROR_1`) even with a body, - contrary to its docstring. It now requires a name only when no body is given. It also passed its type filter under - the old `metadata_element_type` key; it was a 14th ISSUE-131 case that the original grep missed (single quotes), - already covered by the 6.2.1 helper fix, and now uses the canonical key. +**Where seen:** trellis `portal.py` is safe only because its adapter omits +numbers for non-measured states. + +**Candidate fix:** honour an envelope state (see ISSUE-139): draw no chart +unless the state is `measured`, and show the state word instead. --- -### ISSUE-134: With `PYEGERIA_ENABLE_LOGGER_CATCH` on, `@dynamic_catch` turns every failure into a `None` return, indistinguishable from "nothing found" +### ISSUE-139: The executor renders no result envelope, and the Vega bar chart has no subtitle, so a not-established result has nowhere to say so on a chart -**Layer:** pyegeria · **Status:** open, not fixed · **Found:** 2026-10-10, reported by the Resource Explorer -session; confirmed from code. +**Layer:** pyegeria (format_set_executor output rendering) · **Status:** open, logged only (no fix without the owner's explicit go-ahead) · **Found:** 2026-10-12, reported by the Resource Explorer PR/CI session from trellis-analytics (trellis PR #622, `trellis_analytics/portal.py`), which works around it today. -**What:** `dynamic_catch` (`pyegeria/core/utils.py` ~439) returns `logger.catch(func)` when -`app_settings.Debug.enable_logger_catch` is true (`PYEGERIA_ENABLE_LOGGER_CATCH`, `config.py` ~315). loguru's -`catch` defaults to `reraise=False`, so any exception -- 401, 500, timeout, validation error -- is logged and the -method returns `None`. Checked with loguru 0.7.3: both a sync and an `async def` function return `None`. It applies -to all ~1,657 `@dynamic_catch` methods in `pyegeria/omvs/`. The choice is also made once, at import time, so -changing the setting afterwards has no effect. +**What:** a routine's result state (`measured` / `nothing_found` / +`not_established` / `not_applicable`, plus a reason) has nowhere to go on a +chart; it survives only in the DICT output. A tile can therefore show a +chart with no indication that the numbers were not established. -**Effect:** a caller cannot tell a failed read from an empty one; a `None` can render as "no results". Same family as -ISSUE-123 (a failed measurement indistinguishable from a real zero). Resource Explorer now treats `None` from a -wrapped method as "unreadable", never as "no result". +**Where seen:** trellis `tests/test_portal_tile.py` pins the absence, so that +test will fail once this is fixed. -**Fix (not made):** pass `reraise=True` to `logger.catch` (keeps the logging, restores the exception), or document -that the setting is debug-only and must not be used by applications that act on results. +**Candidate fix:** render an optional `envelope`/`subtitle` field from the +result into the Vega spec's `title.subtitle`, and into every output type. --- -### ISSUE-133: Egeria 6.2 returns a solution component's children as `nestedSolutionComponents`; one pyegeria formatter still reads only `subComponents` +### ISSUE-138: Analytic params are JSON-only, and only parameters named `mgr`/`ce`/`expert`/`client` are bound as clients, so a routine taking a reader object can't be named by its own dotted path -**Layer:** pyegeria · **Status:** fixed in #439, released in 6.2.1 (2026-10-10) · **Found:** 2026-10-10, reported by the Resource Explorer session -(`SolutionArchitect.get_solution_component_by_guid` JSON on 6.2); confirmed from Egeria source. +**Layer:** pyegeria (format_set_executor, analytic_registry) · **Status:** open, logged only (no fix without the owner's explicit go-ahead) · **Found:** 2026-10-12, reported by the Resource Explorer PR/CI session from trellis-analytics (trellis PR #622, `trellis_analytics/portal.py`), which works around it today. -**What:** Egeria's REST element (`AttributedMetadataElement.java` ~387) carries child components as -`nestedSolutionComponents` (the `SolutionComposition` relationship, 0730), each a `RelatedMetadataElementSummary` -(`relatedElement.elementHeader.guid`). It has no `subComponents` field; `subComponents` appears in Egeria only in -Java archive builders, not in any REST response. A caller reading `subComponents` sees no children. +**What:** a routine whose first parameter is a reader object (e.g. +`OdsReader`) cannot be named directly by its dotted path; the spec has to +point at an adapter that constructs the reader from JSON params. -**pyegeria readers:** -- `pyegeria/omvs/governance_officer.py` ~274 (`_extract_solution_components_properties`) reads **only** - `subComponents`, so its sub-components column would always be empty. It turned out to have no callers - (`SolutionArchitect` has its own method of the same name), so no live output was affected. -- `solution_architect.py` ~609 and ~5181, and `md_processing/v2/solution_architect.py` ~384, already read - `nestedSolutionComponents`. Each also has a leftover `subComponents` fallback (~675, ~5187, ~388) that never - matches. These do no harm, but they use the old flat shape (`elementHeader` directly, not `relatedElement`), so they - would mis-parse if the field ever came back. +**Where seen:** trellis `trellis_analytics.portal.curation_coverage_tile` (the +adapter), used instead of `trellis_analytics.routines.curation_coverage`. -**Fixed in #439:** `governance_officer.py` reads `nestedSolutionComponents` (through `relatedElement`), and the -three dead `subComponents` fallbacks are removed. Micro-tests in `tests/micro-tests/test_issue_131_133.py`. +**Candidate fix:** a binding hook (e.g. `AnalyticFunctionSpec.binders`) or a +declared factory per parameter type, so non-client objects can be +constructed from params. --- @@ -263,101 +270,143 @@ annotations); do not choose by majority or by what the code already sends. --- -### ISSUE-131: 13 `*_by_name` wrappers pass `metadata_element_type=`, but `_async_get_name_request` reads `metadata_element_type_name`, so the type filter is silently dropped - -**Layer:** pyegeria · **Status:** fixed in #439, released in 6.2.1 (2026-10-10) · **Found:** 2026-10-10, while building Resource Explorer's actor-graph lookups (`get_user_identities_by_name`, `get_actor_profiles_by_name`). +### ISSUE-127: `CollectionManager.delete_collection(cascade=True)` silently never sends `cascadeDelete` -- same shape as ISSUE-62 -**What:** wrappers such as `ActorManager._async_get_user_identities_by_name` (`pyegeria/omvs/actor_manager.py` ~4195) build `params = {"metadata_element_type": metadata_element_type_name, ...}` and call `self._async_get_name_request(url, **params)`. The base method (`pyegeria/core/_server_client.py` ~6581) takes `metadata_element_type_name` and writes it to `"metadataElementTypeName"`, so the wrapper's key falls into `**kwargs` and never reaches the request body. The by-name searches run unfiltered by type. +**Layer:** pyegeria · **Status:** partly fixed in #429 (`68279835`), released in 6.1.29; cascade still blocked by ISSUE-132 · **Found:** 2026-10-05, found by capturing the request body while diagnosing a +failed cleanup. -**Scope (checked 2026-10-10):** `grep -rn '"metadata_element_type": metadata_element_type_name' pyegeria` matches 27 lines, but only 13 are broken. The other 14 are `find_*` wrappers that call `_async_find_request`, whose parameter really is `metadata_element_type` (`_server_client.py` ~6468), so they are correct; do not change them. The 13 broken ones are all the `*_by_name` wrappers that call `_async_get_name_request`: -- `actor_manager.py`: `get_actor_profiles_by_name`, `get_actor_roles_by_name`, `get_user_identities_by_name`, `get_contribution_records_by_name`, `get_contact_details_by_name`, `get_perspectives_by_name`, `get_skills_by_name` -- `collection_manager.py`: `get_collections_by_name` -- `digital_business.py`: `get_business_capabilities_by_name` -- `location_arena.py`: `get_locations_by_name` -- `runtime_manager.py`: `get_metadata_repository_cohorts_by_name` -- `subject_area.py`: `get_subject_areas_by_name` -- `time_keeper.py`: `get_context_events_by_name` +**Fix + scope (2026-10-05):** a scan of all 42 callers of `_async_delete_element_request` found exactly two +that pre-fill a default body *and* pass a cascade flag: `CollectionManager._async_delete_collection` and +`ProductManager._async_delete_digital_product` (so `delete_digital_product(cascade=True)` was never cascading +either -- the live check's own product cleanup included). Both now pass `None` so the helper applies the flag. +`delete_digital_product`'s docstring sample also used the wrong key (`cascadedDelete`); corrected. The helper +itself is unchanged: a caller that passes an explicit body dict *and* `cascade=True` still gets the body's +value, which is documented behaviour ("body supersedes"). -**Root cause:** the two base helpers spell the same argument differently (`_async_find_request(metadata_element_type=...)` vs `_async_get_name_request(metadata_element_type_name=...)`), so a wrapper copied from a `find_*` sibling silently loses the filter. +**What:** `_async_delete_collection` always pre-fills `body = {"class": "DeleteElementRequestBody"}` when none is +given and hands that dict to `_async_delete_element_request(url, body, cascade)`. +`validate_delete_element_request` applies `cascade_delete` only in its *no body* branch, so with that pre-filled dict +the flag is discarded. Captured: `delete_collection("g", cascade=True)` posts +`{'class': 'DeleteElementRequestBody', 'forLineage': False, 'forDuplicateProcessing': False}` with no +`cascadeDelete`. `MetadataExpert.delete_metadata_element(cascade_delete=True)` does send it. -**Effect:** callers relying on the type filter get elements of other types. Resource Explorer works round it by passing a full `FilterRequestBody` with `metadataElementTypeName`. +**Fixed in #429, but cascade still does not reach Egeria -- see ISSUE-132.** #429 made the flag reach the request +body, but under the name `cascadeDelete`. Egeria's real field is `cascadedDelete`, and it ignores unknown fields, so +the flag is still dropped, now on the server side. #429's docstring "correction" from `cascadedDelete` to +`cascadeDelete` was also the wrong way round. The `DataStructure` refusal described below is probably this bug +rather than an Egeria quirk; re-test once ISSUE-132 is fixed. -**Fixed in #439:** `_async_get_name_request` accepts both spellings (`metadata_element_type_name` wins if both are given), and the 13 wrappers now use `metadata_element_type_name`. The 14 `find_*` wrappers are unchanged. Micro-tests in `tests/micro-tests/test_issue_131_133.py` capture the request body for each wrapper. Same family as ISSUE-62, ISSUE-127 and ISSUE-129: an argument accepted and silently dropped. +**Related, an Egeria quirk found alongside:** even with `cascadeDelete: true`, Egeria refused to delete a +`DataStructure` that still had a member `DataField` (`OMAG-GENERIC-HANDLERS-403-005 ... validateNoMemberDataFields`), +so a data contract imported by `import_data_contract*` has to be removed child-first: DataField, DataStructure, +Agreement. --- -### ISSUE-130: The YAML secrets store's user directory caches a user forever, so editing an existing user needs a platform restart +### ISSUE-123: `overview_metrics.count_elements` (and `_find`/`_element_count`) return `0`/`[]` on any failure — a failed measurement is indistinguishable from a real zero -**Layer:** Egeria server (`YAMLSecretsStoreConnector`) · **Status:** open, reported to the Egeria lead · **Found:** 2026-10-09, while adding Resource Explorer's own account (`resourceexplorernpa`). +**Layer:** Pyegeria · **Status:** open · **Found:** 2026-10-04, while +making the portal's Egeria Overview stop showing confident numbers during +an outage · **GitHub:** odpi/egeria-python#424 -**What:** `getUser(userId)` returns the entry from `userAccountMap` if it is already there. `refreshSecrets()` re-reads the YAML file every `refreshTimeInterval` minutes, but `userAccountMap` (and `securityAccessControlMap`) are created only on the first load (when `secretsStoreFile == null`) and are never cleared afterwards. Re-reading the file only helps users the cache has never seen; the comment says "this may be a new user". +**Root cause, from reading `pyegeria/view/overview_metrics.py` (6.1.28):** -**Seen:** the account was first added without its `secrets` block, and the first token request cached it. After the password was added (and matched the servers file), `/api/token` still answered 401 for more than 10 minutes. It worked straight after a platform restart. +- `_find(mgr, body, page_size)` wraps `find_metadata_elements` in a bare + `except Exception` and returns `[]`, logging only at `debug`. +- `_element_count(mgr, body, as_of)` tries the native count; if it is + unsupported *or raises*, it falls back to `len(_find(...))` — so a failed + query becomes `0`. +- `count_elements(...)`'s docstring states "0 on any failure (never + raises)". The callers `people_counts`, `usage_context_counts` and + `counts_by_type` inherit the same behaviour. -**Effect:** any change to an existing user (password, status, named-list membership) needs a platform restart. +**Effect:** a dashboard cannot tell "there are 0 elements" from "Egeria +timed out / returned 401 / 500". An outage or auth failure renders as a +confident `0`. A fallback count that returns exactly the page size is also +only a lower bound, not a total, and is not flagged. -**Fix (Egeria):** clear or rebuild `userAccountMap` and `securityAccessControlMap` in `refreshSecrets()`. +**Proposed fix:** return `None` on failure (or add a strict variant, e.g. +`count_elements(..., strict=True)` / `try_count_elements`, that raises or +returns `None`), keeping today's behaviour behind an explicit opt-in if +existing callers rely on it. Apply the same to `_find`/`_element_count` and +propagate `None` through `people_counts`, `usage_context_counts` and +`counts_by_type`. Log failures above `debug`. Flag a capped fallback count +as a lower bound. + +**Workaround in egeria-workspaces:** the portal's Overview handler calls +`find_metadata_elements` directly and returns `None` for failed or capped +queries, rendering "Not measured" instead of `0` (odpi/egeria-workspaces +#601, #607). --- -### ISSUE-129: `MetadataExpert.get_related_metadata_elements` (and two sibling relationship queries) accept paging, direction and lineage arguments but never send them +### ISSUE-125: `deleteEnumDef` answers 500 "unknown TypeDef" for an enum the server lists -- an enum added through the API cannot be removed -**Layer:** pyegeria · **Status:** open, not fixed · **Found:** 2026-10-09, in a code review of Resource Explorer's -6.2 composition read (`blueprint_materializer.sub_component_guids`). +**Layer:** Egeria Server (not pyegeria) · **Status:** open; reported to the Egeria team (response 2026-10-05); Egeria fix merged to Egeria `main` 2026-10-06 (odpi/egeria#9363), not yet re-tested on a platform built from it · **Found:** +2026-10-05, live verification of `MetadataExpert.delete_enum_def` (PR #428). **Left a stray type behind.** -**What:** `_async_get_related_metadata_elements` (`pyegeria/omvs/metadata_expert.py` ~2886-2964) builds the URL from -`guid` and `relationship_type` only, and posts `body_slimmer(body)`. `starting_at_end`, `start_from`, `page_size`, -`for_lineage` and `for_duplicate_processing` are documented and silently dropped. So the server's default page size -decides how many relationships come back, both ends of the relationship are returned whatever `starting_at_end` says, -and Memento-classified elements are handled by the server default rather than the caller's `for_lineage`. A caller -that guards "a full page means I cannot tell if there are more" is guarding a value that never reached Egeria. Same -shape as ISSUE-62 and ISSUE-127. +**What:** after `add_enum_def` (returned GUID `118441be-...`), `add_type_def` of an entity type using it, and a +successful `delete_type_def` of that entity type, `delete_enum_def(guid, name)` answered +`OMRS-CONTENT-MANAGER-500-001 The repository content manager method getAttributeTypeDef has detected an unknown +TypeDef 118441be-... from qs-metadata-store on behalf of method deleteAttributeTypeDef` ("Open up a Github issue"). +Yet `ValidMetadataManager.get_attribute_types()` still lists the enum, so the server knows it by one route and +not the other. -**Same bug in two siblings (checked 2026-10-10):** `_async_get_all_metadata_element_relationships` (~3056, -`linked-by-any-type`) and `_async_get_metadata_element_relationships` (~3233, `linked-by-type`) also accept -`start_from` and `page_size` and send neither. They also accept `starting_at_end`, but Egeria's endpoints for -these two take no such parameter (both ends are fixed by the URL), so that argument is meaningless there and -should be removed rather than wired up. +**Cause (Egeria team, confirmed in source at `449ad06894` on 2026-10-05):** `OMRSRepositoryContentValidator.java:877` +calls `repositoryContentManager.getAttributeTypeDef(sourceName, guidParameterName, guid, methodName)`, but that +4-argument overload is `(sourceName, attributeTypeDefGUID, attributeTypeDefName, methodName)`. The "GUID" passed is the +parameter-NAME string and the "name" is the real GUID, which is why the error prints our real GUID as the unknown +TypeDef. Every `deleteAttributeTypeDef` through the local repository fails (as do `reIdentifyAttributeTypeDef` and the +enterprise equivalents). **My earlier hypothesis here (a GUID map populated in only one place) was wrong.** -**Workaround (RE):** put `startFrom`/`pageSize` in the `ResultsRequestBody` itself and filter entries by end. +**Left on the shared dev platform:** enum `PyegeriaTmpCuisineType` (`118441be-6e03-4442-96c6-e431f75fcb3f`), +unused, announced to the cohort. Harmless but visible in type listings. Per the Egeria team it is deletable only after a build containing the fix is deployed. -**Server contract (verified 2026-10-10 against Egeria source, `egeria-v6/egeria` at `af400039c2`, 2026-10-06 -- -not just the `.http` file):** `MetadataExpertResource.java` declares `startingAtEnd` as -`@RequestParam(required = false, defaultValue = "0") int` on `/related-elements/{elementGUID}/any-type` and -`/related-elements/{elementGUID}/type/{relationshipTypeName}`, and `MetadataExpertRESTServices` passes it through, so -the `?startingAtEnd=` query parameter is current, not a leftover. The two `linked-by-*` endpoints take only a -`ResultsRequestBody`. `ResultsRequestBody` extends `QueryOptions` -> `PagingOptions` (`startFrom`, `pageSize`) -> -`GetOptions` -> `BasicOptions` (`forLineage`, `forDuplicateProcessing`, `effectiveTime`), so the other four -arguments belong in the body. +**Re-tested 2026-10-05 after the platform was rebuilt and restarted** (image built 14:19Z from egeria main +`449ad06894`, `egeria-main` restarted 14:39Z): the enum survived the restart and is still listed, and +`delete_enum_def` fails with the identical `OMRS-CONTENT-MANAGER-500-001`. So a restart does not help, which the argument-order bug above +explains: the failure does not depend on what was loaded at startup. -**Fix (not made):** in the two `related-elements` methods, add `?startingAtEnd={starting_at_end}` to the URL -(`_async_get_all_related_metadata_elements`, ~2800-2808, already does this and is the pattern to copy), and put -`startFrom`/`pageSize`/`forLineage`/`forDuplicateProcessing` in the body. In the two `linked-by-*` methods, put -paging in the body and drop `starting_at_end`. Only add body fields the caller's body does not already set, so -"body supersedes" (which ISSUE-127 relies on) still holds; the `page_size` default is `max_paging_size`, so -overwriting would silently change callers who set it in the body. +**Workaround:** none through the API. Avoid creating throwaway enums on a shared platform. + +**Egeria team response, relayed 2026-10-05 (source `oak2026` fb3d6fce53; **fixes are in two unmerged PRs, so no build contains them, and nothing was tested against a live platform**):** valid; fix in source on `oak2026`, pending merge to `main`. + +**Update 2026-10-10:** the `oak2026` source (`fb3d6fce53`) is now in Egeria `main`, merged 2026-10-06 (odpi/egeria#9363), so a platform built from current `main` has the fix. Not yet re-tested. --- -### ISSUE-128: Retention cannot be set on Egeria 6.2: the server registers the Jackson subtype as `RetentionProperties.class` (typo), and pyegeria only allows `RetentionClassificationProperties` +### ISSUE-124: `updateTypeDef` (Metadata Expert `open-metadata-types/update`) can never succeed -- the server builds the patch without `updatedBy`, which the repository services require -**Status:** open, found 2026-10-09 by Resource Explorer's first Retention publish on Egeria 6.2. **Layer:** Egeria server (a type registration typo). **Decision (owner, 2026-10-09):** wait for the Egeria fix; no pyegeria workaround. Retention publishing stays blocked until then. +**Layer:** Egeria Server (not pyegeria) · **Status:** open; reported to the Egeria team (response 2026-10-05); Egeria fix merged to Egeria `main` 2026-10-06 (odpi/egeria#9363), not yet re-tested on a platform built from it · **Found:** +2026-10-05, live verification of `MetadataExpert.update_type_def` (PR #428). -**What (all proven live on 6.2, with raw HTTP POSTs to `classification-explorer/elements/{guid}/retention` on a throwaway asset):** -- `"class": "RetentionClassificationProperties"` is refused with `InvalidTypeIdException`. This is what pyegeria documents and whitelists, and it worked on the pre-6.2 server (processors.py comment, 2026-08-03). -- `"class": "RetentionProperties"` (the `.http` ground truth) is also refused with `InvalidTypeIdException`. -- `"class": "RetentionProperties.class"` gets **HTTP 200**, and the classification reads back with `retentionBasis` 2 and the notes. -- The server's known-type list in the error has 96 ids. The only Retention entry is literally `RetentionProperties.class`. That looks like an `@JsonSubTypes.Type(name = "RetentionProperties.class")` typo in Egeria's `ClassificationProperties`. -- pyegeria's `validate_new_classification_request` whitelist (`prop=["RetentionClassificationProperties"]`) refuses anything else client-side, so no Retention can be set through pyegeria on 6.2. +**What:** posting a valid `OpenMetadataTypeDefPatch` (type added through the API moments earlier, +`applyToVersion` 1) answers a 500 wrapping +`OMRS-REPOSITORY-400-069 Method updateTypeDef has detected that a TypeDef patch from qs-metadata-store has the +mandatory field updatedBy set to null ... TypeDefPatch{typeDefGUID=..., typeDefName=`PyegeriaTmpRecipe`, +applyToVersion=1, updateToVersion=2, newVersionName=`2.0`, updatedBy=`null`, ...}`. The request itself arrived +intact (the attribute definition is visible in the message). -(An earlier version of this entry said 6.2 knows `RetentionProperties`. That was a misread of the error text, corrected the same day.) +**Cause, from Egeria's source (`origin/main`, 2026-10-01):** `OpenMetadataStoreRESTServices.updateTypeDef` +calls `converter.getTypeDefPatch(requestBody, methodName)`; `OMRSTypeDefConverter.getTypeDefPatch` takes no user +id and has no `setUpdatedBy` call, while `OMRSMetadataCollection.updateTypeDef` rejects a patch whose +`updatedBy` is null. `OpenMetadataTypeDefPatch` has no `updatedBy` field either, so a client cannot supply it. +Not a pyegeria defect: the SDK sends the documented body. Fix belongs upstream (pass `userId` to the converter and +set it). -**Where seen:** Resource Explorer's Curate classifications step on two repository assets. Both were refused, so nothing was written. Confidentiality, Criticality, Confidence and Impact property classes are registered normally. +**Re-tested 2026-10-05 after the platform was rebuilt and restarted** (image built 14:19Z from egeria main +`449ad06894`): a throwaway entity type (primitive attribute only) was added and patched with one new attribute. +The server rejected the patch with the identical `OMRS-REPOSITORY-400-069 ... updatedBy set to null` +(`TypeDefPatch{... applyToVersion=1, updateToVersion=2, newVersionName=2.0, updatedBy=null ...}`). The type was +then deleted and verified gone. **Confirmed still present on the rebuilt platform.** -**Candidate fix:** -1. **Egeria:** correct the subtype name to `RetentionProperties`, matching the `.http` file. Report it to Mandy. -2. **pyegeria, after Egeria's fix:** change the whitelist and docstring to `RetentionProperties`, the `.http` name. No workaround for the typo, by the owner's decision. -3. Re-check `md_processing/v2/processors.py` and `v2/curation.py`, which both pin `RetentionClassificationProperties`. +**Effect here:** `update_type_def` is correct but unusable against current Egeria; its docstring says so. + +**Egeria team response, relayed 2026-10-05 (source `oak2026` fb3d6fce53; **fixes are in two unmerged PRs, so no build contains them, and nothing was tested against a live platform**):** valid, and `updateTime` was missing as well as `updatedBy` (our analysis was right but incomplete). `getTypeDefPatch` will take the user and set both; new `DynamicTypeFVT`. + +**Update 2026-10-10:** the `oak2026` source (`fb3d6fce53`) is now in Egeria `main`, merged 2026-10-06 (odpi/egeria#9363), so a platform built from current `main` has the fix. Not yet re-tested. + +--- ### ISSUE-117: Cascade delete that takes the soft-delete (Memento) path fails partway with `OMAG-REPOSITORY-HANDLER-400-010` unless `forLineage=true` — leaves the asset live and an anchored element already soft-deleted @@ -507,139 +556,9 @@ downstream investigation. --- -### ISSUE-102: `MemberDataField.minCardinality` silently persists as `maxCardinality`'s value regardless of what's actually sent — server-side, confirmed via a raw request bypassing every pyegeria/Dr.Egeria layer +### ISSUE-90: [fixed?] `qs-engine-host` retries `startMissedEngineActions` forever when one incomplete engine action's anchor is unreadable by the engine-host user -**Layer:** Egeria Server (repository/relationship-property persistence) · **Status:** NO LONGER REPRODUCES on the 2026-10-05 rebuilt platform (see the re-test below) · **Found:** 2026-09-17, live-verifying a Dr.Egeria fix for `Position`/`Minimum Cardinality`/`Maximum Cardinality` on the field↔structure `MemberDataField` relationship (`qs-view-server`/`qs-metadata-store`, versionName `6.2-SNAPSHOT`). - -Confirmed with the type system's own definition -(`ValidMetadataManager._async_get_all_relationship_defs()`, filtered to -`MemberDataField`) that the three real attributes are `position`, -`minCardinality`, `maxCardinality` — all plain `int`, `AT_MOST_ONE` -cardinality, no documented interdependency between `minCardinality` and -`maxCardinality`. - -**Re-tested 2026-10-05 on the rebuilt platform** (Egeria image built 14:19Z from main `449ad06894`, -`egeria-main` restarted 14:39Z). One throwaway `DataStructure` and four `DataField`s were linked with -`MemberDataField` and each relationship was read back via the endpoints lookup -(`get_metadata_element_relationships`). **All four stored correctly** (`position` always right): - -| sent (min, max) | stored (min, max) | -|---|---| -| 1, 5 | 1, 5 | -| 2, 7 | 2, 7 | -| 0, 5 | 0, 5 | -| 3, *omitted* | 3, 0 | - -**Raw capture for the Egeria team, 2026-10-06 (about 02:00Z):** at their request, one more throwaway `DataStructure` + `DataField` were linked with the exact request `NewRelationshipRequestBody{properties: MemberDataFieldProperties{position 3, minCardinality 1, maxCardinality 5}}` (answer `VoidResponse`), and the relationship was read back through the endpoints lookup (`OpenMetadataRelationshipListResponse`): `minCardinality` "1", `maxCardinality` "5". Still no repro, now with the raw request, raw response and read-back captured (kept locally, session scratchpad `live_102_raw.out`; the team's three asks are met). Deleted child-first, verified gone by GUID. - -Everything was deleted child-first and verified gone. **The bug no longer reproduces.** The cause of the fix is not -known: a source comparison of the bean (`MemberDataFieldProperties`/`PartOfRelationshipProperties`), the read -converter and the relationship builder between 2026-09-15 and `449ad06894` found no change, so either the fix is -in code that was not inspected or the build that showed the bug on 2026-09-17 was older than that baseline. Worth -asking the Egeria team which change fixed it before closing this for good. Side observation, separate from this -bug: an omitted `maxCardinality` is stored as `0` (the bean default), below a supplied minimum; the bean's own -documentation says `-1` means unlimited. - -**Repro (isolated with a raw SDK call — no Dr.Egeria markdown, no pyegeria -body-construction logic in the path beyond `EgeriaTech.data_designer`):** -```python -link_body = { - "class": "NewRelationshipRequestBody", - "properties": { - "class": "MemberDataFieldProperties", - "position": 3, - "minCardinality": 1, - "maxCardinality": 5, - }, -} -await client.data_designer._async_link_member_data_field(struct_guid, field_guid, link_body) -# fetch the relationship back: -# position: 3 -- correct, matches what was sent -# minCardinality: 5 -- WRONG, silently coerced to maxCardinality's value (5), not the sent 1 -# maxCardinality: 5 -- correct -``` -`position` round-trips correctly, ruling out a client-side body-construction -bug (the exact same request dict's `position` key is honored, its -`minCardinality` key is not). No error, warning, or validation failure is -returned — the create call reports success and returns a real relationship -GUID; the wrong value is simply what gets stored. - -**Also worth flagging separately, lower confidence:** the -`_async_link_member_data_field` SDK method's own docstring sample body -(`pyegeria/omvs/data_designer.py` ~line 2286) shows `dataFieldPosition` as -the position field's name — that's wrong per the same live type-system -query (the real name is `position`); not filed as its own issue since it's -a pyegeria-side docstring fix, not an Egeria server issue, but noted here -since it was found in the same investigation and would otherwise cause a -second, harder-to-diagnose silent-drop bug for the next person who trusts -that docstring. - -**Ask:** confirm whether `minCardinality` defaulting to `maxCardinality` -when both are supplied is intentional server-side business logic (e.g. a -"min cannot exceed max, so raise min to max" normalization applied even -when min is explicitly below max and both are valid on their own) or a -genuine persistence bug; if intentional, it isn't reflected anywhere in the -`MemberDataField` relationship def's own attribute descriptions. - -**Dr.Egeria-side impact:** `Create Data Field`'s `Minimum Cardinality` -attribute is parsed and sent correctly (see `PYEGERIA_ISSUES.md` ISSUE-101) -but its stored value cannot be trusted independently of `Maximum -Cardinality` until this is resolved server-side — nothing further is -fixable in this repo for that specific symptom. - -**Egeria team response, relayed 2026-10-05 (source `oak2026` fb3d6fce53; **fixes are in two unmerged PRs, so no build contains them, and nothing was tested against a live platform**):** NOT reproduced in Egeria (builder and converter round trip returns `position=3, min=1, max=5`; lower layers not traced). They asked for the exact request, the raw response and the read-back endpoint. Ours: request `NewRelationshipRequestBody` with `properties` `{class: MemberDataFieldProperties, position: 3, minCardinality: 1, maxCardinality: 5}` via `DataDesigner._async_link_member_data_field`; the original raw response was not captured. On the 2026-10-05 rebuilt platform it does not reproduce for us either (see the re-test above). - -**Update 2026-10-10:** the `oak2026` source (`fb3d6fce53`) the team examined is now in Egeria `main` (merged 2026-10-06, odpi/egeria#9363). No fix was made for this issue; the team's questions above still stand. - ---- - -### ISSUE-95: No catalog template registered for the "Apache Kafka Server" technology type — `Create Kafka Server Element` (Asset Maker) fails with a 400 on `qs-view-server` - -**Layer:** Egeria Server (deployment/archive content) · **Status:** open, being investigated by the user (2026-09-11) · **Found:** 2026-09-11, live-verifying the new Asset Maker Dr.Egeria family (#354) - -`AutomatedCuration._async_get_template_guid_for_technology_type("Apache Kafka Server")` -returns `None` on `qs-view-server` — no catalog template is registered for -that technology type on this server — so `Create Kafka Server Element`'s -underlying `_async_create_kafka_server_element_from_template` posts a body -with `templateGUID: null` and gets back `VALIDATION_ERROR_1` ("Request body -failed validation"). Not a pyegeria code defect: every other Asset Maker -technology type tried resolved a real template GUID on the same server -(CSV Data File, PostgreSQL Server, File System Directory, YAML File Secrets -Collection, Unity Catalog Server) — this one is specific to the Kafka -technology type's archive content on this deployment. `Create Kafka Server -Element` itself is otherwise correct (parses/validates cleanly; the failure -is entirely server-side at execution). - -**Ask:** confirm whether the Apache Kafka Server catalog template is -expected to ship with quickstart's default archive content, and if so, why -it's missing here — or add it to whichever archive/content-pack is meant to -carry it. - ---- - -### ISSUE-89: No configurable bearer-token lifetime in `application.properties` — platform tokens are fixed at 3600 s, and the signing key is random per restart - -**Layer:** Egeria Server · **Status:** open · **Found:** 2026-09-04 (trellis-auth token contract) - -With `authentication.mode=token`, every token minted by the platform decodes to `exp - iat = 3600` -(observed on `omag-server-platform-6.2-SNAPSHOT`, quickstart, `authentication.source=platform`). -`application.properties` exposes `authentication.mode`, `authentication.source` and `rsa.key-id`, and -nothing that controls the lifetime; with `rsa.key-id` empty the signing key is regenerated at start, so -every token also dies on a platform restart. Client applications that hold a platform token on the -user's behalf (Portal, Egeria Advisor, Resource Explorer — which now carry the Egeria bearer token in -their session and never the password) cannot offer a session longer than an hour without either -re-prompting the user or caching a password, which is what a bearer token exists to avoid. - -**Ask:** a property such as `authentication.token.lifetime=3600` (seconds, default unchanged), read at -start and applied when a token is issued; optionally a refresh operation that returns a new token for a -valid unexpired one. Full draft issue text: trellis session scratch `egeria-issue-token-lifetime.md`. - -**Correction, 2026-10-05 (Egeria team; confirmed in `RSAGenerator`):** the statement above that an empty `rsa.key-id` is why the signing key is regenerated is wrong. `RSAGenerator.generateRSAKeyPair(keyId)` always makes a fresh key pair and only uses the id as a label, so **no setting makes tokens survive a restart**; persisting the key is planned for the next release. Also: the lifetime property now exists, `bearerTokenTimeout` in HOURS (default 1), added in `a1f228d4a9` (2026-09-19); there is still no refresh operation. - - -### ISSUE-90: [fixed?] `qs-engine-host` retries `startMissedEngineActions` forever when one incomplete engine action's anchor is unreadable by the engine-host user - -**Layer:** Egeria Server (possibly quickstart content) · **Status:** open · **Found:** 2026-09-04 (trevor fresh quickstart) +**Layer:** Egeria Server (possibly quickstart content) · **Status:** open · **Found:** 2026-09-04 (trevor fresh quickstart) On a fresh quickstart repository (6.2-SNAPSHOT builds of 2026-08-31 and 2026-09-04, all standard content packs, stock `coco-user-directory.omsecrets`) the engine host logs, every few seconds from first @@ -705,84 +624,318 @@ read. Full draft: trellis session scratch `egeria-issue-engine-host-403-loop.md` --- -### ISSUE-38 (PY-18): `count_relationships_between_elements("Exception")` (276) disagrees with `ClassificationExplorer.get_relationships("Exception")` (55) +### ISSUE-79: native survey against a template-created `FileFolder` asset fails server-side — `assetConnector` is null in `BasicFolderConnector.getFile()` -**Update 2026-08-30, from the Egeria team (Mandy Chessell).** Leaving this -entry in "Open Egeria Server issues" until the verification below has been -run against real data — the Egeria side is done, but this entry has been -re-measured and re-confirmed several times, and it shouldn't move to -Fixed/Resolved on the strength of a change nobody has yet pointed at the -data that produced the original report. +**Live check 2026-10-06 (about 02:00Z, current pyegeria 6.1.29 code, rebuilt platform): the `deepCopy:false` default WAS the cause of the missing connection.** `create_folder_element_from_template` now sends a request with no `deepCopy` key (keys sent: allowRetrieve, class, forDuplicateProcessing, forLineage, isOwnAnchor, parentAtEnd1, placeholderPropertyValues, templateGUID, templateSubstitute), and the new folder read back through the classification explorer **has 1 connection** (`ResourceConnection` present). Folder deleted (cascade) and verified gone by GUID. **Not run: the survey itself** (a survey creates elements owned by the survey identity that an ordinary user may not be able to delete), so "the survey now completes" is still unproven; the original `assetConnector`-null failure was the symptom of no connection, which is now shown not to occur. Whether the Egeria-side repair (`e01426db86`) also matters is untested. Script: `live_79.py` stage A (stage B `--survey` is opt-in). -**Status: resolved on the Egeria side 2026-08-30** (`omf-metadata-spring`, -`omf-metadata-server`, `generic-handlers`, and the OMF Java client) — -**awaiting verification from the pyegeria side**, see below. The two -numbers are not going to be made equal, because they are answers to two -different questions and both are wanted; what has changed is that the -caller now chooses which one they get. +**Update 2026-08-30, from the Egeria team (Mandy Chessell).** The entry +stays in "Open Egeria Server issues" — the reported failure is not +explained and not fixed. What has changed is that the cause recorded in +this entry, and in Resource Explorer's `re-as-engine-host-plan.md`, is now +known to be wrong, and the error it produces has been made legible. -**What the difference actually was.** `findAttachmentLinks()` puts every -relationship it retrieves through a visibility check on the entities at -both ends, and silently drops the ones whose anchor the caller cannot -read. `countAttachmentLinks()` skipped that check — deliberately, and -documented as such in its own javadoc, because applying it means fetching -every matching relationship, which is the cost the count exists to avoid -— and also ignored `effectiveTime`. So the count was of what matches the -search and the list was of what the caller may see. That answers this -entry's open question ("what does the metadata-expert count include for -`Exception` that the classification-explorer traversal excludes?"): -relationships with an end whose anchor is not readable by the calling -user, plus anything outside the effectivity window. It also explains why -only `Exception` diverged among the types tested — it is the type whose -ends are most likely to have been deleted or anchored to something the -caller cannot see — and why the divergence tracked the demo data rather -than staying at a fixed size. +**Status: still open (Egeria Server)** — but the recorded cause is +disproved, and this entry and `re-as-engine-host-plan.md` should both be +corrected. Investigated 2026-08-30 from the Egeria side, including a new +FVT suite built to run this exact scenario. -**The new option.** Both counting endpoints take a `pushDown` query -parameter, `required=false`, default `true`: +**The premise is false.** This entry, and the design doc's hold, rest on +"template-based asset creation never wires up a working Connection." That +is not true, in either environment it was checked against: + +- **On the quickstart that produced the report.** Every element created + from a template that itself carries a `ResourceConnection` got one — 10 + of 10. Every `FileFolder` in that repository has one, including all + three that were created from a template (`/deployments/content-packs`, + `/deployments/loading-bay/sample-data`, `/deployments/secrets`). +- **In a clean environment built from the current archives.** A folder + catalogued through `FileGovernance::create-file-folder` came back with a + `ResourceConnection` attached. +- **The `FilesContentPack` template itself is also well formed:** + `FileFolder` template `fbdd8efd-...` carries an `Anchors`-classified + `Connection` with both a `ConnectToEndpoint` and a + `ConnectionConnectorType`. + +**The folder survey works.** A new `files-fvt` suite runs the reported +scenario end to end — catalogue a folder from the `FileFolder` template, +then survey it natively — and it passes: ``` -POST .../users/{userId}/relationships/by-search-conditions/count?pushDown=false -POST .../users/{userId}/metadata-elements/by-search-conditions/count?pushDown=false +OMES-SURVEY-ACTION-0019 The survey action service folder-survey-service has completed the +analysis of asset 7050dc50-... with request type survey-folder in 1927 milliseconds; the +results are stored in survey report 7e7892e2-... ``` -- `pushDown=true` (the default, and exactly the existing behaviour) — the - repository counts the matching rows itself. Fast, and counts what - matches the search. -- `pushDown=false` — the relationships are retrieved and counted, so the - answer agrees with the list by construction, at the cost of reading - every one of them. +The report carried six annotations. The test asserts the survey +*completed*, not merely that it ran: a survey that dies on a null +connector still produces an engine action, and the waiter throws on any +terminal status other than `COMPLETED`. -Nothing changes for a caller that does not send the parameter. The same -choice is on the Java client (`countMetadataElements(..., pushDown)` and -`countRelationshipsBetweenMetadataElements(..., pushDown)` on -`OpenMetadataStore` and `OpenMetadataClient`), with the existing -signatures kept and defaulting to `true`. +**A second hypothesis was also eliminated.** While investigating, a real +defect was found in the PostgreSQL repository: a `TypeDefPatch` that +reparents a type does not update the denormalised supertype chain stored +on existing rows, so instances written before the patch become invisible +to searches for the new supertype. That would have hidden connections +exactly as reported, since `AssetConnection` was reparented under +`ResourceConnection` on 2026-08-27. It is fixed — but it was never the +cause here: the quickstart repository stores 202 rows as +`:ResourceConnection:LabeledRelationship:` and none as +`:AssetConnection:`, because it was reloaded after the type change. -**The test the pyegeria side should run to prove it.** Against the -environment that produced the original numbers, using whatever the -current counts are rather than the 58/57 recorded here — the demo data -has moved several times during this entry's life, and what matters is the -relationship between the numbers, not their values: +**What has been fixed, and what it does and does not buy.** The NPE +itself is now a clean error. `getConnectorForAsset()` returns `null` by +contract when an asset has no connection; `FolderSurveyService` cast that +null and dereferenced it, and the framework's own +`performCheckAssetAnalysisStep()` had the same hole on its error path. +Both now report `OPEN-SURVEY-400-008` / `OPEN-SURVEY-0009` +(`NO_ASSET_CONNECTOR`), naming the asset. -1. **The accurate count should equal the list.** For - `relationshipTypeName: "Exception"`, the count with `pushDown=false` - should equal `len(ClassificationExplorer.get_relationships("Exception"))` - exactly. This is the assertion that closes the entry — it is the - discrepancy this entry is about, stated as an equality. -2. **The fast count should be unchanged.** The same call with - `pushDown=true`, and with the parameter omitted altogether, should both - return what the count returns today. If either moves, something - regressed for every existing caller. -3. **A type that already agreed should still agree both ways.** - `SemanticAssignment` matched at 401=401 when this entry was last - measured. Both `pushDown` values should return that same number — the - retrieval route must not lose or double-count anything on a type where - there was never a discrepancy to resolve. +This does not make such a survey succeed — an asset with no connection +describes a resource nothing can open. It makes the failure say *which* +asset, which is what the original report was missing: the NPE named +`BasicFolderConnector`, a class the caller had never heard of, and pointed +away from the asset. -Worth capturing while verifying: the difference between the two counts is -the number of matching relationships the calling user cannot fully see. -If it is not the 1 recorded here, that is not a failure — it is a -measurement of the current data — but it is worth writing down, because a -large difference on a type that should be fully visible would be a +**Why that matters for how this entry reads.** Four further defects were +found in the same connectors, all on paths nothing else exercises: both +file survey services failed on every invocation (annotations created +without a qualified name, in three places), the CSV survey dereferenced a +null root schema type, wrote a relationship property to an entity, and +produced colliding annotation qualified names. The file survey path had, +on this evidence, never run to completion. Someone hitting an NPE in this +family had every reason to suspect the asset — the errors never named +anything else. That is context for the original diagnosis, not a +criticism of it. + +**The naming trap, which is directly actionable on the pyegeria side.** +This entry already records that `initiate_file_folder_survey`'s default +(`FileSurveys:survey-folder`) and the `.http` example +(`AssetSurvey:survey-folder`) are both wrong. Confirmed and now +understood: a governance action type is registered under +`::` — two colons — so the real name is +`FileSurvey::survey-folder`. `AssetSurvey` is not an engine that exists at +all. The `.http` examples have been corrected upstream +(`FileSurvey::survey-folder`, `ApacheKafkaSurvey::survey-kafka-server`, +`PostgreSQLSurvey::survey-postgres-server`, +`PostgreSQLSurvey::survey-postgres-database`) with a note stating the +shape, because the error a wrong name produces — +`OMAG-GENERIC-HANDLERS-400-013`, "the name is not recognized" — does not +say what it wanted instead. `initiate_file_folder_survey`'s default still +needs fixing in pyegeria. + +**What is left, and what would settle it.** The remaining candidate is +the deployment rather than the cataloguing or the survey: which server +and userId the engine host's connected-asset client resolves +`SurveyAssetStore.getConnectorToAsset()` against. That cannot be tested +from the Egeria side — it needs the engine host that produced the +failure. Suggested, in order: + +1. Re-run the original repro against a build carrying the NPE fix. If the + asset genuinely has no connection the run now fails with + `OPEN-SURVEY-400-008` naming it, which turns the question from "why is + this connector null" into "why does this asset have no connection that + this userId can see". If instead the survey completes, the deployment + has moved on and this entry can be closed. +2. Check the folder asset at the moment of failure for a + `ResourceConnection`, using the same userId the engine host runs as — + not an admin one. If the relationship is there but the survey cannot + see it, that is a visibility or server-routing problem, and the userId + is where to look next. +3. Compare against `files-fvt`, which is a known-good baseline for this + exact scenario. Pointing the same two actions at a + differently-configured engine host isolates the variable. + +Also worth correcting: this entry's earlier note about leftover probe +elements (`FileFolder::localhost:/tmp/re-e1-probe-*`, +`FileFolder::localhost:/tmp/re-issue79-recheck-*`) is out of date — the +quickstart repository has been reloaded since and holds none of them, so +that evidence is gone. + +**Status:** open (Egeria Server), reproduced live 2026-08-27, **re-checked +2026-08-28 — still reproduces, identical failure.** Same class/method +(`BasicFolderConnector.getFile()`, `assetConnector` null), same error IDs +(`OMES-SURVEY-ACTION-0018`/`OPEN-SURVEY-500-001`), same +`java.lang.NullPointerException`, against a freshly created `FileFolder` +and a fresh survey run — not a one-off. Filed by content, not by number — +see "why this entry exists" below. + +**Layer:** Egeria Server (survey action framework / `BasicFolderConnector`), +not pyegeria. `create_folder_element_from_template` and the survey-initiate +call both succeed and are correctly formed; the failure happens entirely +inside the server's survey action service. + +**Why this entry exists.** Resource Explorer's `re-as-engine-host-plan.md` +is ON HOLD citing this exact symptom ("connector is null" on a native +survey against a template-created asset) under the name `PYEGERIA_ISSUES.md` +**ISSUE-51** — but this file was renumbered on 2026-08-15, and today's +ISSUE-51 is an unrelated, already-fixed `fetch_element()` shape problem. +The blocker's own content had no surviving entry anywhere in this file. +Investigated fresh rather than guessed at, per the plan's own citation +being untrustworthy. + +**Two candidate bugs were in play, and they are not the same bug:** + +1. **Resource Explorer's superseded tracker, entry E1** (`packages/ + resource-explorer/docs/egeria-pyegeria-issues.md`) describes a *different* + symptom: a 500 from the PostgreSQL repository connector on + `createMetadataElementFromTemplate` (`metadata_collection_guid` null + while saving a classification) — a failure to *create* the element at + all. **Re-tested live 2026-08-27 and does not reproduce**: + `create_folder_element_from_template(path_name=..., folder_name=..., + file_system="localhost")` against `qs-view-server` returned a real GUID + with no error. Whatever this was, it looks fixed. +2. **The design doc's actual blocker** — a template-created asset surveyed + natively fails with a null connector — **does still reproduce**, and is + what this entry tracks. + +**Reproduction** (against `qs-view-server`, no pyegeria bug involved): +```python +from pyegeria import AutomatedCuration +c = AutomatedCuration("qs-view-server", "https://localhost:9443", "erinoverview", "secret") +c.create_egeria_bearer_token() + +folder_guid = c.create_folder_element_from_template( + path_name="/tmp/some-new-path", folder_name="probe-folder", file_system="localhost") + +# Note: initiate_file_folder_survey's own default survey_name +# ("FileSurveys:survey-folder") and the .http ground truth's example +# ("AssetSurvey:survey-folder") are BOTH wrong on this server -- neither +# GovernanceActionType is registered (OMAG-GENERIC-HANDLERS-400-013). The +# real registered qualifiedName, confirmed via +# `EgeriaTech.get_elements("GovernanceActionType")`, is "FileSurvey::survey-folder". +action_guid = c.initiate_file_folder_survey(folder_guid, survey_name="FileSurvey::survey-folder") +``` + +Poll the resulting `EngineAction` (`EgeriaTech.get_elements("EngineAction")`, +match on `elementHeader.guid`) — it completes (fast, ~1s) with +`activityStatus: "FAILED"` and `completionGuards: ["survey-failed"]`. Full +`completionMessage`: +``` +OMES-SURVEY-ACTION-0018 The survey action service folder-survey-service +threw a org.odpi.openmetadata.frameworks.connectors.ffdc.ConnectorCheckedException +exception during the generation of survey report ... for asset + during request type survey-folder in survey action engine +FileSurvey (guid=4168abb9-6c60-46fb-b9c0-b44180d19500). The error message +was OPEN-SURVEY-500-001 Unexpected exception in survey action service +folder-survey-service of type java.lang.NullPointerException detected by +method start. The error message was Cannot invoke +"org.odpi.openmetadata.adapters.connectors.datastore.basicfile.BasicFolderConnector.getFile()" +because "assetConnector" is null +``` + +**Analysis.** `create_folder_element_from_template` creates the `FileFolder` +element and it looks entirely normal (confirmed via `get_elements`: proper +`pathName`, `resourceName`, `Anchors` classification) — but no working +`Connection`/`Connector` got wired to it, so when `folder-survey-service` +tries to open the asset via `BasicFolderConnector.getFile()`, the +connector reference is null and the survey action service crashes with an +NPE rather than a clean error. This matches the design doc's +characterization exactly ("template-based asset creation never wires up a +working Connection, so any native survey against a template-created asset +fails server-side with a NullPointerException") — just with the precise +class/method now on record, which the design doc didn't have. + +**Impact:** blocks Resource Explorer's engine-host participation design +(`re-as-engine-host-plan.md`, cases 1/2/4 — anything needing a native +survey to actually *complete*, not just be triggered; case 3, RE-local +surveying, is unaffected). The hold should be re-evaluated against this +entry specifically, not against the old ISSUE-51 number. As of 2026-08-28 +this is confirmed the *only* thing still blocking that work on the +pyegeria side — ISSUE-78's engine-host method trio shipped in 6.1.5 and RE +has upgraded and verified it end to end, and the two other gaps +identified alongside it (no REST way to create a `GovernanceEngine`/ +`GovernanceService` element; no per-engine "claimable work" listing, +`get_active_engine_actions()` filtered client-side is the only option) are +both genuine server-side gaps, not pyegeria ones. This entry is the real +holdup. + +**Leftover from this investigation:** probe `FileFolder` elements +(`qualifiedName: "FileFolder::localhost:/tmp/re-e1-probe-*"` from +2026-08-27, `"FileFolder::localhost:/tmp/re-issue79-recheck-*"` from the +2026-08-28 re-check) on `qs-metadata-store`, plus their failed +`SurveyReport`/`EngineAction` pairs, left in place as reproduction +evidence for both dates — not yet deleted. + +**Update 2026-10-05 -- probably a pyegeria bug, fixed on our side:** the Egeria team asked us to check that pyegeria did not send `deepCopy:false`. It did: `TemplateRequestBody.deep_copy` defaulted to `False` from 2026-01-09 until fixed on 2026-09-29 (#399, first released in v6.1.23), the model is serialised with `exclude_none` (which keeps `False`), and `create_folder_element_from_template` builds its body without `deepCopy`. Egeria then copies only the top-level element and none of the template's anchored attachments, so no `ResourceConnection`, which is exactly a null `assetConnector`. The Egeria team also points to `e01426db86` (2026-08-30, PostgreSQL supertype-chain repair after the 08-27 `ResourceConnection` re-parenting). **To confirm:** create a folder from the template with current pyegeria, check it has a `ResourceConnection`, then survey it. + +--- + +### ISSUE-38 (PY-18): `count_relationships_between_elements("Exception")` (276) disagrees with `ClassificationExplorer.get_relationships("Exception")` (55) + +**Update 2026-08-30, from the Egeria team (Mandy Chessell).** Leaving this +entry in "Open Egeria Server issues" until the verification below has been +run against real data — the Egeria side is done, but this entry has been +re-measured and re-confirmed several times, and it shouldn't move to +Fixed/Resolved on the strength of a change nobody has yet pointed at the +data that produced the original report. + +**Status: resolved on the Egeria side 2026-08-30** (`omf-metadata-spring`, +`omf-metadata-server`, `generic-handlers`, and the OMF Java client) — +**awaiting verification from the pyegeria side**, see below. The two +numbers are not going to be made equal, because they are answers to two +different questions and both are wanted; what has changed is that the +caller now chooses which one they get. + +**What the difference actually was.** `findAttachmentLinks()` puts every +relationship it retrieves through a visibility check on the entities at +both ends, and silently drops the ones whose anchor the caller cannot +read. `countAttachmentLinks()` skipped that check — deliberately, and +documented as such in its own javadoc, because applying it means fetching +every matching relationship, which is the cost the count exists to avoid +— and also ignored `effectiveTime`. So the count was of what matches the +search and the list was of what the caller may see. That answers this +entry's open question ("what does the metadata-expert count include for +`Exception` that the classification-explorer traversal excludes?"): +relationships with an end whose anchor is not readable by the calling +user, plus anything outside the effectivity window. It also explains why +only `Exception` diverged among the types tested — it is the type whose +ends are most likely to have been deleted or anchored to something the +caller cannot see — and why the divergence tracked the demo data rather +than staying at a fixed size. + +**The new option.** Both counting endpoints take a `pushDown` query +parameter, `required=false`, default `true`: +``` +POST .../users/{userId}/relationships/by-search-conditions/count?pushDown=false +POST .../users/{userId}/metadata-elements/by-search-conditions/count?pushDown=false +``` +- `pushDown=true` (the default, and exactly the existing behaviour) — the + repository counts the matching rows itself. Fast, and counts what + matches the search. +- `pushDown=false` — the relationships are retrieved and counted, so the + answer agrees with the list by construction, at the cost of reading + every one of them. + +Nothing changes for a caller that does not send the parameter. The same +choice is on the Java client (`countMetadataElements(..., pushDown)` and +`countRelationshipsBetweenMetadataElements(..., pushDown)` on +`OpenMetadataStore` and `OpenMetadataClient`), with the existing +signatures kept and defaulting to `true`. + +**The test the pyegeria side should run to prove it.** Against the +environment that produced the original numbers, using whatever the +current counts are rather than the 58/57 recorded here — the demo data +has moved several times during this entry's life, and what matters is the +relationship between the numbers, not their values: + +1. **The accurate count should equal the list.** For + `relationshipTypeName: "Exception"`, the count with `pushDown=false` + should equal `len(ClassificationExplorer.get_relationships("Exception"))` + exactly. This is the assertion that closes the entry — it is the + discrepancy this entry is about, stated as an equality. +2. **The fast count should be unchanged.** The same call with + `pushDown=true`, and with the parameter omitted altogether, should both + return what the count returns today. If either moves, something + regressed for every existing caller. +3. **A type that already agreed should still agree both ways.** + `SemanticAssignment` matched at 401=401 when this entry was last + measured. Both `pushDown` values should return that same number — the + retrieval route must not lose or double-count anything on a type where + there was never a discrepancy to resolve. + +Worth capturing while verifying: the difference between the two counts is +the number of matching relationships the calling user cannot fully see. +If it is not the 1 recorded here, that is not a failure — it is a +measurement of the current data — but it is worth writing down, because a +large difference on a type that should be fully visible would be a separate finding. If test 1 comes out unequal, the remaining difference is not the visibility check and this entry should stay open with both numbers recorded. @@ -954,342 +1107,90 @@ native counting only for **element** counts. --- -### ISSUE-79: native survey against a template-created `FileFolder` asset fails server-side — `assetConnector` is null in `BasicFolderConnector.getFile()` +## Open Egeria Server issues (not fixable in pyegeria) -**Live check 2026-10-06 (about 02:00Z, current pyegeria 6.1.29 code, rebuilt platform): the `deepCopy:false` default WAS the cause of the missing connection.** `create_folder_element_from_template` now sends a request with no `deepCopy` key (keys sent: allowRetrieve, class, forDuplicateProcessing, forLineage, isOwnAnchor, parentAtEnd1, placeholderPropertyValues, templateGUID, templateSubstitute), and the new folder read back through the classification explorer **has 1 connection** (`ResourceConnection` present). Folder deleted (cascade) and verified gone by GUID. **Not run: the survey itself** (a survey creates elements owned by the survey identity that an ordinary user may not be able to delete), so "the survey now completes" is still unproven; the original `assetConnector`-null failure was the symptom of no connection, which is now shown not to occur. Whether the Egeria-side repair (`e01426db86`) also matters is untested. Script: `live_79.py` stage A (stage B `--survey` is opt-in). -**Update 2026-08-30, from the Egeria team (Mandy Chessell).** The entry -stays in "Open Egeria Server issues" — the reported failure is not -explained and not fixed. What has changed is that the cause recorded in -this entry, and in Resource Explorer's `re-as-engine-host-plan.md`, is now -known to be wrong, and the error it produces has been made legible. +These need an Egeria Server-side fix, an upstream bug report, or a +deployment/config change — nothing here can be resolved by editing this +repo. Where an entry has a known pyegeria/Dr.Egeria follow-on once the +Egeria side lands, it's noted inline (also cross-referenced from the +"Open pyegeria items" section below where the follow-on is substantial +enough to track there too). -**Status: still open (Egeria Server)** — but the recorded cause is -disproved, and this entry and `re-as-engine-host-plan.md` should both be -corrected. Investigated 2026-08-30 from the Egeria side, including a new -FVT suite built to run this exact scenario. - -**The premise is false.** This entry, and the design doc's hold, rest on -"template-based asset creation never wires up a working Connection." That -is not true, in either environment it was checked against: - -- **On the quickstart that produced the report.** Every element created - from a template that itself carries a `ResourceConnection` got one — 10 - of 10. Every `FileFolder` in that repository has one, including all - three that were created from a template (`/deployments/content-packs`, - `/deployments/loading-bay/sample-data`, `/deployments/secrets`). -- **In a clean environment built from the current archives.** A folder - catalogued through `FileGovernance::create-file-folder` came back with a - `ResourceConnection` attached. -- **The `FilesContentPack` template itself is also well formed:** - `FileFolder` template `fbdd8efd-...` carries an `Anchors`-classified - `Connection` with both a `ConnectToEndpoint` and a - `ConnectionConnectorType`. - -**The folder survey works.** A new `files-fvt` suite runs the reported -scenario end to end — catalogue a folder from the `FileFolder` template, -then survey it natively — and it passes: -``` -OMES-SURVEY-ACTION-0019 The survey action service folder-survey-service has completed the -analysis of asset 7050dc50-... with request type survey-folder in 1927 milliseconds; the -results are stored in survey report 7e7892e2-... -``` -The report carried six annotations. The test asserts the survey -*completed*, not merely that it ran: a survey that dies on a null -connector still produces an engine action, and the waiter throws on any -terminal status other than `COMPLETED`. - -**A second hypothesis was also eliminated.** While investigating, a real -defect was found in the PostgreSQL repository: a `TypeDefPatch` that -reparents a type does not update the denormalised supertype chain stored -on existing rows, so instances written before the patch become invisible -to searches for the new supertype. That would have hidden connections -exactly as reported, since `AssetConnection` was reparented under -`ResourceConnection` on 2026-08-27. It is fixed — but it was never the -cause here: the quickstart repository stores 202 rows as -`:ResourceConnection:LabeledRelationship:` and none as -`:AssetConnection:`, because it was reloaded after the type change. +--- -**What has been fixed, and what it does and does not buy.** The NPE -itself is now a clean error. `getConnectorForAsset()` returns `null` by -contract when an asset has no connection; `FolderSurveyService` cast that -null and dereferenced it, and the framework's own -`performCheckAssetAnalysisStep()` had the same hole on its error path. -Both now report `OPEN-SURVEY-400-008` / `OPEN-SURVEY-0009` -(`NO_ASSET_CONNECTOR`), naming the asset. +### ISSUE-130: The YAML secrets store's user directory caches a user forever, so editing an existing user needs a platform restart -This does not make such a survey succeed — an asset with no connection -describes a resource nothing can open. It makes the failure say *which* -asset, which is what the original report was missing: the NPE named -`BasicFolderConnector`, a class the caller had never heard of, and pointed -away from the asset. +**Layer:** Egeria server (`YAMLSecretsStoreConnector`) · **Status:** open, reported to the Egeria lead · **Found:** 2026-10-09, while adding Resource Explorer's own account (`resourceexplorernpa`). -**Why that matters for how this entry reads.** Four further defects were -found in the same connectors, all on paths nothing else exercises: both -file survey services failed on every invocation (annotations created -without a qualified name, in three places), the CSV survey dereferenced a -null root schema type, wrote a relationship property to an entity, and -produced colliding annotation qualified names. The file survey path had, -on this evidence, never run to completion. Someone hitting an NPE in this -family had every reason to suspect the asset — the errors never named -anything else. That is context for the original diagnosis, not a -criticism of it. +**What:** `getUser(userId)` returns the entry from `userAccountMap` if it is already there. `refreshSecrets()` re-reads the YAML file every `refreshTimeInterval` minutes, but `userAccountMap` (and `securityAccessControlMap`) are created only on the first load (when `secretsStoreFile == null`) and are never cleared afterwards. Re-reading the file only helps users the cache has never seen; the comment says "this may be a new user". -**The naming trap, which is directly actionable on the pyegeria side.** -This entry already records that `initiate_file_folder_survey`'s default -(`FileSurveys:survey-folder`) and the `.http` example -(`AssetSurvey:survey-folder`) are both wrong. Confirmed and now -understood: a governance action type is registered under -`::` — two colons — so the real name is -`FileSurvey::survey-folder`. `AssetSurvey` is not an engine that exists at -all. The `.http` examples have been corrected upstream -(`FileSurvey::survey-folder`, `ApacheKafkaSurvey::survey-kafka-server`, -`PostgreSQLSurvey::survey-postgres-server`, -`PostgreSQLSurvey::survey-postgres-database`) with a note stating the -shape, because the error a wrong name produces — -`OMAG-GENERIC-HANDLERS-400-013`, "the name is not recognized" — does not -say what it wanted instead. `initiate_file_folder_survey`'s default still -needs fixing in pyegeria. +**Seen:** the account was first added without its `secrets` block, and the first token request cached it. After the password was added (and matched the servers file), `/api/token` still answered 401 for more than 10 minutes. It worked straight after a platform restart. -**What is left, and what would settle it.** The remaining candidate is -the deployment rather than the cataloguing or the survey: which server -and userId the engine host's connected-asset client resolves -`SurveyAssetStore.getConnectorToAsset()` against. That cannot be tested -from the Egeria side — it needs the engine host that produced the -failure. Suggested, in order: +**Effect:** any change to an existing user (password, status, named-list membership) needs a platform restart. -1. Re-run the original repro against a build carrying the NPE fix. If the - asset genuinely has no connection the run now fails with - `OPEN-SURVEY-400-008` naming it, which turns the question from "why is - this connector null" into "why does this asset have no connection that - this userId can see". If instead the survey completes, the deployment - has moved on and this entry can be closed. -2. Check the folder asset at the moment of failure for a - `ResourceConnection`, using the same userId the engine host runs as — - not an admin one. If the relationship is there but the survey cannot - see it, that is a visibility or server-routing problem, and the userId - is where to look next. -3. Compare against `files-fvt`, which is a known-good baseline for this - exact scenario. Pointing the same two actions at a - differently-configured engine host isolates the variable. +**Fix (Egeria):** clear or rebuild `userAccountMap` and `securityAccessControlMap` in `refreshSecrets()`. -Also worth correcting: this entry's earlier note about leftover probe -elements (`FileFolder::localhost:/tmp/re-e1-probe-*`, -`FileFolder::localhost:/tmp/re-issue79-recheck-*`) is out of date — the -quickstart repository has been reloaded since and holds none of them, so -that evidence is gone. +--- -**Status:** open (Egeria Server), reproduced live 2026-08-27, **re-checked -2026-08-28 — still reproduces, identical failure.** Same class/method -(`BasicFolderConnector.getFile()`, `assetConnector` null), same error IDs -(`OMES-SURVEY-ACTION-0018`/`OPEN-SURVEY-500-001`), same -`java.lang.NullPointerException`, against a freshly created `FileFolder` -and a fresh survey run — not a one-off. Filed by content, not by number — -see "why this entry exists" below. +### ISSUE-128: Retention cannot be set on Egeria 6.2: the server registers the Jackson subtype as `RetentionProperties.class` (typo), and pyegeria only allows `RetentionClassificationProperties` -**Layer:** Egeria Server (survey action framework / `BasicFolderConnector`), -not pyegeria. `create_folder_element_from_template` and the survey-initiate -call both succeed and are correctly formed; the failure happens entirely -inside the server's survey action service. +**Status:** open, found 2026-10-09 by Resource Explorer's first Retention publish on Egeria 6.2. **Layer:** Egeria server (a type registration typo). **Decision (owner, 2026-10-09):** wait for the Egeria fix; no pyegeria workaround. Retention publishing stays blocked until then. -**Why this entry exists.** Resource Explorer's `re-as-engine-host-plan.md` -is ON HOLD citing this exact symptom ("connector is null" on a native -survey against a template-created asset) under the name `PYEGERIA_ISSUES.md` -**ISSUE-51** — but this file was renumbered on 2026-08-15, and today's -ISSUE-51 is an unrelated, already-fixed `fetch_element()` shape problem. -The blocker's own content had no surviving entry anywhere in this file. -Investigated fresh rather than guessed at, per the plan's own citation -being untrustworthy. +**What (all proven live on 6.2, with raw HTTP POSTs to `classification-explorer/elements/{guid}/retention` on a throwaway asset):** +- `"class": "RetentionClassificationProperties"` is refused with `InvalidTypeIdException`. This is what pyegeria documents and whitelists, and it worked on the pre-6.2 server (processors.py comment, 2026-08-03). +- `"class": "RetentionProperties"` (the `.http` ground truth) is also refused with `InvalidTypeIdException`. +- `"class": "RetentionProperties.class"` gets **HTTP 200**, and the classification reads back with `retentionBasis` 2 and the notes. +- The server's known-type list in the error has 96 ids. The only Retention entry is literally `RetentionProperties.class`. That looks like an `@JsonSubTypes.Type(name = "RetentionProperties.class")` typo in Egeria's `ClassificationProperties`. +- pyegeria's `validate_new_classification_request` whitelist (`prop=["RetentionClassificationProperties"]`) refuses anything else client-side, so no Retention can be set through pyegeria on 6.2. -**Two candidate bugs were in play, and they are not the same bug:** +(An earlier version of this entry said 6.2 knows `RetentionProperties`. That was a misread of the error text, corrected the same day.) -1. **Resource Explorer's superseded tracker, entry E1** (`packages/ - resource-explorer/docs/egeria-pyegeria-issues.md`) describes a *different* - symptom: a 500 from the PostgreSQL repository connector on - `createMetadataElementFromTemplate` (`metadata_collection_guid` null - while saving a classification) — a failure to *create* the element at - all. **Re-tested live 2026-08-27 and does not reproduce**: - `create_folder_element_from_template(path_name=..., folder_name=..., - file_system="localhost")` against `qs-view-server` returned a real GUID - with no error. Whatever this was, it looks fixed. -2. **The design doc's actual blocker** — a template-created asset surveyed - natively fails with a null connector — **does still reproduce**, and is - what this entry tracks. +**Where seen:** Resource Explorer's Curate classifications step on two repository assets. Both were refused, so nothing was written. Confidentiality, Criticality, Confidence and Impact property classes are registered normally. -**Reproduction** (against `qs-view-server`, no pyegeria bug involved): -```python -from pyegeria import AutomatedCuration -c = AutomatedCuration("qs-view-server", "https://localhost:9443", "erinoverview", "secret") -c.create_egeria_bearer_token() +**Candidate fix:** +1. **Egeria:** correct the subtype name to `RetentionProperties`, matching the `.http` file. Report it to Mandy. +2. **pyegeria, after Egeria's fix:** change the whitelist and docstring to `RetentionProperties`, the `.http` name. No workaround for the typo, by the owner's decision. +3. Re-check `md_processing/v2/processors.py` and `v2/curation.py`, which both pin `RetentionClassificationProperties`. -folder_guid = c.create_folder_element_from_template( - path_name="/tmp/some-new-path", folder_name="probe-folder", file_system="localhost") +--- -# Note: initiate_file_folder_survey's own default survey_name -# ("FileSurveys:survey-folder") and the .http ground truth's example -# ("AssetSurvey:survey-folder") are BOTH wrong on this server -- neither -# GovernanceActionType is registered (OMAG-GENERIC-HANDLERS-400-013). The -# real registered qualifiedName, confirmed via -# `EgeriaTech.get_elements("GovernanceActionType")`, is "FileSurvey::survey-folder". -action_guid = c.initiate_file_folder_survey(folder_guid, survey_name="FileSurvey::survey-folder") -``` +### ISSUE-108: newly created relationships stay invisible to related-element queries for up to ~20 minutes after creation -Poll the resulting `EngineAction` (`EgeriaTech.get_elements("EngineAction")`, -match on `elementHeader.guid`) — it completes (fast, ~1s) with -`activityStatus: "FAILED"` and `completionGuards: ["survey-failed"]`. Full -`completionMessage`: -``` -OMES-SURVEY-ACTION-0018 The survey action service folder-survey-service -threw a org.odpi.openmetadata.frameworks.connectors.ffdc.ConnectorCheckedException -exception during the generation of survey report ... for asset - during request type survey-folder in survey action engine -FileSurvey (guid=4168abb9-6c60-46fb-b9c0-b44180d19500). The error message -was OPEN-SURVEY-500-001 Unexpected exception in survey action service -folder-survey-service of type java.lang.NullPointerException detected by -method start. The error message was Cannot invoke -"org.odpi.openmetadata.adapters.connectors.datastore.basicfile.BasicFolderConnector.getFile()" -because "assetConnector" is null -``` +**Layer:** Egeria Server · **Status:** open · **Found:** 2026-09-20, reported by +the user from a Dr.Egeria bulk load session (1,027 expected field-link +relationships). -**Analysis.** `create_folder_element_from_template` creates the `FileFolder` -element and it looks entirely normal (confirmed via `get_elements`: proper -`pathName`, `resourceName`, `Anchors` classification) — but no working -`Connection`/`Connector` got wired to it, so when `folder-survey-service` -tries to open the asset via `BasicFolderConnector.getFile()`, the -connector reference is null and the survey action service crashes with an -NPE rather than a clean error. This matches the design doc's -characterization exactly ("template-based asset creation never wires up a -working Connection, so any native survey against a template-created asset -fails server-side with a NullPointerException") — just with the precise -class/method now on record, which the design doc didn't have. +**What:** a catalog walk run immediately after a load found only 876 of 1,027 +expected field links. A second walk 12 minutes later still showed 82 +structures short. A third walk, 20 minutes after the load, showed every link +present. No errors were raised at any point in the load itself — the +relationships were created successfully, they simply did not show up in +related-element queries for a long window afterward, then all became visible +at once with no further action taken. -**Impact:** blocks Resource Explorer's engine-host participation design -(`re-as-engine-host-plan.md`, cases 1/2/4 — anything needing a native -survey to actually *complete*, not just be triggered; case 3, RE-local -surveying, is unaffected). The hold should be re-evaluated against this -entry specifically, not against the old ISSUE-51 number. As of 2026-08-28 -this is confirmed the *only* thing still blocking that work on the -pyegeria side — ISSUE-78's engine-host method trio shipped in 6.1.5 and RE -has upgraded and verified it end to end, and the two other gaps -identified alongside it (no REST way to create a `GovernanceEngine`/ -`GovernanceService` element; no per-engine "claimable work" listing, -`get_active_engine_actions()` filtered client-side is the only option) are -both genuine server-side gaps, not pyegeria ones. This entry is the real -holdup. +**Not yet root-caused.** This looks like the same family of symptom as the +2026-08-15 Postgres-checkpoint/background-connector-load entry above (server +falling behind under load, catching up later) rather than a paging defect +like ISSUE-54 (which would not self-heal purely by waiting — ISSUE-54's +bug is about tie-ordering across separately-executed pages, not about data +not existing yet). But this has not been directly correlated with connector +log activity or checkpoint timing the way that entry was — this is a fresh +report, not yet independently reproduced or diagnosed by a pyegeria session. -**Leftover from this investigation:** probe `FileFolder` elements -(`qualifiedName: "FileFolder::localhost:/tmp/re-e1-probe-*"` from -2026-08-27, `"FileFolder::localhost:/tmp/re-issue79-recheck-*"` from the -2026-08-28 re-check) on `qs-metadata-store`, plus their failed -`SurveyReport`/`EngineAction` pairs, left in place as reproduction -evidence for both dates — not yet deleted. +**Impact:** any workflow that verifies a bulk write immediately after +issuing it (a post-load catalog walk, an integration test, an automated +Dr.Egeria "did it work" check) can get a false-negative "missing" result +that resolves itself given enough time — worth knowing before treating an +immediate post-write verification gap as evidence of a real data-loss bug. -**Update 2026-10-05 -- probably a pyegeria bug, fixed on our side:** the Egeria team asked us to check that pyegeria did not send `deepCopy:false`. It did: `TemplateRequestBody.deep_copy` defaulted to `False` from 2026-01-09 until fixed on 2026-09-29 (#399, first released in v6.1.23), the model is serialised with `exclude_none` (which keeps `False`), and `create_folder_element_from_template` builds its body without `deepCopy`. Egeria then copies only the top-level element and none of the template's anchored attachments, so no `ResourceConnection`, which is exactly a null `assetConnector`. The Egeria team also points to `e01426db86` (2026-08-30, PostgreSQL supertype-chain repair after the 08-27 `ResourceConnection` re-parenting). **To confirm:** create a folder from the template with current pyegeria, check it has a `ResourceConnection`, then survey it. - ---- - -### ISSUE-85: No REST endpoint exists to create a `SolutionPort` element — only attach/detach/delegation relationships are exposed - -**Status:** open, confirmed Egeria-side gap 2026-09-03. Found by a -consumer (trellis/Resource Explorer, designing a `PortMaterializer` for -architecture-recovery proposals) hitting the same shape of gap ISSUE-84 -turned out not to be: this time the `.http` ground truth's absence really -does match the live server, not a stale/incomplete `.http` file. - -**Confirmed at every layer:** -- `pyegeria/omvs/solution_architect.py`: `link_solution_component_port` - (attach an *existing* port to a component), `link_solution_port_delegation` - (port-to-port delegation), and both detach counterparts exist. No - `create_solution_port`/`_async_create_solution_port` anywhere. -- `Egeria-api-solution-architect.http`: only `.../solution-components/ - {guid}/solution-ports/{guid}/attach|detach` and `.../solution-ports/ - {guid}/port-delegations/{guid}/attach|detach`. No `POST .../solution-ports` - create path. -- `md_processing/data/compact_commands/commands_solution_architect_compact.json`: - only `Link Solution Component Port`/`Link Solution Port Delegation`. No - `Create Solution Port` command. -- `pyegeria/models/models.py`: no `SolutionPortProperties` model at all — - so even the generic fallback route has nothing typed to build a body - against. - -**Live-verified not just a stale-`.http`-file case** (the precedent this -repo already has for that shape of false alarm — see the multi-link audit -above, where 3 of 4 similarly-unwrapped relationship types turned out to -have zero real endpoint on live-server inspection while others did): -`dwolfson-1b`/trellis fetched the full live OpenAPI spec directly -(`GET /v3/api-docs`, 2.39MB) and searched every path containing "port" — -30 POST paths matched, every one of them attach/detach (component-port, -port-delegation, even a generic process-ports attach under a different -`{urlMarker}` service). Zero creation endpoints anywhere in the spec. -This confirms the `.http` ground truth exactly; not a documentation gap. - -**The generic fallback exists but isn't usable without guessing:** -`/servers/{serverName}/api/open-metadata/{urlMarker}/metadata-elements` -(`MetadataExpert.create_metadata_element`, -`NewOpenMetadataElementRequestBody`) is real and live-confirmed present, -with a top-level `type_name` field that could carry `"SolutionPort"`. But -per this file's own `pyegeria/models/models.py` gotcha, that endpoint's -`properties` field needs the verbose nested `ElementProperties`/ -`propertyValueMap` shape server-side, not the flat dict every bespoke -wrapper uses — and with no `SolutionPortProperties` model to check -property names against, building that body means hand-guessing what -properties a `SolutionPort` actually takes. Consistent with this repo's -established practice (see the multi-link audit's "left unbuilt rather -than guessing at undocumented URLs or building against the awkward -generic path speculatively") — not attempted. - -**Nothing to fix in pyegeria until Egeria adds a real create endpoint.** -Once one exists, the follow-on work is small and well-precedented: a -`create_solution_port`/`_async_create_solution_port` wrapper in -`SolutionArchitect` following the exact pattern ISSUE-84 just corrected -(`NewElementRequestBody` + `contentStatus` inside `properties`, no -fictional separate request-body class), plus a `Create Solution Port` -Dr.Egeria compact command — the family's registration plumbing in -`register_solution_architect_processors()` (`md_processing/dr_egeria.py`) -already handles this shape of addition with no new routing logic needed. - -**Not blocking anything urgent** on the trellis/Resource Explorer side — -they're holding real port materialization until this exists server-side -(tracked as their own Backlog item 8), or until a workaround surfaces. - ---- - -### ISSUE-108: newly created relationships stay invisible to related-element queries for up to ~20 minutes after creation - -**Layer:** Egeria Server · **Status:** open · **Found:** 2026-09-20, reported by -the user from a Dr.Egeria bulk load session (1,027 expected field-link -relationships). - -**What:** a catalog walk run immediately after a load found only 876 of 1,027 -expected field links. A second walk 12 minutes later still showed 82 -structures short. A third walk, 20 minutes after the load, showed every link -present. No errors were raised at any point in the load itself — the -relationships were created successfully, they simply did not show up in -related-element queries for a long window afterward, then all became visible -at once with no further action taken. - -**Not yet root-caused.** This looks like the same family of symptom as the -2026-08-15 Postgres-checkpoint/background-connector-load entry above (server -falling behind under load, catching up later) rather than a paging defect -like ISSUE-54 (which would not self-heal purely by waiting — ISSUE-54's -bug is about tie-ordering across separately-executed pages, not about data -not existing yet). But this has not been directly correlated with connector -log activity or checkpoint timing the way that entry was — this is a fresh -report, not yet independently reproduced or diagnosed by a pyegeria session. - -**Impact:** any workflow that verifies a bulk write immediately after -issuing it (a post-load catalog walk, an integration test, an automated -Dr.Egeria "did it work" check) can get a false-negative "missing" result -that resolves itself given enough time — worth knowing before treating an -immediate post-write verification gap as evidence of a real data-loss bug. - -**Candidate next step (not yet done):** reproduce with a smaller, timed -sample (e.g. create N relationships, poll related-element queries at 1-minute -intervals, log when each becomes visible) to get a real latency -distribution instead of the three anecdotal checkpoints here, and check -whether it correlates with the same background-connector/checkpoint load -already documented above. Worth raising with the Egeria team directly, per -the original report. +**Candidate next step (not yet done):** reproduce with a smaller, timed +sample (e.g. create N relationships, poll related-element queries at 1-minute +intervals, log when each becomes visible) to get a real latency +distribution instead of the three anecdotal checkpoints here, and check +whether it correlates with the same background-connector/checkpoint load +already documented above. Worth raising with the Egeria team directly, per +the original report. **Update 2026-09-20 (peer check, no upstream issue filed yet):** checked `odpi/egeria`'s issue tracker directly — nothing filed for this symptom, so @@ -1393,590 +1294,568 @@ actually test the volume-proportional hypothesis — this run didn't attempt that scale on its own initiative, since committing to it wasn't part of what peers were asked to clear. ---- - -## Open pyegeria items (including follow-ons blocked on an Egeria fix) - -Actionable in this repo. Some of these are fully blocked today — waiting -on an Egeria Server capability that doesn't exist yet — but the pyegeria/ -Dr.Egeria-side work each will need once that capability ships is written -into the entry now, so it isn't rediscovered from scratch later. - **Egeria team response, relayed 2026-10-05 (source `oak2026` fb3d6fce53; **fixes are in two unmerged PRs, so no build contains them, and nothing was tested against a live platform**):** not validated; no cache found. Best lead: a client clock ahead of the server's, with `effectiveFrom` set from it (related-element queries silently hide a relationship whose `effectiveFrom` is later than the server's "now"). They need, for one link still missing at ~12 minutes: its `effectiveFrom` and `createTime`, the server clock then, and whether the loader sets `effectiveFrom`. **Our facts:** pyegeria sets no `effectiveFrom`/`effectiveTo`/`effectiveTime` by default (model defaults are `None`; no clock call is tied to any `effective*` field; Dr.Egeria forwards an "Effective From" only if typed). Host and container clocks agree within 1 s on 2026-10-05; the 2026-09-20 load's values are unavailable. **Update 2026-10-10:** the `oak2026` source (`fb3d6fce53`) the team examined is now in Egeria `main` (merged 2026-10-06, odpi/egeria#9363). No fix was made for this issue; the team's questions above still stand. +--- -### ISSUE-126: `ProductManager` and `DigitalBusiness` never set `collection_command_root`, so every inherited `CollectionManager` method that uses it raises `AttributeError` - -**Layer:** pyegeria · **Status:** fixed in #429 (`68279835`), released in 6.1.29 · **Found:** 2026-10-05, live cleanup of a throwaway data contract (PR #427's -live check). - -**Fix + scope (2026-10-05):** both constructors now call `CollectionManager.__init__`, as `GlossaryManager` -does; that also restores `metadata_expert_command_root`, which the SmartQuery relationship methods use. An AST scan of all 91 classes in -`pyegeria/omvs/` found no other subclass whose `__init__` skips its base's, and -`test_every_omvs_subclass_runs_its_base_constructor` now fails if one ever does. +### ISSUE-95: No catalog template registered for the "Apache Kafka Server" technology type — `Create Kafka Server Element` (Asset Maker) fails with a 400 on `qs-view-server` -**What:** `ProductManager.__init__` and `DigitalBusiness.__init__` call `ServerClient.__init__` directly instead -of `CollectionManager.__init__` (`GlossaryManager` does it correctly). `CollectionManager.__init__` is what sets -`self.collection_command_root`, which 27 of `CollectionManager`'s methods use, so on those two subclasses all of -them fail: +**Layer:** Egeria Server (deployment/archive content) · **Status:** open, being investigated by the user (2026-09-11) · **Found:** 2026-09-11, live-verifying the new Asset Maker Dr.Egeria family (#354) -```python -pm = ProductManager(view_server, platform_url, user, pwd) -pm.delete_collection(guid, cascade=True) -# AttributeError: 'ProductManager' object has no attribute 'collection_command_root' -``` +`AutomatedCuration._async_get_template_guid_for_technology_type("Apache Kafka Server")` +returns `None` on `qs-view-server` — no catalog template is registered for +that technology type on this server — so `Create Kafka Server Element`'s +underlying `_async_create_kafka_server_element_from_template` posts a body +with `templateGUID: null` and gets back `VALIDATION_ERROR_1` ("Request body +failed validation"). Not a pyegeria code defect: every other Asset Maker +technology type tried resolved a real template GUID on the same server +(CSV Data File, PostgreSQL Server, File System Directory, YAML File Secrets +Collection, Unity Catalog Server) — this one is specific to the Kafka +technology type's archive content on this deployment. `Create Kafka Server +Element` itself is otherwise correct (parses/validates cleanly; the failure +is entirely server-side at execution). -**Fixed in #429:** see "Fix + scope" above. +**Ask:** confirm whether the Apache Kafka Server catalog template is +expected to ship with quickstart's default archive content, and if so, why +it's missing here — or add it to whichever archive/content-pack is meant to +carry it. --- -### ISSUE-127: `CollectionManager.delete_collection(cascade=True)` silently never sends `cascadeDelete` -- same shape as ISSUE-62 - -**Layer:** pyegeria · **Status:** partly fixed in #429 (`68279835`), released in 6.1.29; cascade still blocked by ISSUE-132 · **Found:** 2026-10-05, found by capturing the request body while diagnosing a -failed cleanup. +### ISSUE-89: No configurable bearer-token lifetime in `application.properties` — platform tokens are fixed at 3600 s, and the signing key is random per restart -**Fix + scope (2026-10-05):** a scan of all 42 callers of `_async_delete_element_request` found exactly two -that pre-fill a default body *and* pass a cascade flag: `CollectionManager._async_delete_collection` and -`ProductManager._async_delete_digital_product` (so `delete_digital_product(cascade=True)` was never cascading -either -- the live check's own product cleanup included). Both now pass `None` so the helper applies the flag. -`delete_digital_product`'s docstring sample also used the wrong key (`cascadedDelete`); corrected. The helper -itself is unchanged: a caller that passes an explicit body dict *and* `cascade=True` still gets the body's -value, which is documented behaviour ("body supersedes"). +**Layer:** Egeria Server · **Status:** open · **Found:** 2026-09-04 (trellis-auth token contract) -**What:** `_async_delete_collection` always pre-fills `body = {"class": "DeleteElementRequestBody"}` when none is -given and hands that dict to `_async_delete_element_request(url, body, cascade)`. -`validate_delete_element_request` applies `cascade_delete` only in its *no body* branch, so with that pre-filled dict -the flag is discarded. Captured: `delete_collection("g", cascade=True)` posts -`{'class': 'DeleteElementRequestBody', 'forLineage': False, 'forDuplicateProcessing': False}` with no -`cascadeDelete`. `MetadataExpert.delete_metadata_element(cascade_delete=True)` does send it. +With `authentication.mode=token`, every token minted by the platform decodes to `exp - iat = 3600` +(observed on `omag-server-platform-6.2-SNAPSHOT`, quickstart, `authentication.source=platform`). +`application.properties` exposes `authentication.mode`, `authentication.source` and `rsa.key-id`, and +nothing that controls the lifetime; with `rsa.key-id` empty the signing key is regenerated at start, so +every token also dies on a platform restart. Client applications that hold a platform token on the +user's behalf (Portal, Egeria Advisor, Resource Explorer — which now carry the Egeria bearer token in +their session and never the password) cannot offer a session longer than an hour without either +re-prompting the user or caching a password, which is what a bearer token exists to avoid. -**Fixed in #429, but cascade still does not reach Egeria -- see ISSUE-132.** #429 made the flag reach the request -body, but under the name `cascadeDelete`. Egeria's real field is `cascadedDelete`, and it ignores unknown fields, so -the flag is still dropped, now on the server side. #429's docstring "correction" from `cascadedDelete` to -`cascadeDelete` was also the wrong way round. The `DataStructure` refusal described below is probably this bug -rather than an Egeria quirk; re-test once ISSUE-132 is fixed. +**Ask:** a property such as `authentication.token.lifetime=3600` (seconds, default unchanged), read at +start and applied when a token is issued; optionally a refresh operation that returns a new token for a +valid unexpired one. Full draft issue text: trellis session scratch `egeria-issue-token-lifetime.md`. -**Related, an Egeria quirk found alongside:** even with `cascadeDelete: true`, Egeria refused to delete a -`DataStructure` that still had a member `DataField` (`OMAG-GENERIC-HANDLERS-403-005 ... validateNoMemberDataFields`), -so a data contract imported by `import_data_contract*` has to be removed child-first: DataField, DataStructure, -Agreement. +**Correction, 2026-10-05 (Egeria team; confirmed in `RSAGenerator`):** the statement above that an empty `rsa.key-id` is why the signing key is regenerated is wrong. `RSAGenerator.generateRSAKeyPair(keyId)` always makes a fresh key pair and only uses the id as a label, so **no setting makes tokens survive a restart**; persisting the key is planned for the next release. Also: the lifetime property now exists, `bearerTokenTimeout` in HOURS (default 1), added in `a1f228d4a9` (2026-09-19); there is still no refresh operation. --- -### ISSUE-123: `overview_metrics.count_elements` (and `_find`/`_element_count`) return `0`/`[]` on any failure — a failed measurement is indistinguishable from a real zero +### ISSUE-85: No REST endpoint exists to create a `SolutionPort` element — only attach/detach/delegation relationships are exposed -**Layer:** Pyegeria · **Status:** open · **Found:** 2026-10-04, while -making the portal's Egeria Overview stop showing confident numbers during -an outage · **GitHub:** odpi/egeria-python#424 +**Status:** open, confirmed Egeria-side gap 2026-09-03. Found by a +consumer (trellis/Resource Explorer, designing a `PortMaterializer` for +architecture-recovery proposals) hitting the same shape of gap ISSUE-84 +turned out not to be: this time the `.http` ground truth's absence really +does match the live server, not a stale/incomplete `.http` file. -**Root cause, from reading `pyegeria/view/overview_metrics.py` (6.1.28):** +**Confirmed at every layer:** +- `pyegeria/omvs/solution_architect.py`: `link_solution_component_port` + (attach an *existing* port to a component), `link_solution_port_delegation` + (port-to-port delegation), and both detach counterparts exist. No + `create_solution_port`/`_async_create_solution_port` anywhere. +- `Egeria-api-solution-architect.http`: only `.../solution-components/ + {guid}/solution-ports/{guid}/attach|detach` and `.../solution-ports/ + {guid}/port-delegations/{guid}/attach|detach`. No `POST .../solution-ports` + create path. +- `md_processing/data/compact_commands/commands_solution_architect_compact.json`: + only `Link Solution Component Port`/`Link Solution Port Delegation`. No + `Create Solution Port` command. +- `pyegeria/models/models.py`: no `SolutionPortProperties` model at all — + so even the generic fallback route has nothing typed to build a body + against. -- `_find(mgr, body, page_size)` wraps `find_metadata_elements` in a bare - `except Exception` and returns `[]`, logging only at `debug`. -- `_element_count(mgr, body, as_of)` tries the native count; if it is - unsupported *or raises*, it falls back to `len(_find(...))` — so a failed - query becomes `0`. -- `count_elements(...)`'s docstring states "0 on any failure (never - raises)". The callers `people_counts`, `usage_context_counts` and - `counts_by_type` inherit the same behaviour. +**Live-verified not just a stale-`.http`-file case** (the precedent this +repo already has for that shape of false alarm — see the multi-link audit +above, where 3 of 4 similarly-unwrapped relationship types turned out to +have zero real endpoint on live-server inspection while others did): +`dwolfson-1b`/trellis fetched the full live OpenAPI spec directly +(`GET /v3/api-docs`, 2.39MB) and searched every path containing "port" — +30 POST paths matched, every one of them attach/detach (component-port, +port-delegation, even a generic process-ports attach under a different +`{urlMarker}` service). Zero creation endpoints anywhere in the spec. +This confirms the `.http` ground truth exactly; not a documentation gap. -**Effect:** a dashboard cannot tell "there are 0 elements" from "Egeria -timed out / returned 401 / 500". An outage or auth failure renders as a -confident `0`. A fallback count that returns exactly the page size is also -only a lower bound, not a total, and is not flagged. +**The generic fallback exists but isn't usable without guessing:** +`/servers/{serverName}/api/open-metadata/{urlMarker}/metadata-elements` +(`MetadataExpert.create_metadata_element`, +`NewOpenMetadataElementRequestBody`) is real and live-confirmed present, +with a top-level `type_name` field that could carry `"SolutionPort"`. But +per this file's own `pyegeria/models/models.py` gotcha, that endpoint's +`properties` field needs the verbose nested `ElementProperties`/ +`propertyValueMap` shape server-side, not the flat dict every bespoke +wrapper uses — and with no `SolutionPortProperties` model to check +property names against, building that body means hand-guessing what +properties a `SolutionPort` actually takes. Consistent with this repo's +established practice (see the multi-link audit's "left unbuilt rather +than guessing at undocumented URLs or building against the awkward +generic path speculatively") — not attempted. -**Proposed fix:** return `None` on failure (or add a strict variant, e.g. -`count_elements(..., strict=True)` / `try_count_elements`, that raises or -returns `None`), keeping today's behaviour behind an explicit opt-in if -existing callers rely on it. Apply the same to `_find`/`_element_count` and -propagate `None` through `people_counts`, `usage_context_counts` and -`counts_by_type`. Log failures above `debug`. Flag a capped fallback count -as a lower bound. +**Nothing to fix in pyegeria until Egeria adds a real create endpoint.** +Once one exists, the follow-on work is small and well-precedented: a +`create_solution_port`/`_async_create_solution_port` wrapper in +`SolutionArchitect` following the exact pattern ISSUE-84 just corrected +(`NewElementRequestBody` + `contentStatus` inside `properties`, no +fictional separate request-body class), plus a `Create Solution Port` +Dr.Egeria compact command — the family's registration plumbing in +`register_solution_architect_processors()` (`md_processing/dr_egeria.py`) +already handles this shape of addition with no new routing logic needed. -**Workaround in egeria-workspaces:** the portal's Overview handler calls -`find_metadata_elements` directly and returns `None` for failed or capped -queries, rendering "Not measured" instead of `0` (odpi/egeria-workspaces -#601, #607). +**Not blocking anything urgent** on the trellis/Resource Explorer side — +they're holding real port materialization until this exists server-side +(tracked as their own Backlog item 8), or until a workaround surfaces. --- -### ISSUE-124: `updateTypeDef` (Metadata Expert `open-metadata-types/update`) can never succeed -- the server builds the patch without `updatedBy`, which the repository services require - -**Layer:** Egeria Server (not pyegeria) · **Status:** open; reported to the Egeria team (response 2026-10-05); Egeria fix merged to Egeria `main` 2026-10-06 (odpi/egeria#9363), not yet re-tested on a platform built from it · **Found:** -2026-10-05, live verification of `MetadataExpert.update_type_def` (PR #428). - -**What:** posting a valid `OpenMetadataTypeDefPatch` (type added through the API moments earlier, -`applyToVersion` 1) answers a 500 wrapping -`OMRS-REPOSITORY-400-069 Method updateTypeDef has detected that a TypeDef patch from qs-metadata-store has the -mandatory field updatedBy set to null ... TypeDefPatch{typeDefGUID=..., typeDefName=`PyegeriaTmpRecipe`, -applyToVersion=1, updateToVersion=2, newVersionName=`2.0`, updatedBy=`null`, ...}`. The request itself arrived -intact (the attribute definition is visible in the message). +# Quick reference: which OMVS client class for which purpose -**Cause, from Egeria's source (`origin/main`, 2026-10-01):** `OpenMetadataStoreRESTServices.updateTypeDef` -calls `converter.getTypeDefPatch(requestBody, methodName)`; `OMRSTypeDefConverter.getTypeDefPatch` takes no user -id and has no `setUpdatedBy` call, while `OMRSMetadataCollection.updateTypeDef` rejects a patch whose -`updatedBy` is null. `OpenMetadataTypeDefPatch` has no `updatedBy` field either, so a client cannot supply it. -Not a pyegeria defect: the SDK sends the documented body. Fix belongs upstream (pass `userId` to the converter and -set it). +| Need | Class | Notes | +|---|---|---| +| Business reference data (country/currency codes) | `ReferenceDataManager` | Does **not** cover specification properties (ISSUE-19, docs-only) | +| Valid metadata values for a property name | `ReferenceDataManager` or `MetadataExpert` | `get_valid_metadata_values` lives on shared `ServerClient` base; no `as_of_time` support — Egeria endpoint doesn't expose it (ISSUE-18) | +| Specification properties (placeholders, guards, action targets, etc.) | `SpecificationProperties` | `get_specification_property_by_type` now works with either PascalCase or `SCREAMING_SNAKE_CASE` input (ISSUE-17, fixed 2026-08-15); `find_specification_property` with `graph_query_depth=0` also available (ISSUE-15); `get_specification_property_by_guid` works too, `NameError` fixed (ISSUE-28, fixed 2026-08-05, re-verified 2026-08-15) | +| `DataGrain` / `DataClass` listing | `find_data_value_specifications` / `get_data_value_specifications_by_name("*")` | Both fixed (ISSUE-1, ISSUE-2) | +| `DataSpec` (Collection subtype) | `CollectionManager.find_collections(metadata_element_type="DataSpec")` | | +| `DataStructure` / `DataField` | `DataDesigner.find_data_structures` / `find_data_fields` | | +| Solution blueprints/components (any pyegeria version) | `SolutionArchitect.find_solution_blueprints/components(search_string="*")` | Avoid `find_all_*` variants on old versions (ISSUE-11) | +| Note logs (list) | `find_note_logs("*", graph_query_depth=0)` | ISSUE-15 | +| Note logs (entries) | `get_notes_for_note_log(guid, page_size=100)` | ISSUE-3 — never pass `metadata_element_type_name="NoteLog"` | +| Collection members | `get_collection_members(collection_guid)` | ISSUE-8 — now returns members of any type, not just the collection's own type | +| Comparing results across two runs/environments that don't match | — | Check whether the same user's credentials were used in both — governance zone visibility can legitimately change results per-user (ISSUE-29) before assuming a pyegeria bug | +| Multi-classification search (`matchClassifications`, 2+ conditions) | `MetadataExpert.find_metadata_elements` | Fixed in Egeria server (ISSUE-35) | +| Paging a `find_metadata_elements` result | Set `"startFrom"`/`"pageSize"` **in the body dict** | Fixed (ISSUE-34) — these are NOT separate parameters on this method anymore; passing them as kwargs is silently a no-op. Same for `"graphQueryDepth"`. | +| Relationships for a single element by guid | `MetadataExpert.get_all_related_elements(guid)` | **Not** `get_metadata_element_by_guid` — that call never returns relationships, by design (ISSUE-37, not a bug) | +| Project parent/child hierarchy (any linked project, not just hierarchy) | `ProjectManager.get_linked_projects(guid)` | Fixed (ISSUE-42) — was silently returning "No elements found" regardless of real data | +| Checking whether a `get_guid_for_name` lookup found anything | `guid_or_none(client.get_guid_for_name(name))` | ISSUE-114 — a miss returns a truthy sentinel string ("No elements found"), so `if not guid` reads it as found; the helper returns `None` instead | -**Re-tested 2026-10-05 after the platform was rebuilt and restarted** (image built 14:19Z from egeria main -`449ad06894`): a throwaway entity type (primitive attribute only) was added and patched with one new attribute. -The server rejected the patch with the identical `OMRS-REPOSITORY-400-069 ... updatedBy set to null` -(`TypeDefPatch{... applyToVersion=1, updateToVersion=2, newVersionName=2.0, updatedBy=null ...}`). The type was -then deleted and verified gone. **Confirmed still present on the rebuilt platform.** +--- -**Effect here:** `update_type_def` is correct but unusable against current Egeria; its docstring says so. +# Appendix: Closed / Not-a-bug entries -**Egeria team response, relayed 2026-10-05 (source `oak2026` fb3d6fce53; **fixes are in two unmerged PRs, so no build contains them, and nothing was tested against a live platform**):** valid, and `updateTime` was missing as well as `updatedBy` (our analysis was right but incomplete). `getTypeDefPatch` will take the user and set both; new `DynamicTypeFVT`. +## Fixed / Resolved -**Update 2026-10-10:** the `oak2026` source (`fb3d6fce53`) is now in Egeria `main`, merged 2026-10-06 (odpi/egeria#9363), so a platform built from current `main` has the fix. Not yet re-tested. +Sorted newest-first by `ISSUE-#` (2026-10-12). ISSUE-91 has two entries +(the same defect reported twice); both are kept, next to each other. --- -### ISSUE-125: `deleteEnumDef` answers 500 "unknown TypeDef" for an enum the server lists -- an enum added through the API cannot be removed +### ISSUE-141: `ActionParameter.function` is a required string, so analytic-only specs must pass `""`, and the Advisor's validator then warns "No Action Function" -**Layer:** Egeria Server (not pyegeria) · **Status:** open; reported to the Egeria team (response 2026-10-05); Egeria fix merged to Egeria `main` 2026-10-06 (odpi/egeria#9363), not yet re-tested on a platform built from it · **Found:** -2026-10-05, live verification of `MetadataExpert.delete_enum_def` (PR #428). **Left a stray type behind.** +**Layer:** pyegeria (report spec models) · **Status:** fixed 2026-10-12 on branch `docs/issue-136` (PR pending), not yet released · **Found:** 2026-10-12, reported by the Resource Explorer PR/CI session from trellis-analytics (trellis PR #622, `trellis_analytics/portal.py`), which works around it today. -**What:** after `add_enum_def` (returned GUID `118441be-...`), `add_type_def` of an entity type using it, and a -successful `delete_type_def` of that entity type, `delete_enum_def(guid, name)` answered -`OMRS-CONTENT-MANAGER-500-001 The repository content manager method getAttributeTypeDef has detected an unknown -TypeDef 118441be-... from qs-metadata-store on behalf of method deleteAttributeTypeDef` ("Open up a Github issue"). -Yet `ValidMetadataManager.get_attribute_types()` still lists the enum, so the server knows it by one route and -not the other. +**Fixed 2026-10-12:** `function` is now `Optional[str] = None`, and a model +validator rejects an action with neither `function` nor `analytic_function`. An +explicit `""` is still accepted, so every spec that loaded before still loads. +`gen_report_specs` and `analytic_demo_specs` no longer write `function=""`, and +`EgeriaTech.get_report_spec_schema` returns an error row for an analytic-only +spec instead of failing on `None`. **Still to do outside this repo:** the +Advisor's `validate_report_spec` (trellis `packages/egeria-advisor`) should stop +warning "No Action Function" when `analytic_function` is set. Test: +`tests/micro-tests/test_issue_141_action_function_optional.py`. -**Cause (Egeria team, confirmed in source at `449ad06894` on 2026-10-05):** `OMRSRepositoryContentValidator.java:877` -calls `repositoryContentManager.getAttributeTypeDef(sourceName, guidParameterName, guid, methodName)`, but that -4-argument overload is `(sourceName, attributeTypeDefGUID, attributeTypeDefName, methodName)`. The "GUID" passed is the -parameter-NAME string and the "name" is the real GUID, which is why the error prints our real GUID as the unknown -TypeDef. Every `deleteAttributeTypeDef` through the local repository fails (as do `reIdentifyAttributeTypeDef` and the -enterprise equivalents). **My earlier hypothesis here (a GUID map populated in only one place) was wrong.** +**What:** a FormatSet that only names an `analytic_function` still has to +supply an action `function`; pyegeria's own demo specs pass `""`. The +Advisor's `validate_report_spec` then warns "No Action Function" for +analytic-only specs. -**Left on the shared dev platform:** enum `PyegeriaTmpCuisineType` (`118441be-6e03-4442-96c6-e431f75fcb3f`), -unused, announced to the cohort. Harmless but visible in type listings. Per the Egeria team it is deletable only after a build containing the fix is deployed. +**Where seen:** trellis `trellis_analytics.portal.report_specs()`; +egeria-advisor `report_spec_parser` / `validate_report_spec`. -**Re-tested 2026-10-05 after the platform was rebuilt and restarted** (image built 14:19Z from egeria main -`449ad06894`, `egeria-main` restarted 14:39Z): the enum survived the restart and is still listed, and -`delete_enum_def` fails with the identical `OMRS-CONTENT-MANAGER-500-001`. So a restart does not help, which the argument-order bug above -explains: the failure does not depend on what was loaded at startup. +**Candidate fix:** make `function` optional when `analytic_function` is set, +and teach the validator the same rule. -**Workaround:** none through the API. Avoid creating throwaway enums on a shared platform. +--- -**Egeria team response, relayed 2026-10-05 (source `oak2026` fb3d6fce53; **fixes are in two unmerged PRs, so no build contains them, and nothing was tested against a live platform**):** valid; fix in source on `oak2026`, pending merge to `main`. +### ISSUE-137: `exec_report_spec` always builds an `EgeriaTech` client and mints a token, even for an analytic routine that needs no Egeria -**Update 2026-10-10:** the `oak2026` source (`fb3d6fce53`) is now in Egeria `main`, merged 2026-10-06 (odpi/egeria#9363), so a platform built from current `main` has the fix. Not yet re-tested. +**Layer:** pyegeria (format_set_executor) · **Status:** fixed 2026-10-12 on branch `docs/issue-136` (PR pending), not yet released · **Found:** 2026-10-12, reported by the Resource Explorer PR/CI session from trellis-analytics (trellis PR #622, `trellis_analytics/portal.py`), which works around it today. ---- +**Fixed 2026-10-12:** `_run_analytic_function` counts the routine's leading +client parameters (`_count_client_params`) and builds and authenticates an +`EgeriaTech` only when there is at least one. A routine with none is called +directly, so an unreachable `view_url` no longer stops it. Test: +`test_analytic_routine_without_client_param_builds_no_client` in +`tests/micro-tests/test_format_set_executor_token.py`. -### ISSUE-122: `AssetMaker.get_catalog_targets` / `get_catalog_target` send `metadataElementTypeName="CatalogTarget"` (a relationship type) — server rejects with OMAG-COMMON-400-019, surfaced as SERVER_ERROR_500 +**What:** an analytic-function report spec whose routine reads only a local +store (e.g. Resource Explorer's registry through an `OdsReader`) still +triggers `EgeriaTech` construction and token creation before the routine +runs. With an unreachable `view_url`, the call raises before the routine is +ever called. -**Status: FIXED and released.** Merged in #421 (`f2532bbd`, 2026-10-04) and shipped in **pyegeria 6.1.28** (and 6.1.29); present in the portal image's installed copy. Re-verified read-only on 2026-10-05 against the rebuilt platform: `get_catalog_targets` on the JDBC cataloguer returns "No elements found" in JSON, DICT and MD with no server error (it always answered `SERVER_ERROR_500` before the fix). The non-empty case was verified live on 2026-10-04, including the second defect found then (a blank DICT/MD record for the single get) and the `CatalogTargetProperties` field additions. -(logged 2026-10-04 by the Resource Explorer design session; found read-only while -checking which catalog targets the PostgreSQL cataloguers hold). +**Where seen:** trellis `packages/trellis-analytics/src/trellis_analytics/portal.py` +(the `curation_coverage` tile); its test fakes the `EgeriaTech` boundary to get +past this. -**Fix + live verification (2026-10-04):** both methods now pass -`filter_results_by_type=False`; `_async_get_guid_request` gained that flag -(default `True`, mirroring `_async_get_results_body_request`). -`CatalogTargetProperties` gained `connectionName`, `metadataCollectionQualifiedName`, -`permittedSynchronization`, `deleteMethod`. Verified live with one throwaway Asset + -one CatalogTarget on the JDBC cataloguer (both removed afterwards; confirmed -not-found by GUID): list JSON/DICT/MD and single JSON work. **Second defect found -by that run:** `get_catalog_target` DICT/MD output was an all-blank record, because -the endpoint returns a *relationship* (`relationshipGUID`, `elementAtEnd1/2`) and the -element formatter has nothing to read. Added `_generate_catalog_target_output` for -the single get and verified it live in a second throwaway run (also removed, -not-found by GUID): the relationship's two ends are element stubs -(`guid`/`uniqueName`/`type`, no `properties`), so names come from `uniqueName`. -The daemon's log during the ~1s windows was not inspected. +**Candidate fix:** build the client lazily, only when the routine's signature +asks for a client parameter. -Original report follows. +--- -Both methods pass `_type="CatalogTarget"` into the generic results/guid request -helpers, which put it in the request body as `metadataElementTypeName` -(`pyegeria/omvs/asset_maker.py:1371` in `_async_get_catalog_target`, `:1480` in -`_async_get_catalog_targets`, present at 6.1.27). `CatalogTarget` is the -*relationship* type between an integration connector and its target, not an -element type, so the view server answers: +### ISSUE-136: `get_notes_for_note_log` filters by type `Action`, so a NoteLog's Notes don't come back; `get_note_by_guid` calls an endpoint that doesn't exist -``` -OMAG-COMMON-400-019 ... CatalogTarget ... is not a sub-type of OpenMetadataRoot -``` +**Layer:** pyegeria · **Status:** fixed 2026-10-12 on branch `docs/issue-136` (PR pending), not yet released · **Found:** 2026-10-11, reported by the Resource Explorer session (6.2.1, +from a live read and the source); `get_note_by_guid` found while checking the sibling methods. -which pyegeria surfaces as `SERVER_ERROR_500`. The call therefore always fails -as shipped; nobody can list a connector's catalog targets through this client. +**Fixed 2026-10-12, and the diagnosis below was wrong in two places.** +1. Notes did come back. `Note` is a subtype of `Notification`, which is a + subtype of `Action` (`OpenMetadataTypesArchive6_1.getNoteEntity`, + `OpenMetadataTypesArchive1_2.getNotificationEntity`), and type filters + include subtypes. Live and read-only, 2026-10-12 on `qs-view-server`: across + 60 note logs the `"Action"` default returned all 56 Notes, plus 1,859 + `Notification` entries that are not Notes, held in 4 logs. The real defect + was that the default was too wide, not too narrow. +2. `get_note_by_guid`'s endpoint exists. `AssetMakerResource` is mapped at + `/api/open-metadata/{urlMarker}`, so `feedback-manager/assets/{guid}/retrieve` + is Asset Maker's `/assets/{assetGUID}/retrieve`, the same routing pyegeria's + note create, update and find already use. Live, it returned the Note with + either default. -**How to trigger:** -```python -am = AssetMaker("qs-view-server", "https://localhost:9443", user, pw) -am.create_egeria_bearer_token(user, pw) -am.get_catalog_targets("70dcd0b7-9f06-48ad-ad44-ae4d7a7762aa") # JDBC cataloguer -# -> PyegeriaException SERVER_ERROR_500 wrapping OMAG-COMMON-400-019 -``` +Fix: `get_notes_for_note_log`, `get_note_by_guid` and `find_notes` (async and +sync) default to `"Note"`; pass `"Notification"` to get every log entry. +Docstrings corrected. Test: `tests/micro-tests/test_issue_136_note_type_defaults.py`. -**Working alternative (confirmed live 2026-10-04):** supply a body with no -element type, so the helper does not inject `CatalogTarget`: -```python -am.get_catalog_targets( - "70dcd0b7-9f06-48ad-ad44-ae4d7a7762aa", - body={"class": "ResultsRequestBody", "graphQueryDepth": 0}, -) -# -> "No elements found" (zero targets on this platform) -``` -The same applies to `get_catalog_target(relationship_guid)`. +**What:** +- `get_notes_for_note_log(note_log_guid, metadata_element_type_name="Action", ...)` (`core/_server_client.py`, async + and sync). With no body, `_async_get_results_body_request` (`filter_results_by_type=True`) puts + `metadataElementTypeName: "Action"` into the request, so the retrieve on + `feedback-manager/note-logs/{noteLogGUID}/notes/retrieve` is restricted to `Action` elements. The Notes in the log + are type `Note` (`OpenMetadataType.NOTE`), so they are filtered out. The default looks copied from an action + method; the docstrings say so too ("default is \"Action\""). +- `get_note_by_guid(guid, metadata_element_type_name="Action", ...)` has the same default, and its URL is + `feedback-manager/assets/{guid}/retrieve`. `FeedbackManagerResource.java` has no such mapping, and no + single-Note retrieve at all (the only Note read is `/note-logs/{noteLogGUID}/notes/retrieve`). So every call + fails. Dr.Egeria's `Create/Update Note` and the journal-entry paths (`md_processing/v2/feedback.py`) call it to + render the result after a successful write, so those commands would report an error even though the write worked + (not yet confirmed live). -**Likely fix (not made):** drop the `_type="CatalogTarget"` argument in both -methods (or pass the element type the caller wants, default none), since the -endpoint already scopes to catalog targets by URL. Not verified: the -per-target response shape of a non-empty result (no targets exist on the dev -platform to read), and whether `_generate_referenceable_output` handles the -relationship-plus-element shape the server returns. +**Siblings checked:** the other feedback reads with a type default use the right type, and each type name exists in +`OpenMetadataType.java`: `get_attached_comments` (`Comment`), `get_note_logs_by_name` / `get_attached_note_logs` +(`NoteLog`), `get_tags_by_name` / `get_attached_tags` (`InformalTag`), `get_attached_likes` (`Like`), +`get_attached_ratings` (`Rating`), `get_search_keyword_by_keyword` (`SearchKeyword`). -Related: Resource Explorer's `evidence/CATALOGUE-LEVER-FINDINGS.md` §4–5 -(trellis repo), which also notes `CatalogTargetProperties` (`asset_maker.py:46-51`) -lacks `deleteMethod`, `permittedSynchronization`, `connectionName` and -`metadataCollectionQualifiedName`, which the Java relationship accepts. +**Workaround:** pass `metadata_element_type_name="Note"` to `get_notes_for_note_log` (what RE does), or pass a body. +No workaround for `get_note_by_guid`; use a generic element read (e.g. a `MetadataExpert` get-by-guid) instead. +**Proposed fix:** default `get_notes_for_note_log` to `"Note"` in both versions and fix the docstrings. Point +`get_note_by_guid` at a generic by-GUID element retrieve (and default it to `"Note"`), or remove it, and switch the +Dr.Egeria callers to match. Add a micro-test that checks the URL and the request body. --- -### ISSUE-121: `core/mcp_adapter.py` writes the caller's user name and plaintext password to stderr and the log on every report call - -**Layer:** pyegeria (credential leak in diagnostics) · **Status:** fixed in #425, released in 6.1.29 · **Found:** 2026-10-03, read-only security sweep by the Resource -Explorer coordinator session; no value was read or recorded. +### ISSUE-135: 14 `*_by_name` wrappers required `name` even though they accept `body`, so a body-only call raised `TypeError` -**Correction to the original report (2026-10-05, from reading the code):** -there were **three** leak sites in **two** functions, not two in one -- -`_execute_egeria_call_blocking` printed the string to stderr, and `run_report` -both printed it to stderr and `logger.info`'d it. And the fallback path did -**not** leak the configured profile's password: the string is built before the -settings fallback, so it logged `None` there; only a password a caller passed -explicitly was written. **Fix:** one `_describe_call()` helper used by all three -sites; it keeps report/params/server/user and states the *kind* of credential -(`bearer token` / `explicit user/password` / `defaults from settings`), never the -value. Regression tests (`test_mcp_adapter_no_secret_logging.py`) assert a -sentinel password and token appear in neither stderr nor a loguru sink, and fail -on the old code. **Not done by code:** any `debug_log.*` archive written before -this fix may already hold a password, and rotating a real credential used -through this adapter is still the owner's call. +**Layer:** pyegeria · **Status:** fixed in #445 (merged 2026-10-11), not yet released · **Found:** +2026-10-10, reported by the Resource Explorer session (`ActorManager.get_actor_profiles_by_name` on 6.1.15; still +present in 6.2.1). -**What:** the report-execution entry point in `pyegeria/core/mcp_adapter.py` -(the function whose docstring covers the `token` / `user` / `user_pass` -fallback, around lines 170-180) builds a "Format set" diagnostic string that -includes `user` and `user_pass`, then emits it twice: once with -`print(..., file=sys.stderr)` and once with `logger.info(...)`. Both run -before the settings fallback, so whatever the caller passed, and for the -fallback path the configured profile, is written in clear text on every call. -The log sink is loguru, so the password also lands in any rotated debug-log -file (this repo's untracked `debug_log.*.zip` archives are the kind of file -that would carry it) and in anything that captures the MCP server's stderr. +**What:** 14 of the 47 `*_by_name` wrappers declared their first argument (`name` or `filter_string`) with no +default, while also accepting `body`. A body-only call such as `get_actor_profiles_by_name(body={...})` raised +`TypeError`, while siblings such as `get_user_identities_by_name` default `name` to `None`. Affected: +`actor_manager` (actor profiles, actor roles), `asset_maker` (assets, software capabilities), `connection_maker` +(connections, connector types, endpoints), `data_designer` (data fields, data value specifications), +`data_discovery` (analysis reports), `product_manager` (governance action processes), `schema_maker` (schema types, +schema attributes) and `solution_architect` (concept model elements). -**Severity (owner, 2026-10-03):** low for the current demo/dev systems; it becomes a real problem the moment anyone runs this against a production Egeria with real accounts. +**Workaround (before the fix):** pass a placeholder name with the body, e.g. `get_actor_profiles_by_name("*", body=...)`. +An explicit body is sent as given, and the name and the wrapper's other arguments are ignored. -**Why it matters:** the password reaches files and terminals that are -routinely attached to issues, zipped and shared. Rotating the credential -does not remove the copies already written. +**Fix:** the first argument defaults to `None` in both the async and sync versions. `None` already meant "match +everything", like `"*"`, so name-only and name+body calls send exactly what they did before. +`_async_get_data_value_specifications_by_name` also gets the `classification_names=None` default its sync version +already had. Tests: `tests/micro-tests/test_issue_135_by_name_body_only.py`. -**Candidate fix (not applied):** drop `user_pass` from both statements, and -never log `token`. Log `user` only if the owner wants it. Rotate any -Egeria account whose password was used through this adapter, and treat -existing `debug_log.*` archives as containing it. A regression test should -capture stderr and the log sink for a call with a sentinel password and -assert the sentinel is absent. +**Behaviour to know (unchanged):** when a body is given, it is the whole request. The wrapper's default type filter +(e.g. `metadataElementTypeName: "ActorProfile"`) is not added to it, so a body without `metadataElementTypeName` +searches all types. -**Related:** the same sweep found two hard-coded database passwords in -untracked Airflow DAG files in egeria-workspaces-fs; that is tracked in that -repo, not here. +**Also fixed in #445 (found while testing):** five `*_by_name` wrappers that already defaulted `name` still +mishandled a body-only call: +- `SolutionArchitect.get_info_supply_chain_by_name`, `get_solution_blueprints_by_name`, + `get_solution_components_by_name` and `get_solution_roles_by_name` did `body["filter"] = name`, which overwrote + the caller's filter with `None` on a body-only call and mutated the caller's dict. They now copy the body and set + `filter` only when a name is given (a given name still wins). They also sent paging as URL query parameters, which + Egeria ignores: in Egeria source (`SolutionArchitectResource.java`, `ActorManagerResource.java`) the only query + parameter on these endpoints is `addImplementation`, on information supply chains. Paging now goes in the body, + without overriding values the body sets. +- `AutomatedCuration.get_engine_actions_by_name` rejected a missing `name` (`VALIDATION_ERROR_1`) even with a body, + contrary to its docstring. It now requires a name only when no body is given. It also passed its type filter under + the old `metadata_element_type` key; it was a 14th ISSUE-131 case that the original grep missed (single quotes), + already covered by the 6.2.1 helper fix, and now uses the canonical key. --- -### ISSUE-114: `get_guid_for_name`'s miss-sentinel string ("No elements found") is truthy and repeatedly fools callers' existence checks — a caller guideline, not a candidate fix here +### ISSUE-134: With `PYEGERIA_ENABLE_LOGGER_CATCH` on, `@dynamic_catch` turns every failure into a `None` return, indistinguishable from "nothing found" -**Layer:** caller guideline (pyegeria's return convention is working as -designed; this documents it as a footgun rather than proposing a change) · -**Status:** open, doc-only · **Found:** 2026-09-22, live "Phase 1 done-test" -verification in Resource Explorer (trellis repo) of the `data_class_match` -Data Class bootstrap. +**Layer:** pyegeria · **Status:** fixed 2026-10-12 on branch `docs/issue-136` (PR pending), not yet released · **Found:** 2026-10-10, reported by the Resource Explorer +session; confirmed from code. -**What:** `get_guid_for_name(name)` (and the equivalent method on every OMVS -client that shares this convention — confirmed on `DataDesigner` and -`ReferenceDataManager`) returns the literal string `"No elements found"` on -a miss, not `None`, not `""`, and not an exception. That string is truthy -in Python, so any caller that checks the result with a bare -`if not guid`/`guid or default` reads a miss as "found" and skips the -create it should have run. +**Fixed 2026-10-12:** `dynamic_catch` now applies +`logger.catch(reraise=True)(func)`: the failure is still logged with loguru's +traceback, then re-raised. Mind the call shape: `logger.catch(func, reraise=True)` +silently ignores the keyword when the first argument is the function (loguru +0.7.3), so the obvious one-line fix does nothing; the test caught it. Test: +`tests/micro-tests/test_issue_134_dynamic_catch_reraise.py`. -**Confirmed live impact:** Resource Explorer's -`resource_explorer/surveyors/database/bootstrap_data_classes.py` used -exactly that check (`if not data_class_guid: create(...)`) for all three of -its lookups (`DataClass`, `ValidValuesSet`, `ValidValueDefinition` -keywords). Running it against a platform holding zero Data Classes reported -`Created Data Classes: 0, Skipped: 6` — every one of the six standard -classes was silently treated as "already exists" and never created. A -second, independently-introduced instance of the identical bug was found -the same day in the same directory: -`egeria_reference_catalog.py`'s `_find_existing()` used -`str(client.get_guid_for_name(qualified_name) or "")`, which the sentinel -string also defeats (`or ""` never fires on a truthy string). +**What:** `dynamic_catch` (`pyegeria/core/utils.py` ~439) returns `logger.catch(func)` when +`app_settings.Debug.enable_logger_catch` is true (`PYEGERIA_ENABLE_LOGGER_CATCH`, `config.py` ~315). loguru's +`catch` defaults to `reraise=False`, so any exception -- 401, 500, timeout, validation error -- is logged and the +method returns `None`. Checked with loguru 0.7.3: both a sync and an `async def` function return `None`. It applies +to all ~1,657 `@dynamic_catch` methods in `pyegeria/omvs/`. The choice is also made once, at import time, so +changing the setting afterwards has no effect. -**This is not a new pyegeria defect** — it's the same sentinel-return -convention already covered generally in this file's Quick Reference table -and in several fixed entries (e.g. ISSUE-42's `get_linked_projects`), but -this is the first entry pinning the specific "existence-check" failure mode -by name, since `if not guid` / `guid or default` is an easy, natural-looking -way to write that check and it silently does the wrong thing. - -**Caller-side fix applied (not a pyegeria change)**: both RE call sites now -route the result through `survey_definition_reader._as_guid()`, which -rejects the sentinel because it contains whitespace (`any(ch.isspace() ...)` -), unlike a real GUID. `_as_guid` already existed and was already used -correctly elsewhere in RE (`bootstrap.py`'s canary check, -`survey_definition_reader.py` itself); these two call sites had each grown -their own, incorrect, ad hoc check instead of reusing it. See trellis -`re/data-class-seed-and-wiring` branch (bootstrap_data_classes.py, -egeria_reference_catalog.py, tests/test_bootstrap_data_classes.py, -tests/test_postgres_column_profile.py's -`test_a_miss_reading_as_pyegerias_sentinel_string_still_creates`). +**Effect:** a caller cannot tell a failed read from an empty one; a `None` can render as "no results". Same family as +ISSUE-123 (a failed measurement indistinguishable from a real zero). Resource Explorer now treats `None` from a +wrapped method as "unreadable", never as "no result". -**Candidate pyegeria-side improvement, NOT requested/applied here** — per -this repo's standing convention, logging only: `get_guid_for_name` (and -siblings sharing the convention) could raise a dedicated -`ElementNotFoundError`/return `None` instead of the sentinel string, which -would make `if not guid` correct instead of a trap. Not proposing this -without the owner's approval, since existing callers across pyegeria and -its consumers may already depend on the string return value. +**Fix (not made):** pass `reraise=True` to `logger.catch` (keeps the logging, restores the exception), or document +that the setting is debug-only and must not be used by applications that act on results. --- -### ISSUE-113: `NewElementRequestBody` has no `initial_status` field, so `initialStatus` is silently dropped on every create routed through it - -**Layer:** pyegeria · **Status:** open · **Found:** 2026-09-21, verifying -Resource Explorer's multi-resource design doc's probe 4 (`egeria-support- -for-multi-resource.md` §9, trellis repo): whether a client can create a -governance element (DataClass, DataGrain, ValidValueSet, ...) directly in -`DRAFT` status, as a lighter-weight alternative to an RFA-carried proposal. - -**What:** `DataDesigner.create_data_class({"class": "NewElementRequestBody", -"initialStatus": "DRAFT", "properties": {...}})` returns success with a real -guid. Reading that element back by guid -(`MetadataExpert.get_metadata_element_by_guid`) shows `status: ACTIVE` — the -requested `DRAFT` was silently discarded. The failure has no visible signal -at create time; only a read-back exposes it. - -**Root cause, confirmed by reading pyegeria's own source, not guessed**: -`_async_create_element_body_request` (`core/_server_client.py`) validates -every dict body against the `NewElementRequestBody` Pydantic model -(`models/models.py`) before serializing it to JSON. That model has fields -for `anchor_guid`, `is_own_anchor`, `anchor_scope_guids`, -`initial_classifications`, `parent_relationship_properties`, `properties`, -`parent_guid`, `parent_relationship_type_name`, `parent_at_end_1` — no -`initial_status` field at all. Every model inherits `PyegeriaModel`'s -`model_config = ConfigDict(extra='ignore', ...)`, so an unrecognized dict key -like `initialStatus` validates successfully and is dropped before a single -byte reaches the server — the exact shape this repo's own `CLAUDE.md` -documents for ISSUE-62 (`DeleteElementRequestBody` missing -`cascadeDelete`/`deleteMethod`), one level up: this time on the far more -widely-used creation path. +### ISSUE-133: Egeria 6.2 returns a solution component's children as `nestedSolutionComponents`; one pyegeria formatter still reads only `subComponents` -**Not an Egeria server limitation** — `initialStatus` is a real, -server-documented parameter for other creation endpoints: -`collection_manager.py`'s own docstrings list valid values (`DRAFT, -PREPARED, PROPOSED, APPROVED, REJECTED, APPROVED_CONCEPT, ...`) for the -collection-creation calls that route through a body which *does* carry the -field. `NewElementRequestBody` is simply missing it for the endpoints that -use this shared model, including `create_data_class` and (by the same -model) every other `_async_create_element_body_request` caller across the -OMVS clients — this is not narrowly scoped to data classes. +**Layer:** pyegeria · **Status:** fixed in #439, released in 6.2.1 (2026-10-10) · **Found:** 2026-10-10, reported by the Resource Explorer session +(`SolutionArchitect.get_solution_component_by_guid` JSON on 6.2); confirmed from Egeria source. -**Blast radius**: any caller anywhere in `pyegeria/omvs/` that needs to -create an element with a non-default `initialStatus` (DRAFT, PROPOSED, -APPROVED_CONCEPT, ...) via a method routing through -`_async_create_element_body_request` is silently getting `ACTIVE` instead, -with no error. Not audited here which other call sites pass `initialStatus` -today and are similarly affected — worth a grep across this repo's own -callers of `create_data_class`/`create_data_grain`/other `NewElementRequestBody`-based -creates for `initialStatus` once this is fixed, to catch any that were -silently no-op'ing. +**What:** Egeria's REST element (`AttributedMetadataElement.java` ~387) carries child components as +`nestedSolutionComponents` (the `SolutionComposition` relationship, 0730), each a `RelatedMetadataElementSummary` +(`relatedElement.elementHeader.guid`). It has no `subComponents` field; `subComponents` appears in Egeria only in +Java archive builders, not in any REST response. A caller reading `subComponents` sees no children. -**Fix, once approved**: add `initial_status: str | None = None` to -`NewElementRequestBody` (aliased to `initialStatus` via the existing -`alias_generator`), matching how ISSUE-62's fix presumably added the missing -`DeleteElementRequestBody` fields. +**pyegeria readers:** +- `pyegeria/omvs/governance_officer.py` ~274 (`_extract_solution_components_properties`) reads **only** + `subComponents`, so its sub-components column would always be empty. It turned out to have no callers + (`SolutionArchitect` has its own method of the same name), so no live output was affected. +- `solution_architect.py` ~609 and ~5181, and `md_processing/v2/solution_architect.py` ~384, already read + `nestedSolutionComponents`. Each also has a leftover `subComponents` fallback (~675, ~5187, ~388) that never + matches. These do no harm, but they use the old flat shape (`elementHeader` directly, not `relatedElement`), so they + would mis-parse if the field ever came back. -**Not fixed here** — per this repo's standing convention, logged for -approval before a downstream review pass fixes it in place. Full write-up -with the live repro and the Resource Explorer design decision this blocks -(the DRAFT-element governance-proposal path) is in -`docs/design-notes/PROBES-2026-09-21.md`, "Probes 4 and 5, run live" section, -in the trellis repo. +**Fixed in #439:** `governance_officer.py` reads `nestedSolutionComponents` (through `relatedElement`), and the +three dead `subComponents` fallbacks are removed. Micro-tests in `tests/micro-tests/test_issue_131_133.py`. --- -### ISSUE-111: `ConnectionMaker.link_connection_connector_type`/`link_connection_endpoint`/`link_asset_to_connection` silently create no relationship when `body` defaults to `None` +### ISSUE-131: 13 `*_by_name` wrappers pass `metadata_element_type=`, but `_async_get_name_request` reads `metadata_element_type_name`, so the type filter is silently dropped -**Layer:** Pyegeria · **Status:** open · **Found:** 2026-09-21, reported by a -downstream user (Resource Explorer) while building a client-side secrets -store: three separate `link_*` calls "succeeded" (no exception, no error -response) yet none of the relationships existed afterward, discovered only -because a later `save_client_side_secret` call failed with `Null -connectorType property passed in connection ` against a Connection -whose `link_connection_connector_type` had already returned cleanly. +**Layer:** pyegeria · **Status:** fixed in #439, released in 6.2.1 (2026-10-10) · **Found:** 2026-10-10, while building Resource Explorer's actor-graph lookups (`get_user_identities_by_name`, `get_actor_profiles_by_name`). -**Root cause, confirmed by reading `pyegeria/omvs/connection_maker.py` -(pyegeria 6.1.15) and reproducing live against `qs-view-server`:** +**What:** wrappers such as `ActorManager._async_get_user_identities_by_name` (`pyegeria/omvs/actor_manager.py` ~4195) build `params = {"metadata_element_type": metadata_element_type_name, ...}` and call `self._async_get_name_request(url, **params)`. The base method (`pyegeria/core/_server_client.py` ~6581) takes `metadata_element_type_name` and writes it to `"metadataElementTypeName"`, so the wrapper's key falls into `**kwargs` and never reaches the request body. The by-name searches run unfiltered by type. -- `link_connection_connector_type(connection_guid, connector_type_guid, - body: Union[NewRelationshipRequestBody, dict] = None)` — line 950 -- `link_connection_endpoint(connection_guid, endpoint_guid, body=None)` — - line 966 -- `link_asset_to_connection(asset_guid, connection_guid, body=None)` — line - 998 +**Scope (checked 2026-10-10):** `grep -rn '"metadata_element_type": metadata_element_type_name' pyegeria` matches 27 lines, but only 13 are broken. The other 14 are `find_*` wrappers that call `_async_find_request`, whose parameter really is `metadata_element_type` (`_server_client.py` ~6468), so they are correct; do not change them. The 13 broken ones are all the `*_by_name` wrappers that call `_async_get_name_request`: +- `actor_manager.py`: `get_actor_profiles_by_name`, `get_actor_roles_by_name`, `get_user_identities_by_name`, `get_contribution_records_by_name`, `get_contact_details_by_name`, `get_perspectives_by_name`, `get_skills_by_name` +- `collection_manager.py`: `get_collections_by_name` +- `digital_business.py`: `get_business_capabilities_by_name` +- `location_arena.py`: `get_locations_by_name` +- `runtime_manager.py`: `get_metadata_repository_cohorts_by_name` +- `subject_area.py`: `get_subject_areas_by_name` +- `time_keeper.py`: `get_context_events_by_name` -All three forward `body` unchanged into -`_async_new_relationship_request(url, [], body)` -(`pyegeria/core/_server_client.py:7254`), which POSTs whatever `body` -resolves to — including a bare `None` — with **no client-side validation -that a request body was actually required**. The server accepts the POST -(no error surfaces to the caller) but creates no relationship. Confirmed -live: calling any of the three with an explicit -`body={"class": "NewRelationshipRequestBody"}` instead of the default -succeeds and the relationship is genuinely created — the difference is -solely whether *any* body is sent, not its content (an empty -`NewRelationshipRequestBody` with no further fields is sufficient). +**Root cause:** the two base helpers spell the same argument differently (`_async_find_request(metadata_element_type=...)` vs `_async_get_name_request(metadata_element_type_name=...)`), so a wrapper copied from a `find_*` sibling silently loses the filter. -**Repro (isolated, `ConnectionMaker` only, no Dr.Egeria/RE code in the -path):** -```python -maker = ConnectionMaker(view_server, platform_url, user_id, user_pwd) -maker.create_egeria_bearer_token(user_id, user_pwd) -maker.link_connection_connector_type(connection_guid, connector_type_guid) # body defaults to None -# No exception. Relationship does NOT exist -- confirmed via -# MetadataExpert.get_all_related_elements(connection_guid) showing no -# ConnectionConnectorType relationship afterward. -maker.link_connection_connector_type( - connection_guid, connector_type_guid, - body={"class": "NewRelationshipRequestBody"}, -) # now genuinely creates it -``` +**Effect:** callers relying on the type filter get elements of other types. Resource Explorer works round it by passing a full `FilterRequestBody` with `metadataElementTypeName`. -**Suggested fix:** either (a) raise a client-side `PyegeriaInvalidParameterException` -when `body is None` for these (and any other) `link_*`/`_async_new_relationship_request`-based -methods that require a body server-side, rather than silently sending an -empty POST, or (b) default `body` to `{"class": "NewRelationshipRequestBody"}` -internally when the caller passes `None`, matching what a caller who wants -"just link them, no extra relationship properties" actually intends. Not -implemented here — logging per the standing rule (open pyegeria items get -logged, not fixed in place from a downstream review pass). Worth auditing -whether other `link_*`/`detach_*` methods across `ConnectionMaker` (and -other OMVS classes built on the same `_async_new_relationship_request` -helper) share this default-`None`-silently-no-ops shape — not checked -beyond the three methods this investigation actually exercised. +**Fixed in #439:** `_async_get_name_request` accepts both spellings (`metadata_element_type_name` wins if both are given), and the 13 wrappers now use `metadata_element_type_name`. The 14 `find_*` wrappers are unchanged. Micro-tests in `tests/micro-tests/test_issue_131_133.py` capture the request body for each wrapper. Same family as ISSUE-62, ISSUE-127 and ISSUE-129: an argument accepted and silently dropped. --- -### ISSUE-110: `initiate_*_survey` convenience wrappers can't pass `requestParameters` — `finalAnalysisStep`/`ignoreAnalysisSteps`/`analysisLevel` are unreachable through them - -**Layer:** Pyegeria · **Status:** fixed in #388 · **Found:** 2026-09-21, reported by a -downstream user (Resource Explorer) while building a reachability probe that -depends on controlling survey depth/analysis-step parameters. +### ISSUE-129: `MetadataExpert.get_related_metadata_elements` (and two sibling relationship queries) accept paging, direction and lineage arguments but never send them -**Root cause, confirmed by reading `pyegeria/omvs/automated_curation.py`:** -- `_async_initiate_gov_action_type()` (line 3327-ish) and - `initiate_gov_action_type()` (line 3375) both accept and forward a - `request_parameters` dict into the request body's `requestParameters` - field — this path is fine. -- `_async_initiate_survey(survey_name, resource_guid)` (line 3427) is the - shared implementation behind **every** `initiate_*_survey` convenience - wrapper (`initiate_postgres_database_survey`, `initiate_file_folder_survey`, - `initiate_file_survey`, `initiate_kafka_server_survey`, - `initiate_uc_server_survey`, `initiate_uc_schema_survey`, etc., lines - 3470-3609+). It hard-codes the POST body to only - `governanceActionTypeQualifiedName` + a single `serverToSurvey` - `actionTargets` entry (lines 3456-3466) — it takes no `request_parameters` - argument at all, and none of its callers can supply one. -- Net effect: `finalAnalysisStep`, `ignoreAnalysisSteps`, `analysisLevel` (and - any other survey-specific request parameter) cannot be passed through any - of the `initiate_*_survey` wrappers — only by bypassing them and calling - `initiate_gov_action_type()` directly with the survey's own action-type - qualified name, which is why at least one caller - (`egeria_database_surveyor.py:528` in a downstream repo) already does that - instead of using the wrapper. +**Layer:** pyegeria · **Status:** fixed 2026-10-12 on branch `docs/issue-136` (PR pending), not yet released · **Found:** 2026-10-09, in a code review of Resource Explorer's +6.2 composition read (`blueprint_materializer.sub_component_guids`). -**Suggested fix:** add an optional `request_parameters: dict = None` param -to `_async_initiate_survey()` and thread it into the body (mirroring -`_async_initiate_gov_action_type`'s `body_slimmer` pattern so `None` doesn't -serialize), then add the same optional param to each `initiate_*_survey` -wrapper. Not implemented yet — logging per the standing rule (open pyegeria -items get logged here, not fixed in place from a review pass). +**Fixed 2026-10-12:** `_async_get_related_metadata_elements` sends +`?startingAtEnd=`. It and both `linked-by-*` methods put +`startFrom`/`pageSize`/`forLineage`/`forDuplicateProcessing` in the body through +a new `_with_query_options` helper, which never overrides a field the caller's +body sets and accepts a `ResultsRequestBody` model as well as a dict. +`starting_at_end` stays on the two `linked-by-*` methods so existing calls still +work, documented as ignored. Live and read-only, 2026-10-12, on a note log's 500 +`AttachedNoteLogEntry` relationships: `page_size=3` returned 3, `start_from=3` a +different 3 (no overlap), and `starting_at_end=2` returned 0 (the log is at +end 1). Test: `tests/micro-tests/test_issue_129_metadata_expert_relationship_paging.py`. ---- +**What:** `_async_get_related_metadata_elements` (`pyegeria/omvs/metadata_expert.py` ~2886-2964) builds the URL from +`guid` and `relationship_type` only, and posts `body_slimmer(body)`. `starting_at_end`, `start_from`, `page_size`, +`for_lineage` and `for_duplicate_processing` are documented and silently dropped. So the server's default page size +decides how many relationships come back, both ends of the relationship are returned whatever `starting_at_end` says, +and Memento-classified elements are handled by the server default rather than the caller's `for_lineage`. A caller +that guards "a full page means I cannot tell if there are more" is guarding a value that never reached Egeria. Same +shape as ISSUE-62 and ISSUE-127. -# Quick reference: which OMVS client class for which purpose +**Same bug in two siblings (checked 2026-10-10):** `_async_get_all_metadata_element_relationships` (~3056, +`linked-by-any-type`) and `_async_get_metadata_element_relationships` (~3233, `linked-by-type`) also accept +`start_from` and `page_size` and send neither. They also accept `starting_at_end`, but Egeria's endpoints for +these two take no such parameter (both ends are fixed by the URL), so that argument is meaningless there and +should be removed rather than wired up. -| Need | Class | Notes | -|---|---|---| -| Business reference data (country/currency codes) | `ReferenceDataManager` | Does **not** cover specification properties (ISSUE-19, docs-only) | -| Valid metadata values for a property name | `ReferenceDataManager` or `MetadataExpert` | `get_valid_metadata_values` lives on shared `ServerClient` base; no `as_of_time` support — Egeria endpoint doesn't expose it (ISSUE-18) | -| Specification properties (placeholders, guards, action targets, etc.) | `SpecificationProperties` | `get_specification_property_by_type` now works with either PascalCase or `SCREAMING_SNAKE_CASE` input (ISSUE-17, fixed 2026-08-15); `find_specification_property` with `graph_query_depth=0` also available (ISSUE-15); `get_specification_property_by_guid` works too, `NameError` fixed (ISSUE-28, fixed 2026-08-05, re-verified 2026-08-15) | -| `DataGrain` / `DataClass` listing | `find_data_value_specifications` / `get_data_value_specifications_by_name("*")` | Both fixed (ISSUE-1, ISSUE-2) | -| `DataSpec` (Collection subtype) | `CollectionManager.find_collections(metadata_element_type="DataSpec")` | | -| `DataStructure` / `DataField` | `DataDesigner.find_data_structures` / `find_data_fields` | | -| Solution blueprints/components (any pyegeria version) | `SolutionArchitect.find_solution_blueprints/components(search_string="*")` | Avoid `find_all_*` variants on old versions (ISSUE-11) | -| Note logs (list) | `find_note_logs("*", graph_query_depth=0)` | ISSUE-15 | -| Note logs (entries) | `get_notes_for_note_log(guid, page_size=100)` | ISSUE-3 — never pass `metadata_element_type_name="NoteLog"` | -| Collection members | `get_collection_members(collection_guid)` | ISSUE-8 — now returns members of any type, not just the collection's own type | -| Comparing results across two runs/environments that don't match | — | Check whether the same user's credentials were used in both — governance zone visibility can legitimately change results per-user (ISSUE-29) before assuming a pyegeria bug | -| Multi-classification search (`matchClassifications`, 2+ conditions) | `MetadataExpert.find_metadata_elements` | Fixed in Egeria server (ISSUE-35) | -| Paging a `find_metadata_elements` result | Set `"startFrom"`/`"pageSize"` **in the body dict** | Fixed (ISSUE-34) — these are NOT separate parameters on this method anymore; passing them as kwargs is silently a no-op. Same for `"graphQueryDepth"`. | -| Relationships for a single element by guid | `MetadataExpert.get_all_related_elements(guid)` | **Not** `get_metadata_element_by_guid` — that call never returns relationships, by design (ISSUE-37, not a bug) | -| Project parent/child hierarchy (any linked project, not just hierarchy) | `ProjectManager.get_linked_projects(guid)` | Fixed (ISSUE-42) — was silently returning "No elements found" regardless of real data | +**Workaround (RE):** put `startFrom`/`pageSize` in the `ResultsRequestBody` itself and filter entries by end. ---- +**Server contract (verified 2026-10-10 against Egeria source, `egeria-v6/egeria` at `af400039c2`, 2026-10-06 -- +not just the `.http` file):** `MetadataExpertResource.java` declares `startingAtEnd` as +`@RequestParam(required = false, defaultValue = "0") int` on `/related-elements/{elementGUID}/any-type` and +`/related-elements/{elementGUID}/type/{relationshipTypeName}`, and `MetadataExpertRESTServices` passes it through, so +the `?startingAtEnd=` query parameter is current, not a leftover. The two `linked-by-*` endpoints take only a +`ResultsRequestBody`. `ResultsRequestBody` extends `QueryOptions` -> `PagingOptions` (`startFrom`, `pageSize`) -> +`GetOptions` -> `BasicOptions` (`forLineage`, `forDuplicateProcessing`, `effectiveTime`), so the other four +arguments belong in the body. +**Fix (not made):** in the two `related-elements` methods, add `?startingAtEnd={starting_at_end}` to the URL +(`_async_get_all_related_metadata_elements`, ~2800-2808, already does this and is the pattern to copy), and put +`startFrom`/`pageSize`/`forLineage`/`forDuplicateProcessing` in the body. In the two `linked-by-*` methods, put +paging in the body and drop `starting_at_end`. Only add body fields the caller's body does not already set, so +"body supersedes" (which ISSUE-127 relies on) still holds; the `page_size` default is `max_paging_size`, so +overwriting would silently change callers who set it in the body. --- -# Appendix: Closed / Not-a-bug entries - -## Fixed / Resolved - -**Housekeeping, 2026-09-20:** the 14 entries below were moved here from -"Open Egeria Server issues" and "Open pyegeria items" -- all were already -marked `fixed` (or, for one duplicate ISSUE-91 report, corrected to `fixed` -here) but had never been relocated out of the open sections. No content was -changed beyond that one status correction and this note. +### ISSUE-126: `ProductManager` and `DigitalBusiness` never set `collection_command_root`, so every inherited `CollectionManager` method that uses it raises `AttributeError` ---- +**Layer:** pyegeria · **Status:** fixed in #429 (`68279835`), released in 6.1.29 · **Found:** 2026-10-05, live cleanup of a throwaway data contract (PR #427's +live check). -### ISSUE-118: the pyegeria wheel installed `my_egeria` one level too deep, so `my_egeria`, `my_profile`, `serve_my_egeria` and `serve_my_profile` failed in every pip install +**Fix + scope (2026-10-05):** both constructors now call `CollectionManager.__init__`, as `GlossaryManager` +does; that also restores `metadata_expert_command_root`, which the SmartQuery relationship methods use. An AST scan of all 91 classes in +`pyegeria/omvs/` found no other subclass whose `__init__` skips its base's, and +`test_every_omvs_subclass_runs_its_base_constructor` now fails if one ever does. -**Layer:** Pyegeria (packaging) · **Status:** fixed 2026-10-02 · **Found:** -2026-10-02, preparing to serve the new My Profile app from the Egeria-Workspaces portal. +**What:** `ProductManager.__init__` and `DigitalBusiness.__init__` call `ServerClient.__init__` directly instead +of `CollectionManager.__init__` (`GlossaryManager` does it correctly). `CollectionManager.__init__` is what sets +`self.collection_command_root`, which 27 of `CollectionManager`'s methods use, so on those two subclasses all of +them fail: -**What:** `[tool.setuptools.packages.find]` searched only from the repo root, so the -app's real package (`my_egeria/my_egeria/`, inside the `my_egeria/` uv workspace -member) shipped as `site-packages/my_egeria/my_egeria/...`. The four console scripts -point at `my_egeria.main`, `my_egeria.serve`, `my_egeria.DemoCode...`, and the app's -own code imports `from my_egeria.`, so none of it resolved outside this repo's -dev venv (where the workspace member is installed editable). The portal survived only -because its own script launched `my_profile_app.py` by **file path**, found with a glob -for the nested layout. +```python +pm = ProductManager(view_server, platform_url, user, pwd) +pm.delete_collection(guid, cascade=True) +# AttributeError: 'ProductManager' object has no attribute 'collection_command_root' +``` -**Fixed:** `where = [".", "my_egeria"]` with `namespaces = false`, so the inner -package installs as the top-level `my_egeria`. Two folders that had only shipped as -namespace packages (`pyegeria/config`, `commands/deprecated`) gained an -`__init__.py`. A clean-build comparison showed the wheel is otherwise identical: the -same 480 files, with `my_egeria/...` now top-level, plus the two new `__init__.py`. -`my_profile.tcss` now ships too, so the portal Dockerfile's manual copy is unnecessary. -**Downstream impact:** anything launching the app by the old nested file path must -switch to the entry points (`serve_my_profile` etc.). +**Fixed in #429:** see "Fix + scope" above. -**Related, same day:** `serve_my_egeria`/`serve_my_profile` shelled out to the -`textual serve` command, which comes from `textual-dev` (a dev-only dependency), so -they also failed in a plain install. They now use the `textual-serve` library -directly, and honour `MY_EGERIA_PUBLIC_URL`/`MY_PROFILE_PUBLIC_URL` for proxied -deployments. +--- + +### ISSUE-122: `AssetMaker.get_catalog_targets` / `get_catalog_target` send `metadataElementTypeName="CatalogTarget"` (a relationship type) — server rejects with OMAG-COMMON-400-019, surfaced as SERVER_ERROR_500 + +**Status: FIXED and released.** Merged in #421 (`f2532bbd`, 2026-10-04) and shipped in **pyegeria 6.1.28** (and 6.1.29); present in the portal image's installed copy. Re-verified read-only on 2026-10-05 against the rebuilt platform: `get_catalog_targets` on the JDBC cataloguer returns "No elements found" in JSON, DICT and MD with no server error (it always answered `SERVER_ERROR_500` before the fix). The non-empty case was verified live on 2026-10-04, including the second defect found then (a blank DICT/MD record for the single get) and the `CatalogTargetProperties` field additions. +(logged 2026-10-04 by the Resource Explorer design session; found read-only while +checking which catalog targets the PostgreSQL cataloguers hold). + +**Fix + live verification (2026-10-04):** both methods now pass +`filter_results_by_type=False`; `_async_get_guid_request` gained that flag +(default `True`, mirroring `_async_get_results_body_request`). +`CatalogTargetProperties` gained `connectionName`, `metadataCollectionQualifiedName`, +`permittedSynchronization`, `deleteMethod`. Verified live with one throwaway Asset + +one CatalogTarget on the JDBC cataloguer (both removed afterwards; confirmed +not-found by GUID): list JSON/DICT/MD and single JSON work. **Second defect found +by that run:** `get_catalog_target` DICT/MD output was an all-blank record, because +the endpoint returns a *relationship* (`relationshipGUID`, `elementAtEnd1/2`) and the +element formatter has nothing to read. Added `_generate_catalog_target_output` for +the single get and verified it live in a second throwaway run (also removed, +not-found by GUID): the relationship's two ends are element stubs +(`guid`/`uniqueName`/`type`, no `properties`), so names come from `uniqueName`. +The daemon's log during the ~1s windows was not inspected. + +Original report follows. + +Both methods pass `_type="CatalogTarget"` into the generic results/guid request +helpers, which put it in the request body as `metadataElementTypeName` +(`pyegeria/omvs/asset_maker.py:1371` in `_async_get_catalog_target`, `:1480` in +`_async_get_catalog_targets`, present at 6.1.27). `CatalogTarget` is the +*relationship* type between an integration connector and its target, not an +element type, so the view server answers: + +``` +OMAG-COMMON-400-019 ... CatalogTarget ... is not a sub-type of OpenMetadataRoot +``` + +which pyegeria surfaces as `SERVER_ERROR_500`. The call therefore always fails +as shipped; nobody can list a connector's catalog targets through this client. + +**How to trigger:** +```python +am = AssetMaker("qs-view-server", "https://localhost:9443", user, pw) +am.create_egeria_bearer_token(user, pw) +am.get_catalog_targets("70dcd0b7-9f06-48ad-ad44-ae4d7a7762aa") # JDBC cataloguer +# -> PyegeriaException SERVER_ERROR_500 wrapping OMAG-COMMON-400-019 +``` + +**Working alternative (confirmed live 2026-10-04):** supply a body with no +element type, so the helper does not inject `CatalogTarget`: +```python +am.get_catalog_targets( + "70dcd0b7-9f06-48ad-ad44-ae4d7a7762aa", + body={"class": "ResultsRequestBody", "graphQueryDepth": 0}, +) +# -> "No elements found" (zero targets on this platform) +``` +The same applies to `get_catalog_target(relationship_guid)`. + +**Likely fix (not made):** drop the `_type="CatalogTarget"` argument in both +methods (or pass the element type the caller wants, default none), since the +endpoint already scopes to catalog targets by URL. Not verified: the +per-target response shape of a non-empty result (no targets exist on the dev +platform to read), and whether `_generate_referenceable_output` handles the +relationship-plus-element shape the server returns. + +Related: Resource Explorer's `evidence/CATALOGUE-LEVER-FINDINGS.md` §4–5 +(trellis repo), which also notes `CatalogTargetProperties` (`asset_maker.py:46-51`) +lacks `deleteMethod`, `permittedSynchronization`, `connectionName` and +`metadataCollectionQualifiedName`, which the Java relationship accepts. + +--- + +### ISSUE-121: `core/mcp_adapter.py` writes the caller's user name and plaintext password to stderr and the log on every report call + +**Layer:** pyegeria (credential leak in diagnostics) · **Status:** fixed in #425, released in 6.1.29 · **Found:** 2026-10-03, read-only security sweep by the Resource +Explorer coordinator session; no value was read or recorded. + +**Correction to the original report (2026-10-05, from reading the code):** +there were **three** leak sites in **two** functions, not two in one -- +`_execute_egeria_call_blocking` printed the string to stderr, and `run_report` +both printed it to stderr and `logger.info`'d it. And the fallback path did +**not** leak the configured profile's password: the string is built before the +settings fallback, so it logged `None` there; only a password a caller passed +explicitly was written. **Fix:** one `_describe_call()` helper used by all three +sites; it keeps report/params/server/user and states the *kind* of credential +(`bearer token` / `explicit user/password` / `defaults from settings`), never the +value. Regression tests (`test_mcp_adapter_no_secret_logging.py`) assert a +sentinel password and token appear in neither stderr nor a loguru sink, and fail +on the old code. **Not done by code:** any `debug_log.*` archive written before +this fix may already hold a password, and rotating a real credential used +through this adapter is still the owner's call. + +**What:** the report-execution entry point in `pyegeria/core/mcp_adapter.py` +(the function whose docstring covers the `token` / `user` / `user_pass` +fallback, around lines 170-180) builds a "Format set" diagnostic string that +includes `user` and `user_pass`, then emits it twice: once with +`print(..., file=sys.stderr)` and once with `logger.info(...)`. Both run +before the settings fallback, so whatever the caller passed, and for the +fallback path the configured profile, is written in clear text on every call. +The log sink is loguru, so the password also lands in any rotated debug-log +file (this repo's untracked `debug_log.*.zip` archives are the kind of file +that would carry it) and in anything that captures the MCP server's stderr. + +**Severity (owner, 2026-10-03):** low for the current demo/dev systems; it becomes a real problem the moment anyone runs this against a production Egeria with real accounts. + +**Why it matters:** the password reaches files and terminals that are +routinely attached to issues, zipped and shared. Rotating the credential +does not remove the copies already written. + +**Candidate fix (not applied):** drop `user_pass` from both statements, and +never log `token`. Log `user` only if the owner wants it. Rotate any +Egeria account whose password was used through this adapter, and treat +existing `debug_log.*` archives as containing it. A regression test should +capture stderr and the log sink for a call with a sentinel password and +assert the sentinel is absent. + +**Related:** the same sweep found two hard-coded database passwords in +untracked Airflow DAG files in egeria-workspaces-fs; that is tracked in that +repo, not here. --- @@ -2005,6 +1884,37 @@ cases, fixed in the same change by fetching the selected element at depth 1. --- +### ISSUE-118: the pyegeria wheel installed `my_egeria` one level too deep, so `my_egeria`, `my_profile`, `serve_my_egeria` and `serve_my_profile` failed in every pip install + +**Layer:** Pyegeria (packaging) · **Status:** fixed 2026-10-02 · **Found:** +2026-10-02, preparing to serve the new My Profile app from the Egeria-Workspaces portal. + +**What:** `[tool.setuptools.packages.find]` searched only from the repo root, so the +app's real package (`my_egeria/my_egeria/`, inside the `my_egeria/` uv workspace +member) shipped as `site-packages/my_egeria/my_egeria/...`. The four console scripts +point at `my_egeria.main`, `my_egeria.serve`, `my_egeria.DemoCode...`, and the app's +own code imports `from my_egeria.`, so none of it resolved outside this repo's +dev venv (where the workspace member is installed editable). The portal survived only +because its own script launched `my_profile_app.py` by **file path**, found with a glob +for the nested layout. + +**Fixed:** `where = [".", "my_egeria"]` with `namespaces = false`, so the inner +package installs as the top-level `my_egeria`. Two folders that had only shipped as +namespace packages (`pyegeria/config`, `commands/deprecated`) gained an +`__init__.py`. A clean-build comparison showed the wheel is otherwise identical: the +same 480 files, with `my_egeria/...` now top-level, plus the two new `__init__.py`. +`my_profile.tcss` now ships too, so the portal Dockerfile's manual copy is unnecessary. +**Downstream impact:** anything launching the app by the old nested file path must +switch to the entry points (`serve_my_profile` etc.). + +**Related, same day:** `serve_my_egeria`/`serve_my_profile` shelled out to the +`textual serve` command, which comes from `textual-dev` (a dev-only dependency), so +they also failed in a plain install. They now use the `textual-serve` library +directly, and honour `MY_EGERIA_PUBLIC_URL`/`MY_PROFILE_PUBLIC_URL` for proxied +deployments. + +--- + ### ISSUE-116: `get_my_actors` always failed and `get_my_user_identities`/`get_my_roles` always returned nothing — same type-filter defect as ISSUE-115, hidden by tests that swallowed errors **Layer:** Pyegeria · **Status:** fixed 2026-10-02 · **Found:** 2026-10-02 @@ -2086,2818 +1996,2624 @@ from the functional tests then found three more instances: ISSUE-116. --- -### ISSUE-109: `Link Product Dependency` was missing `ISC Qualified Name` — an earlier fix wrongly confirmed it as "correctly absent" against a stale local `.http` copy +### ISSUE-114: `get_guid_for_name`'s miss-sentinel string ("No elements found") is truthy and repeatedly fools callers' existence checks — convention kept; `guid_or_none` helper added -**Layer:** Pyegeria · **Status:** fixed 2026-09-21 · **Found:** 2026-09-20 -(user, reviewing a template fix), confirmed as a real gap 2026-09-21 after -Mandy Chessell (Egeria team) questioned the earlier "not a bug" conclusion -directly. +**Layer:** caller guideline (pyegeria's return convention is working as +designed; this documents it as a footgun rather than proposing a change) · +**Status:** resolved 2026-10-12 — helper added, convention kept · **Found:** 2026-09-22, live "Phase 1 done-test" +verification in Resource Explorer (trellis repo) of the `data_class_match` +Data Class bootstrap. -**What happened, in order — worth recording because the failure mode is -the interesting part, not just the fix:** -1. 2026-09-20: fixing a duplicate `Dependency Description` attribute on - `Link Product Dependency`, the user separately asked why the template - had no `ISC Qualified Name` field. Checked the ground truth this repo's - own docs point to — `pyegeria/http clients/Egeria-api-product-manager.http` - — which showed `DigitalProductDependencyProperties` with only `label`/ - `description`/`effectiveFrom`/`effectiveTo`. Concluded, and documented in - PR #376, that the field was **correctly absent** — not a bug. -2. 2026-09-21: Mandy Chessell approved that PR's template change but asked - directly whether `ISC Qualified Name` was still unmapped. Re-checked - against the **current upstream `odpi/egeria` source** (not the local - `.http` copy) via `gh search code` + - `raw.githubusercontent.com/odpi/egeria/main/...`: - `DigitalProductDependencyProperties` now `extends LineageRelationshipProperties` - (which declares `iscQualifiedName`) rather than being a flat class with - only `label`/`description` — a real, upstream shape change the local - `.http` file predates. **The field genuinely exists server-side; the - original "not a bug" conclusion was wrong**, not because the reasoning - was bad, but because the ground truth it was checked against was stale. +**What:** `get_guid_for_name(name)` (and the equivalent method on every OMVS +client that shares this convention — confirmed on `DataDesigner` and +`ReferenceDataManager`) returns the literal string `"No elements found"` on +a miss, not `None`, not `""`, and not an exception. That string is truthy +in Python, so any caller that checks the result with a bare +`if not guid`/`guid or default` reads a miss as "found" and skips the +create it should have run. -**Lesson:** this repo's own `.http` clients directory -(`pyegeria/http clients/`) is explicitly gitignored and user-managed — -nothing keeps it in sync with upstream automatically, and there is no -signal when it goes stale. When a domain expert questions a "confirmed not -a bug" conclusion, that is exactly the moment to re-check against -`raw.githubusercontent.com/odpi/egeria/main/...` directly rather than -re-trusting the same local copy that produced the wrong answer. +**Confirmed live impact:** Resource Explorer's +`resource_explorer/surveyors/database/bootstrap_data_classes.py` used +exactly that check (`if not data_class_guid: create(...)`) for all three of +its lookups (`DataClass`, `ValidValuesSet`, `ValidValueDefinition` +keywords). Running it against a platform holding zero Data Classes reported +`Created Data Classes: 0, Skipped: 6` — every one of the six standard +classes was silently treated as "already exists" and never created. A +second, independently-introduced instance of the identical bug was found +the same day in the same directory: +`egeria_reference_catalog.py`'s `_find_existing()` used +`str(client.get_guid_for_name(qualified_name) or "")`, which the sentinel +string also defeats (`or ""` never fires on a truthy string). -**Fixed:** added `ISC Qualified Name` (`isc_qualified_name`, level -`Advanced`) to the `Digital Product Dependency` bundle, copying the -identical, pre-existing attribute definition from the Lineage Linker family -rather than reinventing it (same name is already global-namespace-shared -across compact JSON files — see the "Compact Attribute/Bundle Global -Namespace" convention). Wired `iscQualifiedName` into the -`DigitalProductDependencyProperties` body in -`collection_manager_processor.py`'s `"Product Dependency"` branch. -Regenerated templates confirm it now appears in the Advanced template only -(matching its declared level — correctly absent from Basic). Covered by a -new test, -`test_link_product_dependency_sends_isc_qualified_name` in -`tests/micro-tests/test_collection_link_multilink_guid.py`. Full -`tests/micro-tests/` suite re-verified green. - ---- - -### ISSUE-93: `declassify_metadata_element` raises `AttributeError` when `body` is omitted, though the parameter is Optional — and the classification is silently left in place - -**Layer:** Pyegeria · **Status:** fixed (2026-09-13) · **Found:** 2026-09-08 (Resource Explorer, swapping a Project's kind classification during investigation reclassification). +**This is not a new pyegeria defect** — it's the same sentinel-return +convention already covered generally in this file's Quick Reference table +and in several fixed entries (e.g. ISSUE-42's `get_linked_projects`), but +this is the first entry pinning the specific "existence-check" failure mode +by name, since `if not guid` / `guid or default` is an easy, natural-looking +way to write that check and it silently does the wrong thing. -`MetadataExpert.declassify_metadata_element(metadata_element_guid, classification_name, body=None)` -declares `body` as `Optional[dict | MetadataSourceRequestBody] = None`, but the -implementation calls `.model_dump()` on it unconditionally. Omitting it raises: +**Caller-side fix applied (not a pyegeria change)**: both RE call sites now +route the result through `survey_definition_reader._as_guid()`, which +rejects the sentinel because it contains whitespace (`any(ch.isspace() ...)` +), unlike a real GUID. `_as_guid` already existed and was already used +correctly elsewhere in RE (`bootstrap.py`'s canary check, +`survey_definition_reader.py` itself); these two call sites had each grown +their own, incorrect, ad hoc check instead of reusing it. See trellis +`re/data-class-seed-and-wiring` branch (bootstrap_data_classes.py, +egeria_reference_catalog.py, tests/test_bootstrap_data_classes.py, +tests/test_postgres_column_profile.py's +`test_a_miss_reading_as_pyegerias_sentinel_string_still_creates`). -``` -AttributeError: 'NoneType' object has no attribute 'model_dump' -``` +**Candidate pyegeria-side improvement, NOT requested/applied here** — per +this repo's standing convention, logging only: `get_guid_for_name` (and +siblings sharing the convention) could raise a dedicated +`ElementNotFoundError`/return `None` instead of the sentinel string, which +would make `if not guid` correct instead of a trap. Not proposing this +without the owner's approval, since existing callers across pyegeria and +its consumers may already depend on the string return value. -**Why this is worse than an ordinary signature bug:** the exception arrives from -deep inside the client, so a caller that wraps the call defensively — which is -reasonable, since a classification that is already absent should not be fatal — -swallows it and *believes the classification was removed*. It was not. Measured -live: +--- -```python -kinds before: ['Task'] -declassify_metadata_element(guid, "Task") # raises AttributeError -kinds after: ['Task'] # unchanged -``` +## Awaiting re-test (fix landed, not yet verified live) -In our case the element then carried **two** kind classifications at once -(`['Task', 'PersonalProject']`) — the new one added, the old one never removed — -and a check that only asked "is the new classification present?" reported -success. The wrong-but-plausible state was invisible until the element was read -back for *both* names. +Each of these has a fix in place — usually Egeria `main` (odpi/egeria#9363, +merged 2026-10-06), sometimes on the pyegeria side — but nobody has yet +re-run the original repro against a platform that contains it. The next +action for every entry here is the same: re-test, then move it to the +Appendix or back to an open section with what was found. -**Working around it** — pass an explicit body: +**Resolved 2026-10-12 (convention kept, helper added):** the owner chose to +keep the sentinel return, since changing it would break callers across +pyegeria and its consumers with no error to tell them. Instead pyegeria now +ships `guid_or_none(result)` (`pyegeria/core/utils.py`, exported from +`pyegeria`), the same whitespace test as Resource Explorer's `_as_guid`: it +returns the GUID, or `None` for a sentinel, `None`, `""`, or a non-string. +Use it before an existence check: ```python -me.declassify_metadata_element(guid, "Task", {"class": "MetadataSourceRequestBody"}) -# kinds after: [] +guid = guid_or_none(client.get_guid_for_name(qualified_name)) +if guid is None: + ... # create ``` -**Candidate fix:** default the body when `None`, matching the Optional -declaration — the same shape the workaround passes: - -```python -body = body or {"class": "MetadataSourceRequestBody"} -``` +Also added to the Quick reference table. Test: +`tests/micro-tests/test_issue_114_guid_or_none.py`. -**Worth checking while in there:** `classify_metadata_element` and -`reclassify_metadata_element` sit beside it with the same `Optional` body -declaration. We pass a body to both so we have not hit them, but if they share -the unconditional `.model_dump()` they have the same defect — and the -declassify case shows the failure mode is silent at the caller, not loud. +--- -**Where seen:** `trellis/packages/resource-explorer/resource_explorer/surveyors/ -investigation_reclassifier.py::_move_kind_classification`, which now passes the -body explicitly and additionally verifies that the OLD classification is gone -rather than only that the new one is present. +### ISSUE-113: `NewElementRequestBody` has no `initial_status` field, so `initialStatus` is silently dropped on every create routed through it ---- +**Layer:** pyegeria · **Status:** fixed 2026-10-12 on branch `docs/issue-136` (PR pending), not yet released; reclassified: not a missing model field · **Found:** 2026-09-21, verifying +Resource Explorer's multi-resource design doc's probe 4 (`egeria-support- +for-multi-resource.md` §9, trellis repo): whether a client can create a +governance element (DataClass, DataGrain, ValidValueSet, ...) directly in +`DRAFT` status, as a lighter-weight alternative to an RFA-carried proposal. -### ISSUE-92: `Project Type`'s description in `commands_project_compact.json` lists only 4 of its 6 `valid_values` — omits `Project` and `Experiment`, and the stale text is baked into 24 generated files +**Reclassified 2026-10-12.** Egeria removed `initialStatus` from +`NewElementOptions` on 2025-12-15 (`deb8d14dc0`, "Allow EntityProperties use in +replacementAttributes"). At `af400039c2` no request body carries it; only +`OpenMetadataTypeDefPatch` does. Adding the field to pyegeria's model would only +move the drop to the server. In 6.x a draft governance element is expressed with +the `contentStatus` property instead (`DataClassProperties` → +`DataValueSpecificationProperties` → `AuthoredReferenceableProperties`, which +has `contentStatus`), e.g. `properties.contentStatus: "DRAFT"`. + +**Fix (2026-10-12), per the owner's direction** that Dr.Egeria use the kind of +status appropriate to each element type: +- **New `md_processing/md_processing_utils/element_status.py`.** It maps Dr.Egeria's + status attributes onto the property each type has. A table generated from + Egeria's Java source (`scripts/gen_element_status_fields.py` → + `element_status_fields.py`, 660 properties classes, 307 with a status) says + which of `contentStatus` / `activityStatus` / `deploymentStatus` a class + carries, directly or through its superclasses. Actors, plain Assets, Comments + and ContextEvents have none. +- **Precedence.** Values the user wrote come first: a written `Content Status` / + `Activity Status` / `Deployment Status` is sent if the type has that property + (dropped with a warning if not). A written generic `Status` goes to the first + of the type's properties whose enum accepts the value. Spec defaults then fill + only what is still unset, and a defaulted `Status` only ever fills + `contentStatus`, as before, so a default never starts overwriting an activity + or deployment status on an upsert re-run. Types the table does not know keep + the old behaviour. (The first cut let `Content Status`'s own spec default of + `ACTIVE` beat a written `Status`; `dr_egeria --validate` showed it, and a test + now pins it.) +- **Wiring.** `set_element_prop_body` (shared by nearly every Create/Update) + uses the mapping in place of its old "`Content Status` or `Status` → + `contentStatus` for every type" line. `initialStatus` is no longer sent by + `set_create_body` or `v2/solution_architect.py`. The Solution Component + processor's own `deploymentStatus` override, which wrote `None` over a mapped + value, is removed. +- **Glossary terms.** `GlossaryManager`'s CSV import now sends the row's + `Status` as `properties.contentStatus` (Egeria's term status list is the + `ContentStatus` enum). The CSV export reads `contentStatus` (falling back to + the header status) so the two round-trip. `create_term_copy` used a + nonexistent `status` key; it is now `contentStatus`. The `create_glossary_term` + CLI (`commands/cat/glossary_actions.py`) put the term's properties under + `elementProperties`, a key `NewElementRequestBody` drops, so it created terms + with no properties at all. It now uses `properties`, with `contentStatus`. +- **Docstrings.** 13 `initialStatus` lines in sample bodies and 9 "valid values + for initialStatus" paragraphs (`collection_manager.py`, + `governance_officer.py`, `automated_curation.py`) now point at + `properties.contentStatus`. +- **Tests:** `tests/micro-tests/test_issue_113_element_status.py`. + +**Live test 2026-10-12** (`qs-view-server`, erinoverview, with every live +session's clearance). One `dr_egeria --process` run of a two-command document +created a `DataStructure` with `Status: DRAFT` and a `SolutionComponent` with +`Status: UNDER_DEVELOPMENT`. Read back by GUID through `MetadataExpert`, the +DataStructure stored `contentStatus: DRAFT` and the SolutionComponent stored +`deploymentStatus: UNDER_DEVELOPMENT` (plus the default `contentStatus: ACTIVE`). +Both were deleted by GUID, one call each, no cascade, and read back as +`PyegeriaNotFoundException`. + +**The two other senders:** +- Asset Maker's generic `Create Element` (`v2/asset_maker.py`) no longer sends + `initialStatus` either. Egeria's `TemplateRequestBody` (and the + `TemplateOptions`/`NewElementOptions` chain it extends) has no such field, so + the value was always dropped. The copy keeps the template's own status; + `Template Properties` can override it. A written `Initial Status` now logs a + warning saying so. The attribute stays in the compact spec for now. +- The legacy v1 path `pyegeria/view/md_processing_utils.process_term_upsert_command` + (used only by `commands/cat/dr_egeria_jupyter.py`) is left as is. It calls + `create_controlled_glossary_term`, which pyegeria no longer has, so it fails + before the status could matter; its body shape (`ReferenceableRequestBody` + with `elementProperties`) also predates 6.x. -**Layer:** Pyegeria · **Status:** fixed (2026-09-13) · **Found:** 2026-09-07 (Resource -Explorer, designing investigation → Egeria Project classification mapping). +**What:** `DataDesigner.create_data_class({"class": "NewElementRequestBody", +"initialStatus": "DRAFT", "properties": {...}})` returns success with a real +guid. Reading that element back by guid +(`MetadataExpert.get_metadata_element_by_guid`) shows `status: ACTIVE` — the +requested `DRAFT` was silently discarded. The failure has no visible signal +at create time; only a read-back exposes it. -`md_processing/data/compact_commands/commands_project_compact.json`, the -`Project Type` attribute (`variable_name: project_type`): +**Root cause, confirmed by reading pyegeria's own source, not guessed**: +`_async_create_element_body_request` (`core/_server_client.py`) validates +every dict body against the `NewElementRequestBody` Pydantic model +(`models/models.py`) before serializing it to JSON. That model has fields +for `anchor_guid`, `is_own_anchor`, `anchor_scope_guids`, +`initial_classifications`, `parent_relationship_properties`, `properties`, +`parent_guid`, `parent_relationship_type_name`, `parent_at_end_1` — no +`initial_status` field at all. Every model inherits `PyegeriaModel`'s +`model_config = ConfigDict(extra='ignore', ...)`, so an unrecognized dict key +like `initialStatus` validates successfully and is dropped before a single +byte reaches the server — the exact shape this repo's own `CLAUDE.md` +documents for ISSUE-62 (`DeleteElementRequestBody` missing +`cascadeDelete`/`deleteMethod`), one level up: this time on the far more +widely-used creation path. -```json -"valid_values": [ - "Project", "Campaign", "Task", "PersonalProject", "StudyProject", "Experiment" -], -"description": "A string classifying the project. Supported values are Campaign, Task, PersonalProject and StudyProject." -``` +**Not an Egeria server limitation** — `initialStatus` is a real, +server-documented parameter for other creation endpoints: +`collection_manager.py`'s own docstrings list valid values (`DRAFT, +PREPARED, PROPOSED, APPROVED, REJECTED, APPROVED_CONCEPT, ...`) for the +collection-creation calls that route through a body which *does* carry the +field. `NewElementRequestBody` is simply missing it for the endpoints that +use this shared model, including `create_data_class` and (by the same +model) every other `_async_create_element_body_request` caller across the +OMVS clients — this is not narrowly scoped to data classes. -`valid_values` has six entries; the description names four. `Project` (which -is also this attribute's own `default_value`) and `Experiment` are both -missing from the prose. The machine-readable list is correct — it matches -Egeria 6's actual Project classifications in -`OpenMetadataType.java` (0130): `CAMPAIGN_CLASSIFICATION`, -`TASK_CLASSIFICATION`, `PERSONAL_PROJECT_CLASSIFICATION`, -`STUDY_PROJECT_CLASSIFICATION`, `EXPERIMENT_CLASSIFICATION`. Only the -description is behind. +**Blast radius**: any caller anywhere in `pyegeria/omvs/` that needs to +create an element with a non-default `initialStatus` (DRAFT, PROPOSED, +APPROVED_CONCEPT, ...) via a method routing through +`_async_create_element_body_request` is silently getting `ACTIVE` instead, +with no error. Not audited here which other call sites pass `initialStatus` +today and are similarly affected — worth a grep across this repo's own +callers of `create_data_class`/`create_data_grain`/other `NewElementRequestBody`-based +creates for `initialStatus` once this is fixed, to catch any that were +silently no-op'ing. -**Why it matters more than one stale string:** the description is what -`gen_md_cmd_templates` and `gen_dr_help` render, so it has propagated. In -this checkout, 25 files in the main tree carry the sentence verbatim — the -one source JSON plus 24 generated artifacts: +**Fix, once approved**: add `initial_status: str | None = None` to +`NewElementRequestBody` (aliased to `initialStatus` via the existing +`alias_generator`), matching how ISSUE-62's fix presumably added the missing +`DeleteElementRequestBody` fields. -``` -md_processing/data/compact_commands/commands_project_compact.json <- source -md_processing/data/compact_commands_backup/commands_project_compact.json -sample-data/templates/{basic,advanced}/Projects/*.md (16) -sample-data/egeria-inbox/dr-egeria-help-*.md (7) -``` +**Not fixed here** — per this repo's standing convention, logged for +approval before a downstream review pass fixes it in place. Full write-up +with the live repro and the Resource Explorer design decision this blocks +(the DRAFT-element governance-proposal path) is in +`docs/design-notes/PROBES-2026-09-21.md`, "Probes 4 and 5, run live" section, +in the trellis repo. -The sharpest symptom: **`sample-data/templates/{basic,advanced}/Projects/Create_Experiment.md` -tells the reader that `Experiment` is not a supported value** — on the -template whose entire purpose is to create one. `Create_Project.md` has the -same problem for `Project`. The generated help tables are self-contradictory -in a single row, because they print the prose and the `valid_values` list -side by side: +--- -``` -... Supported values are Campaign, Task, PersonalProject and StudyProject. | False | Project, Campaign, Task, PersonalProject, StudyProject, Experiment | Domain | -``` +### ISSUE-111: `ConnectionMaker.link_connection_connector_type`/`link_connection_endpoint`/`link_asset_to_connection` silently create no relationship when `body` defaults to `None` -A reader who trusts the sentence over the column will never reach for -`Experiment`, which is the one classification carrying its own defining -attribute (`hypothesis`), so the omission suppresses a real capability -rather than just reading untidily. +**Layer:** Pyegeria · **Status:** fixed 2026-10-12 on branch `docs/issue-136` (PR pending), not yet released · **Found:** 2026-09-21, reported by a +downstream user (Resource Explorer) while building a client-side secrets +store: three separate `link_*` calls "succeeded" (no exception, no error +response) yet none of the relationships existed afterward, discovered only +because a later `save_client_side_secret` call failed with `Null +connectorType property passed in connection ` against a Connection +whose `link_connection_connector_type` had already returned cleanly. -**Where seen:** found while mapping Resource Explorer's investigation -`project_classification` onto Egeria's Project classifications — the -description was the first thing read, and it made `Experiment` look -unsupported until `valid_values` and the Egeria type source were checked. +**Fixed 2026-10-12; the root cause differs from the one below.** A missing +body was not the problem. With `body=None`, `_async_new_relationship_request` +builds `{"class": "NewRelationshipRequestBody", "properties": {"class": prop[0]}}`, +and these methods passed the relationship type name as `prop[0]` +(`"ConnectionConnectorType"`, ...). That is not a properties class, so Egeria +refuses the request: its REST services accept null properties or the expected +`Properties` class (checked with `instanceof`), nothing else. Why no error +reached the caller is not established; ISSUE-134 (logger-catch turning failures +into `None`) is one way it could happen. An audit of every +`_async_new_relationship_request` caller found 11 methods with this shape: the +10 that passed a type name instead of a `…Properties` class, plus +`link_request_for_action_target`'s near-miss `RequestForActionProperties`. Each +now names the class the Egeria REST service checks for +(`ConnectionMakerRESTServices` / `DataDiscoveryRESTServices` at `af400039c2`). +Several differ from the relationship name: `ConnectToEndpointProperties`, +`ResourceConnectionProperties` (both `assets/…` and `elements/…/connections` +route to `linkResourceToConnection`), `ServerEndpointProperties`, +`ReportedAnnotationProperties`, `AssociatedAnnotationProperties`, +`AnnotationExtensionProperties`, `RequestForActionTargetProperties`. Test: +`tests/micro-tests/test_issue_111_link_default_properties_class.py`. + +**Live test 2026-10-12** (`qs-view-server`, erinoverview, with every other live +session's clearance). I created throwaway elements under `issue111-test::`: a +virtual Connection, a second Connection, two ConnectorTypes, an Endpoint and a +SoftwareServer. Results: +- **Control:** the pre-fix request shape (`properties.class: + "ConnectionConnectorType"`, sent directly through + `_async_new_relationship_request`) got HTTP 400 and created nothing. +- **Fixed calls:** all six ConnectionMaker `link_*` methods, called with + `body=None`, returned cleanly. +- **Read-back** by a different route (`MetadataExpert.get_all_metadata_element_relationships`, + both end orders) found every expected relationship: `ConnectionConnectorType`, + `ConnectToEndpoint`, `EmbeddedConnection`, `ResourceConnection` (×2, from + `link_asset_to_connection` and `link_resource_to_connection`) and + `ServerEndpoint`. The control pair had none. +- **Cleanup:** each element was deleted by GUID, one call each, no cascade. All + six GUIDs then read back as `PyegeriaNotFoundException` (404), and a + starts-with search on the namespace returned nothing. + +The DataDiscovery methods were not exercised live; they need survey reports and +annotations to link. Watch the response shape if you repeat this: `linked-by-*` +returns its list under `relationships`, not `elementList`, and the type name is +at the top level of each entry (`type.typeName`). -**Candidate fix (one string, then regenerate):** edit the `description` in -`commands_project_compact.json` to cover all six, ideally naming what each -means rather than just listing them again — the `valid_values` array -already lists them, so prose that only repeats the list adds nothing and -will drift again the next time a value is added. Suggested: +**Root cause, confirmed by reading `pyegeria/omvs/connection_maker.py` +(pyegeria 6.1.15) and reproducing live against `qs-view-server`:** -> "A string classifying the project. `Project` (the default) applies no -> classification. `Campaign` is a long-term strategic initiative delivered -> through multiple projects; `Task` a self-contained short activity; -> `PersonalProject` an informal project an individual creates to organize -> their own work; `StudyProject` a focused analysis of a topic, person, -> object or situation; `Experiment` a project testing a hypothesis, which -> is recorded in the `hypothesis` attribute." +- `link_connection_connector_type(connection_guid, connector_type_guid, + body: Union[NewRelationshipRequestBody, dict] = None)` — line 950 +- `link_connection_endpoint(connection_guid, endpoint_guid, body=None)` — + line 966 +- `link_asset_to_connection(asset_guid, connection_guid, body=None)` — line + 998 -(Wording taken from the Egeria type definitions themselves so the two -cannot disagree.) +All three forward `body` unchanged into +`_async_new_relationship_request(url, [], body)` +(`pyegeria/core/_server_client.py:7254`), which POSTs whatever `body` +resolves to — including a bare `None` — with **no client-side validation +that a request body was actually required**. The server accepts the POST +(no error surfaces to the caller) but creates no relationship. Confirmed +live: calling any of the three with an explicit +`body={"class": "NewRelationshipRequestBody"}` instead of the default +succeeds and the relationship is genuinely created — the difference is +solely whether *any* body is sent, not its content (an empty +`NewRelationshipRequestBody` with no further fields is sufficient). -Then re-run `refresh_specs`, `gen_md_cmd_templates` and `gen_dr_help` per -the Dr.Egeria command-sync flow and propagate the regenerated templates/help -to `egeria-workspaces` and `egeria-advisor` — the 24 generated files above -will not update on their own, and 16 of them ship as user-facing templates. +**Repro (isolated, `ConnectionMaker` only, no Dr.Egeria/RE code in the +path):** +```python +maker = ConnectionMaker(view_server, platform_url, user_id, user_pwd) +maker.create_egeria_bearer_token(user_id, user_pwd) +maker.link_connection_connector_type(connection_guid, connector_type_guid) # body defaults to None +# No exception. Relationship does NOT exist -- confirmed via +# MetadataExpert.get_all_related_elements(connection_guid) showing no +# ConnectionConnectorType relationship afterward. +maker.link_connection_connector_type( + connection_guid, connector_type_guid, + body={"class": "NewRelationshipRequestBody"}, +) # now genuinely creates it +``` -**Worth checking while in there:** whether any other compact-command -attribute has a `description` that enumerates its `valid_values` in prose. -Any such pair is the same latent defect — two lists that must be edited -together with nothing enforcing it. A cheap guard would be a spec-validation -check that flags a `description` naming a subset of its own `valid_values`. +**Suggested fix:** either (a) raise a client-side `PyegeriaInvalidParameterException` +when `body is None` for these (and any other) `link_*`/`_async_new_relationship_request`-based +methods that require a body server-side, rather than silently sending an +empty POST, or (b) default `body` to `{"class": "NewRelationshipRequestBody"}` +internally when the caller passes `None`, matching what a caller who wants +"just link them, no extra relationship properties" actually intends. Not +implemented here — logging per the standing rule (open pyegeria items get +logged, not fixed in place from a downstream review pass). Worth auditing +whether other `link_*`/`detach_*` methods across `ConnectionMaker` (and +other OMVS classes built on the same `_async_new_relationship_request` +helper) share this default-`None`-silently-no-ops shape — not checked +beyond the three methods this investigation actually exercised. --- -### ISSUE-91: `pyegeria.core.mcp_server` imports `mcp.server.mcpserver` (mcp 2.x only) but `pyproject.toml` declares `mcp >=0.1` — any consumer that resolves mcp 1.x gets a server that dies at import +### ISSUE-110: `initiate_*_survey` convenience wrappers can't pass `requestParameters` — `finalAnalysisStep`/`ignoreAnalysisSteps`/`analysisLevel` are unreachable through them -**Layer:** pyegeria packaging · **Status:** fixed (2026-09-13) · **Found:** 2026-09-05 (Egeria Advisor dev startup on the M3 Max) +**Layer:** Pyegeria · **Status:** fixed in #388 · **Found:** 2026-09-21, reported by a +downstream user (Resource Explorer) while building a reachability probe that +depends on controlling survey depth/analysis-step parameters. -Commit 2b39ba06 (2026-07-30, "migrate mcp_server.py to mcp 2.0.0's MCPServer") changed the -server's import to: - -```python -from mcp.server.mcpserver import MCPServer -``` +**Root cause, confirmed by reading `pyegeria/omvs/automated_curation.py`:** +- `_async_initiate_gov_action_type()` (line 3327-ish) and + `initiate_gov_action_type()` (line 3375) both accept and forward a + `request_parameters` dict into the request body's `requestParameters` + field — this path is fine. +- `_async_initiate_survey(survey_name, resource_guid)` (line 3427) is the + shared implementation behind **every** `initiate_*_survey` convenience + wrapper (`initiate_postgres_database_survey`, `initiate_file_folder_survey`, + `initiate_file_survey`, `initiate_kafka_server_survey`, + `initiate_uc_server_survey`, `initiate_uc_schema_survey`, etc., lines + 3470-3609+). It hard-codes the POST body to only + `governanceActionTypeQualifiedName` + a single `serverToSurvey` + `actionTargets` entry (lines 3456-3466) — it takes no `request_parameters` + argument at all, and none of its callers can supply one. +- Net effect: `finalAnalysisStep`, `ignoreAnalysisSteps`, `analysisLevel` (and + any other survey-specific request parameter) cannot be passed through any + of the `initiate_*_survey` wrappers — only by bypassing them and calling + `initiate_gov_action_type()` directly with the survey's own action-type + qualified name, which is why at least one caller + (`egeria_database_surveyor.py:528` in a downstream repo) already does that + instead of using the wrapper. -That module exists only in `mcp >= 2.0.0` (1.x ships `mcp.server.fastmcp` instead). The -dependency declaration was not updated and still reads `"mcp >=0.1"` — verified in the released -6.1.5 wheel's METADATA (`Requires-Dist: mcp>=0.1`) and in the current `pyproject.toml` at 6.1.10. +**Suggested fix:** add an optional `request_parameters: dict = None` param +to `_async_initiate_survey()` and thread it into the body (mirroring +`_async_initiate_gov_action_type`'s `body_slimmer` pattern so `None` doesn't +serialize), then add the same optional param to each `initiate_*_survey` +wrapper. Not implemented yet — logging per the standing rule (open pyegeria +items get logged here, not fixed in place from a review pass). -**How it shows up.** trellis pinned `mcp>=1.0.0` and its lock resolved mcp 1.29.0, which satisfies -pyegeria's declared range. Launching the server then fails before it can speak MCP: +--- -``` -$ python -m pyegeria.core.mcp_server -MCP import failed. - File ".../pyegeria/core/mcp_server.py", line 24, in - from mcp.server.mcpserver import MCPServer -ModuleNotFoundError: No module named 'mcp.server.mcpserver' -``` +### ISSUE-109: `Link Product Dependency` was missing `ISC Qualified Name` — an earlier fix wrongly confirmed it as "correctly absent" against a stale local `.http` copy -A client sees only "No response from MCP server" because the traceback goes to stderr and the -process exits without writing a JSON-RPC frame. With mcp 2.1.1 installed the same command answers -`initialize` normally (`serverInfo.name = "pyegeria-mcp"`). +**Layer:** Pyegeria · **Status:** fixed 2026-09-21 · **Found:** 2026-09-20 +(user, reviewing a template fix), confirmed as a real gap 2026-09-21 after +Mandy Chessell (Egeria team) questioned the earlier "not a bug" conclusion +directly. -**Why it went unnoticed.** The quickstart containers run mcp 2.1.1 (pyegeria 6.1.9) and use their own -`/app/mcp_server.py` mounted over SSE in `pyegeria_handler`, not this stdio module, so the -egeria-workspaces path never exercised it. The egeria-python checkout's own venv also has mcp 2.0.0. +**What happened, in order — worth recording because the failure mode is +the interesting part, not just the fix:** +1. 2026-09-20: fixing a duplicate `Dependency Description` attribute on + `Link Product Dependency`, the user separately asked why the template + had no `ISC Qualified Name` field. Checked the ground truth this repo's + own docs point to — `pyegeria/http clients/Egeria-api-product-manager.http` + — which showed `DigitalProductDependencyProperties` with only `label`/ + `description`/`effectiveFrom`/`effectiveTo`. Concluded, and documented in + PR #376, that the field was **correctly absent** — not a bug. +2. 2026-09-21: Mandy Chessell approved that PR's template change but asked + directly whether `ISC Qualified Name` was still unmapped. Re-checked + against the **current upstream `odpi/egeria` source** (not the local + `.http` copy) via `gh search code` + + `raw.githubusercontent.com/odpi/egeria/main/...`: + `DigitalProductDependencyProperties` now `extends LineageRelationshipProperties` + (which declares `iscQualifiedName`) rather than being a flat class with + only `label`/`description` — a real, upstream shape change the local + `.http` file predates. **The field genuinely exists server-side; the + original "not a bug" conclusion was wrong**, not because the reasoning + was bad, but because the ground truth it was checked against was stale. -**Proposed fix (backward compatible for callers).** In `pyproject.toml` declare `"mcp >=2.0"`. Nothing -else needs to change: the import is already 2.x-only, so raising the floor only turns a runtime -import crash into a resolver error at install time. If 1.x support is wanted instead, gate the -import (`try: from mcp.server.mcpserver import MCPServer except ImportError: from mcp.server.fastmcp -import FastMCP as MCPServer`) — but the 2.x API differs beyond the class name, so the floor bump is -the honest option. +**Lesson:** this repo's own `.http` clients directory +(`pyegeria/http clients/`) is explicitly gitignored and user-managed — +nothing keeps it in sync with upstream automatically, and there is no +signal when it goes stale. When a domain expert questions a "confirmed not +a bug" conclusion, that is exactly the moment to re-check against +`raw.githubusercontent.com/odpi/egeria/main/...` directly rather than +re-trusting the same local copy that produced the wrong answer. -**Consumer-side workaround applied in trellis (8441efb):** both packages now declare `mcp>=2.0.0` -and the lock carries mcp 2.1.1, matching the quickstart containers. +**Fixed:** added `ISC Qualified Name` (`isc_qualified_name`, level +`Advanced`) to the `Digital Product Dependency` bundle, copying the +identical, pre-existing attribute definition from the Lineage Linker family +rather than reinventing it (same name is already global-namespace-shared +across compact JSON files — see the "Compact Attribute/Bundle Global +Namespace" convention). Wired `iscQualifiedName` into the +`DigitalProductDependencyProperties` body in +`collection_manager_processor.py`'s `"Product Dependency"` branch. +Regenerated templates confirm it now appears in the Advanced template only +(matching its declared level — correctly absent from Basic). Covered by a +new test, +`test_link_product_dependency_sends_isc_qualified_name` in +`tests/micro-tests/test_collection_link_multilink_guid.py`. Full +`tests/micro-tests/` suite re-verified green. --- -### ISSUE-96: get_guid_for_name (and every other sync wrapper using asyncio.get_event_loop().run_until_complete(...)) breaks when a client instance is reused across threads — surfaces as a misleading CLIENT_ERROR_400 "unable to connect" - -Layer: pyegeria core (_base_server_client.py, _base_platform_client.py, every OMVS class's sync wrappers) · Status: fixed (2026-09-13) · Found: 2026-09-12 (Resource Explorer, SurveyDefinitionReader._lookup_question_guid) - -A single client instance (e.g. ClassificationExplorer) constructed once -and reused from multiple threads — a natural pattern for any caller with a -thread pool bridging sync pyegeria calls into async code — fails -non-deterministically once its httpx.AsyncClient (self.session, created -once in __init__, _base_platform_client.py:128) is driven by more than -one event loop over its lifetime. Every sync wrapper (get_guid_for_name, -create_egeria_bearer_token, and every other -`loop = asyncio.get_event_loop(); loop.run_until_complete(...)` call — -~29 call sites in _server_client.py alone) hands the SAME shared client's -connection pool to whichever thread happens to call it, and -pyegeria/__init__.py's import-time nest_asyncio.apply() -(via pyegeria.view.mermaid_utilities) means every calling thread -silently gets its OWN fresh event loop rather than raising — so the -mismatch is invisible until the underlying httpcore/anyio connection's -internal asyncio.locks.Event/lock, bound to loop A, is awaited from loop -B and raises `RuntimeError: <...> is bound to a different event loop`. - -How it shows up. _async_make_request (_base_server_client.py:621) -catches this (and anything else) broadly and re-raises as -PyegeriaUnknownException / CLIENT_ERROR_400, "The client is unable to -connect to the Egeria platform" — the real RuntimeError and its "different -event loop" text never reach the caller. A direct curl to the same -platform at the same moment succeeds in milliseconds, ruling out actual -connectivity. - -How to trigger: -```python -from pyegeria.omvs.classification_explorer import ClassificationExplorer -import threading - -client = ClassificationExplorer(view_server, platform_url, user, pwd) -client.create_egeria_bearer_token(user, pwd) # binds session internals to THIS thread's loop +### ISSUE-107: Dr.Egeria's create pre-check issues up to 4 redundant `guid-by-unique-name` lookups per command, multiplying an already-slow ambiguous-match server response -def call(): - client.get_guid_for_name("some display name", property_name=["displayName"], type_name="GlossaryTerm") +**Layer:** Pyegeria · **Status:** fixed 2026-09-20 · +**Found:** 2026-09-20, reported by the user from a Dr.Egeria bulk load: a +subject-area file created ~25 folders sharing the same three Display Names +("Prime Words", "Modifiers", "Class Words" across parent scopes), and files +using those names went from ~6 seconds to 2+ minutes apiece — 35 of a +56-minute load, by the user's measurement. Postgres itself was not the +cause (27ms queries); ambiguous-name resolution was. -t = threading.Thread(target=call) -t.start(); t.join() # raises PyegeriaUnknownException / CLIENT_ERROR_400 -``` -Reproduced deterministically (100% failure across 52 distinct names, 2 -runs) when the client is called from a shared thread pool or a fresh -thread per call; 0% failure when the client is always driven by the same -single thread/loop throughout its life, or when a fresh client is -constructed inside the very thread that will use it. +**Root cause, confirmed by reading the code (not the client-side "validates +every match" loop the report's phrasing suggested — no such loop exists in +pyegeria):** +- `md_processing/v2/processors.py`'s `resolve_element_guid()` calls the + SDK's `__async_get_guid__` **twice** per lookup (once scoped by + `tech_type`, once broader if the first comes back empty/ambiguous) — + lines 1413 and 1442. +- `execute()`'s step 4a (line 647) runs an **independent, duplicate** + display-name check on top of the one already done inside `fetch_as_is()` + (line 638) — using the raw Display Name rather than the already-derived + qualified name. +- Net: a single `Create` of a not-yet-cached, ambiguously-named element can + trigger up to 4 HTTP round-trips to the same `guid-by-unique-name` + endpoint. `__async_get_guid__` (`pyegeria/core/_server_client.py:224-321`) + POSTs once per call and returns whatever the server reports; the + ambiguous-match resolution itself (and its ~1s-per-candidate cost) is + server-side, not a Python loop — confirmed by reading the full call path. +- **No fast-path existed for an explicit `Qualified Name`.** `fetch_as_is()` + already fast-paths an explicit `### GUID` (ISSUE-59's fix, lines + 1519-1539) but had no equivalent for an unambiguous, user-supplied + qualified name — every Create paid the ambiguous-lookup cost even when + the file already fully disambiguated the target. -Note: pyegeria/core/mcp_server.py:181-184 (this same 6.1.10 release) -documents this exact failure and works around it by constructing a fresh, -per-call EgeriaTech client inside the handler rather than reusing a -module-level one — confirming this is a known, previously-hit instance of -the same defect, not unique to this caller. +**Fixed (this pass):** `execute()` (`md_processing/v2/processors.py`) now +captures whether `Qualified Name` was explicitly authored in the markdown +*before* step 1a's auto-derivation can inject a derived one under the same +key (making the two indistinguishable by the time step 4a runs), and skips +step 4a's duplicate-Display-Name lookup entirely when it was — an explicit +qualified name already unambiguously identifies the target, and +`fetch_as_is()` already looked that exact name up and found nothing, which +is authoritative since no auto-derivation is involved for ISSUE-59's +collision case to apply to. Covered by +`tests/micro-tests/test_step4a_skip_with_explicit_qn.py`: one test confirms +step 4a still runs (≥2 `resolve_element_guid` calls) when the qualified +name is auto-derived, preserving ISSUE-59's protection; a second confirms +exactly one call happens when it's explicit. Full `tests/micro-tests/` +suite re-verified green after the change. -Candidate fix directions (not evaluated for feasibility here): (1) -make sync wrappers detect a loop change since the client's session was -last used and rebuild self.session when it happens; (2) document -plainly that a client instance is not thread-safe / not loop-portable and -must be confined to one thread or reconstructed per thread; (3) provide a -supported "fresh session per call" mode, generalizing the workaround -already used internally in mcp_server.py. +**Fixed (second pass, same day):** `resolve_element_guid()`'s two internal +passes (Pass 1 WITH the `tech_type` constraint, Pass 2 WITHOUT) each now +check a per-batch query cache in `self.context` (`_resolve_guid_query_cache`, +keyed by `(name_or_guid, effective_type)`) before hitting the server, and +populate it afterward — but **only** for a "not found" or "ambiguous" +outcome, deliberately never for a genuine single-match found `guid`: caching +a found match risked going stale within a batch if a later sibling command +creates an element sharing that exact Display Name (step 4a searches by raw +Display Name, which the QN-keyed local element dictionary checked earlier +in the same method can't already cover); a found match is also already the +cheap server-side path, so there was nothing to gain by caching it. This is +the actual fix for the reported 25x-repeated-Display-Name scenario: the +first of the 25 "Prime Words" Creates pays the full ambiguous/not-found +lookup cost (both passes, since neither cache key exists yet); the other 24 +each hit the cache and make zero calls to the server for that name. Ambiguous +results are replayed per-instance too (each sibling's own `parsed_output` +still gets the "Multiple elements found" error appended, even though the +answer came from cache, since that side effect is per-command). Covered by +`tests/micro-tests/test_resolve_guid_query_cache.py`: one test drives 5 +sibling Creates through a client that reports the shared probe name as +ambiguous and asserts the client is only ever called once across all 5; a +second does the same for a "not found" name and asserts exactly 2 calls +total (Pass 1 + Pass 2, both only on the first sibling). Full +`tests/micro-tests/` suite re-verified green. -Suggested fix (pyegeria): either make each sync wrapper use a per-thread session/loop (e.g. `threading.local()` holding the httpx.AsyncClient), or document that a client instance is single-thread-affine and raise a clear error — rather than the broad `except` in `_async_make_request` relabelling `RuntimeError: ... bound to a different event loop` as CLIENT_ERROR_400 'unable to connect', which is what cost the diagnosis. Caller-side mitigation used in Resource Explorer (dwolfson/trellis, `re/question-guid-client-per-thread`): one client per thread, constructed inside the thread that uses it — 0/104 failures after, 52/52 before. +**Other similar patterns checked, none found:** two structurally similar +helpers (`_server_client.py:734-772`, `classification_explorer.py:3428- +3462`) and the legacy v1 path (`extraction_utils.py:416-502`) all do the +same fail-fast "raise if `len(matches) > 1`" with no client-side loop — no +matching perf anti-pattern found elsewhere in `pyegeria/core`, +`pyegeria/omvs`, or `md_processing`. --- -### ISSUE-91: `pyproject.toml` declares `mcp >=0.1`, but `pyegeria.core.mcp_server` needs `mcp>=2.0` — the declared floor lets a resolver install a version too old to import the module at all +### ISSUE-106: `InitialClassifications`'s `model_serializer` popped the wrong dict key (`other_props` instead of the aliased `otherProps`) — every classification property beyond `class` was silently dropped on every `initialClassifications` call across the whole SDK -**Layer:** Pyegeria · **Status:** fixed (2026-09-13) -- duplicate report of the ISSUE-91 entry above (same number, same root cause, found a day later from a different angle: containerized demo deployment rather than dev startup). Confirmed current `pyproject.toml` declares `"mcp >=2.1.1"` · **Found:** 2026-09-06 (Egeria Advisor, containerized demo deployment rebuild against pyegeria 6.1.10). +**Status:** fixed 2026-09-18 (pyegeria — `pyegeria/models/models.py`), +verified with an isolated Pydantic round-trip test and live against +`qs-view-server`. Found while live-verifying the ISSUE-105 Glossary +classification fix below — `Taxonomy`/`CanonicalVocabulary` classifications +were being applied correctly (conditionally, per the ISSUE-105 fix) but +`organizingPrinciple`/`scope` never persisted despite being read and set +correctly on the Python side. -`pyproject.toml`'s `[project.dependencies]` declares `"mcp >=0.1"`, but -`pyegeria/core/mcp_server.py` imports `from mcp.server.mcpserver import -MCPServer` — confirmed live in this checkout's dev venv: -`importlib.metadata.version("mcp")` is `2.0.0`, and -`mcp.server.mcpserver.MCPServer` only exists at that version; the module -path is new to the 2.x line, not present in the 0.x/1.x `mcp` package -history. A resolver that's free to pick anything satisfying `>=0.1` (no -upper or tighter lower bound forcing 2.x) can legitimately land on a much -older `mcp` release, at which point `import pyegeria.core.mcp_server` -fails outright rather than degrading gracefully. +**Root cause:** `InitialClassifications.capture_other_props()` (a +`model_validator`) correctly captures any extra properties beyond `class` +into a Python-side `other_props` field. Its paired +`serialize_model()` (`model_serializer(mode="wrap")`) is supposed to +flatten `other_props` back onto the output dict before sending — but it +does `result.pop("other_props", None)`, the Python field name. Every real +call site in this codebase dumps with `by_alias=True` (confirmed: +`_async_new_relationship_request`, `_async_create_element_body_request`, +etc. all do), and `PyegeriaModel`'s `alias_generator=to_camel_case` means +the serializer's own `handler(self)` call has *already* aliased the field +to `otherProps` by the time `serialize_model` runs — so the `pop("other_props")` +never matched anything, and the extra properties stayed nested under a +stray `otherProps` key the real Egeria DTO doesn't declare, silently +dropped server-side (no error — same "declared but not on model" silent-drop +shape as ISSUE-62, but at the serializer layer instead of the field-declaration +layer). -**Why it matters, found how:** a container rebuild of Egeria Advisor's -demo deployment against pyegeria 6.1.10 pinned `mcp==2.1.1` explicitly -(not resolved from pyegeria's own floor) and confirmed EA's MCP agent -pre-warms cleanly on it — so 6.1.10 itself works fine when the caller -pins a modern `mcp`, but nothing in pyegeria's own declared dependency -would have caught a caller who didn't. +**Scope: every caller of `initialClassifications` with more than a bare +`{"class": ...}`, across the whole SDK, not just Glossary.** Confirmed by +isolated test: +```python +from pyegeria.models.models import NewElementRequestBody +body = {"class": "NewElementRequestBody", "isOwnAnchor": True, + "properties": {"class": "GlossaryProperties", "displayName": "x", "qualifiedName": "x"}, + "initialClassifications": { + "Taxonomy": {"class": "TaxonomyProperties", "organizingPrinciple": "X"}, + }} +NewElementRequestBody.model_validate(body).model_dump_json(by_alias=True) +# before fix: {"Taxonomy": {"class": "TaxonomyProperties", "otherProps": {"organizingPrinciple": "X"}}} +# after fix: {"Taxonomy": {"class": "TaxonomyProperties", "organizingPrinciple": "X"}} +``` -**Ask:** tighten `pyproject.toml`'s `mcp` constraint to actually match -what `mcp_server.py` requires (`mcp>=2.0` at minimum — confirm the exact -version `MCPServer`/`mcp.server.mcpserver` was introduced at, and pin to -that) rather than the inherited placeholder `>=0.1` floor. Not -independently verified against the `mcp` package's own changelog/git -history here — the live import check above shows 2.0.0 works and the -module path is absent from the 0.x/1.x line by inspection, but the exact -first-working version wasn't pinned down. +**Fix:** `serialize_model` now pops either key — +`result.pop("otherProps", None) or result.pop("other_props", None)` — +covering both the aliased (real-world) and unaliased (defensive) cases. + +**Verified live:** re-ran the Glossary classification test from ISSUE-105 +after this fix — `organizingPrinciple`/`scope` both persisted correctly +where they previously silently vanished with the model bug still in place. +`pytest tests/micro-tests -m unit`: no regressions from this shared-model +change. + +**Found, not fixed — adjacent, separate bug, same class as ISSUE-62:** +while cleaning up throwaway test elements for this fix, `_async_delete_collection` +(and likely other `_async_delete_*` wrappers with the same shape) silently +ignores its own `cascade` parameter whenever it's called with a body dict +already present — `validate_delete_element_request()`'s `cascade_delete` +argument is only honored in the `else` branch (body not provided at all); +the `isinstance(body, dict)` branch validates the dict as-is and never +merges `cascade_delete` into it. Several `_async_delete_*` methods +construct a dict body themselves before calling the shared +`_async_delete_element_request` helper (e.g. `CollectionManager._async_delete_collection` +does `body = {"class": "DeleteElementRequestBody"}` when its own `body` arg +is `None`), so `cascade=True` passed to *those* wrappers is silently +ignored too — confirmed live: `_async_delete_collection(guid, cascade=True)` +failed with "not permitted, still has a dependent element" even with +`cascade=True` explicitly passed. Workaround used for cleanup: pass an +explicit `body={"class": "DeleteElementRequestBody", "cascadeDelete": True}` +dict directly. Not fixed here — flagging for a dedicated pass across every +`_async_delete_*` wrapper with this shape, same audit ISSUE-62 should have +covered but evidently didn't catch this specific code path. --- -### ISSUE-87: `ClassificationExplorer.add_ownership_to_element`'s docstring sample body says `"class": "OwnerProperties"` — the method itself only accepts `"OwnershipProperties"`, so the documented body cannot be sent +### ISSUE-105: "Cluster B" — 8 real fixes (casing/phantom-key bugs, missing fields, an unwired relationship, a scope-narrower-than-declared field) plus 6 compact-spec attributes removed as confirmed cruft, plus an active Glossary-classification bug found along the way -**Layer:** Pyegeria · **Status:** fixed 2026-09-05 (Pyegeria — -`pyegeria/omvs/classification_explorer.py`). Both docstrings (async -`_async_add_ownership_to_element` and its sync wrapper -`add_ownership_to_element`) corrected to `"OwnershipProperties"`, matching -the code's own `prop=["OwnershipProperties"]` and the `.http` ground -truth. Verified: `TypeAdapter(NewClassificationRequestBody).validate_python(...)` -with the corrected body now validates cleanly (previously raised -`ValidationError`/`unexpected property class name` from the class-name -mismatch). Swept the rest of the file per the original report's suggestion -(every `_async_new_classification_request` caller, matched by function -boundary rather than naive line-proximity to avoid false positives from -adjacent functions) — this was the only class-name/`prop=` mismatch in -`classification_explorer.py`; not repeated elsewhere. Found 2026-09-04 -(Resource Explorer, `Ownership` on everything it publishes). +**Status:** fixed 2026-09-18 (Dr.Egeria/pyegeria — `md_processing/v2/glossary.py`, +`md_processing/v2/feedback.py`, `md_processing/v2/data_designer.py`, +`md_processing/v2/embedded_process.py`, `md_processing/v2/report.py`, +`md_processing/v2/saved_query.py`, `md_processing/v2/solution_architect.py`, +`md_processing/v2/collection_manager_processor.py`, +`md_processing/v2/project.py`, `pyegeria/omvs/my_profile.py`, plus 5 +compact-spec family files), verified live with a combined throwaway-element +test. Continuing the ISSUE-99/100/101/103/104 follow-up list — "Cluster B", +the lower-confidence remainder of the original attribute-consumption audit +(confirmed absent only by grep, not read in full context). Every item was +independently re-confirmed real-vs-cruft via the live type system before +touching anything, per the discipline ISSUE-103 established. -`classification_explorer.py:8231` `_async_add_ownership_to_element` documents: +**Real fixes:** +- **`Example`** (`Create Glossary Term`, `Create Question`) — real field + `examples` (plural) on `GlossaryTermProperties`; `TermProcessor` read the + phantom key `'Examples'` (plural — spec attribute is singular `Example`); + `QuestionProcessor` never referenced it at all. Both fixed. +- **`Deployed Implementation Type`/`Resource Name`** (`Create Embedded + Process`, `Create Report`, `Create Saved Query`) — real `AssetProperties` + fields (confirmed `.http`), inherited via each command's `Asset` bundle. + All three processors only built their own type-specific extras, never + these two shared-but-inherited fields. Added to all three. +- **`Default Media Usage Other Id`/`Media Type Other Id`** (`Create Related + Media`, plus a bonus find in the adjacent `Cited Document` block) — + case-mismatched phantom keys (`...Other ID` vs spec's `...Other Id`), 3 + sites fixed. +- **`Grain Statement`/`Granularity Basis`/`Interval`** (`Create Data + Grain`) — real `DataGrain` own-attributes, confirmed via the live type + system; `DataGrainProcessor` only called the fully generic + `set_element_prop_body()`. Added all 3. **Also caught and fixed while + verifying live:** `Interval`'s compact-spec `style` was `Simple Float`, + but the real server-side type is `long` — sending `1.0` failed with + `InvalidFormatException`. Corrected style to `Simple Int`. +- **`Role List`** (`Create Solution Blueprint`) — real, unwired + relationship, same shape as the already-fixed `In Data Structure` + (ISSUE-97). `BlueprintProcessor` had no sync of any kind for it. Added a + `_sync_role_list` method mirroring the existing `_sync_components` + pattern (plain `CollectionMembership`, same relationship the standalone + `Link Actor to Blueprint` command already uses). +- **`Objective`** (`Create Meeting` only, per explicit decision — NOT + ToDo/Review/Note, despite the compact spec's shared description implying + otherwise) — real `MeetingProperties` field, confirmed live; absent from + `ToDo`/`Review`/`Action`. `ProjectProcessor`'s Meeting branch calls + `MyProfile._async_create_meeting()`, which had no `objective` parameter + at all — added one (both to the SDK method and the processor call site). -``` -{ - "class" : "NewClassificationRequestBody", - "properties" : { - "class" : "OwnerProperties", - "owner" : "Add value here", - ... -``` +**Active bug, not just a missing field — `Create Glossary`'s +`Is Canonical`/`Is Taxonomy`/`Canonical Scope`/`Organizing Principle`:** +`CollectionManagerProcessor` was unconditionally applying **both** +`Taxonomy` and `CanonicalVocabulary` classifications to every single +Glossary ever created, regardless of what the user set (or didn't set) — +already-running wrong behavior in production, not a dormant gap. Found: +`md_processing/v2/glossary.py`'s `GlossaryProcessor` already had the +*correct* conditional logic written — but `GlossaryProcessor` is **never +registered** in `setup_dispatcher()` (`Glossary` auto-routes via +`COLLECTION_SUBTYPES` to `CollectionManagerProcessor` instead) — fully dead +code, confirmed by grepping the entire dispatcher registration. Fixed in +the code path that actually runs: classifications now applied only when +their respective `Is Canonical`/`Is Taxonomy` flag is true (both default +`False` per the compact spec, matched here), with `Canonical Scope`→`scope` +and `Organizing Principle`→`organizingPrinciple` wired in. **This fix +surfaced ISSUE-106** (below) — the classification-conditional logic was +right, but the properties still didn't persist until that separate, +deeper pyegeria bug was found and fixed too. -and then calls `self._async_new_classification_request(url, prop=["OwnershipProperties"], body=body)` -(`:8285`). `_server_client.py:6175` `validate_new_classification_request` compares -`body["properties"]["class"]` against `prop` and raises -`PyegeriaInvalidParameterException(reason="unexpected property class name")` -when it does not match — **before any HTTP request is made**. So the body in the -docstring is rejected client-side, 100% of the time. +**Confirmed cruft, removed from the compact spec (user decision) — 6 +attributes, cross-file (the `compact-attr-global-namespace` global-namespace +pattern applied to the *removal* side too: some of these existed +identically in multiple family files and needed cleanup in each):** +- `Expected Behavior` (`Create Activity/Blog/Journal Entry`) — zero own + attributes on `Notification`/`JournalEntry`/`ActivityEntry`/`BlogEntry` + in the live type system. +- `Due Time`/`Requested Start Time` (`Meeting`/`ToDo`/`Review`/`Note`) — the + `.http` "confirmation" that looked real turned out to be a copy-paste + artifact (the same oversized field block repeated under 5 unrelated + class headers, including `NotificationProperties`, which the live type + system separately confirms has zero own fields). `MyProfile`'s own SDK + methods have no parameter or override mechanism for either, confirming + no real path ever existed. +- `Role Identifier`/`Role Type` (`Create Solution Role`) — zero own + attributes on `SolutionActorRole` or its super `ActorRole`; `Role + Type`'s own description referenced an unrelated real type + (`GovernanceRole`), reading as confused/stale spec authoring. +- `Subscription Level` (`Create Digital Subscription`) — redundant + duplicate of the already-correctly-wired `Support Level` + (`supportLevel` on `DigitalSubscriptionProperties`, confirmed `.http`); + no `subscriptionLevel` field exists anywhere in the ground truth. -`pyegeria/http clients/Egeria-api-classification-explorer.http:670` uses -`"OwnershipProperties"` and agrees with the code. The docstring is the outlier; -the same sample appears again in the sync wrapper at `:8288`. +**Bundle/attribute removal mechanics:** `Person Action Base` (shared +byte-for-byte across `commands_feedback_compact`, `commands_project_compact`, +`commands_actor_manager_compact`) needed `Due Time`/`Requested Start Time` +stripped from all 3 copies; doing so exposed that 2 of the 3 files were +each already missing a *different* pre-existing shared attribute locally +(`Priority` absent from feedback/actor_manager, `Situation` absent from +project/actor_manager) despite their bundles referencing them successfully +via cross-file resolution at runtime — copied the canonical definitions +into the files missing them (matching the `compact-attr-global-namespace` +"always copy, never reinvent" rule) before the bundle `PUT` would validate. +Not something introduced by this session; a latent inconsistency this +cleanup pass happened to surface and correct as a side effect. -**How it was found:** by running the documented body against the live -quickstart platform. The exception names neither the class it expected nor the -class it got — `unexpected property class name` with an empty context — so -reading the error does not lead to the answer; only reading -`validate_new_classification_request` does. +**Verified live:** one combined throwaway-element test file (Glossary + +Term + Question + Embedded Process + Related Media + Data Grain + Solution +Role + Solution Blueprint with `Role List` + Meeting with `Objective`). +First `--process` run: 1 failure (`Interval`'s float-vs-long style bug, +found and fixed as above), everything else succeeded. Re-ran clean after +the style fix. Fetched every created element back individually and +confirmed: `examples` on both Term and Question; `Taxonomy`/ +`CanonicalVocabulary` classifications present conditionally with +`organizingPrinciple`/`scope` populated (after the ISSUE-106 fix); +`grainStatement`/`granularityBasis`/`interval` on the Data Grain; +`objective` on the Meeting; the `SolutionActorRole`↔`SolutionBlueprint` +membership from `Role List` (confirmed indirectly — a delete of the +blueprint correctly failed with "still has a dependent SolutionActorRole +element" until the role was removed first). All throwaway elements +deleted afterward. `scripts/dr_egeria_attribute_consumption_audit.py` +re-run scoped to every touched command: all 0 findings. +`pytest tests/micro-tests -m unit`: no regressions. -**Ask:** correct both docstrings to `"OwnershipProperties"`. Worth a sweep of -the other `_async_new_classification_request` callers in the same file for the -same docstring/`prop=` divergence — the two were written from different sources -here, so this is unlikely to be the only one. Separately, the validation error -would be far more useful if it named the expected and supplied class names; -`{"reason": "unexpected property class name"}` is a hard error to act on. +**Found, not fixed — a second unrelated dead-code discovery, same shape as +ISSUE-104's "Associated Group":** `Classify Glossary as Canonical`/ +`Classify Glossary as Taxonomy` are declared commands with no registered +processor (surfaced by the audit script's own "unrouted" reporting) — may +be an intended alternative, post-creation classification mechanism that +was never wired up, or may be superseded by this fix's create-time +handling. Not investigated further; flagging for whoever next touches the +Glossary family. --- -### ISSUE-86: `exec_report_spec` (and `_exec_analytic_chart`, `_run_report_spec`) accept only `user`/`user_pass` — no way to run a report with a bearer token the caller already holds +### ISSUE-104: "Cluster A" — 4 relationship-property gaps, a phantom-key relationship-endpoint bug, a genuinely-missing hierarchy relationship, and one Link command with no implementation at all -**Layer:** Pyegeria · **Status:** fixed 2026-09-05 (Pyegeria — -`pyegeria/view/format_set_executor.py`, `pyegeria/core/mcp_adapter.py`). -Added `token: Optional[str] = None` to `exec_report_spec`, -`_exec_analytic_chart`, and `_run_analytic_function` (the function this -entry's title calls `_run_report_spec` — no such name exists in the -codebase; `_run_analytic_function` is the one all three client-building -call sites actually route through), threaded through every place a client -is built from `view_server`/`view_url`/`user`/`user_pass`. When `token` is -given, the client is authenticated with `set_bearer_token(token)` instead -of `create_egeria_bearer_token()`; when omitted, behavior is byte-for-byte -unchanged (`user`/`user_pass` still mint a fresh token exactly as before). -Also threaded through `run_report`/`_execute_egeria_call_blocking` in -`pyegeria/core/mcp_adapter.py`, per the original ask's "same for -run_report/... on the MCP side if they share the builder" — both do, -both now accept `token`. `describe_report` doesn't build a client at all -(pure registry lookup), so it needed no change. The MCP server's own -`run_report` tool (`pyegeria/core/mcp_server.py`) takes a different, -already-token-capable path (`_async_run_report_tool` + a pre-built -`egeria_client`, not `exec_report_spec`'s builder) — out of scope per the -same "if they share the builder" qualifier, not touched. +**Status:** fixed 2026-09-18 (Dr.Egeria/pyegeria — `md_processing/md_processing_utils/common_md_utils.py`, +`md_processing/v2/data_designer.py`, `md_processing/v2/governance.py`, +`pyegeria/models/models.py`), verified live against `qs-view-server` with a +single combined throwaway-element test covering all 7 items. Continuing the +ISSUE-99/100/101/103 follow-up list, picked as "Cluster A" — the +higher-confidence half of what remained, per a live type-system check on +each item before touching code (same discipline ISSUE-103 established). -New unit test `tests/micro-tests/test_exec_report_spec_token.py` (2 tests, -monkeypatched `EgeriaTech`, no live server): confirms `token` routes -through `set_bearer_token` with zero calls to `create_egeria_bearer_token`, -and confirms the no-token path is unchanged. Full `pytest -tests/micro-tests/` green (exit 0) throughout. +**1-3. Three relationship-property gaps, one shared fix.** `Zone +Membership`/`Last Notification`/`Activity Status` (`Link Notification +Subscriber`), `Assignment Type` (`Link Assignment Scope`), and `Expected +Time Allocation Percent` (`Link Person/Team Role Appointment`) were all +declared real by the live type system (`NotificationSubscriber`/ +`AssignmentScope`/`PersonRoleAppointment`/`TeamRoleAppointment` relationship +defs) but never read — all three routes went through the fully generic +`set_rel_prop_body()` in `common_md_utils.py`, which only builds +`description`/`label`/`typeName`/`effectiveFrom`/`effectiveTo`/ +`extendedProperties`. Fixed with one shared edit: `set_rel_prop_body` now +branches on its own computed `prop_name` (the real Egeria type name) to add +each type's own fields, mirroring the `update_gov_body_for_type`/ +`set_collection_manager_body` pattern already used for element bodies. +Also added the missing `last_notification` field to pyegeria's existing +`NotificationSubscriberProperties` Pydantic model (not a live double-gap +today, since this write path uses raw dicts, but worth fixing for +consistency/future use). -**Original report, found 2026-09-04** (Egeria Advisor, trellis-auth token -contract change): +**4. `In Data Field` — same shape as the already-fixed `In Data Structure`, +plus a second bug found while fixing it.** Real (`NestedDataField` +relationship, confirmed via an existing code comment). Also found: +`DataFieldProcessor._sync_all_rels()` read a key, `'Parent Data Field'`, +that doesn't exist anywhere in the compact spec at all — a phantom-key bug +(ISSUE-100 `Allow Duplicates` shape) in code that ISSUE-97/101's fixes to +this exact method had already touched twice this week and missed. Renamed +to `'In Data Field'` with the singular-`guid` fallback (`max_cardinality: 1`, +same pattern as the sibling fixes). -`pyegeria/view/format_set_executor.py:686` `exec_report_spec(..., user=, user_pass=)` builds its own -client (`:189` and `:814`: `client_class(view_server, view_url, user_id=user, user_pwd=user_pass)` -then `client.create_egeria_bearer_token()`) and passes `user`/`user_pass` down to `_exec_analytic_chart` -(`:718`) and the format-row path (`:780`). There is no parameter for a pre-built client or a bearer -token, and `create_egeria_bearer_token()` is unconditional. +**5-6. `In Data Value Specification`/`Specializes Data Value Specification` +— genuinely missing, user confirmed both are the same relationship.** Both +attributes read as near-synonyms in the compact spec; user decided to treat +them as aliases of the one real `DataValueHierarchy` relationship (the same +one `Link Data Value Composition` already uses via +`_async_link_specialized_data_value_specification`). Neither was wired at +all on `Create Data Class`, `Create Data Grain`, or `Create Data Value +Specification` — none of the three processors synced any relationship for +either attribute. Added a `_sync_value_spec_parent`-style sync to all three +(as a new 5th section in `DataClassProcessor`'s existing `_sync_all_rels`; +as new standalone helper methods on `DataGrainProcessor` and +`DataValueSpecificationProcessor`, which previously had *no* relationship +sync of any kind). -**Why it matters:** Egeria Advisor's app JWT carries the user's Egeria bearer token and no longer -carries the password (matching the Portal, whose `X-Egeria-Token` + `set_bearer_token()` pattern -already does this for every PyegeriaWebHandler endpoint). Every other EA call site builds a client -and calls `set_bearer_token(token)`; report execution cannot, so EA falls back to the service -account for that one path and Egeria's provenance records the service account instead of the -person who ran the report (`advisor/report_pipeline.py`, search "ISSUE-86"). +**7. `Link Associated List` — not a field-mapping bug, a completely +unimplemented command.** `governance.py`'s `endpoint_map` had no entry for +`"Associated List"` at all — every invocation raised +`NotImplementedError`. No dedicated SDK method exists for +`AssociatedSecurityList` (`SecurityAccessControl↔SecurityList`, confirmed +real via the live type system), and no `.http` worked example exists +either. Found and reused the actually-correct generic mechanism instead of +writing a new bespoke SDK method: `MetadataExpert._async_create_related_elements`/ +`_async_detach_related_elements_in_store` (wrapping the generic +`createRelatedElementsInStore`/`.../detach-all` endpoints, already proven in +production by `AsyncBaseCommandProcessor._sync_parent_relationship` for the +same "no bespoke wrapper exists" reason). **Explicitly did not** route this +through the neighboring `_async_link_peer_definitions` call used for +`Associated Group`/`Regulation Certification Type` — that method's own +docstring restricts it to `GovernanceDriverLink`/`GovernancePolicyLink`/ +`GovernanceControlLink`, and `SecurityList` is not a governance-definition +peer of `SecurityAccessControl`. One body-shape gotcha caught before +shipping: the generic detach call validates against +`OpenMetadataDeleteRequestBody` (strict `class` literal), not the +`DeleteRelationshipRequestBody` shape every other branch in this method +builds — had to construct that request body separately rather than reusing +the shared one. -**Ask:** add `token: str | None = None` (or `client: EgeriaTech | None = None`) to `exec_report_spec` -and thread it to the two builders; when given, `set_bearer_token(token)` instead of -`create_egeria_bearer_token()`. Same for `run_report`/`describe_report` on the MCP side if they share -the builder. Backward compatible: `user`/`user_pass` keep working when no token is given. +**Found, not fixed — flagged as pre-existing dead code, adjacent to item +7:** `governance.py`'s `"Associated Group"` handling (both the `endpoint_map` +entry and the `elif object_type in {"Associated Group", ...}` branches, in +both the Create and Detach paths) references a relationship type, +`AssociatedSecurityGroup`, that does not exist anywhere in the live type +system, and no compact-spec command anywhere actually has +`object_type == "Associated Group"` — fully unreachable, apparently a stale +leftover. Left as-is (out of scope for this fix); worth a follow-up cleanup +pass. + +**Verified live:** one combined throwaway-element test file exercising all +7 fixes in a single `--process` run (Notification Type + subscriber +Collection + Security Access Control + Security List + Person + Person Role ++ parent/child Data Fields + parent Data Value Specification + child Data +Class), 0 `FAILURE` rows. Fetched every created element back individually +and confirmed: `activityStatus`/`zoneMembership`/`lastNotification` on the +`NotificationSubscriber` relationship; `expectedTimeAllocationPercent` on +`PersonRoleAppointment`; `assignmentType` on `AssignmentScope`; the real +`NestedDataField` relationship linking child→parent Data Field; the real +`DataValueHierarchy` relationship linking Data Class→Data Value +Specification; and the real `AssociatedSecurityList` relationship linking +Security Access Control→Security List with `operationName` set. All +throwaway elements deleted afterward. +`scripts/dr_egeria_attribute_consumption_audit.py` re-run scoped to every +touched command: all 0 findings (Data Grain's separate, untouched +`Grain Statement`/`Granularity Basis`/`Interval` gap still correctly +appears, confirming the audit still works). `pytest tests/micro-tests -m +unit`: no regressions. + +**Still open from the ISSUE-99/100/101 list:** `Operation Name` is now +fixed (covered above as item 7's `Associated List`). Still open: the +longer single-command tail (`Expected Behavior`, `Example`, `Due +Time`/`Objective`/`Requested Start Time`, `Canonical Scope`/`Is +Canonical`/`Is Taxonomy`, `Subscription Level`, `Deployed Implementation +Type`/`Resource Name`, `Default Media Usage Other Id`/`Media Type Other +Id`, `Role Identifier`/`Role Type`, `Role List`, `Grain +Statement`/`Granularity Basis`/`Interval`) — "Cluster B" in earlier +discussion, lower confidence (grep-confirmed absence only, not read in full +context), likely to share root causes once grouped rather than being 15+ +independent bugs. --- -### ISSUE-84: `SolutionArchitect.create_solution_blueprint`'s own docstring documents a `NewSolutionElementRequestBody` body (with `initialStatus`) for Draft-status creation — that class does not exist as a pydantic model, so the documented shape fails client-side validation before any HTTP call +### ISSUE-103: `Link Agreement Terms and Conditions`'s compact spec declared the wrong relationship type entirely (`CollectionMembership` instead of `AgreementItem`), and the code that already implemented it correctly had its own phantom-key bug -**Status:** fixed and live-verified 2026-09-03 — docstring-only fix (see -"Fix landed" below), merged upstream via -[PR #338](https://github.com/odpi/egeria-python/pull/338). Reported by a -consumer (trellis/Resource Explorer's `BlueprintMaterializer`, -`docs/blueprint-materialization-plan.md` Phase A), who also live-verified -it once their shared `quickstart-egeria-main` instance recovered from an -unrelated Postgres connection-pool exhaustion (`jdbcMaximumPoolSize` bump). +**Status:** fixed 2026-09-17 (Dr.Egeria — compact spec via the Spec Editor +API, `md_processing/v2/governance.py`), verified live against +`qs-view-server` (`Create Agreement` → `Create Terms and Conditions` → +`Link Agreement Terms and Conditions`, fetched back, confirmed the real +`AgreementItem` relationship with all 4 properties persisted, then deleted). +Resolves the "investigated, explicitly not fixed" item from ISSUE-101. -**Confirmed in both**: this checkout's installed pyegeria (5.3.4.23, via -the trellis venv) and this repo's own working tree (6.1.9-dev) — -`grep -rn "class NewSolutionElementRequestBody" pyegeria/models/*.py` -returns nothing in either; the class is documented in -`SolutionArchitect.create_solution_blueprint`'s (and its `_async_*` -sibling's) docstring — twice, once per method, `omvs/solution_architect.py` -lines ~2325 and ~2440 in this tree — as the way to set `initialStatus` -(DRAFT/PREPARED/PROPOSED/APPROVED/REJECTED/ACTIVE/DISABLED/DEPRECATED/OTHER) -on a newly-created blueprint, distinct from `NewElementRequestBody` which -"sets the status to ACTIVE." No such model backs that documented shape. +**Root cause:** the compact spec (`OM_TYPE: CollectionMembership`, bundle +`Collection Membership`) and the processor code +(`GovernanceLinkProcessor`'s `elif object_type == "Agreement T&C":` branch, +building `AgreementItemProperties` and calling `_async_link_agreement_item`) +disagreed about which real Egeria relationship type this command uses. +Confirmed via the live type system (`ValidMetadataManager._async_get_all_relationship_defs()`, +filtered to names containing "Agreement"/"Term") that `AgreementItem` is a +real, distinct relationship (`endDef1: Agreement`, `endDef2: Referenceable`, +fields `agreementItemId`/`agreementStart`/`agreementEnd`/`entitlements`/ +`restrictions`/`obligations`/`usageMeasurements`) with no `membershipType` +field at all — the code was right, the spec was wrong. User confirmed +`linkAgreementItem` was the intended relationship before this was +investigated further. -**What actually happens**: `create_solution_blueprint`'s real validation -path (`ServerClient._async_create_element_body_request` → -`validate_new_element_request` → `self._validate_body(self. -_new_element_request_adapter.validate_python, body)`) uses a bare -`TypeAdapter(NewElementRequestBody)` (set in `ServerClient.__init__`) — -`NewElementRequestBody.class_` is `Annotated[Literal["NewElementRequestBody"], -Field(alias="class")]`, a strict literal. Any other `class` value — -including the documented `"NewSolutionElementRequestBody"` — fails pydantic -validation locally, before any network call, raising -`PyegeriaInvalidParameterException` with `additional_info={"reason": -"Request body failed validation", "validation_errors": [...]}`. The -resulting exception message reads exactly like a server-side rejection -("Egeria rejected the new SolutionBlueprint: ...VALIDATION_ERROR_1... -Invalid parameters were provided...") and cost real debugging time on the -consumer side tracing it back to "not actually an Egeria call at all." +**Fix — compact spec:** changed `OM_TYPE` to `AgreementItem` and pointed the +command at a new bundle `Agreement T&C Base` +(`Agreement Item Id`/`Agreement Start Date`/`Agreement End Date`/`Usage +Measurements`) — deliberately narrower than the sibling `Link Agreement +Item` command's own `Agreement Item` bundle, since `Entitlements`/ +`Obligations`/`Restrictions` already live on the linked `Terms and +Conditions` element itself (its own dedicated bundle), not on this +relationship instance, and reusing the sibling's full bundle would have +forced every user to also fill in a redundant `Item Name` (duplicating +`Terms & Conditions Id`, `min_cardinality: 1`) for no reason — confirmed by +trying the wholesale-reuse approach first and hitting exactly that +validation error live. -Reproduced directly: -```python -from pyegeria.models import NewElementRequestBody -from pydantic import TypeAdapter -TypeAdapter(NewElementRequestBody).validate_python({ - "class": "NewSolutionElementRequestBody", "isOwnAnchor": True, - "initialStatus": "DRAFT", - "properties": {"class": "SolutionBlueprintProperties", - "qualifiedName": "test", "displayName": "test"}, -}) -# ValidationError: Input should be 'NewElementRequestBody' -# [type=literal_error, input_value='NewSolutionElementRequestBody', ...] -``` +**Near-miss caught and fixed, not shipped:** the first attempt created a +new bundle also named `Agreement Item` in the Governance Officer family's +own file. Bundle names are merged globally across every compact JSON file +the same way attribute names are (`compact_loader.py`'s `all_bundles`, +last-file-alphabetically-wins) — undocumented anywhere until now. Digital +Product Manager's file (`digital_products` < `governance_officer` +alphabetically) already had a real, working `Agreement Item` bundle used by +`Link Agreement Item`; the new one would have silently replaced it at the +next `refresh_specs`, corrupting that command's attribute set. Caught only +because `Link Agreement Item` was noticed by chance while researching this +fix — not by any tool, since the Spec Editor's structural validation and +`attribute_sharing` reporting are both file-scoped and don't cover this. +Fixed by renaming to the distinct `Agreement T&C Base` instead. Documented +in the `compact-attr-global-namespace` memory and the +`dr-egeria-command-sync` skill's Step 1, both updated to cover bundles, not +just attributes. -**Scope beyond `create_solution_blueprint`**: `create_solution_component` (and -its `_async_*` sibling) carries the identical docstring text and the -identical gap — confirmed by walking each `NewSolutionElementRequestBody` -docstring reference in `omvs/solution_architect.py` (lines 2254/2372/3542/3629 -in this tree) back to its enclosing method: `_async_create_solution_blueprint`/ -`create_solution_blueprint` (the two checked above) and -`_async_create_solution_component`/`create_solution_component`. Both public -create-element methods in this file document the same unbacked Draft-status -path. Not checked beyond `solution_architect.py`: worth a full grep across -`omvs/*.py` for any other method documenting `NewSolutionElementRequestBody`, -`NewGovernanceElementRequestBody`, or any other `New*ElementRequestBody` -variant that isn't `NewElementRequestBody` itself, since the same -"documented in the docstring, no model behind it" shape could recur -per-family. +**Fix — code:** `GovernanceLinkProcessor`'s `Agreement T&C` branch read +`attributes.get("Start Date", ...)`/`attributes.get("End Date", ...)` — a +phantom-key mismatch (this command's bundle, correctly, never provided those +generic names) found only because properly wiring the bundle exposed it. +The sibling `CollectionLinkProcessor`'s `Link Agreement Item` branch already +correctly used `Agreement Start Date`/`Agreement End Date` for the identical +`agreementStart`/`agreementEnd` fields — renamed to match. -**Candidate fix** (not applied — for whoever picks this up): either (a) -define a real `NewSolutionElementRequestBody` pydantic model (subclassing -or extending `NewElementRequestBody` with an `initial_status` field and its -own `class_: Literal["NewSolutionElementRequestBody"]`) and register it -alongside `NewElementRequestBody` wherever `_new_element_request_adapter` -is built (likely needs a `Union`/discriminated-union `TypeAdapter`, not a -single-model one, so both class names validate), or (b) if -`initialStatus` is actually accepted by `NewElementRequestBody` itself on -the Egeria server side despite not being a declared pydantic field, add -`initial_status: str | None = None` directly to `NewElementRequestBody` -and drop the docstring's separate-class framing — cheaper, but needs -confirming server-side that `NewElementRequestBody` + `initialStatus` is -accepted (not verified here; the consumer's own fallback, tracked in their -own Backlog, is to use `NewElementRequestBody` with no `initialStatus` at -all, which creates the element ACTIVE, not Draft, until this is resolved). +**Verified live:** created a throwaway `Agreement`, `Terms and Conditions`, +and linked them via `Link Agreement Terms and Conditions` with `Agreement +Item Id`/`Agreement Start Date`/`Agreement End Date`/`Usage Measurements` +set. Fetched the `Agreement` back and confirmed a real `AgreementItem` +relationship (not `CollectionMembership`) with all 4 values persisted +exactly as sent, linking to the correct `TermsAndConditions` element. Both +`commands_governance_officer_compact` and `commands_digital_products_compact` +re-validated clean (`structural_ok: true`, no new warnings) after cleanup of +the near-miss. `pytest tests/micro-tests -m unit`: no regressions. -**Consumer-side workaround, now supersedable**: `BlueprintMaterializer. -materialize_blueprint_element` (trellis/resource_explorer, `surveyors/ -arch_recovery/blueprint_materializer.py`) sends `class: "NewElementRequestBody"` -with no status field at all, same as their pre-existing `ComponentMaterializer` -— blueprints materialize ACTIVE, not Draft. With this fix landed, that -workaround can be upgraded to set `contentStatus: "DRAFT"` inside -`properties` on the same `NewElementRequestBody` to get the originally -intended Draft-status behavior — not applied here since this is a -consumer-side change, tracked for `dwolfson-1b`/trellis to pick up. +--- -**Fix landed 2026-09-03 — docstring correction, no model change needed.** -Checked ground truth first: `NewSolutionElementRequestBody` and -`initialStatus` appear in **zero** `.http` files anywhere in this repo -(`grep -rln "NewSolutionElementRequestBody\|initialStatus" "pyegeria/http -clients/"` returns nothing) — the documented shape was never a real Egeria -API surface, not merely an unmodeled one, which rules out candidate fix (a) -(defining a real model) and (b) (adding `initial_status` to -`NewElementRequestBody`) from the original write-up. The actual, real -mechanism for setting status on these elements already exists: -`Egeria-api-solution-architect.http`'s own `updateSolutionBlueprintStatus` -example sets a blueprint's status via `"contentStatus": "ACTIVE"` inside an -`AuthoredReferenceableProperties`-shaped `properties` object — and -`content_status: str | None = None` is already a real field on -`ReferenceableProperties` (`pyegeria/models/models.py`), the base class -both `SolutionBlueprintProperties` and (transitively) -`SolutionComponentProperties` derive from. Since `NewElementRequestBody. -properties` is a bare `dict` (no nested pydantic validation), passing -`contentStatus` inside it at *creation* time was always possible — the -docstrings just never said so, and instead pointed at a body shape that -doesn't exist anywhere. +### ISSUE-102: `MemberDataField.minCardinality` silently persists as `maxCardinality`'s value regardless of what's actually sent — server-side, confirmed via a raw request bypassing every pyegeria/Dr.Egeria layer -Rewrote all four docstrings (`_async_create_solution_blueprint`/ -`create_solution_blueprint`, `_async_create_solution_component`/ -`create_solution_component`, `omvs/solution_architect.py`) to drop every -reference to `NewSolutionElementRequestBody`/`initialStatus` and document -`contentStatus` inside `properties` instead, with a `NewElementRequestBody` -example that now sets `"contentStatus": "DRAFT"`. Two extra defects found -and fixed while doing this walk, both worse than the one originally -reported: -- `_async_create_solution_blueprint`'s *first* ("no lifecycle") example - body already carried stray `"userDefinedStatus"`/`"lifecycleStatus"` - fields left over from the fictional shape — with a **missing comma** - after `"lifecycleStatus": "DRAFT"`, making it invalid JSON even as a - copy-paste example, independent of the pydantic-validation bug. -- `_async_create_solution_component`/`create_solution_component`'s only - example already used the real `"class": "NewElementRequestBody"` (so it - passed validation) but paired it with the fictional top-level - `"initialStatus"` field anyway. This is a *different, silent* failure - mode from the reported one: `NewElementRequestBody` inherits - `PyegeriaModel`'s `extra='ignore'`, so `initialStatus` validated - successfully and was silently dropped before serialization — no - exception at all, just an element that always came out ACTIVE regardless - of the caller's intent to set DRAFT. Exactly the class of bug flagged - generally in this file's own `pyegeria/core/` gotcha note about - request-body models silently dropping undeclared fields. +**Layer:** Egeria Server (repository/relationship-property persistence) · **Status:** NO LONGER REPRODUCES on the 2026-10-05 rebuilt platform (see the re-test below) · **Found:** 2026-09-17, live-verifying a Dr.Egeria fix for `Position`/`Minimum Cardinality`/`Maximum Cardinality` on the field↔structure `MemberDataField` relationship (`qs-view-server`/`qs-metadata-store`, versionName `6.2-SNAPSHOT`). -Verified locally (no live server access from this environment): the -original repro's `TypeAdapter(NewElementRequestBody).validate_python(...)` -call with the old `NewSolutionElementRequestBody`/`initialStatus` shape -still fails as expected (proving the understanding of the original bug is -correct), and the corrected shape -(`{"class": "NewElementRequestBody", "properties": {..., -"contentStatus": "DRAFT"}}`) validates cleanly with `contentStatus` -preserved intact through to the serialized JSON. +Confirmed with the type system's own definition +(`ValidMetadataManager._async_get_all_relationship_defs()`, filtered to +`MemberDataField`) that the three real attributes are `position`, +`minCardinality`, `maxCardinality` — all plain `int`, `AT_MOST_ONE` +cardinality, no documented interdependency between `minCardinality` and +`maxCardinality`. -**Live-verified 2026-09-03 by `dwolfson-1b`/trellis**, once their shared -`quickstart-egeria-main` instance recovered from unrelated Postgres pool -exhaustion: `contentStatus: "DRAFT"` inside `properties` on -`NewElementRequestBody` creates without error and round-trips correctly on -read-back (`properties.contentStatus == "DRAFT"`), no separate request-body -class needed — confirmed twice, once as a standalone probe and once through -Resource Explorer's actual application path (accepting a real candidate -blueprint end-to-end via the Curate UI's API, materialized element guid -`809025b5-cca9-4e9a-a2f7-3a5104138f67`, independently queried back by that -guid to confirm). `elementHeader.status` stays `ACTIVE` regardless, as -expected — a different axis (header status vs. `contentStatus`, distinct -concepts in Egeria's status model), not a sign the fix is incomplete. +**Re-tested 2026-10-05 on the rebuilt platform** (Egeria image built 14:19Z from main `449ad06894`, +`egeria-main` restarted 14:39Z). One throwaway `DataStructure` and four `DataField`s were linked with +`MemberDataField` and each relationship was read back via the endpoints lookup +(`get_metadata_element_relationships`). **All four stored correctly** (`position` always right): -`BlueprintMaterializer.materialize_blueprint_element` has been upgraded to -send `contentStatus: "DRAFT"` in `properties`, replacing its earlier -ACTIVE-only workaround. `ComponentMaterializer`'s identical gap is -**still open** — same fix applies, not yet applied there, tracked -separately on the trellis/Resource Explorer side. +| sent (min, max) | stored (min, max) | +|---|---| +| 1, 5 | 1, 5 | +| 2, 7 | 2, 7 | +| 0, 5 | 0, 5 | +| 3, *omitted* | 3, 0 | -**GUID note (2026-09-04):** `quickstart-egeria-main` was redeployed with a full -repository-store wipe the same day this was verified. `809025b5-cca9-4e9a-a2f7-3a5104138f67` -is historical — valid as of the 2026-09-03 verification date, not resolvable against the -platform after the wipe. The finding (`contentStatus` round-trips via `properties`, no -separate request-body class needed) is a property of the API/fix, not of that instance, and -is unaffected. +**Raw capture for the Egeria team, 2026-10-06 (about 02:00Z):** at their request, one more throwaway `DataStructure` + `DataField` were linked with the exact request `NewRelationshipRequestBody{properties: MemberDataFieldProperties{position 3, minCardinality 1, maxCardinality 5}}` (answer `VoidResponse`), and the relationship was read back through the endpoints lookup (`OpenMetadataRelationshipListResponse`): `minCardinality` "1", `maxCardinality` "5". Still no repro, now with the raw request, raw response and read-back captured (kept locally, session scratchpad `live_102_raw.out`; the team's three asks are met). Deleted child-first, verified gone by GUID. ---- +Everything was deleted child-first and verified gone. **The bug no longer reproduces.** The cause of the fix is not +known: a source comparison of the bean (`MemberDataFieldProperties`/`PartOfRelationshipProperties`), the read +converter and the relationship builder between 2026-09-15 and `449ad06894` found no change, so either the fix is +in code that was not inspected or the build that showed the bug on 2026-09-17 was older than that baseline. Worth +asking the Egeria team which change fixed it before closing this for good. Side observation, separate from this +bug: an omitted `maxCardinality` is stored as `0` (the bean default), below a supplied minimum; the bean's own +documentation says `-1` means unlimited. -### ISSUE-82: `pyegeria/omvs/valid_metadata.py` sends the literal query string `typeName=None` whenever `type_name` is Python `None` — breaks every Type-Name-omitted (global) Valid Metadata Value, in 12 of 14 methods across `ValidMetadataManager` +**Repro (isolated with a raw SDK call — no Dr.Egeria markdown, no pyegeria +body-construction logic in the path beyond `EgeriaTech.data_designer`):** +```python +link_body = { + "class": "NewRelationshipRequestBody", + "properties": { + "class": "MemberDataFieldProperties", + "position": 3, + "minCardinality": 1, + "maxCardinality": 5, + }, +} +await client.data_designer._async_link_member_data_field(struct_guid, field_guid, link_body) +# fetch the relationship back: +# position: 3 -- correct, matches what was sent +# minCardinality: 5 -- WRONG, silently coerced to maxCardinality's value (5), not the sent 1 +# maxCardinality: 5 -- correct +``` +`position` round-trips correctly, ruling out a client-side body-construction +bug (the exact same request dict's `position` key is honored, its +`minCardinality` key is not). No error, warning, or validation failure is +returned — the create call reports success and returns a real relationship +GUID; the wrong value is simply what gets stored. -**Status:** fixed and live-verified 2026-08-28 — all 12 affected methods -patched to match the one method that already did this correctly (see "Fix -landed" below); released as pyegeria 6.1.7. Verified with a mocked HTTP -layer first (URL no longer contains the literal string `None`, correctly -omits `typeName` when unset, and still includes it when set), then -confirmed live: `egeria-workspaces-5f` upgraded quickstart-pyegeria-web's -container to 6.1.7 and re-ran all 5 originally-failing files -(`human-resource-management.md`, `health-and-safety.md`, -`biological-agents-and-gmo.md`, `dangerous-goods-transport.md`, -`diversity-equity-inclusion.md`) against a live `qs-view-server` — every -one now exits 0/SUCCESS, zero occurrences of the "not a valid metadata -value", "Missing unresolved reference", or -`PyegeriaNotFoundException`/404 signatures across all 5. Also confirmed -the `type_name` guard is present in the installed package, not just this -checkout. `egeria-workspaces`' own `BACKLOG.md` entry (`PY-25`) closed as -verified on their side. +**Also worth flagging separately, lower confidence:** the +`_async_link_member_data_field` SDK method's own docstring sample body +(`pyegeria/omvs/data_designer.py` ~line 2286) shows `dataFieldPosition` as +the position field's name — that's wrong per the same live type-system +query (the real name is `position`); not filed as its own issue since it's +a pyegeria-side docstring fix, not an Egeria server issue, but noted here +since it was found in the same investigation and would otherwise cause a +second, harder-to-diagnose silent-drop bug for the next person who trusts +that docstring. -**Reported by another Claude session** (`egeria-workspaces-5f`, no local -`egeria-python` checkout) debugging a failed Dr.Egeria batch run. A -`## Setup Valid Metadata Value` command that registers a new -`domainIdentifier` value with `### Type Name` deliberately omitted (i.e. -"applies to all open metadata types") is never visible to the very next -command's `### Domain Identifier` field validation in the same file: +**Ask:** confirm whether `minCardinality` defaulting to `maxCardinality` +when both are supplied is intentional server-side business logic (e.g. a +"min cannot exceed max, so raise min to max" normalization applied even +when min is explicitly below max and both are valid on their own) or a +genuine persistence bug; if intentional, it isn't reflected anywhere in the +`MemberDataField` relationship def's own attribute descriptions. -``` -ERROR | md_processing.v2.processors:615 - Validation failed: Value 'Human -Resource Management' is not a valid metadata value for 'Domain Identifier' -(Validated by Egeria) for Create Business Imperative -``` +**Dr.Egeria-side impact:** `Create Data Field`'s `Minimum Cardinality` +attribute is parsed and sent correctly (see `PYEGERIA_ISSUES.md` ISSUE-101) +but its stored value cannot be trusted independently of `Maximum +Cardinality` until this is resolved server-side — nothing further is +fixable in this repo for that specific symptom. -Confirmed deterministic, not a timing/cache race (re-ran the same file -twice, 20+ minutes apart, identical failure both times). Real-world repro: -`egeria-workspaces`' `compose-configs/egeria-quickstart/PyegeriaWebHandler/ -coco-workbooks/0. data-governance-program/` — exactly the files that -self-register a custom domain before using it -(`human-resource-management.md`, `health-and-safety.md`, -`biological-agents-and-gmo.md`, `dangerous-goods-transport.md`, -`diversity-equity-inclusion.md`) fail 100% reproducibly; the other 12 -files in the same batch, which only use Egeria-builtin/CocoComboArchive -domains, run clean. Downstream cascade: every `Create ` that references the new domain fails validation, so every later -`Link Governance Response/Mechanism/Drivers/Policies` in the same file then -fails with `Missing unresolved reference GUID(s)` (~20+ occurrences per -file), and `Add Member to Collection` 404s (`OMAG-REPOSITORY-HANDLER-404-007`) -trying to add a member that was never created. +**Egeria team response, relayed 2026-10-05 (source `oak2026` fb3d6fce53; **fixes are in two unmerged PRs, so no build contains them, and nothing was tested against a live platform**):** NOT reproduced in Egeria (builder and converter round trip returns `position=3, min=1, max=5`; lower layers not traced). They asked for the exact request, the raw response and the read-back endpoint. Ours: request `NewRelationshipRequestBody` with `properties` `{class: MemberDataFieldProperties, position: 3, minCardinality: 1, maxCardinality: 5}` via `DataDesigner._async_link_member_data_field`; the original raw response was not captured. On the 2026-10-05 rebuilt platform it does not reproduce for us either (see the re-test above). -**Root cause — confirmed in source, this repo's own code.** Every method on -`ValidMetadataManager` (`pyegeria/omvs/valid_metadata.py`) takes an optional -`type_name`, documented as: *"If the typeName is null, this valid value -applies to properties of this name from all types."* Exactly one method -honored that — `_async_get_valid_metadata_values` — with -`if type_name: url += f"&typeName={type_name}"`. Every other method instead -hardcoded the query param unconditionally into an f-string, e.g. -`_async_validate_metadata_value`: +**Update 2026-10-10:** the `oak2026` source (`fb3d6fce53`) the team examined is now in Egeria `main` (merged 2026-10-06, odpi/egeria#9363). No fix was made for this issue; the team's questions above still stand. -```python -url = ( - f".../validate-value/{property_name}?typeName={type_name}&actualValue={actual_value}" -) -``` +--- -When `type_name` is Python `None`, f-string interpolation renders it as the -four-character string `"None"` — so the request sent is literally -`?typeName=None&actualValue=Human Resource Management`, not "no type -filter." Egeria correctly (from its point of view) doesn't recognize a -type named `None` and treats the value as not found/not valid, for **both** -the initial registration and every later lookup — so this isn't a filter -that merely excludes globally-scoped values, it's the *setup* call itself -that never actually registers a true global (`typeName IS NULL`) entry -either; it registers under (or is rejected against) the bogus literal type -`"None"`. That fully explains the reporter's own top hypothesis ("is the -later validation scoped by a Type Name filter that a Type-Name-omitted -registration doesn't satisfy?") — yes, effectively, though the mechanism is -a string-interpolation bug, not an intentional filter. +### ISSUE-101: `Is Case Sensitive` never mapped, field↔structure link sent the wrong wire property name for `Position`, and `Membership Type`/`Dependency Description` were phantom-key or unmapped on 5 relationship-property bodies -Affected methods (12): `_async_setup_valid_metadata_value`, -`_async_setup_valid_metadata_map_name`, -`_async_setup_valid_metadata_map_value`, -`_async_validate_metadata_value`, `_async_validate_metadata_map_name`, -`_async_validate_metadata_map_value`, `_async_get_valid_metadata_value`, -`_async_get_valid_metadata_map_name`, `_async_clear_valid_metadata_value`, -`_async_clear_valid_metadata_map_name`, -`_async_clear_valid_metadata_map_value` (11 async methods; their sync -`loop.run_until_complete` twins inherit the fix for free since they just -delegate). `_async_get_valid_metadata_values` and -`_async_get_valid_metadata_map_value` (the latter passes `type_name` via a -`params=` dict, which httpx already drops when the value is `None`) were -already correct and untouched. - -This is exactly the flow `md_processing/v2/parsing.py`'s `Valid Value` -style hits for the `Domain Identifier` attribute (`property_name` = -`domainIdentifier`, `type_name` = `None` — the compact spec for `Domain -Identifier` has no `type_name` key at all) — so any deployment-added, -type-unscoped `domainIdentifier` (or any other property registered the -same way) was unreachable through this path regardless of caching, exactly -matching "not a race... possibly not ever." +**Status:** fixed 2026-09-17 (Dr.Egeria — `md_processing/v2/data_designer.py`, +`md_processing/v2/collection_manager_processor.py`, +`md_processing/v2/solution_architect.py`), verified live against +`qs-view-server` with throwaway elements, including one raw-SDK isolation +test that caught a second bug this fix's own first attempt introduced. +Continuing the ISSUE-99/100 follow-up list. -**Fix landed:** every affected method now builds the URL without the query -param first, then conditionally appends it — the same pattern -`_async_get_valid_metadata_values` already used: +**`Is Case Sensitive`** (Create Data Class) — confirmed by the user to be a +real `DataClass` field. Added `"isCaseSensitive": attributes.get('Is Case +Sensitive', {}).get('value')` to `DataClassProcessor`'s properties dict. +Verified live: `True` persisted correctly. -```python -url = f".../validate-value/{property_name}?actualValue={actual_value}" -if type_name: - url += f"&typeName={type_name}" -``` +**`Position`/`Minimum Cardinality`/`Maximum Cardinality`** (declared on +`Create Data Field`, meant for the field↔structure `MemberDataField` +relationship) — **the real fix site was not `LinkFieldToStructureProcessor`** +(the standalone `Link Data Field to Data Structure` command has no +cardinality attributes at all) but `DataFieldProcessor._sync_all_rels()`, +whose `add` lambda for the Data Structures sync called +`_async_link_member_data_field(ds, guid, None)` — body always `None`. Fixed +by threading `attributes` through both call sites into `_sync_all_rels` and +building a real `MemberDataFieldProperties` body. -**Nothing left open here.** Fixed, released (pyegeria 6.1.7), and live-verified -end-to-end against the original real-world repro, not just unit-tested. +**Caught during live verification, not before:** the first attempt used +`"dataFieldPosition"` as the wire key, taken from the SDK method's own +docstring sample body (`data_designer.py` line ~2286) — that docstring is +**wrong**. Live verification showed `position` silently staying `0` (server +default) while `minCardinality`/`maxCardinality` round-tripped, which didn't +match "attribute never sent" (that would leave all three at default). +Queried the live type system directly (`ValidMetadataManager._async_get_all_relationship_defs()`, +filtered to `MemberDataField`) — the real, only attribute name is `position`, +not `dataFieldPosition`. Fixed the key; a fresh throwaway round-trip then +showed `position: 3` (sent value) correctly persisted. **The SDK docstring +itself is still wrong** (not fixed here — out of scope for this pass, but a +real, confirmed-live ground-truth error worth a follow-up). ---- +**Not a bug, a genuine Egeria server quirk — do not attempt to fix +client-side:** even with the correct key, `minCardinality` consistently +persists as `maxCardinality`'s value regardless of what's actually sent. +Isolated with a raw SDK call bypassing Dr.Egeria entirely +(`_async_link_member_data_field` called directly with +`{"position": 3, "minCardinality": 1, "maxCardinality": 5}`) — server still +returned `minCardinality: 5`. Confirmed server-side, not a pyegeria or +Dr.Egeria client bug; noting here so nobody re-investigates the client code +for this specific symptom. -### ISSUE-83: `AutomatedCuration.get_technology_type_elements(get_templates=True)` sends `skipClassifiedElements: [""]` (a list containing an empty string) instead of `[]` — Egeria rejects the empty classification name, so every Tech Catalog technology-type listing renders silently empty instead of erroring +**`Membership Type`** — user confirmed this is a real `CollectionMembership` +relationship property (not spec cruft, despite zero hits across every +`.http` ground-truth file — those worked examples are evidently non-exhaustive). +Added to both real fix sites: +- `CollectionLinkProcessor`'s `Add Member to Collection` branch + (`collection_manager_processor.py`). +- `SolutionLinkProcessor`'s generic `om_type == "CollectionMembership"` + branch (`solution_architect.py`) — one fix covers all 4 affected commands + (`Link Actor to Blueprint`, `Link Blueprint Child`, `Link Information + Supply Chain Child`, `Link Solution Component to Blueprint`), confirmed by + checking each command's `OM_TYPE` via the Spec Editor API before editing. -**Status:** fixed and live-verified 2026-09-02. +Verified live (Dr.Egeria pipeline, not raw SDK, to also confirm the parser/ +processor wiring): `membershipType: 'TestMembershipType2'` persisted +correctly on an `Add Member to Collection` command. -**Root cause:** `_async_get_technology_type_elements` -(`pyegeria/omvs/automated_curation.py`) built `skipClassifiedElements` as -`[skip_templates]` where `skip_templates = "Template" if not get_templates -else ""` — when `get_templates=True` (the caller wants templates -*included*, i.e. skip nothing), this sent `["", ]`, a one-element list -containing an empty string, not `[]`. Egeria's server correctly rejects an -empty classification name: +**Investigated, explicitly not fixed — a different, deeper bug:** +`Link Agreement Terms and Conditions` ("T&C") was also on the `Membership +Type` list, but its compact spec (`OM_TYPE: CollectionMembership`, bundle +`Collection Membership`) does not match what the processor code actually +does — `GovernanceLinkProcessor`'s `elif object_type == "Agreement T&C":` +branch builds an entirely different `AgreementItemProperties` body and calls +`_async_link_agreement_item`, which has no `membershipType` field at all. +This is a spec/code type mismatch, not a missing-field gap — adding +`membershipType` to the wrong property class would be a no-op at best. Left +untouched; needs someone to determine whether the compact spec's `OM_TYPE` +is stale or the code's relationship-type choice is wrong, before any fix. -``` -OMAG-COMMON-400-018 The type name passed on method findMetadataElements -of service Open Metadata Store Services is not recognized -``` +**`Dependency Description`** (`Link Product Dependency`) — phantom-key bug, +same shape as `Allow Duplicates` in ISSUE-100: +`CollectionLinkProcessor`'s `Product Dependency` branch read +`attributes.get('Description', ...)`, but this command's `custom_attributes` +are only `["Dependency Description", "Digital Product 1", "Digital Product +2"]` — no generic `Description` attribute exists on it at all. Renamed the +key. **Adjacent phantom key found, not fixed:** the same block also reads +`attributes.get('Label', ...)` — `Label` isn't in this command's +`custom_attributes` either, so it's always `None`; harmless (an optional +field silently unset, not a required one silently dropped) but flagged for +whoever next touches this block. -**Why it went unnoticed:** the failure is masked downstream. `egeria- -workspaces`' `tech_catalog_handler.py` calls this with `get_templates=True` -unconditionally (`get_tech_type_elements`), and its own exception handler -treats any error message containing `"400"` as "no elements found," so the -whole Tech Catalog "Elements" listing for a technology type renders as an -honest-looking, self-consistent **empty list** instead of surfacing the -real 500. Any caller with a similar "400 means empty" convenience handler -would hit the same masking. +**Verification:** all four confirmed fixes re-checked with +`scripts/dr_egeria_attribute_consumption_audit.py` scoped per command — +`Is Case Sensitive` and `Position`/`Minimum Cardinality`/`Maximum +Cardinality` no longer appear on `Data Class`/`Data Field`'s finding lists; +`Add Member to Collection`, `Link Actor to Blueprint`, `Link Product +Dependency` all show 0 findings. `pytest tests/micro-tests -m unit`: no +regressions. -**Found by:** a user report — a known live `RelationalDatabase` asset -(`deployedImplementationType: "PostgreSQL Relational Database"`, confirmed -present via direct GUID lookup) did not appear browsing Technology Types → -Postgres Relational Databases in `egeria-workspaces`' Tech Catalog. -Reproducing the exact call chain (`AutomatedCuration. -get_technology_type_elements(filter_string="PostgreSQL Relational -Database", get_templates=True, ...)`, the parameters `tech_catalog_handler. -py`'s `/api/tech-catalog/tech-types/{qualified_name}/elements` route -actually uses) reproduced the 500 directly; the same call with -`get_templates=False` succeeded and returned the asset correctly (14 -items) — isolating the defect to the `get_templates=True` branch -specifically, not a metadata/zone/visibility problem on the asset itself. +**Still open from the ISSUE-99/100 list:** `Link Agreement Terms and +Conditions`'s spec/code mismatch fixed separately, see ISSUE-103. Still +open: `Operation Name`/`Last Notification`/`Assignment Type`/`Expected Time +Allocation Percent` on other Link commands, and the longer single-command +tail (`Expected Behavior`, `Example`, `Due Time`/`Objective`/`Requested +Start Time`, `Canonical Scope`/`Is Canonical`/`Is Taxonomy`, `Subscription +Level`, `Deployed Implementation Type`/`Resource Name`, `Default Media Usage +Other Id`/`Media Type Other Id`, `Role Identifier`/`Role Type`, `Role List`, +`In Data Value Specification`/`Specializes Data Value Specification`, `In +Data Field`, `Grain Statement`/`Granularity Basis`/`Interval`). -**Fix:** `skip_classified_elements = [] if get_templates else ["Template"]` -— an empty list (skip nothing) when templates should be included, a -one-element list naming the real `Template` classification to skip -otherwise. Also corrected the method's own docstring sample JSON body, -which showed the same `[""]` value as if it were correct usage. +--- -**Live-verified end-to-end**, not just unit-tested: live-patched into -`quickstart-pyegeria-web`'s installed pyegeria (`docker cp` + restart, -diff-confirmed byte-identical), then curled the actual production route — -`GET /api/tech-catalog/tech-types/PostgreSQL%20Relational%20Database/ -elements?display_name=PostgreSQL%20Relational%20Database` — which now -returns the previously-missing asset as the first result, alongside the -other 14 (15 total with templates included, vs. 14 without — consistent). +### ISSUE-100: `Create Data Lens`'s 9 fields were never mapped (same shape as ISSUE-71), and `Allow Duplicate Values` was either read under a phantom key or not read at all ---- +**Status:** fixed 2026-09-17 (Dr.Egeria — +`md_processing/md_processing_utils/common_md_utils.py`, +`md_processing/v2/data_designer.py`), verified live against +`qs-view-server` with throwaway elements. Two of the three follow-up items +from ISSUE-99's list, picked next by priority. -### ISSUE-81: `migrate_question_specs.py` links Perspective↔Question via the wrong relationship type (`AssignmentScope`, not `ScopedBy`) — every bootstrap-migrated perspective silently relies on a filename-inference fallback instead +**Data Lens:** `update_gov_body_for_type()` (`common_md_utils.py`, called by +every `GovernanceProcessor`-routed command) had no branch for `DataLens` — +all 9 of its compact-spec attributes (`Data Collection Start/End Time`, +`Max`/`Min Height`/`Latitude`/`Longitude`, `Scope Elements`) fell through to +the generic fallback at the end of the function, exactly the shape the +`GovernanceActionType`/`GovernanceActionProcessStep` branch immediately +above it already calls out as ISSUE-71. No dedicated Pydantic model exists +for `DataLensProperties` (it's a raw-dict body validated generically by +`_async_create_element_body_request`), so unlike ISSUE-98/99 there was no +second model-level gap to fix. Field names confirmed against both the +`GovernanceOfficer._async_create_data_lens`-style SDK method's own docstring +sample body and the Spec Editor API. **Note:** the SDK docstring also shows +`dataValidityStartTime`/`dataValidityEndTime`/`dataCoverageStartTime`/ +`dataCoverageEndTime` as real wire fields with no corresponding compact-spec +attribute at all — out of scope for this fix, a possible future addition if +wanted, not a bug. -**Status:** fixed 2026-08-28 — script corrected (see "Fix landed" below); -no data repair needed on the checked deployment (see "Checked the live -instance" below). Semantics settled against the canonical docs, not just -source-reading; see "Which relationship is actually correct" below. +**`Allow Duplicate Values` — two different bugs at two different sites, one +false lead:** +- `DataClassProcessor.apply_changes()` (`data_designer.py` ~line 504) read + `attributes.get('Allow Duplicates', ...)` — a key that does not exist + anywhere in the compact spec (real name: `Allow Duplicate Values`, on + `Data Class Base`) — always silently defaulted to `True`. Renamed the key. +- `set_data_field_body()` (`common_md_utils.py`) never referenced the + attribute at all, despite `Allow Duplicate Values` being directly in + `Create Data Field`'s `custom_attributes`. Added the missing line. +- **False lead, not fixed:** the identical `'Allow Duplicates'` string also + appears in `DataValueSpecificationProcessor` (`data_designer.py` line 54). + Confirmed harmless dead code — `Create Data Value Specification`'s own + bundle doesn't declare `Allow Duplicate Values` at all, so this lookup + always returns the default regardless of the key's spelling. Left as-is; + renaming it would be cosmetic, not a fix. -**A real bug**, found 2026-08-28 answering "is the folder-name-inferred -perspective fallback in `load_egeria_report_specs()` actually used -anywhere?" Short answer: yes — for every question migrated through the -Phase 1 bootstrap, it's the *only* thing that has ever worked. +**Verified live:** created a throwaway Data Class (`Allow Duplicate Values: +false`), Data Field (`Allow Duplicate Values: false`), and Data Lens (all 6 +geo fields set), fetched all three back, confirmed +`properties.allowsDuplicateValues` was `False` (not the `True` default) on +both, and all 6 Data Lens fields (`maxHeight`/`minHeight`/`maxLatitude`/ +`minLatitude`/`maxLongitude`/`minLongitude`) persisted with their exact +values — then deleted all three. Re-ran +`scripts/dr_egeria_attribute_consumption_audit.py` scoped to each affected +command: `Data Lens` now 0 findings; `Allow Duplicate Values` no longer +appears in either `Data Class`'s or `Data Field`'s remaining finding list. +`pytest tests/micro-tests -m unit`: no regressions. -**Root cause.** `commands/migrate_question_specs.py`'s `_ensure_scoped_by()` -(despite its name) calls `client.actor_manager.link_assignment_scope(...)` -with `properties: {"class": "AssignmentScopeProperties"}` — that creates -Egeria's **`AssignmentScope`** relationship (`e3fdafe3-...`, "Links a -profile, role, or project to the elements that they are responsible for -managing"). But `load_egeria_report_specs()` reads perspectives back with -`client.get_related_elements(question_guid, relationship_type="ScopedBy")` -— a **different** relationship type (`3845b5cc-...`, "Link between a scope -... and an element restricted the scope"). Confirmed both are genuinely -distinct types, not aliases of one another, via Egeria's own type registry -(`OpenMetadataType.java`: `ASSIGNMENT_SCOPE_RELATIONSHIP` vs -`SCOPED_BY_RELATIONSHIP`, different GUIDs, different Properties classes) -— they just happen to share a doc page (0120-Assignment-Scopes) because -they're related concepts. +**Still open from ISSUE-99's follow-up list** (not touched this round): +`Position`/`Minimum Cardinality`/`Maximum Cardinality` on the field↔structure +link, `Is Case Sensitive` (Data Class), the `Membership Type`/relationship- +property gaps across 6 Link commands, and the longer single-command tail — +see ISSUE-99 for the full list. -Since the two never match, `get_related_elements(..., "ScopedBy")` returns -empty for every question the bootstrap script touched, and -`load_egeria_report_specs()` falls through to its documented fallback: -inferring the perspective from the QuestionSpec folder's own -qualified-name suffix (`QuestionSpec::TypeDef::Developer` → `Developer`). -That fallback is a real, reachable code path with no bug in it — but see -"Checked the live instance" below before assuming it's actually load-bearing -for any specific deployment's data. +--- -**The Dr.Egeria-authored path is correct**, for contrast: -`Link Perspective to Question` (`md_processing/v2/actor_manager.py`, -`object_type == "Perspective to Question"`) calls -`classification_manager._async_add_scope_to_element` — the real `ScopedBy` -endpoint. Anything linked that way going forward works via the intended -relationship; only the one-time bulk migration is affected. +### ISSUE-99: `Purpose` — a required attribute on ~30 Collection-family `Create` commands — was never mapped into the request body at all -**Which relationship is actually correct** (not a majority vote between -the three call sites — checked against the canonical type definitions, -https://egeria-project.org/types/1/0120-Assignment-Scopes): - -- `AssignmentScope`: *"identifies individuals, teams, projects, or other - actors **assigned to manage** resources represented by a linked - element"* — carries `assignmentScope` values `Contributor`, - `Administrator`, `Leader`, `Owner`, `Observer`, `Sponsor`, `Other`. An - operational-responsibility relationship: who administers/owns a - resource. -- `ScopedBy`: *"connects an element to organizations, projects, teams, and - similar entities that **define the impact scope or applicability - boundaries** of that element."* +**Status:** fixed 2026-09-17 (Pyegeria/Dr.Egeria — +`md_processing/md_processing_utils/common_md_utils.py`, +`pyegeria/omvs/collection_manager.py`), verified live against +`qs-view-server` with a throwaway element. Found by +`scripts/dr_egeria_attribute_consumption_audit.py` (see below), a new +static-audit tool built in response to ISSUE-97/98 to find this whole bug +class proactively instead of one report at a time. -A Question isn't "managed" by a Perspective the way an asset is -administered by an owner — none of `AssignmentScope`'s predefined values -(`Contributor`/`Administrator`/`Owner`/...) make sense applied to a -Question. It's *relevant within* the boundary a Perspective defines, -which is exactly `ScopedBy`'s stated purpose. Direction confirms it too: -the (correct) Dr.Egeria command passes Perspective as the scope -(`scoped_by_guid`) and Question as the bounded element (`element_guid`), -matching "connects an element to ... entities that define the ... scope." -**`ScopedBy` is correct; `AssignmentScope` is the bug.** +**Root cause:** `set_element_prop_body()` — the base `Referenceable`-level +body builder every element type ultimately calls — never read `Purpose` at +all. `set_collection_manager_body()` (called for every `CollectionManagerProcessor`- +routed command: Collection, Data Spec, Data Dictionary, the whole Digital +Product family, Glossary, Report Type, Security Group/List/Role, Skill Set, +and ~20 more) only added Digital-Product-specific fields on top, never +`purpose`. **`Purpose` has `min_cardinality: 1` in the compact spec** — it's +presented to the user as required, they fill it in, and it was silently +discarded on every single one of these ~30 commands. Also missing from the +`CollectionProperties` Pydantic model (`pyegeria/omvs/collection_manager.py`) +— the same double-gap pattern as ISSUE-98's `Current Version`. Confirmed +`purpose` is a real wire field via `Egeria-api-collection-manager.http`'s +worked examples (appears in every `createCollection`-family sample body, +right after `authors`). -**Checked the live instance — no repair needed there.** Queried all 84 -Question elements (`get_related_elements(guid, relationship_type=X)` for -both `ScopedBy` and `AssignmentScope`, every one): **all 84 already carry -a real `ScopedBy` relationship; zero carry `AssignmentScope`.** So this -deployment's Question data was bootstrapped via the correct path (the -Dr.Egeria markdown/`run_all.sh` route through `Link Perspective to -Question`, which was never buggy) rather than by running -`migrate_question_specs.py` directly — the folder-name-inference fallback, -while real code with no bug in it, turns out not to be load-bearing for -*this* deployment's data after all. The original "sole source of every -perspective" claim above was inferred from reading the script, not -verified against live data — correcting that here rather than leaving an -overclaim standing. +**Scope note:** `Purpose` is also declared on `Create Solution Blueprint` +and `Create Information Supply Chain`, which use `set_element_prop_body()` +directly (not `set_collection_manager_body()`) and are **not** covered by +this fix — `purpose` does not appear anywhere in +`Egeria-api-solution-architect.http`'s ground truth, so unlike the Collection +case this looks like the same "spec cruft, no real DTO field" shape as +ISSUE-98's `Product Status`/`Product Type`, not a simple omission. Not +resolved here; flagged for the same maintainer call (map vs. remove) ISSUE-98 +got. -**Impact:** the code bug is still real — `migrate_question_specs.py` -would create wrong-typed data on any instance it's actually run against, -and any such data would be silently masked by the inference fallback -exactly as first described. `AssignmentScope` relationships would be real, -queryable, wrong-typed data in Egeria — anyone querying by `ScopedBy` -directly (bypassing the fallback), or any future code trusting -`AssignmentScope` to mean "this profile/role manages this question" (its -actual documented meaning), would get nonsense. Whether any other -deployment actually has this data is unknown — check before assuming -either way. +**Fix:** added `purpose: str | None = None` to `CollectionProperties` (all +Collection subtypes inherit it), and +`prop_bod["purpose"] = attributes.get('Purpose', {}).get('value', None)` to +`set_collection_manager_body()`, unconditionally (applies to every subtype +routed through it, not just Digital Product). Verified live: created a +throwaway Data Specification with `### Purpose` set, fetched it back, +confirmed `properties.purpose` persisted correctly, then deleted it. +`pytest tests/micro-tests -m unit`: no regressions. -**Fix landed:** `migrate_question_specs.py`'s `_ensure_scoped_by()` now -calls `classification_manager.add_scope_to_element` (matching the -Dr.Egeria command) instead of `actor_manager.link_assignment_scope`, so -the properties class actually sent is `ScopedByProperties`, not -`AssignmentScopeProperties`. +**Also from this audit run, not yet fixed — filed for follow-up, not +speculative:** +- `Create Data Lens`'s 9 DataLens-specific fields (`Data Collection Start/End + Time`, `Max`/`Min Height`/`Latitude`/`Longitude`, `Scope Elements`) fall + through to the fully generic governance body builder — same shape as + ISSUE-98, a subtype that never got its own properties branch + (`md_processing/v2/governance.py`, `GovernanceProcessor`). +- `DataClassProcessor`/`DataFieldProcessor` (`data_designer.py` lines + ~54/504) read `attributes.get('Allow Duplicates', ...)` — that key does + not exist anywhere in the compact spec (real name: `Allow Duplicate + Values`) — always silently defaults to `True`. +- `Is Case Sensitive` (Data Class) confirmed absent from the `DataClassProperties` + body. +- `Position`/`Minimum Cardinality`/`Maximum Cardinality`, declared on + `Create Data Field`, are never sent on the actual field↔structure link — + `LinkFieldToStructureProcessor` sends a completely empty + `MemberDataFieldProperties` body, even though the live relationship + carries exactly these 3 fields as server defaults. +- `Membership Type` is declared as a relationship property on 6 different + Link commands (`Add Member to Collection`, `Link Actor to Blueprint`, + `Link Blueprint Child`, `Link Information Supply Chain Child`, + `Link Solution Component to Blueprint`, `Link Agreement Terms and + Conditions`) across 3 different processors + (`CollectionLinkProcessor`/`SolutionLinkProcessor`/`GovernanceLinkProcessor`) + and never read in any of them; same pattern for `Dependency Description` + (`Link Product Dependency`), `Operation Name` (`Link Associated List`), + `Last Notification` (`Link Notification Subscriber`), `Assignment Type` + (`Link Assignment Scope`), `Expected Time Allocation Percent` (`Link + Person/Team Role Appointment`) — all confirmed by reading the relevant + processor's relationship-properties body, not just the audit heuristic. +- A longer tail of single-command gaps (`Expected Behavior`, `Example`, + `Due Time`/`Objective`/`Requested Start Time`, `Canonical Scope`/`Is + Canonical`/`Is Taxonomy`, `Subscription Level`, `Deployed Implementation + Type`/`Resource Name`, `Default Media Usage Other Id`/`Media Type Other + Id`, `Role Identifier`/`Role Type`, `Role List`, `In Data Value + Specification`/`Specializes Data Value Specification` (likely the same + ISSUE-97 singular/plural shape), `In Data Field`, `Grain Statement`/ + `Granularity Basis`/`Interval`) confirmed absent by targeted grep but not + read in full surrounding context — one confidence tier below the items + above; see the audit script's own report for the full list. +- `Estimated Volumetrics` (`Create Information Supply Chain`) is **not** a + new finding — it cross-references the existing ISSUE-64 entry's own + follow-up note (suspected misattributed bundle field, not a simple + "never read" bug). -**No data repair needed** on this deployment (see "Checked the live -instance" above) — nothing here to detach/recreate. If another deployment -did run the buggy script directly, the same live-check -(`get_related_elements(question_guid, relationship_type="AssignmentScope")` -across every Question) will show whether it needs the same repair; write -a small one-off script at that point rather than building one speculatively -now for data that may not exist anywhere. +**New tool, added this session:** `scripts/dr_egeria_attribute_consumption_audit.py` +— a static audit that checks, for every compact-spec command, whether its +processor actually reads each declared attribute (`UNCONSUMED`) and whether +`Reference Name`/`Reference Name List` attributes are read with the matching +`guid`/`guid_list` cardinality key (`CARDINALITY_MISMATCH`, the ISSUE-97 +shape). It's a heuristic, not a proof — see its docstring's "Known +limitations" section — and it has one confirmed class of false positive: +`SolutionLinkProcessor`'s generic `id1_key`/`id2_key` peer-link mechanism +resolves the two link-endpoint attributes dynamically from +`spec.get("custom_attributes")[0]`/`[1]`, never by literal string match, so +its ~21 `Reference Name`-style findings should be disregarded. A full run +also surfaced that `COMMAND_DEFINITIONS["Command Specifications"]` yields +some command names more than once (a few findings appear duplicated) — a +tool-side dedup bug, not a second instance of the underlying finding. --- -### ISSUE-80: `find_report_specs_by_perspective`/`find_report_specs_by_question` are implemented and tested but not exposed anywhere - -**Status:** fixed 2026-09-05 (Pyegeria — `pyegeria/core/mcp_adapter.py`, -`pyegeria/core/mcp_server.py`). Added `run_find_report_specs_by_perspective`/ -`run_find_report_specs_by_question` adapters (same thin-wrapper pattern as -the existing `run_find_report_specs`) and registered them as MCP tools -`find_report_specs_by_perspective`/`find_report_specs_by_question`, -alongside the existing `find_report_specs` tool. No change to the -underlying functions themselves — pure additive wiring, per the candidate -fix below. `hey_egeria` CLI exposure (the "and/or" half of the original -ask) left undone — MCP was the natural fit given the existing -`find_report_specs` tool precedent; a CLI subcommand is separate follow-up -work if wanted. Full `pytest tests/micro-tests/` green (exit 0); the -underlying functions already have their own functional-test coverage -(`tests/functional-tests/test_question_specs.py`, live-server only), not -re-run here. +### ISSUE-98: `Create Digital Product`'s `Product Status`/`Product Type` were spec cruft with no real DTO field, and `Current Version` was silently dropped -**Not a bug — a follow-up.** Found 2026-08-28 while reviewing the Question -Spec migration (`docs/design/report_spec_migration_design.md`, Phase 1 -complete since 2026-05-18/`6c946af`). Both functions -(`pyegeria/view/base_report_formats.py:3678`/`3715`) are real, working -query methods over the Egeria-sourced `question_spec` data that -`load_egeria_report_specs()` merges into the runtime registry — not -leftover pre-migration code, and not candidates for removal (see the -design doc discussion this follow-up came out of). +**Status:** fixed 2026-09-17 (Pyegeria/Dr.Egeria — compact spec via the +Spec Editor API, `md_processing/md_processing_utils/common_md_utils.py`, +`pyegeria/omvs/collection_manager.py`), verified via `refresh_specs` + +unit tests. Reported by an external maintainer/user agent. -**The gap:** neither is exported from `pyegeria/__init__.py`, registered -as an MCP tool (`pyegeria/core/mcp_server.py`), or wired into any -`hey_egeria`/`commands/` CLI surface. The only thing that currently -exercises them is their own functional test, -`tests/functional-tests/test_question_specs.py`. Someone wanting "which -report specs does the Data Steward perspective care about" or "which -report answers this question" today has to import -`pyegeria.view.base_report_formats` directly — there's no MCP tool or CLI -command for either. +**Reported symptom:** `Product Status`, `Product Type`, and `Current +Version` on `Create Digital Product` were not mapped into the request +body at all, so products carried their status in `Maturity` instead. -**Candidate fix, not started:** expose both as MCP tools (natural fit -alongside the existing `find_report_specs` tool the MCP server already -wires at startup via `load_egeria_report_specs`) and/or as a -`hey_egeria cat show` subcommand. Low risk — additive, no changes needed -to the functions themselves. +**Root cause, `Current Version`:** `set_collection_manager_body()`'s +Digital Product branch built `productName`/`maturity`/`serviceLife`/ +`introductionDate`/`withdrawalDate`/`nextVersionDate` but never read +`Current Version` — a plain omission. It's also a real wire field +(`currentVersion`, confirmed in `Egeria-api-product-manager.http`'s +`createDigitalProduct` example) that was additionally missing from the +`DigitalProductProperties` Pydantic model in +`pyegeria/omvs/collection_manager.py` — a double gap per this repo's +known "declared-but-not-on-the-model = silently dropped" pattern +(see ISSUE-62). A second, dead body-builder `set_product_body()` (never +called from anywhere) had the identical gap, left as-is since it's unused. ---- +**Root cause, `Product Status`/`Product Type`:** unlike `Current Version`, +neither corresponds to any real field on the Egeria +`DigitalProductProperties` DTO — confirmed against the `.http` ground +truth. Real lifecycle status is set via a separate +`updateDigitalProductStatus` operation using `contentStatus`/ +`deploymentStatus`, not exposed by any Dr.Egeria command for Create/Update +Digital Product. These two attributes were spec cruft, not simply +unmapped — user confirmed removal rather than inventing a mapping. -### ISSUE-78: engine-host participation trio is implemented on `main` but absent from the released 6.0.18.4 +**Fix:** added `"currentVersion": attributes.get('Current Version', {}).get('value', None)` +to `set_collection_manager_body`'s Digital Product branch, and +`current_version: str | None = None` to `DigitalProductProperties`. +Removed `Product Status`/`Product Type` from the `Digital Product Base` +bundle and their attribute definitions from +`commands_digital_products_compact.json` via the Spec Editor's REST API +(neither was shared with another family — `attribute_sharing` was empty +for both), then ran `refresh_specs --merge-reports` to regenerate +templates/help/report specs. `validate_compact_specs`: 0 errors. Proper +status support for Digital Products (via `contentStatus`/ +`deploymentStatus`) is not implemented — no command currently calls +`updateDigitalProductStatus` — and would need a follow-up if wanted. -> **RESOLVED by pyegeria 6.1.5** (verified 2026-08-27). All three methods ship in -> the wheel. Resource Explorer upgraded from 6.0.18.4 → 6.1.5 and its full suite -> (2613 tests), live Egeria smoke tests (20) and live alignment scan all pass, so -> the minor bump is clean for RE's usage. `claim_engine_action` was exercised -> against the live platform through the client: claiming an action already -> `IN_PROGRESS` under `EgeriaWatchdog` raises `PyegeriaUnauthorizedException` -> carrying `OMAG-GENERIC-HANDLERS-403-003` — a typed, catchable exception, which -> is what an engine-host loop needs to tell "another host got there first" from a -> real failure. Entry kept for history; move to Fixed / Resolved when convenient. +--- -**Not a bug — a release ask.** `claim_engine_action`, -`update_engine_action_status` and `get_active_claimed_engine_actions` exist -in this repo's `main` (commit `8362b1c`, "implement remaining -confirmed-missing methods") and are **not** in the pyegeria 6.0.18.4 that -Resource Explorer resolves from PyPI. Nothing needs writing; the methods -need to ship. +### ISSUE-97: `Create Data Structure`'s `In Data Specification` and `Create Data Field`'s `In Data Structure` were silently dropped — spec declares them singular, processor only read a `guid_list` -Measured 2026-08-26 against the installed package and this checkout: +**Status:** fixed 2026-09-17 (Dr.Egeria — `md_processing/v2/data_designer.py`), +verified live against `qs-view-server` with throwaway elements (created, +relationship confirmed via direct fetch, then deleted). Reported by an +external maintainer/user agent. -| method | 6.0.18.4 | `main` | -|---|---|---| -| `initiate_engine_action` | ✓ | ✓ | -| `get_active_engine_actions` | ✓ | ✓ | -| `get_engine_actions` / `_by_name` / `find_engine_actions` | ✓ | ✓ | -| `cancel_engine_action` | ✓ | ✓ | -| **`claim_engine_action`** | ✗ | ✓ | -| **`update_engine_action_status`** | ✗ | ✓ | -| **`get_active_claimed_engine_actions`** | ✗ | ✓ | +**Reported symptom:** both attributes are declared singular +(`"style": "Reference Name"`, `max_cardinality: 1`) in +`commands_data_designer.json`, but the link was never established — no +error, `--process` reported `SUCCESS`. Users worked around it with +explicit `Add Member to Collection` / `Link Field to Structure` blocks, +deliberately omitting the `In` attributes so a fix couldn't double-link. -**Why these three specifically.** They are the whole participation loop for a -non-Java engine host: find work claimed for my engine → claim it → report -progress. RE is being modelled as an additional engine host -(`packages/resource-explorer/docs/survey-model-and-engine-host-design.md` -§4), and without `claim` there is no arbitration between hosts, while -without `update_engine_action_status` a host could take work and never -report completion — worse than not participating at all. +**Root cause:** the parser (`md_processing/v2/processors.py`, ~line 787-840) +stores a singular `Reference Name` attribute's resolved guid under +`attr_data["guid"]`, and only populates `attr_data["guid_list"]` when the +raw input value is itself a list. `DataStructureProcessor.apply_changes` +(line 220) and `DataFieldProcessor.apply_changes` (line 326) both read +only `.get("guid_list", [])` for `In Data Specification`/`In Data +Structure`, which is never set for these singular attributes — the +existing `if isinstance(x, list) else [x]` wrapper around the *default* +`[]` never triggers, since a default empty list is already a list. The +correct sibling pattern already existed in the same file (`Data Class`, +`Specializes Data Class`, `Glossary Term` all correctly also check the +singular `guid` key) — just not applied here. -**The server side is confirmed working**, so this really is only a client -packaging gap. Tested 2026-08-26 by POSTing -`/engine-actions/{guid}/claim` directly for an action already `IN_PROGRESS` -under the running `EgeriaWatchdog`: +**Also found and fixed as the same bug:** `In Data Dictionary` (also a +singular `Reference Name` attribute per the compact spec) had the +identical `guid_list`-only read at 3 call sites — `DataStructureProcessor`, +`DataFieldProcessor`, and `DataClassProcessor`. -``` -OMAG-GENERIC-HANDLERS-403-003 Engine Host OMAG Server with a userId of -erinoverview is not allowed claim the engine action ... because it is -already claimed -systemAction: The system cannot claim an engine action because another -Engine Host OMAG Server has got there first. -``` +**Fix:** all 4 call sites (`In Data Specification`, `In Data Structure` x2, +`In Data Dictionary` x3) now fall back to the singular `guid` key when +`guid_list` is absent, matching the existing `Glossary Term`/`Specializes +Data Class` pattern. Verified end-to-end against a live server: created a +throwaway `Data Specification` → `Data Structure` (`In Data Specification`) +→ `Data Field` (`In Data Structure`), fetched both back and confirmed the +real `CollectionMembership` and `MemberDataField` relationships were +created, then deleted all three. `pytest tests/micro-tests -m unit`: all +pass, no regressions. -First-claim-wins, server-enforced, and the refused claim left the other -host's action untouched. That is also what makes a polling client safe — -every host may see an action and only one can take it — so **no Kafka -consumer is being asked for**. Delivery is a latency optimisation; -correctness rests on `claim`. +--- -**Blocks:** RE engine-host participation (design note §4.1, §5 step 8). -Nothing else in RE is waiting on it. +### ISSUE-96: get_guid_for_name (and every other sync wrapper using asyncio.get_event_loop().run_until_complete(...)) breaks when a client instance is reused across threads — surfaces as a misleading CLIENT_ERROR_400 "unable to connect" ---- +Layer: pyegeria core (_base_server_client.py, _base_platform_client.py, every OMVS class's sync wrappers) · Status: fixed (2026-09-13) · Found: 2026-09-12 (Resource Explorer, SurveyDefinitionReader._lookup_question_guid) -### ISSUE-107: Dr.Egeria's create pre-check issues up to 4 redundant `guid-by-unique-name` lookups per command, multiplying an already-slow ambiguous-match server response +A single client instance (e.g. ClassificationExplorer) constructed once +and reused from multiple threads — a natural pattern for any caller with a +thread pool bridging sync pyegeria calls into async code — fails +non-deterministically once its httpx.AsyncClient (self.session, created +once in __init__, _base_platform_client.py:128) is driven by more than +one event loop over its lifetime. Every sync wrapper (get_guid_for_name, +create_egeria_bearer_token, and every other +`loop = asyncio.get_event_loop(); loop.run_until_complete(...)` call — +~29 call sites in _server_client.py alone) hands the SAME shared client's +connection pool to whichever thread happens to call it, and +pyegeria/__init__.py's import-time nest_asyncio.apply() +(via pyegeria.view.mermaid_utilities) means every calling thread +silently gets its OWN fresh event loop rather than raising — so the +mismatch is invisible until the underlying httpcore/anyio connection's +internal asyncio.locks.Event/lock, bound to loop A, is awaited from loop +B and raises `RuntimeError: <...> is bound to a different event loop`. -**Layer:** Pyegeria · **Status:** fixed 2026-09-20 · -**Found:** 2026-09-20, reported by the user from a Dr.Egeria bulk load: a -subject-area file created ~25 folders sharing the same three Display Names -("Prime Words", "Modifiers", "Class Words" across parent scopes), and files -using those names went from ~6 seconds to 2+ minutes apiece — 35 of a -56-minute load, by the user's measurement. Postgres itself was not the -cause (27ms queries); ambiguous-name resolution was. +How it shows up. _async_make_request (_base_server_client.py:621) +catches this (and anything else) broadly and re-raises as +PyegeriaUnknownException / CLIENT_ERROR_400, "The client is unable to +connect to the Egeria platform" — the real RuntimeError and its "different +event loop" text never reach the caller. A direct curl to the same +platform at the same moment succeeds in milliseconds, ruling out actual +connectivity. -**Root cause, confirmed by reading the code (not the client-side "validates -every match" loop the report's phrasing suggested — no such loop exists in -pyegeria):** -- `md_processing/v2/processors.py`'s `resolve_element_guid()` calls the - SDK's `__async_get_guid__` **twice** per lookup (once scoped by - `tech_type`, once broader if the first comes back empty/ambiguous) — - lines 1413 and 1442. -- `execute()`'s step 4a (line 647) runs an **independent, duplicate** - display-name check on top of the one already done inside `fetch_as_is()` - (line 638) — using the raw Display Name rather than the already-derived - qualified name. -- Net: a single `Create` of a not-yet-cached, ambiguously-named element can - trigger up to 4 HTTP round-trips to the same `guid-by-unique-name` - endpoint. `__async_get_guid__` (`pyegeria/core/_server_client.py:224-321`) - POSTs once per call and returns whatever the server reports; the - ambiguous-match resolution itself (and its ~1s-per-candidate cost) is - server-side, not a Python loop — confirmed by reading the full call path. -- **No fast-path existed for an explicit `Qualified Name`.** `fetch_as_is()` - already fast-paths an explicit `### GUID` (ISSUE-59's fix, lines - 1519-1539) but had no equivalent for an unambiguous, user-supplied - qualified name — every Create paid the ambiguous-lookup cost even when - the file already fully disambiguated the target. +How to trigger: +```python +from pyegeria.omvs.classification_explorer import ClassificationExplorer +import threading -**Fixed (this pass):** `execute()` (`md_processing/v2/processors.py`) now -captures whether `Qualified Name` was explicitly authored in the markdown -*before* step 1a's auto-derivation can inject a derived one under the same -key (making the two indistinguishable by the time step 4a runs), and skips -step 4a's duplicate-Display-Name lookup entirely when it was — an explicit -qualified name already unambiguously identifies the target, and -`fetch_as_is()` already looked that exact name up and found nothing, which -is authoritative since no auto-derivation is involved for ISSUE-59's -collision case to apply to. Covered by -`tests/micro-tests/test_step4a_skip_with_explicit_qn.py`: one test confirms -step 4a still runs (≥2 `resolve_element_guid` calls) when the qualified -name is auto-derived, preserving ISSUE-59's protection; a second confirms -exactly one call happens when it's explicit. Full `tests/micro-tests/` -suite re-verified green after the change. +client = ClassificationExplorer(view_server, platform_url, user, pwd) +client.create_egeria_bearer_token(user, pwd) # binds session internals to THIS thread's loop -**Fixed (second pass, same day):** `resolve_element_guid()`'s two internal -passes (Pass 1 WITH the `tech_type` constraint, Pass 2 WITHOUT) each now -check a per-batch query cache in `self.context` (`_resolve_guid_query_cache`, -keyed by `(name_or_guid, effective_type)`) before hitting the server, and -populate it afterward — but **only** for a "not found" or "ambiguous" -outcome, deliberately never for a genuine single-match found `guid`: caching -a found match risked going stale within a batch if a later sibling command -creates an element sharing that exact Display Name (step 4a searches by raw -Display Name, which the QN-keyed local element dictionary checked earlier -in the same method can't already cover); a found match is also already the -cheap server-side path, so there was nothing to gain by caching it. This is -the actual fix for the reported 25x-repeated-Display-Name scenario: the -first of the 25 "Prime Words" Creates pays the full ambiguous/not-found -lookup cost (both passes, since neither cache key exists yet); the other 24 -each hit the cache and make zero calls to the server for that name. Ambiguous -results are replayed per-instance too (each sibling's own `parsed_output` -still gets the "Multiple elements found" error appended, even though the -answer came from cache, since that side effect is per-command). Covered by -`tests/micro-tests/test_resolve_guid_query_cache.py`: one test drives 5 -sibling Creates through a client that reports the shared probe name as -ambiguous and asserts the client is only ever called once across all 5; a -second does the same for a "not found" name and asserts exactly 2 calls -total (Pass 1 + Pass 2, both only on the first sibling). Full -`tests/micro-tests/` suite re-verified green. +def call(): + client.get_guid_for_name("some display name", property_name=["displayName"], type_name="GlossaryTerm") -**Other similar patterns checked, none found:** two structurally similar -helpers (`_server_client.py:734-772`, `classification_explorer.py:3428- -3462`) and the legacy v1 path (`extraction_utils.py:416-502`) all do the -same fail-fast "raise if `len(matches) > 1`" with no client-side loop — no -matching perf anti-pattern found elsewhere in `pyegeria/core`, -`pyegeria/omvs`, or `md_processing`. +t = threading.Thread(target=call) +t.start(); t.join() # raises PyegeriaUnknownException / CLIENT_ERROR_400 +``` +Reproduced deterministically (100% failure across 52 distinct names, 2 +runs) when the client is called from a shared thread pool or a fresh +thread per call; 0% failure when the client is always driven by the same +single thread/loop throughout its life, or when a fresh client is +constructed inside the very thread that will use it. ---- +Note: pyegeria/core/mcp_server.py:181-184 (this same 6.1.10 release) +documents this exact failure and works around it by constructing a fresh, +per-call EgeriaTech client inside the handler rather than reusing a +module-level one — confirming this is a known, previously-hit instance of +the same defect, not unique to this caller. + +Candidate fix directions (not evaluated for feasibility here): (1) +make sync wrappers detect a loop change since the client's session was +last used and rebuild self.session when it happens; (2) document +plainly that a client instance is not thread-safe / not loop-portable and +must be confined to one thread or reconstructed per thread; (3) provide a +supported "fresh session per call" mode, generalizing the workaround +already used internally in mcp_server.py. +Suggested fix (pyegeria): either make each sync wrapper use a per-thread session/loop (e.g. `threading.local()` holding the httpx.AsyncClient), or document that a client instance is single-thread-affine and raise a clear error — rather than the broad `except` in `_async_make_request` relabelling `RuntimeError: ... bound to a different event loop` as CLIENT_ERROR_400 'unable to connect', which is what cost the diagnosis. Caller-side mitigation used in Resource Explorer (dwolfson/trellis, `re/question-guid-client-per-thread`): one client per thread, constructed inside the thread that uses it — 0/104 failures after, 52/52 before. -### ISSUE-106: `InitialClassifications`'s `model_serializer` popped the wrong dict key (`other_props` instead of the aliased `otherProps`) — every classification property beyond `class` was silently dropped on every `initialClassifications` call across the whole SDK +--- -**Status:** fixed 2026-09-18 (pyegeria — `pyegeria/models/models.py`), -verified with an isolated Pydantic round-trip test and live against -`qs-view-server`. Found while live-verifying the ISSUE-105 Glossary -classification fix below — `Taxonomy`/`CanonicalVocabulary` classifications -were being applied correctly (conditionally, per the ISSUE-105 fix) but -`organizingPrinciple`/`scope` never persisted despite being read and set -correctly on the Python side. +### ISSUE-94: `SchemaMaker._async_delete_schema_type`/`_async_delete_schema_attribute` sent `MetadataSourceRequestBody` — the live server's schema-maker delete endpoints reject it outright -**Root cause:** `InitialClassifications.capture_other_props()` (a -`model_validator`) correctly captures any extra properties beyond `class` -into a Python-side `other_props` field. Its paired -`serialize_model()` (`model_serializer(mode="wrap")`) is supposed to -flatten `other_props` back onto the output dict before sending — but it -does `result.pop("other_props", None)`, the Python field name. Every real -call site in this codebase dumps with `by_alias=True` (confirmed: -`_async_new_relationship_request`, `_async_create_element_body_request`, -etc. all do), and `PyegeriaModel`'s `alias_generator=to_camel_case` means -the serializer's own `handler(self)` call has *already* aliased the field -to `otherProps` by the time `serialize_model` runs — so the `pop("other_props")` -never matched anything, and the extra properties stayed nested under a -stray `otherProps` key the real Egeria DTO doesn't declare, silently -dropped server-side (no error — same "declared but not on model" silent-drop -shape as ISSUE-62, but at the serializer layer instead of the field-declaration -layer). +**Status:** fixed 2026-09-11 (Pyegeria — `pyegeria/omvs/schema_maker.py`), +verified live against `qs-view-server`. -**Scope: every caller of `initialClassifications` with more than a bare -`{"class": ...}`, across the whole SDK, not just Glossary.** Confirmed by -isolated test: -```python -from pyegeria.models.models import NewElementRequestBody -body = {"class": "NewElementRequestBody", "isOwnAnchor": True, - "properties": {"class": "GlossaryProperties", "displayName": "x", "qualifiedName": "x"}, - "initialClassifications": { - "Taxonomy": {"class": "TaxonomyProperties", "organizingPrinciple": "X"}, - }} -NewElementRequestBody.model_validate(body).model_dump_json(by_alias=True) -# before fix: {"Taxonomy": {"class": "TaxonomyProperties", "otherProps": {"organizingPrinciple": "X"}}} -# after fix: {"Taxonomy": {"class": "TaxonomyProperties", "organizingPrinciple": "X"}} +Found while live-verifying the new Asset Maker / Schema Maker Dr.Egeria +command families (#354). Both methods built `{"class": +"MetadataSourceRequestBody", ...}` and posted it via +`_async_metadata_source_body_request` — matching the ground-truth +`Egeria-api-schema-maker.http` file's documented body for these two +`POST .../schema-types/{guid}/delete` and +`.../schema-attributes/{guid}/delete` endpoints. Both nonetheless failed +against a live server with: + +``` +InvalidTypeIdException: Could not resolve type id 'MetadataSourceRequestBody' +as a subtype of DeleteElementRequestBody: known type ids = +[DeleteClassificationRequestBody, DeleteElementRequestBody, DeleteRelationshipRequestBody] ``` -**Fix:** `serialize_model` now pops either key — -`result.pop("otherProps", None) or result.pop("other_props", None)` — -covering both the aliased (real-world) and unaliased (defensive) cases. +i.e. the server's real Jackson type registry for these endpoints never +recognized `MetadataSourceRequestBody` as a valid subtype — the `.http` +ground truth was out of date against this server (confirmed by the user, +who requested the fix directly rather than filing an Egeria-side issue). -**Verified live:** re-ran the Glossary classification test from ISSUE-105 -after this fix — `organizingPrinciple`/`scope` both persisted correctly -where they previously silently vanished with the model bug still in place. -`pytest tests/micro-tests -m unit`: no regressions from this shared-model -change. - -**Found, not fixed — adjacent, separate bug, same class as ISSUE-62:** -while cleaning up throwaway test elements for this fix, `_async_delete_collection` -(and likely other `_async_delete_*` wrappers with the same shape) silently -ignores its own `cascade` parameter whenever it's called with a body dict -already present — `validate_delete_element_request()`'s `cascade_delete` -argument is only honored in the `else` branch (body not provided at all); -the `isinstance(body, dict)` branch validates the dict as-is and never -merges `cascade_delete` into it. Several `_async_delete_*` methods -construct a dict body themselves before calling the shared -`_async_delete_element_request` helper (e.g. `CollectionManager._async_delete_collection` -does `body = {"class": "DeleteElementRequestBody"}` when its own `body` arg -is `None`), so `cascade=True` passed to *those* wrappers is silently -ignored too — confirmed live: `_async_delete_collection(guid, cascade=True)` -failed with "not permitted, still has a dependent element" even with -`cascade=True` explicitly passed. Workaround used for cleanup: pass an -explicit `body={"class": "DeleteElementRequestBody", "cascadeDelete": True}` -dict directly. Not fixed here — flagging for a dedicated pass across every -`_async_delete_*` wrapper with this shape, same audit ISSUE-62 should have -covered but evidently didn't catch this specific code path. +**Fix:** switched both methods to `DeleteElementRequestBody` via the +standard `_async_delete_element_request` helper (`cascade_delete` param +added to both, matching every other `_async_delete_*` in this codebase); +corrected `Egeria-api-schema-maker.http`'s two delete sections to match. +Verified live: created a throwaway `SchemaType` and `SchemaAttribute`, +deleted each with the default (no-body) call — both succeeded. --- -### ISSUE-105: "Cluster B" — 8 real fixes (casing/phantom-key bugs, missing fields, an unwired relationship, a scope-narrower-than-declared field) plus 6 compact-spec attributes removed as confirmed cruft, plus an active Glossary-classification bug found along the way +### ISSUE-93: `declassify_metadata_element` raises `AttributeError` when `body` is omitted, though the parameter is Optional — and the classification is silently left in place -**Status:** fixed 2026-09-18 (Dr.Egeria/pyegeria — `md_processing/v2/glossary.py`, -`md_processing/v2/feedback.py`, `md_processing/v2/data_designer.py`, -`md_processing/v2/embedded_process.py`, `md_processing/v2/report.py`, -`md_processing/v2/saved_query.py`, `md_processing/v2/solution_architect.py`, -`md_processing/v2/collection_manager_processor.py`, -`md_processing/v2/project.py`, `pyegeria/omvs/my_profile.py`, plus 5 -compact-spec family files), verified live with a combined throwaway-element -test. Continuing the ISSUE-99/100/101/103/104 follow-up list — "Cluster B", -the lower-confidence remainder of the original attribute-consumption audit -(confirmed absent only by grep, not read in full context). Every item was -independently re-confirmed real-vs-cruft via the live type system before -touching anything, per the discipline ISSUE-103 established. +**Layer:** Pyegeria · **Status:** fixed (2026-09-13) · **Found:** 2026-09-08 (Resource Explorer, swapping a Project's kind classification during investigation reclassification). -**Real fixes:** -- **`Example`** (`Create Glossary Term`, `Create Question`) — real field - `examples` (plural) on `GlossaryTermProperties`; `TermProcessor` read the - phantom key `'Examples'` (plural — spec attribute is singular `Example`); - `QuestionProcessor` never referenced it at all. Both fixed. -- **`Deployed Implementation Type`/`Resource Name`** (`Create Embedded - Process`, `Create Report`, `Create Saved Query`) — real `AssetProperties` - fields (confirmed `.http`), inherited via each command's `Asset` bundle. - All three processors only built their own type-specific extras, never - these two shared-but-inherited fields. Added to all three. -- **`Default Media Usage Other Id`/`Media Type Other Id`** (`Create Related - Media`, plus a bonus find in the adjacent `Cited Document` block) — - case-mismatched phantom keys (`...Other ID` vs spec's `...Other Id`), 3 - sites fixed. -- **`Grain Statement`/`Granularity Basis`/`Interval`** (`Create Data - Grain`) — real `DataGrain` own-attributes, confirmed via the live type - system; `DataGrainProcessor` only called the fully generic - `set_element_prop_body()`. Added all 3. **Also caught and fixed while - verifying live:** `Interval`'s compact-spec `style` was `Simple Float`, - but the real server-side type is `long` — sending `1.0` failed with - `InvalidFormatException`. Corrected style to `Simple Int`. -- **`Role List`** (`Create Solution Blueprint`) — real, unwired - relationship, same shape as the already-fixed `In Data Structure` - (ISSUE-97). `BlueprintProcessor` had no sync of any kind for it. Added a - `_sync_role_list` method mirroring the existing `_sync_components` - pattern (plain `CollectionMembership`, same relationship the standalone - `Link Actor to Blueprint` command already uses). -- **`Objective`** (`Create Meeting` only, per explicit decision — NOT - ToDo/Review/Note, despite the compact spec's shared description implying - otherwise) — real `MeetingProperties` field, confirmed live; absent from - `ToDo`/`Review`/`Action`. `ProjectProcessor`'s Meeting branch calls - `MyProfile._async_create_meeting()`, which had no `objective` parameter - at all — added one (both to the SDK method and the processor call site). +`MetadataExpert.declassify_metadata_element(metadata_element_guid, classification_name, body=None)` +declares `body` as `Optional[dict | MetadataSourceRequestBody] = None`, but the +implementation calls `.model_dump()` on it unconditionally. Omitting it raises: -**Active bug, not just a missing field — `Create Glossary`'s -`Is Canonical`/`Is Taxonomy`/`Canonical Scope`/`Organizing Principle`:** -`CollectionManagerProcessor` was unconditionally applying **both** -`Taxonomy` and `CanonicalVocabulary` classifications to every single -Glossary ever created, regardless of what the user set (or didn't set) — -already-running wrong behavior in production, not a dormant gap. Found: -`md_processing/v2/glossary.py`'s `GlossaryProcessor` already had the -*correct* conditional logic written — but `GlossaryProcessor` is **never -registered** in `setup_dispatcher()` (`Glossary` auto-routes via -`COLLECTION_SUBTYPES` to `CollectionManagerProcessor` instead) — fully dead -code, confirmed by grepping the entire dispatcher registration. Fixed in -the code path that actually runs: classifications now applied only when -their respective `Is Canonical`/`Is Taxonomy` flag is true (both default -`False` per the compact spec, matched here), with `Canonical Scope`→`scope` -and `Organizing Principle`→`organizingPrinciple` wired in. **This fix -surfaced ISSUE-106** (below) — the classification-conditional logic was -right, but the properties still didn't persist until that separate, -deeper pyegeria bug was found and fixed too. +``` +AttributeError: 'NoneType' object has no attribute 'model_dump' +``` -**Confirmed cruft, removed from the compact spec (user decision) — 6 -attributes, cross-file (the `compact-attr-global-namespace` global-namespace -pattern applied to the *removal* side too: some of these existed -identically in multiple family files and needed cleanup in each):** -- `Expected Behavior` (`Create Activity/Blog/Journal Entry`) — zero own - attributes on `Notification`/`JournalEntry`/`ActivityEntry`/`BlogEntry` - in the live type system. -- `Due Time`/`Requested Start Time` (`Meeting`/`ToDo`/`Review`/`Note`) — the - `.http` "confirmation" that looked real turned out to be a copy-paste - artifact (the same oversized field block repeated under 5 unrelated - class headers, including `NotificationProperties`, which the live type - system separately confirms has zero own fields). `MyProfile`'s own SDK - methods have no parameter or override mechanism for either, confirming - no real path ever existed. -- `Role Identifier`/`Role Type` (`Create Solution Role`) — zero own - attributes on `SolutionActorRole` or its super `ActorRole`; `Role - Type`'s own description referenced an unrelated real type - (`GovernanceRole`), reading as confused/stale spec authoring. -- `Subscription Level` (`Create Digital Subscription`) — redundant - duplicate of the already-correctly-wired `Support Level` - (`supportLevel` on `DigitalSubscriptionProperties`, confirmed `.http`); - no `subscriptionLevel` field exists anywhere in the ground truth. +**Why this is worse than an ordinary signature bug:** the exception arrives from +deep inside the client, so a caller that wraps the call defensively — which is +reasonable, since a classification that is already absent should not be fatal — +swallows it and *believes the classification was removed*. It was not. Measured +live: -**Bundle/attribute removal mechanics:** `Person Action Base` (shared -byte-for-byte across `commands_feedback_compact`, `commands_project_compact`, -`commands_actor_manager_compact`) needed `Due Time`/`Requested Start Time` -stripped from all 3 copies; doing so exposed that 2 of the 3 files were -each already missing a *different* pre-existing shared attribute locally -(`Priority` absent from feedback/actor_manager, `Situation` absent from -project/actor_manager) despite their bundles referencing them successfully -via cross-file resolution at runtime — copied the canonical definitions -into the files missing them (matching the `compact-attr-global-namespace` -"always copy, never reinvent" rule) before the bundle `PUT` would validate. -Not something introduced by this session; a latent inconsistency this -cleanup pass happened to surface and correct as a side effect. +```python +kinds before: ['Task'] +declassify_metadata_element(guid, "Task") # raises AttributeError +kinds after: ['Task'] # unchanged +``` -**Verified live:** one combined throwaway-element test file (Glossary + -Term + Question + Embedded Process + Related Media + Data Grain + Solution -Role + Solution Blueprint with `Role List` + Meeting with `Objective`). -First `--process` run: 1 failure (`Interval`'s float-vs-long style bug, -found and fixed as above), everything else succeeded. Re-ran clean after -the style fix. Fetched every created element back individually and -confirmed: `examples` on both Term and Question; `Taxonomy`/ -`CanonicalVocabulary` classifications present conditionally with -`organizingPrinciple`/`scope` populated (after the ISSUE-106 fix); -`grainStatement`/`granularityBasis`/`interval` on the Data Grain; -`objective` on the Meeting; the `SolutionActorRole`↔`SolutionBlueprint` -membership from `Role List` (confirmed indirectly — a delete of the -blueprint correctly failed with "still has a dependent SolutionActorRole -element" until the role was removed first). All throwaway elements -deleted afterward. `scripts/dr_egeria_attribute_consumption_audit.py` -re-run scoped to every touched command: all 0 findings. -`pytest tests/micro-tests -m unit`: no regressions. +In our case the element then carried **two** kind classifications at once +(`['Task', 'PersonalProject']`) — the new one added, the old one never removed — +and a check that only asked "is the new classification present?" reported +success. The wrong-but-plausible state was invisible until the element was read +back for *both* names. -**Found, not fixed — a second unrelated dead-code discovery, same shape as -ISSUE-104's "Associated Group":** `Classify Glossary as Canonical`/ -`Classify Glossary as Taxonomy` are declared commands with no registered -processor (surfaced by the audit script's own "unrouted" reporting) — may -be an intended alternative, post-creation classification mechanism that -was never wired up, or may be superseded by this fix's create-time -handling. Not investigated further; flagging for whoever next touches the -Glossary family. +**Working around it** — pass an explicit body: ---- +```python +me.declassify_metadata_element(guid, "Task", {"class": "MetadataSourceRequestBody"}) +# kinds after: [] +``` -### ISSUE-104: "Cluster A" — 4 relationship-property gaps, a phantom-key relationship-endpoint bug, a genuinely-missing hierarchy relationship, and one Link command with no implementation at all +**Candidate fix:** default the body when `None`, matching the Optional +declaration — the same shape the workaround passes: -**Status:** fixed 2026-09-18 (Dr.Egeria/pyegeria — `md_processing/md_processing_utils/common_md_utils.py`, -`md_processing/v2/data_designer.py`, `md_processing/v2/governance.py`, -`pyegeria/models/models.py`), verified live against `qs-view-server` with a -single combined throwaway-element test covering all 7 items. Continuing the -ISSUE-99/100/101/103 follow-up list, picked as "Cluster A" — the -higher-confidence half of what remained, per a live type-system check on -each item before touching code (same discipline ISSUE-103 established). +```python +body = body or {"class": "MetadataSourceRequestBody"} +``` -**1-3. Three relationship-property gaps, one shared fix.** `Zone -Membership`/`Last Notification`/`Activity Status` (`Link Notification -Subscriber`), `Assignment Type` (`Link Assignment Scope`), and `Expected -Time Allocation Percent` (`Link Person/Team Role Appointment`) were all -declared real by the live type system (`NotificationSubscriber`/ -`AssignmentScope`/`PersonRoleAppointment`/`TeamRoleAppointment` relationship -defs) but never read — all three routes went through the fully generic -`set_rel_prop_body()` in `common_md_utils.py`, which only builds -`description`/`label`/`typeName`/`effectiveFrom`/`effectiveTo`/ -`extendedProperties`. Fixed with one shared edit: `set_rel_prop_body` now -branches on its own computed `prop_name` (the real Egeria type name) to add -each type's own fields, mirroring the `update_gov_body_for_type`/ -`set_collection_manager_body` pattern already used for element bodies. -Also added the missing `last_notification` field to pyegeria's existing -`NotificationSubscriberProperties` Pydantic model (not a live double-gap -today, since this write path uses raw dicts, but worth fixing for -consistency/future use). +**Worth checking while in there:** `classify_metadata_element` and +`reclassify_metadata_element` sit beside it with the same `Optional` body +declaration. We pass a body to both so we have not hit them, but if they share +the unconditional `.model_dump()` they have the same defect — and the +declassify case shows the failure mode is silent at the caller, not loud. -**4. `In Data Field` — same shape as the already-fixed `In Data Structure`, -plus a second bug found while fixing it.** Real (`NestedDataField` -relationship, confirmed via an existing code comment). Also found: -`DataFieldProcessor._sync_all_rels()` read a key, `'Parent Data Field'`, -that doesn't exist anywhere in the compact spec at all — a phantom-key bug -(ISSUE-100 `Allow Duplicates` shape) in code that ISSUE-97/101's fixes to -this exact method had already touched twice this week and missed. Renamed -to `'In Data Field'` with the singular-`guid` fallback (`max_cardinality: 1`, -same pattern as the sibling fixes). +**Where seen:** `trellis/packages/resource-explorer/resource_explorer/surveyors/ +investigation_reclassifier.py::_move_kind_classification`, which now passes the +body explicitly and additionally verifies that the OLD classification is gone +rather than only that the new one is present. -**5-6. `In Data Value Specification`/`Specializes Data Value Specification` -— genuinely missing, user confirmed both are the same relationship.** Both -attributes read as near-synonyms in the compact spec; user decided to treat -them as aliases of the one real `DataValueHierarchy` relationship (the same -one `Link Data Value Composition` already uses via -`_async_link_specialized_data_value_specification`). Neither was wired at -all on `Create Data Class`, `Create Data Grain`, or `Create Data Value -Specification` — none of the three processors synced any relationship for -either attribute. Added a `_sync_value_spec_parent`-style sync to all three -(as a new 5th section in `DataClassProcessor`'s existing `_sync_all_rels`; -as new standalone helper methods on `DataGrainProcessor` and -`DataValueSpecificationProcessor`, which previously had *no* relationship -sync of any kind). +--- -**7. `Link Associated List` — not a field-mapping bug, a completely -unimplemented command.** `governance.py`'s `endpoint_map` had no entry for -`"Associated List"` at all — every invocation raised -`NotImplementedError`. No dedicated SDK method exists for -`AssociatedSecurityList` (`SecurityAccessControl↔SecurityList`, confirmed -real via the live type system), and no `.http` worked example exists -either. Found and reused the actually-correct generic mechanism instead of -writing a new bespoke SDK method: `MetadataExpert._async_create_related_elements`/ -`_async_detach_related_elements_in_store` (wrapping the generic -`createRelatedElementsInStore`/`.../detach-all` endpoints, already proven in -production by `AsyncBaseCommandProcessor._sync_parent_relationship` for the -same "no bespoke wrapper exists" reason). **Explicitly did not** route this -through the neighboring `_async_link_peer_definitions` call used for -`Associated Group`/`Regulation Certification Type` — that method's own -docstring restricts it to `GovernanceDriverLink`/`GovernancePolicyLink`/ -`GovernanceControlLink`, and `SecurityList` is not a governance-definition -peer of `SecurityAccessControl`. One body-shape gotcha caught before -shipping: the generic detach call validates against -`OpenMetadataDeleteRequestBody` (strict `class` literal), not the -`DeleteRelationshipRequestBody` shape every other branch in this method -builds — had to construct that request body separately rather than reusing -the shared one. +### ISSUE-92: `Project Type`'s description in `commands_project_compact.json` lists only 4 of its 6 `valid_values` — omits `Project` and `Experiment`, and the stale text is baked into 24 generated files -**Found, not fixed — flagged as pre-existing dead code, adjacent to item -7:** `governance.py`'s `"Associated Group"` handling (both the `endpoint_map` -entry and the `elif object_type in {"Associated Group", ...}` branches, in -both the Create and Detach paths) references a relationship type, -`AssociatedSecurityGroup`, that does not exist anywhere in the live type -system, and no compact-spec command anywhere actually has -`object_type == "Associated Group"` — fully unreachable, apparently a stale -leftover. Left as-is (out of scope for this fix); worth a follow-up cleanup -pass. +**Layer:** Pyegeria · **Status:** fixed (2026-09-13) · **Found:** 2026-09-07 (Resource +Explorer, designing investigation → Egeria Project classification mapping). -**Verified live:** one combined throwaway-element test file exercising all -7 fixes in a single `--process` run (Notification Type + subscriber -Collection + Security Access Control + Security List + Person + Person Role -+ parent/child Data Fields + parent Data Value Specification + child Data -Class), 0 `FAILURE` rows. Fetched every created element back individually -and confirmed: `activityStatus`/`zoneMembership`/`lastNotification` on the -`NotificationSubscriber` relationship; `expectedTimeAllocationPercent` on -`PersonRoleAppointment`; `assignmentType` on `AssignmentScope`; the real -`NestedDataField` relationship linking child→parent Data Field; the real -`DataValueHierarchy` relationship linking Data Class→Data Value -Specification; and the real `AssociatedSecurityList` relationship linking -Security Access Control→Security List with `operationName` set. All -throwaway elements deleted afterward. -`scripts/dr_egeria_attribute_consumption_audit.py` re-run scoped to every -touched command: all 0 findings (Data Grain's separate, untouched -`Grain Statement`/`Granularity Basis`/`Interval` gap still correctly -appears, confirming the audit still works). `pytest tests/micro-tests -m -unit`: no regressions. +`md_processing/data/compact_commands/commands_project_compact.json`, the +`Project Type` attribute (`variable_name: project_type`): -**Still open from the ISSUE-99/100/101 list:** `Operation Name` is now -fixed (covered above as item 7's `Associated List`). Still open: the -longer single-command tail (`Expected Behavior`, `Example`, `Due -Time`/`Objective`/`Requested Start Time`, `Canonical Scope`/`Is -Canonical`/`Is Taxonomy`, `Subscription Level`, `Deployed Implementation -Type`/`Resource Name`, `Default Media Usage Other Id`/`Media Type Other -Id`, `Role Identifier`/`Role Type`, `Role List`, `Grain -Statement`/`Granularity Basis`/`Interval`) — "Cluster B" in earlier -discussion, lower confidence (grep-confirmed absence only, not read in full -context), likely to share root causes once grouped rather than being 15+ -independent bugs. +```json +"valid_values": [ + "Project", "Campaign", "Task", "PersonalProject", "StudyProject", "Experiment" +], +"description": "A string classifying the project. Supported values are Campaign, Task, PersonalProject and StudyProject." +``` ---- +`valid_values` has six entries; the description names four. `Project` (which +is also this attribute's own `default_value`) and `Experiment` are both +missing from the prose. The machine-readable list is correct — it matches +Egeria 6's actual Project classifications in +`OpenMetadataType.java` (0130): `CAMPAIGN_CLASSIFICATION`, +`TASK_CLASSIFICATION`, `PERSONAL_PROJECT_CLASSIFICATION`, +`STUDY_PROJECT_CLASSIFICATION`, `EXPERIMENT_CLASSIFICATION`. Only the +description is behind. -### ISSUE-103: `Link Agreement Terms and Conditions`'s compact spec declared the wrong relationship type entirely (`CollectionMembership` instead of `AgreementItem`), and the code that already implemented it correctly had its own phantom-key bug +**Why it matters more than one stale string:** the description is what +`gen_md_cmd_templates` and `gen_dr_help` render, so it has propagated. In +this checkout, 25 files in the main tree carry the sentence verbatim — the +one source JSON plus 24 generated artifacts: -**Status:** fixed 2026-09-17 (Dr.Egeria — compact spec via the Spec Editor -API, `md_processing/v2/governance.py`), verified live against -`qs-view-server` (`Create Agreement` → `Create Terms and Conditions` → -`Link Agreement Terms and Conditions`, fetched back, confirmed the real -`AgreementItem` relationship with all 4 properties persisted, then deleted). -Resolves the "investigated, explicitly not fixed" item from ISSUE-101. +``` +md_processing/data/compact_commands/commands_project_compact.json <- source +md_processing/data/compact_commands_backup/commands_project_compact.json +sample-data/templates/{basic,advanced}/Projects/*.md (16) +sample-data/egeria-inbox/dr-egeria-help-*.md (7) +``` -**Root cause:** the compact spec (`OM_TYPE: CollectionMembership`, bundle -`Collection Membership`) and the processor code -(`GovernanceLinkProcessor`'s `elif object_type == "Agreement T&C":` branch, -building `AgreementItemProperties` and calling `_async_link_agreement_item`) -disagreed about which real Egeria relationship type this command uses. -Confirmed via the live type system (`ValidMetadataManager._async_get_all_relationship_defs()`, -filtered to names containing "Agreement"/"Term") that `AgreementItem` is a -real, distinct relationship (`endDef1: Agreement`, `endDef2: Referenceable`, -fields `agreementItemId`/`agreementStart`/`agreementEnd`/`entitlements`/ -`restrictions`/`obligations`/`usageMeasurements`) with no `membershipType` -field at all — the code was right, the spec was wrong. User confirmed -`linkAgreementItem` was the intended relationship before this was -investigated further. +The sharpest symptom: **`sample-data/templates/{basic,advanced}/Projects/Create_Experiment.md` +tells the reader that `Experiment` is not a supported value** — on the +template whose entire purpose is to create one. `Create_Project.md` has the +same problem for `Project`. The generated help tables are self-contradictory +in a single row, because they print the prose and the `valid_values` list +side by side: -**Fix — compact spec:** changed `OM_TYPE` to `AgreementItem` and pointed the -command at a new bundle `Agreement T&C Base` -(`Agreement Item Id`/`Agreement Start Date`/`Agreement End Date`/`Usage -Measurements`) — deliberately narrower than the sibling `Link Agreement -Item` command's own `Agreement Item` bundle, since `Entitlements`/ -`Obligations`/`Restrictions` already live on the linked `Terms and -Conditions` element itself (its own dedicated bundle), not on this -relationship instance, and reusing the sibling's full bundle would have -forced every user to also fill in a redundant `Item Name` (duplicating -`Terms & Conditions Id`, `min_cardinality: 1`) for no reason — confirmed by -trying the wholesale-reuse approach first and hitting exactly that -validation error live. +``` +... Supported values are Campaign, Task, PersonalProject and StudyProject. | False | Project, Campaign, Task, PersonalProject, StudyProject, Experiment | Domain | +``` -**Near-miss caught and fixed, not shipped:** the first attempt created a -new bundle also named `Agreement Item` in the Governance Officer family's -own file. Bundle names are merged globally across every compact JSON file -the same way attribute names are (`compact_loader.py`'s `all_bundles`, -last-file-alphabetically-wins) — undocumented anywhere until now. Digital -Product Manager's file (`digital_products` < `governance_officer` -alphabetically) already had a real, working `Agreement Item` bundle used by -`Link Agreement Item`; the new one would have silently replaced it at the -next `refresh_specs`, corrupting that command's attribute set. Caught only -because `Link Agreement Item` was noticed by chance while researching this -fix — not by any tool, since the Spec Editor's structural validation and -`attribute_sharing` reporting are both file-scoped and don't cover this. -Fixed by renaming to the distinct `Agreement T&C Base` instead. Documented -in the `compact-attr-global-namespace` memory and the -`dr-egeria-command-sync` skill's Step 1, both updated to cover bundles, not -just attributes. +A reader who trusts the sentence over the column will never reach for +`Experiment`, which is the one classification carrying its own defining +attribute (`hypothesis`), so the omission suppresses a real capability +rather than just reading untidily. -**Fix — code:** `GovernanceLinkProcessor`'s `Agreement T&C` branch read -`attributes.get("Start Date", ...)`/`attributes.get("End Date", ...)` — a -phantom-key mismatch (this command's bundle, correctly, never provided those -generic names) found only because properly wiring the bundle exposed it. -The sibling `CollectionLinkProcessor`'s `Link Agreement Item` branch already -correctly used `Agreement Start Date`/`Agreement End Date` for the identical -`agreementStart`/`agreementEnd` fields — renamed to match. +**Where seen:** found while mapping Resource Explorer's investigation +`project_classification` onto Egeria's Project classifications — the +description was the first thing read, and it made `Experiment` look +unsupported until `valid_values` and the Egeria type source were checked. -**Verified live:** created a throwaway `Agreement`, `Terms and Conditions`, -and linked them via `Link Agreement Terms and Conditions` with `Agreement -Item Id`/`Agreement Start Date`/`Agreement End Date`/`Usage Measurements` -set. Fetched the `Agreement` back and confirmed a real `AgreementItem` -relationship (not `CollectionMembership`) with all 4 values persisted -exactly as sent, linking to the correct `TermsAndConditions` element. Both -`commands_governance_officer_compact` and `commands_digital_products_compact` -re-validated clean (`structural_ok: true`, no new warnings) after cleanup of -the near-miss. `pytest tests/micro-tests -m unit`: no regressions. +**Candidate fix (one string, then regenerate):** edit the `description` in +`commands_project_compact.json` to cover all six, ideally naming what each +means rather than just listing them again — the `valid_values` array +already lists them, so prose that only repeats the list adds nothing and +will drift again the next time a value is added. Suggested: ---- +> "A string classifying the project. `Project` (the default) applies no +> classification. `Campaign` is a long-term strategic initiative delivered +> through multiple projects; `Task` a self-contained short activity; +> `PersonalProject` an informal project an individual creates to organize +> their own work; `StudyProject` a focused analysis of a topic, person, +> object or situation; `Experiment` a project testing a hypothesis, which +> is recorded in the `hypothesis` attribute." -### ISSUE-101: `Is Case Sensitive` never mapped, field↔structure link sent the wrong wire property name for `Position`, and `Membership Type`/`Dependency Description` were phantom-key or unmapped on 5 relationship-property bodies +(Wording taken from the Egeria type definitions themselves so the two +cannot disagree.) -**Status:** fixed 2026-09-17 (Dr.Egeria — `md_processing/v2/data_designer.py`, -`md_processing/v2/collection_manager_processor.py`, -`md_processing/v2/solution_architect.py`), verified live against -`qs-view-server` with throwaway elements, including one raw-SDK isolation -test that caught a second bug this fix's own first attempt introduced. -Continuing the ISSUE-99/100 follow-up list. +Then re-run `refresh_specs`, `gen_md_cmd_templates` and `gen_dr_help` per +the Dr.Egeria command-sync flow and propagate the regenerated templates/help +to `egeria-workspaces` and `egeria-advisor` — the 24 generated files above +will not update on their own, and 16 of them ship as user-facing templates. -**`Is Case Sensitive`** (Create Data Class) — confirmed by the user to be a -real `DataClass` field. Added `"isCaseSensitive": attributes.get('Is Case -Sensitive', {}).get('value')` to `DataClassProcessor`'s properties dict. -Verified live: `True` persisted correctly. +**Worth checking while in there:** whether any other compact-command +attribute has a `description` that enumerates its `valid_values` in prose. +Any such pair is the same latent defect — two lists that must be edited +together with nothing enforcing it. A cheap guard would be a spec-validation +check that flags a `description` naming a subset of its own `valid_values`. -**`Position`/`Minimum Cardinality`/`Maximum Cardinality`** (declared on -`Create Data Field`, meant for the field↔structure `MemberDataField` -relationship) — **the real fix site was not `LinkFieldToStructureProcessor`** -(the standalone `Link Data Field to Data Structure` command has no -cardinality attributes at all) but `DataFieldProcessor._sync_all_rels()`, -whose `add` lambda for the Data Structures sync called -`_async_link_member_data_field(ds, guid, None)` — body always `None`. Fixed -by threading `attributes` through both call sites into `_sync_all_rels` and -building a real `MemberDataFieldProperties` body. +--- -**Caught during live verification, not before:** the first attempt used -`"dataFieldPosition"` as the wire key, taken from the SDK method's own -docstring sample body (`data_designer.py` line ~2286) — that docstring is -**wrong**. Live verification showed `position` silently staying `0` (server -default) while `minCardinality`/`maxCardinality` round-tripped, which didn't -match "attribute never sent" (that would leave all three at default). -Queried the live type system directly (`ValidMetadataManager._async_get_all_relationship_defs()`, -filtered to `MemberDataField`) — the real, only attribute name is `position`, -not `dataFieldPosition`. Fixed the key; a fresh throwaway round-trip then -showed `position: 3` (sent value) correctly persisted. **The SDK docstring -itself is still wrong** (not fixed here — out of scope for this pass, but a -real, confirmed-live ground-truth error worth a follow-up). +### ISSUE-91: `pyegeria.core.mcp_server` imports `mcp.server.mcpserver` (mcp 2.x only) but `pyproject.toml` declares `mcp >=0.1` — any consumer that resolves mcp 1.x gets a server that dies at import -**Not a bug, a genuine Egeria server quirk — do not attempt to fix -client-side:** even with the correct key, `minCardinality` consistently -persists as `maxCardinality`'s value regardless of what's actually sent. -Isolated with a raw SDK call bypassing Dr.Egeria entirely -(`_async_link_member_data_field` called directly with -`{"position": 3, "minCardinality": 1, "maxCardinality": 5}`) — server still -returned `minCardinality: 5`. Confirmed server-side, not a pyegeria or -Dr.Egeria client bug; noting here so nobody re-investigates the client code -for this specific symptom. +**Layer:** pyegeria packaging · **Status:** fixed (2026-09-13) · **Found:** 2026-09-05 (Egeria Advisor dev startup on the M3 Max) -**`Membership Type`** — user confirmed this is a real `CollectionMembership` -relationship property (not spec cruft, despite zero hits across every -`.http` ground-truth file — those worked examples are evidently non-exhaustive). -Added to both real fix sites: -- `CollectionLinkProcessor`'s `Add Member to Collection` branch - (`collection_manager_processor.py`). -- `SolutionLinkProcessor`'s generic `om_type == "CollectionMembership"` - branch (`solution_architect.py`) — one fix covers all 4 affected commands - (`Link Actor to Blueprint`, `Link Blueprint Child`, `Link Information - Supply Chain Child`, `Link Solution Component to Blueprint`), confirmed by - checking each command's `OM_TYPE` via the Spec Editor API before editing. +Commit 2b39ba06 (2026-07-30, "migrate mcp_server.py to mcp 2.0.0's MCPServer") changed the +server's import to: -Verified live (Dr.Egeria pipeline, not raw SDK, to also confirm the parser/ -processor wiring): `membershipType: 'TestMembershipType2'` persisted -correctly on an `Add Member to Collection` command. +```python +from mcp.server.mcpserver import MCPServer +``` -**Investigated, explicitly not fixed — a different, deeper bug:** -`Link Agreement Terms and Conditions` ("T&C") was also on the `Membership -Type` list, but its compact spec (`OM_TYPE: CollectionMembership`, bundle -`Collection Membership`) does not match what the processor code actually -does — `GovernanceLinkProcessor`'s `elif object_type == "Agreement T&C":` -branch builds an entirely different `AgreementItemProperties` body and calls -`_async_link_agreement_item`, which has no `membershipType` field at all. -This is a spec/code type mismatch, not a missing-field gap — adding -`membershipType` to the wrong property class would be a no-op at best. Left -untouched; needs someone to determine whether the compact spec's `OM_TYPE` -is stale or the code's relationship-type choice is wrong, before any fix. +That module exists only in `mcp >= 2.0.0` (1.x ships `mcp.server.fastmcp` instead). The +dependency declaration was not updated and still reads `"mcp >=0.1"` — verified in the released +6.1.5 wheel's METADATA (`Requires-Dist: mcp>=0.1`) and in the current `pyproject.toml` at 6.1.10. -**`Dependency Description`** (`Link Product Dependency`) — phantom-key bug, -same shape as `Allow Duplicates` in ISSUE-100: -`CollectionLinkProcessor`'s `Product Dependency` branch read -`attributes.get('Description', ...)`, but this command's `custom_attributes` -are only `["Dependency Description", "Digital Product 1", "Digital Product -2"]` — no generic `Description` attribute exists on it at all. Renamed the -key. **Adjacent phantom key found, not fixed:** the same block also reads -`attributes.get('Label', ...)` — `Label` isn't in this command's -`custom_attributes` either, so it's always `None`; harmless (an optional -field silently unset, not a required one silently dropped) but flagged for -whoever next touches this block. +**How it shows up.** trellis pinned `mcp>=1.0.0` and its lock resolved mcp 1.29.0, which satisfies +pyegeria's declared range. Launching the server then fails before it can speak MCP: -**Verification:** all four confirmed fixes re-checked with -`scripts/dr_egeria_attribute_consumption_audit.py` scoped per command — -`Is Case Sensitive` and `Position`/`Minimum Cardinality`/`Maximum -Cardinality` no longer appear on `Data Class`/`Data Field`'s finding lists; -`Add Member to Collection`, `Link Actor to Blueprint`, `Link Product -Dependency` all show 0 findings. `pytest tests/micro-tests -m unit`: no -regressions. +``` +$ python -m pyegeria.core.mcp_server +MCP import failed. + File ".../pyegeria/core/mcp_server.py", line 24, in + from mcp.server.mcpserver import MCPServer +ModuleNotFoundError: No module named 'mcp.server.mcpserver' +``` -**Still open from the ISSUE-99/100 list:** `Link Agreement Terms and -Conditions`'s spec/code mismatch fixed separately, see ISSUE-103. Still -open: `Operation Name`/`Last Notification`/`Assignment Type`/`Expected Time -Allocation Percent` on other Link commands, and the longer single-command -tail (`Expected Behavior`, `Example`, `Due Time`/`Objective`/`Requested -Start Time`, `Canonical Scope`/`Is Canonical`/`Is Taxonomy`, `Subscription -Level`, `Deployed Implementation Type`/`Resource Name`, `Default Media Usage -Other Id`/`Media Type Other Id`, `Role Identifier`/`Role Type`, `Role List`, -`In Data Value Specification`/`Specializes Data Value Specification`, `In -Data Field`, `Grain Statement`/`Granularity Basis`/`Interval`). +A client sees only "No response from MCP server" because the traceback goes to stderr and the +process exits without writing a JSON-RPC frame. With mcp 2.1.1 installed the same command answers +`initialize` normally (`serverInfo.name = "pyegeria-mcp"`). ---- +**Why it went unnoticed.** The quickstart containers run mcp 2.1.1 (pyegeria 6.1.9) and use their own +`/app/mcp_server.py` mounted over SSE in `pyegeria_handler`, not this stdio module, so the +egeria-workspaces path never exercised it. The egeria-python checkout's own venv also has mcp 2.0.0. -### ISSUE-100: `Create Data Lens`'s 9 fields were never mapped (same shape as ISSUE-71), and `Allow Duplicate Values` was either read under a phantom key or not read at all +**Proposed fix (backward compatible for callers).** In `pyproject.toml` declare `"mcp >=2.0"`. Nothing +else needs to change: the import is already 2.x-only, so raising the floor only turns a runtime +import crash into a resolver error at install time. If 1.x support is wanted instead, gate the +import (`try: from mcp.server.mcpserver import MCPServer except ImportError: from mcp.server.fastmcp +import FastMCP as MCPServer`) — but the 2.x API differs beyond the class name, so the floor bump is +the honest option. -**Status:** fixed 2026-09-17 (Dr.Egeria — -`md_processing/md_processing_utils/common_md_utils.py`, -`md_processing/v2/data_designer.py`), verified live against -`qs-view-server` with throwaway elements. Two of the three follow-up items -from ISSUE-99's list, picked next by priority. +**Consumer-side workaround applied in trellis (8441efb):** both packages now declare `mcp>=2.0.0` +and the lock carries mcp 2.1.1, matching the quickstart containers. -**Data Lens:** `update_gov_body_for_type()` (`common_md_utils.py`, called by -every `GovernanceProcessor`-routed command) had no branch for `DataLens` — -all 9 of its compact-spec attributes (`Data Collection Start/End Time`, -`Max`/`Min Height`/`Latitude`/`Longitude`, `Scope Elements`) fell through to -the generic fallback at the end of the function, exactly the shape the -`GovernanceActionType`/`GovernanceActionProcessStep` branch immediately -above it already calls out as ISSUE-71. No dedicated Pydantic model exists -for `DataLensProperties` (it's a raw-dict body validated generically by -`_async_create_element_body_request`), so unlike ISSUE-98/99 there was no -second model-level gap to fix. Field names confirmed against both the -`GovernanceOfficer._async_create_data_lens`-style SDK method's own docstring -sample body and the Spec Editor API. **Note:** the SDK docstring also shows -`dataValidityStartTime`/`dataValidityEndTime`/`dataCoverageStartTime`/ -`dataCoverageEndTime` as real wire fields with no corresponding compact-spec -attribute at all — out of scope for this fix, a possible future addition if -wanted, not a bug. +--- -**`Allow Duplicate Values` — two different bugs at two different sites, one -false lead:** -- `DataClassProcessor.apply_changes()` (`data_designer.py` ~line 504) read - `attributes.get('Allow Duplicates', ...)` — a key that does not exist - anywhere in the compact spec (real name: `Allow Duplicate Values`, on - `Data Class Base`) — always silently defaulted to `True`. Renamed the key. -- `set_data_field_body()` (`common_md_utils.py`) never referenced the - attribute at all, despite `Allow Duplicate Values` being directly in - `Create Data Field`'s `custom_attributes`. Added the missing line. -- **False lead, not fixed:** the identical `'Allow Duplicates'` string also - appears in `DataValueSpecificationProcessor` (`data_designer.py` line 54). - Confirmed harmless dead code — `Create Data Value Specification`'s own - bundle doesn't declare `Allow Duplicate Values` at all, so this lookup - always returns the default regardless of the key's spelling. Left as-is; - renaming it would be cosmetic, not a fix. +### ISSUE-91: `pyproject.toml` declares `mcp >=0.1`, but `pyegeria.core.mcp_server` needs `mcp>=2.0` — the declared floor lets a resolver install a version too old to import the module at all -**Verified live:** created a throwaway Data Class (`Allow Duplicate Values: -false`), Data Field (`Allow Duplicate Values: false`), and Data Lens (all 6 -geo fields set), fetched all three back, confirmed -`properties.allowsDuplicateValues` was `False` (not the `True` default) on -both, and all 6 Data Lens fields (`maxHeight`/`minHeight`/`maxLatitude`/ -`minLatitude`/`maxLongitude`/`minLongitude`) persisted with their exact -values — then deleted all three. Re-ran -`scripts/dr_egeria_attribute_consumption_audit.py` scoped to each affected -command: `Data Lens` now 0 findings; `Allow Duplicate Values` no longer -appears in either `Data Class`'s or `Data Field`'s remaining finding list. -`pytest tests/micro-tests -m unit`: no regressions. +**Layer:** Pyegeria · **Status:** fixed (2026-09-13) -- duplicate report of the ISSUE-91 entry above (same number, same root cause, found a day later from a different angle: containerized demo deployment rather than dev startup). Confirmed current `pyproject.toml` declares `"mcp >=2.1.1"` · **Found:** 2026-09-06 (Egeria Advisor, containerized demo deployment rebuild against pyegeria 6.1.10). -**Still open from ISSUE-99's follow-up list** (not touched this round): -`Position`/`Minimum Cardinality`/`Maximum Cardinality` on the field↔structure -link, `Is Case Sensitive` (Data Class), the `Membership Type`/relationship- -property gaps across 6 Link commands, and the longer single-command tail — -see ISSUE-99 for the full list. +`pyproject.toml`'s `[project.dependencies]` declares `"mcp >=0.1"`, but +`pyegeria/core/mcp_server.py` imports `from mcp.server.mcpserver import +MCPServer` — confirmed live in this checkout's dev venv: +`importlib.metadata.version("mcp")` is `2.0.0`, and +`mcp.server.mcpserver.MCPServer` only exists at that version; the module +path is new to the 2.x line, not present in the 0.x/1.x `mcp` package +history. A resolver that's free to pick anything satisfying `>=0.1` (no +upper or tighter lower bound forcing 2.x) can legitimately land on a much +older `mcp` release, at which point `import pyegeria.core.mcp_server` +fails outright rather than degrading gracefully. + +**Why it matters, found how:** a container rebuild of Egeria Advisor's +demo deployment against pyegeria 6.1.10 pinned `mcp==2.1.1` explicitly +(not resolved from pyegeria's own floor) and confirmed EA's MCP agent +pre-warms cleanly on it — so 6.1.10 itself works fine when the caller +pins a modern `mcp`, but nothing in pyegeria's own declared dependency +would have caught a caller who didn't. + +**Ask:** tighten `pyproject.toml`'s `mcp` constraint to actually match +what `mcp_server.py` requires (`mcp>=2.0` at minimum — confirm the exact +version `MCPServer`/`mcp.server.mcpserver` was introduced at, and pin to +that) rather than the inherited placeholder `>=0.1` floor. Not +independently verified against the `mcp` package's own changelog/git +history here — the live import check above shows 2.0.0 works and the +module path is absent from the 0.x/1.x line by inspection, but the exact +first-working version wasn't pinned down. --- -### ISSUE-99: `Purpose` — a required attribute on ~30 Collection-family `Create` commands — was never mapped into the request body at all +### ISSUE-87: `ClassificationExplorer.add_ownership_to_element`'s docstring sample body says `"class": "OwnerProperties"` — the method itself only accepts `"OwnershipProperties"`, so the documented body cannot be sent -**Status:** fixed 2026-09-17 (Pyegeria/Dr.Egeria — -`md_processing/md_processing_utils/common_md_utils.py`, -`pyegeria/omvs/collection_manager.py`), verified live against -`qs-view-server` with a throwaway element. Found by -`scripts/dr_egeria_attribute_consumption_audit.py` (see below), a new -static-audit tool built in response to ISSUE-97/98 to find this whole bug -class proactively instead of one report at a time. +**Layer:** Pyegeria · **Status:** fixed 2026-09-05 (Pyegeria — +`pyegeria/omvs/classification_explorer.py`). Both docstrings (async +`_async_add_ownership_to_element` and its sync wrapper +`add_ownership_to_element`) corrected to `"OwnershipProperties"`, matching +the code's own `prop=["OwnershipProperties"]` and the `.http` ground +truth. Verified: `TypeAdapter(NewClassificationRequestBody).validate_python(...)` +with the corrected body now validates cleanly (previously raised +`ValidationError`/`unexpected property class name` from the class-name +mismatch). Swept the rest of the file per the original report's suggestion +(every `_async_new_classification_request` caller, matched by function +boundary rather than naive line-proximity to avoid false positives from +adjacent functions) — this was the only class-name/`prop=` mismatch in +`classification_explorer.py`; not repeated elsewhere. Found 2026-09-04 +(Resource Explorer, `Ownership` on everything it publishes). -**Root cause:** `set_element_prop_body()` — the base `Referenceable`-level -body builder every element type ultimately calls — never read `Purpose` at -all. `set_collection_manager_body()` (called for every `CollectionManagerProcessor`- -routed command: Collection, Data Spec, Data Dictionary, the whole Digital -Product family, Glossary, Report Type, Security Group/List/Role, Skill Set, -and ~20 more) only added Digital-Product-specific fields on top, never -`purpose`. **`Purpose` has `min_cardinality: 1` in the compact spec** — it's -presented to the user as required, they fill it in, and it was silently -discarded on every single one of these ~30 commands. Also missing from the -`CollectionProperties` Pydantic model (`pyegeria/omvs/collection_manager.py`) -— the same double-gap pattern as ISSUE-98's `Current Version`. Confirmed -`purpose` is a real wire field via `Egeria-api-collection-manager.http`'s -worked examples (appears in every `createCollection`-family sample body, -right after `authors`). +`classification_explorer.py:8231` `_async_add_ownership_to_element` documents: -**Scope note:** `Purpose` is also declared on `Create Solution Blueprint` -and `Create Information Supply Chain`, which use `set_element_prop_body()` -directly (not `set_collection_manager_body()`) and are **not** covered by -this fix — `purpose` does not appear anywhere in -`Egeria-api-solution-architect.http`'s ground truth, so unlike the Collection -case this looks like the same "spec cruft, no real DTO field" shape as -ISSUE-98's `Product Status`/`Product Type`, not a simple omission. Not -resolved here; flagged for the same maintainer call (map vs. remove) ISSUE-98 -got. +``` +{ + "class" : "NewClassificationRequestBody", + "properties" : { + "class" : "OwnerProperties", + "owner" : "Add value here", + ... +``` -**Fix:** added `purpose: str | None = None` to `CollectionProperties` (all -Collection subtypes inherit it), and -`prop_bod["purpose"] = attributes.get('Purpose', {}).get('value', None)` to -`set_collection_manager_body()`, unconditionally (applies to every subtype -routed through it, not just Digital Product). Verified live: created a -throwaway Data Specification with `### Purpose` set, fetched it back, -confirmed `properties.purpose` persisted correctly, then deleted it. -`pytest tests/micro-tests -m unit`: no regressions. +and then calls `self._async_new_classification_request(url, prop=["OwnershipProperties"], body=body)` +(`:8285`). `_server_client.py:6175` `validate_new_classification_request` compares +`body["properties"]["class"]` against `prop` and raises +`PyegeriaInvalidParameterException(reason="unexpected property class name")` +when it does not match — **before any HTTP request is made**. So the body in the +docstring is rejected client-side, 100% of the time. -**Also from this audit run, not yet fixed — filed for follow-up, not -speculative:** -- `Create Data Lens`'s 9 DataLens-specific fields (`Data Collection Start/End - Time`, `Max`/`Min Height`/`Latitude`/`Longitude`, `Scope Elements`) fall - through to the fully generic governance body builder — same shape as - ISSUE-98, a subtype that never got its own properties branch - (`md_processing/v2/governance.py`, `GovernanceProcessor`). -- `DataClassProcessor`/`DataFieldProcessor` (`data_designer.py` lines - ~54/504) read `attributes.get('Allow Duplicates', ...)` — that key does - not exist anywhere in the compact spec (real name: `Allow Duplicate - Values`) — always silently defaults to `True`. -- `Is Case Sensitive` (Data Class) confirmed absent from the `DataClassProperties` - body. -- `Position`/`Minimum Cardinality`/`Maximum Cardinality`, declared on - `Create Data Field`, are never sent on the actual field↔structure link — - `LinkFieldToStructureProcessor` sends a completely empty - `MemberDataFieldProperties` body, even though the live relationship - carries exactly these 3 fields as server defaults. -- `Membership Type` is declared as a relationship property on 6 different - Link commands (`Add Member to Collection`, `Link Actor to Blueprint`, - `Link Blueprint Child`, `Link Information Supply Chain Child`, - `Link Solution Component to Blueprint`, `Link Agreement Terms and - Conditions`) across 3 different processors - (`CollectionLinkProcessor`/`SolutionLinkProcessor`/`GovernanceLinkProcessor`) - and never read in any of them; same pattern for `Dependency Description` - (`Link Product Dependency`), `Operation Name` (`Link Associated List`), - `Last Notification` (`Link Notification Subscriber`), `Assignment Type` - (`Link Assignment Scope`), `Expected Time Allocation Percent` (`Link - Person/Team Role Appointment`) — all confirmed by reading the relevant - processor's relationship-properties body, not just the audit heuristic. -- A longer tail of single-command gaps (`Expected Behavior`, `Example`, - `Due Time`/`Objective`/`Requested Start Time`, `Canonical Scope`/`Is - Canonical`/`Is Taxonomy`, `Subscription Level`, `Deployed Implementation - Type`/`Resource Name`, `Default Media Usage Other Id`/`Media Type Other - Id`, `Role Identifier`/`Role Type`, `Role List`, `In Data Value - Specification`/`Specializes Data Value Specification` (likely the same - ISSUE-97 singular/plural shape), `In Data Field`, `Grain Statement`/ - `Granularity Basis`/`Interval`) confirmed absent by targeted grep but not - read in full surrounding context — one confidence tier below the items - above; see the audit script's own report for the full list. -- `Estimated Volumetrics` (`Create Information Supply Chain`) is **not** a - new finding — it cross-references the existing ISSUE-64 entry's own - follow-up note (suspected misattributed bundle field, not a simple - "never read" bug). +`pyegeria/http clients/Egeria-api-classification-explorer.http:670` uses +`"OwnershipProperties"` and agrees with the code. The docstring is the outlier; +the same sample appears again in the sync wrapper at `:8288`. -**New tool, added this session:** `scripts/dr_egeria_attribute_consumption_audit.py` -— a static audit that checks, for every compact-spec command, whether its -processor actually reads each declared attribute (`UNCONSUMED`) and whether -`Reference Name`/`Reference Name List` attributes are read with the matching -`guid`/`guid_list` cardinality key (`CARDINALITY_MISMATCH`, the ISSUE-97 -shape). It's a heuristic, not a proof — see its docstring's "Known -limitations" section — and it has one confirmed class of false positive: -`SolutionLinkProcessor`'s generic `id1_key`/`id2_key` peer-link mechanism -resolves the two link-endpoint attributes dynamically from -`spec.get("custom_attributes")[0]`/`[1]`, never by literal string match, so -its ~21 `Reference Name`-style findings should be disregarded. A full run -also surfaced that `COMMAND_DEFINITIONS["Command Specifications"]` yields -some command names more than once (a few findings appear duplicated) — a -tool-side dedup bug, not a second instance of the underlying finding. +**How it was found:** by running the documented body against the live +quickstart platform. The exception names neither the class it expected nor the +class it got — `unexpected property class name` with an empty context — so +reading the error does not lead to the answer; only reading +`validate_new_classification_request` does. + +**Ask:** correct both docstrings to `"OwnershipProperties"`. Worth a sweep of +the other `_async_new_classification_request` callers in the same file for the +same docstring/`prop=` divergence — the two were written from different sources +here, so this is unlikely to be the only one. Separately, the validation error +would be far more useful if it named the expected and supplied class names; +`{"reason": "unexpected property class name"}` is a hard error to act on. --- -### ISSUE-98: `Create Digital Product`'s `Product Status`/`Product Type` were spec cruft with no real DTO field, and `Current Version` was silently dropped +### ISSUE-86: `exec_report_spec` (and `_exec_analytic_chart`, `_run_report_spec`) accept only `user`/`user_pass` — no way to run a report with a bearer token the caller already holds -**Status:** fixed 2026-09-17 (Pyegeria/Dr.Egeria — compact spec via the -Spec Editor API, `md_processing/md_processing_utils/common_md_utils.py`, -`pyegeria/omvs/collection_manager.py`), verified via `refresh_specs` + -unit tests. Reported by an external maintainer/user agent. +**Layer:** Pyegeria · **Status:** fixed 2026-09-05 (Pyegeria — +`pyegeria/view/format_set_executor.py`, `pyegeria/core/mcp_adapter.py`). +Added `token: Optional[str] = None` to `exec_report_spec`, +`_exec_analytic_chart`, and `_run_analytic_function` (the function this +entry's title calls `_run_report_spec` — no such name exists in the +codebase; `_run_analytic_function` is the one all three client-building +call sites actually route through), threaded through every place a client +is built from `view_server`/`view_url`/`user`/`user_pass`. When `token` is +given, the client is authenticated with `set_bearer_token(token)` instead +of `create_egeria_bearer_token()`; when omitted, behavior is byte-for-byte +unchanged (`user`/`user_pass` still mint a fresh token exactly as before). +Also threaded through `run_report`/`_execute_egeria_call_blocking` in +`pyegeria/core/mcp_adapter.py`, per the original ask's "same for +run_report/... on the MCP side if they share the builder" — both do, +both now accept `token`. `describe_report` doesn't build a client at all +(pure registry lookup), so it needed no change. The MCP server's own +`run_report` tool (`pyegeria/core/mcp_server.py`) takes a different, +already-token-capable path (`_async_run_report_tool` + a pre-built +`egeria_client`, not `exec_report_spec`'s builder) — out of scope per the +same "if they share the builder" qualifier, not touched. -**Reported symptom:** `Product Status`, `Product Type`, and `Current -Version` on `Create Digital Product` were not mapped into the request -body at all, so products carried their status in `Maturity` instead. +New unit test `tests/micro-tests/test_exec_report_spec_token.py` (2 tests, +monkeypatched `EgeriaTech`, no live server): confirms `token` routes +through `set_bearer_token` with zero calls to `create_egeria_bearer_token`, +and confirms the no-token path is unchanged. Full `pytest +tests/micro-tests/` green (exit 0) throughout. -**Root cause, `Current Version`:** `set_collection_manager_body()`'s -Digital Product branch built `productName`/`maturity`/`serviceLife`/ -`introductionDate`/`withdrawalDate`/`nextVersionDate` but never read -`Current Version` — a plain omission. It's also a real wire field -(`currentVersion`, confirmed in `Egeria-api-product-manager.http`'s -`createDigitalProduct` example) that was additionally missing from the -`DigitalProductProperties` Pydantic model in -`pyegeria/omvs/collection_manager.py` — a double gap per this repo's -known "declared-but-not-on-the-model = silently dropped" pattern -(see ISSUE-62). A second, dead body-builder `set_product_body()` (never -called from anywhere) had the identical gap, left as-is since it's unused. +**Original report, found 2026-09-04** (Egeria Advisor, trellis-auth token +contract change): -**Root cause, `Product Status`/`Product Type`:** unlike `Current Version`, -neither corresponds to any real field on the Egeria -`DigitalProductProperties` DTO — confirmed against the `.http` ground -truth. Real lifecycle status is set via a separate -`updateDigitalProductStatus` operation using `contentStatus`/ -`deploymentStatus`, not exposed by any Dr.Egeria command for Create/Update -Digital Product. These two attributes were spec cruft, not simply -unmapped — user confirmed removal rather than inventing a mapping. +`pyegeria/view/format_set_executor.py:686` `exec_report_spec(..., user=, user_pass=)` builds its own +client (`:189` and `:814`: `client_class(view_server, view_url, user_id=user, user_pwd=user_pass)` +then `client.create_egeria_bearer_token()`) and passes `user`/`user_pass` down to `_exec_analytic_chart` +(`:718`) and the format-row path (`:780`). There is no parameter for a pre-built client or a bearer +token, and `create_egeria_bearer_token()` is unconditional. -**Fix:** added `"currentVersion": attributes.get('Current Version', {}).get('value', None)` -to `set_collection_manager_body`'s Digital Product branch, and -`current_version: str | None = None` to `DigitalProductProperties`. -Removed `Product Status`/`Product Type` from the `Digital Product Base` -bundle and their attribute definitions from -`commands_digital_products_compact.json` via the Spec Editor's REST API -(neither was shared with another family — `attribute_sharing` was empty -for both), then ran `refresh_specs --merge-reports` to regenerate -templates/help/report specs. `validate_compact_specs`: 0 errors. Proper -status support for Digital Products (via `contentStatus`/ -`deploymentStatus`) is not implemented — no command currently calls -`updateDigitalProductStatus` — and would need a follow-up if wanted. +**Why it matters:** Egeria Advisor's app JWT carries the user's Egeria bearer token and no longer +carries the password (matching the Portal, whose `X-Egeria-Token` + `set_bearer_token()` pattern +already does this for every PyegeriaWebHandler endpoint). Every other EA call site builds a client +and calls `set_bearer_token(token)`; report execution cannot, so EA falls back to the service +account for that one path and Egeria's provenance records the service account instead of the +person who ran the report (`advisor/report_pipeline.py`, search "ISSUE-86"). + +**Ask:** add `token: str | None = None` (or `client: EgeriaTech | None = None`) to `exec_report_spec` +and thread it to the two builders; when given, `set_bearer_token(token)` instead of +`create_egeria_bearer_token()`. Same for `run_report`/`describe_report` on the MCP side if they share +the builder. Backward compatible: `user`/`user_pass` keep working when no token is given. --- -### ISSUE-97: `Create Data Structure`'s `In Data Specification` and `Create Data Field`'s `In Data Structure` were silently dropped — spec declares them singular, processor only read a `guid_list` +### ISSUE-84: `SolutionArchitect.create_solution_blueprint`'s own docstring documents a `NewSolutionElementRequestBody` body (with `initialStatus`) for Draft-status creation — that class does not exist as a pydantic model, so the documented shape fails client-side validation before any HTTP call -**Status:** fixed 2026-09-17 (Dr.Egeria — `md_processing/v2/data_designer.py`), -verified live against `qs-view-server` with throwaway elements (created, -relationship confirmed via direct fetch, then deleted). Reported by an -external maintainer/user agent. +**Status:** fixed and live-verified 2026-09-03 — docstring-only fix (see +"Fix landed" below), merged upstream via +[PR #338](https://github.com/odpi/egeria-python/pull/338). Reported by a +consumer (trellis/Resource Explorer's `BlueprintMaterializer`, +`docs/blueprint-materialization-plan.md` Phase A), who also live-verified +it once their shared `quickstart-egeria-main` instance recovered from an +unrelated Postgres connection-pool exhaustion (`jdbcMaximumPoolSize` bump). -**Reported symptom:** both attributes are declared singular -(`"style": "Reference Name"`, `max_cardinality: 1`) in -`commands_data_designer.json`, but the link was never established — no -error, `--process` reported `SUCCESS`. Users worked around it with -explicit `Add Member to Collection` / `Link Field to Structure` blocks, -deliberately omitting the `In` attributes so a fix couldn't double-link. +**Confirmed in both**: this checkout's installed pyegeria (5.3.4.23, via +the trellis venv) and this repo's own working tree (6.1.9-dev) — +`grep -rn "class NewSolutionElementRequestBody" pyegeria/models/*.py` +returns nothing in either; the class is documented in +`SolutionArchitect.create_solution_blueprint`'s (and its `_async_*` +sibling's) docstring — twice, once per method, `omvs/solution_architect.py` +lines ~2325 and ~2440 in this tree — as the way to set `initialStatus` +(DRAFT/PREPARED/PROPOSED/APPROVED/REJECTED/ACTIVE/DISABLED/DEPRECATED/OTHER) +on a newly-created blueprint, distinct from `NewElementRequestBody` which +"sets the status to ACTIVE." No such model backs that documented shape. -**Root cause:** the parser (`md_processing/v2/processors.py`, ~line 787-840) -stores a singular `Reference Name` attribute's resolved guid under -`attr_data["guid"]`, and only populates `attr_data["guid_list"]` when the -raw input value is itself a list. `DataStructureProcessor.apply_changes` -(line 220) and `DataFieldProcessor.apply_changes` (line 326) both read -only `.get("guid_list", [])` for `In Data Specification`/`In Data -Structure`, which is never set for these singular attributes — the -existing `if isinstance(x, list) else [x]` wrapper around the *default* -`[]` never triggers, since a default empty list is already a list. The -correct sibling pattern already existed in the same file (`Data Class`, -`Specializes Data Class`, `Glossary Term` all correctly also check the -singular `guid` key) — just not applied here. +**What actually happens**: `create_solution_blueprint`'s real validation +path (`ServerClient._async_create_element_body_request` → +`validate_new_element_request` → `self._validate_body(self. +_new_element_request_adapter.validate_python, body)`) uses a bare +`TypeAdapter(NewElementRequestBody)` (set in `ServerClient.__init__`) — +`NewElementRequestBody.class_` is `Annotated[Literal["NewElementRequestBody"], +Field(alias="class")]`, a strict literal. Any other `class` value — +including the documented `"NewSolutionElementRequestBody"` — fails pydantic +validation locally, before any network call, raising +`PyegeriaInvalidParameterException` with `additional_info={"reason": +"Request body failed validation", "validation_errors": [...]}`. The +resulting exception message reads exactly like a server-side rejection +("Egeria rejected the new SolutionBlueprint: ...VALIDATION_ERROR_1... +Invalid parameters were provided...") and cost real debugging time on the +consumer side tracing it back to "not actually an Egeria call at all." -**Also found and fixed as the same bug:** `In Data Dictionary` (also a -singular `Reference Name` attribute per the compact spec) had the -identical `guid_list`-only read at 3 call sites — `DataStructureProcessor`, -`DataFieldProcessor`, and `DataClassProcessor`. +Reproduced directly: +```python +from pyegeria.models import NewElementRequestBody +from pydantic import TypeAdapter +TypeAdapter(NewElementRequestBody).validate_python({ + "class": "NewSolutionElementRequestBody", "isOwnAnchor": True, + "initialStatus": "DRAFT", + "properties": {"class": "SolutionBlueprintProperties", + "qualifiedName": "test", "displayName": "test"}, +}) +# ValidationError: Input should be 'NewElementRequestBody' +# [type=literal_error, input_value='NewSolutionElementRequestBody', ...] +``` -**Fix:** all 4 call sites (`In Data Specification`, `In Data Structure` x2, -`In Data Dictionary` x3) now fall back to the singular `guid` key when -`guid_list` is absent, matching the existing `Glossary Term`/`Specializes -Data Class` pattern. Verified end-to-end against a live server: created a -throwaway `Data Specification` → `Data Structure` (`In Data Specification`) -→ `Data Field` (`In Data Structure`), fetched both back and confirmed the -real `CollectionMembership` and `MemberDataField` relationships were -created, then deleted all three. `pytest tests/micro-tests -m unit`: all -pass, no regressions. +**Scope beyond `create_solution_blueprint`**: `create_solution_component` (and +its `_async_*` sibling) carries the identical docstring text and the +identical gap — confirmed by walking each `NewSolutionElementRequestBody` +docstring reference in `omvs/solution_architect.py` (lines 2254/2372/3542/3629 +in this tree) back to its enclosing method: `_async_create_solution_blueprint`/ +`create_solution_blueprint` (the two checked above) and +`_async_create_solution_component`/`create_solution_component`. Both public +create-element methods in this file document the same unbacked Draft-status +path. Not checked beyond `solution_architect.py`: worth a full grep across +`omvs/*.py` for any other method documenting `NewSolutionElementRequestBody`, +`NewGovernanceElementRequestBody`, or any other `New*ElementRequestBody` +variant that isn't `NewElementRequestBody` itself, since the same +"documented in the docstring, no model behind it" shape could recur +per-family. ---- +**Candidate fix** (not applied — for whoever picks this up): either (a) +define a real `NewSolutionElementRequestBody` pydantic model (subclassing +or extending `NewElementRequestBody` with an `initial_status` field and its +own `class_: Literal["NewSolutionElementRequestBody"]`) and register it +alongside `NewElementRequestBody` wherever `_new_element_request_adapter` +is built (likely needs a `Union`/discriminated-union `TypeAdapter`, not a +single-model one, so both class names validate), or (b) if +`initialStatus` is actually accepted by `NewElementRequestBody` itself on +the Egeria server side despite not being a declared pydantic field, add +`initial_status: str | None = None` directly to `NewElementRequestBody` +and drop the docstring's separate-class framing — cheaper, but needs +confirming server-side that `NewElementRequestBody` + `initialStatus` is +accepted (not verified here; the consumer's own fallback, tracked in their +own Backlog, is to use `NewElementRequestBody` with no `initialStatus` at +all, which creates the element ACTIVE, not Draft, until this is resolved). -### ISSUE-94: `SchemaMaker._async_delete_schema_type`/`_async_delete_schema_attribute` sent `MetadataSourceRequestBody` — the live server's schema-maker delete endpoints reject it outright +**Consumer-side workaround, now supersedable**: `BlueprintMaterializer. +materialize_blueprint_element` (trellis/resource_explorer, `surveyors/ +arch_recovery/blueprint_materializer.py`) sends `class: "NewElementRequestBody"` +with no status field at all, same as their pre-existing `ComponentMaterializer` +— blueprints materialize ACTIVE, not Draft. With this fix landed, that +workaround can be upgraded to set `contentStatus: "DRAFT"` inside +`properties` on the same `NewElementRequestBody` to get the originally +intended Draft-status behavior — not applied here since this is a +consumer-side change, tracked for `dwolfson-1b`/trellis to pick up. -**Status:** fixed 2026-09-11 (Pyegeria — `pyegeria/omvs/schema_maker.py`), -verified live against `qs-view-server`. +**Fix landed 2026-09-03 — docstring correction, no model change needed.** +Checked ground truth first: `NewSolutionElementRequestBody` and +`initialStatus` appear in **zero** `.http` files anywhere in this repo +(`grep -rln "NewSolutionElementRequestBody\|initialStatus" "pyegeria/http +clients/"` returns nothing) — the documented shape was never a real Egeria +API surface, not merely an unmodeled one, which rules out candidate fix (a) +(defining a real model) and (b) (adding `initial_status` to +`NewElementRequestBody`) from the original write-up. The actual, real +mechanism for setting status on these elements already exists: +`Egeria-api-solution-architect.http`'s own `updateSolutionBlueprintStatus` +example sets a blueprint's status via `"contentStatus": "ACTIVE"` inside an +`AuthoredReferenceableProperties`-shaped `properties` object — and +`content_status: str | None = None` is already a real field on +`ReferenceableProperties` (`pyegeria/models/models.py`), the base class +both `SolutionBlueprintProperties` and (transitively) +`SolutionComponentProperties` derive from. Since `NewElementRequestBody. +properties` is a bare `dict` (no nested pydantic validation), passing +`contentStatus` inside it at *creation* time was always possible — the +docstrings just never said so, and instead pointed at a body shape that +doesn't exist anywhere. -Found while live-verifying the new Asset Maker / Schema Maker Dr.Egeria -command families (#354). Both methods built `{"class": -"MetadataSourceRequestBody", ...}` and posted it via -`_async_metadata_source_body_request` — matching the ground-truth -`Egeria-api-schema-maker.http` file's documented body for these two -`POST .../schema-types/{guid}/delete` and -`.../schema-attributes/{guid}/delete` endpoints. Both nonetheless failed -against a live server with: +Rewrote all four docstrings (`_async_create_solution_blueprint`/ +`create_solution_blueprint`, `_async_create_solution_component`/ +`create_solution_component`, `omvs/solution_architect.py`) to drop every +reference to `NewSolutionElementRequestBody`/`initialStatus` and document +`contentStatus` inside `properties` instead, with a `NewElementRequestBody` +example that now sets `"contentStatus": "DRAFT"`. Two extra defects found +and fixed while doing this walk, both worse than the one originally +reported: +- `_async_create_solution_blueprint`'s *first* ("no lifecycle") example + body already carried stray `"userDefinedStatus"`/`"lifecycleStatus"` + fields left over from the fictional shape — with a **missing comma** + after `"lifecycleStatus": "DRAFT"`, making it invalid JSON even as a + copy-paste example, independent of the pydantic-validation bug. +- `_async_create_solution_component`/`create_solution_component`'s only + example already used the real `"class": "NewElementRequestBody"` (so it + passed validation) but paired it with the fictional top-level + `"initialStatus"` field anyway. This is a *different, silent* failure + mode from the reported one: `NewElementRequestBody` inherits + `PyegeriaModel`'s `extra='ignore'`, so `initialStatus` validated + successfully and was silently dropped before serialization — no + exception at all, just an element that always came out ACTIVE regardless + of the caller's intent to set DRAFT. Exactly the class of bug flagged + generally in this file's own `pyegeria/core/` gotcha note about + request-body models silently dropping undeclared fields. -``` -InvalidTypeIdException: Could not resolve type id 'MetadataSourceRequestBody' -as a subtype of DeleteElementRequestBody: known type ids = -[DeleteClassificationRequestBody, DeleteElementRequestBody, DeleteRelationshipRequestBody] -``` +Verified locally (no live server access from this environment): the +original repro's `TypeAdapter(NewElementRequestBody).validate_python(...)` +call with the old `NewSolutionElementRequestBody`/`initialStatus` shape +still fails as expected (proving the understanding of the original bug is +correct), and the corrected shape +(`{"class": "NewElementRequestBody", "properties": {..., +"contentStatus": "DRAFT"}}`) validates cleanly with `contentStatus` +preserved intact through to the serialized JSON. -i.e. the server's real Jackson type registry for these endpoints never -recognized `MetadataSourceRequestBody` as a valid subtype — the `.http` -ground truth was out of date against this server (confirmed by the user, -who requested the fix directly rather than filing an Egeria-side issue). +**Live-verified 2026-09-03 by `dwolfson-1b`/trellis**, once their shared +`quickstart-egeria-main` instance recovered from unrelated Postgres pool +exhaustion: `contentStatus: "DRAFT"` inside `properties` on +`NewElementRequestBody` creates without error and round-trips correctly on +read-back (`properties.contentStatus == "DRAFT"`), no separate request-body +class needed — confirmed twice, once as a standalone probe and once through +Resource Explorer's actual application path (accepting a real candidate +blueprint end-to-end via the Curate UI's API, materialized element guid +`809025b5-cca9-4e9a-a2f7-3a5104138f67`, independently queried back by that +guid to confirm). `elementHeader.status` stays `ACTIVE` regardless, as +expected — a different axis (header status vs. `contentStatus`, distinct +concepts in Egeria's status model), not a sign the fix is incomplete. -**Fix:** switched both methods to `DeleteElementRequestBody` via the -standard `_async_delete_element_request` helper (`cascade_delete` param -added to both, matching every other `_async_delete_*` in this codebase); -corrected `Egeria-api-schema-maker.http`'s two delete sections to match. -Verified live: created a throwaway `SchemaType` and `SchemaAttribute`, -deleted each with the default (no-body) call — both succeeded. +`BlueprintMaterializer.materialize_blueprint_element` has been upgraded to +send `contentStatus: "DRAFT"` in `properties`, replacing its earlier +ACTIVE-only workaround. `ComponentMaterializer`'s identical gap is +**still open** — same fix applies, not yet applied there, tracked +separately on the trellis/Resource Explorer side. ---- +**GUID note (2026-09-04):** `quickstart-egeria-main` was redeployed with a full +repository-store wipe the same day this was verified. `809025b5-cca9-4e9a-a2f7-3a5104138f67` +is historical — valid as of the 2026-09-03 verification date, not resolvable against the +platform after the wipe. The finding (`contentStatus` round-trips via `properties`, no +separate request-body class needed) is a property of the API/fix, not of that instance, and +is unaffected. -### ISSUE-52: `qs-nanny-daemon`/`qs-integration-daemon`'s own connectors generate sustained, heavy background write load against the shared repository — starves interactive requests, plausible cause of "frequent Postgres checkpoints" +--- -**Status:** fixed (Egeria server — `postgres-repository-connector`, -`jdbc-resource-connector`), confirmed 2026-08-30. The fix predates this -confirmation: the mechanism this entry identified on 2026-08-16 — 38 connections -stuck `idle in transaction` on the same `entity`-table query shape, waiting on -their own client rather than on Postgres — is a JDBC transaction leak, and it was -fixed the same day in `a4ab1c090a` ("Fix JDBC connection and transaction leaks in -the PostgreSQL connectors"). `DatabaseStore` became `AutoCloseable` with `close()` -rolling back, so a read leaves its connection idle rather than idle-in-transaction, -and all 28 of its uses became try-with-resources. `7359136cb1` then replaced the -per-thread connection cache with a HikariCP pool and `c52c1ba8bd` externalised the -pool size. The 6.2-SNAPSHOT build of 2026-08-24 that later re-checks were run -against already carried all three, which is consistent with this entry's own "net -assessment" that the acute symptom stopped reproducing. +### ISSUE-83: `AutomatedCuration.get_technology_type_elements(get_templates=True)` sends `skipClassifiedElements: [""]` (a list containing an empty string) instead of `[]` — Egeria rejects the empty classification name, so every Tech Catalog technology-type listing renders silently empty instead of erroring -The `OpenAPICataloguer` half of this entry does not reproduce in the code either: -`OpenAPIMonitorIntegrationConnector.getAPIOperationGUID()` looks up the existing -`APIOperation` by qualified name and only creates one when it finds nothing, so the -crawl converges rather than re-cataloguing. That matches the revised "cold-start -crawl that converges" framing this entry already reached, and means the remaining -`JacquardDigitalProductLoom` question is about refresh interval tuning, not about a -defect. +**Status:** fixed and live-verified 2026-09-02. -**Not independently re-measured against a live environment for this update** — the -basis is the code plus this entry's own post-restart observations. If the timeout -symptom ever recurs on a build newer than 2026-08-16, that would be a new problem -rather than this one, and worth a fresh entry. +**Root cause:** `_async_get_technology_type_elements` +(`pyegeria/omvs/automated_curation.py`) built `skipClassifiedElements` as +`[skip_templates]` where `skip_templates = "Template" if not get_templates +else ""` — when `get_templates=True` (the caller wants templates +*included*, i.e. skip nothing), this sent `["", ]`, a one-element list +containing an empty string, not `[]`. Egeria's server correctly rejects an +empty classification name: -**Moved here from "Open Egeria Server issues" on the basis of the above.** +``` +OMAG-COMMON-400-018 The type name passed on method findMetadataElements +of service Open Metadata Store Services is not recognized +``` -**Update 2026-08-16, more specific root-cause evidence.** Recurred while -retrying the `dr-egeria` help-Glossary `--process` step (see ISSUE-52's own -earlier "cold-start crawl" postscript — this shows the earlier optimism was -premature, it recurs under real batch load, not just right after a -restart): `docker stats` showed `egeria-shared-postgres` at up to **650% -CPU**, and connecting directly (`docker exec ... psql -U postgres -p 5442 --d egeria`) found the real mechanism — **38 connections stuck `idle in -transaction`** against the `egeria` database, all holding open the same -query shape (`select distinct * from entity where (version_end_time is -null) and (exists (select 1 from en...`). This is a connection/transaction -pileup, not simple CPU starvation as originally framed — many connections -opened a transaction, are waiting on something (`wait_event_type: Client`/ -`ClientRead`, i.e. waiting on their own client, not blocked by Postgres -itself), and never committed/closed, so newer requests (including a plain -`get_glossaries_by_name` lookup) queue behind them. CPU eased from 650% → -340% → 298% over ~15 minutes while sitting at this same connection count, -without by-name-search latency improving at all (stayed pinned at -25-30s/timeout) — the two aren't tightly coupled, reinforcing that this is -a connection-holding problem, not a raw compute-load problem. Whatever is -opening these `entity`-table transactions and not releasing them (almost -certainly the same `qs-nanny-daemon`/`qs-integration-daemon` connectors -already suspected) is the actual thing to fix, not just "connector load" -in the abstract. +**Why it went unnoticed:** the failure is masked downstream. `egeria- +workspaces`' `tech_catalog_handler.py` calls this with `get_templates=True` +unconditionally (`get_tech_type_elements`), and its own exception handler +treats any error message containing `"400"` as "no elements found," so the +whole Tech Catalog "Elements" listing for a technology type renders as an +honest-looking, self-consistent **empty list** instead of surfacing the +real 500. Any caller with a similar "400 means empty" convenience handler +would hit the same masking. -**Original status:** open (Egeria server / deployment config), found 2026-08-15 (Dan) -investigating why a `dr_egeria --validate`/`--process` run against the -`dr-egeria` help file (100+ commands) was taking 70+ minutes and timing out -individual calls (see ISSUE-51), and independently reported by the user as -"frequent checkpoints" observed in the Postgres console. +**Found by:** a user report — a known live `RelationalDatabase` asset +(`deployedImplementationType: "PostgreSQL Relational Database"`, confirmed +present via direct GUID lookup) did not appear browsing Technology Types → +Postgres Relational Databases in `egeria-workspaces`' Tech Catalog. +Reproducing the exact call chain (`AutomatedCuration. +get_technology_type_elements(filter_string="PostgreSQL Relational +Database", get_templates=True, ...)`, the parameters `tech_catalog_handler. +py`'s `/api/tech-catalog/tech-types/{qualified_name}/elements` route +actually uses) reproduced the 500 directly; the same call with +`get_templates=False` succeeded and returned the asset correctly (14 +items) — isolating the defect to the `get_templates=True` branch +specifically, not a metadata/zone/visibility problem on the asset itself. -**Layer:** Egeria Server (`egeria-quickstart` deployment's integration -daemon connector configuration), not pyegeria — this is a deployment/config -issue in the `egeria-shared-postgres`-backed local quickstart stack, not a -pyegeria code defect. +**Fix:** `skip_classified_elements = [] if get_templates else ["Template"]` +— an empty list (skip nothing) when templates should be included, a +one-element list naming the real `Template` classification to skip +otherwise. Also corrected the method's own docstring sample JSON body, +which showed the same `[""]` value as if it were correct usage. -**What:** confirmed live, same session: -- `egeria-shared-postgres` (the `pgvector/pgvector:pg17` container backing - `qs-view-server`'s repository, port 5442) is issuing time-based - checkpoints every 5 minutes (the Postgres default `checkpoint_timeout`) - with real, non-trivial WAL volume behind each one (~10-22 MB/checkpoint, - i.e. roughly 35-75 KB/s of sustained write throughput) continuously, - including in windows where no interactive Dr.Egeria/pyegeria work was - running — meaning the load is coming from something else running inside - the platform itself, not from client-side testing. -- `docker logs quickstart-egeria-main` shows the source: `qs-nanny-daemon`'s - `JacquardDigitalProductLoom` integration connector logged a single - refresh cycle that took **4,370,395 ms (~73 minutes)** to complete - (`INTEGRATION-DAEMON-SERVICES-0043`). `qs-integration-daemon`'s - `OpenAPICataloguer` connector is continuously creating new `APIOperation` - catalog entities — confirmed 703 total in the container's full log - history, 36 of them in the last 30 minutes of a single ~2 hour window - sampled — each one a real metadata write, evidently crawling/re-crawling - the platform's own REST API surface (dozens of - `/open-metadata/access-services/open-metadata-store/...` and - `/open-metadata/conformance-suite/...` paths) rather than converging to a - steady state. -- Directly reproduced the contention: a single, otherwise-simple - `client._async_get_element_by_guid_(guid)` call against `qs-view-server`, - issued from a fresh script with nothing else running client-side, hit the - 30-second client timeout and raised `PyegeriaTimeoutException` — - confirming the server itself, not the client or network, is the - bottleneck. +**Live-verified end-to-end**, not just unit-tested: live-patched into +`quickstart-pyegeria-web`'s installed pyegeria (`docker cp` + restart, +diff-confirmed byte-identical), then curled the actual production route — +`GET /api/tech-catalog/tech-types/PostgreSQL%20Relational%20Database/ +elements?display_name=PostgreSQL%20Relational%20Database` — which now +returns the previously-missing asset as the first result, alongside the +other 14 (15 total with templates included, vs. 14 without — consistent). -**Impact:** any pyegeria/Dr.Egeria workload that does more than a handful -of sequential server calls (bulk `--process` runs, the help-file Glossary -sync in particular) becomes unreliable — individual calls time out -(`TIMEOUT_ERROR_408`) or the whole run takes an order of magnitude longer -than expected — while these background connectors are active. This is very -likely the direct cause of ISSUE-51's crash (the ClassificationExplorer -call it depends on timed out under this exact load) and of the multi-hour -`dr_egeria --validate` run in this same session. +--- -**Update 2026-08-15, same day:** user restarted the Egeria environment with -the latest configuration. Post-restart, a `dr_egeria --process` run against -the same 100+-command help file completed its first ~4 minutes with 0 -errors (vs. timing out repeatedly before), and `OpenAPICataloguer` created -0 new `APIOperation` entities in a 10-minute post-restart sample (vs. 36 in -30 minutes before) — consistent with the crawl having converged/settled -after the restart rather than continuously re-cataloguing. Not yet -confirmed whether this is a durable fix or the connectors will resume the -same pattern once they hit their next scheduled refresh; worth re-checking -`docker logs quickstart-egeria-main | grep JacquardDigitalProductLoom` after -it's been up for a few hours. +### ISSUE-82: `pyegeria/omvs/valid_metadata.py` sends the literal query string `typeName=None` whenever `type_name` is Python `None` — breaks every Type-Name-omitted (global) Valid Metadata Value, in 12 of 14 methods across `ValidMetadataManager` -**Candidate next step (not yet done):** narrow which specific connector(s) -are the dominant contributor — `JacquardDigitalProductLoom`'s single -73-minute cycle and `OpenAPICataloguer`'s continuous entity creation are -the two strongest leads — and either increase their refresh interval, -narrow `OpenAPICataloguer`'s crawl scope (it may be re-cataloguing -`localhost:9443`'s own REST surface on every refresh instead of once), or -disable/reconfigure them in the `egeria-quickstart` compose config if -they're not needed for this deployment's actual use case. This needs -Egeria-server-side (or deployment-config-side) investigation — nothing here -is fixable from pyegeria's side. +**Status:** fixed and live-verified 2026-08-28 — all 12 affected methods +patched to match the one method that already did this correctly (see "Fix +landed" below); released as pyegeria 6.1.7. Verified with a mocked HTTP +layer first (URL no longer contains the literal string `None`, correctly +omits `typeName` when unset, and still includes it when set), then +confirmed live: `egeria-workspaces-5f` upgraded quickstart-pyegeria-web's +container to 6.1.7 and re-ran all 5 originally-failing files +(`human-resource-management.md`, `health-and-safety.md`, +`biological-agents-and-gmo.md`, `dangerous-goods-transport.md`, +`diversity-equity-inclusion.md`) against a live `qs-view-server` — every +one now exits 0/SUCCESS, zero occurrences of the "not a valid metadata +value", "Missing unresolved reference", or +`PyegeriaNotFoundException`/404 signatures across all 5. Also confirmed +the `type_name` guard is present in the installed package, not just this +checkout. `egeria-workspaces`' own `BACKLOG.md` entry (`PY-25`) closed as +verified on their side. -**Update 2026-08-15, done (partial):** `quickstart-egeria-main` had been -restarted again since the note above (container uptime ~1h at check time, -vs. the earlier 8h+ uptime of sibling containers) — so this was actually a -fresh post-restart observation window, not a stale one. Findings: -- **`OpenAPICataloguer`'s behavior is better characterized as a one-time - cold-start catalog crawl that converges, not continuous unbounded - re-crawling** — the earlier framing was too pessimistic. Creation rate - measured in shrinking trailing windows since restart: 134 (50 min) → 86 - (40 min) → 48 (30 min) → 26 (20 min) → 8 (10 min) → 6 (5 min) new - `APIOperation` entities — a clear deceleration, not a steady/repeating - rate. 201 total `APIOperation`s exist in the full container log history; - 171 of them were created in just this last hour, meaning the connector - is doing its first post-restart crawl right now and tapering off, not - perpetually re-cataloguing the same ground. -- **`JacquardDigitalProductLoom` logged zero refresh-cycle activity in this - 1-hour post-restart window** — can't yet say whether its 73-minute cycle - is gone for good or just hasn't fired again yet (container too young to - rule out a recurrence on its next scheduled interval). -- **Server responsiveness confirmed fast right now**: 3 sequential - `get_metadata_element_by_guid` calls against `qs-view-server` completed - in 0.19s/0.03s/0.02s — no timeouts, nothing close to the earlier - 30-second `PyegeriaTimeoutException` symptom. -- Could not directly inspect Postgres checkpoint stats this pass (`psql` - isn't installed as a client tool inside the `egeria-shared-postgres` - container in a way this session could reach it) — server-response timing - above stands in as an indirect but consistent signal instead. +**Reported by another Claude session** (`egeria-workspaces-5f`, no local +`egeria-python` checkout) debugging a failed Dr.Egeria batch run. A +`## Setup Valid Metadata Value` command that registers a new +`domainIdentifier` value with `### Type Name` deliberately omitted (i.e. +"applies to all open metadata types") is never visible to the very next +command's `### Domain Identifier` field validation in the same file: -**Net assessment:** the acute symptom (timeouts, multi-hour bulk-processing -runs) is not currently reproducing, and the mechanism now looks more like -"cold-start crawl briefly saturates the server after every restart, then -settles" than "sustained, unbounded background load." Still genuinely -Egeria-server/deployment-config territory, not pyegeria-fixable — if this -pattern recurs and is disruptive on every restart, the original candidate -fixes (narrow `OpenAPICataloguer`'s crawl scope, stagger connector startup, -or increase refresh intervals) remain the right next step, just lower -urgency than originally assessed. +``` +ERROR | md_processing.v2.processors:615 - Validation failed: Value 'Human +Resource Management' is not a valid metadata value for 'Domain Identifier' +(Validated by Egeria) for Create Business Imperative +``` ---- +Confirmed deterministic, not a timing/cache race (re-ran the same file +twice, 20+ minutes apart, identical failure both times). Real-world repro: +`egeria-workspaces`' `compose-configs/egeria-quickstart/PyegeriaWebHandler/ +coco-workbooks/0. data-governance-program/` — exactly the files that +self-register a custom domain before using it +(`human-resource-management.md`, `health-and-safety.md`, +`biological-agents-and-gmo.md`, `dangerous-goods-transport.md`, +`diversity-equity-inclusion.md`) fail 100% reproducibly; the other 12 +files in the same batch, which only use Egeria-builtin/CocoComboArchive +domains, run clean. Downstream cascade: every `Create ` that references the new domain fails validation, so every later +`Link Governance Response/Mechanism/Drivers/Policies` in the same file then +fails with `Missing unresolved reference GUID(s)` (~20+ occurrences per +file), and `Add Member to Collection` 404s (`OMAG-REPOSITORY-HANDLER-404-007`) +trying to add a member that was never created. -### ISSUE-54: `findMetadataElements` scoped to `Referenceable` silently returns an incomplete, arbitrary subset instead of the true population +**Root cause — confirmed in source, this repo's own code.** Every method on +`ValidMetadataManager` (`pyegeria/omvs/valid_metadata.py`) takes an optional +`type_name`, documented as: *"If the typeName is null, this valid value +applies to properties of this name from all types."* Exactly one method +honored that — `_async_get_valid_metadata_values` — with +`if type_name: url += f"&typeName={type_name}"`. Every other method instead +hardcoded the query param unconditionally into an f-string, e.g. +`_async_validate_metadata_value`: -**Status:** fixed 2026-08-30 (Egeria server — `postgres-repository-connector`, -`QueryBuilder.java`, commit `0d8d079c50`). The root cause is the one this entry -worked its way to: `getSequencingOrder()` emitted an `ORDER BY` on a single -non-unique column — creation time, update time, or a property value — and -`getPaging()` appended `limit`/`offset` to it. Each page is a separate execution of -that query rather than a server-side cursor, so rows tying on the sort column could -be ordered differently by each execution: an element could move between offset -windows from one page fetch to the next and be returned twice, or skipped, while the -traversal still terminated normally and reported nothing wrong. This is the -`OFFSET`/`LIMIT`-without-a-unique-`ORDER BY` anti-pattern the 2026-08-18 analysis -predicted, confirmed in the source rather than inferred from behaviour. +```python +url = ( + f".../validate-value/{property_name}?typeName={type_name}&actualValue={actual_value}" +) +``` -The `ORDER BY` now always ends with the principle table's primary key, making it a -deterministic total order. The primary key is used rather than `instance_guid` alone -because it is what defines a unique row for the table being paged — -`(instance_guid, version)` for entity and relationship, and -`(instance_guid, classification_name, version)` for classification, where one GUID -legitimately has a row per classification. +When `type_name` is Python `None`, f-string interpolation renders it as the +four-character string `"None"` — so the request sent is literally +`?typeName=None&actualValue=Human Resource Management`, not "no type +filter." Egeria correctly (from its point of view) doesn't recognize a +type named `None` and treats the value as not found/not valid, for **both** +the initial registration and every later lookup — so this isn't a filter +that merely excludes globally-scoped values, it's the *setup* call itself +that never actually registers a true global (`typeName IS NULL`) entry +either; it registers under (or is rejected against) the bogus literal type +`"None"`. That fully explains the reporter's own top hypothesis ("is the +later validation scoped by a Type Name filter that a Type-Name-omitted +registration doesn't satisfy?") — yes, effectively, though the mechanism is +a string-interpolation bug, not an intentional filter. -This also explains the two results that looked like separate problems. -`sequencingProperty=GUID` worked because it was the one branch already sorting on a -unique column — it is no longer a workaround anyone needs. And the residual ~4% -omission under `qualifiedName` sequencing, concentrated in `ValidMetadataValue` and -`PersonRole`, was the same defect in its other form: with the property null for -every element of a type, those elements all tie at the null position and the tie is -resolved differently per page. +Affected methods (12): `_async_setup_valid_metadata_value`, +`_async_setup_valid_metadata_map_name`, +`_async_setup_valid_metadata_map_value`, +`_async_validate_metadata_value`, `_async_validate_metadata_map_name`, +`_async_validate_metadata_map_value`, `_async_get_valid_metadata_value`, +`_async_get_valid_metadata_map_name`, `_async_clear_valid_metadata_value`, +`_async_clear_valid_metadata_map_name`, +`_async_clear_valid_metadata_map_value` (11 async methods; their sync +`loop.run_until_complete` twins inherit the fix for free since they just +delegate). `_async_get_valid_metadata_values` and +`_async_get_valid_metadata_map_value` (the latter passes `type_name` via a +`params=` dict, which httpx already drops when the value is `None`) were +already correct and untouched. -**Regression cover:** `PagingStabilityFVT` (in `open-metadata-test/open-metadata-fvt/ -query-fvt`) pages a set arranged so every sequencing is a complete tie — including a -property that is null for every element — and checks each traversal visits every -element exactly once. With the fix reverted it fails with "Element … was returned on -more than one page when paging with PROPERTY_ASCENDING on a property every element -shares a value for". The pre-existing `PagingFVT` could not see any of this: it asks -for `SequencingOrder.GUID`, a total order already. +This is exactly the flow `md_processing/v2/parsing.py`'s `Valid Value` +style hits for the `Domain Identifier` attribute (`property_name` = +`domainIdentifier`, `type_name` = `None` — the compact spec for `Domain +Identifier` has no `type_name` key at all) — so any deployment-added, +type-unscoped `domainIdentifier` (or any other property registered the +same way) was unreachable through this path regardless of caching, exactly +matching "not a race... possibly not ever." -**Still to confirm, and worth doing before this is treated as closed:** the fix was -proven on a 45-element tie set, not on the ~9,600-element `Referenceable` population -this entry measured. The numbers recorded here are consistent with tie instability -accounting for the whole gap, but that is inference — re-running the exhaustive -`Referenceable` scan against a build carrying `0d8d079c50` is what settles it. Note -also that the fix is in the **PostgreSQL** repository connector; the in-memory -connector has not been checked for the same defect. +**Fix landed:** every affected method now builds the URL without the query +param first, then conditionally appends it — the same pattern +`_async_get_valid_metadata_values` already used: -**Correction to this entry's own side notes:** the remark that -`metadataElementSubtypeNames` is "confirmed non-functional" is out of date — subtype -names are pushed down to the repository in current code -(`OpenMetadataStoreRESTServices` passes `getMetadataElementSubtypeNames()` through to -the handler). +```python +url = f".../validate-value/{property_name}?actualValue={actual_value}" +if type_name: + url += f"&typeName={type_name}" +``` -**Moved here from "Open Egeria Server issues" on the basis of the above — re-run the -exhaustive `Referenceable` scan against a build carrying `0d8d079c50` before treating -the ~9,600-element population number as settled.** +**Nothing left open here.** Fixed, released (pyegeria 6.1.7), and live-verified +end-to-end against the original real-world repro, not just unit-tested. -**Update 2026-08-26: re-confirmed again, unchanged.** Re-ran the exhaustive -`Referenceable` scan (unsequenced, `pageSize=500`, advancing unconditionally, -stopping only on an empty page — the correct pagination contract per the -note below) against the current server build (`6.2-SNAPSHOT`, platform -timestamp `2026-08-24T18:27:04Z` — the same redeploy that turned out to -have fixed ISSUE-69, checked in the same pass on the chance this one had -also moved). It hadn't: 9,344 raw elements fetched across 19 pages, only -**6,789 distinct GUIDs** (2,555 duplicates) against a native -`count_metadata_elements` population of **9,636** — ~70% coverage, in the -same range as every prior check (54-73% across past re-checks). Cross-check -against a direct exhaustive `GlossaryTerm` scan (262 real, distinct GUIDs): -only **80 (31%)** of them appear anywhere in the broad `Referenceable` -scan, **183 missing**. Same shape as originally documented — this is a -live, unresolved Egeria server bug, not something that rides along with -whatever fixed ISSUE-69. +--- -**Status:** still open (Egeria server) — **NOT fixed, re-confirmed -2026-08-18, and this pass corrects a misdiagnosis made earlier the same -day.** First re-check (prompted by dwolfson asking to double-check after -ISSUE-53/57/60) found `pageSize=500`/`startFrom=0` on a `Referenceable` -scan returning only 494 elements, and mistook that short page for the bug -itself (reasoning: "a naive `len(page) < page_size` pagination loop halts -here"). **dwolfson caught this and pointed at Egeria's own paging docs** -(https://egeria-project.org/guides/developer/finding-metadata/overview/#paging), -which explicitly document that a short-but-nonempty page does **not** mean -"last page" — server-side filtering can legitimately shorten an individual -page without ending the result set, and the only correct termination -signal is a genuinely **empty** page. `len(page) < page_size` is a -documented anti-pattern, not evidence of an Egeria bug — re-running the -scan correctly (always advance `startFrom` by the full `pageSize`, -regardless of how many came back, stop only on `not page`) confirms this: -`startFrom=500` (not `startFrom=494`) picks up cleanly with no gap. +### ISSUE-81: `migrate_question_specs.py` links Perspective↔Question via the wrong relationship type (`AssignmentScope`, not `ScopedBy`) — every bootstrap-migrated perspective silently relies on a filename-inference fallback instead -**The real remaining defect, once paginated correctly:** exhaustively -scanning `Referenceable` with the *correct* stop condition (23 pages, -`pageSize=500`, advancing by 500 unconditionally) fetched **10,548** -elements total against a true native-count population of **11,019** — but -those 10,548 contained **1,394 duplicate GUIDs**, leaving only **9,154 -distinct** elements, still **~17% short** of the true 11,019. `GlossaryTerm` -coverage in this corrected scan: **243/357 (68%)** — much better than the -27% the flawed short-page-as-stop-signal method found, but still -genuinely incomplete. So the original 2026-08-06 hypothesis (unstable -server-side result ordering across pages causing both duplication *and* -omission) remains the best-supported explanation — this is a real, -still-open Egeria server pagination-stability bug for this specific -broad-type query, just not the mechanism this session initially -misdiagnosed. Nothing fixable client-side. +**Status:** fixed 2026-08-28 — script corrected (see "Fix landed" below); +no data repair needed on the checked deployment (see "Checked the live +instance" below). Semantics settled against the canonical docs, not just +source-reading; see "Which relationship is actually correct" below. -**Checked dwolfson's hypothesis that the duplicates might just be multiple -versions of the same object (not a true dupe)** — ruled out definitively. -Re-ran the corrected exhaustive scan and compared `versions.version`/ -`versions.updateTime` across every occurrence of every duplicated GUID -(1,112 duplicated GUIDs found this run — close to, not identical to, the -1,394 above; population drifts slightly between runs since this is a live -shared server). **Every single occurrence of every duplicate has the -identical version number and `updateTime`** — 0 of 1,112 duplicated GUIDs -showed two different versions of the same element; all 1,112 were the -exact same version appearing 2-3 times (`{2: 1092, 3: 20}` occurrence-count -distribution). So this is not "the repository returned two historical -versions of one element and we're not deduplicating" — it's the literal -same element-state appearing more than once in the raw paginated response, -which is consistent only with unstable result ordering shifting an -element's position between page fetches (the same mechanism suspected -since the original 2026-08-06 finding), not a versioning artifact. -Duplicated types skew heavily toward `ValidMetadataValue` (659 of 1,112) -and `SpecificationPropertyValue` (300) — both likely large, frequently- -reordered collections, consistent with an ordering-instability theory. +**A real bug**, found 2026-08-28 answering "is the folder-name-inferred +perspective fallback in `load_egeria_report_specs()` actually used +anywhere?" Short answer: yes — for every question migrated through the +Phase 1 bootstrap, it's the *only* thing that has ever worked. -**Follow-up, same day** — dwolfson asked the natural next question: how can -a duplicate GUID even happen if the repository enforces uniqueness? Ruled -out every "this is actually fine" explanation with direct evidence rather -than just asserting server-side instability: -- **Not a live-mutating dataset causing drift mid-scan.** Native - `count_metadata_elements` for `Referenceable` stayed exactly stable - (11,079 → 11,079) across a full 7.4-second, 23-page scan. Page 1 - (`startFrom=0`) came back byte-for-byte identical before and after the - entire scan. Two immediate back-to-back identical requests also returned - the exact same elements in the exact same order — no evidence of - anything being written to this dataset during or around the scan. -- **Not a simple page-boundary tie-break either** — checked how far apart - (in page-index terms) each duplicated GUID's two occurrences were: only - 245/1,619 (15%) were adjacent pages; 1,374 (85%) spanned 2+ pages apart, - with a max observed gap of **10 pages** (~5,000 elements apart). An - element legitimately can't shift ~5,000 positions between two `startFrom` - calls unless the server's ordering for this query isn't a true - deterministic total order at all. -- **Best-supported mechanism:** `findMetadataElements` on a broad - `Referenceable` scan most likely sorts by something like creation time - with no unique tiebreaker (e.g. GUID) appended to the effective - `ORDER BY`. `ValidMetadataValue`/`SpecificationPropertyValue` (the two - most-duplicated types, 659 + 300 of 1,112) are both bulk-loaded from an - archive, so plausibly thousands of rows share the *exact same* creation - timestamp. Each `startFrom`/`pageSize` call is a fresh query execution, - not a stable server-side cursor — when many rows tie on the sort key, the - database is free to resolve that tie differently on separate query - executions, so the identical single row can legitimately land in very - different offset windows across two calls. This is the well-known - `OFFSET`/`LIMIT`-without-a-fully-unique-`ORDER BY` SQL pagination - anti-pattern, surfacing here in Egeria's repository query — not a - pyegeria bug, not a real repository-level duplicate GUID. +**Root cause.** `commands/migrate_question_specs.py`'s `_ensure_scoped_by()` +(despite its name) calls `client.actor_manager.link_assignment_scope(...)` +with `properties: {"class": "AssignmentScopeProperties"}` — that creates +Egeria's **`AssignmentScope`** relationship (`e3fdafe3-...`, "Links a +profile, role, or project to the elements that they are responsible for +managing"). But `load_egeria_report_specs()` reads perspectives back with +`client.get_related_elements(question_guid, relationship_type="ScopedBy")` +— a **different** relationship type (`3845b5cc-...`, "Link between a scope +... and an element restricted the scope"). Confirmed both are genuinely +distinct types, not aliases of one another, via Egeria's own type registry +(`OpenMetadataType.java`: `ASSIGNMENT_SCOPE_RELATIONSHIP` vs +`SCOPED_BY_RELATIONSHIP`, different GUIDs, different Properties classes) +— they just happen to share a doc page (0120-Assignment-Scopes) because +they're related concepts. -**Theory tested directly, same day** — dwolfson's suggestion: force a -deterministic sort by a genuinely unique property and see whether -duplicates disappear. They did, confirming the theory outright, and the -test surfaced a second, related defect in the process: +Since the two never match, `get_related_elements(..., "ScopedBy")` returns +empty for every question the bootstrap script touched, and +`load_egeria_report_specs()` falls through to its documented fallback: +inferring the perspective from the QuestionSpec folder's own +qualified-name suffix (`QuestionSpec::TypeDef::Developer` → `Developer`). +That fallback is a real, reachable code path with no bug in it — but see +"Checked the live instance" below before assuming it's actually load-bearing +for any specific deployment's data. -| Sort | Duplicates | Completeness (distinct / true count) | -|---|---|---| -| None (baseline, this run) | 853 | 9,752 / 10,605 (92%) | -| `sequencingProperty=GUID` | **1** (plausibly one live edit mid-scan, not a leftover tie) | not measured — session blocked mid-check by an unrelated concurrent edit breaking `pyegeria` imports (`pyegeria/omvs/feedback_manager.py`, not touched by this investigation) | -| `sequencingProperty=qualifiedName` | **0** | 10,597 / 11,081 (95.6%) — still short | +**The Dr.Egeria-authored path is correct**, for contrast: +`Link Perspective to Question` (`md_processing/v2/actor_manager.py`, +`object_type == "Perspective to Question"`) calls +`classification_manager._async_add_scope_to_element` — the real `ScopedBy` +endpoint. Anything linked that way going forward works via the intended +relationship; only the one-time bulk migration is affected. -Sorting by `qualifiedName` fully eliminates duplication (confirms the -theory) but the resulting scan is still incomplete — and *not* randomly: -every element of the omission is concentrated in exactly two types, -`ValidMetadataValue` (211 missing of 2,296) and `PersonRole` (63 missing of -334), while every other type in the scan matched its native count exactly -(`SpecificationPropertyValue`, `GovernanceActionProcessStep`, -`GovernanceActionProcess`, `GlossaryTerm`, `NotificationType`, -`SolutionComponent` all diff=0). Root cause of the omission: **every -single element of both `ValidMetadataValue` and `PersonRole` has -`qualifiedName = None`** — confirmed directly (2,296/2,296 and 323/323 -have a null `qualifiedName`, not just "some"). Sorting by `qualifiedName` -ties every element of these two types together into one giant unresolved -group at the null position — the *same* missing-tiebreaker defect as the -duplication case, just manifesting as omission instead of duplication when -the caller's own chosen `sequencingProperty` happens to be null for an -entire type. This means `GUID` is the objectively safer workaround for -anyone needing a reliable exhaustive scan today — it's the one property -guaranteed non-null and unique for every element regardless of type, so it -shouldn't hit either failure mode (consistent with its near-zero-duplicate -result above; completeness under `GUID` sequencing not yet confirmed, see -blocker note in the table). +**Which relationship is actually correct** (not a majority vote between +the three call sites — checked against the canonical type definitions, +https://egeria-project.org/types/1/0120-Assignment-Scopes): -**Also found as a byproduct of this correction:** pyegeria's own -`pyegeria/view/base_report_formats.py` (`load_egeria_report_specs()`) had -exactly the `len(page) < page_size` anti-pattern in a real fetch-all loop -(`find_collections` for `ReportType` collections) — fixed 2026-08-18 to -advance unconditionally and stop only on an empty page, per the now-clear -Egeria paging contract. Worth grepping the rest of this codebase (and any -sibling repo — `egeria-workspaces-fs`, `egeria-advisor`/`trellis`) for the -same pattern (`len(page) <` / `< page_size` / `< _page_size`); not -exhaustively audited beyond this one hit in this pass. +- `AssignmentScope`: *"identifies individuals, teams, projects, or other + actors **assigned to manage** resources represented by a linked + element"* — carries `assignmentScope` values `Contributor`, + `Administrator`, `Leader`, `Owner`, `Observer`, `Sponsor`, `Other`. An + operational-responsibility relationship: who administers/owns a + resource. +- `ScopedBy`: *"connects an element to organizations, projects, teams, and + similar entities that **define the impact scope or applicability + boundaries** of that element."* -**Status:** still open (Egeria server), re-confirmed 2026-08-15 against the -current, restarted `qs-view-server` — magnitude has shifted (as with -ISSUE-38/52's re-checks) but the core defect persists. Re-ran the same two -independent cross-checks with a fully-paginated exhaustive scan -(`page_size=200`, following pagination to exhaustion): -- Direct exhaustive `metadataElementTypeName="GlossaryTerm"`: **532** - elements. Of the exhaustive `Referenceable` scan's 19,166 elements - (19,127 distinct GUIDs — see below), only **324** carry `typeName: - "GlossaryTerm"` — **61%** coverage (better than the original run's ~54%, - still clearly incomplete, not just a rounding gap). -- Independent check against `SemanticAssignment` relationship participants - (`ClassificationExplorer.get_relationships`, unrelated API path): 414 - distinct participant GUIDs; only **302 (73%)** appear anywhere in the - 19,166-element `Referenceable` scan (up from the original run's 23/410 = - 5.6%, but still missing more than a quarter of real, independently-known - participants). -- **New observation this pass:** the exhaustive `Referenceable` scan itself - returned 19,166 total elements across pages but only 19,127 **distinct** - GUIDs — 39 duplicate entries showing up more than once across different - pages of the same paginated scan. Not previously noted; suggests server-side - result ordering/stability issues for this specific broad-type scan, which - would also explain why elements can be silently skipped between pages - (an element shifting position between page fetches, due to unstable - ordering, could cause both duplication *and* omission depending on which - way it moves) — a plausible mechanism for the core bug, not confirmed as - the actual root cause. +A Question isn't "managed" by a Perspective the way an asset is +administered by an owner — none of `AssignmentScope`'s predefined values +(`Contributor`/`Administrator`/`Owner`/...) make sense applied to a +Question. It's *relevant within* the boundary a Perspective defines, +which is exactly `ScopedBy`'s stated purpose. Direction confirms it too: +the (correct) Dr.Egeria command passes Perspective as the scope +(`scoped_by_guid`) and Question as the bounded element (`element_guid`), +matching "connects an element to ... entities that define the ... scope." +**`ScopedBy` is correct; `AssignmentScope` is the bug.** -Both cross-checks still clearly demonstrate the defect; nothing changed -client-side to warrant re-testing pyegeria's own pagination logic (already -ruled out in the original investigation — direct exhaustive scans of real -types are complete and correct, only the broad-base-type scan is affected). +**Checked the live instance — no repair needed there.** Queried all 84 +Question elements (`get_related_elements(guid, relationship_type=X)` for +both `ScopedBy` and `AssignmentScope`, every one): **all 84 already carry +a real `ScopedBy` relationship; zero carry `AssignmentScope`.** So this +deployment's Question data was bootstrapped via the correct path (the +Dr.Egeria markdown/`run_all.sh` route through `Link Perspective to +Question`, which was never buggy) rather than by running +`migrate_question_specs.py` directly — the folder-name-inference fallback, +while real code with no bug in it, turns out not to be load-bearing for +*this* deployment's data after all. The original "sole source of every +perspective" claim above was inferred from reading the script, not +verified against live data — correcting that here rather than leaving an +overclaim standing. -**Original status:** open (Egeria server), found 2026-08-06 fixing egeria-workspaces-fs's -relationship-only search (see ISSUE-45's same investigation thread — -looking for a safe fallback type once `metadataElementTypeName="Asset"` -was confirmed wrong for `SemanticAssignment`, and `metadataElementSubtypeNames` -confirmed non-functional). +**Impact:** the code bug is still real — `migrate_question_specs.py` +would create wrong-typed data on any instance it's actually run against, +and any such data would be silently masked by the inference fallback +exactly as first described. `AssignmentScope` relationships would be real, +queryable, wrong-typed data in Egeria — anyone querying by `ScopedBy` +directly (bypassing the fallback), or any future code trusting +`AssignmentScope` to mean "this profile/role manages this question" (its +actual documented meaning), would get nonsense. Whether any other +deployment actually has this data is unknown — check before assuming +either way. -**Layer:** Egeria Server — not fixable in pyegeria. +**Fix landed:** `migrate_question_specs.py`'s `_ensure_scoped_by()` now +calls `classification_manager.add_scope_to_element` (matching the +Dr.Egeria command) instead of `actor_manager.link_assignment_scope`, so +the properties class actually sent is `ScopedByProperties`, not +`AssignmentScopeProperties`. -**What:** an exhaustive, fully-paginated `find_metadata_elements` scoped to -`metadataElementTypeName="Referenceable"` (**not** the true type-hierarchy -root — that's `OpenMetadataRoot`, confirmed live via `get_all_entity_defs`; -`Referenceable` is one of five direct subtypes of `OpenMetadataRoot` -alongside `SearchKeyword`/`Rating`/`Like`/`TranslationDetail`, but it's the -practical common ancestor of essentially every entity type an -application-level search actually cares about, which is why it's the type -most callers reach for as a "find everything" scope) returns a small, -arbitrary subset instead of the true population, with no error, no truncation flag, -and pagination genuinely terminating normally (`added == 0`/`len(page) < -page_size` on the last page — the loop believes it's done). Confirmed live -against `qs-view-server` by direct comparison: +**No data repair needed** on this deployment (see "Checked the live +instance" above) — nothing here to detach/recreate. If another deployment +did run the buggy script directly, the same live-check +(`get_related_elements(question_guid, relationship_type="AssignmentScope")` +across every Question) will show whether it needs the same repair; write +a small one-off script at that point rather than building one speculatively +now for data that may not exist anywhere. -| Scope | Elements found | -|---|---| -| `metadataElementTypeName="Referenceable"` (exhaustive) | 3,999 total | -| ...of which `typeName="GlossaryTerm"` | 241 | -| ...of which `typeName="GovernanceActionProcess"` | 22 | -| `metadataElementTypeName="GlossaryTerm"` (exhaustive, direct) | **450** | -| `metadataElementTypeName="GovernanceActionProcess"` (exhaustive, direct) | **378** | +--- -Cross-checked against a real, independently-known population: fetching -every participant GUID of the `SemanticAssignment` relationship type -(`ClassificationExplorer.get_relationships`, a separate, unrelated API -path, previously verified complete) gives 410 distinct GUIDs. Only 23 of -those 410 appear anywhere in the 3,999-element `Referenceable` scan — 387 -real participants (94%) are simply absent from a scan of the type that is -supposed to be their common ancestor and therefore cover all of them. +### ISSUE-80: `find_report_specs_by_perspective`/`find_report_specs_by_question` are implemented and tested but not exposed anywhere -**Impact:** `metadataElementTypeName="Referenceable"` cannot be used as a -"safe, unscoped, find-everything" fallback the way its practical role as -the common ancestor of nearly every real entity type implies — a caller -that scopes a search this broadly on purpose (not just as an accidental fallback) will silently miss -the majority of real results, not just cap them at a boundary. Confirmed -this is specific to the base-type-wide scan, not pagination itself — -directly-typed exhaustive searches for the exact same real types -(`GlossaryTerm`, `GovernanceActionProcess`) are complete and correct. +**Status:** fixed 2026-09-05 (Pyegeria — `pyegeria/core/mcp_adapter.py`, +`pyegeria/core/mcp_server.py`). Added `run_find_report_specs_by_perspective`/ +`run_find_report_specs_by_question` adapters (same thin-wrapper pattern as +the existing `run_find_report_specs`) and registered them as MCP tools +`find_report_specs_by_perspective`/`find_report_specs_by_question`, +alongside the existing `find_report_specs` tool. No change to the +underlying functions themselves — pure additive wiring, per the candidate +fix below. `hey_egeria` CLI exposure (the "and/or" half of the original +ask) left undone — MCP was the natural fit given the existing +`find_report_specs` tool precedent; a CLI subcommand is separate follow-up +work if wanted. Full `pytest tests/micro-tests/` green (exit 0); the +underlying functions already have their own functional-test coverage +(`tests/functional-tests/test_question_specs.py`, live-server only), not +re-run here. -**Candidate fix:** none client-side. `egeria-workspaces-fs`'s workaround -(see its `EGERIA_INSIGHTS_QUERY_MODEL.md`/`insights_handler.py`, commit -`015916d0`) is to never scope a search to `Referenceable` (or any other -broad base type) as a "safe fallback" — when the real target type isn't -known, derive the actual candidate types from other data (e.g. a -relationship's real participants) and search each directly instead. +**Not a bug — a follow-up.** Found 2026-08-28 while reviewing the Question +Spec migration (`docs/design/report_spec_migration_design.md`, Phase 1 +complete since 2026-05-18/`6c946af`). Both functions +(`pyegeria/view/base_report_formats.py:3678`/`3715`) are real, working +query methods over the Egeria-sourced `question_spec` data that +`load_egeria_report_specs()` merges into the runtime registry — not +leftover pre-migration code, and not candidates for removal (see the +design doc discussion this follow-up came out of). ---- +**The gap:** neither is exported from `pyegeria/__init__.py`, registered +as an MCP tool (`pyegeria/core/mcp_server.py`), or wired into any +`hey_egeria`/`commands/` CLI surface. The only thing that currently +exercises them is their own functional test, +`tests/functional-tests/test_question_specs.py`. Someone wanting "which +report specs does the Data Steward perspective care about" or "which +report answers this question" today has to import +`pyegeria.view.base_report_formats` directly — there's no MCP tool or CLI +command for either. -### ISSUE-41 (PY-21): `find_glossary_terms(sequencing_order=..., include_only_classified_elements=...)` returns ZERO results when combined — each filter alone works fine +**Candidate fix, not started:** expose both as MCP tools (natural fit +alongside the existing `find_report_specs` tool the MCP server already +wires at startup via `load_egeria_report_specs`) and/or as a +`hey_egeria cat show` subcommand. Low risk — additive, no changes needed +to the functions themselves. -**Status:** fixed 2026-08-30 (Egeria server — `omf-metadata-server`, -`OpenMetadataStoreRESTServices.java`, commit `0d8d079c50`). The cause turned out not -to be `sequencing_order` at all, which is why this entry's trigger condition kept -shifting under re-testing. +--- -`includeOnlyClassifiedElements`/`skipClassifiedElements` are carried on every search -request body, but on the by-search-string path they were never expressed as part of -the query. The repository chose the page and the filter was applied afterwards to -whatever came back — so `find_glossary_terms` was handed whatever fraction of the -first page happened to carry the classification. Adding `sequencing_order` changed -which 200 elements landed on page 1, and so changed how many survived the filter; -zero was an ordinary outcome, not a special one. The 2026-08-27 "regression" — the -classification filter alone returning 0, with no sequencing involved — was the same -cause with more data in the repository, not a strictly worse second defect. The same -reasoning explains why `["Template"]` also returned 0 while an unfiltered call -returned a full page, and why `get_term_by_guid` on a "missing" term correctly showed -the classification: the data was always intact and always findable; it just was not -on the page the repository had already chosen. +### ISSUE-78: engine-host participation trio is implemented on `main` but absent from the released 6.0.18.4 -The filters are now pushed into the query, merged with the anchor conditions where a -search already has them. Only one of the two can be pushed down at a time, because a -classification search carries a single match criteria and "must have these" cannot be -combined with "must not have those" — the required classifications win and the -forbidden ones fall through to the existing post-retrieval check. +> **RESOLVED by pyegeria 6.1.5** (verified 2026-08-27). All three methods ship in +> the wheel. Resource Explorer upgraded from 6.0.18.4 → 6.1.5 and its full suite +> (2613 tests), live Egeria smoke tests (20) and live alignment scan all pass, so +> the minor bump is clean for RE's usage. `claim_engine_action` was exercised +> against the live platform through the client: claiming an action already +> `IN_PROGRESS` under `EgeriaWatchdog` raises `PyegeriaUnauthorizedException` +> carrying `OMAG-GENERIC-HANDLERS-403-003` — a typed, catchable exception, which +> is what an engine-host loop needs to tell "another host got there first" from a +> real failure. Entry kept for history; move to Fixed / Resolved when convenient. -**Regression cover:** `ClassificationFilterPushdownFVT` (in -`open-metadata-test/open-metadata-fvt/query-fvt`) puts more unclassified elements -than fit on a page ahead of the classified ones and checks that a single request -returns the classified elements. With the fix reverted it fails with -`expected: <[…6 GUIDs…]> but was: <[]>` — this entry's symptom exactly. +**Not a bug — a release ask.** `claim_engine_action`, +`update_engine_action_status` and `get_active_claimed_engine_actions` exist +in this repo's `main` (commit `8362b1c`, "implement remaining +confirmed-missing methods") and are **not** in the pyegeria 6.0.18.4 that +Resource Explorer resolves from PyPI. Nothing needs writing; the methods +need to ship. -**pyegeria follow-on:** `perspectives_handler.py`'s `get_questions()` can drop the -2026-08-27 workaround (fetch unfiltered at `page_size=1000`, filter client-side on -`otherClassifications`) and go back to a server-side -`include_only_classified_elements` call. Two things to preserve when doing so: +Measured 2026-08-26 against the installed package and this checkout: -- **Page until a null, not until an empty list.** Empty pages still occur — from - `skipClassifiedElements` where it cannot be pushed down, from the visibility - filtering every search applies, and from the client-side subtype/classification - checks. A single call that stops on an empty list can still under-report. This is - Egeria's documented paging contract and is unchanged by the fix. -- **`findMetadataElements`/`countMetadataElements` are not covered.** Those request - bodies still ignore `includeOnlyClassifiedElements` — they take the OMF-framework - `SearchClassifications` rather than the OMRS one, so merging there needs a second - typed variant that was left out of this change. Callers that reach the repository - through those two endpoints still get no server-side classification filtering. +| method | 6.0.18.4 | `main` | +|---|---|---| +| `initiate_engine_action` | ✓ | ✓ | +| `get_active_engine_actions` | ✓ | ✓ | +| `get_engine_actions` / `_by_name` / `find_engine_actions` | ✓ | ✓ | +| `cancel_engine_action` | ✓ | ✓ | +| **`claim_engine_action`** | ✗ | ✓ | +| **`update_engine_action_status`** | ✗ | ✓ | +| **`get_active_claimed_engine_actions`** | ✗ | ✓ | -**Moved here from "Open Egeria Server issues" on the basis of the above.** +**Why these three specifically.** They are the whole participation loop for a +non-Java engine host: find work claimed for my engine → claim it → report +progress. RE is being modelled as an additional engine host +(`packages/resource-explorer/docs/survey-model-and-engine-host-design.md` +§4), and without `claim` there is no arbitration between hosts, while +without `update_engine_action_status` a host could take work and never +report completion — worse than not participating at all. -**Update 2026-08-27: regression — `include_only_classified_elements` alone -now also returns ZERO, no `sequencing_order` involved at all.** Found -debugging Egeria Explorer's Questions tab showing no questions on the -left (`egeria-workspaces-fs`'s `perspectives_handler.py`'s `get_questions()`, -which had already dropped `sequencing_order` per this issue's earlier -guidance). Reproduced directly against `qs-view-server`: -`find_glossary_terms(search_string="*", starts_with=True, graph_query_depth=0, -include_only_classified_elements=["Question"], page_size=200)` → **0** hits, -where the **2026-08-26** update immediately below recorded this exact call -(classification filter alone) returning **43** hits one day earlier. Ruled -out a `"Question"`-specific cause: swapped in `include_only_classified_elements= -["Template"]` (a long-established classification, unrelated to anything -created recently) → also **0**. An unfiltered call on the same connection -(`find_glossary_terms(search_string="*", starts_with=True, graph_query_depth=0, -page_size=200)`, no classification filter at all) → **200** hits (page-size -ceiling), so the connection/server itself is healthy and the classification -data is intact (confirmed via direct `get_term_by_guid` on one of the -"missing" terms — its `elementHeader.otherClassifications` correctly carries -`classificationName: "Question"`). So as of this server instance today, -`include_only_classified_elements` appears to return zero regardless of -`sequencing_order` — a strictly worse regression than this issue's original -scope. Workaround applied in `perspectives_handler.py`: drop the server-side -classification filter entirely, fetch unfiltered (`page_size=1000`, the -server's own max), and filter client-side on `otherClassifications`. -Worth re-testing the classification-filter-alone case again on a later -server build, the way this issue's own history already does for the -combined-filter case. +**The server side is confirmed working**, so this really is only a client +packaging gap. Tested 2026-08-26 by POSTing +`/engine-actions/{guid}/claim` directly for an action already `IN_PROGRESS` +under the running `EgeriaWatchdog`: -**Update 2026-08-26: re-confirmed again, unchanged.** Re-ran the exact -repro against the current server build (`6.2-SNAPSHOT`, platform -timestamp `2026-08-24T18:27:04Z` — the same build that turned out to have -resolved ISSUE-69, checked in the same pass on the chance this one had -also moved). It hadn't: `include_only_classified_elements=["Question"]` -alone returns 43 hits (count has drifted from the 33 in earlier checks — -unrelated to the bug, just live data), `sequencing_order= -"PROPERTY_ASCENDING"` alone returns 200 (page-size ceiling), and combining -both still returns exactly **0**. Same symptom as every prior check. +``` +OMAG-GENERIC-HANDLERS-403-003 Engine Host OMAG Server with a userId of +erinoverview is not allowed claim the engine action ... because it is +already claimed +systemAction: The system cannot claim an engine action because another +Engine Host OMAG Server has got there first. +``` -**Status:** double-checked again 2026-08-18 (per dwolfson's explicit -request) — core defect persists, and this pass rules out a client-side -cause definitively by capturing the actual outgoing request body via a -traced `_async_make_request`: `include_only_classified_elements=["Question"]` -alone sends a well-formed body (`includeOnlyClassifiedElements: ["Question"]`, -no `sequencingOrder` key) and returns **2 hits** (data has changed since the -original 33 — unrelated to the bug). Adding *only* `sequencing_order= -"PROPERTY_ASCENDING"` to that exact same body (confirmed via the trace — -every other field byte-identical) drops the result to **0**. Also tried -adding `sequencing_property="displayName"` on top — still 0. Since ISSUE-60 -(sequencing order itself) is now fixed server-side, this entry's trigger -condition may have shifted from what it was originally — worth a fresh -root-cause dig rather than assuming the original mechanism still applies -unchanged — but the *symptom* (combining `sequencing_order` with a -classification filter zeroes the result) is unambiguously still present, -confirmed via real request/response, not just method-level return values. +First-claim-wins, server-enforced, and the refused claim left the other +host's action untouched. That is also what makes a polling client safe — +every host may see an action and only one can take it — so **no Kafka +consumer is being asked for**. Delivery is a latency optimisation; +correctness rests on `claim`. -**Status:** confirmed bug (Egeria server) — found 2026-07-28 debugging -Egeria Explorer's Perspectives page showing Perspectives but no Questions. -Related to ISSUE-40 below (same broken parameter, different — and more -severe — failure mode: not just wrong order, but zero rows). Consolidated -in from `egeria-workspaces-fs/PYEGERIA_ISSUES.md` 2026-08-05. +**Blocks:** RE engine-host participation (design note §4.1, §5 step 8). +Nothing else in RE is waiting on it. -**How to trigger** (`GlossaryManager.find_glossary_terms`, qs-view-server, -33 `GlossaryTerm`s classified `Question`): -```python -# classification filter alone: 33 hits -mgr.find_glossary_terms(search_string="*", starts_with=True, output_format="JSON", - page_size=200, graph_query_depth=0, - include_only_classified_elements=["Question"]) +--- -# sequencing_order alone (no classification filter): 200 hits (unrelated terms, page_size ceiling) -mgr.find_glossary_terms(search_string="*", starts_with=True, output_format="JSON", - page_size=200, graph_query_depth=0, - sequencing_order="PROPERTY_ASCENDING") +### ISSUE-77: 5 classification attributes on the shared `Referenceable` bundle, and `Anchor Scope IDs`/`Make Anchor` on the shared `Link Command Base` bundle, were parsed and validated but never applied -# BOTH together: 0 hits -mgr.find_glossary_terms(search_string="*", starts_with=True, output_format="JSON", - page_size=200, graph_query_depth=0, - sequencing_order="PROPERTY_ASCENDING", - include_only_classified_elements=["Question"]) -# -> [] (or a "No elements found" string, depending on call shape) -``` -Isolated further: `sequencing_order="PROPERTY_ASCENDING"` is the trigger — -`sequencing_property` alone (no `sequencing_order`) does **not** break it -(still 33 hits). It's specifically `sequencing_order` + a classification -filter. +**Status:** fixed 2026-08-26 (Dr.Egeria — `md_processing/v2/processors.py`, +`md_processing/v2/glossary.py`, `md_processing/md_processing_utils/common_md_utils.py`, +`md_processing/data/compact_commands/*.json`; tests — +`tests/micro-tests/test_referenceable_classification_sync.py`, +`tests/micro-tests/test_link_body_anchor_fields.py`). -**Expected:** the classification filter's 33 matches, sorted by the given -sequencing property (or, per ISSUE-40, at least returned in server-internal -order — but not silently emptied). +**Layer:** Dr.Egeria. Found by `scripts/dr_egeria_audit.py`'s DEAD_ATTRIBUTE +check (see the tool's own PR, #307) — its ~1162 raw hits collapsed to ~30 +distinct attribute names once bundle-inheritance duplication is accounted +for; this entry covers the 7 dominant ones, dwolfson's explicit priority. -**Actual:** zero rows, with no error — the query silently looks like -"nothing matches" rather than failing loudly, which is what made this hard -to spot (the egeria-workspaces-fs `/api/questions` endpoint returned -`{"total": 0}` with a 200 status; only comparing against a live count of -Question-classified terms in Egeria surfaced that this was wrong, not just -an empty demo). +**`Security Tags` / `Policy Management Point`** — both real, generic +classifications (`SecurityTagsProperties`/`PolicyManagementPointProperties`, +confirmed against `Egeria-api-classification-explorer.http`/ +`Egeria-api-governance-officer.http` — both endpoints are +`elements/{guid}/...`, valid for any `Referenceable`) already had a fully +working standalone `Classify X as Security Tags`/`Classify X as Policy +Management Point` command (`CurationClassifyProcessor`, +`CLASSIFICATION_METHODS` in `md_processing/v2/curation.py`) — but the +*embedded* copy of these same attributes on the `Referenceable` bundle +(inherited by nearly every `Create` command) was never synced after +create/update, unlike its 5 siblings +(`Confidence`/`Confidentiality`/`Criticality`/`Impact`/`Retention +Classification`, handled by the pre-existing `_sync_governance_classifications`). +Fixed by adding a new `AsyncBaseCommandProcessor._sync_referenceable_classifications()`, +called alongside `_sync_governance_classifications` for every Create/Update, +reusing the same `CLASSIFICATION_METHODS`/`CURATION_CLASSIFICATION_CLIENTS` +spec the standalone commands already use (deferred import from +`curation.py` to avoid the circular import — `curation.py` imports +`AsyncBaseCommandProcessor` from `processors.py`). -**Impact / workaround:** egeria-workspaces-fs's `perspectives_handler.py`'s -`get_questions()` used exactly this broken combination. Fixed by dropping -`sequencing_order`/`sequencing_property` from the call — the endpoint -already sorts client-side, so the server-side sequencing was redundant -even before this bug was found. No other known callers currently combine -`sequencing_order` with a classification filter, but worth checking -`include_only_classified_elements`/`matchClassifications` callers -generally if new zero-result reports show up elsewhere. +**`Class Word` / `Modifier` / `Prime Word` Classification — a different, +more serious shape of the same bug, since simply reusing the pattern above +would have introduced a *new* live failure.** Their real endpoints +(`_async_set_is_class_word` etc., `pyegeria/omvs/glossary_manager.py`) are +`/glossaries/terms/{term_guid}/is-class-word` — **GlossaryTerm-only**, not +generic — confirmed against the actual URL. The embedded attributes were +on the same widely-shared `Referenceable` bundle as `Security Tags` above, +inherited by ~100 unrelated Create commands (`Create Governance Action +Process`, `Create Digital Product`, ...). Syncing these generically for +every Create/Update guid would 400 against the term-only endpoint for +every non-term element. Per dwolfson's explicit direction: **removed from +the shared `Referenceable` bundle in all 11 families that had it** (only +`Create Glossary Term`/`Create Question` legitimately need them — both +inherit the term-specific `Glossary Term Base` bundle in +`commands_glossary_compact.json`, which now carries the 3 attributes +instead). Wired into a new `AsyncBaseCommandProcessor._sync_term_naming_classifications()`, +called only from `TermProcessor`/`QuestionProcessor` +(`md_processing/v2/glossary.py`) where the guid is known to be a real term +— never from the generic Create/Update sync path. ---- +(`commands_curation_compact.json`/`commands_report_compact.json` also +define a `Referenceable` bundle, but neither has any command that actually +uses it — confirmed dead/orphaned in both, pre-existing and unrelated to +this fix, left as-is.) -### ISSUE-69: Egeria's repository (server-side) doubles every apostrophe on persist (`what's` -> `what''s`) — corrupts displayName/qualifiedName, making affected elements unfindable by their real name +**`Anchor Scope IDs` / `Make Anchor`** — a naming trap: there are *two* +separate "anchor scope" attributes in the compact spec. `Anchor Scope ID` +(singular) lives on the `New-Element` bundle (Create commands) and was +already correctly wired to `anchorScopeGUID` on `NewElementRequestBody`. +`Anchor Scope IDs` (plural, `Reference Name List` style) is a *different* +attribute living on `Link Command Base` (relationship-establishing +commands) alongside `Make Anchor` — neither was ever included in +`set_rel_request_body()`'s outer `NewRelationshipRequestBody`, confirmed +missing against `Egeria-api-lineage-linker.http`, which shows both +`anchorScopeGUIDs` (plural list) and `makeAnchor` as real top-level fields +on that body. Fixed by adding both to `set_rel_request_body()`. -**Layer:** Egeria Server / repository connector — **not pyegeria, not -Dr.Egeria**. Originally logged (2026-08-19) as a Dr.Egeria write-path bug -with a theory pointing at a nonexistent "SQLite-backed spec/journal -store" in `md_processing/v2/` — re-investigated 2026-08-19 by tracing the -*actual outgoing HTTP request body* (not just Dr.Egeria's own echoed -attribute table) and the theory doesn't hold: there is no sqlite3 usage -anywhere in `md_processing/`/`pyegeria/` (`grep -rn "sqlite3"` — zero -hits), and the wire payload itself is provably clean. +**Deliberately not fixed here — flagged, not guessed at:** +- **`Request ID`** — parsed and validated everywhere via the shared + `Request Base` bundle, but no generic `requestId` field exists anywhere + in the real Egeria REST API (searched the entire `.http` ground truth: + one match, in an unrelated user-management endpoint). The only real + "request ID" concept in this codebase is `self.context["request_id"]`, + an internally auto-generated `uuid.uuid4()` in + `AsyncBaseCommandProcessor.__init__` that nothing else currently reads. + Per dwolfson (a rare/Advanced-level corner case): an explicit + `Request ID` value now overrides the auto-generated one for that + command's execution, added in `execute()` right after parsing — minimal, + since there's no real consumer to fully wire yet. -**Status: RESOLVED 2026-08-26 (server-side).** Re-ran the exact repro -(`_async_create_glossary_term` with an apostrophe in `displayName`, -`qualifiedName`, and `description`, immediate read-back by GUID) against -the currently-running server (`6.2-SNAPSHOT`, platform build timestamp -`2026-08-24T18:27:04Z` per `/api/about` — a newer build than whatever was -running on 2026-08-19). All three properties round-tripped byte-for-byte -correct, single apostrophes, no doubling. Ran twice independently, both -clean; test terms deleted after confirming. This looks like a server-side -fix landed via a platform redeploy between 2026-08-19 and now, not a -pyegeria/Dr.Egeria change (nothing in this repo touched the write path in -between). Moved here from "Open Egeria Server issues" on that basis — one -caveat worth keeping in mind: this is a SNAPSHOT build, not a pinned -release, so re-check if it ever resurfaces on a later build. +**Tests:** `test_referenceable_classification_sync.py` (6 tests) and +`test_link_body_anchor_fields.py` (2 tests), fake client, no live server +required. Full `pytest tests/micro-tests/` green throughout. -**Status (as originally logged, kept for history): was open, root cause was correctly located as follows.** +--- -**Re-traced live, instrumenting `_async_make_request` directly** (not -just Dr.Egeria's parser echo, which the original report already showed -was clean) to capture the exact JSON string sent over the wire for a -`Create Glossary Term` with `Description` = `What's the owner's policy?`: +### ISSUE-76: `TermRelationshipProcessor` offered obsolete/fictional term-relationship types, and swallowed the resulting server failure as a false "success" -``` -[POST] .../glossary-manager/glossaries/terms -PAYLOAD: ...,"description": "What's the owner's policy?","category": null... -``` +**Status:** fixed 2026-08-26 (Dr.Egeria — `md_processing/v2/glossary.py`, +`md_processing/data/compact_commands/commands_glossary_compact.json`; +tests — `tests/micro-tests/test_term_relationship_processor.py`). -Single apostrophes, correctly JSON-encoded, exactly as authored — **the -outgoing request from pyegeria/Dr.Egeria is clean.** Immediately reading -the same element back by GUID after creation: +**Layer:** Dr.Egeria (`TermRelationshipProcessor`, `TermAsContextProcessor`). -```python -term = await gm._async_get_term_by_guid(new_guid) -props["description"] # -> "What''s the owner''s policy?" -- DOUBLED -``` +**Reported by dwolfson** (user-forwarded field report): `dr_egeria --directive +process` on a real glossary file produced repeated `SERVER_ERROR_500`s — +`OMAG-COMMON-400-018 The type name TermISATYPEOFRelationship ... is not +recognized` (also seen for `TermHASARelationship`, `TermTYPEDBYRelationship`) +— and, worse, **the run's final summary reported success anyway** if left +to complete, with no visible sign anything had failed except scrollback the +user had to notice and read. -**Reproduced again with zero pyegeria/Python code in the loop at all** — -plain `curl`, using `Egeria-api-glossary-manager.http`'s own -`createGlossaryTerm`/`getGlossaryTermByGUID` requests as the template -(same URLs, same body shape), to rule out any possibility this was an -httpx/Pydantic serialization artifact rather than a genuine server round -trip: +**Bug 1 — obsolete/fictional type names.** `TermRelationshipProcessor`'s +`rel_mapping` mapped `HASA`/`HAS A` → `TermHASARelationship`, `TYPED BY` → +`TermTYPEDBYRelationship`, and `TYPE OF` → `TermISATYPEOFRelationship`. +None of these three target names has ever existed as a real Egeria +relationship type — confirmed against a live server's +`get_all_relationship_defs()` (only `Synonym`, `PreferredTerm`, `Antonym`, +`ReplacementTerm`, `RelatedTerm`, `ISARelationship` are real +`GlossaryTerm`<->`GlossaryTerm` relationship types) and against every +`open-metadata-types` archive version in odpi/egeria's own source history +(no hit for any of the three names, ever). The compact spec's +`Relationship Type` enum made this worse by also offering `Translation` +and `ValidValue` as selectable values — real Egeria type names exist under +similar spellings (`TranslationLink`, several `ValidValue*` relationships) +but none of them connects two `GlossaryTerm`s, so both would also have +failed the same way. -```bash -TOKEN=$(curl -sk -X POST https://localhost:9443/api/token \ - -H "Content-Type: application/json" \ - -d '{"userId":"erinoverview","password":"secret"}') +**Bug 2 — the false "success" (the more serious one).** +`TermRelationshipProcessor.apply_changes()` caught `PyegeriaException` +internally, logged an error, set `self.parsed_output['valid'] = False`, +and returned `self.command.raw_block` — which *looks* like a correct +error-handling path, but isn't: `AsyncBaseCommandProcessor.execute()` only +checks `parsed_output['valid']` in its pre-flight validation step (step 2), +which runs *before* `apply_changes()` (step 7) is ever called. By the time +`apply_changes()` discovers the failure, that check has already passed. +Catching the exception inside `apply_changes()` meant it never reached +`execute()`'s own `try/except PyegeriaException` (which does correctly set +`"status": "failure"`), so `execute()` fell through to its normal +success-path return unconditionally. Checked every other +`except PyegeriaException` in `md_processing/v2/*.py` for the same +pattern — this was the only one; everywhere else is either the dispatcher's +own correct top-level catch or a benign best-effort read (`fetch_element`/ +optional-fallback lookups) that doesn't guard a write operation. -curl -sk -X POST \ - https://localhost:9443/servers/qs-view-server/api/open-metadata/glossary-manager/glossaries/terms \ - -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{ - "class": "NewElementRequestBody", - "isOwnAnchor": true, - "properties": { - "class": "GlossaryTermProperties", - "displayName": "Native Curl Apostrophe Test QRS456", - "qualifiedName": "Coco Pharmaceuticals::Term::Native-Curl-Apostrophe-Test-QRS456::1.0", - "description": "What'\''s the owner'\''s policy?", - "contentStatus": "ACTIVE", - "versionIdentifier": "1.0" - }, - "parentAtEnd1": true - }' -# -> {"class":"GUIDResponse", ..., "guid":"494d35d8-..."} +**Fix:** +- `rel_mapping` now only maps `ISA`/`IS A` → `ISARelationship` (the one + real alias). A new `REAL_TERM_RELATIONSHIP_TYPES` set is checked before + any request is built; an unrecognized value raises a clear `ValueError` + naming the real options, instead of reaching the server and 400ing. +- The internal `try/except PyegeriaException` around the actual link/unlink + calls is removed entirely — the exception now propagates to `execute()`, + which correctly reports `"status": "failure"` in the batch summary. +- Compact spec's `Relationship Type` enum trimmed to the 6 real values + (`RelatedTerm`, `Synonym`, `Antonym`, `PreferredTerm`, `ReplacementTerm`, + `ISA`/`ISARelationship`) via the Spec Editor's REST API; `refresh_specs` + regenerated the affected templates. +- **Found while fixing this**: `Link Term as Context` (`UsedInContext`, a + *fixed* relationship type — not user-selectable) was incorrectly sharing + the same `Term-Term Link Base` bundle as `Link Term-Term Relationship`, + which meant it inherited a **required** `Relationship Type` attribute + that `TermAsContextProcessor.apply_changes()` never reads at all — users + were forced to supply a value for a field that does nothing. Gave it its + own `Term as Context Base` bundle containing only the attributes the + processor actually reads (`Confidence`/`Expression`/`Source`/`Steward`/ + `Term Relationship Status`, all confirmed genuinely used); `Term 1`/ + `Element Id` stay as the command's own `custom_attributes`, unchanged. -curl -sk -X POST \ - https://localhost:9443/servers/qs-view-server/api/open-metadata/glossary-manager/glossaries/terms/494d35d8-a939-47dc-9504-a984117d2b76 \ - -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"class":"GetRequestBody","forLineage":false,"forDuplicateProcessing":false}' -# -> "description": "What''s the owner''s policy?" -- DOUBLED, same as via pyegeria -``` +**Tests:** `test_term_relationship_processor.py` (5 tests, fake client, no +live server required) — a real type still links successfully, `ISA` maps +correctly, an obsolete type raises `ValueError` before ever reaching the +client, and a genuine `PyegeriaException` from the client propagates out +of `apply_changes()` rather than being swallowed. Full `pytest +tests/micro-tests/` green. -Identical result via raw `curl` as via pyegeria — this is conclusive: -the doubling happens inside Egeria itself, full stop, with no dependency -on any client library. (Both test terms cleaned up after confirming.) +--- -So the corruption happens strictly **between the server receiving a -correct single-apostrophe payload and what it persists/returns** — inside -Egeria itself (most likely the active repository connector's handling of -string properties, e.g. a SQL string-literal escape (`'` → `''`, the -standard SQL-92 way to escape a quote inside a literal) applied even -though the actual write should be going through a parameterized -query/placeholder that needs no manual escaping — but not confirmed to -that level of detail; flagging the mechanism as a plausible lead for -whoever picks this up server-side, not asserting it). +### ISSUE-75: `ExternalReferenceLink`/`MediaReference`/`CitedDocumentLink` — same create-time relationship GUID discarded, Detach had no way to target a specific instance (ISSUE-73 follow-up) -**Everything from the original report still holds and is folded in -here** (re-verified, not re-run): scope is every string property, not -just names (description/usage/summary/displayName/qualifiedName all -affected in the original 500-term scan); harmful specifically on -`displayName`/`qualifiedName` since those are lookup keys -(`get_guid_for_name` misses on the true value, hits on the doubled one); -doesn't compound across re-runs (escaping applied consistently to both -the upsert lookup key and the stored value, so it's internally -self-consistent within Dr.Egeria's own round-trip — which is likely why -it went unnoticed for so long); breaks Resource Explorer's display-name -resolution for the 2 affected Questions out of 41, and is an active -hazard for RE's Dr.Egeria bootstrap self-heal (an apostrophe-containing -canary is permanently unresolvable, re-running its batch on every check — -duplicate-link damage risk for non-idempotent commands like `Link -First/Next Process Step`). +**Status:** fixed 2026-08-25 (Dr.Egeria — `md_processing/v2/feedback.py`, +`md_processing/data/compact_commands/commands_external_references_compact.json`; +tests — `tests/micro-tests/test_feedback_link_multilink_guid.py`). -**Repro:** `Create Glossary Term` with `Display Name` = `What's the -owner's policy?` via `dr_egeria --process`; confirm the outgoing payload -is clean by tracing `_async_make_request`, then read the created element -back by GUID — the returned `displayName`/`description` have every `'` -doubled. No pyegeria/Dr.Egeria code path touches the value in between. +**Layer:** Dr.Egeria (`FeedbackLinkProcessor`). -**Nothing to fix here in this repo** — pyegeria sends the correct string; -Egeria's repository persists/returns a corrupted one. Track this against -the Egeria server, not this repo. +Same root cause and shape as ISSUE-73, found by extending that audit's +"two detach overloads" check across every `MULTI_LINK` type with a Dr.Egeria +command: `ExternalReferenceLink`, `MediaReference`, and `CitedDocumentLink` +are all `MULTI_LINK`. ISSUE-68's own audit already fixed +`_async_link_external_reference`/`_async_link_media_reference`/ +`_async_link_cited_document` to return the new relationship's GUID, but +`FeedbackLinkProcessor.apply_changes()` — the Dr.Egeria processor above +them — discarded it for all three, same as `CollectionLinkProcessor` did +before ISSUE-73. On Detach, all three went straight to a pair-based call +(`_async_detach_external_reference`/`_async_detach_media_reference`/ +`_async_detach_cited_document`) which, per Egeria's own multi-link API +pattern (see the egeria.git docs work on `site/docs/concepts/ +uni-multi-link.md`), removes *every* matching relationship between the +pair rather than one specific instance — even though GUID-targeted +`_by_id` detach methods (`_async_detach_external_reference_by_id`, +`_async_detach_media_reference_by_id`, +`_async_detach_cited_document_reference_by_id`) already existed, added by +the same PR #294 that added ISSUE-73's `_by_id` methods. ---- +**Fix:** identical shape to ISSUE-73 — `FeedbackLinkProcessor`'s three +Link branches capture the returned GUID into `parsed_output["guid"]` and +the shared final return now shows it under a "Relationship GUID" heading +when set. Added a `GUID` attribute to the `External Reference Link`/ +`Media Reference Link`/`Cited Document Link` compact-spec bundles via the +Spec Editor's REST API. Detach now checks for an explicit `GUID` attribute +first, routing to the `_by_id` method when present; falls back to the +previous pair-based call when absent (backward compatible, correct for +the common single-instance-per-pair case). -### ISSUE-70: `ActionAuthor.update_next_action_process_step()` always raises `AttributeError` — calls a method that does not exist +**Tests:** `test_feedback_link_multilink_guid.py` (7 tests, fake client, +no live server required) — GUID capture/display on all three Link +branches, explicit-GUID and pair-based-fallback paths on Detach for all +three. Full `pytest tests/micro-tests/` green. -**Status:** fixed 2026-08-21 (Pyegeria — `pyegeria/omvs/action_author.py`, -`tests/micro-tests/test_update_next_action_process_step.py`). Routed through -the real shared helper, `_async_update_relationship_request(url, -["NextGovernanceActionProcessStepProperties"], body)`, matching every other -relationship-update call site in the package (e.g. `asset_maker.py`'s -`_async_update_catalog_target`) rather than the nonexistent -`_async_update_relationship_body_request`. +**Caught while checking the "second detach overload" gotcha** described in +new egeria.git documentation (`site/docs/concepts/uni-multi-link.md`) — +that doc names the exact failure mode this and ISSUE-73 both guard +against: repeatedly calling `attach`/Link on a multi-link relationship +silently accumulates duplicate relationships (no error), most visible on +a schedule (e.g. an integration connector's `refresh()`), and a +subsequent Detach hits the "removes every matching relationship" ambiguity +instead of targeting one. Checked every remaining `MULTI_LINK` type +against this pattern while investigating: `AgreementActor`/`Certification`/ +`License`/`NextGovernanceActionProcessStep` never had a pair-based +overload to begin with (GUID-only by construction); `SolutionLinkingWire`'s +processor already prefers its GUID-based detach correctly; +`ValidValuesImplementation` has the same `_by_id` gap but no Dr.Egeria +command exists yet to fix (nothing to wire). -**Layer:** pyegeria. Single call site, straightforward fix. +--- -`_async_update_next_action_process_step` (pyegeria/omvs/action_author.py:328) -awaits `self._async_update_relationship_body_request(url, body)`. No method of -that name exists anywhere in the package — `grep -rn` returns exactly one hit, -the call itself. The method on the shared base is -`_async_update_relationship_request` (pyegeria/core/_server_client.py:7289). +### ISSUE-74: `ProjectManager`'s `ProjectProperties` body-builders sent `name` instead of `displayName` — rename/create silently sets nothing -So the public `update_next_action_process_step()` can never succeed; it raises -before issuing any request: - -``` -AttributeError: 'ActionAuthor' object has no attribute -'_async_update_relationship_body_request'. Did you mean: -'_async_update_relationship_request'? -``` - -**Found:** 2026-08-21, trying to change the `guard` on an existing -`NextGovernanceActionProcessStep` relationship from Resource Explorer. - -**Verified as version-current:** pyegeria 6.0.18.4 as installed in the trellis -venv. - -**Repro:** -```python -from pyegeria.omvs.action_author import ActionAuthor -aa = ActionAuthor(view_server, platform_url, user_id, user_pwd) -aa.create_egeria_bearer_token(user_id, user_pwd) -aa.update_next_action_process_step(relationship_guid, { - "class": "UpdateRelationshipRequestBody", - "properties": {"class": "NextGovernanceActionProcessStepProperties", - "guard": "some-guard", "mandatoryGuard": False}}) -``` - -**Likely fix:** rename the call to `_async_update_relationship_request`, matching -`_async_setup_next_action_process_step` immediately above it, which correctly -uses `_async_new_relationship_request(url, ["NextGovernanceActionProcessStepProperties"], body)` -— note that one also passes the properties-class list, which the update path -probably needs too. Worth checking the signature of -`_async_update_relationship_request` rather than assuming the argument shape. - -**Not blocking RE today.** Guards can already be authored through Dr.Egeria's -`Link Next Process Step` command (its `custom_attributes` include `Guard` and -`Mandatory Guard`), which is how RE writes them now — every link RE has authored -carries `guard: "Any"`, confirmed by reading a live definition back. This only -blocks *amending* an existing link's guard in place via the client, which is a -tidier path than re-authoring a whole document and re-running the reconciler. - ---- - -### ISSUE-71: Dr.Egeria silently drops `Produced Guards` — the value never reaches the element, and processing still reports SUCCESS - -**Status:** fixed 2026-08-21 (Dr.Egeria — -`md_processing/md_processing_utils/common_md_utils.py`, -`tests/micro-tests/governance/test_governance_action_type_properties.py`). -Root cause was broader than the "Simple List" theory this entry started -with: `update_gov_body_for_type()` had **no branch at all** for -`GovernanceActionType`/`GovernanceActionProcessStep` — neither type is in -`GENERAL_GOVERNANCE_DEFINITIONS` or `GOVERNANCE_CONTROLS` — so every -attribute their compact-spec bundles add (Implementation Description, -Produced Guards, Wait Time, and, for Process Step only, Ignore Multiple -Triggers) fell through to the generic "no dedicated mapping" fallback, -which only preserves base Referenceable/GovernanceDefinition fields. That -also means the live probe's own read of `waitTime = '0'` as "persisted" -was almost certainly a coincidental match against the server's default, -not a genuine round-trip — the value was never in the outgoing body either. -Added a dedicated branch mapping all four fields; the other `Simple List` -attributes checked elsewhere in this file were unaffected, since the actual -defect was per-type dispatch, not list handling. - -**Layer:** Dr.Egeria (`md_processing`) parse/build path, not the server. - -`Produced Guards` is declared in `commands_action_author.json` as a -`Simple List` attribute ("List of guards that this action type produces"), -inherited via the `Governance Action Type Base` bundle, so it applies to both -`Create Governance Action Type` and `Create Governance Action Process Step`. -Supplying it has no effect: the element is created with every other attribute and -no `producedGuards`. - -**Silent.** Processing reports SUCCESS, and Dr.Egeria's own outbox echo renders -the attribute as `None` — indistinguishable from an attribute that was never -supplied. - -**Verified live 2026-08-21** against this deployment, both commands, same result: - -| document said | echoed in outbox | stored on the element | -|---|---|---| -| `### Produced Guards`
`recovered, partial, no_signal, unverified, regression` | `None` | *(absent)* | -| `### Wait Time`
`0` | `0` | `waitTime = '0'` | - -`Wait Time` (Simple Int) and `Ignore Multiple Triggers` (Bool) on the same -element both persisted, so this is specific to the attribute, not to the command -or the document. - -**Repro** — process this and read the element back: -```markdown -## Create Governance Action Type - -### Display Name -Probe Guards On Action Type - -### Qualified Name -GovActionType::ProbeOutcomeVocab::probe - -### Produced Guards -recovered, partial, no_signal, unverified, regression -``` -```python -me.get_metadata_element_by_unique_name("GovActionType::ProbeOutcomeVocab::probe") -# elementProperties.propertiesAsStrings has no producedGuards -``` -The comma-separated form is the one used by this repo's own working test document -(`tests/dr-egeria-command-tests/dr_test_action_author.md`, `SUCCESS, FAILURE`), so -it is the documented spelling rather than a guess. - -**Two probe elements were left in the qs-metadata-store by this investigation**, -both named for deletion: -`GovActionProcessStep::ProbeOutcomeVocab::probe` and -`GovActionType::ProbeOutcomeVocab::probe`. Neither is linked into any process. - -**Why it matters beyond the attribute.** `producedGuards` is how a step declares -the outcomes it can complete with — the catalog-side half of the outcome -vocabulary Resource Explorer adopted on 2026-08-21 -(`resource_explorer/step_outcome.py`). Until this works, that vocabulary can be -recorded on results locally but cannot be *declared* in Egeria, so nothing else -can discover what a step is capable of producing. Not urgent — RE routes on -nothing today — but it blocks the specification-in-Egeria direction. - -**Worth checking whether other `Simple List` attributes are affected**, since the -failure looks like list-style handling rather than anything about guards. This -probe only exercised `Produced Guards`. - ---- - -### ISSUE-72: `GetRequestBody`/`ResultsRequestBody`'s `class` field was a hardcoded `Literal` matching only the base model's own name — rejected every real subclass tag, including ones this SDK's own docstrings tell callers to use - -**Status:** fixed 2026-08-23 (Pyegeria — `pyegeria/models/models.py`, -`tests/micro-tests/test_get_request_body_class_literal.py`). Reported -upstream as [GitHub issue #298](https://github.com/odpi/egeria-python/issues/298) -by the egeria-workspaces session, found live via `PyegeriaWebHandler` after -upgrading to pyegeria 6.1.1. +**Status:** fixed 2026-08-25 (Pyegeria — `pyegeria/omvs/project_manager.py`; +tests — `tests/micro-tests/test_project_manager_display_name.py`). -**Layer:** pyegeria. Single root cause, widely reached. +**Layer:** Pyegeria (`ProjectManager`). Reported by another concurrent +session doing Resource Explorer investigation work (found live via +`qs-view-server`, reproduced against `_async_update_project`), initially +filed as "ISSUE-73" — renumbered here since that number was already taken +by the merged multi-link fix above (this file's established collision +convention: keep the number on whichever entry's number is harder to +change at the point of collision). That original report only tested +`_async_update_project` and asserted `_async_create_project`'s convenience +path "works" for comparison; on inspection, `_async_create_project`'s own +default-body construction has the identical `"name"` mistake (confirmed +against the real behavior — its success in the original report's Resource +Explorer workaround came from callers passing an explicit `body=` with +`displayName`, the same workaround used for update, not from the +convenience path actually being correct). So the real scope is three call +sites, not one. -`ServerClient._async_get_guid_request`'s (and -`_async_get_request_body_request`'s, `_async_get_results_body_request`'s) -dict-body branch always validated against -`TypeAdapter(GetRequestBody)`/`TypeAdapter(ResultsRequestBody)`, and both -models' `class_` field was `Literal["GetRequestBody"]` / -`Literal["ResultsRequestBody"]` — an exact match against the *base* model's -own name. Any caller-supplied dict naming a real Egeria polymorphism -subclass — structurally identical, different `class` tag — was rejected -client-side before the request ever reached the server. Two live hits: +`ProjectProperties`'s real field is `displayName` (confirmed against +`Egeria-api-project-manager.http`, the ground truth — every worked example +uses `"displayName"`, never `"name"`). Three of `project_manager.py`'s own +default-body builders used `"name"` instead: +`_async_create_project`, `_async_update_project`, and +`_async_create_project_task`. Two docstring sample bodies had the same +wrong key baked in. -1. `SolutionArchitect.get_solution_component_by_guid`/ - `get_solution_blueprint_by_guid` — the method's own docstring documents - `"class": "AnyTimeRequestBody"`, which doesn't exist as its own Pydantic - model, so following the docstring produced a `PyegeriaInvalidParameterException` - / user-facing 500. -2. `ProjectManager.get_linked_projects` — same failure with `"class": - "RelationshipRequestBody"`. Worse in practice: callers wrapping this in - a reasonable try/except ("does this project have children" is often - best-effort) got a silent empty list instead of a visible error. +**Why this is worse than a plain failure:** for `_async_update_project`, +the body is a plain `dict` and `_async_update_element_body_request`'s +dict-branch does no Pydantic validation at all (`body_slimmer(body)` only) +— the wrong key goes out over the wire completely unchecked, Egeria's +deserializer just doesn't recognize it, and the call returns normally with +no exception. A caller renaming a project via the convenience parameter +ends up with the local record and the catalog permanently disagreeing +about the same element, with nothing anywhere indicating a problem. For +`_async_create_project`/`_async_create_project_task`, the body does go +through Pydantic validation (`validate_new_element_request`), but the +practical effect is the same: the display name is silently never set. -Since nothing downstream branches on the exact string in `class_` (these -two base models are never dispatched through a discriminated union — -`TypeAdapter(GetRequestBody)` is a single-type adapter), the fix loosens -`class_` on both models from `Literal[...]` to plain `str`: the field -still must be present, but any real subclass tag now round-trips through -unchanged in `model_dump_json`, matching the ~90 call sites across the SDK -that pass a `dict | GetRequestBody` body through this shared helper family. +**Fix:** all three `"name": display_name` sites (plus the two docstring +copies) changed to `"displayName": display_name`. Checked every other +OMVS file for the same substitution (`grep` for the literal pattern) — +this was isolated to `project_manager.py`; no other module has it. -**Not changing** the other ~60 `Literal`-tagged `class_` fields elsewhere -in `models.py` — those model narrower, single-purpose Properties/RequestBody -shapes where the literal genuinely pins down which typed model applies; -this defect was specific to `GetRequestBody`/`ResultsRequestBody` being -generic "workhorse" bodies reused, by Egeria's own design, under many -different polymorphism tags. +**Tests:** `test_project_manager_display_name.py` (3 tests, one per call +site) — patches `_async_make_request` to capture the JSON body that would +have been sent and asserts `displayName` carries the right value with no +stray `name` key. Verified each test genuinely catches the regression +(temporarily reverted the fix, confirmed all three fail with the exact +"unchanged" symptom, then restored it). Full `pytest tests/micro-tests/` +green throughout. --- @@ -4993,721 +4709,556 @@ without checking real ground truth first. --- -### ISSUE-74: `ProjectManager`'s `ProjectProperties` body-builders sent `name` instead of `displayName` — rename/create silently sets nothing +### ISSUE-72: `GetRequestBody`/`ResultsRequestBody`'s `class` field was a hardcoded `Literal` matching only the base model's own name — rejected every real subclass tag, including ones this SDK's own docstrings tell callers to use -**Status:** fixed 2026-08-25 (Pyegeria — `pyegeria/omvs/project_manager.py`; -tests — `tests/micro-tests/test_project_manager_display_name.py`). +**Status:** fixed 2026-08-23 (Pyegeria — `pyegeria/models/models.py`, +`tests/micro-tests/test_get_request_body_class_literal.py`). Reported +upstream as [GitHub issue #298](https://github.com/odpi/egeria-python/issues/298) +by the egeria-workspaces session, found live via `PyegeriaWebHandler` after +upgrading to pyegeria 6.1.1. -**Layer:** Pyegeria (`ProjectManager`). Reported by another concurrent -session doing Resource Explorer investigation work (found live via -`qs-view-server`, reproduced against `_async_update_project`), initially -filed as "ISSUE-73" — renumbered here since that number was already taken -by the merged multi-link fix above (this file's established collision -convention: keep the number on whichever entry's number is harder to -change at the point of collision). That original report only tested -`_async_update_project` and asserted `_async_create_project`'s convenience -path "works" for comparison; on inspection, `_async_create_project`'s own -default-body construction has the identical `"name"` mistake (confirmed -against the real behavior — its success in the original report's Resource -Explorer workaround came from callers passing an explicit `body=` with -`displayName`, the same workaround used for update, not from the -convenience path actually being correct). So the real scope is three call -sites, not one. +**Layer:** pyegeria. Single root cause, widely reached. -`ProjectProperties`'s real field is `displayName` (confirmed against -`Egeria-api-project-manager.http`, the ground truth — every worked example -uses `"displayName"`, never `"name"`). Three of `project_manager.py`'s own -default-body builders used `"name"` instead: -`_async_create_project`, `_async_update_project`, and -`_async_create_project_task`. Two docstring sample bodies had the same -wrong key baked in. +`ServerClient._async_get_guid_request`'s (and +`_async_get_request_body_request`'s, `_async_get_results_body_request`'s) +dict-body branch always validated against +`TypeAdapter(GetRequestBody)`/`TypeAdapter(ResultsRequestBody)`, and both +models' `class_` field was `Literal["GetRequestBody"]` / +`Literal["ResultsRequestBody"]` — an exact match against the *base* model's +own name. Any caller-supplied dict naming a real Egeria polymorphism +subclass — structurally identical, different `class` tag — was rejected +client-side before the request ever reached the server. Two live hits: -**Why this is worse than a plain failure:** for `_async_update_project`, -the body is a plain `dict` and `_async_update_element_body_request`'s -dict-branch does no Pydantic validation at all (`body_slimmer(body)` only) -— the wrong key goes out over the wire completely unchecked, Egeria's -deserializer just doesn't recognize it, and the call returns normally with -no exception. A caller renaming a project via the convenience parameter -ends up with the local record and the catalog permanently disagreeing -about the same element, with nothing anywhere indicating a problem. For -`_async_create_project`/`_async_create_project_task`, the body does go -through Pydantic validation (`validate_new_element_request`), but the -practical effect is the same: the display name is silently never set. +1. `SolutionArchitect.get_solution_component_by_guid`/ + `get_solution_blueprint_by_guid` — the method's own docstring documents + `"class": "AnyTimeRequestBody"`, which doesn't exist as its own Pydantic + model, so following the docstring produced a `PyegeriaInvalidParameterException` + / user-facing 500. +2. `ProjectManager.get_linked_projects` — same failure with `"class": + "RelationshipRequestBody"`. Worse in practice: callers wrapping this in + a reasonable try/except ("does this project have children" is often + best-effort) got a silent empty list instead of a visible error. -**Fix:** all three `"name": display_name` sites (plus the two docstring -copies) changed to `"displayName": display_name`. Checked every other -OMVS file for the same substitution (`grep` for the literal pattern) — -this was isolated to `project_manager.py`; no other module has it. +Since nothing downstream branches on the exact string in `class_` (these +two base models are never dispatched through a discriminated union — +`TypeAdapter(GetRequestBody)` is a single-type adapter), the fix loosens +`class_` on both models from `Literal[...]` to plain `str`: the field +still must be present, but any real subclass tag now round-trips through +unchanged in `model_dump_json`, matching the ~90 call sites across the SDK +that pass a `dict | GetRequestBody` body through this shared helper family. -**Tests:** `test_project_manager_display_name.py` (3 tests, one per call -site) — patches `_async_make_request` to capture the JSON body that would -have been sent and asserts `displayName` carries the right value with no -stray `name` key. Verified each test genuinely catches the regression -(temporarily reverted the fix, confirmed all three fail with the exact -"unchanged" symptom, then restored it). Full `pytest tests/micro-tests/` -green throughout. +**Not changing** the other ~60 `Literal`-tagged `class_` fields elsewhere +in `models.py` — those model narrower, single-purpose Properties/RequestBody +shapes where the literal genuinely pins down which typed model applies; +this defect was specific to `GetRequestBody`/`ResultsRequestBody` being +generic "workhorse" bodies reused, by Egeria's own design, under many +different polymorphism tags. --- -### ISSUE-75: `ExternalReferenceLink`/`MediaReference`/`CitedDocumentLink` — same create-time relationship GUID discarded, Detach had no way to target a specific instance (ISSUE-73 follow-up) - -**Status:** fixed 2026-08-25 (Dr.Egeria — `md_processing/v2/feedback.py`, -`md_processing/data/compact_commands/commands_external_references_compact.json`; -tests — `tests/micro-tests/test_feedback_link_multilink_guid.py`). +### ISSUE-71: Dr.Egeria silently drops `Produced Guards` — the value never reaches the element, and processing still reports SUCCESS -**Layer:** Dr.Egeria (`FeedbackLinkProcessor`). +**Status:** fixed 2026-08-21 (Dr.Egeria — +`md_processing/md_processing_utils/common_md_utils.py`, +`tests/micro-tests/governance/test_governance_action_type_properties.py`). +Root cause was broader than the "Simple List" theory this entry started +with: `update_gov_body_for_type()` had **no branch at all** for +`GovernanceActionType`/`GovernanceActionProcessStep` — neither type is in +`GENERAL_GOVERNANCE_DEFINITIONS` or `GOVERNANCE_CONTROLS` — so every +attribute their compact-spec bundles add (Implementation Description, +Produced Guards, Wait Time, and, for Process Step only, Ignore Multiple +Triggers) fell through to the generic "no dedicated mapping" fallback, +which only preserves base Referenceable/GovernanceDefinition fields. That +also means the live probe's own read of `waitTime = '0'` as "persisted" +was almost certainly a coincidental match against the server's default, +not a genuine round-trip — the value was never in the outgoing body either. +Added a dedicated branch mapping all four fields; the other `Simple List` +attributes checked elsewhere in this file were unaffected, since the actual +defect was per-type dispatch, not list handling. -Same root cause and shape as ISSUE-73, found by extending that audit's -"two detach overloads" check across every `MULTI_LINK` type with a Dr.Egeria -command: `ExternalReferenceLink`, `MediaReference`, and `CitedDocumentLink` -are all `MULTI_LINK`. ISSUE-68's own audit already fixed -`_async_link_external_reference`/`_async_link_media_reference`/ -`_async_link_cited_document` to return the new relationship's GUID, but -`FeedbackLinkProcessor.apply_changes()` — the Dr.Egeria processor above -them — discarded it for all three, same as `CollectionLinkProcessor` did -before ISSUE-73. On Detach, all three went straight to a pair-based call -(`_async_detach_external_reference`/`_async_detach_media_reference`/ -`_async_detach_cited_document`) which, per Egeria's own multi-link API -pattern (see the egeria.git docs work on `site/docs/concepts/ -uni-multi-link.md`), removes *every* matching relationship between the -pair rather than one specific instance — even though GUID-targeted -`_by_id` detach methods (`_async_detach_external_reference_by_id`, -`_async_detach_media_reference_by_id`, -`_async_detach_cited_document_reference_by_id`) already existed, added by -the same PR #294 that added ISSUE-73's `_by_id` methods. +**Layer:** Dr.Egeria (`md_processing`) parse/build path, not the server. -**Fix:** identical shape to ISSUE-73 — `FeedbackLinkProcessor`'s three -Link branches capture the returned GUID into `parsed_output["guid"]` and -the shared final return now shows it under a "Relationship GUID" heading -when set. Added a `GUID` attribute to the `External Reference Link`/ -`Media Reference Link`/`Cited Document Link` compact-spec bundles via the -Spec Editor's REST API. Detach now checks for an explicit `GUID` attribute -first, routing to the `_by_id` method when present; falls back to the -previous pair-based call when absent (backward compatible, correct for -the common single-instance-per-pair case). +`Produced Guards` is declared in `commands_action_author.json` as a +`Simple List` attribute ("List of guards that this action type produces"), +inherited via the `Governance Action Type Base` bundle, so it applies to both +`Create Governance Action Type` and `Create Governance Action Process Step`. +Supplying it has no effect: the element is created with every other attribute and +no `producedGuards`. -**Tests:** `test_feedback_link_multilink_guid.py` (7 tests, fake client, -no live server required) — GUID capture/display on all three Link -branches, explicit-GUID and pair-based-fallback paths on Detach for all -three. Full `pytest tests/micro-tests/` green. +**Silent.** Processing reports SUCCESS, and Dr.Egeria's own outbox echo renders +the attribute as `None` — indistinguishable from an attribute that was never +supplied. -**Caught while checking the "second detach overload" gotcha** described in -new egeria.git documentation (`site/docs/concepts/uni-multi-link.md`) — -that doc names the exact failure mode this and ISSUE-73 both guard -against: repeatedly calling `attach`/Link on a multi-link relationship -silently accumulates duplicate relationships (no error), most visible on -a schedule (e.g. an integration connector's `refresh()`), and a -subsequent Detach hits the "removes every matching relationship" ambiguity -instead of targeting one. Checked every remaining `MULTI_LINK` type -against this pattern while investigating: `AgreementActor`/`Certification`/ -`License`/`NextGovernanceActionProcessStep` never had a pair-based -overload to begin with (GUID-only by construction); `SolutionLinkingWire`'s -processor already prefers its GUID-based detach correctly; -`ValidValuesImplementation` has the same `_by_id` gap but no Dr.Egeria -command exists yet to fix (nothing to wire). +**Verified live 2026-08-21** against this deployment, both commands, same result: ---- +| document said | echoed in outbox | stored on the element | +|---|---|---| +| `### Produced Guards`
`recovered, partial, no_signal, unverified, regression` | `None` | *(absent)* | +| `### Wait Time`
`0` | `0` | `waitTime = '0'` | -### ISSUE-76: `TermRelationshipProcessor` offered obsolete/fictional term-relationship types, and swallowed the resulting server failure as a false "success" +`Wait Time` (Simple Int) and `Ignore Multiple Triggers` (Bool) on the same +element both persisted, so this is specific to the attribute, not to the command +or the document. -**Status:** fixed 2026-08-26 (Dr.Egeria — `md_processing/v2/glossary.py`, -`md_processing/data/compact_commands/commands_glossary_compact.json`; -tests — `tests/micro-tests/test_term_relationship_processor.py`). +**Repro** — process this and read the element back: +```markdown +## Create Governance Action Type -**Layer:** Dr.Egeria (`TermRelationshipProcessor`, `TermAsContextProcessor`). +### Display Name +Probe Guards On Action Type -**Reported by dwolfson** (user-forwarded field report): `dr_egeria --directive -process` on a real glossary file produced repeated `SERVER_ERROR_500`s — -`OMAG-COMMON-400-018 The type name TermISATYPEOFRelationship ... is not -recognized` (also seen for `TermHASARelationship`, `TermTYPEDBYRelationship`) -— and, worse, **the run's final summary reported success anyway** if left -to complete, with no visible sign anything had failed except scrollback the -user had to notice and read. +### Qualified Name +GovActionType::ProbeOutcomeVocab::probe -**Bug 1 — obsolete/fictional type names.** `TermRelationshipProcessor`'s -`rel_mapping` mapped `HASA`/`HAS A` → `TermHASARelationship`, `TYPED BY` → -`TermTYPEDBYRelationship`, and `TYPE OF` → `TermISATYPEOFRelationship`. -None of these three target names has ever existed as a real Egeria -relationship type — confirmed against a live server's -`get_all_relationship_defs()` (only `Synonym`, `PreferredTerm`, `Antonym`, -`ReplacementTerm`, `RelatedTerm`, `ISARelationship` are real -`GlossaryTerm`<->`GlossaryTerm` relationship types) and against every -`open-metadata-types` archive version in odpi/egeria's own source history -(no hit for any of the three names, ever). The compact spec's -`Relationship Type` enum made this worse by also offering `Translation` -and `ValidValue` as selectable values — real Egeria type names exist under -similar spellings (`TranslationLink`, several `ValidValue*` relationships) -but none of them connects two `GlossaryTerm`s, so both would also have -failed the same way. +### Produced Guards +recovered, partial, no_signal, unverified, regression +``` +```python +me.get_metadata_element_by_unique_name("GovActionType::ProbeOutcomeVocab::probe") +# elementProperties.propertiesAsStrings has no producedGuards +``` +The comma-separated form is the one used by this repo's own working test document +(`tests/dr-egeria-command-tests/dr_test_action_author.md`, `SUCCESS, FAILURE`), so +it is the documented spelling rather than a guess. -**Bug 2 — the false "success" (the more serious one).** -`TermRelationshipProcessor.apply_changes()` caught `PyegeriaException` -internally, logged an error, set `self.parsed_output['valid'] = False`, -and returned `self.command.raw_block` — which *looks* like a correct -error-handling path, but isn't: `AsyncBaseCommandProcessor.execute()` only -checks `parsed_output['valid']` in its pre-flight validation step (step 2), -which runs *before* `apply_changes()` (step 7) is ever called. By the time -`apply_changes()` discovers the failure, that check has already passed. -Catching the exception inside `apply_changes()` meant it never reached -`execute()`'s own `try/except PyegeriaException` (which does correctly set -`"status": "failure"`), so `execute()` fell through to its normal -success-path return unconditionally. Checked every other -`except PyegeriaException` in `md_processing/v2/*.py` for the same -pattern — this was the only one; everywhere else is either the dispatcher's -own correct top-level catch or a benign best-effort read (`fetch_element`/ -optional-fallback lookups) that doesn't guard a write operation. +**Two probe elements were left in the qs-metadata-store by this investigation**, +both named for deletion: +`GovActionProcessStep::ProbeOutcomeVocab::probe` and +`GovActionType::ProbeOutcomeVocab::probe`. Neither is linked into any process. -**Fix:** -- `rel_mapping` now only maps `ISA`/`IS A` → `ISARelationship` (the one - real alias). A new `REAL_TERM_RELATIONSHIP_TYPES` set is checked before - any request is built; an unrecognized value raises a clear `ValueError` - naming the real options, instead of reaching the server and 400ing. -- The internal `try/except PyegeriaException` around the actual link/unlink - calls is removed entirely — the exception now propagates to `execute()`, - which correctly reports `"status": "failure"` in the batch summary. -- Compact spec's `Relationship Type` enum trimmed to the 6 real values - (`RelatedTerm`, `Synonym`, `Antonym`, `PreferredTerm`, `ReplacementTerm`, - `ISA`/`ISARelationship`) via the Spec Editor's REST API; `refresh_specs` - regenerated the affected templates. -- **Found while fixing this**: `Link Term as Context` (`UsedInContext`, a - *fixed* relationship type — not user-selectable) was incorrectly sharing - the same `Term-Term Link Base` bundle as `Link Term-Term Relationship`, - which meant it inherited a **required** `Relationship Type` attribute - that `TermAsContextProcessor.apply_changes()` never reads at all — users - were forced to supply a value for a field that does nothing. Gave it its - own `Term as Context Base` bundle containing only the attributes the - processor actually reads (`Confidence`/`Expression`/`Source`/`Steward`/ - `Term Relationship Status`, all confirmed genuinely used); `Term 1`/ - `Element Id` stay as the command's own `custom_attributes`, unchanged. +**Why it matters beyond the attribute.** `producedGuards` is how a step declares +the outcomes it can complete with — the catalog-side half of the outcome +vocabulary Resource Explorer adopted on 2026-08-21 +(`resource_explorer/step_outcome.py`). Until this works, that vocabulary can be +recorded on results locally but cannot be *declared* in Egeria, so nothing else +can discover what a step is capable of producing. Not urgent — RE routes on +nothing today — but it blocks the specification-in-Egeria direction. -**Tests:** `test_term_relationship_processor.py` (5 tests, fake client, no -live server required) — a real type still links successfully, `ISA` maps -correctly, an obsolete type raises `ValueError` before ever reaching the -client, and a genuine `PyegeriaException` from the client propagates out -of `apply_changes()` rather than being swallowed. Full `pytest -tests/micro-tests/` green. +**Worth checking whether other `Simple List` attributes are affected**, since the +failure looks like list-style handling rather than anything about guards. This +probe only exercised `Produced Guards`. --- -### ISSUE-77: 5 classification attributes on the shared `Referenceable` bundle, and `Anchor Scope IDs`/`Make Anchor` on the shared `Link Command Base` bundle, were parsed and validated but never applied +### ISSUE-70: `ActionAuthor.update_next_action_process_step()` always raises `AttributeError` — calls a method that does not exist -**Status:** fixed 2026-08-26 (Dr.Egeria — `md_processing/v2/processors.py`, -`md_processing/v2/glossary.py`, `md_processing/md_processing_utils/common_md_utils.py`, -`md_processing/data/compact_commands/*.json`; tests — -`tests/micro-tests/test_referenceable_classification_sync.py`, -`tests/micro-tests/test_link_body_anchor_fields.py`). +**Status:** fixed 2026-08-21 (Pyegeria — `pyegeria/omvs/action_author.py`, +`tests/micro-tests/test_update_next_action_process_step.py`). Routed through +the real shared helper, `_async_update_relationship_request(url, +["NextGovernanceActionProcessStepProperties"], body)`, matching every other +relationship-update call site in the package (e.g. `asset_maker.py`'s +`_async_update_catalog_target`) rather than the nonexistent +`_async_update_relationship_body_request`. -**Layer:** Dr.Egeria. Found by `scripts/dr_egeria_audit.py`'s DEAD_ATTRIBUTE -check (see the tool's own PR, #307) — its ~1162 raw hits collapsed to ~30 -distinct attribute names once bundle-inheritance duplication is accounted -for; this entry covers the 7 dominant ones, dwolfson's explicit priority. +**Layer:** pyegeria. Single call site, straightforward fix. -**`Security Tags` / `Policy Management Point`** — both real, generic -classifications (`SecurityTagsProperties`/`PolicyManagementPointProperties`, -confirmed against `Egeria-api-classification-explorer.http`/ -`Egeria-api-governance-officer.http` — both endpoints are -`elements/{guid}/...`, valid for any `Referenceable`) already had a fully -working standalone `Classify X as Security Tags`/`Classify X as Policy -Management Point` command (`CurationClassifyProcessor`, -`CLASSIFICATION_METHODS` in `md_processing/v2/curation.py`) — but the -*embedded* copy of these same attributes on the `Referenceable` bundle -(inherited by nearly every `Create` command) was never synced after -create/update, unlike its 5 siblings -(`Confidence`/`Confidentiality`/`Criticality`/`Impact`/`Retention -Classification`, handled by the pre-existing `_sync_governance_classifications`). -Fixed by adding a new `AsyncBaseCommandProcessor._sync_referenceable_classifications()`, -called alongside `_sync_governance_classifications` for every Create/Update, -reusing the same `CLASSIFICATION_METHODS`/`CURATION_CLASSIFICATION_CLIENTS` -spec the standalone commands already use (deferred import from -`curation.py` to avoid the circular import — `curation.py` imports -`AsyncBaseCommandProcessor` from `processors.py`). - -**`Class Word` / `Modifier` / `Prime Word` Classification — a different, -more serious shape of the same bug, since simply reusing the pattern above -would have introduced a *new* live failure.** Their real endpoints -(`_async_set_is_class_word` etc., `pyegeria/omvs/glossary_manager.py`) are -`/glossaries/terms/{term_guid}/is-class-word` — **GlossaryTerm-only**, not -generic — confirmed against the actual URL. The embedded attributes were -on the same widely-shared `Referenceable` bundle as `Security Tags` above, -inherited by ~100 unrelated Create commands (`Create Governance Action -Process`, `Create Digital Product`, ...). Syncing these generically for -every Create/Update guid would 400 against the term-only endpoint for -every non-term element. Per dwolfson's explicit direction: **removed from -the shared `Referenceable` bundle in all 11 families that had it** (only -`Create Glossary Term`/`Create Question` legitimately need them — both -inherit the term-specific `Glossary Term Base` bundle in -`commands_glossary_compact.json`, which now carries the 3 attributes -instead). Wired into a new `AsyncBaseCommandProcessor._sync_term_naming_classifications()`, -called only from `TermProcessor`/`QuestionProcessor` -(`md_processing/v2/glossary.py`) where the guid is known to be a real term -— never from the generic Create/Update sync path. +`_async_update_next_action_process_step` (pyegeria/omvs/action_author.py:328) +awaits `self._async_update_relationship_body_request(url, body)`. No method of +that name exists anywhere in the package — `grep -rn` returns exactly one hit, +the call itself. The method on the shared base is +`_async_update_relationship_request` (pyegeria/core/_server_client.py:7289). -(`commands_curation_compact.json`/`commands_report_compact.json` also -define a `Referenceable` bundle, but neither has any command that actually -uses it — confirmed dead/orphaned in both, pre-existing and unrelated to -this fix, left as-is.) +So the public `update_next_action_process_step()` can never succeed; it raises +before issuing any request: -**`Anchor Scope IDs` / `Make Anchor`** — a naming trap: there are *two* -separate "anchor scope" attributes in the compact spec. `Anchor Scope ID` -(singular) lives on the `New-Element` bundle (Create commands) and was -already correctly wired to `anchorScopeGUID` on `NewElementRequestBody`. -`Anchor Scope IDs` (plural, `Reference Name List` style) is a *different* -attribute living on `Link Command Base` (relationship-establishing -commands) alongside `Make Anchor` — neither was ever included in -`set_rel_request_body()`'s outer `NewRelationshipRequestBody`, confirmed -missing against `Egeria-api-lineage-linker.http`, which shows both -`anchorScopeGUIDs` (plural list) and `makeAnchor` as real top-level fields -on that body. Fixed by adding both to `set_rel_request_body()`. +``` +AttributeError: 'ActionAuthor' object has no attribute +'_async_update_relationship_body_request'. Did you mean: +'_async_update_relationship_request'? +``` -**Deliberately not fixed here — flagged, not guessed at:** -- **`Request ID`** — parsed and validated everywhere via the shared - `Request Base` bundle, but no generic `requestId` field exists anywhere - in the real Egeria REST API (searched the entire `.http` ground truth: - one match, in an unrelated user-management endpoint). The only real - "request ID" concept in this codebase is `self.context["request_id"]`, - an internally auto-generated `uuid.uuid4()` in - `AsyncBaseCommandProcessor.__init__` that nothing else currently reads. - Per dwolfson (a rare/Advanced-level corner case): an explicit - `Request ID` value now overrides the auto-generated one for that - command's execution, added in `execute()` right after parsing — minimal, - since there's no real consumer to fully wire yet. +**Found:** 2026-08-21, trying to change the `guard` on an existing +`NextGovernanceActionProcessStep` relationship from Resource Explorer. -**Tests:** `test_referenceable_classification_sync.py` (6 tests) and -`test_link_body_anchor_fields.py` (2 tests), fake client, no live server -required. Full `pytest tests/micro-tests/` green throughout. +**Verified as version-current:** pyegeria 6.0.18.4 as installed in the trellis +venv. ---- +**Repro:** +```python +from pyegeria.omvs.action_author import ActionAuthor +aa = ActionAuthor(view_server, platform_url, user_id, user_pwd) +aa.create_egeria_bearer_token(user_id, user_pwd) +aa.update_next_action_process_step(relationship_guid, { + "class": "UpdateRelationshipRequestBody", + "properties": {"class": "NextGovernanceActionProcessStepProperties", + "guard": "some-guard", "mandatoryGuard": False}}) +``` -### ISSUE-48: No OMVS wrapper exists for the `SchemaAttributeDefinition` relationship (physical `SchemaAttribute` ↔ logical `DataField`) +**Likely fix:** rename the call to `_async_update_relationship_request`, matching +`_async_setup_next_action_process_step` immediately above it, which correctly +uses `_async_new_relationship_request(url, ["NextGovernanceActionProcessStepProperties"], body)` +— note that one also passes the properties-class list, which the update path +probably needs too. Worth checking the signature of +`_async_update_relationship_request` rather than assuming the argument shape. -**Status:** implemented 2026-08-19 via Option 2 (the generic -`MetadataExpert` relationship mechanism) — deferred 2026-08-15, re-checked -2026-08-19 (still no bespoke Egeria endpoint, confirmed against the full -`/v3/api-docs` spec; Option 2 confirmed viable live), then built the same -day per the user's go-ahead. +**Not blocking RE today.** Guards can already be authored through Dr.Egeria's +`Link Next Process Step` command (its `custom_attributes` include `Guard` and +`Mandatory Guard`), which is how RE writes them now — every link RE has authored +carries `guard: "Any"`, confirmed by reading a live definition back. This only +blocks *amending* an existing link's guard in place via the client, which is a +tidier path than re-authoring a whole document and re-running the reconciler. -**What shipped:** a new Dr.Egeria command family, `Link/Detach Schema -Attribute Definition` (`Data Designer` family, -`commands_data_designer.json`, new `Schema Attribute` attribute alongside -the existing `Data Field`), backed by a new -`LinkSchemaAttributeDefinitionProcessor` -(`md_processing/v2/data_designer.py`): -- **Link/Attach/Add:** calls - `MetadataExpert._async_create_related_elements` with a - `NewRelatedElementsRequestBody` (`typeName: SchemaAttributeDefinition`), - captures the returned relationship GUID into - `parsed_output["guid"]`, and displays it in the result markdown. -- **Detach/Unlink/Remove:** looks the relationship up by its element pair - via `_async_find_relationships_between_elements` - (`end1EntityGUIDs`/`end2EntityGUIDs`) rather than requiring a separate - GUID attribute — safe because `SchemaAttributeDefinition`'s - `relationshipCategory` is `UNI_LINK` with `AT_MOST_ONE` cardinality on - both ends (confirmed live via - `ValidMetadataManager.get_all_relationship_defs()`), so there's no - multi-instance ambiguity to resolve. Then deletes via - `_async_delete_related_elements(relationship_guid, ...)`. -- `supports_target_element_lookup()` overridden to `False` (relationship- - only processor — see ISSUE-68's `AsyncBaseCommandProcessor` upsert- - rewrite-bug fix for why this matters; avoids the same latent Update-verb - hazard flagged there for any future relationship processor). +--- -**No new pyegeria SDK surface required** — `_async_create_related_elements`/ -`_async_find_relationships_between_elements`/`_async_delete_related_elements` -all already existed on `MetadataExpert`. +### ISSUE-69: Egeria's repository (server-side) doubles every apostrophe on persist (`what's` -> `what''s`) — corrupts displayName/qualifiedName, making affected elements unfindable by their real name -**Verified live**, full round trip against a real `DataField`/ -`SchemaAttribute` pair: `Link Schema Attribute Definition` via -`--process` created the relationship and displayed its GUID; a direct -`_async_find_relationships_between_elements` query confirmed it existed; -`Detach Schema Attribute Definition` via `--process` removed it; the same -query afterward returned "No elements found", confirming clean deletion. +**Layer:** Egeria Server / repository connector — **not pyegeria, not +Dr.Egeria**. Originally logged (2026-08-19) as a Dr.Egeria write-path bug +with a theory pointing at a nonexistent "SQLite-backed spec/journal +store" in `md_processing/v2/` — re-investigated 2026-08-19 by tracing the +*actual outgoing HTTP request body* (not just Dr.Egeria's own echoed +attribute table) and the theory doesn't hold: there is no sqlite3 usage +anywhere in `md_processing/`/`pyegeria/` (`grep -rn "sqlite3"` — zero +hits), and the wire payload itself is provably clean. -**Tests:** `test_link_schema_attribute_definition.py` (5 tests, fake -client, no live server needed). +**Status: RESOLVED 2026-08-26 (server-side).** Re-ran the exact repro +(`_async_create_glossary_term` with an apostrophe in `displayName`, +`qualifiedName`, and `description`, immediate read-back by GUID) against +the currently-running server (`6.2-SNAPSHOT`, platform build timestamp +`2026-08-24T18:27:04Z` per `/api/about` — a newer build than whatever was +running on 2026-08-19). All three properties round-tripped byte-for-byte +correct, single apostrophes, no doubling. Ran twice independently, both +clean; test terms deleted after confirming. This looks like a server-side +fix landed via a platform redeploy between 2026-08-19 and now, not a +pyegeria/Dr.Egeria change (nothing in this repo touched the write path in +between). Moved here from "Open Egeria Server issues" on that basis — one +caveat worth keeping in mind: this is a SNAPSHOT build, not a pinned +release, so re-check if it ever resurfaces on a later build. -**Explicitly temporary, per the user's own framing** ("implement Option 2 -... and we'll replace it once the new endpoint is available") — re-check -this issue's own "still deferred" 2026-08-19 OpenAPI-spec check -periodically; once Egeria ships a bespoke `SchemaAttributeDefinition` -endpoint, migrate `LinkSchemaAttributeDefinitionProcessor` to a dedicated -`DataDesigner._async_link_schema_attribute_definition`/ -`_async_detach_schema_attribute_definition` wrapper (Option 1) instead — -same compact-spec command, same attributes, just a different -implementation underneath. +**Status (as originally logged, kept for history): was open, root cause was correctly located as follows.** ---- +**Re-traced live, instrumenting `_async_make_request` directly** (not +just Dr.Egeria's parser echo, which the original report already showed +was clean) to capture the exact JSON string sent over the wire for a +`Create Glossary Term` with `Description` = `What's the owner's policy?`: -### ISSUE-55: Desired enhancement — true "exclude type" / NOT semantics for `findMetadataElements` +``` +[POST] .../glossary-manager/glossaries/terms +PAYLOAD: ...,"description": "What's the owner's policy?","category": null... +``` -**Status:** fixed 2026-08-18 — Egeria shipped it. Raised 2026-08-05 -alongside ISSUE-45/ISSUE-53, from the same Egeria Insights use case -("elements that have a SemanticAssignment relationship but are NOT type -Notification"), flagged then as needing real Egeria-side API design work -with no shape prescribed. dwolfson found the implementation: -[odpi/egeria#9215](https://github.com/odpi/egeria/pull/9215). +Single apostrophes, correctly JSON-encoded, exactly as authored — **the +outgoing request from pyegeria/Dr.Egeria is clean.** Immediately reading +the same element back by GUID after creation: -**What Egeria shipped:** a new `skipSubtypes` boolean on `QueryOptions` -(server-side). When `true`, `metadataElementSubtypeNames` is treated as -the list of subtypes to *exclude* from the results instead of the only -subtypes to *include* (the pre-existing default, -`skipSubtypes=false`/absent) — exactly the "NOT semantics" this issue -asked for, without needing a second field or new operator. -`metadataElementSubtypeNames` itself also moved off `GetOptions` onto -`QueryOptions` in the same PR (it never did anything on a plain get-by-guid -style request; now it's only ever relevant to find/search bodies, matching -how it actually behaves). Relationship-side got the equivalent treatment: -`FindRelationshipRequestBody.relationshipSubtypeGUIDs` was renamed to -`relationshipSubtypeNames` (GUID-based → type-name-based, matching the -element side) with the same `skipSubtypes` exclude-semantics threaded -through the relationship find/count handlers — though (see "Not done" -below) `FindRelationshipRequestBody` itself has no `skipSubtypes` field on -the wire, unlike the element side. +```python +term = await gm._async_get_term_by_guid(new_guid) +props["description"] # -> "What''s the owner''s policy?" -- DOUBLED +``` -**pyegeria fix:** `FindRequestBody` (`pyegeria/models/models.py`) — the -model backing the *validated* find-elements path (e.g. -`ClassificationExplorer._async_find_root_elements`, anything going through -`FindRequestBody.model_validate()`/the `TypeAdapter` in -`_server_client.py`) — had no `skip_subtypes` field, so a caller setting -`"skipSubtypes": true` there would have been silently dropped by -`PyegeriaModel`'s `extra='ignore'`, same bug class as ISSUE-62/63. Added -`skip_subtypes: bool | None = Field(None, alias="skipSubtypes")`. -`MetadataExpert._async_find_metadata_elements` needed no code change — it -sends its body dict as-is with no validation, so `"skipSubtypes": true` -already worked there even before this fix (confirmed live, both paths). -Also left a comment on `GetRequestBody.metadata_element_subtype_names` -noting it's now vestigial server-side (kept, not removed — the server -ignores unknown/inapplicable fields rather than rejecting them, so no live -breakage either way). Updated sample-body docstrings in -`metadata_expert.py` (`find_metadata_elements`/`find_relationships_between_elements`/ -`count_relationships_between_elements`) and `classification_explorer.py` -(`find_root_elements`) to show `skipSubtypes`/`relationshipSubtypeNames` -for discoverability. +**Reproduced again with zero pyegeria/Python code in the loop at all** — +plain `curl`, using `Egeria-api-glossary-manager.http`'s own +`createGlossaryTerm`/`getGlossaryTermByGUID` requests as the template +(same URLs, same body shape), to rule out any possibility this was an +httpx/Pydantic serialization artifact rather than a genuine server round +trip: -**Not done:** no `FindRelationshipRequestBody` typed model exists in -pyegeria at all (every relationship-find call site builds a raw dict) — -checked the real server-side class directly -(`frameworkservices/omf/rest/FindRelationshipRequestBody.java`): it -extends the older `commonservices.ffdc.rest.ResultsRequestBody`, not -`QueryOptions`, so `skipSubtypes` is genuinely **not** an accepted field -on relationship-find/count request bodies today, only on element-find — -confirmed by reading the class, not assumed. If Egeria adds it there -later, the fix is the same shape (a typed model + the field), but there's -nothing to add on the pyegeria side until the server does. +```bash +TOKEN=$(curl -sk -X POST https://localhost:9443/api/token \ + -H "Content-Type: application/json" \ + -d '{"userId":"erinoverview","password":"secret"}') -**Dr.Egeria:** no command-level change — checked `find_constraints` -(`md_processing/v2/processors.py`), Dr.Egeria's only existing -subtype-related mechanism, and it's a different, narrower thing (a single -include-type used to disambiguate a name-based element lookup during -processing, not a `FindRequestBody`-style batch search with an -include/exclude subtype list). No compact command currently exposes -element-search subtype filtering to a markdown author at all, so there's -no existing user-facing surface for `skip_subtypes` to plug into yet. The -natural future consumer is `pyegeria/view/overview_metrics.py`'s analytic -functions (several already build `FindRequestBody`s) if a metric ever -needs "count everything except X" — not built speculatively here. - -**Verified live** against a running server, both request paths: -- Raw-dict pass-through (`MetadataExpert._async_find_metadata_elements`): - `metadataElementSubtypeNames: ["GlossaryTerm"], skipSubtypes: true` on - `Referenceable` returned zero `GlossaryTerm` elements in the sample - (types seen: `ConnectorActivityReport`, `GovernanceZone`). - Cross-checked the include-list side too (ISSUE-53's own fix, still - holding): restricting to `["GlossaryTerm"]` without `skipSubtypes` - correctly returned only `GlossaryTerm`. -- Validated `FindRequestBody` path - (`ClassificationExplorer._async_find_root_elements`): identical - `skipSubtypes: true` behavior after the model fix — confirmed the field - actually reaches the server through Pydantic validation, not just the - raw-dict shortcut. +curl -sk -X POST \ + https://localhost:9443/servers/qs-view-server/api/open-metadata/glossary-manager/glossaries/terms \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{ + "class": "NewElementRequestBody", + "isOwnAnchor": true, + "properties": { + "class": "GlossaryTermProperties", + "displayName": "Native Curl Apostrophe Test QRS456", + "qualifiedName": "Coco Pharmaceuticals::Term::Native-Curl-Apostrophe-Test-QRS456::1.0", + "description": "What'\''s the owner'\''s policy?", + "contentStatus": "ACTIVE", + "versionIdentifier": "1.0" + }, + "parentAtEnd1": true + }' +# -> {"class":"GUIDResponse", ..., "guid":"494d35d8-..."} -**Tests:** `test_find_request_body_skip_subtypes.py` (4 tests, model-level, -no live server needed). +curl -sk -X POST \ + https://localhost:9443/servers/qs-view-server/api/open-metadata/glossary-manager/glossaries/terms/494d35d8-a939-47dc-9504-a984117d2b76 \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"class":"GetRequestBody","forLineage":false,"forDuplicateProcessing":false}' +# -> "description": "What''s the owner''s policy?" -- DOUBLED, same as via pyegeria +``` ---- +Identical result via raw `curl` as via pyegeria — this is conclusive: +the doubling happens inside Egeria itself, full stop, with no dependency +on any client library. (Both test terms cleaned up after confirming.) -### ISSUE-61: No Dr.Egeria command exists to attach a `GovernanceMetric` to its implementation (the `GovernanceResults` relationship) — the OMVS wrapper already exists, it's just never wired to a compact-spec command +So the corruption happens strictly **between the server receiving a +correct single-apostrophe payload and what it persists/returns** — inside +Egeria itself (most likely the active repository connector's handling of +string properties, e.g. a SQL string-literal escape (`'` → `''`, the +standard SQL-92 way to escape a quote inside a literal) applied even +though the actual write should be going through a parameterized +query/placeholder that needs no manual escaping — but not confirmed to +that level of detail; flagging the mechanism as a plausible lead for +whoever picks this up server-side, not asserting it). -**Status:** fixed 2026-08-17 (Pyegeria/Dr.Egeria — Dan, via the Dr.Egeria -Spec Editor's REST API). Added `Link Governance Results` (`OM_TYPE -GovernanceResults`, bundle `Link Command Base`) to the `Governance Officer` -family, with two new Reference Name-style attributes ("Governance Metric" → -`GovernanceMetric`, "Data Asset" → `Asset`) matching the sibling "Governance -Definition"/"Referenceable" pair on `Link Governed By`. `GovernanceLinkProcessor` -(`md_processing/v2/governance.py`) already handles Link/Detach/Unlink/Remove -variants from one spec entry via `build_command_variants` — no separate -registration needed; wired via a new `endpoint_map` entry + one -`apply_changes` branch per direction, calling -`_async_link_governance_results`/`_async_detach_governance_results` with a -body built via the existing generic `set_rel_prop_body()` (no bespoke body -builder needed). +**Everything from the original report still holds and is folded in +here** (re-verified, not re-run): scope is every string property, not +just names (description/usage/summary/displayName/qualifiedName all +affected in the original 500-term scan); harmful specifically on +`displayName`/`qualifiedName` since those are lookup keys +(`get_guid_for_name` misses on the true value, hits on the doubled one); +doesn't compound across re-runs (escaping applied consistently to both +the upsert lookup key and the stored value, so it's internally +self-consistent within Dr.Egeria's own round-trip — which is likely why +it went unnoticed for so long); breaks Resource Explorer's display-name +resolution for the 2 affected Questions out of 41, and is an active +hazard for RE's Dr.Egeria bootstrap self-heal (an apostrophe-containing +canary is permanently unresolvable, re-running its batch on every check — +duplicate-link damage risk for non-idempotent commands like `Link +First/Next Process Step`). -Originally raised 2026-08-17 while designing how the Egeria Overview -dashboard's ~25 `pyegeria.view.overview_metrics` functions could be -described as real Egeria metadata rather than only Python docstrings — see -`egeria-workspaces-fs`'s `OVERVIEW_NEXT_STEPS.md` for the full design -reasoning (`GovernanceMetric` for the definition, `GovernanceResults` → -`Report` for the implementation, no new relationship type needed since -`Report` is already a `DataSet` subtype). +**Repro:** `Create Glossary Term` with `Display Name` = `What's the +owner's policy?` via `dr_egeria --process`; confirm the outgoing payload +is clean by tracing `_async_make_request`, then read the created element +back by GUID — the returned `displayName`/`description` have every `'` +doubled. No pyegeria/Dr.Egeria code path touches the value in between. -**Verified live 2026-08-17** against the pilot `GovernanceMetric`/`Report` -pair created during the original design investigation: `--validate` and -`--process` both succeed; detached the pilot's original raw-API-created -`GovernanceResults` relationship (hit the known -`deleteRelationshipInStore`-rejects-its-own-default-`deleteMethod` bug — -worked around with an explicit `{"deleteMethod": "SOFT_DELETE"}` body, same -issue this repo's `DeleteRelationshipRequestBody.delete_method` field -already exists to work around elsewhere) and re-created it via the new -`Link Governance Results` command; `mgr.get_all_related_elements()` confirms the relationship resolves correctly afterward. +**Nothing to fix here in this repo** — pyegeria sends the correct string; +Egeria's repository persists/returns a corrupted one. Track this against +the Egeria server, not this repo. --- -### ISSUE-62: `DeleteElementRequestBody` had no `cascadeDelete`/`deleteMethod` fields — `extra='ignore'` silently discarded both, not just left them unsupported - -**Status:** fixed 2026-08-18 (Pyegeria — `pyegeria/models/models.py`, -`tests/micro-tests/test_delete_element_request_body.py`). - -Raised by dwolfson noticing `_async_delete_element_request` and its -Pydantic model didn't appear to support `deleteMethod`. Investigation found -two gaps, not one: - -1. **`deleteMethod` missing entirely.** Ground truth (`.http` reference - files, e.g. `Egeria-api-actor-manager.http` lines 1452-1460, 2127-2135, - 2409+) shows real `DeleteElementRequestBody` bodies legitimately - carrying `"deleteMethod": "LOOK_FOR_LINEAGE"`; the pydantic model had no - such field. Same bug class as ISSUE-61's mention of - `DeleteRelationshipRequestBody.delete_method` (added 2026-08-17 for - `OMAG-COMMON-400-032`) — that sibling class got the fix, this one never - did. -2. **`cascadeDelete` was *also* silently dropped, and already - live-impacting.** `validate_delete_element_request`'s no-body-provided - path builds `{"class": "DeleteElementRequestBody", "cascadeDelete": - cascade_delete}` and validates it against the same model. Since the - model had neither field and `PyegeriaModel`'s `extra='ignore'`, both - values vanished before serialization — confirmed directly (no live - server needed): - ```python - >>> DeleteElementRequestBody.model_validate( - ... {'class': 'DeleteElementRequestBody', 'cascadeDelete': True, 'deleteMethod': 'PURGE'} - ... ).model_dump_json(exclude_none=True) - {"forLineage": false, "forDuplicateProcessing": false, "class": "DeleteElementRequestBody"} - ``` - This wasn't hypothetical: every `_async_delete_X` wrapper that threads - `cascade_delete` through `_async_delete_element_request` (e.g. - `DataDesigner.delete_data_structure(guid, cascade_delete=True)`, - exercised in `tests/scenario-tests/test_data_designer_scenarios.py`) has - been silently sending `cascade_delete` as unset every time, regardless - of caller intent — Egeria falls back to its own non-cascading default - instead. - -Checked the adjacent classes for the same gap: `DeleteClassificationRequestBody` -and `OpenMetadataDeleteRequestBody` both correctly have neither field — -matches their real ground-truth bodies, no bug there. The gap was isolated -to `DeleteElementRequestBody`. +### ISSUE-68: Multi-link relationships — relationship GUID silently discarded on create, both in the SDK and in several Dr.Egeria processors -**Fix (deliberately scoped — see "Possible future work" below for what was -declined):** added both fields directly to `DeleteElementRequestBody`, -mirroring `DeleteRelationshipRequestBody`'s existing fix pattern: -- `delete_method: Optional[DeleteMethod] = None` — no spelling ambiguity - found in ground truth for this one, uses the model's normal - `alias_generator` (→ `deleteMethod`). -- `cascade_delete: Optional[bool] = None` with - `validation_alias=AliasChoices("cascadeDelete", "cascadedDelete")` and - `serialization_alias="cascadeDelete"`. Ground-truth `.http` files are - genuinely split roughly 50/50 between `cascadeDelete` and - `cascadedDelete` for this same class — even within a single file - (`Egeria-api-actor-manager.http` uses both). Not resolved here; - `cascadeDelete` was chosen as the *outgoing* name because it's what this - repo's own code (`validate_delete_element_request`'s no-body-provided - branch) has already been trying to send since before this fix. Both - spellings are accepted on input so a caller-supplied dict using either - one still populates correctly. +**Layer:** Pyegeria (`pyegeria/core/_server_client.py`, several +`pyegeria/omvs/*.py` files) + Dr.Egeria +(`md_processing/v2/solution_architect.py`, +`md_processing/v2/governance.py`). -No changes needed to `_server_client.py` — `validate_delete_element_request`'s -existing shape was already correct in intent, it was purely the missing -model fields silently eating the values. Verified: 5 new unit tests -(`test_delete_element_request_body.py`) covering both-spelling validation, -canonical-name serialization, and the actual outgoing JSON body via a -mocked `_async_make_request` on both the no-body-provided and -caller-supplied-dict paths; full `pytest tests/micro-tests/` passes. +**Status:** fixed 2026-08-18 (two commits). -**Possible future work (deliberately not done here):** `_async_delete_element_request` -still has no first-class `delete_method` parameter (only `cascade_delete`) -— a caller wanting to set it today must hand-build a body dict rather than -pass a kwarg, asymmetric with how `cascade_delete` already works. Adding -one would mean threading it through every `_async_delete_X`/`delete_X` -wrapper across most OMVS modules that currently thread `cascade_delete` -(dozens of call sites, each with its own docstring/"Full sample body" -block) — declined for this pass because (a) it's convenience, not a -correctness fix, the model fix alone already unblocks any caller that -passes a body dict or a `DeleteElementRequestBody` instance directly, and -(b) not every delete endpoint necessarily accepts every `DeleteMethod` -value uniformly (`deleteRelationshipInStore` already rejects its own -default `LookForLineage` — see ISSUE-61's live-verification note — so -Egeria's validation isn't uniform across delete routes), and rolling a new -kwarg out to dozens of wrappers without live-verifying each endpoint's -accepted values individually risks wiring up a parameter some routes will -400 on. Also still open: the `cascadeDelete`/`cascadedDelete` spelling -split above was never resolved against a live server per-endpoint — if a -given delete endpoint actually expects `cascadedDelete`, this fix's choice -would need revisiting for that specific endpoint. +**Raised by dwolfson**: "We need to do some updates to Dr.Egeria to +support multi-link relationships... the action verbs and semantics of +multi-link relationships are different and we need both additional +attributes — like a relationship_guid when updating a multi-link +relationship, and we need to display the relationship guid when we +create a new relationship." ---- +**"Multi-link"** means more than one instance of a relationship type may +exist between the same ordered pair of elements (e.g. `DataFlow`, +`Certification`) — as opposed to the common case of at most one instance +per pair. There is no literal boolean field for this in Egeria's type-def +JSON; the real, live-queryable signal is `relationshipCategory` on +`ValidMetadataManager.get_all_relationship_defs()`'s output +(`MULTI_LINK`/`UNI_LINK`/`REVERSIBLE`) — confirmed live: 21 `MULTI_LINK` +types, 169 `UNI_LINK`, 9 `REVERSIBLE`. For a `MULTI_LINK` type, a +relationship's own GUID (not the pair of element GUIDs) is required to +target one specific instance for Update/Detach. -### ISSUE-30: `updateNote` REST operation returns 404 on a live server — misdiagnosed as an Egeria server gap for three separate re-checks; actually a stale URL in both pyegeria and the `.http` ground truth it was checked against +**Root cause:** `_async_new_relationship_request` (the shared helper +behind most `_async_link_X` wrappers) discarded the GUID Egeria's +relationship-create endpoints return +(`{"class": "GUIDResponse", "guid": "..."}`), so every wrapper built on +it was silently unable to report the new relationship's own identity — +even in cases where the *same* OMVS file's Update/Detach methods already +required that GUID as a parameter, meaning the only way to get it was a +separate `find` call after the fact. -**Layer:** ~~Egeria Server~~ **Pyegeria** — corrected. `_server_client.py`'s -`_async_update_note` built `{command_root}feedback-manager/notes/{note_guid}`. -Every prior re-check (2026-08-05, 2026-08-15, 2026-08-18) cross-checked this -exact URL against `Egeria-api-feedback-manager.http`'s `@name updateNote` -worked example and found it matched byte-for-byte — which is why this got -classified as an Egeria server bug (unregistered/unshipped endpoint) rather -than a pyegeria bug for three separate passes. The `.http` reference file -itself was stale/wrong at the time, so "matches ground truth" gave false -confidence. +**Fix, commit 1** (SDK foundation): `_async_new_relationship_request` now +returns `Optional[str]` (the GUID), purely additive — existing callers +that ignore the return value are unaffected. +`_async_link_solution_linking_wire`/`link_solution_linking_wire` +(`solution_architect.py`) now return it, and +`SolutionLinkProcessor.apply_changes()`'s new-`SolutionLinkingWire`-create +branch captures it into `parsed_output["guid"]` and shows "Created wire +{guid} ..." instead of a generic message — mirroring the sibling +"Updated wire {guid} ..." branch. New +`pyegeria/core/relationship_multiplicity.py`: +`async_is_multi_link()`/`async_get_relationship_category()`, backed by +`get_all_relationship_defs()`, cached per server. Live-verified: +`DataFlow=MULTI_LINK`, `SolutionLinkingWire=UNI_LINK`, +`GovernedBy=UNI_LINK`, `Synonym=REVERSIBLE`. -**Status:** fixed 2026-08-18 (Pyegeria — `pyegeria/core/_server_client.py`, -`tests/micro-tests/test_update_note_url.py`). Fixed by dwolfson after -refreshing the local `Egeria-api-feedback-manager.http` copy and noticing -its `updateNote` worked example now shows a different URL: -`feedback-manager/assets/{noteGUID}/update` (the same `assets/{guid}/update` -shape the feedback-manager service uses elsewhere), not -`feedback-manager/notes/{noteGUID}`. `_async_update_note`'s URL construction -corrected to match. - -**Verified live** against `qs-view-server`: created a throwaway -`DataStructure` → `NoteLog` → `Note` (as in every prior re-check), called -`update_note(note_guid, display_name=..., description=...)` — succeeded -(previously 404'd every time). Went further than a bare success check: -fetched the note log's notes back via `get_notes_for_note_log` and -confirmed both `displayName` and `description` actually persisted the new -values, not just a 200 with no real effect. 2 new unit tests -(`test_update_note_url.py`, mocked `_async_make_request`, no live server) -lock in the corrected URL and body shape; full `pytest tests/micro-tests/` -passes. - -**Lesson for future re-verifications:** "the client-side URL matches the -`.http` ground truth exactly" is only as trustworthy as that ground-truth -file itself — this file can drift out of date the same way pyegeria's own -code can, and when both sides of a comparison share the same stale source, -byte-for-byte agreement stops being meaningful evidence. Worth being more -skeptical of a "confirmed matches ground truth, must be server-side" verdict -when the `.http` file hasn't been independently re-verified against a -current server/Swagger spec recently. - ---- - -### ISSUE-63: `MetadataExpert.delete_related_elements()`/`delete_metadata_element()` were missed when the `deleteMethod`-dropped-silently bug was fixed everywhere else — still could never succeed against a stock Egeria server +**Fix, commit 2** (audit of the remaining `MULTI_LINK` types): every +`_async_link_X` wrapper for a type with an existing OMVS +create-relationship method was checked against its own Update/Detach +signature (GUID-based already, in every case checked) and fixed to +return the GUID: `_async_link_agreement_actor`/`_async_link_agreement_item` +(`collection_manager.py`), `_async_add_catalog_target` +(`asset_maker.py` — was already documented as returning a GUID but had a +`"Relationship created"` placeholder string), `_async_link_external_reference`/ +`_async_link_media_reference`/`_async_link_cited_document` +(`external_links.py`), `_async_setup_next_action_process_step` +(`action_author.py`), `_async_link_valid_value_implementation` +(`reference_data.py` — noted exception: Egeria's own REST API for this +one relationship has no relationship-guid-targeted detach endpoint at +all, per `Egeria-api-reference-data.http`, so the GUID is currently only +useful for record-keeping). `GovernanceLinkProcessor.apply_changes()`'s +Certification/License/Agreement T&C create branches now capture and +display the GUID the same way — its own Detach branch already raises +`ValueError` without one. -**Layer:** Pyegeria (`pyegeria/models/models.py`, `pyegeria/omvs/metadata_expert.py`). +**Left unchanged:** `governance_officer.py`'s `_async_certify_element`/ +`_async_license_element` and `classification_explorer.py`'s +`_async_add_certification_to_element`/`_async_add_license_to_element` +already returned the GUID correctly (bypass the broken shared helper) — +only the Dr.Egeria processor above them was discarding it. +`lineage_linker.py`'s `_async_link_lineage`/`_async_link_data_flow` +(covers `DataFlow`/`ControlFlow`/`ProcessCall`/`LineageMapping`/ +`UltimateSource`/`UltimateDestination`) and its Dr.Egeria processor +already had fully correct GUID-based multi-link semantics end-to-end, +confirmed via `git log` (`b17f71e`) — no change needed. +`AssociatedSecurityList`/`DataLineageRelationship`/`NetworkGatewayLink`/ +`SupportedGovernanceService` have no OMVS wrapper implemented at all — +nothing to fix in this commit (`SupportedGovernanceService` was added in +a follow-up commit, see below). -**Status:** fixed 2026-08-18 (Pyegeria — `pyegeria/omvs/metadata_expert.py`, -`tests/micro-tests/test_metadata_expert_delete_methods.py`). Reported by -dwolfson from Resource Explorer's own investigation (full text below, -verbatim from the report). +**Known discrepancy documented, not changed:** `SolutionLinkingWire` is +`UNI_LINK` in the live type registry despite being treated as multi-link +by existing Dr.Egeria code and comments (Egeria PR #9156). Per dwolfson's +explicit direction, `relationshipCategory` is the source of truth going +forward for new/updated detection logic; reconciling +`SolutionLinkingWire`'s existing multi-link-shaped code against that is +left as a follow-up, not addressed here. -**Report as received:** +**Update 2026-08-24:** re-queried the live server while auditing the full +multi-link inventory (see ISSUE-73) — `SolutionLinkingWire` is now +`MULTI_LINK` (25 `MULTI_LINK` types total, up from 21 above). The buggy 6.1 +Egeria patch that should have set this (referenced above) was corrected in +a later patch server-side; Dr.Egeria's existing multi-link-shaped code for +`SolutionLinkingWire` is therefore now consistent with the live server +again, with no client-side change needed. This discrepancy is resolved. -**Status: partially fixed in 6.0.18.3 — re-verified 2026-08-18, `MetadataExpert`'s two delete methods were left behind.** The underlying bug class (a caller-supplied `deleteMethod` silently validating-then-dropping instead of reaching the server, because `PyegeriaModel`'s `extra='ignore'` swallows any field the target model doesn't declare) is now genuinely fixed for essentially every relationship-delete call site in the SDK: a new `DeleteRelationshipRequestBody` (`pyegeria/models/models.py`) declares `delete_method: Optional[DeleteMethod] = None`, and a new `_async_delete_relationship_request()` helper (`_server_client.py`) was wired into ~15 OMVS modules (`asset_maker.py`, `data_designer.py`, `connection_manager.py`, `classification_explorer.py`, `governance_officer.py`, `collection_manager.py`, `project_manager.py`, `glossary_manager.py`, `solution_architect.py`, `actor_manager.py`, `external_links.py`, `reference_data.py`, `lineage_linker.py`, and others). Confirmed live via direct model round-trip on the freshly-installed 6.0.18.3: -```python -DeleteRelationshipRequestBody.model_validate( - {"class": "DeleteRelationshipRequestBody", "deleteMethod": "SOFT_DELETE"} -).model_dump(by_alias=True, exclude_none=True) -# -> {'forLineage': False, 'forDuplicateProcessing': False, -# 'class': 'DeleteRelationshipRequestBody', 'deleteMethod': 'SOFT_DELETE'} # preserved -``` +**Follow-up, commit 3** (`asset_maker.py` — dwolfson asked whether the 4 +missing-wrapper types needed building): checked each of the 4 against a +*live server's OpenAPI spec* (`/v3/api-docs`), not just this repo's +cached `.http` files, since the `.http` files could themselves be stale. +Result: 3 of the 4 (`AssociatedSecurityList`, `DataLineageRelationship`, +`NetworkGatewayLink`) have **no dedicated REST endpoint anywhere** — +`AssociatedSecurityListProperties` exists only as an orphaned OpenAPI +schema referenced by zero paths, and the other two don't appear in the +spec at all. Only the generic `MetadataExpert` endpoint could create +them, which needs the verbose typed `ElementProperties`/`propertyValueMap` +body shape (see the `pyegeria/models/models.py` gotcha at the top of this +file) — a materially different, larger task than wrapping a real +dedicated endpoint, so these 3 were left unbuilt rather than guessing at +undocumented URLs or building against the awkward generic path +speculatively. Separately, `AssociatedSecurityList` **does** have an +orphaned Dr.Egeria compact command (`Link Associated List`, in +`commands_governance_officer_compact.json`) that was never registered in +`dr_egeria.py` — confirmed dead (`--process` would report "no processor +registered"), left as-is since there's still no backing wrapper to +register it against. -**What's still broken:** `MetadataExpert.delete_related_elements()`/`_async_delete_related_elements()` and `delete_metadata_element()`/`_async_delete_metadata_element()` (`pyegeria/omvs/metadata_expert.py`) were **not** migrated to the new pattern — both still declare `body: Optional[dict | OpenMetadataDeleteRequestBody]` and route through the old, still-unfixed `_async_open_metadata_delete_body_request()`/`OpenMetadataDeleteRequestBody` (`models.py`, unchanged: declares only `class_`, no `delete_method` field). Same direct test against 6.0.18.3, same input shape, different outcome: -```python -OpenMetadataDeleteRequestBody.model_validate( - {"class": "OpenMetadataDeleteRequestBody", "deleteMethod": "SOFT_DELETE"} -).model_dump(by_alias=True, exclude_none=True) -# -> {'forLineage': False, 'forDuplicateProcessing': False, 'class': 'OpenMetadataDeleteRequestBody'} -# deleteMethod silently gone — the exact original bug, unchanged. -``` -So `MetadataExpert`'s generic relationship-delete (no dedicated "unlink" method exists for most relationship types, so this is the fallback every caller without a bespoke delete method reaches for) is still unusable end-to-end on a stock server: the server's own default (`LookForLineage`) is still rejected by its own `deleteRelationshipInStore` validation (`OMAG-COMMON-400-032`), and pyegeria still gives no way to override it for this specific class of caller. +The 4th, `SupportedGovernanceService`, **does** have real, dedicated, +GUID-based-for-update/detach endpoints (`Egeria-api-asset-maker.http`'s +own comment already documents it as multi-link: "the same governance +engine may call the same governance service many times... The unique +identifier of the new relationship is returned so it can be updated or +removed later") — built the full wrapper: +`_async_link_supported_governance_service`/`link_supported_governance_service` +(returns the GUID), `_async_update_supported_governance_service`/ +`update_supported_governance_service`, +`_async_detach_supported_governance_service`/ +`detach_supported_governance_service`. Live-verified routing against a +running server (fake GUIDs correctly reach `createRelatedElementsInStore` +and return 404s, not URL/shape errors). No Dr.Egeria compact command +exists for this relationship type — out of scope for this commit, which +is the OMVS wrapper only. -**Where seen:** Resource Explorer's `SurveyDefinitionReader.reconcile_step_links()` (`packages/resource-explorer/resource_explorer/surveyors/survey_definition_reader.py`), deleting a stale/duplicate `Link Next Process Step` relationship left behind by a separate Dr.Egeria non-idempotency issue. +**Follow-up on `ValidValuesImplementation`'s pair-only-API note above:** +re-checked all remaining implemented `MULTI_LINK` types' Detach methods +for GUID- vs pair-based targeting, and found 4 more with the same +Egeria-side limitation as `ValidValuesImplementation` — no +relationship-guid-targeted detach endpoint exists in Egeria's own REST +API at all, confirmed against the `.http` ground truth for each: +`MediaReference`, `ExternalReferenceLink`, `CitedDocumentLink` (all +`Egeria-api-external-links.http`, pair-based `elements/{elementGUID}/ +media-references|external-references|cited-document-references/ +{refGUID}/detach` only) and `AgreementItem` +(`Egeria-api-collection-manager.http` — its sibling `AgreementActor` +*does* have a GUID-based +`agreements/agreement-actors/{agreementActorRelationshipGUID}/detach`, +but `AgreementItem` only has pair-based +`agreements/{agreementGUID}/agreement-items/{agreementItemGUID}/detach`). +For these 5 types total, the create-time GUID this issue's earlier fixes +now surface is real and correctly returned, but there is currently no +API to *use* it for a later Update/Detach — that gap can only be closed +by Egeria adding the missing endpoints, not by anything in pyegeria. -**Workaround in RE** (not upstream — this repo's own calling code, needed until this fix landed): bypass `delete_related_elements()`/pydantic validation entirely for this one call — hit `MetadataExpert`'s own `command_root` + `.../related-elements/{guid}/delete` URL directly via its private `_async_make_request()` transport method with a raw dict body, wrapped in `asyncio.run(...)`. Verified live: the duplicate link deletes successfully with this workaround, `scripts/reconcile_survey_definition_links.py` runs clean, and the previously-unrunnable "Git Statistics Survey" Survey Definition (`GovActionProcess::RepoCoarseScout`) now fetches without the branching-guard exception. RE's own copy of this workaround was left in place, not reverted here — no need to force RE to re-deploy against a specific pyegeria patch version right away. +**Follow-up, Dr.Egeria `Update` commands added** (dwolfson: "do all the +Dr.Egeria upgrades"): confirmed via `build_command_variants` that `Update` +is never auto-generated for a Link-family command — `LINK_VERBS` is +`(Link, Attach, Add, Detach, Unlink, Remove)`, no `Update`; the only +family with a working `Update ` command before this was +Lineage Linker, which hand-adds a separate compact-spec entry. Added the +same pattern (new compact commands via the Spec Editor's REST API, one +new bundle each reusing existing attributes, `refresh_specs` regeneration) +for the 3 relationship types where the OMVS layer's Update is already +GUID-based *and* a Dr.Egeria Link command already exists: `Update +Certification`/`Update License` (`GovernanceLinkProcessor`, registered +explicitly — the family loop otherwise routes any non-Link verb to +`GovernanceProcessor`, the *element* processor, same override pattern as +`Create Embedded Process` above), `Update Next Process Step` +(`ActionProcessStepLinkProcessor`, auto-routed via the family loop's +existing `om_type` special-case). **`Update Agreement Actor` was +considered and dropped**: `AgreementActor`'s OMVS layer only has a +GUID-based *detach*, not update — checked `Egeria-api-collection- +manager.http` directly and Egeria has no `/agreement-actors/{guid}/ +update` endpoint at all, only `attach`/`{guid}/detach`. `CatalogTarget` +is also GUID-based but has no Dr.Egeria command at all yet — net-new +command family, treated as future work, not an "upgrade" to an existing +one. Also fixed the pre-existing gap where `Link Next Process Step` +discarded the GUID `_async_setup_next_action_process_step` now returns +(same pattern as the other Link-branch fixes above) — it's displayed the +same way the other newly-fixed create paths are. -**Fix applied here (2026-08-18):** migrated exactly as the report's own -"Candidate fix" suggested — `_async_delete_related_elements()` now -validates against `DeleteRelationshipRequestBody` via -`_async_delete_relationship_request()` (mirroring the ~15 already-migrated -modules); `_async_delete_metadata_element()` now validates against -`DeleteElementRequestBody` via `_async_delete_element_request()` (mirroring -`_async_archive_metadata_element` right above it in the same file, which -already used `DeleteElementRequestBody`). Both sync wrappers and -`delete_metadata_element`'s signature gained a `cascade_delete` parameter -to match the pattern used everywhere else `DeleteElementRequestBody` is -used. `OpenMetadataDeleteRequestBody` itself is untouched — it's correctly -fieldless per its own real ground truth (confirmed in ISSUE-62's -investigation), just no longer the class these two methods route through. +**Second bug found and fixed while verifying these live** (not part of +the original request, but directly blocked it): +`AsyncBaseCommandProcessor.execute()`'s step 5 — the shared Create↔Update +upsert-transition logic every processor goes through — was **not gated +by `supports_target_element_lookup()`**, unlike steps 1a/3/7 which are. +A relationship-only processor (no target Referenceable element, +`fetch_as_is()` always `None`, no `qualified_name`) fell through step 5's +"doesn't exist anywhere" branch unconditionally and had its verb silently +rewritten `Update` → `Create` — confirmed live via `--validate` on both +the 3 new commands above *and*, independently, the **pre-existing** +`Update Lineage Relationship` (shipped earlier, apparently never +exercised through `--validate`/`--process` with a complete valid +attribute set before now — with only a relationship GUID + label +supplied it fails pre-flight validation first and never reaches this +code path, which is exactly what happened the first time it was tried +here too). Fixed in two parts: `current_qn` is now read unconditionally +(it's referenced later in `execute()` regardless of the flag) but the +existence/rewrite side effects are now wrapped in `if +self.supports_target_element_lookup():`; and +`GovernanceLinkProcessor`/`ActionProcessStepLinkProcessor`/ +`LineageLinkProcessor`/`UpdateLineageRelationshipProcessor` now override +`supports_target_element_lookup()` to return `False`. **Not +comprehensively audited**: every other relationship-only processor in the +codebase (`SolutionLinkProcessor`, `CollectionLinkProcessor`, +`CurationLinkProcessor`, `ActorManagerLinkProcessor`, +`FeedbackLinkProcessor`, `TermRelationshipProcessor`, +`ProjectLinkProcessor`, ...) has this same latent exposure, but none of +them currently register an `Update`-verb command, so the bug is dormant +for them today — flagged here as a structural risk for whoever adds the +next relationship-`Update` command, not fixed preemptively. -**Verified live** against `qs-view-server`: created two throwaway assets -and a real `DataFlow` relationship between them. -`delete_related_elements(rel_guid)` with no body still fails with the same -`OMAG-COMMON-400-032`-class 500 (expected and correct — Egeria's own -default `deleteMethod` is still rejected by this endpoint; that half of -the behavior is unchanged and not this fix's job to change). -`delete_related_elements(rel_guid, {"deleteMethod": "SOFT_DELETE"})` -**succeeded** — the override that was previously impossible now works -end-to-end. `delete_metadata_element()` on both throwaway assets also -succeeded. 6 new unit tests -(`test_metadata_expert_delete_methods.py`) cover both methods' outgoing -request bodies (mocked, no live server) plus a direct model round-trip -matching the report's own verification snippets; full -`pytest tests/micro-tests/` passes. +**Tests:** `test_relationship_multiplicity.py` (7 tests, the detection +utility), `test_solution_linking_wire_multilink.py` (2 tests), +`test_governance_link_multilink_guid.py` (2 tests), +`test_supported_governance_service.py` (3 tests), +`test_multilink_update_commands.py` (6 tests, the 3 new Update commands) +— all with fake clients, no live server required. Full `pytest +tests/micro-tests/` green throughout. `relationship_multiplicity`, +`_async_link_supported_governance_service`, and all 3 new `Update` +commands (plus the pre-existing `Update Lineage Relationship`, to +confirm the shared-code fix) also live-verified via `--validate` against +a running server — each now correctly keeps `verb=Update` through +`execute()` instead of silently becoming `Create`. --- -### ISSUE-66: `Create Design Pattern` had no `Usage` attribute at all +### ISSUE-67: `pyegeria/view/base_report_formats.py` had a dead, shadowed `select_report_spec` function plus a real ambiguous-lookup bug it was masking -**Layer:** Pyegeria/Dr.Egeria (`md_processing/data/compact_commands/commands_solution_architect_compact.json`, `md_processing/md_processing_utils/common_md_utils.py`). - -**Status:** fixed 2026-08-18 (Pyegeria/Dr.Egeria — compact spec via the -Dr.Egeria Spec Editor's REST API, `common_md_utils.py`, -`tests/micro-tests/test_design_pattern_usage.py`). Reported by dwolfson. - -**What was missing:** `usage` is a real `DesignPatternProperties` field — -confirmed in `Egeria-api-solution-architect.http`'s -`createDesignPattern`/`updateDesignPattern` worked examples, right -alongside `context`/`problemStatement`/`solutionDescription`/etc., which -`Create Design Pattern` already supported. But the "Design Pattern Base" -bundle's `own_attributes` never declared a `Usage` attribute at all, and -`set_solution_architect_body()`'s `DesignPattern` branch (the function that -builds the outgoing properties for both Create and Update) never included -`usage` in what it constructs. Unlike most gaps in this tracker, this -wasn't even a silent-drop: a `### Usage` block in a Dr.Egeria markdown file -would have failed `--validate` outright as an undeclared attribute — the -property was simply unreachable from Dr.Egeria, not misrouted once -supplied. - -**Fix:** added `Usage` to the "Design Pattern Base" bundle's -`own_attributes` via the Spec Editor API (reusing the existing shared -`Usage` attribute definition already used elsewhere in this same family — -no new attribute definition needed, `style: Simple`). Added -`"usage": attributes.get('Usage', {}).get('value', None)` to -`set_solution_architect_body()`'s `DesignPattern` branch, alongside its -sibling fields. `refresh_specs` regenerated both basic/advanced -`Create_Design_Pattern.md` templates correctly; -`validate_compact_specs` still reports 0 errors (same 26 pre-existing -warnings, none new). - -**Verified live** against `qs-view-server`: created a real `DesignPattern` -with `Usage` set, fetched it back — `usage` persisted correctly. 3 new -unit tests (`test_design_pattern_usage.py`, pure function test against -`set_solution_architect_body`, no live server needed) cover the happy -path, the unset-is-None case, and confirm `SolutionBlueprint` (which has -no such field on its own real properties class) doesn't pick up a stray -`usage` key. Full `pytest tests/micro-tests/` passes. - ---- - -### ISSUE-67: `pyegeria/view/base_report_formats.py` had a dead, shadowed `select_report_spec` function plus a real ambiguous-lookup bug it was masking - -**Layer:** Pyegeria (`pyegeria/view/base_report_formats.py`). +**Layer:** Pyegeria (`pyegeria/view/base_report_formats.py`). **Status:** fixed 2026-08-18 (Pyegeria — `pyegeria/view/base_report_formats.py`, @@ -5786,286 +5337,100 @@ individually. --- -### ISSUE-68: Multi-link relationships — relationship GUID silently discarded on create, both in the SDK and in several Dr.Egeria processors +### ISSUE-66: `Create Design Pattern` had no `Usage` attribute at all -**Layer:** Pyegeria (`pyegeria/core/_server_client.py`, several -`pyegeria/omvs/*.py` files) + Dr.Egeria -(`md_processing/v2/solution_architect.py`, -`md_processing/v2/governance.py`). +**Layer:** Pyegeria/Dr.Egeria (`md_processing/data/compact_commands/commands_solution_architect_compact.json`, `md_processing/md_processing_utils/common_md_utils.py`). -**Status:** fixed 2026-08-18 (two commits). +**Status:** fixed 2026-08-18 (Pyegeria/Dr.Egeria — compact spec via the +Dr.Egeria Spec Editor's REST API, `common_md_utils.py`, +`tests/micro-tests/test_design_pattern_usage.py`). Reported by dwolfson. -**Raised by dwolfson**: "We need to do some updates to Dr.Egeria to -support multi-link relationships... the action verbs and semantics of -multi-link relationships are different and we need both additional -attributes — like a relationship_guid when updating a multi-link -relationship, and we need to display the relationship guid when we -create a new relationship." +**What was missing:** `usage` is a real `DesignPatternProperties` field — +confirmed in `Egeria-api-solution-architect.http`'s +`createDesignPattern`/`updateDesignPattern` worked examples, right +alongside `context`/`problemStatement`/`solutionDescription`/etc., which +`Create Design Pattern` already supported. But the "Design Pattern Base" +bundle's `own_attributes` never declared a `Usage` attribute at all, and +`set_solution_architect_body()`'s `DesignPattern` branch (the function that +builds the outgoing properties for both Create and Update) never included +`usage` in what it constructs. Unlike most gaps in this tracker, this +wasn't even a silent-drop: a `### Usage` block in a Dr.Egeria markdown file +would have failed `--validate` outright as an undeclared attribute — the +property was simply unreachable from Dr.Egeria, not misrouted once +supplied. -**"Multi-link"** means more than one instance of a relationship type may -exist between the same ordered pair of elements (e.g. `DataFlow`, -`Certification`) — as opposed to the common case of at most one instance -per pair. There is no literal boolean field for this in Egeria's type-def -JSON; the real, live-queryable signal is `relationshipCategory` on -`ValidMetadataManager.get_all_relationship_defs()`'s output -(`MULTI_LINK`/`UNI_LINK`/`REVERSIBLE`) — confirmed live: 21 `MULTI_LINK` -types, 169 `UNI_LINK`, 9 `REVERSIBLE`. For a `MULTI_LINK` type, a -relationship's own GUID (not the pair of element GUIDs) is required to -target one specific instance for Update/Detach. +**Fix:** added `Usage` to the "Design Pattern Base" bundle's +`own_attributes` via the Spec Editor API (reusing the existing shared +`Usage` attribute definition already used elsewhere in this same family — +no new attribute definition needed, `style: Simple`). Added +`"usage": attributes.get('Usage', {}).get('value', None)` to +`set_solution_architect_body()`'s `DesignPattern` branch, alongside its +sibling fields. `refresh_specs` regenerated both basic/advanced +`Create_Design_Pattern.md` templates correctly; +`validate_compact_specs` still reports 0 errors (same 26 pre-existing +warnings, none new). -**Root cause:** `_async_new_relationship_request` (the shared helper -behind most `_async_link_X` wrappers) discarded the GUID Egeria's -relationship-create endpoints return -(`{"class": "GUIDResponse", "guid": "..."}`), so every wrapper built on -it was silently unable to report the new relationship's own identity — -even in cases where the *same* OMVS file's Update/Detach methods already -required that GUID as a parameter, meaning the only way to get it was a -separate `find` call after the fact. +**Verified live** against `qs-view-server`: created a real `DesignPattern` +with `Usage` set, fetched it back — `usage` persisted correctly. 3 new +unit tests (`test_design_pattern_usage.py`, pure function test against +`set_solution_architect_body`, no live server needed) cover the happy +path, the unset-is-None case, and confirm `SolutionBlueprint` (which has +no such field on its own real properties class) doesn't pick up a stray +`usage` key. Full `pytest tests/micro-tests/` passes. -**Fix, commit 1** (SDK foundation): `_async_new_relationship_request` now -returns `Optional[str]` (the GUID), purely additive — existing callers -that ignore the return value are unaffected. -`_async_link_solution_linking_wire`/`link_solution_linking_wire` -(`solution_architect.py`) now return it, and -`SolutionLinkProcessor.apply_changes()`'s new-`SolutionLinkingWire`-create -branch captures it into `parsed_output["guid"]` and shows "Created wire -{guid} ..." instead of a generic message — mirroring the sibling -"Updated wire {guid} ..." branch. New -`pyegeria/core/relationship_multiplicity.py`: -`async_is_multi_link()`/`async_get_relationship_category()`, backed by -`get_all_relationship_defs()`, cached per server. Live-verified: -`DataFlow=MULTI_LINK`, `SolutionLinkingWire=UNI_LINK`, -`GovernedBy=UNI_LINK`, `Synonym=REVERSIBLE`. +--- -**Fix, commit 2** (audit of the remaining `MULTI_LINK` types): every -`_async_link_X` wrapper for a type with an existing OMVS -create-relationship method was checked against its own Update/Detach -signature (GUID-based already, in every case checked) and fixed to -return the GUID: `_async_link_agreement_actor`/`_async_link_agreement_item` -(`collection_manager.py`), `_async_add_catalog_target` -(`asset_maker.py` — was already documented as returning a GUID but had a -`"Relationship created"` placeholder string), `_async_link_external_reference`/ -`_async_link_media_reference`/`_async_link_cited_document` -(`external_links.py`), `_async_setup_next_action_process_step` -(`action_author.py`), `_async_link_valid_value_implementation` -(`reference_data.py` — noted exception: Egeria's own REST API for this -one relationship has no relationship-guid-targeted detach endpoint at -all, per `Egeria-api-reference-data.http`, so the GUID is currently only -useful for record-keeping). `GovernanceLinkProcessor.apply_changes()`'s -Certification/License/Agreement T&C create branches now capture and -display the GUID the same way — its own Detach branch already raises -`ValueError` without one. +### ISSUE-64: `Create Information Supply Chain`'s `Purposes`/`Scope` attributes were silently dropped — never read from `attributes` at all -**Left unchanged:** `governance_officer.py`'s `_async_certify_element`/ -`_async_license_element` and `classification_explorer.py`'s -`_async_add_certification_to_element`/`_async_add_license_to_element` -already returned the GUID correctly (bypass the broken shared helper) — -only the Dr.Egeria processor above them was discarding it. -`lineage_linker.py`'s `_async_link_lineage`/`_async_link_data_flow` -(covers `DataFlow`/`ControlFlow`/`ProcessCall`/`LineageMapping`/ -`UltimateSource`/`UltimateDestination`) and its Dr.Egeria processor -already had fully correct GUID-based multi-link semantics end-to-end, -confirmed via `git log` (`b17f71e`) — no change needed. -`AssociatedSecurityList`/`DataLineageRelationship`/`NetworkGatewayLink`/ -`SupportedGovernanceService` have no OMVS wrapper implemented at all — -nothing to fix in this commit (`SupportedGovernanceService` was added in -a follow-up commit, see below). +**Status:** fixed 2026-08-18 (Pyegeria/Dr.Egeria — +`md_processing/v2/solution_architect.py`, +`tests/micro-tests/test_supply_chain_processor.py`). **Renumbered twice +from collisions**: originally logged as `ISSUE-62` by dwolfson, colliding +with this file's own `ISSUE-62` (`DeleteElementRequestBody`, above) added +the same day — renumbered to `ISSUE-63`. Then a second, real `ISSUE-63` +arrived the same day from Resource Explorer (below — `MetadataExpert`'s +delete methods missing the `deleteMethod` fix) with its number already +fixed by that report's own author, so this entry moved again to `ISSUE-64` +per this file's established collision convention (see the "Renumbered +2026-08-15" note near the top) — kept the number on whichever entry's +number was harder to change at the point of collision. -**Known discrepancy documented, not changed:** `SolutionLinkingWire` is -`UNI_LINK` in the live type registry despite being treated as multi-link -by existing Dr.Egeria code and comments (Egeria PR #9156). Per dwolfson's -explicit direction, `relationshipCategory` is the source of truth going -forward for new/updated detection logic; reconciling -`SolutionLinkingWire`'s existing multi-link-shaped code against that is -left as a follow-up, not addressed here. +**Originally found** 2026-08-18 building `gen_governance_metrics.py` +(egeria-workspaces-fs, `OVERVIEW_GOVERNANCE_METRICS.dr-egeria.md`) while +representing a metric's conceptual data-flow as an `InformationSupplyChain`. +`--validate`/`--process` both reported `SUCCESS` with `### Purposes` set, +but the created element never persisted it — created 17 ISCs with a real +`Purposes` value, none retained it. -**Update 2026-08-24:** re-queried the live server while auditing the full -multi-link inventory (see ISSUE-73) — `SolutionLinkingWire` is now -`MULTI_LINK` (25 `MULTI_LINK` types total, up from 21 above). The buggy 6.1 -Egeria patch that should have set this (referenced above) was corrected in -a later patch server-side; Dr.Egeria's existing multi-link-shaped code for -`SolutionLinkingWire` is therefore now consistent with the live server -again, with no client-side change needed. This discrepancy is resolved. +**Root cause, found and fixed:** `SupplyChainProcessor.apply_changes()` +(`md_processing/v2/solution_architect.py`) built the create/update +properties body purely from `set_element_prop_body()` — the generic +`Referenceable`-level builder, which has no knowledge of +`InformationSupplyChainProperties`-specific fields — and never added +`Purposes`/`Scope` on top, unlike sibling domain processors (e.g. +`set_collection_manager_body` adding Digital-Product-specific fields). +So `--validate`/`--process` reporting `SUCCESS` was accurate for what the +processor actually did (create the element with valid base properties); +`Purposes`/`Scope` were just never included in the outgoing body at all. -**Follow-up, commit 3** (`asset_maker.py` — dwolfson asked whether the 4 -missing-wrapper types needed building): checked each of the 4 against a -*live server's OpenAPI spec* (`/v3/api-docs`), not just this repo's -cached `.http` files, since the `.http` files could themselves be stale. -Result: 3 of the 4 (`AssociatedSecurityList`, `DataLineageRelationship`, -`NetworkGatewayLink`) have **no dedicated REST endpoint anywhere** — -`AssociatedSecurityListProperties` exists only as an orphaned OpenAPI -schema referenced by zero paths, and the other two don't appear in the -spec at all. Only the generic `MetadataExpert` endpoint could create -them, which needs the verbose typed `ElementProperties`/`propertyValueMap` -body shape (see the `pyegeria/models/models.py` gotcha at the top of this -file) — a materially different, larger task than wrapping a real -dedicated endpoint, so these 3 were left unbuilt rather than guessing at -undocumented URLs or building against the awkward generic path -speculatively. Separately, `AssociatedSecurityList` **does** have an -orphaned Dr.Egeria compact command (`Link Associated List`, in -`commands_governance_officer_compact.json`) that was never registered in -`dr_egeria.py` — confirmed dead (`--process` would report "no processor -registered"), left as-is since there's still no backing wrapper to -register it against. +**Also found while root-causing:** the real Egeria wire property is +**`dataProcessingPurposes`**, not `purposes` (confirmed against +`Egeria-api-solution-architect.http`'s `createInformationSupplyChain`/ +`updateInformationSupplyChain` worked examples) — the compact spec's +`Purposes` attribute has no `property_name` override pointing at this, so +even a naive "just pass `attributes` through" fix would have sent the +wrong key. `Scope` matches its attribute name exactly. -The 4th, `SupportedGovernanceService`, **does** have real, dedicated, -GUID-based-for-update/detach endpoints (`Egeria-api-asset-maker.http`'s -own comment already documents it as multi-link: "the same governance -engine may call the same governance service many times... The unique -identifier of the new relationship is returned so it can be updated or -removed later") — built the full wrapper: -`_async_link_supported_governance_service`/`link_supported_governance_service` -(returns the GUID), `_async_update_supported_governance_service`/ -`update_supported_governance_service`, -`_async_detach_supported_governance_service`/ -`detach_supported_governance_service`. Live-verified routing against a -running server (fake GUIDs correctly reach `createRelatedElementsInStore` -and return 404s, not URL/shape errors). No Dr.Egeria compact command -exists for this relationship type — out of scope for this commit, which -is the OMVS wrapper only. - -**Follow-up on `ValidValuesImplementation`'s pair-only-API note above:** -re-checked all remaining implemented `MULTI_LINK` types' Detach methods -for GUID- vs pair-based targeting, and found 4 more with the same -Egeria-side limitation as `ValidValuesImplementation` — no -relationship-guid-targeted detach endpoint exists in Egeria's own REST -API at all, confirmed against the `.http` ground truth for each: -`MediaReference`, `ExternalReferenceLink`, `CitedDocumentLink` (all -`Egeria-api-external-links.http`, pair-based `elements/{elementGUID}/ -media-references|external-references|cited-document-references/ -{refGUID}/detach` only) and `AgreementItem` -(`Egeria-api-collection-manager.http` — its sibling `AgreementActor` -*does* have a GUID-based -`agreements/agreement-actors/{agreementActorRelationshipGUID}/detach`, -but `AgreementItem` only has pair-based -`agreements/{agreementGUID}/agreement-items/{agreementItemGUID}/detach`). -For these 5 types total, the create-time GUID this issue's earlier fixes -now surface is real and correctly returned, but there is currently no -API to *use* it for a later Update/Detach — that gap can only be closed -by Egeria adding the missing endpoints, not by anything in pyegeria. - -**Follow-up, Dr.Egeria `Update` commands added** (dwolfson: "do all the -Dr.Egeria upgrades"): confirmed via `build_command_variants` that `Update` -is never auto-generated for a Link-family command — `LINK_VERBS` is -`(Link, Attach, Add, Detach, Unlink, Remove)`, no `Update`; the only -family with a working `Update ` command before this was -Lineage Linker, which hand-adds a separate compact-spec entry. Added the -same pattern (new compact commands via the Spec Editor's REST API, one -new bundle each reusing existing attributes, `refresh_specs` regeneration) -for the 3 relationship types where the OMVS layer's Update is already -GUID-based *and* a Dr.Egeria Link command already exists: `Update -Certification`/`Update License` (`GovernanceLinkProcessor`, registered -explicitly — the family loop otherwise routes any non-Link verb to -`GovernanceProcessor`, the *element* processor, same override pattern as -`Create Embedded Process` above), `Update Next Process Step` -(`ActionProcessStepLinkProcessor`, auto-routed via the family loop's -existing `om_type` special-case). **`Update Agreement Actor` was -considered and dropped**: `AgreementActor`'s OMVS layer only has a -GUID-based *detach*, not update — checked `Egeria-api-collection- -manager.http` directly and Egeria has no `/agreement-actors/{guid}/ -update` endpoint at all, only `attach`/`{guid}/detach`. `CatalogTarget` -is also GUID-based but has no Dr.Egeria command at all yet — net-new -command family, treated as future work, not an "upgrade" to an existing -one. Also fixed the pre-existing gap where `Link Next Process Step` -discarded the GUID `_async_setup_next_action_process_step` now returns -(same pattern as the other Link-branch fixes above) — it's displayed the -same way the other newly-fixed create paths are. - -**Second bug found and fixed while verifying these live** (not part of -the original request, but directly blocked it): -`AsyncBaseCommandProcessor.execute()`'s step 5 — the shared Create↔Update -upsert-transition logic every processor goes through — was **not gated -by `supports_target_element_lookup()`**, unlike steps 1a/3/7 which are. -A relationship-only processor (no target Referenceable element, -`fetch_as_is()` always `None`, no `qualified_name`) fell through step 5's -"doesn't exist anywhere" branch unconditionally and had its verb silently -rewritten `Update` → `Create` — confirmed live via `--validate` on both -the 3 new commands above *and*, independently, the **pre-existing** -`Update Lineage Relationship` (shipped earlier, apparently never -exercised through `--validate`/`--process` with a complete valid -attribute set before now — with only a relationship GUID + label -supplied it fails pre-flight validation first and never reaches this -code path, which is exactly what happened the first time it was tried -here too). Fixed in two parts: `current_qn` is now read unconditionally -(it's referenced later in `execute()` regardless of the flag) but the -existence/rewrite side effects are now wrapped in `if -self.supports_target_element_lookup():`; and -`GovernanceLinkProcessor`/`ActionProcessStepLinkProcessor`/ -`LineageLinkProcessor`/`UpdateLineageRelationshipProcessor` now override -`supports_target_element_lookup()` to return `False`. **Not -comprehensively audited**: every other relationship-only processor in the -codebase (`SolutionLinkProcessor`, `CollectionLinkProcessor`, -`CurationLinkProcessor`, `ActorManagerLinkProcessor`, -`FeedbackLinkProcessor`, `TermRelationshipProcessor`, -`ProjectLinkProcessor`, ...) has this same latent exposure, but none of -them currently register an `Update`-verb command, so the bug is dormant -for them today — flagged here as a structural risk for whoever adds the -next relationship-`Update` command, not fixed preemptively. - -**Tests:** `test_relationship_multiplicity.py` (7 tests, the detection -utility), `test_solution_linking_wire_multilink.py` (2 tests), -`test_governance_link_multilink_guid.py` (2 tests), -`test_supported_governance_service.py` (3 tests), -`test_multilink_update_commands.py` (6 tests, the 3 new Update commands) -— all with fake clients, no live server required. Full `pytest -tests/micro-tests/` green throughout. `relationship_multiplicity`, -`_async_link_supported_governance_service`, and all 3 new `Update` -commands (plus the pre-existing `Update Lineage Relationship`, to -confirm the shared-code fix) also live-verified via `--validate` against -a running server — each now correctly keeps `verb=Update` through -`execute()` instead of silently becoming `Create`. - -### ISSUE-64: `Create Information Supply Chain`'s `Purposes`/`Scope` attributes were silently dropped — never read from `attributes` at all - -**Status:** fixed 2026-08-18 (Pyegeria/Dr.Egeria — -`md_processing/v2/solution_architect.py`, -`tests/micro-tests/test_supply_chain_processor.py`). **Renumbered twice -from collisions**: originally logged as `ISSUE-62` by dwolfson, colliding -with this file's own `ISSUE-62` (`DeleteElementRequestBody`, above) added -the same day — renumbered to `ISSUE-63`. Then a second, real `ISSUE-63` -arrived the same day from Resource Explorer (below — `MetadataExpert`'s -delete methods missing the `deleteMethod` fix) with its number already -fixed by that report's own author, so this entry moved again to `ISSUE-64` -per this file's established collision convention (see the "Renumbered -2026-08-15" note near the top) — kept the number on whichever entry's -number was harder to change at the point of collision. - -**Originally found** 2026-08-18 building `gen_governance_metrics.py` -(egeria-workspaces-fs, `OVERVIEW_GOVERNANCE_METRICS.dr-egeria.md`) while -representing a metric's conceptual data-flow as an `InformationSupplyChain`. -`--validate`/`--process` both reported `SUCCESS` with `### Purposes` set, -but the created element never persisted it — created 17 ISCs with a real -`Purposes` value, none retained it. - -**Root cause, found and fixed:** `SupplyChainProcessor.apply_changes()` -(`md_processing/v2/solution_architect.py`) built the create/update -properties body purely from `set_element_prop_body()` — the generic -`Referenceable`-level builder, which has no knowledge of -`InformationSupplyChainProperties`-specific fields — and never added -`Purposes`/`Scope` on top, unlike sibling domain processors (e.g. -`set_collection_manager_body` adding Digital-Product-specific fields). -So `--validate`/`--process` reporting `SUCCESS` was accurate for what the -processor actually did (create the element with valid base properties); -`Purposes`/`Scope` were just never included in the outgoing body at all. - -**Also found while root-causing:** the real Egeria wire property is -**`dataProcessingPurposes`**, not `purposes` (confirmed against -`Egeria-api-solution-architect.http`'s `createInformationSupplyChain`/ -`updateInformationSupplyChain` worked examples) — the compact spec's -`Purposes` attribute has no `property_name` override pointing at this, so -even a naive "just pass `attributes` through" fix would have sent the -wrong key. `Scope` matches its attribute name exactly. - -**Fix:** `SupplyChainProcessor.apply_changes()` now explicitly sets -`prop_body["dataProcessingPurposes"]` (from `Purposes`, a `Simple List` -style attribute — value is already a Python list) and `prop_body["scope"]` -(from `Scope`) on both the Create and Update paths, right after the -generic `set_element_prop_body()` call. Verified live: created a real ISC -with both fields set, fetched it back — `dataProcessingPurposes` and -`scope` both persisted correctly. 3 new unit tests -(`test_supply_chain_processor.py`) cover Create, Update, and the -None-when-unset case via a fake client capturing the outgoing body; full -`pytest tests/micro-tests/` passes. +**Fix:** `SupplyChainProcessor.apply_changes()` now explicitly sets +`prop_body["dataProcessingPurposes"]` (from `Purposes`, a `Simple List` +style attribute — value is already a Python list) and `prop_body["scope"]` +(from `Scope`) on both the Create and Update paths, right after the +generic `set_element_prop_body()` call. Verified live: created a real ISC +with both fields set, fetched it back — `dataProcessingPurposes` and +`scope` both persisted correctly. 3 new unit tests +(`test_supply_chain_processor.py`) cover Create, Update, and the +None-when-unset case via a fake client capturing the outgoing body; full +`pytest tests/micro-tests/` passes. **Not resolved, flagged for a follow-up investigation:** `Integration Style` and `Estimated Volumetrics` are also declared in the `ISC Base` @@ -6088,83 +5453,215 @@ before assuming either explanation. --- -### ISSUE-57: `GovernanceResults` relationship rejects the exact end1/end2 GUID order the `.http` ground truth and pyegeria's URL both use +### ISSUE-63: `MetadataExpert.delete_related_elements()`/`delete_metadata_element()` were missed when the `deleteMethod`-dropped-silently bug was fixed everywhere else — still could never succeed against a stock Egeria server -**Status:** fixed server-side, confirmed 2026-08-18. Found 2026-08-15 as a -byproduct of live-verifying the ISSUE-27 fix (`detach_governance_results`'s -duplicate-argument bug). Re-verified as part of a full open-issue sweep -prompted by dwolfson's recent bug-fix pass — created a throwaway -`GovernanceMetric` + `DataSet` asset live against `qs-view-server` and -called `GovernanceOfficer.link_governance_results(metric_guid, dataset_guid)` -directly: **succeeded**, no 500, relationship created correctly. Cleaned up -both elements afterward. No pyegeria code change involved — this was -purely a server-side fix/redeploy between 2026-08-15 and now. +**Layer:** Pyegeria (`pyegeria/models/models.py`, `pyegeria/omvs/metadata_expert.py`). -**Original finding (2026-08-15):** calling -`GovernanceOfficer._async_link_governance_results(governance_metric_guid, -data_asset_guid)` — which builds -`POST .../governance-metrics/{governanceMetricGUID}/measurements/{dataAssetGUID}/attach`, -matching `Egeria-api-governance-officer.http`'s `linkGovernanceResults` -worked example exactly — failed with a 500 -(`OMRS-REPOSITORY-400-047`: end1 assigned to the *second* URL path GUID and -end2 to the *first*, the reverse of what the `GovernanceResults` -relationship type and the URL's own naming both imply). Confirmed at the -time nothing was fixable client-side; the server has since been corrected -(or the underlying type/endpoint definition fixed), consistent with the -`Link Governance Results` Dr.Egeria command (ISSUE-61, closed 2026-08-17) -also live-verifying successfully around the same window. +**Status:** fixed 2026-08-18 (Pyegeria — `pyegeria/omvs/metadata_expert.py`, +`tests/micro-tests/test_metadata_expert_delete_methods.py`). Reported by +dwolfson from Resource Explorer's own investigation (full text below, +verbatim from the report). ---- +**Report as received:** -### ISSUE-53: `findMetadataElements`'s `metadataElementSubtypeNames` is silently ignored — restricting to specific subtypes has no effect on results +**Status: partially fixed in 6.0.18.3 — re-verified 2026-08-18, `MetadataExpert`'s two delete methods were left behind.** The underlying bug class (a caller-supplied `deleteMethod` silently validating-then-dropping instead of reaching the server, because `PyegeriaModel`'s `extra='ignore'` swallows any field the target model doesn't declare) is now genuinely fixed for essentially every relationship-delete call site in the SDK: a new `DeleteRelationshipRequestBody` (`pyegeria/models/models.py`) declares `delete_method: Optional[DeleteMethod] = None`, and a new `_async_delete_relationship_request()` helper (`_server_client.py`) was wired into ~15 OMVS modules (`asset_maker.py`, `data_designer.py`, `connection_manager.py`, `classification_explorer.py`, `governance_officer.py`, `collection_manager.py`, `project_manager.py`, `glossary_manager.py`, `solution_architect.py`, `actor_manager.py`, `external_links.py`, `reference_data.py`, `lineage_linker.py`, and others). Confirmed live via direct model round-trip on the freshly-installed 6.0.18.3: +```python +DeleteRelationshipRequestBody.model_validate( + {"class": "DeleteRelationshipRequestBody", "deleteMethod": "SOFT_DELETE"} +).model_dump(by_alias=True, exclude_none=True) +# -> {'forLineage': False, 'forDuplicateProcessing': False, +# 'class': 'DeleteRelationshipRequestBody', 'deleteMethod': 'SOFT_DELETE'} # preserved +``` -**Status:** fixed server-side, confirmed 2026-08-18. Re-verified as part of -a full open-issue sweep: `MetadataExpert.find_metadata_elements` with -`metadataElementTypeName: "Referenceable"` + -`metadataElementSubtypeNames: ["GlossaryTerm"]` now correctly returns only -`GlossaryTerm` elements (confirmed: 10/10 results in a sample page all -`typeName: "GlossaryTerm"`, versus the previous mixed-type-junk behavior). -Cross-checked via the native count endpoint too: -`count_metadata_elements` with the same subtype filter returns **357**, -identical to a direct `metadataElementTypeName: "GlossaryTerm"` count with -no subtype filter at all — exact agreement, not just "closer." No pyegeria -code change involved — purely a server-side fix/redeploy since the -2026-08-05/2026-08-15 findings below. +**What's still broken:** `MetadataExpert.delete_related_elements()`/`_async_delete_related_elements()` and `delete_metadata_element()`/`_async_delete_metadata_element()` (`pyegeria/omvs/metadata_expert.py`) were **not** migrated to the new pattern — both still declare `body: Optional[dict | OpenMetadataDeleteRequestBody]` and route through the old, still-unfixed `_async_open_metadata_delete_body_request()`/`OpenMetadataDeleteRequestBody` (`models.py`, unchanged: declares only `class_`, no `delete_method` field). Same direct test against 6.0.18.3, same input shape, different outcome: +```python +OpenMetadataDeleteRequestBody.model_validate( + {"class": "OpenMetadataDeleteRequestBody", "deleteMethod": "SOFT_DELETE"} +).model_dump(by_alias=True, exclude_none=True) +# -> {'forLineage': False, 'forDuplicateProcessing': False, 'class': 'OpenMetadataDeleteRequestBody'} +# deleteMethod silently gone — the exact original bug, unchanged. +``` +So `MetadataExpert`'s generic relationship-delete (no dedicated "unlink" method exists for most relationship types, so this is the fallback every caller without a bespoke delete method reaches for) is still unusable end-to-end on a stock server: the server's own default (`LookForLineage`) is still rejected by its own `deleteRelationshipInStore` validation (`OMAG-COMMON-400-032`), and pyegeria still gives no way to override it for this specific class of caller. -**Original finding:** found 2026-08-05 investigating whether Egeria -Insights could express "elements that have a SemanticAssignment but are NOT -type Notification" via an allow-list of subtypes (`Referenceable` + -`metadataElementSubtypeNames: ["GlossaryTerm", "DataAsset"]`, per -`Egeria-api-metadata-expert.http`'s worked "findMetadataElements (nested -condition)" example). `metadataElementSubtypeNames` was accepted (HTTP 200, -no error) but had zero effect on results — byte-identical to the same -query with the field omitted, reproduced across three different -base-type/subtype pairs and re-confirmed again 2026-08-15 against a -restarted `qs-view-server`. Blocked the Egeria Insights use case that -prompted the investigation (excluding noisy `Notification`/`ToDo`/ -`Meeting`/`Review` `Asset` subtypes from a broader search) — that use case -is now unblocked as a byproduct of this fix. See ISSUE-55 (now also -fixed, 2026-08-18) for the true exclude/NOT semantics (`skipSubtypes`) -that arrived shortly after this fix, in the same Egeria PR that reworked -this code path. +**Where seen:** Resource Explorer's `SurveyDefinitionReader.reconcile_step_links()` (`packages/resource-explorer/resource_explorer/surveyors/survey_definition_reader.py`), deleting a stale/duplicate `Link Next Process Step` relationship left behind by a separate Dr.Egeria non-idempotency issue. ---- +**Workaround in RE** (not upstream — this repo's own calling code, needed until this fix landed): bypass `delete_related_elements()`/pydantic validation entirely for this one call — hit `MetadataExpert`'s own `command_root` + `.../related-elements/{guid}/delete` URL directly via its private `_async_make_request()` transport method with a raw dict body, wrapped in `asyncio.run(...)`. Verified live: the duplicate link deletes successfully with this workaround, `scripts/reconcile_survey_definition_links.py` runs clean, and the previously-unrunnable "Git Statistics Survey" Survey Definition (`GovActionProcess::RepoCoarseScout`) now fetches without the branching-guard exception. RE's own copy of this workaround was left in place, not reverted here — no need to force RE to re-deploy against a specific pyegeria patch version right away. -### ISSUE-60: `find_glossary_terms`'s `sequencing_order`/`sequencing_property` sorts only the fetched page, not the full result set — a "fetch one page, sort in JS" pattern silently drops alphabetically-early/late terms once the collection exceeds the page +**Fix applied here (2026-08-18):** migrated exactly as the report's own +"Candidate fix" suggested — `_async_delete_related_elements()` now +validates against `DeleteRelationshipRequestBody` via +`_async_delete_relationship_request()` (mirroring the ~15 already-migrated +modules); `_async_delete_metadata_element()` now validates against +`DeleteElementRequestBody` via `_async_delete_element_request()` (mirroring +`_async_archive_metadata_element` right above it in the same file, which +already used `DeleteElementRequestBody`). Both sync wrappers and +`delete_metadata_element`'s signature gained a `cascade_delete` parameter +to match the pattern used everywhere else `DeleteElementRequestBody` is +used. `OpenMetadataDeleteRequestBody` itself is untouched — it's correctly +fieldless per its own real ground truth (confirmed in ISSUE-62's +investigation), just no longer the class these two methods route through. -**Status:** fixed server-side, confirmed 2026-08-18. Re-verified as part of -a full open-issue sweep: `find_glossary_terms(sequencing_order= -"PROPERTY_ASCENDING", sequencing_property="displayName")` now produces a -genuine, continuous alphabetical sort **across pages**, not just within -one — page 1 (`start_from=0, page_size=10`) ends `'...Amount', 'Approved', -'Area'`, page 2 (`start_from=10`) picks up immediately at `'Attending', -'Attribution factor', 'Avoided emissions', ...` with no gap or overlap. -`PROPERTY_DESCENDING` correctly reverses (`'World Resources Institute', -'World Input Output Database (WIOD)', ...`). No pyegeria code change -involved — purely a server-side fix/redeploy since the 2026-08-15 finding -below (this repo's own request shape was already confirmed byte-identical -to `Egeria-api-glossary-manager.http`'s worked example at that time, so -there was never anything to fix client-side). +**Verified live** against `qs-view-server`: created two throwaway assets +and a real `DataFlow` relationship between them. +`delete_related_elements(rel_guid)` with no body still fails with the same +`OMAG-COMMON-400-032`-class 500 (expected and correct — Egeria's own +default `deleteMethod` is still rejected by this endpoint; that half of +the behavior is unchanged and not this fix's job to change). +`delete_related_elements(rel_guid, {"deleteMethod": "SOFT_DELETE"})` +**succeeded** — the override that was previously impossible now works +end-to-end. `delete_metadata_element()` on both throwaway assets also +succeeded. 6 new unit tests +(`test_metadata_expert_delete_methods.py`) cover both methods' outgoing +request bodies (mocked, no live server) plus a direct model round-trip +matching the report's own verification snippets; full +`pytest tests/micro-tests/` passes. + +--- + +### ISSUE-62: `DeleteElementRequestBody` had no `cascadeDelete`/`deleteMethod` fields — `extra='ignore'` silently discarded both, not just left them unsupported + +**Status:** fixed 2026-08-18 (Pyegeria — `pyegeria/models/models.py`, +`tests/micro-tests/test_delete_element_request_body.py`). + +Raised by dwolfson noticing `_async_delete_element_request` and its +Pydantic model didn't appear to support `deleteMethod`. Investigation found +two gaps, not one: + +1. **`deleteMethod` missing entirely.** Ground truth (`.http` reference + files, e.g. `Egeria-api-actor-manager.http` lines 1452-1460, 2127-2135, + 2409+) shows real `DeleteElementRequestBody` bodies legitimately + carrying `"deleteMethod": "LOOK_FOR_LINEAGE"`; the pydantic model had no + such field. Same bug class as ISSUE-61's mention of + `DeleteRelationshipRequestBody.delete_method` (added 2026-08-17 for + `OMAG-COMMON-400-032`) — that sibling class got the fix, this one never + did. +2. **`cascadeDelete` was *also* silently dropped, and already + live-impacting.** `validate_delete_element_request`'s no-body-provided + path builds `{"class": "DeleteElementRequestBody", "cascadeDelete": + cascade_delete}` and validates it against the same model. Since the + model had neither field and `PyegeriaModel`'s `extra='ignore'`, both + values vanished before serialization — confirmed directly (no live + server needed): + ```python + >>> DeleteElementRequestBody.model_validate( + ... {'class': 'DeleteElementRequestBody', 'cascadeDelete': True, 'deleteMethod': 'PURGE'} + ... ).model_dump_json(exclude_none=True) + {"forLineage": false, "forDuplicateProcessing": false, "class": "DeleteElementRequestBody"} + ``` + This wasn't hypothetical: every `_async_delete_X` wrapper that threads + `cascade_delete` through `_async_delete_element_request` (e.g. + `DataDesigner.delete_data_structure(guid, cascade_delete=True)`, + exercised in `tests/scenario-tests/test_data_designer_scenarios.py`) has + been silently sending `cascade_delete` as unset every time, regardless + of caller intent — Egeria falls back to its own non-cascading default + instead. + +Checked the adjacent classes for the same gap: `DeleteClassificationRequestBody` +and `OpenMetadataDeleteRequestBody` both correctly have neither field — +matches their real ground-truth bodies, no bug there. The gap was isolated +to `DeleteElementRequestBody`. + +**Fix (deliberately scoped — see "Possible future work" below for what was +declined):** added both fields directly to `DeleteElementRequestBody`, +mirroring `DeleteRelationshipRequestBody`'s existing fix pattern: +- `delete_method: Optional[DeleteMethod] = None` — no spelling ambiguity + found in ground truth for this one, uses the model's normal + `alias_generator` (→ `deleteMethod`). +- `cascade_delete: Optional[bool] = None` with + `validation_alias=AliasChoices("cascadeDelete", "cascadedDelete")` and + `serialization_alias="cascadeDelete"`. Ground-truth `.http` files are + genuinely split roughly 50/50 between `cascadeDelete` and + `cascadedDelete` for this same class — even within a single file + (`Egeria-api-actor-manager.http` uses both). Not resolved here; + `cascadeDelete` was chosen as the *outgoing* name because it's what this + repo's own code (`validate_delete_element_request`'s no-body-provided + branch) has already been trying to send since before this fix. Both + spellings are accepted on input so a caller-supplied dict using either + one still populates correctly. + +No changes needed to `_server_client.py` — `validate_delete_element_request`'s +existing shape was already correct in intent, it was purely the missing +model fields silently eating the values. Verified: 5 new unit tests +(`test_delete_element_request_body.py`) covering both-spelling validation, +canonical-name serialization, and the actual outgoing JSON body via a +mocked `_async_make_request` on both the no-body-provided and +caller-supplied-dict paths; full `pytest tests/micro-tests/` passes. + +**Possible future work (deliberately not done here):** `_async_delete_element_request` +still has no first-class `delete_method` parameter (only `cascade_delete`) +— a caller wanting to set it today must hand-build a body dict rather than +pass a kwarg, asymmetric with how `cascade_delete` already works. Adding +one would mean threading it through every `_async_delete_X`/`delete_X` +wrapper across most OMVS modules that currently thread `cascade_delete` +(dozens of call sites, each with its own docstring/"Full sample body" +block) — declined for this pass because (a) it's convenience, not a +correctness fix, the model fix alone already unblocks any caller that +passes a body dict or a `DeleteElementRequestBody` instance directly, and +(b) not every delete endpoint necessarily accepts every `DeleteMethod` +value uniformly (`deleteRelationshipInStore` already rejects its own +default `LookForLineage` — see ISSUE-61's live-verification note — so +Egeria's validation isn't uniform across delete routes), and rolling a new +kwarg out to dozens of wrappers without live-verifying each endpoint's +accepted values individually risks wiring up a parameter some routes will +400 on. Also still open: the `cascadeDelete`/`cascadedDelete` spelling +split above was never resolved against a live server per-endpoint — if a +given delete endpoint actually expects `cascadedDelete`, this fix's choice +would need revisiting for that specific endpoint. + +--- + +### ISSUE-61: No Dr.Egeria command exists to attach a `GovernanceMetric` to its implementation (the `GovernanceResults` relationship) — the OMVS wrapper already exists, it's just never wired to a compact-spec command + +**Status:** fixed 2026-08-17 (Pyegeria/Dr.Egeria — Dan, via the Dr.Egeria +Spec Editor's REST API). Added `Link Governance Results` (`OM_TYPE +GovernanceResults`, bundle `Link Command Base`) to the `Governance Officer` +family, with two new Reference Name-style attributes ("Governance Metric" → +`GovernanceMetric`, "Data Asset" → `Asset`) matching the sibling "Governance +Definition"/"Referenceable" pair on `Link Governed By`. `GovernanceLinkProcessor` +(`md_processing/v2/governance.py`) already handles Link/Detach/Unlink/Remove +variants from one spec entry via `build_command_variants` — no separate +registration needed; wired via a new `endpoint_map` entry + one +`apply_changes` branch per direction, calling +`_async_link_governance_results`/`_async_detach_governance_results` with a +body built via the existing generic `set_rel_prop_body()` (no bespoke body +builder needed). + +Originally raised 2026-08-17 while designing how the Egeria Overview +dashboard's ~25 `pyegeria.view.overview_metrics` functions could be +described as real Egeria metadata rather than only Python docstrings — see +`egeria-workspaces-fs`'s `OVERVIEW_NEXT_STEPS.md` for the full design +reasoning (`GovernanceMetric` for the definition, `GovernanceResults` → +`Report` for the implementation, no new relationship type needed since +`Report` is already a `DataSet` subtype). + +**Verified live 2026-08-17** against the pilot `GovernanceMetric`/`Report` +pair created during the original design investigation: `--validate` and +`--process` both succeed; detached the pilot's original raw-API-created +`GovernanceResults` relationship (hit the known +`deleteRelationshipInStore`-rejects-its-own-default-`deleteMethod` bug — +worked around with an explicit `{"deleteMethod": "SOFT_DELETE"}` body, same +issue this repo's `DeleteRelationshipRequestBody.delete_method` field +already exists to work around elsewhere) and re-created it via the new +`Link Governance Results` command; `mgr.get_all_related_elements()` confirms the relationship resolves correctly afterward. + +--- + +### ISSUE-60: `find_glossary_terms`'s `sequencing_order`/`sequencing_property` sorts only the fetched page, not the full result set — a "fetch one page, sort in JS" pattern silently drops alphabetically-early/late terms once the collection exceeds the page + +**Status:** fixed server-side, confirmed 2026-08-18. Re-verified as part of +a full open-issue sweep: `find_glossary_terms(sequencing_order= +"PROPERTY_ASCENDING", sequencing_property="displayName")` now produces a +genuine, continuous alphabetical sort **across pages**, not just within +one — page 1 (`start_from=0, page_size=10`) ends `'...Amount', 'Approved', +'Area'`, page 2 (`start_from=10`) picks up immediately at `'Attending', +'Attribution factor', 'Avoided emissions', ...` with no gap or overlap. +`PROPERTY_DESCENDING` correctly reverses (`'World Resources Institute', +'World Input Output Database (WIOD)', ...`). No pyegeria code change +involved — purely a server-side fix/redeploy since the 2026-08-15 finding +below (this repo's own request shape was already confirmed byte-identical +to `Egeria-api-glossary-manager.http`'s worked example at that time, so +there was never anything to fix client-side). **Original finding (2026-08-15):** `sequencing_property="displayName"` and `="qualifiedName"` both still returned server-internal order, not @@ -6285,6 +5782,34 @@ element." --- +### ISSUE-57: `GovernanceResults` relationship rejects the exact end1/end2 GUID order the `.http` ground truth and pyegeria's URL both use + +**Status:** fixed server-side, confirmed 2026-08-18. Found 2026-08-15 as a +byproduct of live-verifying the ISSUE-27 fix (`detach_governance_results`'s +duplicate-argument bug). Re-verified as part of a full open-issue sweep +prompted by dwolfson's recent bug-fix pass — created a throwaway +`GovernanceMetric` + `DataSet` asset live against `qs-view-server` and +called `GovernanceOfficer.link_governance_results(metric_guid, dataset_guid)` +directly: **succeeded**, no 500, relationship created correctly. Cleaned up +both elements afterward. No pyegeria code change involved — this was +purely a server-side fix/redeploy between 2026-08-15 and now. + +**Original finding (2026-08-15):** calling +`GovernanceOfficer._async_link_governance_results(governance_metric_guid, +data_asset_guid)` — which builds +`POST .../governance-metrics/{governanceMetricGUID}/measurements/{dataAssetGUID}/attach`, +matching `Egeria-api-governance-officer.http`'s `linkGovernanceResults` +worked example exactly — failed with a 500 +(`OMRS-REPOSITORY-400-047`: end1 assigned to the *second* URL path GUID and +end2 to the *first*, the reverse of what the `GovernanceResults` +relationship type and the URL's own naming both imply). Confirmed at the +time nothing was fixable client-side; the server has since been corrected +(or the underlying type/endpoint definition fixed), consistent with the +`Link Governance Results` Dr.Egeria command (ISSUE-61, closed 2026-08-17) +also live-verifying successfully around the same window. + +--- + ### ISSUE-56: `ReferenceDataManager`'s 6 relationship-link methods all route through the element-body validator, not the relationship-body validator — every call built per the method's own documented sample body fails Pydantic validation **Re-verified 2026-08-15 (same day, third check — asked twice more after @@ -6409,614 +5934,852 @@ approval, not patched directly by an assisting session. --- -### ISSUE-51: `AsyncBaseCommandProcessor.fetch_element()`'s two fallback paths return incompatible shapes — the MetadataExpert fallback crashes downstream code that assumes ClassificationExplorer's envelope - -**Status:** fixed 2026-08-15 (Pyegeria — `md_processing/v2/processors.py`). -Fix shape changed from the original candidate below after discussion: on a -`PyegeriaTimeoutException` specifically, `fetch_element()` now retries the -*same* ClassificationExplorer call (up to 2 attempts) instead of falling -through to the shape-incompatible MetadataExpert path — a timeout means -either the request was transient (retrying the same call is exactly as -likely to succeed as switching endpoints) or the server is under sustained -load (in which case a different endpoint is no more likely to help, and -risks the exact shape-mismatch crash this issue describes). MetadataExpert -remains the fallback for every *non*-timeout failure (not found, -unsupported type, etc.), and as a last resort after timeout retries are -exhausted. Verified: `pytest tests/ -m unit` passes; re-ran the same -100+-command help file that originally triggered the crash against a -freshly-restarted server with 0 errors in the first several minutes (vs. -crashing on the very first command before). The underlying shape -inconsistency between the two paths (described below) still exists — this -fix avoids hitting it in the timeout case rather than resolving the shape -mismatch itself; a genuine non-timeout MetadataExpert-fallback call could -still hit it, so the original diagnosis is left below for reference. +### ISSUE-55: Desired enhancement — true "exclude type" / NOT semantics for `findMetadataElements` -**Original status:** open, found 2026-08-15 (Dan) processing the regenerated -`dr-egeria` help file (100+ `Create`/`Update Term` commands) against -`qs-view-server` while the server was under heavy, unrelated background -load (see ISSUE-52 — same session). +**Status:** fixed 2026-08-18 — Egeria shipped it. Raised 2026-08-05 +alongside ISSUE-45/ISSUE-53, from the same Egeria Insights use case +("elements that have a SemanticAssignment relationship but are NOT type +Notification"), flagged then as needing real Egeria-side API design work +with no shape prescribed. dwolfson found the implementation: +[odpi/egeria#9215](https://github.com/odpi/egeria/pull/9215). -**Layer:** Pyegeria (`md_processing/v2/processors.py`). +**What Egeria shipped:** a new `skipSubtypes` boolean on `QueryOptions` +(server-side). When `true`, `metadataElementSubtypeNames` is treated as +the list of subtypes to *exclude* from the results instead of the only +subtypes to *include* (the pre-existing default, +`skipSubtypes=false`/absent) — exactly the "NOT semantics" this issue +asked for, without needing a second field or new operator. +`metadataElementSubtypeNames` itself also moved off `GetOptions` onto +`QueryOptions` in the same PR (it never did anything on a plain get-by-guid +style request; now it's only ever relevant to find/search bodies, matching +how it actually behaves). Relationship-side got the equivalent treatment: +`FindRelationshipRequestBody.relationshipSubtypeGUIDs` was renamed to +`relationshipSubtypeNames` (GUID-based → type-name-based, matching the +element side) with the same `skipSubtypes` exclude-semantics threaded +through the relationship find/count handlers — though (see "Not done" +below) `FindRelationshipRequestBody` itself has no `skipSubtypes` field on +the wire, unlike the element side. -**What:** `AsyncBaseCommandProcessor.fetch_element()` (`processors.py:1146`) -tries ClassificationExplorer first (`self.client._async_get_element_by_guid_`), -unwraps `res["element"]` if present, and returns that. If that call raises -(caught silently, logged at `debug` only), it falls back to -`self.client._async_get_metadata_element_by_guid(guid)` (MetadataExpert) and -returns *that* result directly, with no unwrapping. Every caller of -`fetch_as_is()`/`fetch_element()` downstream (e.g. -`CollectionManagerProcessor.apply_changes()`) then does -`self.as_is_element['elementHeader']['guid']` unconditionally, assuming both -paths hand back the same `{"elementHeader": {...}, "properties"/"element"...}` -envelope shape. +**pyegeria fix:** `FindRequestBody` (`pyegeria/models/models.py`) — the +model backing the *validated* find-elements path (e.g. +`ClassificationExplorer._async_find_root_elements`, anything going through +`FindRequestBody.model_validate()`/the `TypeAdapter` in +`_server_client.py`) — had no `skip_subtypes` field, so a caller setting +`"skipSubtypes": true` there would have been silently dropped by +`PyegeriaModel`'s `extra='ignore'`, same bug class as ISSUE-62/63. Added +`skip_subtypes: bool | None = Field(None, alias="skipSubtypes")`. +`MetadataExpert._async_find_metadata_elements` needed no code change — it +sends its body dict as-is with no validation, so `"skipSubtypes": true` +already worked there even before this fix (confirmed live, both paths). +Also left a comment on `GetRequestBody.metadata_element_subtype_names` +noting it's now vestigial server-side (kept, not removed — the server +ignores unknown/inapplicable fields rather than rejecting them, so no live +breakage either way). Updated sample-body docstrings in +`metadata_expert.py` (`find_metadata_elements`/`find_relationships_between_elements`/ +`count_relationships_between_elements`) and `classification_explorer.py` +(`find_root_elements`) to show `skipSubtypes`/`relationshipSubtypeNames` +for discoverability. -**Reproduced:** processing `## Update Glossary` for the pre-existing -`dr-egeria` glossary crashed with `KeyError: 'elementHeader'` — -`self.as_is_element` at the point of the crash was -`{'headerVersion': 0, 'status': 'ACTIVE', 'type': {..., 'typeName': -'Glossary', ...}, ...}`, i.e. the *contents* of an `elementHeader` object, -not a dict containing one — consistent with the ClassificationExplorer call -timing out under load (confirmed independently: a standalone repro of -`client._async_get_element_by_guid_(guid)` against the same server, same -session, hit a hard 30s `PyegeriaTimeoutException` on this exact endpoint), -falling through to the MetadataExpert path, whose -`_async_get_metadata_element_by_guid` routes through -`_async_get_guid_request(..., output_format="JSON")` — that helper's -`output_format == "JSON"` branch returns `resp_json.get("element", ...)` -raw, without the normalization/unwrapping ClassificationExplorer's own path -already does elsewhere in the codebase, for at least this element type. +**Not done:** no `FindRelationshipRequestBody` typed model exists in +pyegeria at all (every relationship-find call site builds a raw dict) — +checked the real server-side class directly +(`frameworkservices/omf/rest/FindRelationshipRequestBody.java`): it +extends the older `commonservices.ffdc.rest.ResultsRequestBody`, not +`QueryOptions`, so `skipSubtypes` is genuinely **not** an accepted field +on relationship-find/count request bodies today, only on element-find — +confirmed by reading the class, not assumed. If Egeria adds it there +later, the fix is the same shape (a typed model + the field), but there's +nothing to add on the pyegeria side until the server does. -**Impact:** any `Update`/upsert-style Dr.Egeria command whose primary -element-lookup call fails or times out (not narrow to Glossary — any type -routed through the shared base `fetch_element()`) crashes with an -unhandled `KeyError` instead of retrying, warning, or falling back to -`Create`, and aborts that one command while the rest of a batch continues -(confirmed: the batch run kept going after this crash). +**Dr.Egeria:** no command-level change — checked `find_constraints` +(`md_processing/v2/processors.py`), Dr.Egeria's only existing +subtype-related mechanism, and it's a different, narrower thing (a single +include-type used to disambiguate a name-based element lookup during +processing, not a `FindRequestBody`-style batch search with an +include/exclude subtype list). No compact command currently exposes +element-search subtype filtering to a markdown author at all, so there's +no existing user-facing surface for `skip_subtypes` to plug into yet. The +natural future consumer is `pyegeria/view/overview_metrics.py`'s analytic +functions (several already build `FindRequestBody`s) if a metric ever +needs "count everything except X" — not built speculatively here. -**Candidate fix:** make the two fallback paths agree on shape before -`fetch_element()` returns — either normalize the MetadataExpert path to -always produce `{"elementHeader": ..., ...}` (checking what -`_async_get_metadata_element_by_guid` actually returns for a few element -types to confirm whether it's ever *already* wrapped, or always flat), or -make the single downstream contract explicit and defensive -(`self.as_is_element.get('elementHeader', self.as_is_element).get('guid')` -as a minimal guard) so a shape mismatch degrades to "treat as not found" -rather than crashing. Needs a live server that isn't under load (see -ISSUE-52) to safely re-verify the exact MetadataExpert raw shape without -every repro attempt itself timing out. +**Verified live** against a running server, both request paths: +- Raw-dict pass-through (`MetadataExpert._async_find_metadata_elements`): + `metadataElementSubtypeNames: ["GlossaryTerm"], skipSubtypes: true` on + `Referenceable` returned zero `GlossaryTerm` elements in the sample + (types seen: `ConnectorActivityReport`, `GovernanceZone`). + Cross-checked the include-list side too (ISSUE-53's own fix, still + holding): restricting to `["GlossaryTerm"]` without `skipSubtypes` + correctly returned only `GlossaryTerm`. +- Validated `FindRequestBody` path + (`ClassificationExplorer._async_find_root_elements`): identical + `skipSubtypes: true` behavior after the model fix — confirmed the field + actually reaches the server through Pydantic validation, not just the + raw-dict shortcut. + +**Tests:** `test_find_request_body_skip_subtypes.py` (4 tests, model-level, +no live server needed). --- -### ISSUE-49: `MetadataExpert.get_metadata_element_relationships` (lookup a relationship by its two endpoint GUIDs) returns "No elements found" for a relationship confirmed to exist +### ISSUE-54: `findMetadataElements` scoped to `Referenceable` silently returns an incomplete, arbitrary subset instead of the true population -**Re-verified 2026-08-15 (same day, follow-up run), no regression.** -Recreated the exact repro fresh (new `ResultsSet` + `SavedQuery` pair, -real `SmartQuery` link) — `get_metadata_element_relationships(rs_guid, -sq_guid, "SmartQuery", None)` correctly returns the real relationship -(`relationshipGUID`, both ends populated) instead of "No elements found". -Test elements cleaned up. `pytest tests/ -m unit` passes. +**Status:** fixed 2026-08-30 (Egeria server — `postgres-repository-connector`, +`QueryBuilder.java`, commit `0d8d079c50`). The root cause is the one this entry +worked its way to: `getSequencingOrder()` emitted an `ORDER BY` on a single +non-unique column — creation time, update time, or a property value — and +`getPaging()` appended `limit`/`offset` to it. Each page is a separate execution of +that query rather than a server-side cursor, so rows tying on the sort column could +be ordered differently by each execution: an element could move between offset +windows from one page fetch to the next and be returned twice, or skipped, while the +traversal still terminated normally and reported nothing wrong. This is the +`OFFSET`/`LIMIT`-without-a-unique-`ORDER BY` anti-pattern the 2026-08-18 analysis +predicted, confirmed in the source rather than inferred from behaviour. -**Status:** fixed 2026-08-15 (Pyegeria — `pyegeria/omvs/metadata_expert.py`). -Root cause: the shared `process_related_element_list()` helper has two -envelope shapes it can parse — a related-*element* envelope -(`{"relatedElementList": {"elementList": [...]}}`) and a relationship-list -envelope (`{"relationshipList": {"relationships": [...]}}`) — selected by -its `relationship_list` parameter. Two bugs stacked: (1) -`_async_get_metadata_element_relationships` (and its sibling -`_async_get_all_metadata_element_relationships`) called the helper with -the default `relationship_list=False`, so it looked for the wrong -top-level key (`relatedElementList`, which doesn't exist in a relationship -response) and returned the `NO_ELEMENTS_FOUND` sentinel immediately; (2) -even with `relationship_list=True` set (as `_async_find_relationships_ -between_elements`, ISSUE-39's method, already did), the helper's inner -lookup used the wrong key too — `elementList` instead of the real -`relationships` key nested inside `relationshipList`. Confirmed live via -raw response inspection: `POST .../linked-by-type/{type}/to-elements/{guid}` -returns `{"relationshipList": {"relationships": [...]}}` — never -`relatedElementList`/`elementList` at all. +The `ORDER BY` now always ends with the principle table's primary key, making it a +deterministic total order. The primary key is used rather than `instance_guid` alone +because it is what defines a unique row for the table being paged — +`(instance_guid, version)` for entity and relationship, and +`(instance_guid, classification_name, version)` for classification, where one GUID +legitimately has a row per classification. -**Fixed both bugs**: `process_related_element_list()` now looks up -`"relationships"` (not `"elementList"`) when `relationship_list=True`; the -two relationship-endpoint call sites that weren't already passing -`relationship_list=True` now do -(`_async_get_metadata_element_relationships`, -`_async_get_all_metadata_element_relationships`). +This also explains the two results that looked like separate problems. +`sequencingProperty=GUID` worked because it was the one branch already sorting on a +unique column — it is no longer a workaround anyone needs. And the residual ~4% +omission under `qualifiedName` sequencing, concentrated in `ValidMetadataValue` and +`PersonRole`, was the same defect in its other form: with the property null for +every element of a type, those elements all tie at the null position and the tie is +resolved differently per page. -**Verified live**: recreated the exact repro below (`ResultsSet` + -`SavedQuery` + real `SmartQuery` link via -`link_saved_query_to_results_set`) — `get_metadata_element_relationships` -now returns the relationship with the matching GUID instead of "No -elements found". Also fixed ISSUE-39 as a side effect (same root cause, -same helper) — see that entry. `pytest tests/ -m unit` passes throughout. +**Regression cover:** `PagingStabilityFVT` (in `open-metadata-test/open-metadata-fvt/ +query-fvt`) pages a set arranged so every sequencing is a complete tie — including a +property that is null for every element — and checks each traversal visits every +element exactly once. With the fix reverted it fails with "Element … was returned on +more than one page when paging with PROPERTY_ASCENDING on a property every element +shares a value for". The pre-existing `PagingFVT` could not see any of this: it asks +for `SequencingOrder.GUID`, a total order already. -**Original status:** open, found 2026-08-10 (Dan) building egeria-workspaces-fs's -migration of Egeria Insights' saved queries onto the real `SavedQuery`/ -`SmartQuery` types (`insights_handler.py`, Track C.1 of -`EGERIA_INSIGHTS_QUERY_MODEL.md`). +**Still to confirm, and worth doing before this is treated as closed:** the fix was +proven on a 45-element tie set, not on the ~9,600-element `Referenceable` population +this entry measured. The numbers recorded here are consistent with tie instability +accounting for the whole gap, but that is inference — re-running the exhaustive +`Referenceable` scan against a build carrying `0d8d079c50` is what settles it. Note +also that the fix is in the **PostgreSQL** repository connector; the in-memory +connector has not been checked for the same defect. -**Layer:** Pyegeria (`pyegeria/omvs/metadata_expert.py`) — or possibly the -view-service endpoint it calls; not yet narrowed further, see Candidate fix. +**Correction to this entry's own side notes:** the remark that +`metadataElementSubtypeNames` is "confirmed non-functional" is out of date — subtype +names are pushed down to the repository in current code +(`OpenMetadataStoreRESTServices` passes `getMetadataElementSubtypeNames()` through to +the handler). -**What:** created a `SmartQuery` relationship between a `ResultsSet` and a -`SavedQuery` via `CollectionManager.link_saved_query_to_results_set()` -(itself new in 6.0.17.18, part of the same feature — see that method's own -docstring), which returned a real relationship GUID. Immediately after, -looked the same relationship up by its two endpoint GUIDs via -`MetadataExpert.get_metadata_element_relationships(end1_guid=, end2_guid=, relationship_type="SmartQuery", -body={"class": "ResultsRequestBody"})` — the exact body shape from the -method's own docstring sample — and got back the string `"No elements -found"` rather than the relationship. Tried both `body={}` and -`body={"class": "ResultsRequestBody"}`; same result both times. +**Moved here from "Open Egeria Server issues" on the basis of the above — re-run the +exhaustive `Referenceable` scan against a build carrying `0d8d079c50` before treating +the ~9,600-element population number as settled.** -The relationship is genuinely there — independently confirmed via -`ClassificationExplorer.get_relationships(relationship_type="SmartQuery", -output_format="JSON")`, which lists it with the correct `end1`/`end2` and -the same GUID `link_saved_query_to_results_set()` returned. +**Update 2026-08-26: re-confirmed again, unchanged.** Re-ran the exhaustive +`Referenceable` scan (unsequenced, `pageSize=500`, advancing unconditionally, +stopping only on an empty page — the correct pagination contract per the +note below) against the current server build (`6.2-SNAPSHOT`, platform +timestamp `2026-08-24T18:27:04Z` — the same redeploy that turned out to +have fixed ISSUE-69, checked in the same pass on the chance this one had +also moved). It hadn't: 9,344 raw elements fetched across 19 pages, only +**6,789 distinct GUIDs** (2,555 duplicates) against a native +`count_metadata_elements` population of **9,636** — ~70% coverage, in the +same range as every prior check (54-73% across past re-checks). Cross-check +against a direct exhaustive `GlossaryTerm` scan (262 real, distinct GUIDs): +only **80 (31%)** of them appear anywhere in the broad `Referenceable` +scan, **183 missing**. Same shape as originally documented — this is a +live, unresolved Egeria server bug, not something that rides along with +whatever fixed ISSUE-69. -**Where seen:** `insights_handler.py`'s saved-query delete path would have -used this method to look up the `SmartQuery` relationship's GUID from the -`ResultsSet`/`SavedQuery` GUIDs alone, if it needed to. Worked around -entirely rather than root-caused further — see Candidate fix. +**Status:** still open (Egeria server) — **NOT fixed, re-confirmed +2026-08-18, and this pass corrects a misdiagnosis made earlier the same +day.** First re-check (prompted by dwolfson asking to double-check after +ISSUE-53/57/60) found `pageSize=500`/`startFrom=0` on a `Referenceable` +scan returning only 494 elements, and mistook that short page for the bug +itself (reasoning: "a naive `len(page) < page_size` pagination loop halts +here"). **dwolfson caught this and pointed at Egeria's own paging docs** +(https://egeria-project.org/guides/developer/finding-metadata/overview/#paging), +which explicitly document that a short-but-nonempty page does **not** mean +"last page" — server-side filtering can legitimately shorten an individual +page without ending the result set, and the only correct termination +signal is a genuinely **empty** page. `len(page) < page_size` is a +documented anti-pattern, not evidence of an Egeria bug — re-running the +scan correctly (always advance `startFrom` by the full `pageSize`, +regardless of how many came back, stop only on `not page`) confirms this: +`startFrom=500` (not `startFrom=494`) picks up cleanly with no gap. -**Repro:** -```python -from pyegeria import CollectionManager, MetadataExpert -cm = CollectionManager(view_server="qs-view-server", platform_url=url, user_id=user, user_pwd=pwd) -cm.create_egeria_bearer_token() -rel_guid = cm.link_saved_query_to_results_set(results_set_guid, saved_query_guid) # succeeds, real guid +**The real remaining defect, once paginated correctly:** exhaustively +scanning `Referenceable` with the *correct* stop condition (23 pages, +`pageSize=500`, advancing by 500 unconditionally) fetched **10,548** +elements total against a true native-count population of **11,019** — but +those 10,548 contained **1,394 duplicate GUIDs**, leaving only **9,154 +distinct** elements, still **~17% short** of the true 11,019. `GlossaryTerm` +coverage in this corrected scan: **243/357 (68%)** — much better than the +27% the flawed short-page-as-stop-signal method found, but still +genuinely incomplete. So the original 2026-08-06 hypothesis (unstable +server-side result ordering across pages causing both duplication *and* +omission) remains the best-supported explanation — this is a real, +still-open Egeria server pagination-stability bug for this specific +broad-type query, just not the mechanism this session initially +misdiagnosed. Nothing fixable client-side. -me = MetadataExpert(view_server="qs-view-server", platform_url=url, user_id=user, user_pwd=pwd) -me.create_egeria_bearer_token() -me.get_metadata_element_relationships( - end1_guid=results_set_guid, end2_guid=saved_query_guid, - relationship_type="SmartQuery", body={"class": "ResultsRequestBody"}, -) -# -> "No elements found" +**Checked dwolfson's hypothesis that the duplicates might just be multiple +versions of the same object (not a true dupe)** — ruled out definitively. +Re-ran the corrected exhaustive scan and compared `versions.version`/ +`versions.updateTime` across every occurrence of every duplicated GUID +(1,112 duplicated GUIDs found this run — close to, not identical to, the +1,394 above; population drifts slightly between runs since this is a live +shared server). **Every single occurrence of every duplicate has the +identical version number and `updateTime`** — 0 of 1,112 duplicated GUIDs +showed two different versions of the same element; all 1,112 were the +exact same version appearing 2-3 times (`{2: 1092, 3: 20}` occurrence-count +distribution). So this is not "the repository returned two historical +versions of one element and we're not deduplicating" — it's the literal +same element-state appearing more than once in the raw paginated response, +which is consistent only with unstable result ordering shifting an +element's position between page fetches (the same mechanism suspected +since the original 2026-08-06 finding), not a versioning artifact. +Duplicated types skew heavily toward `ValidMetadataValue` (659 of 1,112) +and `SpecificationPropertyValue` (300) — both likely large, frequently- +reordered collections, consistent with an ordering-instability theory. -from pyegeria import ClassificationExplorer -ce = ClassificationExplorer(view_server="qs-view-server", platform_url=url, user_id=user, user_pwd=pwd) -ce.create_egeria_bearer_token() -ce.get_relationships(relationship_type="SmartQuery", output_format="JSON", body=None) -# -> [{"relationshipHeader": {..."guid": rel_guid...}, "end1": {...ResultsSet...}, "end2": {...SavedQuery...}}] -# same rel_guid, so the relationship is real and correctly formed -``` +**Follow-up, same day** — dwolfson asked the natural next question: how can +a duplicate GUID even happen if the repository enforces uniqueness? Ruled +out every "this is actually fine" explanation with direct evidence rather +than just asserting server-side instability: +- **Not a live-mutating dataset causing drift mid-scan.** Native + `count_metadata_elements` for `Referenceable` stayed exactly stable + (11,079 → 11,079) across a full 7.4-second, 23-page scan. Page 1 + (`startFrom=0`) came back byte-for-byte identical before and after the + entire scan. Two immediate back-to-back identical requests also returned + the exact same elements in the exact same order — no evidence of + anything being written to this dataset during or around the scan. +- **Not a simple page-boundary tie-break either** — checked how far apart + (in page-index terms) each duplicated GUID's two occurrences were: only + 245/1,619 (15%) were adjacent pages; 1,374 (85%) spanned 2+ pages apart, + with a max observed gap of **10 pages** (~5,000 elements apart). An + element legitimately can't shift ~5,000 positions between two `startFrom` + calls unless the server's ordering for this query isn't a true + deterministic total order at all. +- **Best-supported mechanism:** `findMetadataElements` on a broad + `Referenceable` scan most likely sorts by something like creation time + with no unique tiebreaker (e.g. GUID) appended to the effective + `ORDER BY`. `ValidMetadataValue`/`SpecificationPropertyValue` (the two + most-duplicated types, 659 + 300 of 1,112) are both bulk-loaded from an + archive, so plausibly thousands of rows share the *exact same* creation + timestamp. Each `startFrom`/`pageSize` call is a fresh query execution, + not a stable server-side cursor — when many rows tie on the sort key, the + database is free to resolve that tie differently on separate query + executions, so the identical single row can legitimately land in very + different offset windows across two calls. This is the well-known + `OFFSET`/`LIMIT`-without-a-fully-unique-`ORDER BY` SQL pagination + anti-pattern, surfacing here in Egeria's repository query — not a + pyegeria bug, not a real repository-level duplicate GUID. -**Candidate fix:** not yet investigated — could be a body-shape mismatch -this repro hasn't found, a wrong end1/end2 ordering assumption in the -method's URL construction (`.../metadata-elements/{end1_guid}/linked-by- -type/{relationship_type}/to-elements/{end2_guid}`), or a real server-side -gap specific to this endpoint (`SmartQuery` has no bespoke view-service -endpoint of its own — types/properties only, per Egeria PR #9200 — so it's -plausible the generic linked-by-type endpoint doesn't handle every -relationship type uniformly). Whoever picks this up should try swapping -end1/end2, and try a relationship type with an established working example -of this same method (if one exists) to isolate whether `SmartQuery` -specifically is the problem or the method itself. +**Theory tested directly, same day** — dwolfson's suggestion: force a +deterministic sort by a genuinely unique property and see whether +duplicates disappear. They did, confirming the theory outright, and the +test surfaced a second, related defect in the process: -**Workaround in use:** don't rely on this lookup at all — capture the -relationship GUID from `link_saved_query_to_results_set()`'s own return -value at creation time and persist it (in `insights_handler.py`'s case, in -the `SavedQuery`'s own `additionalProperties`), so any later operation that -needs to detach the relationship already has its GUID in hand. +| Sort | Duplicates | Completeness (distinct / true count) | +|---|---|---| +| None (baseline, this run) | 853 | 9,752 / 10,605 (92%) | +| `sequencingProperty=GUID` | **1** (plausibly one live edit mid-scan, not a leftover tie) | not measured — session blocked mid-check by an unrelated concurrent edit breaking `pyegeria` imports (`pyegeria/omvs/feedback_manager.py`, not touched by this investigation) | +| `sequencingProperty=qualifiedName` | **0** | 10,597 / 11,081 (95.6%) — still short | ---- +Sorting by `qualifiedName` fully eliminates duplication (confirms the +theory) but the resulting scan is still incomplete — and *not* randomly: +every element of the omission is concentrated in exactly two types, +`ValidMetadataValue` (211 missing of 2,296) and `PersonRole` (63 missing of +334), while every other type in the scan matched its native count exactly +(`SpecificationPropertyValue`, `GovernanceActionProcessStep`, +`GovernanceActionProcess`, `GlossaryTerm`, `NotificationType`, +`SolutionComponent` all diff=0). Root cause of the omission: **every +single element of both `ValidMetadataValue` and `PersonRole` has +`qualifiedName = None`** — confirmed directly (2,296/2,296 and 323/323 +have a null `qualifiedName`, not just "some"). Sorting by `qualifiedName` +ties every element of these two types together into one giant unresolved +group at the null position — the *same* missing-tiebreaker defect as the +duplication case, just manifesting as omission instead of duplication when +the caller's own chosen `sequencingProperty` happens to be null for an +entire type. This means `GUID` is the objectively safer workaround for +anyone needing a reliable exhaustive scan today — it's the one property +guaranteed non-null and unique for every element regardless of type, so it +shouldn't hit either failure mode (consistent with its near-zero-duplicate +result above; completeness under `GUID` sequencing not yet confirmed, see +blocker note in the table). -### ISSUE-23: `max_mermaid_node_count` defaults to 5 across every shared find/get request helper, silently truncating server-generated mermaid graphs +**Also found as a byproduct of this correction:** pyegeria's own +`pyegeria/view/base_report_formats.py` (`load_egeria_report_specs()`) had +exactly the `len(page) < page_size` anti-pattern in a real fetch-all loop +(`find_collections` for `ReportType` collections) — fixed 2026-08-18 to +advance unconditionally and stop only on an empty page, per the now-clear +Egeria paging contract. Worth grepping the rest of this codebase (and any +sibling repo — `egeria-workspaces-fs`, `egeria-advisor`/`trellis`) for the +same pattern (`len(page) <` / `< page_size` / `< _page_size`); not +exhaustively audited beyond this one hit in this pass. -**Status:** fixed 2026-08-15 (Pyegeria). Raised every remaining -`max_mermaid_node_count` default from 5 to 10, matching -`_async_find_request`'s default (already 10 from an earlier pass): -`_async_get_name_request`, `_async_get_guid_request`, -`_async_get_request_body_request`, `_async_activity_status_search_request`, -and the content-status/deployment-status search variants (all in -`pyegeria/core/_server_client.py`), plus `AssetCatalog`'s three lineage-graph -methods in `pyegeria/omvs/asset_catalog.py` (`_async_get_asset_lineage_graph_by_guid`, -`get_asset_lineage_graph_by_guid`, `get_asset_lineage_mermaid_graph`, added -as part of ISSUE-24 at 5, now raised to 10 for consistency). Verified live -via request-body spies that the new default (10) actually reaches the wire -for `_async_get_guid_request` and `_async_get_asset_lineage_graph_by_guid`. +**Status:** still open (Egeria server), re-confirmed 2026-08-15 against the +current, restarted `qs-view-server` — magnitude has shifted (as with +ISSUE-38/52's re-checks) but the core defect persists. Re-ran the same two +independent cross-checks with a fully-paginated exhaustive scan +(`page_size=200`, following pagination to exhaustion): +- Direct exhaustive `metadataElementTypeName="GlossaryTerm"`: **532** + elements. Of the exhaustive `Referenceable` scan's 19,166 elements + (19,127 distinct GUIDs — see below), only **324** carry `typeName: + "GlossaryTerm"` — **61%** coverage (better than the original run's ~54%, + still clearly incomplete, not just a rounding gap). +- Independent check against `SemanticAssignment` relationship participants + (`ClassificationExplorer.get_relationships`, unrelated API path): 414 + distinct participant GUIDs; only **302 (73%)** appear anywhere in the + 19,166-element `Referenceable` scan (up from the original run's 23/410 = + 5.6%, but still missing more than a quarter of real, independently-known + participants). +- **New observation this pass:** the exhaustive `Referenceable` scan itself + returned 19,166 total elements across pages but only 19,127 **distinct** + GUIDs — 39 duplicate entries showing up more than once across different + pages of the same paginated scan. Not previously noted; suggests server-side + result ordering/stability issues for this specific broad-type scan, which + would also explain why elements can be silently skipped between pages + (an element shifting position between page fetches, due to unstable + ordering, could cause both duplication *and* omission depending on which + way it moves) — a plausible mechanism for the core bug, not confirmed as + the actual root cause. -Also fixed the "third case" this issue flagged as a candidate follow-up: -`get_solution_blueprint_by_guid`/`_async_get_solution_blueprint_by_guid` -(`pyegeria/omvs/solution_architect.py`) built no body at all when the -caller left `body=None`, so `graph_query_depth`/`max_mermaid_node_count` -were genuinely dead parameters regardless of the shared default — same -"dead parameter" shape ISSUE-26 already fixed for -`get_info_supply_chain_by_guid`/`get_solution_role_by_guid`. Fixed the same -way: build an `AnyTimeRequestBody` with `graphQueryDepth`/ -`maxMermaidNodeCount` populated when `body is None`, and added -`max_mermaid_node_count` as an explicit parameter (was previously not even -accepted). Verified live: a request-body spy confirmed caller-supplied -`graph_query_depth=9`/`max_mermaid_node_count=77` now reach the wire, where -previously neither did. `get_solution_component_by_guid` turned out **not** -to share this bug — it already routes through the (now-also-fixed) -`_async_get_guid_request` shared helper, so it correctly forwards both -parameters; verified live the same way (`graph_query_depth=8`/ -`max_mermaid_node_count=66` both reached the wire) before ruling it out. +Both cross-checks still clearly demonstrate the defect; nothing changed +client-side to warrant re-testing pyegeria's own pagination logic (already +ruled out in the original investigation — direct exhaustive scans of real +types are complete and correct, only the broad-base-type scan is affected). -`pytest tests/ -m unit -q` passes (exit 0) after all changes. +**Original status:** open (Egeria server), found 2026-08-06 fixing egeria-workspaces-fs's +relationship-only search (see ISSUE-45's same investigation thread — +looking for a safe fallback type once `metadataElementTypeName="Asset"` +was confirmed wrong for `SemanticAssignment`, and `metadataElementSubtypeNames` +confirmed non-functional). -**Original status:** open at the pyegeria-default level (the user is raising the -shared default from 5 to 10 separately); worked around explicitly in -egeria-workspaces-fs across all detail-view call sites that render a -mermaid diagram — `isc_handler.py`, `mermaid_handler.py`, -`solution_architect_handler.py`, `tech_catalog_handler.py`, -`digital_products_handler.py`, `action_center_handler.py`, -`actor_handler.py`, `community_handler.py`, `context_events_handler.py`, -`data_design_handler.py`, `external_links_handler.py`, -`glossary_handler.py`, `governance_definitions_handler.py`, -`lineage_handler.py` (asset graph only, see ISSUE-24 for the lineage -graph endpoint), `location_handler.py`, `perspectives_handler.py`, -`reference_data_handler.py` — found 2026-08-04 investigating a report -that Egeria Explorer's Information Supply Chain view wasn't rendering the -full chain. `audit_handler.py`'s two `get_element_by_guid` calls are -deliberately left at `graphQueryDepth: 0` (a lightweight side-panel -lookup with no relationship traversal, so no mermaid graph is generated -there anyway) and `notelog_handler.py` doesn't render mermaid fields — -both confirmed not affected, not just skipped. +**Layer:** Egeria Server — not fixable in pyegeria. -**Two distinct fix shapes were needed depending on the call site**, -learned the hard way after an initial fix silently didn't work: -- If the portal handler passes **no** `body=` argument (stays `None`), - a `max_mermaid_node_count=250` kwarg on the outer call correctly - threads through — every `_by_guid`/`find_*` wrapper method in pyegeria - follows the same `params.update(kwargs); ...; **params` idiom that - forwards it into the shared helper's own `max_mermaid_node_count` - parameter. -- If the portal handler already builds an explicit `body={"class": - "GetRequestBody", ...}` dict (common wherever `as_of_time` needs to be - conditionally added), `_async_get_guid_request`/`_async_find_request` - validate that body **as-is** and silently ignore the sibling - `max_mermaid_node_count`/`graph_query_depth` kwargs entirely — the - field must be embedded directly in the body dict - (`"maxMermaidNodeCount": 250`) instead. -- A third shape: `get_solution_blueprint_by_guid`/ - `get_solution_component_by_guid` hit dedicated `.../retrieve` REST - endpoints whose own `graph_query_depth` **parameter is dead** — never - reaches the request at all, body or no body. Passing an explicit - `AnyTimeRequestBody` with `graphQueryDepth`/`maxMermaidNodeCount` set - is the only way to raise it (verified live: 74 → 187 mermaid lines). - **Update 2026-08-04:** `get_info_supply_chain_by_guid`/ - `get_solution_role_by_guid` have the identical shape — see ISSUE-26. +**What:** an exhaustive, fully-paginated `find_metadata_elements` scoped to +`metadataElementTypeName="Referenceable"` (**not** the true type-hierarchy +root — that's `OpenMetadataRoot`, confirmed live via `get_all_entity_defs`; +`Referenceable` is one of five direct subtypes of `OpenMetadataRoot` +alongside `SearchKeyword`/`Rating`/`Like`/`TranslationDetail`, but it's the +practical common ancestor of essentially every entity type an +application-level search actually cares about, which is why it's the type +most callers reach for as a "find everything" scope) returns a small, +arbitrary subset instead of the true population, with no error, no truncation flag, +and pagination genuinely terminating normally (`added == 0`/`len(page) < +page_size` on the last page — the loop believes it's done). Confirmed live +against `qs-view-server` by direct comparison: -**Layer:** Pyegeria (`pyegeria/core/_server_client.py`). +| Scope | Elements found | +|---|---| +| `metadataElementTypeName="Referenceable"` (exhaustive) | 3,999 total | +| ...of which `typeName="GlossaryTerm"` | 241 | +| ...of which `typeName="GovernanceActionProcess"` | 22 | +| `metadataElementTypeName="GlossaryTerm"` (exhaustive, direct) | **450** | +| `metadataElementTypeName="GovernanceActionProcess"` (exhaustive, direct) | **378** | -**What:** `_async_find_request`, `_async_get_name_request`, and -`_async_get_guid_request` — the three shared helpers behind essentially -every `find_*`/`get_*_by_guid` OMVS method — all declare -`max_mermaid_node_count: int = 5`, which gets sent to the server as -`maxMermaidNodeCount` in the request body. The server caps the generated -mermaid flowchart at that many nodes. 5 is far too small for any real -graph with more than a handful of related elements — a value seemingly -picked as a lightweight default for broad/unbounded listings, not for a -"show me everything about one element" detail view, but it applies -identically to both. +Cross-checked against a real, independently-known population: fetching +every participant GUID of the `SemanticAssignment` relationship type +(`ClassificationExplorer.get_relationships`, a separate, unrelated API +path, previously verified complete) gives 410 distinct GUIDs. Only 23 of +those 410 appear anywhere in the 3,999-element `Referenceable` scan — 387 +real participants (94%) are simply absent from a scan of the type that is +supposed to be their common ancestor and therefore cover all of them. -**Where seen:** live repro against a Coco Pharmaceuticals demo server — -`SolutionArchitect.find_information_supply_chains("Open Metadata Highway -Information Supply Chain")`'s `mermaidGraph` field was 165 lines at the -default `max_mermaid_node_count=5`, vs. 619 lines at `250` (graph actually -complete at that point — verified by also trying `50`/`100`, no further -growth past ~200). `graph_query_depth` (default 3), by contrast, made no -measurable difference for any information supply chain tested (1 through -10) — the node-count cap, not query depth, is the actual bottleneck for -mermaid graph completeness. Every OMVS wrapper method that calls one of -these three shared helpers (`find_information_supply_chains`, -`get_element_by_guid`, `find_solution_blueprints`, -`find_solution_components`, `get_solution_component_by_guid`, etc.) -inherits the same 5-node cap unless the caller explicitly overrides it — -confirmed via grep that essentially none of egeria-workspaces-fs's -`PyegeriaWebHandler` handlers do (`isc_handler.py`, now fixed; -`solution_architect_handler.py`, `tech_catalog_handler.py`, -`mermaid_handler.py`, `digital_products_handler.py`, `audit_handler.py` -all still rely on the pyegeria default). +**Impact:** `metadataElementTypeName="Referenceable"` cannot be used as a +"safe, unscoped, find-everything" fallback the way its practical role as +the common ancestor of nearly every real entity type implies — a caller +that scopes a search this broadly on purpose (not just as an accidental fallback) will silently miss +the majority of real results, not just cap them at a boundary. Confirmed +this is specific to the base-type-wide scan, not pagination itself — +directly-typed exhaustive searches for the exact same real types +(`GlossaryTerm`, `GovernanceActionProcess`) are complete and correct. -**Candidate fix (still open):** raise the shared `max_mermaid_node_count` -default itself (planned separately by the repo owner, 5 → 10) — the -per-handler overrides above are generous enough (250) that they don't -depend on that change, but the two "dead parameter" methods -(`get_solution_blueprint_by_guid`/`get_solution_component_by_guid`) would -benefit from an actual fix to thread `graph_query_depth`/ -`max_mermaid_node_count` into their request the same way every other -`_by_guid` method does, rather than requiring callers to know the -`AnyTimeRequestBody` workaround. +**Candidate fix:** none client-side. `egeria-workspaces-fs`'s workaround +(see its `EGERIA_INSIGHTS_QUERY_MODEL.md`/`insights_handler.py`, commit +`015916d0`) is to never scope a search to `Referenceable` (or any other +broad base type) as a "safe fallback" — when the real target type isn't +known, derive the actual candidate types from other data (e.g. a +relationship's real participants) and search each directly instead. --- -### ISSUE-24: `AssetCatalog._async_get_asset_lineage_graph_by_guid` hardcodes `queryGraphDepth: 5` with no override parameter at all +### ISSUE-53: `findMetadataElements`'s `metadataElementSubtypeNames` is silently ignored — restricting to specific subtypes has no effect on results -**Status:** fixed 2026-08-15 (Pyegeria — `pyegeria/omvs/asset_catalog.py`). -Turned out to be worse than "no override" — checked the real -`AssetLineageGraphRequestBody` Java class (`-> QueryOptions -> GetOptions`) -and the field pyegeria was sending, `"queryGraphDepth"`, **doesn't exist on -the real body at all**; the actual field is `"graphQueryDepth"`. So the -hardcoded `5` was always silently dropped by the server regardless — the -server's own default (also 5, coincidentally) was what actually applied -the whole time, even before this fix. Found the same problem one field -over: `"relationshipTypes"` isn't real either — the actual field, -`GetOptions.includeOnlyRelationships`, means that caller-supplied -relationship-type filter has also never worked. Fixed both field names, -added `graph_query_depth`/`max_mermaid_node_count` parameters (the real -`GetOptions.maxMermaidNodeCount` field was never sent at all before) to -all three call layers (`_async_get_asset_lineage_graph_by_guid`, -`get_asset_lineage_graph_by_guid`, `get_asset_lineage_mermaid_graph`). +**Status:** fixed server-side, confirmed 2026-08-18. Re-verified as part of +a full open-issue sweep: `MetadataExpert.find_metadata_elements` with +`metadataElementTypeName: "Referenceable"` + +`metadataElementSubtypeNames: ["GlossaryTerm"]` now correctly returns only +`GlossaryTerm` elements (confirmed: 10/10 results in a sample page all +`typeName: "GlossaryTerm"`, versus the previous mixed-type-junk behavior). +Cross-checked via the native count endpoint too: +`count_metadata_elements` with the same subtype filter returns **357**, +identical to a direct `metadataElementTypeName: "GlossaryTerm"` count with +no subtype filter at all — exact agreement, not just "closer." No pyegeria +code change involved — purely a server-side fix/redeploy since the +2026-08-05/2026-08-15 findings below. -**Verified live**: created a real throwaway asset, spied on the outgoing -request body, and confirmed `graph_query_depth=7`/`max_mermaid_node_count=42` -now actually appear in the POST body as `graphQueryDepth`/ -`maxMermaidNodeCount` and the server accepts the call successfully. -`pytest tests/ -m unit` passes. +**Original finding:** found 2026-08-05 investigating whether Egeria +Insights could express "elements that have a SemanticAssignment but are NOT +type Notification" via an allow-list of subtypes (`Referenceable` + +`metadataElementSubtypeNames: ["GlossaryTerm", "DataAsset"]`, per +`Egeria-api-metadata-expert.http`'s worked "findMetadataElements (nested +condition)" example). `metadataElementSubtypeNames` was accepted (HTTP 200, +no error) but had zero effect on results — byte-identical to the same +query with the field omitted, reproduced across three different +base-type/subtype pairs and re-confirmed again 2026-08-15 against a +restarted `qs-view-server`. Blocked the Egeria Insights use case that +prompted the investigation (excluding noisy `Notification`/`ToDo`/ +`Meeting`/`Review` `Asset` subtypes from a broader search) — that use case +is now unblocked as a byproduct of this fix. See ISSUE-55 (now also +fixed, 2026-08-18) for the true exclude/NOT semantics (`skipSubtypes`) +that arrived shortly after this fix, in the same Egeria PR that reworked +this code path. -**Original status:** open, found 2026-08-04 alongside ISSUE-23 while auditing -`egeria-workspaces-fs/.../lineage_handler.py`'s mermaid-graph call sites. +--- + +### ISSUE-52: `qs-nanny-daemon`/`qs-integration-daemon`'s own connectors generate sustained, heavy background write load against the shared repository — starves interactive requests, plausible cause of "frequent Postgres checkpoints" + +**Status:** fixed (Egeria server — `postgres-repository-connector`, +`jdbc-resource-connector`), confirmed 2026-08-30. The fix predates this +confirmation: the mechanism this entry identified on 2026-08-16 — 38 connections +stuck `idle in transaction` on the same `entity`-table query shape, waiting on +their own client rather than on Postgres — is a JDBC transaction leak, and it was +fixed the same day in `a4ab1c090a` ("Fix JDBC connection and transaction leaks in +the PostgreSQL connectors"). `DatabaseStore` became `AutoCloseable` with `close()` +rolling back, so a read leaves its connection idle rather than idle-in-transaction, +and all 28 of its uses became try-with-resources. `7359136cb1` then replaced the +per-thread connection cache with a HikariCP pool and `c52c1ba8bd` externalised the +pool size. The 6.2-SNAPSHOT build of 2026-08-24 that later re-checks were run +against already carried all three, which is consistent with this entry's own "net +assessment" that the acute symptom stopped reproducing. + +The `OpenAPICataloguer` half of this entry does not reproduce in the code either: +`OpenAPIMonitorIntegrationConnector.getAPIOperationGUID()` looks up the existing +`APIOperation` by qualified name and only creates one when it finds nothing, so the +crawl converges rather than re-cataloguing. That matches the revised "cold-start +crawl that converges" framing this entry already reached, and means the remaining +`JacquardDigitalProductLoom` question is about refresh interval tuning, not about a +defect. + +**Not independently re-measured against a live environment for this update** — the +basis is the code plus this entry's own post-restart observations. If the timeout +symptom ever recurs on a build newer than 2026-08-16, that would be a new problem +rather than this one, and worth a fresh entry. + +**Moved here from "Open Egeria Server issues" on the basis of the above.** + +**Update 2026-08-16, more specific root-cause evidence.** Recurred while +retrying the `dr-egeria` help-Glossary `--process` step (see ISSUE-52's own +earlier "cold-start crawl" postscript — this shows the earlier optimism was +premature, it recurs under real batch load, not just right after a +restart): `docker stats` showed `egeria-shared-postgres` at up to **650% +CPU**, and connecting directly (`docker exec ... psql -U postgres -p 5442 +-d egeria`) found the real mechanism — **38 connections stuck `idle in +transaction`** against the `egeria` database, all holding open the same +query shape (`select distinct * from entity where (version_end_time is +null) and (exists (select 1 from en...`). This is a connection/transaction +pileup, not simple CPU starvation as originally framed — many connections +opened a transaction, are waiting on something (`wait_event_type: Client`/ +`ClientRead`, i.e. waiting on their own client, not blocked by Postgres +itself), and never committed/closed, so newer requests (including a plain +`get_glossaries_by_name` lookup) queue behind them. CPU eased from 650% → +340% → 298% over ~15 minutes while sitting at this same connection count, +without by-name-search latency improving at all (stayed pinned at +25-30s/timeout) — the two aren't tightly coupled, reinforcing that this is +a connection-holding problem, not a raw compute-load problem. Whatever is +opening these `entity`-table transactions and not releasing them (almost +certainly the same `qs-nanny-daemon`/`qs-integration-daemon` connectors +already suspected) is the actual thing to fix, not just "connector load" +in the abstract. + +**Original status:** open (Egeria server / deployment config), found 2026-08-15 (Dan) +investigating why a `dr_egeria --validate`/`--process` run against the +`dr-egeria` help file (100+ commands) was taking 70+ minutes and timing out +individual calls (see ISSUE-51), and independently reported by the user as +"frequent checkpoints" observed in the Postgres console. + +**Layer:** Egeria Server (`egeria-quickstart` deployment's integration +daemon connector configuration), not pyegeria — this is a deployment/config +issue in the `egeria-shared-postgres`-backed local quickstart stack, not a +pyegeria code defect. + +**What:** confirmed live, same session: +- `egeria-shared-postgres` (the `pgvector/pgvector:pg17` container backing + `qs-view-server`'s repository, port 5442) is issuing time-based + checkpoints every 5 minutes (the Postgres default `checkpoint_timeout`) + with real, non-trivial WAL volume behind each one (~10-22 MB/checkpoint, + i.e. roughly 35-75 KB/s of sustained write throughput) continuously, + including in windows where no interactive Dr.Egeria/pyegeria work was + running — meaning the load is coming from something else running inside + the platform itself, not from client-side testing. +- `docker logs quickstart-egeria-main` shows the source: `qs-nanny-daemon`'s + `JacquardDigitalProductLoom` integration connector logged a single + refresh cycle that took **4,370,395 ms (~73 minutes)** to complete + (`INTEGRATION-DAEMON-SERVICES-0043`). `qs-integration-daemon`'s + `OpenAPICataloguer` connector is continuously creating new `APIOperation` + catalog entities — confirmed 703 total in the container's full log + history, 36 of them in the last 30 minutes of a single ~2 hour window + sampled — each one a real metadata write, evidently crawling/re-crawling + the platform's own REST API surface (dozens of + `/open-metadata/access-services/open-metadata-store/...` and + `/open-metadata/conformance-suite/...` paths) rather than converging to a + steady state. +- Directly reproduced the contention: a single, otherwise-simple + `client._async_get_element_by_guid_(guid)` call against `qs-view-server`, + issued from a fresh script with nothing else running client-side, hit the + 30-second client timeout and raised `PyegeriaTimeoutException` — + confirming the server itself, not the client or network, is the + bottleneck. + +**Impact:** any pyegeria/Dr.Egeria workload that does more than a handful +of sequential server calls (bulk `--process` runs, the help-file Glossary +sync in particular) becomes unreliable — individual calls time out +(`TIMEOUT_ERROR_408`) or the whole run takes an order of magnitude longer +than expected — while these background connectors are active. This is very +likely the direct cause of ISSUE-51's crash (the ClassificationExplorer +call it depends on timed out under this exact load) and of the multi-hour +`dr_egeria --validate` run in this same session. -**Layer:** Pyegeria (`pyegeria/omvs/asset_catalog.py`). +**Update 2026-08-15, same day:** user restarted the Egeria environment with +the latest configuration. Post-restart, a `dr_egeria --process` run against +the same 100+-command help file completed its first ~4 minutes with 0 +errors (vs. timing out repeatedly before), and `OpenAPICataloguer` created +0 new `APIOperation` entities in a 10-minute post-restart sample (vs. 36 in +30 minutes before) — consistent with the crawl having converged/settled +after the restart rather than continuously re-cataloguing. Not yet +confirmed whether this is a durable fix or the connectors will resume the +same pattern once they hit their next scheduled refresh; worth re-checking +`docker logs quickstart-egeria-main | grep JacquardDigitalProductLoom` after +it's been up for a few hours. -**What:** unlike every other `_by_guid` method in this codebase, -`_async_get_asset_lineage_graph_by_guid` builds its `AssetLineageGraphRequestBody` -manually with `"queryGraphDepth": 5` hardcoded, and does not spread -`**kwargs` into that body at all — the method's own `**kwargs` parameter -exists but is never used. There is currently no way to override this from -a caller, unlike `get_asset_graph_by_guid` (a different, sibling method on -the same class going through `_async_get_results_body_request`, which -**does** correctly forward a `max_mermaid_node_count` kwarg — confirmed -and fixed in `lineage_handler.py`'s `get_asset_graph` route this session). +**Candidate next step (not yet done):** narrow which specific connector(s) +are the dominant contributor — `JacquardDigitalProductLoom`'s single +73-minute cycle and `OpenAPICataloguer`'s continuous entity creation are +the two strongest leads — and either increase their refresh interval, +narrow `OpenAPICataloguer`'s crawl scope (it may be re-cataloguing +`localhost:9443`'s own REST surface on every refresh instead of once), or +disable/reconfigure them in the `egeria-quickstart` compose config if +they're not needed for this deployment's actual use case. This needs +Egeria-server-side (or deployment-config-side) investigation — nothing here +is fixable from pyegeria's side. -**Where seen:** `egeria-workspaces-fs/compose-configs/*/PyegeriaWebHandler/lineage_handler.py`'s -`get_asset_lineage_graph` route (`/api/lineage/asset/{guid}/lineage-graph`) -— serves `mermaidGraph`/`fullLineageMermaidGraph`/`edgeMermaidGraph` for -the lineage explorer's full end-to-end diagram, exactly the kind of view -most likely to need a depth greater than 5 for a long pipeline. Left -unmodified in the portal since there's no parameter to pass. +**Update 2026-08-15, done (partial):** `quickstart-egeria-main` had been +restarted again since the note above (container uptime ~1h at check time, +vs. the earlier 8h+ uptime of sibling containers) — so this was actually a +fresh post-restart observation window, not a stale one. Findings: +- **`OpenAPICataloguer`'s behavior is better characterized as a one-time + cold-start catalog crawl that converges, not continuous unbounded + re-crawling** — the earlier framing was too pessimistic. Creation rate + measured in shrinking trailing windows since restart: 134 (50 min) → 86 + (40 min) → 48 (30 min) → 26 (20 min) → 8 (10 min) → 6 (5 min) new + `APIOperation` entities — a clear deceleration, not a steady/repeating + rate. 201 total `APIOperation`s exist in the full container log history; + 171 of them were created in just this last hour, meaning the connector + is doing its first post-restart crawl right now and tapering off, not + perpetually re-cataloguing the same ground. +- **`JacquardDigitalProductLoom` logged zero refresh-cycle activity in this + 1-hour post-restart window** — can't yet say whether its 73-minute cycle + is gone for good or just hasn't fired again yet (container too young to + rule out a recurrence on its next scheduled interval). +- **Server responsiveness confirmed fast right now**: 3 sequential + `get_metadata_element_by_guid` calls against `qs-view-server` completed + in 0.19s/0.03s/0.02s — no timeouts, nothing close to the earlier + 30-second `PyegeriaTimeoutException` symptom. +- Could not directly inspect Postgres checkpoint stats this pass (`psql` + isn't installed as a client tool inside the `egeria-shared-postgres` + container in a way this session could reach it) — server-response timing + above stands in as an indirect but consistent signal instead. -**Candidate fix:** add `graph_query_depth`/`max_mermaid_node_count` -parameters to `_async_get_asset_lineage_graph_by_guid`/ -`get_asset_lineage_graph_by_guid`, threading them into -`queryGraphDepth`/a new `maxMermaidNodeCount` body field (need to confirm -the server actually honors a `maxMermaidNodeCount` field on -`AssetLineageGraphRequestBody` specifically — this body class differs -from the more common `GetRequestBody`/`ResultsRequestBody` shapes seen -elsewhere in this file). +**Net assessment:** the acute symptom (timeouts, multi-hour bulk-processing +runs) is not currently reproducing, and the mechanism now looks more like +"cold-start crawl briefly saturates the server after every restart, then +settles" than "sustained, unbounded background load." Still genuinely +Egeria-server/deployment-config territory, not pyegeria-fixable — if this +pattern recurs and is disruptive on every restart, the original candidate +fixes (narrow `OpenAPICataloguer`'s crawl scope, stagger connector startup, +or increase refresh intervals) remain the right next step, just lower +urgency than originally assessed. --- -### ISSUE-27: ~50 more sync methods delegate to their async counterpart with an all-positional argument list — unaudited for the same scrambling bug as ISSUE-21/25 - -**Status:** fixed 2026-08-15 (Pyegeria). Full audit via a script (not manual -grepping): every multi-line all-positional `self._async_*(...)` delegation -across `pyegeria/omvs/*.py` (62 sites, dominated by -`classification_explorer.py`), plus every single-line all-positional call -with 3+ bare-identifier args (285 more candidates), each cross-checked -against its target async method's *actual* parameter order (careful to use -the target's **last** definition in the file, matching Python's real -same-name-method resolution — one earlier false lead came from a stale, -fully-shadowed duplicate `_async_delete_solution_blueprint` definition still -sitting in `solution_architect.py`, dead code but harmless since it's never -reachable). +### ISSUE-51: `AsyncBaseCommandProcessor.fetch_element()`'s two fallback paths return incompatible shapes — the MetadataExpert fallback crashes downstream code that assumes ClassificationExplorer's envelope -Of ~350 candidate sites, found and fixed **4 real bugs**, all in -`pyegeria/omvs/governance_officer.py` and `pyegeria/omvs/solution_architect.py` -(the rest were either exact matches or "different local variable name, same -value, same position" false positives — e.g. `add_term_to_folder(folder_guid, -term_guid, body)` calling `_async_add_to_collection(folder_guid, term_guid, -body)` against a target expecting `(collection_guid, element_guid, body)` — -positionally correct, just domain-specific local names): +**Status:** fixed 2026-08-15 (Pyegeria — `md_processing/v2/processors.py`). +Fix shape changed from the original candidate below after discussion: on a +`PyegeriaTimeoutException` specifically, `fetch_element()` now retries the +*same* ClassificationExplorer call (up to 2 attempts) instead of falling +through to the shape-incompatible MetadataExpert path — a timeout means +either the request was transient (retrying the same call is exactly as +likely to succeed as switching endpoints) or the server is under sustained +load (in which case a different endpoint is no more likely to help, and +risks the exact shape-mismatch crash this issue describes). MetadataExpert +remains the fallback for every *non*-timeout failure (not found, +unsupported type, etc.), and as a last resort after timeout retries are +exhausted. Verified: `pytest tests/ -m unit` passes; re-ran the same +100+-command help file that originally triggered the crash against a +freshly-restarted server with 0 errors in the first several minutes (vs. +crashing on the very first command before). The underlying shape +inconsistency between the two paths (described below) still exists — this +fix avoids hitting it in the timeout case rather than resolving the shape +mismatch itself; a genuine non-timeout MetadataExpert-fallback call could +still hit it, so the original diagnosis is left below for reference. -1. **`solution_architect.py`: `delete_solution_role`** — sync wrapper's own - signature is `(guid, body=None, cascade_delete=False)` (correct order, - matching the async target), but it called - `self._async_delete_solution_role(guid, cascade_delete, body)` — - transposed, sending the boolean into the `body` slot and the body dict - into the `cascade_delete` slot. Fixed to - `self._async_delete_solution_role(guid, body, cascade_delete)`. +**Original status:** open, found 2026-08-15 (Dan) processing the regenerated +`dr-egeria` help file (100+ `Create`/`Update Term` commands) against +`qs-view-server` while the server was under heavy, unrelated background +load (see ISSUE-52 — same session). -2. **`governance_officer.py`: `add_regulator_to_regulation`** — called the - **wrong async method entirely**: `self._async_link_governance_results(...)` - (a copy-paste leftover — a completely different relationship, hitting - `governance-metrics/{gov_metric_guid}/measurements/{data_asset_guid}/attach` - with `GovernanceResultsProperties`) instead of the correct - `self._async_add_regulator_to_regulation(...)`. Fixed the call target. +**Layer:** Pyegeria (`md_processing/v2/processors.py`). -3. **`governance_officer.py`: `_async_add_regulator_to_regulation`/ - `_async_detach_regulator_from_regulation`** — found while fixing #2: both - built the wrong URL (`.../{url_marker}/governance-officer/regulations/...` - — `self.url_marker` is itself `"governance-officer"`, so this duplicated - the segment; ground truth per `Egeria-api-governance-officer.http` is - `.../governance-officer/regulations/{regulationGUID}/regulators/organizations/{regulatorGUID}/attach`, - single segment) and the attach method also used the wrong properties - class (`"GovernanceResultsProperties"` instead of `"RegulatorProperties"`, - also confirmed against the `.http` file). Fixed both URLs and the - properties class. +**What:** `AsyncBaseCommandProcessor.fetch_element()` (`processors.py:1146`) +tries ClassificationExplorer first (`self.client._async_get_element_by_guid_`), +unwraps `res["element"]` if present, and returns that. If that call raises +(caught silently, logged at `debug` only), it falls back to +`self.client._async_get_metadata_element_by_guid(guid)` (MetadataExpert) and +returns *that* result directly, with no unwrapping. Every caller of +`fetch_as_is()`/`fetch_element()` downstream (e.g. +`CollectionManagerProcessor.apply_changes()`) then does +`self.as_is_element['elementHeader']['guid']` unconditionally, assuming both +paths hand back the same `{"elementHeader": {...}, "properties"/"element"...}` +envelope shape. -4. **`governance_officer.py`: `detach_governance_results`** — sync wrapper - passed `data_asset_guid` **twice**: - `self._async_detach_governance_results(gov_metric_guid, data_asset_guid, - data_asset_guid, body)` against a 3-parameter target - `(gov_metric_guid, data_asset_guid, body)` — the extra positional arg - landed in the `body` slot, and the real `body` argument was silently - dropped (extra arg beyond the target's parameter count). Fixed by - removing the duplicate. +**Reproduced:** processing `## Update Glossary` for the pre-existing +`dr-egeria` glossary crashed with `KeyError: 'elementHeader'` — +`self.as_is_element` at the point of the crash was +`{'headerVersion': 0, 'status': 'ACTIVE', 'type': {..., 'typeName': +'Glossary', ...}, ...}`, i.e. the *contents* of an `elementHeader` object, +not a dict containing one — consistent with the ClassificationExplorer call +timing out under load (confirmed independently: a standalone repro of +`client._async_get_element_by_guid_(guid)` against the same server, same +session, hit a hard 30s `PyegeriaTimeoutException` on this exact endpoint), +falling through to the MetadataExpert path, whose +`_async_get_metadata_element_by_guid` routes through +`_async_get_guid_request(..., output_format="JSON")` — that helper's +`output_format == "JSON"` branch returns `resp_json.get("element", ...)` +raw, without the normalization/unwrapping ClassificationExplorer's own path +already does elsewhere in the codebase, for at least this element type. -**Verified live** against `qs-view-server`: `delete_solution_role`'s fix -confirmed via a request-body spy (cascade/body no longer transposed); -`add_regulator_to_regulation`/`detach_regulator_from_regulation` verified -fully end-to-end — created a throwaway `Regulation` + `Organization`, -linked via the fixed method, confirmed via `.http`-matching URL/body, -detached, cleaned up. `detach_governance_results`'s fix confirmed via a -request-body spy showing the correct 2-argument call now reaches -`_async_delete_relationship_request` cleanly. +**Impact:** any `Update`/upsert-style Dr.Egeria command whose primary +element-lookup call fails or times out (not narrow to Glossary — any type +routed through the shared base `fetch_element()`) crashes with an +unhandled `KeyError` instead of retrying, warning, or falling back to +`Create`, and aborts that one command while the rest of a batch continues +(confirmed: the batch run kept going after this crash). -**Found as a byproduct, NOT part of this issue's scope, tracked separately:** -attempting to live-verify `_async_link_governance_results` (unrelated, -untouched method) surfaced a real server-side/endpoint-design bug — see -ISSUE-57. +**Candidate fix:** make the two fallback paths agree on shape before +`fetch_element()` returns — either normalize the MetadataExpert path to +always produce `{"elementHeader": ..., ...}` (checking what +`_async_get_metadata_element_by_guid` actually returns for a few element +types to confirm whether it's ever *already* wrapped, or always flat), or +make the single downstream contract explicit and defensive +(`self.as_is_element.get('elementHeader', self.as_is_element).get('guid')` +as a minimal guard) so a shape mismatch degrades to "treat as not found" +rather than crashing. Needs a live server that isn't under load (see +ISSUE-52) to safely re-verify the exact MetadataExpert raw shape without +every repro attempt itself timing out. -`pytest tests/ -m unit -q` passes (exit 0). +--- -**Original status:** open, found 2026-08-04 as a byproduct of the ISSUE-25 sentinel -audit. +### ISSUE-50: `base_report_formats.py`'s `Collections` FormatSet aliased the real Egeria type `ResultsSet` as `"ResultSet"` (missing the "s") -**Layer:** Pyegeria (`classification_explorer.py`, `runtime_manager.py`, -`project_manager.py`, `automated_curation.py`, `glossary_manager.py`). +**Status:** fixed 2026-08-09, alongside adding pyegeria SDK support for +Egeria PR #9200 (SavedQuery/SmartQuery/ResultsSet). Originally found +2026-08-05 building egeria-workspaces-fs's saved-query prototype (Egeria +Insights `EGERIA_INSIGHTS_QUERY_MODEL.md`, Track A), which needed to +create a real `ResultsSet` collection. -**What:** grep for `self\._async_\w+\(\s*\n(?:\s+\w+,?\s*\n)+\s*\)` (a -multi-line, all-positional call with no `=`, `*`, or `**` on any line) in -`pyegeria/omvs/*.py` turns up ~50 more call sites, mostly write/action -methods (set/clear classification, add/remove relationship) that don't -carry `graph_query_depth` — so ISSUE-25's sentinel-injection method (which -specifically hunted for that one parameter) couldn't confirm or rule them -out. ISSUE-21 and ISSUE-25's `find_root_elements` finding both prove this -call shape is a real, recurring risk in this codebase whenever a -delegate's parameter order doesn't exactly match the caller's. +Root cause: `pyegeria/view/base_report_formats.py`'s `"Collections"` +`FormatSet` listed `aliases=[..., "ResultSet", ...]` — missing the "s". +The real Egeria entity type is spelled **`ResultsSet`** (confirmed live +against `qs-view-server`'s `/api/types` entity catalog); passing +`"ResultSet"` as a `typeName` to `CollectionManager.create_collection` was +rejected outright by the server (`OMAG-COMMON-400-018`). Impact was +narrow but real: any caller rendering a genuine `ResultsSet` element +through `generate_output()`'s alias-based `FormatSet` lookup would miss +the match and silently fall through to a generic/default format instead +of `Collections`. -**Candidate fix:** for each of the ~50 call sites, diff the caller's -positional argument order against the target method's actual parameter -order (the same manual check done for `find_root_elements` in ISSUE-25) -and convert to keyword arguments wherever they've drifted. Not attempted -here — flagged for a dedicated follow-up pass. +Fix: one-line change, `"ResultSet"` → `"ResultsSet"` in the `aliases` +list (`pyegeria/view/base_report_formats.py`, `"Collections"` FormatSet). --- -### ISSUE-31: `MetadataExpert.delete_metadata_element` crashes with `AttributeError: 'NoneType' object has no attribute 'model_dump'` when called with no explicit body +### ISSUE-49: `MetadataExpert.get_metadata_element_relationships` (lookup a relationship by its two endpoint GUIDs) returns "No elements found" for a relationship confirmed to exist -**Status:** fixed 2026-08-15 (Pyegeria — `pyegeria/core/_server_client.py`). -`validate_open_metadata_delete_request` now builds a default -`{"class": "OpenMetadataDeleteRequestBody"}` when `body` is `None`, -matching `validate_delete_element_request`'s existing pattern, instead of -returning `None` straight into `.model_dump()`. Verified live: -`client.metadata_expert.delete_metadata_element()` with no body now -raises a clean `PyegeriaNotFoundException` for a nonexistent GUID instead -of crashing with `AttributeError`. Fixes the wider blast radius too, since -every other caller of the shared `_async_open_metadata_delete_body_request` -helper goes through this same validator. +**Re-verified 2026-08-15 (same day, follow-up run), no regression.** +Recreated the exact repro fresh (new `ResultsSet` + `SavedQuery` pair, +real `SmartQuery` link) — `get_metadata_element_relationships(rs_guid, +sq_guid, "SmartQuery", None)` correctly returns the real relationship +(`relationshipGUID`, both ends populated) instead of "No elements found". +Test elements cleaned up. `pytest tests/ -m unit` passes. -**Original status:** open, found 2026-08-04/05 while cleaning up throwaway test -elements created during the ISSUE-32 (`Note` type) live verification. +**Status:** fixed 2026-08-15 (Pyegeria — `pyegeria/omvs/metadata_expert.py`). +Root cause: the shared `process_related_element_list()` helper has two +envelope shapes it can parse — a related-*element* envelope +(`{"relatedElementList": {"elementList": [...]}}`) and a relationship-list +envelope (`{"relationshipList": {"relationships": [...]}}`) — selected by +its `relationship_list` parameter. Two bugs stacked: (1) +`_async_get_metadata_element_relationships` (and its sibling +`_async_get_all_metadata_element_relationships`) called the helper with +the default `relationship_list=False`, so it looked for the wrong +top-level key (`relatedElementList`, which doesn't exist in a relationship +response) and returned the `NO_ELEMENTS_FOUND` sentinel immediately; (2) +even with `relationship_list=True` set (as `_async_find_relationships_ +between_elements`, ISSUE-39's method, already did), the helper's inner +lookup used the wrong key too — `elementList` instead of the real +`relationships` key nested inside `relationshipList`. Confirmed live via +raw response inspection: `POST .../linked-by-type/{type}/to-elements/{guid}` +returns `{"relationshipList": {"relationships": [...]}}` — never +`relatedElementList`/`elementList` at all. -**Layer:** Pyegeria (`pyegeria/core/_server_client.py`). +**Fixed both bugs**: `process_related_element_list()` now looks up +`"relationships"` (not `"elementList"`) when `relationship_list=True`; the +two relationship-endpoint call sites that weren't already passing +`relationship_list=True` now do +(`_async_get_metadata_element_relationships`, +`_async_get_all_metadata_element_relationships`). -**What:** `_async_open_metadata_delete_body_request(url, body=None)` calls -`self.validate_open_metadata_delete_request(body)` and immediately calls -`.model_dump(...)` on the result — but `validate_open_metadata_delete_request` -returns `None` (not a default `OpenMetadataDeleteRequestBody()` instance) -when `body` is `None`, so the very next line always crashes for the common -"just delete it, no special options" case: +**Verified live**: recreated the exact repro below (`ResultsSet` + +`SavedQuery` + real `SmartQuery` link via +`link_saved_query_to_results_set`) — `get_metadata_element_relationships` +now returns the relationship with the matching GUID instead of "No +elements found". Also fixed ISSUE-39 as a side effect (same root cause, +same helper) — see that entry. `pytest tests/ -m unit` passes throughout. -```python -from pyegeria import EgeriaTech -client = EgeriaTech("qs-view-server", "https://localhost:9443", user_id="erinoverview") -client.create_egeria_bearer_token("erinoverview", "secret") -client.metadata_expert.delete_metadata_element("") -# AttributeError: 'NoneType' object has no attribute 'model_dump' -``` +**Original status:** open, found 2026-08-10 (Dan) building egeria-workspaces-fs's +migration of Egeria Insights' saved queries onto the real `SavedQuery`/ +`SmartQuery` types (`insights_handler.py`, Track C.1 of +`EGERIA_INSIGHTS_QUERY_MODEL.md`). -**Workaround (used during this investigation):** always pass an explicit -`body={"class": "OpenMetadataDeleteRequestBody"}` — works correctly once a -non-`None` body is supplied. +**Layer:** Pyegeria (`pyegeria/omvs/metadata_expert.py`) — or possibly the +view-service endpoint it calls; not yet narrowed further, see Candidate fix. -**Candidate fix:** `validate_open_metadata_delete_request` should construct -and return a default `OpenMetadataDeleteRequestBody()` when `body is None`, -matching the pattern every other `validate_*_request` helper in this file -uses (they all handle the "no body given" case by building sensible -defaults, not returning `None`). Not attempted here — found in passing, -outside the scope of the `Note` type work that surfaced it. +**What:** created a `SmartQuery` relationship between a `ResultsSet` and a +`SavedQuery` via `CollectionManager.link_saved_query_to_results_set()` +(itself new in 6.0.17.18, part of the same feature — see that method's own +docstring), which returned a real relationship GUID. Immediately after, +looked the same relationship up by its two endpoint GUIDs via +`MetadataExpert.get_metadata_element_relationships(end1_guid=, end2_guid=, relationship_type="SmartQuery", +body={"class": "ResultsRequestBody"})` — the exact body shape from the +method's own docstring sample — and got back the string `"No elements +found"` rather than the relationship. Tried both `body={}` and +`body={"class": "ResultsRequestBody"}`; same result both times. -**Confirmed wider blast radius (2026-08-09):** this isn't specific to -`delete_metadata_element` — `_async_open_metadata_delete_body_request` is a -shared base-class helper, so every caller with an optional/default-`None` -body hits the same crash. Reproduced live via the new -`CollectionManager._async_detach_saved_query_from_results_set` (added for -Egeria PR #9200's SmartQuery relationship, which itself calls -`_async_open_metadata_delete_body_request` the same way -`MetadataExpert._async_delete_related_elements` does) — worked around -locally in that method the same way (`body or {"class": -"OpenMetadataDeleteRequestBody"}`), rather than waiting on this issue. -Any other no-args delete-style call sharing this helper (grep -`_async_open_metadata_delete_body_request` for the full caller list) is -suspect until the base helper itself is fixed. +The relationship is genuinely there — independently confirmed via +`ClassificationExplorer.get_relationships(relationship_type="SmartQuery", +output_format="JSON")`, which lists it with the correct `end1`/`end2` and +the same GUID `link_saved_query_to_results_set()` returned. ---- +**Where seen:** `insights_handler.py`'s saved-query delete path would have +used this method to look up the `SmartQuery` relationship's GUID from the +`ResultsSet`/`SavedQuery` GUIDs alone, if it needed to. Worked around +entirely rather than root-caused further — see Candidate fix. -### ISSUE-45: `tests/functional-tests/test_my_profile.py::test_create_my_todo` has no teardown — leaves a live ToDo behind on every run +**Repro:** +```python +from pyegeria import CollectionManager, MetadataExpert +cm = CollectionManager(view_server="qs-view-server", platform_url=url, user_id=user, user_pwd=pwd) +cm.create_egeria_bearer_token() +rel_guid = cm.link_saved_query_to_results_set(results_set_guid, saved_query_guid) # succeeds, real guid -**Re-verified 2026-08-15 (same day, follow-up run), no regression.** -Re-ran the test directly (`PYEG_LIVE_EGERIA=1 pytest -tests/functional-tests/test_my_profile.py::TestMyProfile::test_create_my_todo --v`) — passes, prints its own "Deleted test to-do " confirmation, and -a direct `get_metadata_element_by_guid` on that exact GUID immediately -afterward correctly raises `PyegeriaNotFoundException` — genuinely gone -server-side, not just claimed by the test's own output. +me = MetadataExpert(view_server="qs-view-server", platform_url=url, user_id=user, user_pwd=pwd) +me.create_egeria_bearer_token() +me.get_metadata_element_relationships( + end1_guid=results_set_guid, end2_guid=saved_query_guid, + relationship_type="SmartQuery", body={"class": "ResultsRequestBody"}, +) +# -> "No elements found" -**Status:** fixed 2026-08-15 (test hygiene — -`tests/functional-tests/test_my_profile.py`). Added a `finally` block that -deletes the created `ToDo` via a `MetadataExpert` client -(`delete_metadata_element(guid, body={"class": -"OpenMetadataDeleteRequestBody"})`), matching the candidate fix below. -Verified live: ran the test, confirmed the `ToDo` was created (real GUID -returned), then confirmed via a direct `get_metadata_element_by_guid` call -immediately after the test run that the same GUID is now soft-deleted -(`OMRS-REPOSITORY-404-013`). `pytest tests/ -m unit` passes. +from pyegeria import ClassificationExplorer +ce = ClassificationExplorer(view_server="qs-view-server", platform_url=url, user_id=user, user_pwd=pwd) +ce.create_egeria_bearer_token() +ce.get_relationships(relationship_type="SmartQuery", output_format="JSON", body=None) +# -> [{"relationshipHeader": {..."guid": rel_guid...}, "end1": {...ResultsSet...}, "end2": {...SavedQuery...}}] +# same rel_guid, so the relationship is real and correctly formed +``` -**Original status:** open (test hygiene) — found 2026-08-05 while investigating -ISSUE-44's follow-up. Running the test suite against the live -`qs-view-server` created a real `ToDo` (`do-my-backup`, confirmed via -`find_metadata_elements(metadataElementTypeName="ToDo")` immediately after a -test run, `createTime` matching the run) that isn't deleted afterward. Over -repeated test runs this silently accumulates orphaned `ToDo` entities on -whatever server the suite is pointed at. Same class of issue as the -already-known pattern in this file of "confirm via live GUID, then clean up -afterward" — this test doesn't do the second half. +**Candidate fix:** not yet investigated — could be a body-shape mismatch +this repro hasn't found, a wrong end1/end2 ordering assumption in the +method's URL construction (`.../metadata-elements/{end1_guid}/linked-by- +type/{relationship_type}/to-elements/{end2_guid}`), or a real server-side +gap specific to this endpoint (`SmartQuery` has no bespoke view-service +endpoint of its own — types/properties only, per Egeria PR #9200 — so it's +plausible the generic linked-by-type endpoint doesn't handle every +relationship type uniformly). Whoever picks this up should try swapping +end1/end2, and try a relationship type with an established working example +of this same method (if one exists) to isolate whether `SmartQuery` +specifically is the problem or the method itself. -**Candidate fix:** add a fixture/teardown that deletes the created GUID via -`MetadataExpert.delete_metadata_element(guid, body={"class": -"OpenMetadataDeleteRequestBody"})`, matching the pattern used everywhere else -in this file's own verification steps. +**Workaround in use:** don't rely on this lookup at all — capture the +relationship GUID from `link_saved_query_to_results_set()`'s own return +value at creation time and persist it (in `insights_handler.py`'s case, in +the `SavedQuery`'s own `additionalProperties`), so any later operation that +needs to detach the relationship already has its GUID in hand. --- -### ISSUE-46: `logger.info(self.__str__(), ip=..., http_code=..., pyegeria_code=...)` crashed with `KeyError` on any exception whose message contained literal `{}` — masked the real error behind an opaque traceback across 5 of 7 exception classes +### ISSUE-48: No OMVS wrapper exists for the `SchemaAttributeDefinition` relationship (physical `SchemaAttribute` ↔ logical `DataField`) -**Status:** fixed 2026-08-05 (Pyegeria — `pyegeria/core/_exceptions.py`). Found -while investigating the user's report that "many CLI scripts under -`/commands` are broken" — the first repro (`create_todo` CLI) crashed not -with a clean Egeria error but with a bare `KeyError: '\`parameterName\`'`. +**Status:** implemented 2026-08-19 via Option 2 (the generic +`MetadataExpert` relationship mechanism) — deferred 2026-08-15, re-checked +2026-08-19 (still no bespoke Egeria endpoint, confirmed against the full +`/v3/api-docs` spec; Option 2 confirmed viable live), then built the same +day per the user's go-ahead. -**Root cause:** loguru's `Logger.info(message, *args, **kwargs)` always calls -`message.format(*args, **kwargs)` when any args/kwargs are given (this is -loguru's structured-logging feature — `logger.info("Processing {file}", -file=name)`). `PyegeriaException.__str__()` embeds `additional_info` values -(e.g. `exceptionProperties`) via `str(value).replace('"','\`').replace("'", -'\`')`, which preserves any literal `{`/`}` characters already present in a -dict repr (e.g. `` `{`parameterName`: `elementGUID`}` ``). Six call sites -across five exception classes did `logger.info(self.__str__(), ip=..., -http_code=..., pyegeria_code=...)` — passing that literal-brace string as -the format string. Whenever the underlying Egeria error included any -`exceptionProperties` (common - it's present on most `InvalidParameterException` -translations), `.format()` tried to resolve `parameterName` as a kwarg, -found only `ip`/`http_code`/`pyegeria_code`, and raised `KeyError` from -*inside the exception's own constructor* - so the caller never even got a -`PyegeriaException` to catch; it got a raw `KeyError` traceback instead. +**What shipped:** a new Dr.Egeria command family, `Link/Detach Schema +Attribute Definition` (`Data Designer` family, +`commands_data_designer.json`, new `Schema Attribute` attribute alongside +the existing `Data Field`), backed by a new +`LinkSchemaAttributeDefinitionProcessor` +(`md_processing/v2/data_designer.py`): +- **Link/Attach/Add:** calls + `MetadataExpert._async_create_related_elements` with a + `NewRelatedElementsRequestBody` (`typeName: SchemaAttributeDefinition`), + captures the returned relationship GUID into + `parsed_output["guid"]`, and displays it in the result markdown. +- **Detach/Unlink/Remove:** looks the relationship up by its element pair + via `_async_find_relationships_between_elements` + (`end1EntityGUIDs`/`end2EntityGUIDs`) rather than requiring a separate + GUID attribute — safe because `SchemaAttributeDefinition`'s + `relationshipCategory` is `UNI_LINK` with `AT_MOST_ONE` cardinality on + both ends (confirmed live via + `ValidMetadataManager.get_all_relationship_defs()`), so there's no + multi-instance ambiguity to resolve. Then deletes via + `_async_delete_related_elements(relationship_guid, ...)`. +- `supports_target_element_lookup()` overridden to `False` (relationship- + only processor — see ISSUE-68's `AsyncBaseCommandProcessor` upsert- + rewrite-bug fix for why this matters; avoids the same latent Update-verb + hazard flagged there for any future relationship processor). -**Not a new regression** - `PyegeriaAPIException.__init__` (the class -actually raised for real Egeria error responses) already had the fix, -complete with an explanatory comment: `msg.replace("{", "{{").replace("}", -"}}")`. But the identical `logger.info(self.__str__(), ...)` pattern in -`PyegeriaInvalidParameterException`, `PyegeriaClientException`, -`PyegeriaUnauthorizedException`, `PyegeriaNotFoundException`, and -`PyegeriaUnknownException` never got the same treatment - so any 404, 401, -or invalid-parameter response with a curly-brace-shaped error message -crashed instead of printing. +**No new pyegeria SDK surface required** — `_async_create_related_elements`/ +`_async_find_relationships_between_elements`/`_async_delete_related_elements` +all already existed on `MetadataExpert`. -**Fix:** applied the same `.replace("{", "{{").replace("}", "}}")` escaping -to all five remaining call sites. +**Verified live**, full round trip against a real `DataField`/ +`SchemaAttribute` pair: `Link Schema Attribute Definition` via +`--process` created the relationship and displayed its GUID; a direct +`_async_find_relationships_between_elements` query confirmed it existed; +`Detach Schema Attribute Definition` via `--process` removed it; the same +query afterward returned "No elements found", confirming clean deletion. -**Verified live:** re-ran `create_todo --name ... --description ...` (no -`--assigned-to`, so the CLI's stale hardcoded `peter_guid` default 404s) - -before the fix this crashed with `KeyError: '\`parameterName\`'`; after the -fix it prints a clean, readable Egeria error message via -`print_basic_exception` with no traceback. `tests/micro-tests` clean (same -one pre-existing unrelated failure). +**Tests:** `test_link_schema_attribute_definition.py` (5 tests, fake +client, no live server needed). + +**Explicitly temporary, per the user's own framing** ("implement Option 2 +... and we'll replace it once the new endpoint is available") — re-check +this issue's own "still deferred" 2026-08-19 OpenAPI-spec check +periodically; once Egeria ships a bespoke `SchemaAttributeDefinition` +endpoint, migrate `LinkSchemaAttributeDefinitionProcessor` to a dedicated +`DataDesigner._async_link_schema_attribute_definition`/ +`_async_detach_schema_attribute_definition` wrapper (Option 1) instead — +same compact-spec command, same attributes, just a different +implementation underneath. --- @@ -7107,258 +6870,523 @@ reporting → reassign/update → Delete) for ToDo/Meeting/Review that asserts real failures rather than swallowing them - written specifically to catch a regression on ISSUE-44's three bugs. ---- +--- + +### ISSUE-46: `logger.info(self.__str__(), ip=..., http_code=..., pyegeria_code=...)` crashed with `KeyError` on any exception whose message contained literal `{}` — masked the real error behind an opaque traceback across 5 of 7 exception classes + +**Status:** fixed 2026-08-05 (Pyegeria — `pyegeria/core/_exceptions.py`). Found +while investigating the user's report that "many CLI scripts under +`/commands` are broken" — the first repro (`create_todo` CLI) crashed not +with a clean Egeria error but with a bare `KeyError: '\`parameterName\`'`. + +**Root cause:** loguru's `Logger.info(message, *args, **kwargs)` always calls +`message.format(*args, **kwargs)` when any args/kwargs are given (this is +loguru's structured-logging feature — `logger.info("Processing {file}", +file=name)`). `PyegeriaException.__str__()` embeds `additional_info` values +(e.g. `exceptionProperties`) via `str(value).replace('"','\`').replace("'", +'\`')`, which preserves any literal `{`/`}` characters already present in a +dict repr (e.g. `` `{`parameterName`: `elementGUID`}` ``). Six call sites +across five exception classes did `logger.info(self.__str__(), ip=..., +http_code=..., pyegeria_code=...)` — passing that literal-brace string as +the format string. Whenever the underlying Egeria error included any +`exceptionProperties` (common - it's present on most `InvalidParameterException` +translations), `.format()` tried to resolve `parameterName` as a kwarg, +found only `ip`/`http_code`/`pyegeria_code`, and raised `KeyError` from +*inside the exception's own constructor* - so the caller never even got a +`PyegeriaException` to catch; it got a raw `KeyError` traceback instead. + +**Not a new regression** - `PyegeriaAPIException.__init__` (the class +actually raised for real Egeria error responses) already had the fix, +complete with an explanatory comment: `msg.replace("{", "{{").replace("}", +"}}")`. But the identical `logger.info(self.__str__(), ...)` pattern in +`PyegeriaInvalidParameterException`, `PyegeriaClientException`, +`PyegeriaUnauthorizedException`, `PyegeriaNotFoundException`, and +`PyegeriaUnknownException` never got the same treatment - so any 404, 401, +or invalid-parameter response with a curly-brace-shaped error message +crashed instead of printing. + +**Fix:** applied the same `.replace("{", "{{").replace("}", "}}")` escaping +to all five remaining call sites. + +**Verified live:** re-ran `create_todo --name ... --description ...` (no +`--assigned-to`, so the CLI's stale hardcoded `peter_guid` default 404s) - +before the fix this crashed with `KeyError: '\`parameterName\`'`; after the +fix it prints a clean, readable Egeria error message via +`print_basic_exception` with no traceback. `tests/micro-tests` clean (same +one pre-existing unrelated failure). + +--- + +### ISSUE-45: `tests/functional-tests/test_my_profile.py::test_create_my_todo` has no teardown — leaves a live ToDo behind on every run + +**Re-verified 2026-08-15 (same day, follow-up run), no regression.** +Re-ran the test directly (`PYEG_LIVE_EGERIA=1 pytest +tests/functional-tests/test_my_profile.py::TestMyProfile::test_create_my_todo +-v`) — passes, prints its own "Deleted test to-do " confirmation, and +a direct `get_metadata_element_by_guid` on that exact GUID immediately +afterward correctly raises `PyegeriaNotFoundException` — genuinely gone +server-side, not just claimed by the test's own output. + +**Status:** fixed 2026-08-15 (test hygiene — +`tests/functional-tests/test_my_profile.py`). Added a `finally` block that +deletes the created `ToDo` via a `MetadataExpert` client +(`delete_metadata_element(guid, body={"class": +"OpenMetadataDeleteRequestBody"})`), matching the candidate fix below. +Verified live: ran the test, confirmed the `ToDo` was created (real GUID +returned), then confirmed via a direct `get_metadata_element_by_guid` call +immediately after the test run that the same GUID is now soft-deleted +(`OMRS-REPOSITORY-404-013`). `pytest tests/ -m unit` passes. + +**Original status:** open (test hygiene) — found 2026-08-05 while investigating +ISSUE-44's follow-up. Running the test suite against the live +`qs-view-server` created a real `ToDo` (`do-my-backup`, confirmed via +`find_metadata_elements(metadataElementTypeName="ToDo")` immediately after a +test run, `createTime` matching the run) that isn't deleted afterward. Over +repeated test runs this silently accumulates orphaned `ToDo` entities on +whatever server the suite is pointed at. Same class of issue as the +already-known pattern in this file of "confirm via live GUID, then clean up +afterward" — this test doesn't do the second half. + +**Candidate fix:** add a fixture/teardown that deletes the created GUID via +`MetadataExpert.delete_metadata_element(guid, body={"class": +"OpenMetadataDeleteRequestBody"})`, matching the pattern used everywhere else +in this file's own verification steps. + +--- + +### ISSUE-44: `create_my_todo`/`create_meeting`/`create_review` (my_profile.py) always invented their own `qualifiedName` (random timestamp suffix, wrong casing), silently diverging from what Dr.Egeria's `Create ToDo`/`Create Meeting`/`Create Review` reported having created + +**Status:** fixed 2026-08-05 (Pyegeria — `pyegeria/omvs/my_profile.py` + +`md_processing/v2/actor_manager.py`/`project.py`/`feedback.py`). Reported +directly by the user: "the Dr.Egeria command Create ToDo does not appear to +create a todo." + +**What actually happened:** `Create ToDo` genuinely succeeded — a real +`ToDo` entity was created, correct `displayName`/`description`/`situation`/ +`priority`/`activityStatus`, `SUCCESS` reported with a real GUID. But +`_async_create_my_todo(todo_name, ...)` has no parameter at all for +accepting a caller-supplied qualified name — it always computes its own via +`self.__create_qualified_name__("Todo", todo_name)+f"-{int(time.time())}"`, +ignoring whatever qualified name Dr.Egeria's dispatcher already derived and +displays in its own analysis table. Confirmed live: Dr.Egeria reported +creating `Coco Pharmaceuticals::ToDo::Go-to-Lunch-with-Peter`; the entity +actually stored was `Coco Pharmaceuticals::Todo::Go-to-Lunch-with-Peter-1785955953` +— different casing (`Todo` vs `ToDo`) *and* an unpredictable numeric +suffix. Anyone searching for the element by the name Dr.Egeria reported — +Egeria Explorer, a fresh `find_*` call, a later Dr.Egeria command +referencing it by qualified name in a separate run — finds nothing, which +is exactly why it "does not appear to" have been created even though it +was. + +**Systemic, not isolated:** `_async_create_meeting` and `_async_create_review` +have the identical pattern (confirmed by reading both) — both already wired +as Dr.Egeria commands (`Create Meeting` via `ProjectProcessor`, `Create +Review` via `FeedbackProcessor`), so both had the same latent bug. + +**Fix:** added an optional `qualified_name: Optional[str] = None` parameter +to all three async methods and their sync wrappers — used verbatim when +provided (falls back to the existing auto-generation only when the caller +doesn't supply one, so no other caller's behavior changes). Updated all +three Dr.Egeria processor call sites +(`actor_manager.py`'s `Create ToDo` branch, `project.py`'s `Create Meeting` +branch, `feedback.py`'s `Create Review` branch) to pass the `qualified_name` +the dispatcher already computed. + +**Verified live:** re-ran the user's exact `actor_actions.md` — `Create +ToDo` still `SUCCESS`, and the real stored element's `qualifiedName` now +matches Dr.Egeria's reported qualified name exactly (`Coco +Pharmaceuticals::ToDo::Go-to-Lunch-with-Peter`, confirmed via direct +`get_element_by_guid` comparison). Cleaned up both the original mismatched +throwaway and the corrected one afterward. Full `tests/micro-tests` suite +clean (same one pre-existing unrelated failure). + +**Follow-up 2026-08-05 — the deeper bug: assigned ToDos never showed up in +`get_my_to_dos()` at all.** User reported: "I ran test_get_to_dos in +test_my_profile.py as erinoverview and don't see any Todos?" Root-caused in +three layers, all now fixed: + +1. **`_async_get_my_assigned_actions` had a parameter-name bug** (typo + `metadtata_element_subtypes` plus wrong field names `metadata_element_type`/ + `metadata_element_subtypes` instead of the real `GetRequestBody` fields + `metadata_element_type_name`/`metadata_element_subtype_names`) — silently + dropped by pydantic's `extra='ignore'` on every call. Fixing the names + uncovered that this endpoint's own worked example + (`Egeria-api-my-profile.http`) never sends these fields at all, and doing + so live produces `OMAG-COMMON-400-019 ... type name ToDo ... is not a + sub-type of UserIdentity` — the endpoint evidently applies them against + the assigned actor's type, not the action's. Fix: stop forwarding these + two fields into the request body for this endpoint (kept as inert public + parameters for call-site compatibility with `get_my_to_dos`/ + `get_my_sponsored_actions`). +2. **The actual root cause: `_async_create_my_todo`/`_async_create_meeting`/ + `_async_create_review` put `originatorGUID`/`assignToActorGUID` *inside* + `properties` (i.e. as fields of `ToDoProperties`/`MeetingProperties`/ + `ReviewProperties`)**, instead of at the top level of `ActionRequestBody` + as siblings of `properties` — confirmed against + `AssetMaker._async_create_action`'s own docstring/worked example, which + shows `originatorGUID`/`actionSponsorGUID`/`assignToActorGUID` as + top-level `ActionRequestBody` fields. Since `ToDoProperties` etc. have no + such fields, they were silently dropped there too — the create call always + succeeded, but no `ActionRequester`/`AssignmentScope` relationship was + ever created, so the new ToDo was structurally invisible to any + "assigned to me" query, no matter how the query itself was fixed. This + was misdiagnosed mid-investigation as a possible Egeria-server-side gap + (a raw REST call reproducing the same nested-properties mistake showed + the identical missing-relationship symptom) before the nesting bug was + spotted — flagged by the user, not found independently. Fix: moved both + fields to the top level of the body in all three methods. +3. Also added `qualified_name: Optional[str] = None` support to + `_async_log_my_activity`/`_async_journal_my_activity`/`_async_blog_my_activity` + (the milder, deterministic-but-still-caller-ignored variant noted below in + the original write-up), and added hand-maintained `ToDo-DrE`/`Meeting-DrE`/ + `Review-DrE` `FormatSet`s to `base_report_specs` (same fallback pattern as + the existing `Journal-Entry-DrE`), fixing the `Report spec + 'ToDo-DrE-Basic' not found. Falling back to default.` warning noted below. + +**Verified live (2026-08-05):** created a fresh ToDo via `create_my_todo()`; +`get_all_related_elements(guid)` now shows a real `ActionRequester` +relationship to the `Person` entity; `get_my_to_dos(output_format='JSON')` +returns the new ToDo's GUID. `select_report_spec('ToDo-DrE-Basic', 'MD')` +correctly falls through to the new `ToDo-DrE` spec. Cleaned up all +throwaway test elements afterward. + +**Second follow-up 2026-08-05 — checked Meeting/Review through to reporting +too, not just ToDo:** + +4. Verified live that `create_meeting`/`create_review` also now produce real + `ActionRequester`/`AssignmentScope` relationships after the fix, and both + show up via `get_my_assigned_actions()`. +5. **Found and fixed a separate, `Meeting`-specific bug**: + `ProjectProcessor.fetch_element()` (`md_processing/v2/project.py`) + unconditionally calls `_async_get_project_by_guid` — but `Meeting` is a + Person Action Base type routed through `my_profile.create_meeting`, not a + `Project` at all (see the `Meeting` branch in `apply_changes()`). Calling + the Project-typed getter on a Meeting GUID 404s server-side + (`OMAG-REPOSITORY-HANDLER-404-001 ... retrieved an object ... of type + Meeting rather than type Project`), which `fetch_element` silently + swallowed and turned into "Could not fetch element" — so + `render_result_markdown` never rendered a Meeting's report at all, always + falling back to `raw_block`. `Review` was unaffected (`FeedbackProcessor` + doesn't override `fetch_element`, so it already used the generic + `ClassificationExplorer`-based fetch in the base class). Fix: special-case + `Meeting` in `ProjectProcessor.fetch_element()` to use the inherited + base-class fetch instead. Verified live end-to-end (`fetch_element` + + `render_result_markdown`) — a Meeting now renders its full `Meeting-DrE` + report with no warnings. +6. Confirmed `AssetMaker._async_create_action`'s own worked examples in + `Egeria-api-asset-maker.http` were already correct (top-level + `originatorGUID`/`actionSponsorGUID`/`assignToActorGUID`) — the bug was + purely in `my_profile.py` not following its own ground truth, not a gap in + the `.http` file. Confirmed no other caller of `_async_create_action` + exists, so the bug's blast radius was exactly the three methods fixed + above. +7. Added a `KNOWN ISSUE` comment directly above the `getMyAssignedActions` + worked example in `Egeria-api-my-profile.http`, documenting the confirmed + `metadataElementTypeName`/`metadataElementSubtypeNames` 400 so it isn't + reintroduced by a future edit that "helpfully" adds those fields back + without re-checking live. +8. **Pre-existing broken ToDos found and deleted** (user's explicit + decision): 6 `ToDo` entities existed from before this fix (`Go to Lunch + with Peter` ×2, `Have a burger`/`Burger` ×2, `Curation Smoke ToDo`, plus + one test-run leftover `do-my-backup`) — all created via the pre-fix nested + -properties bug, so none had an `ActionRequester`/`AssignmentScope` + relationship and none would ever appear in `get_my_to_dos()`. There is no + retroactive repair (the relationship was simply never created); user chose + to delete all 6 rather than recreate them. Also note: `do-my-backup` + reveals `tests/functional-tests/test_my_profile.py::test_create_my_todo` + has no teardown/cleanup — it leaves a live ToDo behind on every run. + +--- + +### ISSUE-43: Pagination location for `find_metadata_elements`'s endpoint — audit finding, plausible but not yet confirmed live + +**Status:** fixed 2026-08-15 (Pyegeria — `pyegeria/omvs/runtime_manager.py`). +Resolved the "not yet empirically confirmed" `_async_get_server_templates_by_dep_impl_type` +finding via ground-truth comparison rather than waiting for 2+-page demo +data (the blocker noted below, still unavailable): checked +`Egeria-api-runtime-manager.http`'s `getServerTemplatesByDeployedImplementationType` +worked example — a plain `POST .../software-servers/by-deployed-implementation-type` +with **no query string at all**, pagination and the `Template` filter both +carried entirely in the `FilterRequestBody` (`includeOnlyClassifiedElements: +["Template"]`). The sibling, non-templates method in the same file +(`_async_get_servers_by_dep_impl_type`) already builds the URL with no query +string, confirming the correct pattern exists right next to the bug. Removed +the spurious `?startFrom={..}&pageSize={..}&getTemplates=true` suffix +entirely — none of those three query params exist on the real endpoint; +pagination was already being built correctly into the body by the shared +`_async_get_name_request` helper regardless (confirmed via a request-body +spy, before and after: `startFrom`/`pageSize` land in the body either way, +so the URL suffix was purely spurious/redundant, not silently required for +anything to work). + +**Verified live:** the call now succeeds against `qs-view-server` +(`get_server_templates_by_dep_impl_type("cots-application-server", ...)` → +`No elements found`, same empty-but-successful result as before, for the +reason already documented below — no `Template`-classified software servers +of any tested type exist in this demo dataset, so a real 2+-page comparison +still isn't possible here). `pytest tests/ -m unit` passes. -### ISSUE-22: `Ownership`/`Impact`/`Confidence`/`Confidentiality`/`Criticality` classification `status` field expects an int enum, not the free-text value the Dr.Egeria "Status" attribute style implies +**Original status:** open (pyegeria, suspected) — found 2026-08-05 while auditing +for the same drift class as ISSUE-34 (see that entry's "design principle" +paragraph — Egeria's pagination convention moved from URL query params to +request-body fields for at least one endpoint a few months ago; worth +checking any method still doing URL-based `startFrom`/`pageSize` +injection). Grepped `pyegeria/omvs/*.py` for the same +`?startFrom={..}&pageSize={..}` URL-construction shape ISSUE-34 had; found +2 more matches besides `metadata_expert.py` (already fixed there) — +`runtime_manager.py` and `valid_metadata.py`. -**Status:** fixed 2026-08-15. `Ownership` was never actually affected — its -`ClassificationSpec` uses its own explicit field map (`owner`/ -`ownerTypeName`/`ownerPropertyName`), not `_GOVERNANCE_SHARED_FIELDS`, so -the issue title overstated its scope; `Retention` *is* affected (it also -consumes `_GOVERNANCE_SHARED_FIELDS`) and is fixed by the same change as -`Impact`/`Confidence`/`Confidentiality`/`Criticality`. +**`valid_metadata.py`'s `_async_get_valid_metadata_values`: checked, NOT a +bug.** This is a plain `GET` request (`get-valid-metadata-values/{property_name}`) +— there is no request body at all for a GET, so URL query parameters are +the only place `startFrom`/`pageSize` *can* go. Confirmed against the +`.http` ground truth (`Egeria-api-classification-explorer.http`) — also a +bare GET, no body shown. Correctly implemented as-is. -Two separate bugs were involved: -1. **Attribute-definition gap.** Between this issue being filed and now, - the compact-spec attribute had already been renamed `Status` → - `Governance Status` and migrated to `style: "Valid Value"` — but with no - explicit `property_name`, so it auto-derived to `governanceStatus` (a - property that doesn't exist), causing the live valid-metadata-value - lookup to silently return empty and fall through to a stale ALL-CAPS - fallback list, passing through an un-converted string. Fixed via the - Spec Editor API: `Governance Status` now has `property_name: - "statusIdentifier"` and `valid_values` = `["Discovered", "Proposed", - "Imported", "Validated", "Deprecated", "Obsolete", "Other"]`, matching - `GovernanceClassificationStatus`. -2. **Wire-key mapping bug**, in `md_processing/v2/curation.py`: - `_GOVERNANCE_SHARED_FIELDS` mapped `"Governance Status"` to the outgoing - JSON key `"status"`, but the real field on `ImpactProperties`/ - `ConfidenceProperties`/`ConfidentialityProperties`/`CriticalityProperties`/ - `RetentionClassificationProperties` is `statusIdentifier` — `"status"` is - silently ignored by the server (confirmed live: sending `"status": 4` - left readback `statusIdentifier: "0"`; sending `"statusIdentifier": 4` - directly persisted correctly). Fixed: `_GOVERNANCE_SHARED_FIELDS = - {"Governance Status": "statusIdentifier", ...}`. +**`runtime_manager.py`'s `_async_get_server_templates_by_dep_impl_type`: +suspicious, not yet confirmed.** Line ~2612 unconditionally appends +`?startFrom={start_from}&pageSize={page_size}&getTemplates=true` to the URL +*regardless* of whether `start_from`/`page_size` are also meaningful via +the `body` path — and the shared helper it delegates to +(`ServerClient._async_get_name_request`) **already** builds a correct +`"startFrom"`/`"pageSize"` pair into the body it constructs when the +caller doesn't supply an explicit `body` (confirmed by reading +`_async_get_name_request`'s source directly, `_server_client.py` ~line +6493). So the URL suffix here is at best redundant with what the body path +already does correctly, and at worst sends conflicting information when a +caller supplies their own explicit `body` without `startFrom`/`pageSize` +set (in that case pagination could only come from the URL, which per +ISSUE-34's finding is exactly the convention that stopped working for a +sibling endpoint). -Verified end-to-end through the real Dr.Egeria pipeline (not just manual -body construction): `Create Data Structure` + `Classify Impact` with -`Governance Status: Validated`, `Level Identifier: 2`, `Steward: -test-steward` — both commands SUCCESS, and a direct element fetch afterward -showed `statusIdentifier: 3` (the correct ordinal for `Validated`) alongside -`severityLevel: 2` and `steward: "test-steward"`. Test element deleted -afterward. `pytest tests/ -m unit -q` passes (exit 0). +**Not yet empirically confirmed**, unlike ISSUE-34's `find_metadata_elements` +finding — this demo dataset doesn't have enough `Template`-classified +servers of the same `deployedImplementationType` to construct a real +2-page pagination test (tried `cots-application-server` — 5 total +`SoftwareServer`s of that type exist, but zero are `Template`-classified; +every filter value tested returned either 0 or 1 `Template`-classified +result, never 2+, so a same-content-vs-distinct-content comparison isn't +possible with this data). **Next step:** either seed enough demo data to +get a real 2+ result test, or read the Java source for this specific +endpoint the way ISSUE-15's `QueryBuilder` investigation did, rather than +guessing from the pattern match alone. -**Original status:** open, found 2026-08-03 during a live smoke test of `Classify -Impact` (Curation family). +--- -**Layer:** Not strictly a pyegeria bug — a Dr.Egeria compact-spec design -gap (`md_processing/data/compact_commands/commands_curation_compact.json`, -`Status` attribute, style `Simple`) combined with `curation.py`'s field -mapping (`_GOVERNANCE_SHARED_FIELDS = {"Status": "status", ...}`). +### ISSUE-42 (PY-22): `ProjectManager.get_linked_projects()` didn't surface real `ProjectHierarchy` relationships -**What:** `ImpactProperties`/etc.'s real `status` field is an integer -`GovernanceClassificationStatus` enum ordinal (per -`Egeria-api-classification-explorer.http`'s worked example: `"status" : 0`), -not a free-text string. Passing `"ACTIVE"` (a plausible-looking value given -the attribute's `Simple` style) doesn't error — it's silently dropped by -the server, and the classification's `statusIdentifier` (the field's -readback name) stays at its default `0`. +**Status: FIXED** (2026-08-05, `pyegeria/core/_server_client.py`). -**Where seen:** live repro — `Classify Impact` with `Status: ACTIVE`, -`Level Identifier: 2`, `Steward`/`Source`/`Description` set. Fetching the -classified element back showed `severityLevel: "2"`, `steward`, `source`, -`notes` all correct, but `statusIdentifier: "0"` regardless of the `Status` -value sent — confirms `status` never took effect as text. +**Layer:** Pyegeria (shared response-parsing helper, not the Egeria +server). Originally found 2026-07-31 while finishing an `as_of_time` +verification pass; consolidated in from `egeria-workspaces-fs/ +PYEGERIA_ISSUES.md` 2026-08-05 as open, then investigated and fixed the +same day. -**Candidate fix:** change the `Status` attribute's style to `Valid Value`/ -`Enum` with the real `GovernanceClassificationStatus` ordinal labels (needs -confirming the exact enum from the Egeria type system rather than guessing -further), authored in Tinderbox per the usual compact-spec workflow — not -attempted here, since it's a spec authoring change, not a Curation -processor-wiring bug. Doesn't block Impact/Confidence/etc. classification -overall — `severityLevel`, `steward`, `source`, `notes` all confirmed -working correctly. +**Root cause:** `get_linked_projects` delegates to the shared +`ServerClient._async_get_guid_request` helper (used by 40+ callers across +the OMVS classes), which parsed a response by checking only two keys — +singular `"element"`, then `"elementGraph"` — before giving up and +returning `NO_ELEMENTS_FOUND`. But the real Egeria response for this +endpoint (`.../project-manager/metadata-elements/{guid}/projects`) returns +a genuine list under the **plural** `"elements"` key, which was never +checked at all. Confirmed live via a request-spy on `_async_make_request`: ---- +``` +RAW RESPONSE KEYS: ['class', 'requestId', 'relatedHTTPCode', 'elements'] +get_linked_projects("5d0057f6-...") -> "No elements found" # wrong +# but the raw body's "elements" list genuinely had 1 item +``` -### ISSUE-14 (PY-4): `update_comment` demands `qualifiedName` even with `mergeUpdate: true` +**Confirmed not a data-availability problem** (as originally suspected +when this was found): checked all 29 qs demo projects — every single one +returned `"No elements found"` regardless of whether it had real linked +projects (confirmed via `get_project_by_guid`'s own `managedProjects` +field, which does surface the same `ProjectHierarchy` relationships +correctly through a different code path). -**Status:** tracker was stale — re-checked 2026-08-15 and found the -workaround has actually been **in pyegeria itself** since 2026-08-03 -(`92cde6a`), not "application code" as this entry previously said. -`_async_update_comment` (`pyegeria/core/_server_client.py`) already -auto-fetches the comment's current `qualifiedName` via -`_async_get_comment_by_guid` whenever the caller's body doesn't supply one, -and injects it before sending — the exact workaround this entry describes, -just already shipped and never reflected here. **Verified live**: created a -throwaway `DataStructure`, attached a comment via `add_comment_to_element`, -then called `update_comment(comment_guid, comment="edited text")` with no -`qualifiedName` in the body at all — succeeds cleanly (previously would -have hit `OMAG-METADATA-400-004`). Cleaned up the test element afterward. -Underlying server behavior (still demanding `qualifiedName` despite -`mergeUpdate: true`) is unchanged and still worth an upstream Egeria report, -but there is nothing left to do on the pyegeria side — every caller of -`update_comment`/`_async_update_comment` already gets the workaround for -free, no application-level workaround needed. +**Fix:** added the plural `"elements"` key as a third fallback in +`_async_get_guid_request`, checked *last* (only reached once both +`"element"` and `"elementGraph"` have already failed) — purely additive, +so the other 40+ callers whose endpoints genuinely return the singular +shape are unaffected; none of their existing behavior changes since they +never reach the new branch. -**Original status:** open (Egeria server) — workaround shipped in application code, -not a pyegeria fix. Passing `"mergeUpdate": true` should allow a partial -update (only supplied fields required), but the server still demands -`qualifiedName` regardless. +**Verified:** +- Live against `qs-view-server`: `get_linked_projects("5d0057f6-...")` (no + body, matching the original repro exactly) now returns a real 1-element + list (`"Sustainability Bootstrap Project"`); a second project + ("Next 30 Days (Initiation Phase)") returns 11 linked projects — matches + its 10 `managedProjects` (children) + 1 `managingProjects` (parent), + consistent with the method's own docstring ("any relationship will do"). +- New unit tests, `tests/micro-tests/test_get_guid_request.py` (4 tests, + mocked `_async_make_request`): the plural-`elements` case now parses + correctly; the singular-`element` and `elementGraph` cases are + unaffected (still checked first); the true-empty case still degrades to + `NO_ELEMENTS_FOUND`, not an exception. +- Full micro-test suite: 228 passed (was 224; +4 new), same single + pre-existing unrelated failure (`test_dashboard_sheet_models.py`) as + before this change — no regressions across the other 40+ callers of the + shared helper. -**Original workaround (superseded, no longer needed by callers):** fetch the -element first via `get_comment_by_guid`, extract `qualifiedName`, and always -include it in the update body regardless of `mergeUpdate`. +--- -```python -from pyegeria import EgeriaTech -client = EgeriaTech(view_server="qs-view-server", platform_url="https://localhost:9443", - user_id="erinoverview", user_pwd="secret") -client.create_egeria_bearer_token() -client.update_comment(comment_guid="", body={"commentText": "edited text"}) -# OMAG-METADATA-400-004 demanding qualifiedName even though mergeUpdate: true is set. -``` +### ISSUE-41 (PY-21): `find_glossary_terms(sequencing_order=..., include_only_classified_elements=...)` returns ZERO results when combined — each filter alone works fine ---- +**Status:** fixed 2026-08-30 (Egeria server — `omf-metadata-server`, +`OpenMetadataStoreRESTServices.java`, commit `0d8d079c50`). The cause turned out not +to be `sequencing_order` at all, which is why this entry's trigger condition kept +shifting under re-testing. -### ISSUE-17 (PY-13): `SpecificationProperties.get_specification_property_by_type` always returns 400 +`includeOnlyClassifiedElements`/`skipClassifiedElements` are carried on every search +request body, but on the by-search-string path they were never expressed as part of +the query. The repository chose the page and the filter was applied afterwards to +whatever came back — so `find_glossary_terms` was handed whatever fraction of the +first page happened to carry the classification. Adding `sequencing_order` changed +which 200 elements landed on page 1, and so changed how many survived the filter; +zero was an ordinary outcome, not a special one. The 2026-08-27 "regression" — the +classification filter alone returning 0, with no sequencing involved — was the same +cause with more data in the repository, not a strictly worse second defect. The same +reasoning explains why `["Template"]` also returned 0 while an unfiltered call +returned a full page, and why `get_term_by_guid` on a "missing" term correctly showed +the classification: the data was always intact and always findable; it just was not +on the page the repository had already chosen. -**Status:** fixed 2026-08-15 (Pyegeria — `pyegeria/omvs/valid_metadata.py`). -**The original diagnosis was wrong** — not a server-side enum-binding drift. -Checked `Egeria-api-valid-metadata.http`'s `getSpecificationPropertyByType` -worked example: it uses `specificationPropertyType=PRODUCED_GUARD` — -`SCREAMING_SNAKE_CASE`, not the value shape the original repro tried -(`"SpecificationPropertyType{placeholderProperty}"`, nor the raw -PascalCase keys `get_specification_property_types()` returns, e.g. -`"ProducedGuard"`). Tried `PRODUCED_GUARD` directly — **succeeded -immediately**, no 400 at all. Root cause: `get_specification_property_types()` -returns its `stringMap` verbatim from the server's own -`specification-properties/type-names` endpoint, which genuinely uses -PascalCase keys (`"ProducedGuard"`) — a different server endpoint using a -different casing convention for the same enum than `by-type` expects. This -is a real cross-endpoint inconsistency in the server, but the client-visible -symptom ("every input 400s") was entirely avoidable: `get_specification_property_by_type` -did zero conversion, so the single most natural, discoverable input for a -caller (feeding it a key from `get_specification_property_types()`) always -failed. +The filters are now pushed into the query, merged with the anchor conditions where a +search already has them. Only one of the two can be pushed down at a time, because a +classification search carries a single match criteria and "must have these" cannot be +combined with "must not have those" — the required classifications win and the +forbidden ones fall through to the existing post-retrieval check. -**Fix:** `_async_get_specification_property_by_type` now auto-converts a -PascalCase/camelCase input to `SCREAMING_SNAKE_CASE` before building the URL -(a plain regex insert-underscore-before-each-capital + uppercase — no -hardcoded type list, so it isn't tied to today's enum values); an -already-`SCREAMING_SNAKE_CASE` value passes through untouched. +**Regression cover:** `ClassificationFilterPushdownFVT` (in +`open-metadata-test/open-metadata-fvt/query-fvt`) puts more unclassified elements +than fit on a page ahead of the classified ones and checks that a single request +returns the classified elements. With the fix reverted it fails with +`expected: <[…6 GUIDs…]> but was: <[]>` — this entry's symptom exactly. -**Verified live** against `qs-view-server`: all 12 real specification -property types returned by `get_specification_property_types()`, fed -straight into `get_specification_property_by_type()` unmodified, now -resolve successfully (10 return real results, 2 correctly return "No -elements found" — no data of that type, not an error); the already-correct -`PRODUCED_GUARD` form still works unchanged. `pytest tests/ -m unit` -passes. +**pyegeria follow-on:** `perspectives_handler.py`'s `get_questions()` can drop the +2026-08-27 workaround (fetch unfiltered at `page_size=1000`, filter client-side on +`otherClassifications`) and go back to a server-side +`include_only_classified_elements` call. Two things to preserve when doing so: -**Impact on the workaround downstream:** `find_specification_property("*", -...)` + client-side filtering is no longer necessary to work around this -specific method, though it may still be independently useful/faster for -some callers. +- **Page until a null, not until an empty list.** Empty pages still occur — from + `skipClassifiedElements` where it cannot be pushed down, from the visibility + filtering every search applies, and from the client-side subtype/classification + checks. A single call that stops on an empty list can still under-report. This is + Egeria's documented paging contract and is unchanged by the fix. +- **`findMetadataElements`/`countMetadataElements` are not covered.** Those request + bodies still ignore `includeOnlyClassifiedElements` — they take the OMF-framework + `SearchClassifications` rather than the OMRS one, so merging there needs a second + typed variant that was left out of this change. Callers that reach the repository + through those two endpoints still get no server-side classification filtering. -**Original status:** open (Egeria server), re-verified 2026-07-31 (live) — still -reproduces exactly as described. Root cause is server-side: the OpenAPI -schema declares `specificationPropertyType` as a required enum query param -with values like `"SpecificationPropertyType{placeholderProperty}"`, but -every form of that value (plain, enum-wrapped, percent-encoded) still 400s -— the Spring `@RequestParam` enum binding appears to have drifted from the -OpenAPI-declared enum. No pyegeria code change can fix a 400 the server -returns for every input. +**Moved here from "Open Egeria Server issues" on the basis of the above.** -**Original workaround (superseded, no longer needed for this method):** use -`find_specification_property("*", ...)` (by-search-string) instead, and -filter client-side on `element["properties"]["identifier"]` (camelCase form -of the type name). +**Update 2026-08-27: regression — `include_only_classified_elements` alone +now also returns ZERO, no `sequencing_order` involved at all.** Found +debugging Egeria Explorer's Questions tab showing no questions on the +left (`egeria-workspaces-fs`'s `perspectives_handler.py`'s `get_questions()`, +which had already dropped `sequencing_order` per this issue's earlier +guidance). Reproduced directly against `qs-view-server`: +`find_glossary_terms(search_string="*", starts_with=True, graph_query_depth=0, +include_only_classified_elements=["Question"], page_size=200)` → **0** hits, +where the **2026-08-26** update immediately below recorded this exact call +(classification filter alone) returning **43** hits one day earlier. Ruled +out a `"Question"`-specific cause: swapped in `include_only_classified_elements= +["Template"]` (a long-established classification, unrelated to anything +created recently) → also **0**. An unfiltered call on the same connection +(`find_glossary_terms(search_string="*", starts_with=True, graph_query_depth=0, +page_size=200)`, no classification filter at all) → **200** hits (page-size +ceiling), so the connection/server itself is healthy and the classification +data is intact (confirmed via direct `get_term_by_guid` on one of the +"missing" terms — its `elementHeader.otherClassifications` correctly carries +`classificationName: "Question"`). So as of this server instance today, +`include_only_classified_elements` appears to return zero regardless of +`sequencing_order` — a strictly worse regression than this issue's original +scope. Workaround applied in `perspectives_handler.py`: drop the server-side +classification filter entirely, fetch unfiltered (`page_size=1000`, the +server's own max), and filter client-side on `otherClassifications`. +Worth re-testing the classification-filter-alone case again on a later +server build, the way this issue's own history already does for the +combined-filter case. -```python -from pyegeria import SpecificationProperties -mgr = SpecificationProperties(view_server="qs-view-server", platform_url="https://localhost:9443", - user_id="erinoverview", user_pwd="secret") -mgr.create_egeria_bearer_token() -types = mgr.get_specification_property_types() -mgr.get_specification_property_by_type(list(types.keys())[0]) -# Now succeeds -- was previously 400ing for every input, confirmed 2026-07-31. -``` +**Update 2026-08-26: re-confirmed again, unchanged.** Re-ran the exact +repro against the current server build (`6.2-SNAPSHOT`, platform +timestamp `2026-08-24T18:27:04Z` — the same build that turned out to have +resolved ISSUE-69, checked in the same pass on the chance this one had +also moved). It hadn't: `include_only_classified_elements=["Question"]` +alone returns 43 hits (count has drifted from the 33 in earlier checks — +unrelated to the bug, just live data), `sequencing_order= +"PROPERTY_ASCENDING"` alone returns 200 (page-size ceiling), and combining +both still returns exactly **0**. Same symptom as every prior check. ---- +**Status:** double-checked again 2026-08-18 (per dwolfson's explicit +request) — core defect persists, and this pass rules out a client-side +cause definitively by capturing the actual outgoing request body via a +traced `_async_make_request`: `include_only_classified_elements=["Question"]` +alone sends a well-formed body (`includeOnlyClassifiedElements: ["Question"]`, +no `sequencingOrder` key) and returns **2 hits** (data has changed since the +original 33 — unrelated to the bug). Adding *only* `sequencing_order= +"PROPERTY_ASCENDING"` to that exact same body (confirmed via the trace — +every other field byte-identical) drops the result to **0**. Also tried +adding `sequencing_property="displayName"` on top — still 0. Since ISSUE-60 +(sequencing order itself) is now fixed server-side, this entry's trigger +condition may have shifted from what it was originally — worth a fresh +root-cause dig rather than assuming the original mechanism still applies +unchanged — but the *symptom* (combining `sequencing_order` with a +classification filter zeroes the result) is unambiguously still present, +confirmed via real request/response, not just method-level return values. -### ISSUE-43: Pagination location for `find_metadata_elements`'s endpoint — audit finding, plausible but not yet confirmed live +**Status:** confirmed bug (Egeria server) — found 2026-07-28 debugging +Egeria Explorer's Perspectives page showing Perspectives but no Questions. +Related to ISSUE-40 below (same broken parameter, different — and more +severe — failure mode: not just wrong order, but zero rows). Consolidated +in from `egeria-workspaces-fs/PYEGERIA_ISSUES.md` 2026-08-05. -**Status:** fixed 2026-08-15 (Pyegeria — `pyegeria/omvs/runtime_manager.py`). -Resolved the "not yet empirically confirmed" `_async_get_server_templates_by_dep_impl_type` -finding via ground-truth comparison rather than waiting for 2+-page demo -data (the blocker noted below, still unavailable): checked -`Egeria-api-runtime-manager.http`'s `getServerTemplatesByDeployedImplementationType` -worked example — a plain `POST .../software-servers/by-deployed-implementation-type` -with **no query string at all**, pagination and the `Template` filter both -carried entirely in the `FilterRequestBody` (`includeOnlyClassifiedElements: -["Template"]`). The sibling, non-templates method in the same file -(`_async_get_servers_by_dep_impl_type`) already builds the URL with no query -string, confirming the correct pattern exists right next to the bug. Removed -the spurious `?startFrom={..}&pageSize={..}&getTemplates=true` suffix -entirely — none of those three query params exist on the real endpoint; -pagination was already being built correctly into the body by the shared -`_async_get_name_request` helper regardless (confirmed via a request-body -spy, before and after: `startFrom`/`pageSize` land in the body either way, -so the URL suffix was purely spurious/redundant, not silently required for -anything to work). +**How to trigger** (`GlossaryManager.find_glossary_terms`, qs-view-server, +33 `GlossaryTerm`s classified `Question`): +```python +# classification filter alone: 33 hits +mgr.find_glossary_terms(search_string="*", starts_with=True, output_format="JSON", + page_size=200, graph_query_depth=0, + include_only_classified_elements=["Question"]) -**Verified live:** the call now succeeds against `qs-view-server` -(`get_server_templates_by_dep_impl_type("cots-application-server", ...)` → -`No elements found`, same empty-but-successful result as before, for the -reason already documented below — no `Template`-classified software servers -of any tested type exist in this demo dataset, so a real 2+-page comparison -still isn't possible here). `pytest tests/ -m unit` passes. +# sequencing_order alone (no classification filter): 200 hits (unrelated terms, page_size ceiling) +mgr.find_glossary_terms(search_string="*", starts_with=True, output_format="JSON", + page_size=200, graph_query_depth=0, + sequencing_order="PROPERTY_ASCENDING") -**Original status:** open (pyegeria, suspected) — found 2026-08-05 while auditing -for the same drift class as ISSUE-34 (see that entry's "design principle" -paragraph — Egeria's pagination convention moved from URL query params to -request-body fields for at least one endpoint a few months ago; worth -checking any method still doing URL-based `startFrom`/`pageSize` -injection). Grepped `pyegeria/omvs/*.py` for the same -`?startFrom={..}&pageSize={..}` URL-construction shape ISSUE-34 had; found -2 more matches besides `metadata_expert.py` (already fixed there) — -`runtime_manager.py` and `valid_metadata.py`. +# BOTH together: 0 hits +mgr.find_glossary_terms(search_string="*", starts_with=True, output_format="JSON", + page_size=200, graph_query_depth=0, + sequencing_order="PROPERTY_ASCENDING", + include_only_classified_elements=["Question"]) +# -> [] (or a "No elements found" string, depending on call shape) +``` +Isolated further: `sequencing_order="PROPERTY_ASCENDING"` is the trigger — +`sequencing_property` alone (no `sequencing_order`) does **not** break it +(still 33 hits). It's specifically `sequencing_order` + a classification +filter. -**`valid_metadata.py`'s `_async_get_valid_metadata_values`: checked, NOT a -bug.** This is a plain `GET` request (`get-valid-metadata-values/{property_name}`) -— there is no request body at all for a GET, so URL query parameters are -the only place `startFrom`/`pageSize` *can* go. Confirmed against the -`.http` ground truth (`Egeria-api-classification-explorer.http`) — also a -bare GET, no body shown. Correctly implemented as-is. +**Expected:** the classification filter's 33 matches, sorted by the given +sequencing property (or, per ISSUE-40, at least returned in server-internal +order — but not silently emptied). -**`runtime_manager.py`'s `_async_get_server_templates_by_dep_impl_type`: -suspicious, not yet confirmed.** Line ~2612 unconditionally appends -`?startFrom={start_from}&pageSize={page_size}&getTemplates=true` to the URL -*regardless* of whether `start_from`/`page_size` are also meaningful via -the `body` path — and the shared helper it delegates to -(`ServerClient._async_get_name_request`) **already** builds a correct -`"startFrom"`/`"pageSize"` pair into the body it constructs when the -caller doesn't supply an explicit `body` (confirmed by reading -`_async_get_name_request`'s source directly, `_server_client.py` ~line -6493). So the URL suffix here is at best redundant with what the body path -already does correctly, and at worst sends conflicting information when a -caller supplies their own explicit `body` without `startFrom`/`pageSize` -set (in that case pagination could only come from the URL, which per -ISSUE-34's finding is exactly the convention that stopped working for a -sibling endpoint). +**Actual:** zero rows, with no error — the query silently looks like +"nothing matches" rather than failing loudly, which is what made this hard +to spot (the egeria-workspaces-fs `/api/questions` endpoint returned +`{"total": 0}` with a 200 status; only comparing against a live count of +Question-classified terms in Egeria surfaced that this was wrong, not just +an empty demo). -**Not yet empirically confirmed**, unlike ISSUE-34's `find_metadata_elements` -finding — this demo dataset doesn't have enough `Template`-classified -servers of the same `deployedImplementationType` to construct a real -2-page pagination test (tried `cots-application-server` — 5 total -`SoftwareServer`s of that type exist, but zero are `Template`-classified; -every filter value tested returned either 0 or 1 `Template`-classified -result, never 2+, so a same-content-vs-distinct-content comparison isn't -possible with this data). **Next step:** either seed enough demo data to -get a real 2+ result test, or read the Java source for this specific -endpoint the way ISSUE-15's `QueryBuilder` investigation did, rather than -guessing from the pattern match alone. +**Impact / workaround:** egeria-workspaces-fs's `perspectives_handler.py`'s +`get_questions()` used exactly this broken combination. Fixed by dropping +`sequencing_order`/`sequencing_property` from the call — the endpoint +already sorts client-side, so the server-side sequencing was redundant +even before this bug was found. No other known callers currently combine +`sequencing_order` with a classification filter, but worth checking +`include_only_classified_elements`/`matchClassifications` callers +generally if new zero-result reports show up elsewhere. --- @@ -7410,545 +7438,649 @@ count. --- -### ISSUE-19 (PY-12): `ReferenceDataManager` has no specification-property or valid-metadata-value methods +### ISSUE-36 (PY-16): `ClassificationExplorer.link_elements_as_peer_duplicates` (and its `_async_*` twin) POST to the wrong URL path — always 404s -**Status:** fixed — re-verified 2026-08-15, found already resolved (never -marked closed here). `ReferenceDataManager`'s class docstring -(`pyegeria/omvs/reference_data.py`) now has a `Note` section stating it -does NOT cover Specification Properties and pointing callers at -`SpecificationProperties`/`MetadataExpert` instead — exactly the -"suggested improvement" below, already applied. Not clear which session -added it since this entry was last touched; recommending a class-selection -decision tree across `ReferenceDataManager`/`SpecificationProperties`/ -`ValidMetadataLists`/`ValidTypeLists`/`MetadataExpert` is still open as a -nice-to-have, not a defect. +**Status: FIXED** — confirmed 2026-07-17 on pyegeria 6.0.16.20. +Consolidated in from `egeria-workspaces-fs/PYEGERIA_ISSUES.md` 2026-08-05. -**Originally filed as:** open (docs/organization), re-verified 2026-07-31 — still true. -`ReferenceDataManager` inherits only from `ServerClient` (it's for -*business* reference data — country codes, currency codes, etc.), not from -`ValidMetadataManager`. `get_valid_metadata_values` happens to work on it -because that method lives on the shared `ServerClient` base, not on -`ValidMetadataManager` — easy to misjudge the class boundary from that -alone. The specification-property methods -(`get_specification_property_types`, `get_specification_property_by_type`, -`get_specification_property_by_name`, `get_specification_property_by_guid`, -`find_specification_property`) only exist on -**`pyegeria.SpecificationProperties`** (also on `ValidMetadataManager` -subclasses `ValidMetadataLists`, `ValidTypeLists`). +`_async_link_elements_as_peer_duplicates` now builds the URL from +`f"{self.classification_command_root}/related-elements/{element_guid}/peer-duplicate/{peer_duplicate_guid}/attach"` +— verified directly against the running `quickstart-pyegeria-web` +container's installed pyegeria source. Originally confirmed as a +client-side bug 2026-07-16, while seeding demo data for +egeria-workspaces-fs's Duplicate Resolution Review pane +(`duplicate_review_handler.py`). -**Fix:** use `pyegeria.SpecificationProperties(...)` for these calls -instead of `ReferenceDataManager`. +**Root cause:** the client posted to +`.../classification-explorer/elements/{elementGUID}/peer-duplicate/{peerDuplicateGUID}/attach` +but the real Spring endpoint +(`ClassificationExplorerResource.linkElementsAsPeerDuplicates`) is mapped +at `.../classification-explorer/related-elements/{elementGUID}/peer-duplicate/{peerDuplicateGUID}/attach` +— `elements` vs `related-elements`. Same root cause likely affected +`unlink_elements_as_peer_duplicates` (detach, same path shape). +Classification calls (`set_known_duplicate_classification`/ +`set_consolidated_duplicate_classification`) and the read path +(`get_relationships`/`get_elements_by_classification`) were unaffected. -**Suggested improvement:** either a class-level docstring note on -`ReferenceDataManager` clarifying the boundary, or a single unified -decision tree for "which OMVS client class do I need" across -`ReferenceDataManager` / `SpecificationProperties` / `ValidMetadataLists` / -`ValidTypeLists` / `MetadataExpert`, since several overlap in purpose. +**Fix:** `pyegeria/omvs/classification_explorer.py`, +`_async_link_elements_as_peer_duplicates` (and its detach twin) now build +the URL from the `related-elements` path. --- -### ISSUE-50: `base_report_formats.py`'s `Collections` FormatSet aliased the real Egeria type `ResultsSet` as `"ResultSet"` (missing the "s") - -**Status:** fixed 2026-08-09, alongside adding pyegeria SDK support for -Egeria PR #9200 (SavedQuery/SmartQuery/ResultsSet). Originally found -2026-08-05 building egeria-workspaces-fs's saved-query prototype (Egeria -Insights `EGERIA_INSIGHTS_QUERY_MODEL.md`, Track A), which needed to -create a real `ResultsSet` collection. +### ISSUE-35 (PY-15): Postgres repository connector ignored `matchCriteria` on `SearchClassifications` — multi-classification search always returned 0 -Root cause: `pyegeria/view/base_report_formats.py`'s `"Collections"` -`FormatSet` listed `aliases=[..., "ResultSet", ...]` — missing the "s". -The real Egeria entity type is spelled **`ResultsSet`** (confirmed live -against `qs-view-server`'s `/api/types` entity catalog); passing -`"ResultSet"` as a `typeName` to `CollectionManager.create_collection` was -rejected outright by the server (`OMAG-COMMON-400-018`). Impact was -narrow but real: any caller rendering a genuine `ResultsSet` element -through `generate_output()`'s alias-based `FormatSet` lookup would miss -the match and silently fall through to a generic/default format instead -of `Collections`. +**Status: FIXED and CLOSED (Egeria server, not pyegeria)** — verified +2026-07-17. Consolidated in from `egeria-workspaces-fs/PYEGERIA_ISSUES.md` +2026-08-05 (both files independently tracked pyegeria/Egeria issues; this +one and ISSUE-36 through ISSUE-41 below were unique to that copy — see this +file's top-of-doc note for the consolidation). -Fix: one-line change, `"ResultSet"` → `"ResultsSet"` in the `aliases` -list (`pyegeria/view/base_report_formats.py`, `"Collections"` FormatSet). +Originally confirmed as a server-side bug 2026-07-15, while building Egeria +Insights (`insights_handler.py`) — not a pyegeria client bug, the client +sent the body faithfully; the Postgres repository connector's SQL +generation dropped `matchCriteria` on the classification-matching path +only. Fixed server-side and re-verified live against `qs-view-server` once +the fixed server was deployed: ---- +``` +ZoneMembership ANY: 150 (unchanged -- single-condition baseline) +Confidentiality ANY: 1 (unchanged -- single-condition baseline) +Both ANY: 150 (was 0 -- now a real, differentiated union) +Both ALL: 0 ("No elements found" -- correct, empty intersection) +Both NONE: 1000 (was 0 -- now a real, differentiated count) +``` -### ISSUE-44: `create_my_todo`/`create_meeting`/`create_review` (my_profile.py) always invented their own `qualifiedName` (random timestamp suffix, wrong casing), silently diverging from what Dr.Egeria's `Create ToDo`/`Create Meeting`/`Create Review` reported having created +`ANY`/`ALL`/`NONE` now produce distinct, semantically correct results +instead of all being an unconditional AND that always returned zero. -**Status:** fixed 2026-08-05 (Pyegeria — `pyegeria/omvs/my_profile.py` + -`md_processing/v2/actor_manager.py`/`project.py`/`feedback.py`). Reported -directly by the user: "the Dr.Egeria command Create ToDo does not appear to -create a todo." +**Root cause:** `QueryBuilder.getSearchClassificationsClause()` in +`open-metadata-implementation/adapters/open-connectors/repository-services-connectors/open-metadata-collection-store-connectors/postgres-repository-connector/src/main/java/org/odpi/openmetadata/adapters/repositoryservices/postgres/repositoryconnector/database/QueryBuilder.java` +(lines 1036–1078) unconditionally `AND`s an `AND (type_name LIKE +'%::%' ...)` clause per classification condition and never reads +`matchClassifications.getMatchCriteria()` at all — confirmed by grepping +the whole file: `MatchCriteria` is read in the *property*-matching path +(`getPropertyComparisonFromPropertyConditions()`, ~line 896) but never in +the classification-matching path. So the generated SQL always required +every named classification to appear on the same classification-table join +simultaneously, regardless of what `matchCriteria` the caller asked for. -**What actually happened:** `Create ToDo` genuinely succeeded — a real -`ToDo` entity was created, correct `displayName`/`description`/`situation`/ -`priority`/`activityStatus`, `SUCCESS` reported with a real GUID. But -`_async_create_my_todo(todo_name, ...)` has no parameter at all for -accepting a caller-supplied qualified name — it always computes its own via -`self.__create_qualified_name__("Todo", todo_name)+f"-{int(time.time())}"`, -ignoring whatever qualified name Dr.Egeria's dispatcher already derived and -displays in its own analysis table. Confirmed live: Dr.Egeria reported -creating `Coco Pharmaceuticals::ToDo::Go-to-Lunch-with-Peter`; the entity -actually stored was `Coco Pharmaceuticals::Todo::Go-to-Lunch-with-Peter-1785955953` -— different casing (`Todo` vs `ToDo`) *and* an unpredictable numeric -suffix. Anyone searching for the element by the name Dr.Egeria reported — -Egeria Explorer, a fresh `find_*` call, a later Dr.Egeria command -referencing it by qualified name in a separate run — finds nothing, which -is exactly why it "does not appear to" have been created even though it -was. +**Regression coverage:** +- `egeria-python/tests/functional-tests/test_metadata_expert.py::test_find_metadata_elements_multi_classification_any_match_criteria` + (pytest, asserts ANY's count >= max of the two single-condition counts) — + passes as of 2026-07-17. +- `egeria-python/pyegeria/http clients/Egeria-PY15-matchClassifications-bug.http` + (PyCharm/IntelliJ HTTP Client collection, same assertions via raw REST + calls). -**Systemic, not isolated:** `_async_create_meeting` and `_async_create_review` -have the identical pattern (confirmed by reading both) — both already wired -as Dr.Egeria commands (`Create Meeting` via `ProjectProcessor`, `Create -Review` via `FeedbackProcessor`), so both had the same latent bug. +--- -**Fix:** added an optional `qualified_name: Optional[str] = None` parameter -to all three async methods and their sync wrappers — used verbatim when -provided (falls back to the existing auto-generation only when the caller -doesn't supply one, so no other caller's behavior changes). Updated all -three Dr.Egeria processor call sites -(`actor_manager.py`'s `Create ToDo` branch, `project.py`'s `Create Meeting` -branch, `feedback.py`'s `Create Review` branch) to pass the `qualified_name` -the dispatcher already computed. +### ISSUE-34: `MetadataExpert.find_metadata_elements` ignored `start_from`/`page_size` — traced through THREE fix attempts before landing on the actual root cause (a moved Egeria convention), and a design change to prevent the whole bug class recurring -**Verified live:** re-ran the user's exact `actor_actions.md` — `Create -ToDo` still `SUCCESS`, and the real stored element's `qualifiedName` now -matches Dr.Egeria's reported qualified name exactly (`Coco -Pharmaceuticals::ToDo::Go-to-Lunch-with-Peter`, confirmed via direct -`get_element_by_guid` comparison). Cleaned up both the original mismatched -throwaway and the corrected one afterward. Full `tests/micro-tests` suite -clean (same one pre-existing unrelated failure). +**Status: FIXED** (final fix 2026-08-05, `pyegeria/omvs/metadata_expert.py` + +`pyegeria/view/overview_metrics.py` + tests). Worth reading in full even +though it's closed — the misdiagnosis in the middle is as instructive as +the fix. -**Follow-up 2026-08-05 — the deeper bug: assigned ToDos never showed up in -`get_my_to_dos()` at all.** User reported: "I ran test_get_to_dos in -test_my_profile.py as erinoverview and don't see any Todos?" Root-caused in -three layers, all now fixed: +**Attempt 1 (dropped parameter).** `_async_find_metadata_elements` +originally built its request with a bare URL and the caller's `body` +passed straight through unmodified — `start_from`/`page_size`/ +`graph_query_depth` were silently dropped, never merged into `body`, never +appended to `url`. Root-caused against the `.http` ground truth +(`Egeria-api-metadata-expert.http`'s worked examples): `graphQueryDepth` +**is** a body field there; `startFrom`/`pageSize` are not shown in the body +at all, so the fix assumed they were URL query parameters (matching the +older, pre-migration Egeria convention this same file's `.http` examples +still document) — appended `?startFrom={..}&pageSize={..}` to the URL and +merged `graphQueryDepth` into `body` when not already present. Verified via +runtime capture that the values reached the wire correctly. -1. **`_async_get_my_assigned_actions` had a parameter-name bug** (typo - `metadtata_element_subtypes` plus wrong field names `metadata_element_type`/ - `metadata_element_subtypes` instead of the real `GetRequestBody` fields - `metadata_element_type_name`/`metadata_element_subtype_names`) — silently - dropped by pydantic's `extra='ignore'` on every call. Fixing the names - uncovered that this endpoint's own worked example - (`Egeria-api-my-profile.http`) never sends these fields at all, and doing - so live produces `OMAG-COMMON-400-019 ... type name ToDo ... is not a - sub-type of UserIdentity` — the endpoint evidently applies them against - the assigned actor's type, not the action's. Fix: stop forwarding these - two fields into the request body for this endpoint (kept as inert public - parameters for call-site compatibility with `get_my_to_dos`/ - `get_my_sponsored_actions`). -2. **The actual root cause: `_async_create_my_todo`/`_async_create_meeting`/ - `_async_create_review` put `originatorGUID`/`assignToActorGUID` *inside* - `properties` (i.e. as fields of `ToDoProperties`/`MeetingProperties`/ - `ReviewProperties`)**, instead of at the top level of `ActionRequestBody` - as siblings of `properties` — confirmed against - `AssetMaker._async_create_action`'s own docstring/worked example, which - shows `originatorGUID`/`actionSponsorGUID`/`assignToActorGUID` as - top-level `ActionRequestBody` fields. Since `ToDoProperties` etc. have no - such fields, they were silently dropped there too — the create call always - succeeded, but no `ActionRequester`/`AssignmentScope` relationship was - ever created, so the new ToDo was structurally invisible to any - "assigned to me" query, no matter how the query itself was fixed. This - was misdiagnosed mid-investigation as a possible Egeria-server-side gap - (a raw REST call reproducing the same nested-properties mistake showed - the identical missing-relationship symptom) before the nesting bug was - spotted — flagged by the user, not found independently. Fix: moved both - fields to the top level of the body in all three methods. -3. Also added `qualified_name: Optional[str] = None` support to - `_async_log_my_activity`/`_async_journal_my_activity`/`_async_blog_my_activity` - (the milder, deterministic-but-still-caller-ignored variant noted below in - the original write-up), and added hand-maintained `ToDo-DrE`/`Meeting-DrE`/ - `Review-DrE` `FormatSet`s to `base_report_specs` (same fallback pattern as - the existing `Journal-Entry-DrE`), fixing the `Report spec - 'ToDo-DrE-Basic' not found. Falling back to default.` warning noted below. +**Attempt 2 (misdiagnosed as an Egeria-server bug).** Re-verified live +against `qs-view-server` and found real pagination still didn't work — +`start_from=0`/`start_from=5` (both `page_size=5`) returned the identical +full ~1,837-element population both times, 100% GUID overlap. Since the +*client* was confirmed (via request-spy) to be sending the spec-correct +URL, this was logged as an open **Egeria-server-side** gap — "the server +ignores these query parameters regardless of client version." **This +conclusion was wrong.** -**Verified live (2026-08-05):** created a fresh ToDo via `create_my_todo()`; -`get_all_related_elements(guid)` now shows a real `ActionRequester` -relationship to the `Person` entity; `get_my_to_dos(output_format='JSON')` -returns the new ToDo's GUID. `select_report_spec('ToDo-DrE-Basic', 'MD')` -correctly falls through to the new `ToDo-DrE` spec. Cleaned up all -throwaway test elements afterward. +**Attempt 3 (the actual root cause, found by the user).** Egeria's +pagination convention for this endpoint changed a few months before this +investigation — from URL query parameters to **request-body fields**. The +`.http` contract files pyegeria's fix was root-caused against were +themselves stale for this specific endpoint. Confirmed empirically: +```python +body1 = {..., "startFrom": 0, "pageSize": 5} +body2 = {..., "startFrom": 5, "pageSize": 5} +# len(r1) == len(r2) == 5 (page_size genuinely respected, not the full 1837) +# set(guids in r1) & set(guids in r2) == set() -- two real, distinct pages +``` +There is no Egeria-server-side bug here at all — pagination works +correctly and always has; pyegeria was sending the right values to the +wrong location, and "the server ignores it" was the visible symptom of +that, not evidence the server itself was broken. -**Second follow-up 2026-08-05 — checked Meeting/Review through to reporting -too, not just ToDo:** +**Final fix — a design change, not just a relocated injection.** Rather +than moving the URL-based injection into a body-based injection (which +would just be the same "pyegeria guesses where pagination goes for this +endpoint" model, now proven to go stale at least once already), +`start_from`/`page_size` were **removed from the method signature +entirely** — same treatment `graph_query_depth` had already gotten a few +hours earlier in this same investigation, for the same underlying reason +(pushback from the user: "I think that they should have full control over +the body and not have me inject anything in it"). For endpoints where the +caller already constructs the full `body` themselves, a second parameter +channel for the same information is exactly the shape that goes stale — +this is the second time in one investigation (dropped parameter, then +wrong location) that pyegeria's own tracking of "where does this value +belong" broke a caller. The caller already has to know Egeria's current +contract to build a correct body; they're better positioned to keep +`startFrom`/`pageSize` current for a given endpoint than pyegeria's +hardcoded assumption is. `body` is now sent to `_async_make_request` +completely unmodified, in every code path, for both the async and sync +forms of this method. -4. Verified live that `create_meeting`/`create_review` also now produce real - `ActionRequester`/`AssignmentScope` relationships after the fix, and both - show up via `get_my_assigned_actions()`. -5. **Found and fixed a separate, `Meeting`-specific bug**: - `ProjectProcessor.fetch_element()` (`md_processing/v2/project.py`) - unconditionally calls `_async_get_project_by_guid` — but `Meeting` is a - Person Action Base type routed through `my_profile.create_meeting`, not a - `Project` at all (see the `Meeting` branch in `apply_changes()`). Calling - the Project-typed getter on a Meeting GUID 404s server-side - (`OMAG-REPOSITORY-HANDLER-404-001 ... retrieved an object ... of type - Meeting rather than type Project`), which `fetch_element` silently - swallowed and turned into "Could not fetch element" — so - `render_result_markdown` never rendered a Meeting's report at all, always - falling back to `raw_block`. `Review` was unaffected (`FeedbackProcessor` - doesn't override `fetch_element`, so it already used the generic - `ClassificationExplorer`-based fetch in the base class). Fix: special-case - `Meeting` in `ProjectProcessor.fetch_element()` to use the inherited - base-class fetch instead. Verified live end-to-end (`fetch_element` + - `render_result_markdown`) — a Meeting now renders its full `Meeting-DrE` - report with no warnings. -6. Confirmed `AssetMaker._async_create_action`'s own worked examples in - `Egeria-api-asset-maker.http` were already correct (top-level - `originatorGUID`/`actionSponsorGUID`/`assignToActorGUID`) — the bug was - purely in `my_profile.py` not following its own ground truth, not a gap in - the `.http` file. Confirmed no other caller of `_async_create_action` - exists, so the bug's blast radius was exactly the three methods fixed - above. -7. Added a `KNOWN ISSUE` comment directly above the `getMyAssignedActions` - worked example in `Egeria-api-my-profile.http`, documenting the confirmed - `metadataElementTypeName`/`metadataElementSubtypeNames` 400 so it isn't - reintroduced by a future edit that "helpfully" adds those fields back - without re-checking live. -8. **Pre-existing broken ToDos found and deleted** (user's explicit - decision): 6 `ToDo` entities existed from before this fix (`Go to Lunch - with Peter` ×2, `Have a burger`/`Burger` ×2, `Curation Smoke ToDo`, plus - one test-run leftover `do-my-backup`) — all created via the pre-fix nested - -properties bug, so none had an `ActionRequester`/`AssignmentScope` - relationship and none would ever appear in `get_my_to_dos()`. There is no - retroactive repair (the relationship was simply never created); user chose - to delete all 6 rather than recreate them. Also note: `do-my-backup` - reveals `tests/functional-tests/test_my_profile.py::test_create_my_todo` - has no teardown/cleanup — it leaves a live ToDo behind on every run. +**Design principle adopted from this investigation** (see also ISSUE-43, +an audit for the same drift shape elsewhere): for any method where the +caller is required to supply the complete request body, do not *also* +accept overlapping parameters (`start_from`, `page_size`, +`graph_query_depth`, or similar) that pyegeria tries to inject somewhere — +document where the field belongs in the body and let the caller put it +there. This does **not** apply to convenience methods that build their own +body internally from named kwargs (most of the OMVS-specific `find_*`/ +`get_*` wrappers) — there, pyegeria owns body construction either way, so +there's no second channel to remove. ---- +**Callers updated to the new signature:** +`pyegeria/view/overview_metrics.py`'s `_find()` (production code — now +puts `"startFrom": 0, "pageSize": page_size` directly in the body dict +alongside the already-updated `"graphQueryDepth": 0`) and 5 call sites +across `tests/functional-tests/test_metadata_expert.py` / +`test_metadata_explorer.py` (previously passed `page_size=`/`start_from=` +as kwargs; now set `"pageSize"`/`"startFrom"` in the body dict itself). A +stale `PY-15` cross-reference in `test_metadata_expert.py` (predating this +file's 2026-08-05 consolidation) was also corrected to `ISSUE-35 (PY-15)` +while in there. -### ISSUE-1 (PY-1): `DataDesigner.find_data_value_specifications` called non-existent `_async_post` +**Verified:** live against `qs-view-server`, two real distinct 5-element +pages via the *public* `find_metadata_elements(body)` call (no separate +pagination parameters at all) — confirms the fix works through the whole +call chain, not just the internal request construction. Full +`tests/micro-tests` suite: 228 passed, same one pre-existing unrelated +failure as throughout this session, no regressions. -**Status:** fixed, re-verified 2026-07-31 — `_async_post` no longer exists -anywhere in the codebase (`grep -rn "_async_post\b" pyegeria/` returns -nothing); `find_data_value_specifications` now routes through the shared -`_async_find_request` helper. +**Follow-up cleanup, same day:** `output_format`, `for_lineage`, +`for_duplicate_processing`, and `**kwargs` were also removed from both +`_async_find_metadata_elements` and `find_metadata_elements` — none of the +four were ever referenced anywhere in either method's body (confirmed by +reading the full implementation), so declaring them implied controls that +never existed, the same "extraneous parameter" shape as `start_from`/ +`page_size`/`graph_query_depth` above. Final signature for both: `(self, +body: dict, timeout: int = default_timeout)` — nothing else. Verified: live +call still succeeds (a `pageSize: 1` body field correctly limits to 1 +result), full `tests/micro-tests` suite still clean. -```python -from pyegeria import DataDesigner -mgr = DataDesigner(view_server="qs-view-server", platform_url="https://localhost:9443", - user_id="erinoverview", user_pwd="secret") -mgr.create_egeria_bearer_token() -mgr.find_data_value_specifications(search_string="*") -``` +**This raises the severity of the note below** — removing `**kwargs` +specifically means a caller still passing `start_from=`/`page_size=`/ +`graph_query_depth=` as keyword arguments no longer gets silently ignored, +it now gets a hard `TypeError: find_metadata_elements() got an unexpected +keyword argument`. Worth knowing before upgrading pyegeria in any consumer +that hasn't been updated yet. + +**Not carried over from pyegeria's fix:** `egeria-workspaces-fs/ +insights_handler.py`'s `search_elements()` still calls `find_metadata_elements` +with `start_from=`/`page_size=`/`graph_query_depth=` as keyword arguments — +as of the follow-up cleanup above, this will now **raise `TypeError` on +every call** rather than silently no-op (see previous paragraph; this is a +stronger warning than when this entry was first written). That repo's own +call site needs a follow-up update to put `startFrom`/`pageSize` in its own +`find_body` dict to get real, efficient pagination — flagged, not fixed +here (different repo), but now genuinely blocking rather than just a +missed optimization. --- -### ISSUE-2 (PY-2): `get_data_value_specifications_by_name("*")` rejected the wildcard +### ISSUE-33: `Create Journal Activity`/`Create Log Activity` used the `Notification` placeholder typeName; renamed to match real `JournalEntry`/`ActivityEntry` types, and the orphaned dead `Create Journal Entry` registration from ISSUE-32 is now a real, working command -**Status:** fixed, re-verified 2026-07-31 — the shared -`_async_get_name_request` helper (`_server_client.py`) still correctly maps -`filter_string = None if filter_string == "*" else filter_string`, aligned -with every sibling helper's "any" sentinel handling. +**Status:** fixed 2026-08-05 (Pyegeria + Dr.Egeria compact spec, same day as +ISSUE-32, follow-up in the same session). User question: since `JournalEntry` +is a real `Notification` subtype, why is Dr.Egeria's command called `Journal +Activity` instead — could it be renamed for consistency? -Root cause was the `"*"` convenience sentinel being sent to Egeria as the -literal string `".*"` instead of `None` — Egeria's by-name `filter` field is -not a regex, so `".*"` either matched nothing or matched literally. +**Investigation, grounded in the actual Egeria type system and docs (not +guessed):** +- Live type query confirmed `JournalEntry`, `ActivityEntry`, `BlogEntry` are + all real, distinct `Notification` subtypes (siblings of `Note` from + ISSUE-32) — `superType: Notification` for all three, live on + `qs-view-server`. +- Fetched `egeria-project.org/concepts/notification/`, + `/types/1/0160-Notes/`, and `/types/1/0135-Actions-For-People/` before + proceeding (per explicit user instruction) — confirmed `Note` and + `JournalEntry`/`ActivityEntry`/`BlogEntry` are siblings under the *same* + `NoteLog` + `AttachedNoteLogEntry` mechanism (`AttachedNoteLogEntry` + connects a NoteLog to "an Action (**typically a Notification**)" — not + restricted to one subtype), differing only in target (any element, for + `Note`; the calling user's own profile, for the other three) and + visibility (`JournalEntry` private; `ActivityEntry`/`BlogEntry` public). +- This also surfaced that `Create Journal Activity`/`Create Log Activity` + (bundle `My Journal Base`, mapping to `my_profile.py`'s + `journal_my_activity`/`log_my_activity`) were **not registered in the + dispatcher at all** — a bigger gap than a naming mismatch. `Create Blog + Entry` already had the correct `OM_TYPE: "BlogEntry"` but was equally + unregistered. +- Confirmed the pre-existing `reg("Create Journal Entry", FeedbackProcessor)` + (ISSUE-32's finding) was genuinely dead/orphaned — no command by that name + existed in the compact spec, and its code branch referenced attributes + that don't exist anywhere in `commands_feedback_compact.json`. Renaming + `Journal Activity` → `Journal Entry` would otherwise collide with that + dead registration, so it was removed first. -```python -from pyegeria import DataDesigner -mgr = DataDesigner(view_server="qs-view-server", platform_url="https://localhost:9443", - user_id="erinoverview", user_pwd="secret") -mgr.create_egeria_bearer_token() -mgr.get_data_value_specifications_by_name("*") -``` +**Fixed, via the Dr.Egeria Spec Editor's REST API** (compact spec — see +ISSUE-32's entry for why raw JSON edits aren't used): +- Deleted `Create Journal Activity`, created `Create Journal Entry` + (`OM_TYPE: "JournalEntry"`, `alternate_names: ["Journal Activity"]`). +- Deleted `Create Log Activity`, created `Create Activity Entry` + (`OM_TYPE: "ActivityEntry"`, `alternate_names: ["Log Activity"]`) — same + consistency rationale as the Journal rename, so it wasn't left half-fixed. +- `refresh_specs --merge-reports` regenerated templates/help/report-specs; + unit tests pass. + +**Fixed, code (`md_processing/dr_egeria.py`, `md_processing/v2/feedback.py`):** +- Removed the dead `reg("Create Journal Entry", FeedbackProcessor)` + + its broken code branch. +- Registered `Create Journal Entry`, `Create Activity Entry`, `Create Blog + Entry` → `FeedbackProcessor`. +- Implemented one shared branch (`object_type in ("Journal Entry", + "Activity Entry", "Blog Entry")`) dispatching to + `my_profile._async_journal_my_activity`/`_async_log_my_activity`/ + `_async_blog_my_activity` respectively. + +**Verified live, all three, end-to-end** (not just `--validate`): processed +a throwaway smoke-test doc through the real `dr_egeria --process` pipeline — +all three `SUCCESS` with real GUIDs — then fetched each back and confirmed +`elementHeader.type.typeName` was exactly `JournalEntry`/`ActivityEntry`/ +`BlogEntry` respectively. Cleaned up all three afterward. + +**Not touched:** `my_profile.py`'s journal/log/blog-my-activity NoteLogs +(the containers these entries were created in) — unlike `Create Note`'s +throwaway per-note `NoteLog`, these are the user's own persistent +activity-stream containers and weren't deleted during cleanup, only the +three created entries were. + +**Downstream propagation completed same session:** templates regenerated +and mirrored into `egeria-workspaces-fs`/`egeria-advisor` (old +`Create_Journal_Activity.md`/`Create_Log_Activity.md` deleted, new +`Create_Journal_Entry.md`/`Create_Activity_Entry.md` added, confirmed via +`git status` in both downstream repos); `docs/dr_egeria_manual.md`'s +Feedback family description updated; `egeria-workspaces-fs/portal-docs/ +tools/dr-egeria/templates-basic.md`'s Journal Entry example corrected to +use real attribute names (`Display Name`/`Description`, not the +placeholder `Subject`/`Entry` it previously showed) and noted the sibling +commands; dr-egeria Glossary (help Terms) regenerated and reprocessed +against the live server. --- -### ISSUE-3 (PY-5): `get_notes_for_note_log` broken on pyegeria 6.0.14.4/.5 +### ISSUE-32: `create_note`/`update_note` used `"Notification"` as a placeholder `typeName`/`class` — replaced with the real `Note` type now that it's shipped (Egeria PR #9191) -**Status:** fixed, re-verified 2026-07-31 (live). The default -`metadata_element_type_name="Action"` in `_async_get_notes_for_note_log` -(`_server_client.py`) looks superficially like it could still trigger the -originally-reported 404, but a live call against a real note log guid -(`901aab3a-e6f9-4bd1-8899-f013150fbcd4`) returned the correct note list (1 -note, no error) — confirmed by direct test, not just by reading the code. +**Status:** fixed 2026-08-05 (Pyegeria — `pyegeria/core/_server_client.py`, +`md_processing/v2/feedback.py`), found/actioned same day the user redeployed +Egeria with the new type. -Originally on 6.0.14.4/.5: the default caused -`OMAG-REPOSITORY-HANDLER-404-001` ("guid is of type NoteLog rather than -Action"); passing `metadata_element_type_name="NoteLog"` returned the log -itself, not its notes. Fixed in 6.0.14.6+. +**Background:** Dr.Egeria's `Create Note` command was already a known, +explicitly-flagged placeholder — its compact-spec entry +(`md_processing/data/compact_commands/commands_feedback_compact.json`) reads +*"PARSE-ONLY STUB: no NoteProperties/NoteLogEntryProperties type exists +anywhere in pyegeria yet — a new Egeria type is in progress. ... do NOT wire +a processor for this command until the type ships and is confirmed."* with +`OM_TYPE: "Notification"` as the placeholder. Separately, +`ServerClient.create_note`/`update_note` (the note-log-entry creation/update +methods used by `_async_add_journal_entry` and directly by tests) built +their default request bodies with `"class": "NotificationProperties", +"typeName": "Notification"` for the same reason — no dedicated type existed. -**Regression risk carried forward:** a `pip install pyegeria --upgrade` -that lands on exactly 6.0.14.4/.5 (a yanked/reordered PyPI point release) -would resurface this. Floor-pin `pyegeria>=6.0.14.6` in any deployment that -installs from PyPI rather than an editable local checkout. Also: do NOT -pass `metadata_element_type_name="NoteLog"` — it returns 0 notes even on -fixed versions. +**What changed:** confirmed via the actual Egeria source (not the stale +local `egeria-v6/egeria` checkout, which was 13 commits behind — checked +`git fetch` + `origin/main` directly) that commit `8d3b8d9a99` ("Add Note +type - a subtype of Notification", merged via PR #9191) added a real +`Note` `EntityDef` (typeGUID `391f73ff-c184-41b6-8620-310b4284039f`, +superType `Notification`, model area 0160 "Notes" — the same area as +`NoteLog`/`AttachedNoteLogEntry`, confirming `Note` is exactly the proper +type for what `create_note` already builds via the `AttachedNoteLogEntry` +relationship, not a separate new concept). `NoteProperties extends +NotificationProperties` with **zero new fields** — purely a type-identity +fix, no new attributes needed anywhere. -```python -from pyegeria import EgeriaTech -client = EgeriaTech(view_server="qs-view-server", platform_url="https://localhost:9443", - user_id="erinoverview", user_pwd="secret") -client.create_egeria_bearer_token() -client.get_notes_for_note_log(note_log_guid="") -``` +**Verified live** (user redeployed Egeria with the new type same day): +- `ValidMetadataManager.get_typedef_by_name("Note")` confirms the type is + registered on `qs-view-server` (`beanClassName: NoteProperties`, correct + GUID). +- Created a real `Note`-typed element end-to-end via the fixed + `create_note` default body (no manual override needed) and fetched it + back — `elementHeader.type.typeName == "Note"`, confirmed. +- Cleaned up all throwaway test elements afterward + (`MetadataExpert.delete_metadata_element`, see ISSUE-31 for a bug hit + along the way). + +**Fixed:** +- `_server_client.py`: `_async_create_note`/`create_note` and + `_async_update_note`/`update_note` — default body's `"class"`/`"typeName"` + changed from `"NotificationProperties"`/`"Notification"` to + `"NoteProperties"`/`"Note"` (6 docstring sample-body occurrences too, for + consistency); the dead/vestigial `prop` type-list arguments passed to + `_async_create_element_body_request`/`_async_update_element_body_request` + updated to match (`['NoteProperties']`/`['Note']`) even though currently + unused by the validation code, to avoid leaving stale documentation of + intent. +- `md_processing/v2/feedback.py`: `FeedbackProcessor`'s `"Note" in + object_type` branch — `set_element_prop_body("Notification", ...)` → + `set_element_prop_body("Note", ...)` (this one has real effect — produces + `"class": "NoteProperties", "typeName": "Note"`); `set_update_body(...)`'s + first argument also updated for consistency, though that helper doesn't + actually use it (dead parameter, confirmed by reading the function body). + +**Update 2026-08-05 — Dr.Egeria's `Create Note` command is now fully wired, +not just unblocked at the SDK level.** The Tinderbox-only bottleneck this +entry originally described no longer applies — as of the same day, compact- +spec edits go through the Dr.Egeria Spec Editor's local REST API instead +(`commands/tech/spec_editor.py`, `http://localhost:8420`; see `CLAUDE.md`'s +"Dr.Egeria Spec Editor" section and the `dr-egeria-command-sync` skill). +Using that API: `Create Note`'s `OM_TYPE` changed `"Notification"` → +`"Note"`, its stub description replaced with a real one, and +`"Commented On Element"` (the same reference attribute `Create Comment` +already uses, conveniently pre-aliased `"Associated Element"`) added to its +`custom_attributes` — the spec previously had no way to say what element a +note was about at all. `refresh_specs --merge-reports` regenerated +templates/help/report-specs; unit tests pass. Registered +`FeedbackProcessor` for `Create Note` in `setup_dispatcher()` and +implemented its `Create` branch (previously a stub with no code at all): it +now creates a dedicated `NoteLog` for the target element +(`_async_create_note_log`) and the `Note` entry on it +(`_async_create_note`, using the fixed `Note`/`NoteProperties` typing from +above). **Verified live end-to-end**, not just `--validate`: processed a +throwaway `dr_test_create_note_smoketest.md`, confirmed +`elementHeader.type.typeName == "Note"` on the created element via direct +fetch, then cleaned up both the `Note` and its `NoteLog` +(`MetadataExpert.delete_metadata_element`, see ISSUE-31 for a bug hit +along the way). Note: each `Create Note` call currently creates its own +private `NoteLog` rather than reusing an existing one on repeated notes +against the same element — not wrong (a `NoteLog` can hold many notes, +this just doesn't dedupe), flagged as a possible future improvement, not +attempted here. + +**Still not fixed:** +- `update_note`'s underlying REST operation 404s on this server regardless + of the type fix — see ISSUE-30, unrelated to this change, blocks live + verification of the update path specifically (create + fetch are + confirmed working). `Create Note`'s compact spec has no companion + `Update Note` command at all today (only `Create` was ever defined), so + this doesn't block Dr.Egeria usage — only direct SDK callers of + `update_note`/`_async_update_note`. +- **Fixed 2026-08-05, same day, follow-up session:** the orphaned + `"Create Journal Entry"` registration flagged above is resolved — see + ISSUE-33. --- -### ISSUE-4 (PY-7/8/11): `as_of_time` missing or silently dropped on several find methods +### ISSUE-31: `MetadataExpert.delete_metadata_element` crashes with `AttributeError: 'NoneType' object has no attribute 'model_dump'` when called with no explicit body -**Status:** fixed, re-verified 2026-07-31 — `find_information_supply_chains`, -`find_governance_definitions`, `find_note_logs`, `find_collections`, -`find_data_structures`, and `get_technology_type_elements` all route -through shared helpers (`_async_find_request` / `_async_get_name_request`) -that take `as_of_time` as an explicit named parameter or splat `**kwargs` -into the validated request body. +**Status:** fixed 2026-08-15 (Pyegeria — `pyegeria/core/_server_client.py`). +`validate_open_metadata_delete_request` now builds a default +`{"class": "OpenMetadataDeleteRequestBody"}` when `body` is `None`, +matching `validate_delete_element_request`'s existing pattern, instead of +returning `None` straight into `.model_dump()`. Verified live: +`client.metadata_expert.delete_metadata_element()` with no body now +raises a clean `PyegeriaNotFoundException` for a nonexistent GUID instead +of crashing with `AttributeError`. Fixes the wider blast radius too, since +every other caller of the shared `_async_open_metadata_delete_body_request` +helper goes through this same validator. -`get_valid_metadata_values` was reclassified, not a pyegeria fix — see -ISSUE-11 (Egeria Server section): the underlying REST endpoint is a plain -GET that never accepted `asOfTime` in the first place. +**Original status:** open, found 2026-08-04/05 while cleaning up throwaway test +elements created during the ISSUE-32 (`Note` type) live verification. -Originally (surfaced during a "time travel" audit): all of the following -accepted `as_of_time` either not at all, or silently swallowed it because -the method built its request body from explicit named parameters instead -of forwarding kwargs — `find_information_supply_chains`, -`find_governance_definitions`, `find_note_logs`, `find_collections`, -`find_data_structures`, `get_technology_type_elements`. +**Layer:** Pyegeria (`pyegeria/core/_server_client.py`). + +**What:** `_async_open_metadata_delete_body_request(url, body=None)` calls +`self.validate_open_metadata_delete_request(body)` and immediately calls +`.model_dump(...)` on the result — but `validate_open_metadata_delete_request` +returns `None` (not a default `OpenMetadataDeleteRequestBody()` instance) +when `body` is `None`, so the very next line always crashes for the common +"just delete it, no special options" case: ```python from pyegeria import EgeriaTech -client = EgeriaTech(view_server="qs-view-server", platform_url="https://localhost:9443", - user_id="erinoverview", user_pwd="secret") -client.create_egeria_bearer_token() -r1 = client.find_note_logs("*", graph_query_depth=0) -r2 = client.find_note_logs("*", graph_query_depth=0, as_of_time="2020-01-01T00:00:00Z") -# Fixed: r2 now genuinely reflects historical state instead of being identical to r1. +client = EgeriaTech("qs-view-server", "https://localhost:9443", user_id="erinoverview") +client.create_egeria_bearer_token("erinoverview", "secret") +client.metadata_expert.delete_metadata_element("") +# AttributeError: 'NoneType' object has no attribute 'model_dump' ``` +**Workaround (used during this investigation):** always pass an explicit +`body={"class": "OpenMetadataDeleteRequestBody"}` — works correctly once a +non-`None` body is supplied. + +**Candidate fix:** `validate_open_metadata_delete_request` should construct +and return a default `OpenMetadataDeleteRequestBody()` when `body is None`, +matching the pattern every other `validate_*_request` helper in this file +uses (they all handle the "no body given" case by building sensible +defaults, not returning `None`). Not attempted here — found in passing, +outside the scope of the `Note` type work that surfaced it. + +**Confirmed wider blast radius (2026-08-09):** this isn't specific to +`delete_metadata_element` — `_async_open_metadata_delete_body_request` is a +shared base-class helper, so every caller with an optional/default-`None` +body hits the same crash. Reproduced live via the new +`CollectionManager._async_detach_saved_query_from_results_set` (added for +Egeria PR #9200's SmartQuery relationship, which itself calls +`_async_open_metadata_delete_body_request` the same way +`MetadataExpert._async_delete_related_elements` does) — worked around +locally in that method the same way (`body or {"class": +"OpenMetadataDeleteRequestBody"}`), rather than waiting on this issue. +Any other no-args delete-style call sharing this helper (grep +`_async_open_metadata_delete_body_request` for the full caller list) is +suspect until the base helper itself is fixed. + --- -### ISSUE-6 (GAP-1): `PyegeriaAPIException` was a catch-all; typed subclasses existed but weren't raised +### ISSUE-30: `updateNote` REST operation returns 404 on a live server — misdiagnosed as an Egeria server gap for three separate re-checks; actually a stale URL in both pyegeria and the `.http` ground truth it was checked against -**Status:** fixed 2026-07-31. `_async_make_request` in -`_base_server_client.py` now branches on `related_http_code` and raises -`PyegeriaNotFoundException` (404) / `PyegeriaUnauthorizedException` -(401/403) instead of the generic `PyegeriaAPIException`. Both exception -classes re-parented from `PyegeriaException` to subclass -`PyegeriaAPIException` (with `__init__` calling `PyegeriaException.__init__` -directly, bypassing `PyegeriaAPIException.__init__`'s `response.json()` -call, since these two are now also raised for bare HTTP error responses -with no JSON body — see ISSUE-7) so all 63 existing -`except PyegeriaAPIException` handlers found across the codebase keep -working unchanged. +**Layer:** ~~Egeria Server~~ **Pyegeria** — corrected. `_server_client.py`'s +`_async_update_note` built `{command_root}feedback-manager/notes/{note_guid}`. +Every prior re-check (2026-08-05, 2026-08-15, 2026-08-18) cross-checked this +exact URL against `Egeria-api-feedback-manager.http`'s `@name updateNote` +worked example and found it matched byte-for-byte — which is why this got +classified as an Egeria server bug (unregistered/unshipped endpoint) rather +than a pyegeria bug for three separate passes. The `.http` reference file +itself was stale/wrong at the time, so "matches ground truth" gave false +confidence. -**Regression caught and fixed post-merge:** bypassing -`PyegeriaAPIException.__init__` also skipped the line that sets -`self.related_http_code`, which existing code relies on -(`except PyegeriaAPIException as e: if e.related_http_code == 404`). Fixed -by setting it explicitly in both subclasses' `__init__`. +**Status:** fixed 2026-08-18 (Pyegeria — `pyegeria/core/_server_client.py`, +`tests/micro-tests/test_update_note_url.py`). Fixed by dwolfson after +refreshing the local `Egeria-api-feedback-manager.http` copy and noticing +its `updateNote` worked example now shows a different URL: +`feedback-manager/assets/{noteGUID}/update` (the same `assets/{guid}/update` +shape the feedback-manager service uses elsewhere), not +`feedback-manager/notes/{noteGUID}`. `_async_update_note`'s URL construction +corrected to match. -```python -from pyegeria import EgeriaTech -from pyegeria.core._exceptions import PyegeriaNotFoundException -client = EgeriaTech(view_server="qs-view-server", platform_url="https://localhost:9443", - user_id="erinoverview", user_pwd="secret") -client.create_egeria_bearer_token() -try: - client.get_digital_product_by_guid("00000000-0000-0000-0000-000000000000") -except PyegeriaNotFoundException as e: - print(e.related_http_code) # 404 -``` +**Verified live** against `qs-view-server`: created a throwaway +`DataStructure` → `NoteLog` → `Note` (as in every prior re-check), called +`update_note(note_guid, display_name=..., description=...)` — succeeded +(previously 404'd every time). Went further than a bare success check: +fetched the note log's notes back via `get_notes_for_note_log` and +confirmed both `displayName` and `description` actually persisted the new +values, not just a 200 with no real effect. 2 new unit tests +(`test_update_note_url.py`, mocked `_async_make_request`, no live server) +lock in the corrected URL and body shape; full `pytest tests/micro-tests/` +passes. + +**Lesson for future re-verifications:** "the client-side URL matches the +`.http` ground truth exactly" is only as trustworthy as that ground-truth +file itself — this file can drift out of date the same way pyegeria's own +code can, and when both sides of a comparison share the same stale source, +byte-for-byte agreement stops being meaningful evidence. Worth being more +skeptical of a "confirmed matches ground truth, must be server-side" verdict +when the `.http` file hasn't been independently re-verified against a +current server/Swagger spec recently. --- -### ISSUE-7 (GAP-6): expired/invalid bearer token had no specific exception and no auto-renew +### ISSUE-28: `get_specification_property_by_guid` raised a bare `NameError: name 'validate_guid' is not defined` — cannot be called at all -**Status:** fixed 2026-07-31. Connects to the (now closed) `BACKLOG.md` -issue "Bearer token expires mid-run on long `dr_egeria --process` batches." +**Re-verified 2026-08-15, no regression.** Bogus GUID now correctly raises +`PyegeriaNotFoundException` (not `NameError`); a real round trip (find a +specification property, fetch it by the returned GUID) succeeds and +returns the correct `identifier` (`supportedAnalysisStep`). Also fixed a +stale "currently broken outright" note pointing at this issue in the +Quick-reference table further down this file. -Confirmed by direct repro against `qs-view-server`: a bad/expired bearer -token returns a bare HTTP 401 with an **empty response body** — no JSON, -nothing to distinguish "expired" from "wrong credentials" from -"insufficient permission." Since there's no way to detect "expired" -specifically from the response, `_async_make_request` now treats any 401 -as a candidate for "the token might be stale": if the client holds stored -credentials, it calls the already-existing -`_async_create_egeria_bearer_token()` once and retries the original -request once before giving up. Default-on, no opt-out flag; at most one -retry ever (internal `_retry_on_auth` guard). +**Status:** fixed 2026-08-05 (Pyegeria — `pyegeria/omvs/valid_metadata.py`). -**Bonus bug found and fixed while implementing this:** -`_async_create_egeria_bearer_token()` (both the `BaseServerClient` and -`BasePlatformClient` copies) sent the token-creation POST with -`headers=self.headers` — which still carried the stale Authorization header -being replaced. The `/api/token` endpoint rejects requests with a bad -Authorization header outright (confirmed via direct curl repro), so the -refresh call itself always 401'd, silently defeating the whole mechanism -(both this new auto-retry and the pre-existing manual "call this if your -token expired" use case). Fixed to send a plain `Content-Type`-only header -for that one request. +**What:** `validate_guid` was referenced in +`_async_get_specification_property_by_guid` but never imported anywhere in +`valid_metadata.py`. Every call failed immediately with `NameError`, +regardless of arguments — affecting `SpecificationProperties`, +`ValidMetadataManager`, `ValidMetadataLists`, `ValidTypeLists` (all share +this one implementation via inheritance). -**Verified live:** corrupted a real client's Authorization header, made a -real API call, confirmed exactly one auto-retry occurred and the call -succeeded transparently with a freshly-obtained token. +**Fix:** added `from pyegeria.core._validators import validate_guid`, +matching the exact import convention already used in sibling OMVS modules +(`solution_architect.py`, `glossary_manager.py`, `automated_curation.py`, +`core_omag_server_config.py`). + +**Verified live, end-to-end:** confirmed the `NameError` is gone (a bogus +guid now correctly raises `PyegeriaNotFoundException` instead), then ran a +full round trip — found a real specification property via +`find_specification_property`, fetched it by guid, got back the correct +`identifier` (`placeholderProperty`). --- -### ISSUE-8 (GAP-5): `get_collection_members` silently dropped non-Collection members +### ISSUE-27: ~50 more sync methods delegate to their async counterpart with an all-positional argument list — unaudited for the same scrambling bug as ISSUE-21/25 -**Status:** fixed 2026-07-31. `_type` was overloaded in -`_async_get_results_body_request` for two purposes — genuine results -filtering (correct for most of its ~30 callers, e.g. "attached comments are -always type Comment") vs. a pure output-rendering hint (wrong only for -`_async_get_collection_members`'s `/{guid}/members` endpoint, since a -collection's members are never guaranteed to share its own type — e.g. a -`WorkItemList`'s members are `Project`s). +**Status:** fixed 2026-08-15 (Pyegeria). Full audit via a script (not manual +grepping): every multi-line all-positional `self._async_*(...)` delegation +across `pyegeria/omvs/*.py` (62 sites, dominated by +`classification_explorer.py`), plus every single-line all-positional call +with 3+ bare-identifier args (285 more candidates), each cross-checked +against its target async method's *actual* parameter order (careful to use +the target's **last** definition in the file, matching Python's real +same-name-method resolution — one earlier false lead came from a stale, +fully-shadowed duplicate `_async_delete_solution_blueprint` definition still +sitting in `solution_architect.py`, dead code but harmless since it's never +reachable). -Rather than dropping the default filter for all ~30 callers, added an -explicit `filter_results_by_type: bool = True` parameter (default preserves -existing behavior everywhere) and set it `False` only at the one -confirmed-broken call site. Checked `get_collection_hierarchy` (same file, -same `_type="Collection"` default) separately — its endpoint's own docs say -"return a hierarchy of nested collections," so that filter is legitimate -and was left unchanged. Also added `"body"` to the `"Collection Members"` -FormatSet's `optional_params` in `base_report_formats.py` so report callers -can pass a fully custom body. +Of ~350 candidate sites, found and fixed **4 real bugs**, all in +`pyegeria/omvs/governance_officer.py` and `pyegeria/omvs/solution_architect.py` +(the rest were either exact matches or "different local variable name, same +value, same position" false positives — e.g. `add_term_to_folder(folder_guid, +term_guid, body)` calling `_async_add_to_collection(folder_guid, term_guid, +body)` against a target expecting `(collection_guid, element_guid, body)` — +positionally correct, just domain-specific local names): -Live-verified against the original golden anchor (`WorkItemList` guid -`0affb580-fa81-4d00-9438-b26faf11845d`) — now correctly returns all 5 -`Project`-typed members instead of an empty list. +1. **`solution_architect.py`: `delete_solution_role`** — sync wrapper's own + signature is `(guid, body=None, cascade_delete=False)` (correct order, + matching the async target), but it called + `self._async_delete_solution_role(guid, cascade_delete, body)` — + transposed, sending the boolean into the `body` slot and the body dict + into the `cascade_delete` slot. Fixed to + `self._async_delete_solution_role(guid, body, cascade_delete)`. ---- +2. **`governance_officer.py`: `add_regulator_to_regulation`** — called the + **wrong async method entirely**: `self._async_link_governance_results(...)` + (a copy-paste leftover — a completely different relationship, hitting + `governance-metrics/{gov_metric_guid}/measurements/{data_asset_guid}/attach` + with `GovernanceResultsProperties`) instead of the correct + `self._async_add_regulator_to_regulation(...)`. Fixed the call target. -### ISSUE-9 (GAP-3): malformed `as_of_time` reached pydantic as a raw `ValidationError` +3. **`governance_officer.py`: `_async_add_regulator_to_regulation`/ + `_async_detach_regulator_from_regulation`** — found while fixing #2: both + built the wrong URL (`.../{url_marker}/governance-officer/regulations/...` + — `self.url_marker` is itself `"governance-officer"`, so this duplicated + the segment; ground truth per `Egeria-api-governance-officer.http` is + `.../governance-officer/regulations/{regulationGUID}/regulators/organizations/{regulatorGUID}/attach`, + single segment) and the attach method also used the wrong properties + class (`"GovernanceResultsProperties"` instead of `"RegulatorProperties"`, + also confirmed against the `.http` file). Fixed both URLs and the + properties class. -**Status:** fixed 2026-07-31. Scope turned out bigger than originally -suspected: `as_of_time` is inherited by the whole get/find/search model -family (`GetRequestBody` and its ~4 subclasses), and 51 -`.model_validate(...)`/`.validate_python(...)` call sites in -`_server_client.py` shared the same unguarded-validation pattern. Added a -generic `ServerClient._validate_body(validator, body)` static helper -(handles both the `self._xxx_adapter.validate_python` and -`SomeModel.model_validate` call shapes uniformly) that catches -`pydantic.ValidationError` and re-raises as -`PyegeriaInvalidParameterException` naming the offending field(s). Routed -all 49 active call sites through it (2 were already commented out). +4. **`governance_officer.py`: `detach_governance_results`** — sync wrapper + passed `data_asset_guid` **twice**: + `self._async_detach_governance_results(gov_metric_guid, data_asset_guid, + data_asset_guid, body)` against a 3-parameter target + `(gov_metric_guid, data_asset_guid, body)` — the extra positional arg + landed in the `body` slot, and the real `body` argument was silently + dropped (extra arg beyond the target's parameter count). Fixed by + removing the duplicate. -```python -from pyegeria import EgeriaTech -client = EgeriaTech(view_server="qs-view-server", platform_url="https://localhost:9443", - user_id="erinoverview", user_pwd="secret") -client.create_egeria_bearer_token() -client.find_collections("*", as_of_time="null") -# Now: PyegeriaInvalidParameterException citing the asOfTime field. -# Previously: raw pydantic_core.ValidationError. -``` +**Verified live** against `qs-view-server`: `delete_solution_role`'s fix +confirmed via a request-body spy (cascade/body no longer transposed); +`add_regulator_to_regulation`/`detach_regulator_from_regulation` verified +fully end-to-end — created a throwaway `Regulation` + `Organization`, +linked via the fixed method, confirmed via `.http`-matching URL/body, +detached, cleaned up. `detach_governance_results`'s fix confirmed via a +request-body spy showing the correct 2-argument call now reaches +`_async_delete_relationship_request` cleanly. ---- +**Found as a byproduct, NOT part of this issue's scope, tracked separately:** +attempting to live-verify `_async_link_governance_results` (unrelated, +untouched method) surfaced a real server-side/endpoint-design bug — see +ISSUE-57. -### ISSUE-10 (GAP-2): `user_id`/`server_name` with URL-unsafe characters weren't validated +`pytest tests/ -m unit -q` passes (exit 0). -**Status:** fixed 2026-07-31. `user_id` is embedded raw (no URL-encoding) -into several URL paths (e.g. `pyegeria/omvs/server_operations.py`'s -`.../users/{self.user_id}/status`). Neither `validate_server_name` nor -`validate_user_id` previously checked for characters unsafe in a URL path, -and `validate_user_id` wasn't even called during client construction. +**Original status:** open, found 2026-08-04 as a byproduct of the ISSUE-25 sentinel +audit. -Note: the original report's exact symptom ("a raw `httpx.InvalidURL` -propagates") didn't fully match what was found — the existing bare -`except Exception` in `_async_make_request` already wrapped it as the -generic `PyegeriaUnknownException`, just not a helpful, specific type. +**Layer:** Pyegeria (`classification_explorer.py`, `runtime_manager.py`, +`project_manager.py`, `automated_curation.py`, `glossary_manager.py`). -Added a shared `_validate_url_path_safe(value, param_name)` check in -`_validators.py` (rejects ASCII control characters and URL-structural -characters `/ ? # \` plus whitespace), wired into both -`validate_server_name` (already called in both client constructors, so -`server_name` was fixed for free) and `validate_user_id` (now also called -explicitly in both constructors). Now raises -`PyegeriaInvalidParameterException` immediately at construction time, -naming the exact bad character. +**What:** grep for `self\._async_\w+\(\s*\n(?:\s+\w+,?\s*\n)+\s*\)` (a +multi-line, all-positional call with no `=`, `*`, or `**` on any line) in +`pyegeria/omvs/*.py` turns up ~50 more call sites, mostly write/action +methods (set/clear classification, add/remove relationship) that don't +carry `graph_query_depth` — so ISSUE-25's sentinel-injection method (which +specifically hunted for that one parameter) couldn't confirm or rule them +out. ISSUE-21 and ISSUE-25's `find_root_elements` finding both prove this +call shape is a real, recurring risk in this codebase whenever a +delegate's parameter order doesn't exactly match the caller's. -```python -from pyegeria.omvs.server_operations import ServerOps -ServerOps("qs-view-server", "https://localhost:9443", user_id="bad\x00user", user_pwd="secret") -# Now: PyegeriaInvalidParameterException at construction, naming the character. -``` +**Candidate fix:** for each of the ~50 call sites, diff the caller's +positional argument order against the target method's actual parameter +order (the same manual check done for `find_root_elements` in ISSUE-25) +and convert to keyword arguments wherever they've drifted. Not attempted +here — flagged for a dedicated follow-up pass. --- -### ISSUE-20: `Analytic Parameters`/`Report Parameters` stringify every value, breaking list-valued analytic function params (e.g. `type_map`) - -**Status:** fixed 2026-08-01 (Pyegeria — `md_processing/v2/report.py`), -found 2026-07-31 while building an Analytic Function demo (Projects/Terms -comparison) for the Local Dashboards sample. +### ISSUE-26: `get_info_supply_chain_by_guid`/`get_solution_role_by_guid` accept `graph_query_depth`/`max_mermaid_node_count` but never use them — same "dead parameter" shape as ISSUE-23's third case -**What:** `parse_key_value()` (`md_processing/v2/utils.py`) parses every -Dictionary-style attribute value (simple/list/table form) as a plain string -— there's no JSON/type coercion. `_report_additional_properties()` -(`md_processing/v2/report.py:132-135`) then does -`{str(k): str(v) for k, v in analytic_params.items() if v is not None}` -before `json.dumps(...)` into the stored `analyticParams` key. For a scalar -param (`window: 90d`, `type_name: Project`) this round-trips fine — the -string IS the value. But for a param whose real type is a list (e.g. -`counts_by_type`/`growth_series`'s `type_map: list[tuple[str, str]]`), the -markdown author's only way to express it is typing the literal text -`type_map: [["Projects", "Project"], ["Terms", "GlossaryTerm"]]` on one -line — `parse_key_value` stores that whole thing as ONE string value, which -then gets `str()`'d (no-op, already a string) and JSON-encoded as a STRING, -not a real JSON array. At read time (`local_dashboards_handler.py`'s -`json.loads(raw_analytic_params)`, or any other consumer), `type_map` comes -back as a Python `str`, not a `list`. When `format_set_executor.py`'s -`_run_analytic_function` forwards it as a kwarg, -`counts_by_type(type_map="[[\"Projects\", \"Project\"], ...]")` tries to -iterate the string character-by-character and unpack each character into -`(label, type_name)` → `ValueError: not enough values to unpack (expected -2, got 1)`. +**Status:** fixed 2026-08-05 (Pyegeria — `pyegeria/omvs/solution_architect.py`). -**Where seen:** live repro — created a throwaway `Report` via `Create -Report` with `Report Spec: Analytic Demo - Assets by Type Breakdown` and -`Analytic Parameters: type_map: [["Projects", "Project"], ["Terms", -"GlossaryTerm"]]`. `--validate`/`--process` both succeeded (no attribute- -level error — the value parses as *a* string just fine). Fetching the -created element's `additionalProperties.analyticParams` showed -`{"type_map": "[[\"Projects\", \"Project\"], [\"Terms\", \"GlossaryTerm\"]]"}` -— the outer JSON is valid, but `type_map`'s VALUE is itself a string, not a -nested array. Calling `POST /api/report-specs/execute` with that exact -params dict reproduced the crash: `"not enough values to unpack (expected -2, got 1)"`. Cleaned up the throwaway Report afterward (`AssetMaker.delete_asset`). +**What:** `_async_get_info_supply_chain_by_guid` and +`_async_get_solution_role_by_guid` declared `graph_query_depth` but, when +`body` was `None`, sent **no body at all** +(`self._async_make_request("POST", url, **kwargs)`); passing +`max_mermaid_node_count` explicitly landed in `**kwargs` and was forwarded +straight into `_async_make_request()`, which doesn't accept it — a +`TypeError` on every attempt. Confirmed live before the fix: -**Impact:** any analytic function whose parameter type is a list/dict -(`counts_by_type`'s `type_map`, `growth_series`'s custom `type_map`, -`metric_trend`'s `metric_params`) cannot be set via Dr.Egeria's `Analytic -Parameters` (or `Report Parameters`, same code path) today — only flat -scalar params (`type_name`, `window`, `points`, `collection_guid`, etc.) -work. Worked around in the Local Dashboards sample/tutorial by sticking to -`count_elements(type_name=...)` (scalar) instead of `counts_by_type` -(list) for the Projects-vs-Terms comparison. +```python +sa.get_info_supply_chain_by_guid("", graph_query_depth=10, max_mermaid_node_count=10) +# TypeError: BaseServerClient._async_make_request() got an unexpected keyword argument 'max_mermaid_node_count' +``` -**Fix:** added `_coerce_analytic_value()` in `md_processing/v2/report.py` — -attempts `json.loads(v)` per-value before it's packed into the -`analyticParams` JSON blob, falling back to the original string if it isn't -valid JSON. Applied in `_report_additional_properties()`'s `Analytic -Parameters` handling (`Create Report`) and, since `View Report` -(`md_processing/v2/view.py`) turned out to have a second, related gap — -`Analytic Parameters` wasn't handled there **at all**, falling through to -the generic per-attribute branch and being treated as one param literally -named `analytic_parameters` holding the whole dict — added a matching -`Analytic Parameters` branch there too (merges flat into `params`, coerced, -mirroring the existing `Report Parameters` branch immediately above it). +Same shape as ISSUE-23's "third case" for `get_solution_blueprint_by_guid`/ +`get_solution_component_by_guid` (those two remain open/unfixed — this +entry only covers the two methods ISSUE-26 was scoped to). -Deliberately scoped to `analyticParams` only, not `_REPORT_PARAM_ATTRS` or -`Report Parameters` — those remain `Map` at the Egeria -element level (a real constraint, not a bug) and must stay strings; -`analyticParams` is a single JSON-encoded blob consumed purely by Python's -own `**kwargs` at execution time, where real types matter. +**Fix:** when `body is None`, build an `AnyTimeRequestBody` with +`graphQueryDepth`/`maxMermaidNodeCount` populated from the (now also +added) `max_mermaid_node_count` parameter, instead of sending no body / +dumping stray kwargs into `_async_make_request`. -**Verified live** (not just `--validate`): created a throwaway `Report` -with the exact original repro (`type_map: [["Projects", "Project"], -["Terms", "GlossaryTerm"]]`), confirmed `additionalProperties.analyticParams` -now stores `type_map` as a real nested JSON array, then called -`POST /api/report-specs/execute` with it — `counts_by_type` returned the -correct per-type breakdown (`Projects: 29, Terms: 591`) instead of -crashing. Also caught and fixed a second latent bug the same coercion -exposes: an `int`-typed scalar param (`points: 3`) previously round-tripped -as the string `"3"` too, which broke `growth_series` outright -(`TypeError: unsupported operand type(s) for -: 'str' and 'int'`) — now -correctly coerces to a real `int`. Ran -`tests/micro-tests/test_overview_metrics.py`, -`test_gen_report_specs.py`, `test_base_report_formats_mermaid.py` (34 -tests) — all pass, no regressions. Cleaned up the throwaway Report -afterward. Not yet released to PyPI — still pinned at `6.0.17.8` in -egeria-workspaces-fs's `requirements.txt`; needs a version bump when the -next release goes out. +**Verified live, both methods:** calls that previously raised `TypeError` +on every attempt now succeed and return the expected element. --- @@ -8106,185 +8238,254 @@ file. --- -### ISSUE-32: `create_note`/`update_note` used `"Notification"` as a placeholder `typeName`/`class` — replaced with the real `Note` type now that it's shipped (Egeria PR #9191) +### ISSUE-24: `AssetCatalog._async_get_asset_lineage_graph_by_guid` hardcodes `queryGraphDepth: 5` with no override parameter at all + +**Status:** fixed 2026-08-15 (Pyegeria — `pyegeria/omvs/asset_catalog.py`). +Turned out to be worse than "no override" — checked the real +`AssetLineageGraphRequestBody` Java class (`-> QueryOptions -> GetOptions`) +and the field pyegeria was sending, `"queryGraphDepth"`, **doesn't exist on +the real body at all**; the actual field is `"graphQueryDepth"`. So the +hardcoded `5` was always silently dropped by the server regardless — the +server's own default (also 5, coincidentally) was what actually applied +the whole time, even before this fix. Found the same problem one field +over: `"relationshipTypes"` isn't real either — the actual field, +`GetOptions.includeOnlyRelationships`, means that caller-supplied +relationship-type filter has also never worked. Fixed both field names, +added `graph_query_depth`/`max_mermaid_node_count` parameters (the real +`GetOptions.maxMermaidNodeCount` field was never sent at all before) to +all three call layers (`_async_get_asset_lineage_graph_by_guid`, +`get_asset_lineage_graph_by_guid`, `get_asset_lineage_mermaid_graph`). + +**Verified live**: created a real throwaway asset, spied on the outgoing +request body, and confirmed `graph_query_depth=7`/`max_mermaid_node_count=42` +now actually appear in the POST body as `graphQueryDepth`/ +`maxMermaidNodeCount` and the server accepts the call successfully. +`pytest tests/ -m unit` passes. + +**Original status:** open, found 2026-08-04 alongside ISSUE-23 while auditing +`egeria-workspaces-fs/.../lineage_handler.py`'s mermaid-graph call sites. + +**Layer:** Pyegeria (`pyegeria/omvs/asset_catalog.py`). + +**What:** unlike every other `_by_guid` method in this codebase, +`_async_get_asset_lineage_graph_by_guid` builds its `AssetLineageGraphRequestBody` +manually with `"queryGraphDepth": 5` hardcoded, and does not spread +`**kwargs` into that body at all — the method's own `**kwargs` parameter +exists but is never used. There is currently no way to override this from +a caller, unlike `get_asset_graph_by_guid` (a different, sibling method on +the same class going through `_async_get_results_body_request`, which +**does** correctly forward a `max_mermaid_node_count` kwarg — confirmed +and fixed in `lineage_handler.py`'s `get_asset_graph` route this session). + +**Where seen:** `egeria-workspaces-fs/compose-configs/*/PyegeriaWebHandler/lineage_handler.py`'s +`get_asset_lineage_graph` route (`/api/lineage/asset/{guid}/lineage-graph`) +— serves `mermaidGraph`/`fullLineageMermaidGraph`/`edgeMermaidGraph` for +the lineage explorer's full end-to-end diagram, exactly the kind of view +most likely to need a depth greater than 5 for a long pipeline. Left +unmodified in the portal since there's no parameter to pass. + +**Candidate fix:** add `graph_query_depth`/`max_mermaid_node_count` +parameters to `_async_get_asset_lineage_graph_by_guid`/ +`get_asset_lineage_graph_by_guid`, threading them into +`queryGraphDepth`/a new `maxMermaidNodeCount` body field (need to confirm +the server actually honors a `maxMermaidNodeCount` field on +`AssetLineageGraphRequestBody` specifically — this body class differs +from the more common `GetRequestBody`/`ResultsRequestBody` shapes seen +elsewhere in this file). + +--- + +### ISSUE-23: `max_mermaid_node_count` defaults to 5 across every shared find/get request helper, silently truncating server-generated mermaid graphs + +**Status:** fixed 2026-08-15 (Pyegeria). Raised every remaining +`max_mermaid_node_count` default from 5 to 10, matching +`_async_find_request`'s default (already 10 from an earlier pass): +`_async_get_name_request`, `_async_get_guid_request`, +`_async_get_request_body_request`, `_async_activity_status_search_request`, +and the content-status/deployment-status search variants (all in +`pyegeria/core/_server_client.py`), plus `AssetCatalog`'s three lineage-graph +methods in `pyegeria/omvs/asset_catalog.py` (`_async_get_asset_lineage_graph_by_guid`, +`get_asset_lineage_graph_by_guid`, `get_asset_lineage_mermaid_graph`, added +as part of ISSUE-24 at 5, now raised to 10 for consistency). Verified live +via request-body spies that the new default (10) actually reaches the wire +for `_async_get_guid_request` and `_async_get_asset_lineage_graph_by_guid`. + +Also fixed the "third case" this issue flagged as a candidate follow-up: +`get_solution_blueprint_by_guid`/`_async_get_solution_blueprint_by_guid` +(`pyegeria/omvs/solution_architect.py`) built no body at all when the +caller left `body=None`, so `graph_query_depth`/`max_mermaid_node_count` +were genuinely dead parameters regardless of the shared default — same +"dead parameter" shape ISSUE-26 already fixed for +`get_info_supply_chain_by_guid`/`get_solution_role_by_guid`. Fixed the same +way: build an `AnyTimeRequestBody` with `graphQueryDepth`/ +`maxMermaidNodeCount` populated when `body is None`, and added +`max_mermaid_node_count` as an explicit parameter (was previously not even +accepted). Verified live: a request-body spy confirmed caller-supplied +`graph_query_depth=9`/`max_mermaid_node_count=77` now reach the wire, where +previously neither did. `get_solution_component_by_guid` turned out **not** +to share this bug — it already routes through the (now-also-fixed) +`_async_get_guid_request` shared helper, so it correctly forwards both +parameters; verified live the same way (`graph_query_depth=8`/ +`max_mermaid_node_count=66` both reached the wire) before ruling it out. -**Status:** fixed 2026-08-05 (Pyegeria — `pyegeria/core/_server_client.py`, -`md_processing/v2/feedback.py`), found/actioned same day the user redeployed -Egeria with the new type. +`pytest tests/ -m unit -q` passes (exit 0) after all changes. -**Background:** Dr.Egeria's `Create Note` command was already a known, -explicitly-flagged placeholder — its compact-spec entry -(`md_processing/data/compact_commands/commands_feedback_compact.json`) reads -*"PARSE-ONLY STUB: no NoteProperties/NoteLogEntryProperties type exists -anywhere in pyegeria yet — a new Egeria type is in progress. ... do NOT wire -a processor for this command until the type ships and is confirmed."* with -`OM_TYPE: "Notification"` as the placeholder. Separately, -`ServerClient.create_note`/`update_note` (the note-log-entry creation/update -methods used by `_async_add_journal_entry` and directly by tests) built -their default request bodies with `"class": "NotificationProperties", -"typeName": "Notification"` for the same reason — no dedicated type existed. +**Original status:** open at the pyegeria-default level (the user is raising the +shared default from 5 to 10 separately); worked around explicitly in +egeria-workspaces-fs across all detail-view call sites that render a +mermaid diagram — `isc_handler.py`, `mermaid_handler.py`, +`solution_architect_handler.py`, `tech_catalog_handler.py`, +`digital_products_handler.py`, `action_center_handler.py`, +`actor_handler.py`, `community_handler.py`, `context_events_handler.py`, +`data_design_handler.py`, `external_links_handler.py`, +`glossary_handler.py`, `governance_definitions_handler.py`, +`lineage_handler.py` (asset graph only, see ISSUE-24 for the lineage +graph endpoint), `location_handler.py`, `perspectives_handler.py`, +`reference_data_handler.py` — found 2026-08-04 investigating a report +that Egeria Explorer's Information Supply Chain view wasn't rendering the +full chain. `audit_handler.py`'s two `get_element_by_guid` calls are +deliberately left at `graphQueryDepth: 0` (a lightweight side-panel +lookup with no relationship traversal, so no mermaid graph is generated +there anyway) and `notelog_handler.py` doesn't render mermaid fields — +both confirmed not affected, not just skipped. -**What changed:** confirmed via the actual Egeria source (not the stale -local `egeria-v6/egeria` checkout, which was 13 commits behind — checked -`git fetch` + `origin/main` directly) that commit `8d3b8d9a99` ("Add Note -type - a subtype of Notification", merged via PR #9191) added a real -`Note` `EntityDef` (typeGUID `391f73ff-c184-41b6-8620-310b4284039f`, -superType `Notification`, model area 0160 "Notes" — the same area as -`NoteLog`/`AttachedNoteLogEntry`, confirming `Note` is exactly the proper -type for what `create_note` already builds via the `AttachedNoteLogEntry` -relationship, not a separate new concept). `NoteProperties extends -NotificationProperties` with **zero new fields** — purely a type-identity -fix, no new attributes needed anywhere. +**Two distinct fix shapes were needed depending on the call site**, +learned the hard way after an initial fix silently didn't work: +- If the portal handler passes **no** `body=` argument (stays `None`), + a `max_mermaid_node_count=250` kwarg on the outer call correctly + threads through — every `_by_guid`/`find_*` wrapper method in pyegeria + follows the same `params.update(kwargs); ...; **params` idiom that + forwards it into the shared helper's own `max_mermaid_node_count` + parameter. +- If the portal handler already builds an explicit `body={"class": + "GetRequestBody", ...}` dict (common wherever `as_of_time` needs to be + conditionally added), `_async_get_guid_request`/`_async_find_request` + validate that body **as-is** and silently ignore the sibling + `max_mermaid_node_count`/`graph_query_depth` kwargs entirely — the + field must be embedded directly in the body dict + (`"maxMermaidNodeCount": 250`) instead. +- A third shape: `get_solution_blueprint_by_guid`/ + `get_solution_component_by_guid` hit dedicated `.../retrieve` REST + endpoints whose own `graph_query_depth` **parameter is dead** — never + reaches the request at all, body or no body. Passing an explicit + `AnyTimeRequestBody` with `graphQueryDepth`/`maxMermaidNodeCount` set + is the only way to raise it (verified live: 74 → 187 mermaid lines). + **Update 2026-08-04:** `get_info_supply_chain_by_guid`/ + `get_solution_role_by_guid` have the identical shape — see ISSUE-26. -**Verified live** (user redeployed Egeria with the new type same day): -- `ValidMetadataManager.get_typedef_by_name("Note")` confirms the type is - registered on `qs-view-server` (`beanClassName: NoteProperties`, correct - GUID). -- Created a real `Note`-typed element end-to-end via the fixed - `create_note` default body (no manual override needed) and fetched it - back — `elementHeader.type.typeName == "Note"`, confirmed. -- Cleaned up all throwaway test elements afterward - (`MetadataExpert.delete_metadata_element`, see ISSUE-31 for a bug hit - along the way). +**Layer:** Pyegeria (`pyegeria/core/_server_client.py`). -**Fixed:** -- `_server_client.py`: `_async_create_note`/`create_note` and - `_async_update_note`/`update_note` — default body's `"class"`/`"typeName"` - changed from `"NotificationProperties"`/`"Notification"` to - `"NoteProperties"`/`"Note"` (6 docstring sample-body occurrences too, for - consistency); the dead/vestigial `prop` type-list arguments passed to - `_async_create_element_body_request`/`_async_update_element_body_request` - updated to match (`['NoteProperties']`/`['Note']`) even though currently - unused by the validation code, to avoid leaving stale documentation of - intent. -- `md_processing/v2/feedback.py`: `FeedbackProcessor`'s `"Note" in - object_type` branch — `set_element_prop_body("Notification", ...)` → - `set_element_prop_body("Note", ...)` (this one has real effect — produces - `"class": "NoteProperties", "typeName": "Note"`); `set_update_body(...)`'s - first argument also updated for consistency, though that helper doesn't - actually use it (dead parameter, confirmed by reading the function body). +**What:** `_async_find_request`, `_async_get_name_request`, and +`_async_get_guid_request` — the three shared helpers behind essentially +every `find_*`/`get_*_by_guid` OMVS method — all declare +`max_mermaid_node_count: int = 5`, which gets sent to the server as +`maxMermaidNodeCount` in the request body. The server caps the generated +mermaid flowchart at that many nodes. 5 is far too small for any real +graph with more than a handful of related elements — a value seemingly +picked as a lightweight default for broad/unbounded listings, not for a +"show me everything about one element" detail view, but it applies +identically to both. -**Update 2026-08-05 — Dr.Egeria's `Create Note` command is now fully wired, -not just unblocked at the SDK level.** The Tinderbox-only bottleneck this -entry originally described no longer applies — as of the same day, compact- -spec edits go through the Dr.Egeria Spec Editor's local REST API instead -(`commands/tech/spec_editor.py`, `http://localhost:8420`; see `CLAUDE.md`'s -"Dr.Egeria Spec Editor" section and the `dr-egeria-command-sync` skill). -Using that API: `Create Note`'s `OM_TYPE` changed `"Notification"` → -`"Note"`, its stub description replaced with a real one, and -`"Commented On Element"` (the same reference attribute `Create Comment` -already uses, conveniently pre-aliased `"Associated Element"`) added to its -`custom_attributes` — the spec previously had no way to say what element a -note was about at all. `refresh_specs --merge-reports` regenerated -templates/help/report-specs; unit tests pass. Registered -`FeedbackProcessor` for `Create Note` in `setup_dispatcher()` and -implemented its `Create` branch (previously a stub with no code at all): it -now creates a dedicated `NoteLog` for the target element -(`_async_create_note_log`) and the `Note` entry on it -(`_async_create_note`, using the fixed `Note`/`NoteProperties` typing from -above). **Verified live end-to-end**, not just `--validate`: processed a -throwaway `dr_test_create_note_smoketest.md`, confirmed -`elementHeader.type.typeName == "Note"` on the created element via direct -fetch, then cleaned up both the `Note` and its `NoteLog` -(`MetadataExpert.delete_metadata_element`, see ISSUE-31 for a bug hit -along the way). Note: each `Create Note` call currently creates its own -private `NoteLog` rather than reusing an existing one on repeated notes -against the same element — not wrong (a `NoteLog` can hold many notes, -this just doesn't dedupe), flagged as a possible future improvement, not -attempted here. +**Where seen:** live repro against a Coco Pharmaceuticals demo server — +`SolutionArchitect.find_information_supply_chains("Open Metadata Highway +Information Supply Chain")`'s `mermaidGraph` field was 165 lines at the +default `max_mermaid_node_count=5`, vs. 619 lines at `250` (graph actually +complete at that point — verified by also trying `50`/`100`, no further +growth past ~200). `graph_query_depth` (default 3), by contrast, made no +measurable difference for any information supply chain tested (1 through +10) — the node-count cap, not query depth, is the actual bottleneck for +mermaid graph completeness. Every OMVS wrapper method that calls one of +these three shared helpers (`find_information_supply_chains`, +`get_element_by_guid`, `find_solution_blueprints`, +`find_solution_components`, `get_solution_component_by_guid`, etc.) +inherits the same 5-node cap unless the caller explicitly overrides it — +confirmed via grep that essentially none of egeria-workspaces-fs's +`PyegeriaWebHandler` handlers do (`isc_handler.py`, now fixed; +`solution_architect_handler.py`, `tech_catalog_handler.py`, +`mermaid_handler.py`, `digital_products_handler.py`, `audit_handler.py` +all still rely on the pyegeria default). -**Still not fixed:** -- `update_note`'s underlying REST operation 404s on this server regardless - of the type fix — see ISSUE-30, unrelated to this change, blocks live - verification of the update path specifically (create + fetch are - confirmed working). `Create Note`'s compact spec has no companion - `Update Note` command at all today (only `Create` was ever defined), so - this doesn't block Dr.Egeria usage — only direct SDK callers of - `update_note`/`_async_update_note`. -- **Fixed 2026-08-05, same day, follow-up session:** the orphaned - `"Create Journal Entry"` registration flagged above is resolved — see - ISSUE-33. +**Candidate fix (still open):** raise the shared `max_mermaid_node_count` +default itself (planned separately by the repo owner, 5 → 10) — the +per-handler overrides above are generous enough (250) that they don't +depend on that change, but the two "dead parameter" methods +(`get_solution_blueprint_by_guid`/`get_solution_component_by_guid`) would +benefit from an actual fix to thread `graph_query_depth`/ +`max_mermaid_node_count` into their request the same way every other +`_by_guid` method does, rather than requiring callers to know the +`AnyTimeRequestBody` workaround. --- -### ISSUE-33: `Create Journal Activity`/`Create Log Activity` used the `Notification` placeholder typeName; renamed to match real `JournalEntry`/`ActivityEntry` types, and the orphaned dead `Create Journal Entry` registration from ISSUE-32 is now a real, working command +### ISSUE-22: `Ownership`/`Impact`/`Confidence`/`Confidentiality`/`Criticality` classification `status` field expects an int enum, not the free-text value the Dr.Egeria "Status" attribute style implies -**Status:** fixed 2026-08-05 (Pyegeria + Dr.Egeria compact spec, same day as -ISSUE-32, follow-up in the same session). User question: since `JournalEntry` -is a real `Notification` subtype, why is Dr.Egeria's command called `Journal -Activity` instead — could it be renamed for consistency? +**Status:** fixed 2026-08-15. `Ownership` was never actually affected — its +`ClassificationSpec` uses its own explicit field map (`owner`/ +`ownerTypeName`/`ownerPropertyName`), not `_GOVERNANCE_SHARED_FIELDS`, so +the issue title overstated its scope; `Retention` *is* affected (it also +consumes `_GOVERNANCE_SHARED_FIELDS`) and is fixed by the same change as +`Impact`/`Confidence`/`Confidentiality`/`Criticality`. -**Investigation, grounded in the actual Egeria type system and docs (not -guessed):** -- Live type query confirmed `JournalEntry`, `ActivityEntry`, `BlogEntry` are - all real, distinct `Notification` subtypes (siblings of `Note` from - ISSUE-32) — `superType: Notification` for all three, live on - `qs-view-server`. -- Fetched `egeria-project.org/concepts/notification/`, - `/types/1/0160-Notes/`, and `/types/1/0135-Actions-For-People/` before - proceeding (per explicit user instruction) — confirmed `Note` and - `JournalEntry`/`ActivityEntry`/`BlogEntry` are siblings under the *same* - `NoteLog` + `AttachedNoteLogEntry` mechanism (`AttachedNoteLogEntry` - connects a NoteLog to "an Action (**typically a Notification**)" — not - restricted to one subtype), differing only in target (any element, for - `Note`; the calling user's own profile, for the other three) and - visibility (`JournalEntry` private; `ActivityEntry`/`BlogEntry` public). -- This also surfaced that `Create Journal Activity`/`Create Log Activity` - (bundle `My Journal Base`, mapping to `my_profile.py`'s - `journal_my_activity`/`log_my_activity`) were **not registered in the - dispatcher at all** — a bigger gap than a naming mismatch. `Create Blog - Entry` already had the correct `OM_TYPE: "BlogEntry"` but was equally - unregistered. -- Confirmed the pre-existing `reg("Create Journal Entry", FeedbackProcessor)` - (ISSUE-32's finding) was genuinely dead/orphaned — no command by that name - existed in the compact spec, and its code branch referenced attributes - that don't exist anywhere in `commands_feedback_compact.json`. Renaming - `Journal Activity` → `Journal Entry` would otherwise collide with that - dead registration, so it was removed first. +Two separate bugs were involved: +1. **Attribute-definition gap.** Between this issue being filed and now, + the compact-spec attribute had already been renamed `Status` → + `Governance Status` and migrated to `style: "Valid Value"` — but with no + explicit `property_name`, so it auto-derived to `governanceStatus` (a + property that doesn't exist), causing the live valid-metadata-value + lookup to silently return empty and fall through to a stale ALL-CAPS + fallback list, passing through an un-converted string. Fixed via the + Spec Editor API: `Governance Status` now has `property_name: + "statusIdentifier"` and `valid_values` = `["Discovered", "Proposed", + "Imported", "Validated", "Deprecated", "Obsolete", "Other"]`, matching + `GovernanceClassificationStatus`. +2. **Wire-key mapping bug**, in `md_processing/v2/curation.py`: + `_GOVERNANCE_SHARED_FIELDS` mapped `"Governance Status"` to the outgoing + JSON key `"status"`, but the real field on `ImpactProperties`/ + `ConfidenceProperties`/`ConfidentialityProperties`/`CriticalityProperties`/ + `RetentionClassificationProperties` is `statusIdentifier` — `"status"` is + silently ignored by the server (confirmed live: sending `"status": 4` + left readback `statusIdentifier: "0"`; sending `"statusIdentifier": 4` + directly persisted correctly). Fixed: `_GOVERNANCE_SHARED_FIELDS = + {"Governance Status": "statusIdentifier", ...}`. + +Verified end-to-end through the real Dr.Egeria pipeline (not just manual +body construction): `Create Data Structure` + `Classify Impact` with +`Governance Status: Validated`, `Level Identifier: 2`, `Steward: +test-steward` — both commands SUCCESS, and a direct element fetch afterward +showed `statusIdentifier: 3` (the correct ordinal for `Validated`) alongside +`severityLevel: 2` and `steward: "test-steward"`. Test element deleted +afterward. `pytest tests/ -m unit -q` passes (exit 0). -**Fixed, via the Dr.Egeria Spec Editor's REST API** (compact spec — see -ISSUE-32's entry for why raw JSON edits aren't used): -- Deleted `Create Journal Activity`, created `Create Journal Entry` - (`OM_TYPE: "JournalEntry"`, `alternate_names: ["Journal Activity"]`). -- Deleted `Create Log Activity`, created `Create Activity Entry` - (`OM_TYPE: "ActivityEntry"`, `alternate_names: ["Log Activity"]`) — same - consistency rationale as the Journal rename, so it wasn't left half-fixed. -- `refresh_specs --merge-reports` regenerated templates/help/report-specs; - unit tests pass. +**Original status:** open, found 2026-08-03 during a live smoke test of `Classify +Impact` (Curation family). -**Fixed, code (`md_processing/dr_egeria.py`, `md_processing/v2/feedback.py`):** -- Removed the dead `reg("Create Journal Entry", FeedbackProcessor)` + - its broken code branch. -- Registered `Create Journal Entry`, `Create Activity Entry`, `Create Blog - Entry` → `FeedbackProcessor`. -- Implemented one shared branch (`object_type in ("Journal Entry", - "Activity Entry", "Blog Entry")`) dispatching to - `my_profile._async_journal_my_activity`/`_async_log_my_activity`/ - `_async_blog_my_activity` respectively. +**Layer:** Not strictly a pyegeria bug — a Dr.Egeria compact-spec design +gap (`md_processing/data/compact_commands/commands_curation_compact.json`, +`Status` attribute, style `Simple`) combined with `curation.py`'s field +mapping (`_GOVERNANCE_SHARED_FIELDS = {"Status": "status", ...}`). -**Verified live, all three, end-to-end** (not just `--validate`): processed -a throwaway smoke-test doc through the real `dr_egeria --process` pipeline — -all three `SUCCESS` with real GUIDs — then fetched each back and confirmed -`elementHeader.type.typeName` was exactly `JournalEntry`/`ActivityEntry`/ -`BlogEntry` respectively. Cleaned up all three afterward. +**What:** `ImpactProperties`/etc.'s real `status` field is an integer +`GovernanceClassificationStatus` enum ordinal (per +`Egeria-api-classification-explorer.http`'s worked example: `"status" : 0`), +not a free-text string. Passing `"ACTIVE"` (a plausible-looking value given +the attribute's `Simple` style) doesn't error — it's silently dropped by +the server, and the classification's `statusIdentifier` (the field's +readback name) stays at its default `0`. -**Not touched:** `my_profile.py`'s journal/log/blog-my-activity NoteLogs -(the containers these entries were created in) — unlike `Create Note`'s -throwaway per-note `NoteLog`, these are the user's own persistent -activity-stream containers and weren't deleted during cleanup, only the -three created entries were. +**Where seen:** live repro — `Classify Impact` with `Status: ACTIVE`, +`Level Identifier: 2`, `Steward`/`Source`/`Description` set. Fetching the +classified element back showed `severityLevel: "2"`, `steward`, `source`, +`notes` all correct, but `statusIdentifier: "0"` regardless of the `Status` +value sent — confirms `status` never took effect as text. -**Downstream propagation completed same session:** templates regenerated -and mirrored into `egeria-workspaces-fs`/`egeria-advisor` (old -`Create_Journal_Activity.md`/`Create_Log_Activity.md` deleted, new -`Create_Journal_Entry.md`/`Create_Activity_Entry.md` added, confirmed via -`git status` in both downstream repos); `docs/dr_egeria_manual.md`'s -Feedback family description updated; `egeria-workspaces-fs/portal-docs/ -tools/dr-egeria/templates-basic.md`'s Journal Entry example corrected to -use real attribute names (`Display Name`/`Description`, not the -placeholder `Subject`/`Entry` it previously showed) and noted the sibling -commands; dr-egeria Glossary (help Terms) regenerated and reprocessed -against the live server. +**Candidate fix:** change the `Status` attribute's style to `Valid Value`/ +`Enum` with the real `GovernanceClassificationStatus` ordinal labels (needs +confirming the exact enum from the Egeria type system rather than guessing +further), authored in Tinderbox per the usual compact-spec workflow — not +attempted here, since it's a spec authoring change, not a Curation +processor-wiring bug. Doesn't block Impact/Confidence/etc. classification +overall — `severityLevel`, `steward`, `source`, `notes` all confirmed +working correctly. --- @@ -8319,348 +8520,508 @@ is `.../classification-explorer/elements/scoped-by/{scopeGUID}`, no **Fix:** removed the stray `glossaries/` segment. -**Verified live:** `get_scoped_elements`/`get_scopes` against a real -element guid both now return `"No elements found"` (correct — that -element has no `ScopedBy` relationships) instead of a 404 exception. -Confirmed the fix doesn't regress the write path -(`add_scope_to_element`/`clear_scope_from_element`, unaffected by this -change, still used successfully elsewhere in this session's work). +**Verified live:** `get_scoped_elements`/`get_scopes` against a real +element guid both now return `"No elements found"` (correct — that +element has no `ScopedBy` relationships) instead of a 404 exception. +Confirmed the fix doesn't regress the write path +(`add_scope_to_element`/`clear_scope_from_element`, unaffected by this +change, still used successfully elsewhere in this session's work). + +--- + +### ISSUE-20: `Analytic Parameters`/`Report Parameters` stringify every value, breaking list-valued analytic function params (e.g. `type_map`) + +**Status:** fixed 2026-08-01 (Pyegeria — `md_processing/v2/report.py`), +found 2026-07-31 while building an Analytic Function demo (Projects/Terms +comparison) for the Local Dashboards sample. + +**What:** `parse_key_value()` (`md_processing/v2/utils.py`) parses every +Dictionary-style attribute value (simple/list/table form) as a plain string +— there's no JSON/type coercion. `_report_additional_properties()` +(`md_processing/v2/report.py:132-135`) then does +`{str(k): str(v) for k, v in analytic_params.items() if v is not None}` +before `json.dumps(...)` into the stored `analyticParams` key. For a scalar +param (`window: 90d`, `type_name: Project`) this round-trips fine — the +string IS the value. But for a param whose real type is a list (e.g. +`counts_by_type`/`growth_series`'s `type_map: list[tuple[str, str]]`), the +markdown author's only way to express it is typing the literal text +`type_map: [["Projects", "Project"], ["Terms", "GlossaryTerm"]]` on one +line — `parse_key_value` stores that whole thing as ONE string value, which +then gets `str()`'d (no-op, already a string) and JSON-encoded as a STRING, +not a real JSON array. At read time (`local_dashboards_handler.py`'s +`json.loads(raw_analytic_params)`, or any other consumer), `type_map` comes +back as a Python `str`, not a `list`. When `format_set_executor.py`'s +`_run_analytic_function` forwards it as a kwarg, +`counts_by_type(type_map="[[\"Projects\", \"Project\"], ...]")` tries to +iterate the string character-by-character and unpack each character into +`(label, type_name)` → `ValueError: not enough values to unpack (expected +2, got 1)`. + +**Where seen:** live repro — created a throwaway `Report` via `Create +Report` with `Report Spec: Analytic Demo - Assets by Type Breakdown` and +`Analytic Parameters: type_map: [["Projects", "Project"], ["Terms", +"GlossaryTerm"]]`. `--validate`/`--process` both succeeded (no attribute- +level error — the value parses as *a* string just fine). Fetching the +created element's `additionalProperties.analyticParams` showed +`{"type_map": "[[\"Projects\", \"Project\"], [\"Terms\", \"GlossaryTerm\"]]"}` +— the outer JSON is valid, but `type_map`'s VALUE is itself a string, not a +nested array. Calling `POST /api/report-specs/execute` with that exact +params dict reproduced the crash: `"not enough values to unpack (expected +2, got 1)"`. Cleaned up the throwaway Report afterward (`AssetMaker.delete_asset`). + +**Impact:** any analytic function whose parameter type is a list/dict +(`counts_by_type`'s `type_map`, `growth_series`'s custom `type_map`, +`metric_trend`'s `metric_params`) cannot be set via Dr.Egeria's `Analytic +Parameters` (or `Report Parameters`, same code path) today — only flat +scalar params (`type_name`, `window`, `points`, `collection_guid`, etc.) +work. Worked around in the Local Dashboards sample/tutorial by sticking to +`count_elements(type_name=...)` (scalar) instead of `counts_by_type` +(list) for the Projects-vs-Terms comparison. + +**Fix:** added `_coerce_analytic_value()` in `md_processing/v2/report.py` — +attempts `json.loads(v)` per-value before it's packed into the +`analyticParams` JSON blob, falling back to the original string if it isn't +valid JSON. Applied in `_report_additional_properties()`'s `Analytic +Parameters` handling (`Create Report`) and, since `View Report` +(`md_processing/v2/view.py`) turned out to have a second, related gap — +`Analytic Parameters` wasn't handled there **at all**, falling through to +the generic per-attribute branch and being treated as one param literally +named `analytic_parameters` holding the whole dict — added a matching +`Analytic Parameters` branch there too (merges flat into `params`, coerced, +mirroring the existing `Report Parameters` branch immediately above it). + +Deliberately scoped to `analyticParams` only, not `_REPORT_PARAM_ATTRS` or +`Report Parameters` — those remain `Map` at the Egeria +element level (a real constraint, not a bug) and must stay strings; +`analyticParams` is a single JSON-encoded blob consumed purely by Python's +own `**kwargs` at execution time, where real types matter. + +**Verified live** (not just `--validate`): created a throwaway `Report` +with the exact original repro (`type_map: [["Projects", "Project"], +["Terms", "GlossaryTerm"]]`), confirmed `additionalProperties.analyticParams` +now stores `type_map` as a real nested JSON array, then called +`POST /api/report-specs/execute` with it — `counts_by_type` returned the +correct per-type breakdown (`Projects: 29, Terms: 591`) instead of +crashing. Also caught and fixed a second latent bug the same coercion +exposes: an `int`-typed scalar param (`points: 3`) previously round-tripped +as the string `"3"` too, which broke `growth_series` outright +(`TypeError: unsupported operand type(s) for -: 'str' and 'int'`) — now +correctly coerces to a real `int`. Ran +`tests/micro-tests/test_overview_metrics.py`, +`test_gen_report_specs.py`, `test_base_report_formats_mermaid.py` (34 +tests) — all pass, no regressions. Cleaned up the throwaway Report +afterward. Not yet released to PyPI — still pinned at `6.0.17.8` in +egeria-workspaces-fs's `requirements.txt`; needs a version bump when the +next release goes out. + +--- + +### ISSUE-19 (PY-12): `ReferenceDataManager` has no specification-property or valid-metadata-value methods + +**Status:** fixed — re-verified 2026-08-15, found already resolved (never +marked closed here). `ReferenceDataManager`'s class docstring +(`pyegeria/omvs/reference_data.py`) now has a `Note` section stating it +does NOT cover Specification Properties and pointing callers at +`SpecificationProperties`/`MetadataExpert` instead — exactly the +"suggested improvement" below, already applied. Not clear which session +added it since this entry was last touched; recommending a class-selection +decision tree across `ReferenceDataManager`/`SpecificationProperties`/ +`ValidMetadataLists`/`ValidTypeLists`/`MetadataExpert` is still open as a +nice-to-have, not a defect. + +**Originally filed as:** open (docs/organization), re-verified 2026-07-31 — still true. +`ReferenceDataManager` inherits only from `ServerClient` (it's for +*business* reference data — country codes, currency codes, etc.), not from +`ValidMetadataManager`. `get_valid_metadata_values` happens to work on it +because that method lives on the shared `ServerClient` base, not on +`ValidMetadataManager` — easy to misjudge the class boundary from that +alone. The specification-property methods +(`get_specification_property_types`, `get_specification_property_by_type`, +`get_specification_property_by_name`, `get_specification_property_by_guid`, +`find_specification_property`) only exist on +**`pyegeria.SpecificationProperties`** (also on `ValidMetadataManager` +subclasses `ValidMetadataLists`, `ValidTypeLists`). + +**Fix:** use `pyegeria.SpecificationProperties(...)` for these calls +instead of `ReferenceDataManager`. + +**Suggested improvement:** either a class-level docstring note on +`ReferenceDataManager` clarifying the boundary, or a single unified +decision tree for "which OMVS client class do I need" across +`ReferenceDataManager` / `SpecificationProperties` / `ValidMetadataLists` / +`ValidTypeLists` / `MetadataExpert`, since several overlap in purpose. + +--- + +### ISSUE-17 (PY-13): `SpecificationProperties.get_specification_property_by_type` always returns 400 + +**Status:** fixed 2026-08-15 (Pyegeria — `pyegeria/omvs/valid_metadata.py`). +**The original diagnosis was wrong** — not a server-side enum-binding drift. +Checked `Egeria-api-valid-metadata.http`'s `getSpecificationPropertyByType` +worked example: it uses `specificationPropertyType=PRODUCED_GUARD` — +`SCREAMING_SNAKE_CASE`, not the value shape the original repro tried +(`"SpecificationPropertyType{placeholderProperty}"`, nor the raw +PascalCase keys `get_specification_property_types()` returns, e.g. +`"ProducedGuard"`). Tried `PRODUCED_GUARD` directly — **succeeded +immediately**, no 400 at all. Root cause: `get_specification_property_types()` +returns its `stringMap` verbatim from the server's own +`specification-properties/type-names` endpoint, which genuinely uses +PascalCase keys (`"ProducedGuard"`) — a different server endpoint using a +different casing convention for the same enum than `by-type` expects. This +is a real cross-endpoint inconsistency in the server, but the client-visible +symptom ("every input 400s") was entirely avoidable: `get_specification_property_by_type` +did zero conversion, so the single most natural, discoverable input for a +caller (feeding it a key from `get_specification_property_types()`) always +failed. + +**Fix:** `_async_get_specification_property_by_type` now auto-converts a +PascalCase/camelCase input to `SCREAMING_SNAKE_CASE` before building the URL +(a plain regex insert-underscore-before-each-capital + uppercase — no +hardcoded type list, so it isn't tied to today's enum values); an +already-`SCREAMING_SNAKE_CASE` value passes through untouched. + +**Verified live** against `qs-view-server`: all 12 real specification +property types returned by `get_specification_property_types()`, fed +straight into `get_specification_property_by_type()` unmodified, now +resolve successfully (10 return real results, 2 correctly return "No +elements found" — no data of that type, not an error); the already-correct +`PRODUCED_GUARD` form still works unchanged. `pytest tests/ -m unit` +passes. + +**Impact on the workaround downstream:** `find_specification_property("*", +...)` + client-side filtering is no longer necessary to work around this +specific method, though it may still be independently useful/faster for +some callers. + +**Original status:** open (Egeria server), re-verified 2026-07-31 (live) — still +reproduces exactly as described. Root cause is server-side: the OpenAPI +schema declares `specificationPropertyType` as a required enum query param +with values like `"SpecificationPropertyType{placeholderProperty}"`, but +every form of that value (plain, enum-wrapped, percent-encoded) still 400s +— the Spring `@RequestParam` enum binding appears to have drifted from the +OpenAPI-declared enum. No pyegeria code change can fix a 400 the server +returns for every input. + +**Original workaround (superseded, no longer needed for this method):** use +`find_specification_property("*", ...)` (by-search-string) instead, and +filter client-side on `element["properties"]["identifier"]` (camelCase form +of the type name). + +```python +from pyegeria import SpecificationProperties +mgr = SpecificationProperties(view_server="qs-view-server", platform_url="https://localhost:9443", + user_id="erinoverview", user_pwd="secret") +mgr.create_egeria_bearer_token() +types = mgr.get_specification_property_types() +mgr.get_specification_property_by_type(list(types.keys())[0]) +# Now succeeds -- was previously 400ing for every input, confirmed 2026-07-31. +``` --- -### ISSUE-26: `get_info_supply_chain_by_guid`/`get_solution_role_by_guid` accept `graph_query_depth`/`max_mermaid_node_count` but never use them — same "dead parameter" shape as ISSUE-23's third case +### ISSUE-14 (PY-4): `update_comment` demands `qualifiedName` even with `mergeUpdate: true` -**Status:** fixed 2026-08-05 (Pyegeria — `pyegeria/omvs/solution_architect.py`). +**Status:** tracker was stale — re-checked 2026-08-15 and found the +workaround has actually been **in pyegeria itself** since 2026-08-03 +(`92cde6a`), not "application code" as this entry previously said. +`_async_update_comment` (`pyegeria/core/_server_client.py`) already +auto-fetches the comment's current `qualifiedName` via +`_async_get_comment_by_guid` whenever the caller's body doesn't supply one, +and injects it before sending — the exact workaround this entry describes, +just already shipped and never reflected here. **Verified live**: created a +throwaway `DataStructure`, attached a comment via `add_comment_to_element`, +then called `update_comment(comment_guid, comment="edited text")` with no +`qualifiedName` in the body at all — succeeds cleanly (previously would +have hit `OMAG-METADATA-400-004`). Cleaned up the test element afterward. +Underlying server behavior (still demanding `qualifiedName` despite +`mergeUpdate: true`) is unchanged and still worth an upstream Egeria report, +but there is nothing left to do on the pyegeria side — every caller of +`update_comment`/`_async_update_comment` already gets the workaround for +free, no application-level workaround needed. -**What:** `_async_get_info_supply_chain_by_guid` and -`_async_get_solution_role_by_guid` declared `graph_query_depth` but, when -`body` was `None`, sent **no body at all** -(`self._async_make_request("POST", url, **kwargs)`); passing -`max_mermaid_node_count` explicitly landed in `**kwargs` and was forwarded -straight into `_async_make_request()`, which doesn't accept it — a -`TypeError` on every attempt. Confirmed live before the fix: +**Original status:** open (Egeria server) — workaround shipped in application code, +not a pyegeria fix. Passing `"mergeUpdate": true` should allow a partial +update (only supplied fields required), but the server still demands +`qualifiedName` regardless. + +**Original workaround (superseded, no longer needed by callers):** fetch the +element first via `get_comment_by_guid`, extract `qualifiedName`, and always +include it in the update body regardless of `mergeUpdate`. ```python -sa.get_info_supply_chain_by_guid("", graph_query_depth=10, max_mermaid_node_count=10) -# TypeError: BaseServerClient._async_make_request() got an unexpected keyword argument 'max_mermaid_node_count' +from pyegeria import EgeriaTech +client = EgeriaTech(view_server="qs-view-server", platform_url="https://localhost:9443", + user_id="erinoverview", user_pwd="secret") +client.create_egeria_bearer_token() +client.update_comment(comment_guid="", body={"commentText": "edited text"}) +# OMAG-METADATA-400-004 demanding qualifiedName even though mergeUpdate: true is set. ``` -Same shape as ISSUE-23's "third case" for `get_solution_blueprint_by_guid`/ -`get_solution_component_by_guid` (those two remain open/unfixed — this -entry only covers the two methods ISSUE-26 was scoped to). +--- -**Fix:** when `body is None`, build an `AnyTimeRequestBody` with -`graphQueryDepth`/`maxMermaidNodeCount` populated from the (now also -added) `max_mermaid_node_count` parameter, instead of sending no body / -dumping stray kwargs into `_async_make_request`. +### ISSUE-10 (GAP-2): `user_id`/`server_name` with URL-unsafe characters weren't validated -**Verified live, both methods:** calls that previously raised `TypeError` -on every attempt now succeed and return the expected element. +**Status:** fixed 2026-07-31. `user_id` is embedded raw (no URL-encoding) +into several URL paths (e.g. `pyegeria/omvs/server_operations.py`'s +`.../users/{self.user_id}/status`). Neither `validate_server_name` nor +`validate_user_id` previously checked for characters unsafe in a URL path, +and `validate_user_id` wasn't even called during client construction. ---- +Note: the original report's exact symptom ("a raw `httpx.InvalidURL` +propagates") didn't fully match what was found — the existing bare +`except Exception` in `_async_make_request` already wrapped it as the +generic `PyegeriaUnknownException`, just not a helpful, specific type. -### ISSUE-28: `get_specification_property_by_guid` raised a bare `NameError: name 'validate_guid' is not defined` — cannot be called at all +Added a shared `_validate_url_path_safe(value, param_name)` check in +`_validators.py` (rejects ASCII control characters and URL-structural +characters `/ ? # \` plus whitespace), wired into both +`validate_server_name` (already called in both client constructors, so +`server_name` was fixed for free) and `validate_user_id` (now also called +explicitly in both constructors). Now raises +`PyegeriaInvalidParameterException` immediately at construction time, +naming the exact bad character. -**Re-verified 2026-08-15, no regression.** Bogus GUID now correctly raises -`PyegeriaNotFoundException` (not `NameError`); a real round trip (find a -specification property, fetch it by the returned GUID) succeeds and -returns the correct `identifier` (`supportedAnalysisStep`). Also fixed a -stale "currently broken outright" note pointing at this issue in the -Quick-reference table further down this file. +```python +from pyegeria.omvs.server_operations import ServerOps +ServerOps("qs-view-server", "https://localhost:9443", user_id="bad\x00user", user_pwd="secret") +# Now: PyegeriaInvalidParameterException at construction, naming the character. +``` -**Status:** fixed 2026-08-05 (Pyegeria — `pyegeria/omvs/valid_metadata.py`). +--- -**What:** `validate_guid` was referenced in -`_async_get_specification_property_by_guid` but never imported anywhere in -`valid_metadata.py`. Every call failed immediately with `NameError`, -regardless of arguments — affecting `SpecificationProperties`, -`ValidMetadataManager`, `ValidMetadataLists`, `ValidTypeLists` (all share -this one implementation via inheritance). +### ISSUE-9 (GAP-3): malformed `as_of_time` reached pydantic as a raw `ValidationError` -**Fix:** added `from pyegeria.core._validators import validate_guid`, -matching the exact import convention already used in sibling OMVS modules -(`solution_architect.py`, `glossary_manager.py`, `automated_curation.py`, -`core_omag_server_config.py`). +**Status:** fixed 2026-07-31. Scope turned out bigger than originally +suspected: `as_of_time` is inherited by the whole get/find/search model +family (`GetRequestBody` and its ~4 subclasses), and 51 +`.model_validate(...)`/`.validate_python(...)` call sites in +`_server_client.py` shared the same unguarded-validation pattern. Added a +generic `ServerClient._validate_body(validator, body)` static helper +(handles both the `self._xxx_adapter.validate_python` and +`SomeModel.model_validate` call shapes uniformly) that catches +`pydantic.ValidationError` and re-raises as +`PyegeriaInvalidParameterException` naming the offending field(s). Routed +all 49 active call sites through it (2 were already commented out). -**Verified live, end-to-end:** confirmed the `NameError` is gone (a bogus -guid now correctly raises `PyegeriaNotFoundException` instead), then ran a -full round trip — found a real specification property via -`find_specification_property`, fetched it by guid, got back the correct -`identifier` (`placeholderProperty`). +```python +from pyegeria import EgeriaTech +client = EgeriaTech(view_server="qs-view-server", platform_url="https://localhost:9443", + user_id="erinoverview", user_pwd="secret") +client.create_egeria_bearer_token() +client.find_collections("*", as_of_time="null") +# Now: PyegeriaInvalidParameterException citing the asOfTime field. +# Previously: raw pydantic_core.ValidationError. +``` --- -### ISSUE-35 (PY-15): Postgres repository connector ignored `matchCriteria` on `SearchClassifications` — multi-classification search always returned 0 +### ISSUE-8 (GAP-5): `get_collection_members` silently dropped non-Collection members -**Status: FIXED and CLOSED (Egeria server, not pyegeria)** — verified -2026-07-17. Consolidated in from `egeria-workspaces-fs/PYEGERIA_ISSUES.md` -2026-08-05 (both files independently tracked pyegeria/Egeria issues; this -one and ISSUE-36 through ISSUE-41 below were unique to that copy — see this -file's top-of-doc note for the consolidation). +**Status:** fixed 2026-07-31. `_type` was overloaded in +`_async_get_results_body_request` for two purposes — genuine results +filtering (correct for most of its ~30 callers, e.g. "attached comments are +always type Comment") vs. a pure output-rendering hint (wrong only for +`_async_get_collection_members`'s `/{guid}/members` endpoint, since a +collection's members are never guaranteed to share its own type — e.g. a +`WorkItemList`'s members are `Project`s). -Originally confirmed as a server-side bug 2026-07-15, while building Egeria -Insights (`insights_handler.py`) — not a pyegeria client bug, the client -sent the body faithfully; the Postgres repository connector's SQL -generation dropped `matchCriteria` on the classification-matching path -only. Fixed server-side and re-verified live against `qs-view-server` once -the fixed server was deployed: +Rather than dropping the default filter for all ~30 callers, added an +explicit `filter_results_by_type: bool = True` parameter (default preserves +existing behavior everywhere) and set it `False` only at the one +confirmed-broken call site. Checked `get_collection_hierarchy` (same file, +same `_type="Collection"` default) separately — its endpoint's own docs say +"return a hierarchy of nested collections," so that filter is legitimate +and was left unchanged. Also added `"body"` to the `"Collection Members"` +FormatSet's `optional_params` in `base_report_formats.py` so report callers +can pass a fully custom body. -``` -ZoneMembership ANY: 150 (unchanged -- single-condition baseline) -Confidentiality ANY: 1 (unchanged -- single-condition baseline) -Both ANY: 150 (was 0 -- now a real, differentiated union) -Both ALL: 0 ("No elements found" -- correct, empty intersection) -Both NONE: 1000 (was 0 -- now a real, differentiated count) -``` +Live-verified against the original golden anchor (`WorkItemList` guid +`0affb580-fa81-4d00-9438-b26faf11845d`) — now correctly returns all 5 +`Project`-typed members instead of an empty list. -`ANY`/`ALL`/`NONE` now produce distinct, semantically correct results -instead of all being an unconditional AND that always returned zero. +--- -**Root cause:** `QueryBuilder.getSearchClassificationsClause()` in -`open-metadata-implementation/adapters/open-connectors/repository-services-connectors/open-metadata-collection-store-connectors/postgres-repository-connector/src/main/java/org/odpi/openmetadata/adapters/repositoryservices/postgres/repositoryconnector/database/QueryBuilder.java` -(lines 1036–1078) unconditionally `AND`s an `AND (type_name LIKE -'%::%' ...)` clause per classification condition and never reads -`matchClassifications.getMatchCriteria()` at all — confirmed by grepping -the whole file: `MatchCriteria` is read in the *property*-matching path -(`getPropertyComparisonFromPropertyConditions()`, ~line 896) but never in -the classification-matching path. So the generated SQL always required -every named classification to appear on the same classification-table join -simultaneously, regardless of what `matchCriteria` the caller asked for. +### ISSUE-7 (GAP-6): expired/invalid bearer token had no specific exception and no auto-renew -**Regression coverage:** -- `egeria-python/tests/functional-tests/test_metadata_expert.py::test_find_metadata_elements_multi_classification_any_match_criteria` - (pytest, asserts ANY's count >= max of the two single-condition counts) — - passes as of 2026-07-17. -- `egeria-python/pyegeria/http clients/Egeria-PY15-matchClassifications-bug.http` - (PyCharm/IntelliJ HTTP Client collection, same assertions via raw REST - calls). +**Status:** fixed 2026-07-31. Connects to the (now closed) `BACKLOG.md` +issue "Bearer token expires mid-run on long `dr_egeria --process` batches." ---- +Confirmed by direct repro against `qs-view-server`: a bad/expired bearer +token returns a bare HTTP 401 with an **empty response body** — no JSON, +nothing to distinguish "expired" from "wrong credentials" from +"insufficient permission." Since there's no way to detect "expired" +specifically from the response, `_async_make_request` now treats any 401 +as a candidate for "the token might be stale": if the client holds stored +credentials, it calls the already-existing +`_async_create_egeria_bearer_token()` once and retries the original +request once before giving up. Default-on, no opt-out flag; at most one +retry ever (internal `_retry_on_auth` guard). -### ISSUE-36 (PY-16): `ClassificationExplorer.link_elements_as_peer_duplicates` (and its `_async_*` twin) POST to the wrong URL path — always 404s +**Bonus bug found and fixed while implementing this:** +`_async_create_egeria_bearer_token()` (both the `BaseServerClient` and +`BasePlatformClient` copies) sent the token-creation POST with +`headers=self.headers` — which still carried the stale Authorization header +being replaced. The `/api/token` endpoint rejects requests with a bad +Authorization header outright (confirmed via direct curl repro), so the +refresh call itself always 401'd, silently defeating the whole mechanism +(both this new auto-retry and the pre-existing manual "call this if your +token expired" use case). Fixed to send a plain `Content-Type`-only header +for that one request. -**Status: FIXED** — confirmed 2026-07-17 on pyegeria 6.0.16.20. -Consolidated in from `egeria-workspaces-fs/PYEGERIA_ISSUES.md` 2026-08-05. +**Verified live:** corrupted a real client's Authorization header, made a +real API call, confirmed exactly one auto-retry occurred and the call +succeeded transparently with a freshly-obtained token. -`_async_link_elements_as_peer_duplicates` now builds the URL from -`f"{self.classification_command_root}/related-elements/{element_guid}/peer-duplicate/{peer_duplicate_guid}/attach"` -— verified directly against the running `quickstart-pyegeria-web` -container's installed pyegeria source. Originally confirmed as a -client-side bug 2026-07-16, while seeding demo data for -egeria-workspaces-fs's Duplicate Resolution Review pane -(`duplicate_review_handler.py`). +--- -**Root cause:** the client posted to -`.../classification-explorer/elements/{elementGUID}/peer-duplicate/{peerDuplicateGUID}/attach` -but the real Spring endpoint -(`ClassificationExplorerResource.linkElementsAsPeerDuplicates`) is mapped -at `.../classification-explorer/related-elements/{elementGUID}/peer-duplicate/{peerDuplicateGUID}/attach` -— `elements` vs `related-elements`. Same root cause likely affected -`unlink_elements_as_peer_duplicates` (detach, same path shape). -Classification calls (`set_known_duplicate_classification`/ -`set_consolidated_duplicate_classification`) and the read path -(`get_relationships`/`get_elements_by_classification`) were unaffected. +### ISSUE-6 (GAP-1): `PyegeriaAPIException` was a catch-all; typed subclasses existed but weren't raised + +**Status:** fixed 2026-07-31. `_async_make_request` in +`_base_server_client.py` now branches on `related_http_code` and raises +`PyegeriaNotFoundException` (404) / `PyegeriaUnauthorizedException` +(401/403) instead of the generic `PyegeriaAPIException`. Both exception +classes re-parented from `PyegeriaException` to subclass +`PyegeriaAPIException` (with `__init__` calling `PyegeriaException.__init__` +directly, bypassing `PyegeriaAPIException.__init__`'s `response.json()` +call, since these two are now also raised for bare HTTP error responses +with no JSON body — see ISSUE-7) so all 63 existing +`except PyegeriaAPIException` handlers found across the codebase keep +working unchanged. -**Fix:** `pyegeria/omvs/classification_explorer.py`, -`_async_link_elements_as_peer_duplicates` (and its detach twin) now build -the URL from the `related-elements` path. +**Regression caught and fixed post-merge:** bypassing +`PyegeriaAPIException.__init__` also skipped the line that sets +`self.related_http_code`, which existing code relies on +(`except PyegeriaAPIException as e: if e.related_http_code == 404`). Fixed +by setting it explicitly in both subclasses' `__init__`. + +```python +from pyegeria import EgeriaTech +from pyegeria.core._exceptions import PyegeriaNotFoundException +client = EgeriaTech(view_server="qs-view-server", platform_url="https://localhost:9443", + user_id="erinoverview", user_pwd="secret") +client.create_egeria_bearer_token() +try: + client.get_digital_product_by_guid("00000000-0000-0000-0000-000000000000") +except PyegeriaNotFoundException as e: + print(e.related_http_code) # 404 +``` --- -### ISSUE-42 (PY-22): `ProjectManager.get_linked_projects()` didn't surface real `ProjectHierarchy` relationships +### ISSUE-5 (PY-9): local `as_of_time` fixes not shipped to the deployed package -**Status: FIXED** (2026-08-05, `pyegeria/core/_server_client.py`). +**Status:** fixed / moot, re-verified 2026-07-31 — `get_linked_projects`, +`get_collection_members`, and `get_data_field_by_guid` all accept +`as_of_time` (directly or via `**kwargs`) in current source. This was a +deployment-timing issue, not a code defect — see ISSUE-12, below. -**Layer:** Pyegeria (shared response-parsing helper, not the Egeria -server). Originally found 2026-07-31 while finishing an `as_of_time` -verification pass; consolidated in from `egeria-workspaces-fs/ -PYEGERIA_ISSUES.md` 2026-08-05 as open, then investigated and fixed the -same day. +--- -**Root cause:** `get_linked_projects` delegates to the shared -`ServerClient._async_get_guid_request` helper (used by 40+ callers across -the OMVS classes), which parsed a response by checking only two keys — -singular `"element"`, then `"elementGraph"` — before giving up and -returning `NO_ELEMENTS_FOUND`. But the real Egeria response for this -endpoint (`.../project-manager/metadata-elements/{guid}/projects`) returns -a genuine list under the **plural** `"elements"` key, which was never -checked at all. Confirmed live via a request-spy on `_async_make_request`: +### ISSUE-4 (PY-7/8/11): `as_of_time` missing or silently dropped on several find methods -``` -RAW RESPONSE KEYS: ['class', 'requestId', 'relatedHTTPCode', 'elements'] -get_linked_projects("5d0057f6-...") -> "No elements found" # wrong -# but the raw body's "elements" list genuinely had 1 item -``` +**Status:** fixed, re-verified 2026-07-31 — `find_information_supply_chains`, +`find_governance_definitions`, `find_note_logs`, `find_collections`, +`find_data_structures`, and `get_technology_type_elements` all route +through shared helpers (`_async_find_request` / `_async_get_name_request`) +that take `as_of_time` as an explicit named parameter or splat `**kwargs` +into the validated request body. -**Confirmed not a data-availability problem** (as originally suspected -when this was found): checked all 29 qs demo projects — every single one -returned `"No elements found"` regardless of whether it had real linked -projects (confirmed via `get_project_by_guid`'s own `managedProjects` -field, which does surface the same `ProjectHierarchy` relationships -correctly through a different code path). +`get_valid_metadata_values` was reclassified, not a pyegeria fix — see +ISSUE-11 (Egeria Server section): the underlying REST endpoint is a plain +GET that never accepted `asOfTime` in the first place. -**Fix:** added the plural `"elements"` key as a third fallback in -`_async_get_guid_request`, checked *last* (only reached once both -`"element"` and `"elementGraph"` have already failed) — purely additive, -so the other 40+ callers whose endpoints genuinely return the singular -shape are unaffected; none of their existing behavior changes since they -never reach the new branch. +Originally (surfaced during a "time travel" audit): all of the following +accepted `as_of_time` either not at all, or silently swallowed it because +the method built its request body from explicit named parameters instead +of forwarding kwargs — `find_information_supply_chains`, +`find_governance_definitions`, `find_note_logs`, `find_collections`, +`find_data_structures`, `get_technology_type_elements`. -**Verified:** -- Live against `qs-view-server`: `get_linked_projects("5d0057f6-...")` (no - body, matching the original repro exactly) now returns a real 1-element - list (`"Sustainability Bootstrap Project"`); a second project - ("Next 30 Days (Initiation Phase)") returns 11 linked projects — matches - its 10 `managedProjects` (children) + 1 `managingProjects` (parent), - consistent with the method's own docstring ("any relationship will do"). -- New unit tests, `tests/micro-tests/test_get_guid_request.py` (4 tests, - mocked `_async_make_request`): the plural-`elements` case now parses - correctly; the singular-`element` and `elementGraph` cases are - unaffected (still checked first); the true-empty case still degrades to - `NO_ELEMENTS_FOUND`, not an exception. -- Full micro-test suite: 228 passed (was 224; +4 new), same single - pre-existing unrelated failure (`test_dashboard_sheet_models.py`) as - before this change — no regressions across the other 40+ callers of the - shared helper. +```python +from pyegeria import EgeriaTech +client = EgeriaTech(view_server="qs-view-server", platform_url="https://localhost:9443", + user_id="erinoverview", user_pwd="secret") +client.create_egeria_bearer_token() +r1 = client.find_note_logs("*", graph_query_depth=0) +r2 = client.find_note_logs("*", graph_query_depth=0, as_of_time="2020-01-01T00:00:00Z") +# Fixed: r2 now genuinely reflects historical state instead of being identical to r1. +``` --- -### ISSUE-34: `MetadataExpert.find_metadata_elements` ignored `start_from`/`page_size` — traced through THREE fix attempts before landing on the actual root cause (a moved Egeria convention), and a design change to prevent the whole bug class recurring +### ISSUE-3 (PY-5): `get_notes_for_note_log` broken on pyegeria 6.0.14.4/.5 -**Status: FIXED** (final fix 2026-08-05, `pyegeria/omvs/metadata_expert.py` + -`pyegeria/view/overview_metrics.py` + tests). Worth reading in full even -though it's closed — the misdiagnosis in the middle is as instructive as -the fix. +**Status:** fixed, re-verified 2026-07-31 (live). The default +`metadata_element_type_name="Action"` in `_async_get_notes_for_note_log` +(`_server_client.py`) looks superficially like it could still trigger the +originally-reported 404, but a live call against a real note log guid +(`901aab3a-e6f9-4bd1-8899-f013150fbcd4`) returned the correct note list (1 +note, no error) — confirmed by direct test, not just by reading the code. -**Attempt 1 (dropped parameter).** `_async_find_metadata_elements` -originally built its request with a bare URL and the caller's `body` -passed straight through unmodified — `start_from`/`page_size`/ -`graph_query_depth` were silently dropped, never merged into `body`, never -appended to `url`. Root-caused against the `.http` ground truth -(`Egeria-api-metadata-expert.http`'s worked examples): `graphQueryDepth` -**is** a body field there; `startFrom`/`pageSize` are not shown in the body -at all, so the fix assumed they were URL query parameters (matching the -older, pre-migration Egeria convention this same file's `.http` examples -still document) — appended `?startFrom={..}&pageSize={..}` to the URL and -merged `graphQueryDepth` into `body` when not already present. Verified via -runtime capture that the values reached the wire correctly. +Originally on 6.0.14.4/.5: the default caused +`OMAG-REPOSITORY-HANDLER-404-001` ("guid is of type NoteLog rather than +Action"); passing `metadata_element_type_name="NoteLog"` returned the log +itself, not its notes. Fixed in 6.0.14.6+. -**Attempt 2 (misdiagnosed as an Egeria-server bug).** Re-verified live -against `qs-view-server` and found real pagination still didn't work — -`start_from=0`/`start_from=5` (both `page_size=5`) returned the identical -full ~1,837-element population both times, 100% GUID overlap. Since the -*client* was confirmed (via request-spy) to be sending the spec-correct -URL, this was logged as an open **Egeria-server-side** gap — "the server -ignores these query parameters regardless of client version." **This -conclusion was wrong.** +**Regression risk carried forward:** a `pip install pyegeria --upgrade` +that lands on exactly 6.0.14.4/.5 (a yanked/reordered PyPI point release) +would resurface this. Floor-pin `pyegeria>=6.0.14.6` in any deployment that +installs from PyPI rather than an editable local checkout. Also: do NOT +pass `metadata_element_type_name="NoteLog"` — it returns 0 notes even on +fixed versions. -**Attempt 3 (the actual root cause, found by the user).** Egeria's -pagination convention for this endpoint changed a few months before this -investigation — from URL query parameters to **request-body fields**. The -`.http` contract files pyegeria's fix was root-caused against were -themselves stale for this specific endpoint. Confirmed empirically: ```python -body1 = {..., "startFrom": 0, "pageSize": 5} -body2 = {..., "startFrom": 5, "pageSize": 5} -# len(r1) == len(r2) == 5 (page_size genuinely respected, not the full 1837) -# set(guids in r1) & set(guids in r2) == set() -- two real, distinct pages +from pyegeria import EgeriaTech +client = EgeriaTech(view_server="qs-view-server", platform_url="https://localhost:9443", + user_id="erinoverview", user_pwd="secret") +client.create_egeria_bearer_token() +client.get_notes_for_note_log(note_log_guid="") ``` -There is no Egeria-server-side bug here at all — pagination works -correctly and always has; pyegeria was sending the right values to the -wrong location, and "the server ignores it" was the visible symptom of -that, not evidence the server itself was broken. - -**Final fix — a design change, not just a relocated injection.** Rather -than moving the URL-based injection into a body-based injection (which -would just be the same "pyegeria guesses where pagination goes for this -endpoint" model, now proven to go stale at least once already), -`start_from`/`page_size` were **removed from the method signature -entirely** — same treatment `graph_query_depth` had already gotten a few -hours earlier in this same investigation, for the same underlying reason -(pushback from the user: "I think that they should have full control over -the body and not have me inject anything in it"). For endpoints where the -caller already constructs the full `body` themselves, a second parameter -channel for the same information is exactly the shape that goes stale — -this is the second time in one investigation (dropped parameter, then -wrong location) that pyegeria's own tracking of "where does this value -belong" broke a caller. The caller already has to know Egeria's current -contract to build a correct body; they're better positioned to keep -`startFrom`/`pageSize` current for a given endpoint than pyegeria's -hardcoded assumption is. `body` is now sent to `_async_make_request` -completely unmodified, in every code path, for both the async and sync -forms of this method. - -**Design principle adopted from this investigation** (see also ISSUE-43, -an audit for the same drift shape elsewhere): for any method where the -caller is required to supply the complete request body, do not *also* -accept overlapping parameters (`start_from`, `page_size`, -`graph_query_depth`, or similar) that pyegeria tries to inject somewhere — -document where the field belongs in the body and let the caller put it -there. This does **not** apply to convenience methods that build their own -body internally from named kwargs (most of the OMVS-specific `find_*`/ -`get_*` wrappers) — there, pyegeria owns body construction either way, so -there's no second channel to remove. -**Callers updated to the new signature:** -`pyegeria/view/overview_metrics.py`'s `_find()` (production code — now -puts `"startFrom": 0, "pageSize": page_size` directly in the body dict -alongside the already-updated `"graphQueryDepth": 0`) and 5 call sites -across `tests/functional-tests/test_metadata_expert.py` / -`test_metadata_explorer.py` (previously passed `page_size=`/`start_from=` -as kwargs; now set `"pageSize"`/`"startFrom"` in the body dict itself). A -stale `PY-15` cross-reference in `test_metadata_expert.py` (predating this -file's 2026-08-05 consolidation) was also corrected to `ISSUE-35 (PY-15)` -while in there. +--- -**Verified:** live against `qs-view-server`, two real distinct 5-element -pages via the *public* `find_metadata_elements(body)` call (no separate -pagination parameters at all) — confirms the fix works through the whole -call chain, not just the internal request construction. Full -`tests/micro-tests` suite: 228 passed, same one pre-existing unrelated -failure as throughout this session, no regressions. +### ISSUE-2 (PY-2): `get_data_value_specifications_by_name("*")` rejected the wildcard -**Follow-up cleanup, same day:** `output_format`, `for_lineage`, -`for_duplicate_processing`, and `**kwargs` were also removed from both -`_async_find_metadata_elements` and `find_metadata_elements` — none of the -four were ever referenced anywhere in either method's body (confirmed by -reading the full implementation), so declaring them implied controls that -never existed, the same "extraneous parameter" shape as `start_from`/ -`page_size`/`graph_query_depth` above. Final signature for both: `(self, -body: dict, timeout: int = default_timeout)` — nothing else. Verified: live -call still succeeds (a `pageSize: 1` body field correctly limits to 1 -result), full `tests/micro-tests` suite still clean. +**Status:** fixed, re-verified 2026-07-31 — the shared +`_async_get_name_request` helper (`_server_client.py`) still correctly maps +`filter_string = None if filter_string == "*" else filter_string`, aligned +with every sibling helper's "any" sentinel handling. -**This raises the severity of the note below** — removing `**kwargs` -specifically means a caller still passing `start_from=`/`page_size=`/ -`graph_query_depth=` as keyword arguments no longer gets silently ignored, -it now gets a hard `TypeError: find_metadata_elements() got an unexpected -keyword argument`. Worth knowing before upgrading pyegeria in any consumer -that hasn't been updated yet. +Root cause was the `"*"` convenience sentinel being sent to Egeria as the +literal string `".*"` instead of `None` — Egeria's by-name `filter` field is +not a regex, so `".*"` either matched nothing or matched literally. -**Not carried over from pyegeria's fix:** `egeria-workspaces-fs/ -insights_handler.py`'s `search_elements()` still calls `find_metadata_elements` -with `start_from=`/`page_size=`/`graph_query_depth=` as keyword arguments — -as of the follow-up cleanup above, this will now **raise `TypeError` on -every call** rather than silently no-op (see previous paragraph; this is a -stronger warning than when this entry was first written). That repo's own -call site needs a follow-up update to put `startFrom`/`pageSize` in its own -`find_body` dict to get real, efficient pagination — flagged, not fixed -here (different repo), but now genuinely blocking rather than just a -missed optimization. +```python +from pyegeria import DataDesigner +mgr = DataDesigner(view_server="qs-view-server", platform_url="https://localhost:9443", + user_id="erinoverview", user_pwd="secret") +mgr.create_egeria_bearer_token() +mgr.get_data_value_specifications_by_name("*") +``` --- -### ISSUE-5 (PY-9): local `as_of_time` fixes not shipped to the deployed package +### ISSUE-1 (PY-1): `DataDesigner.find_data_value_specifications` called non-existent `_async_post` + +**Status:** fixed, re-verified 2026-07-31 — `_async_post` no longer exists +anywhere in the codebase (`grep -rn "_async_post\b" pyegeria/` returns +nothing); `find_data_value_specifications` now routes through the shared +`_async_find_request` helper. -**Status:** fixed / moot, re-verified 2026-07-31 — `get_linked_projects`, -`get_collection_members`, and `get_data_field_by_guid` all accept -`as_of_time` (directly or via `**kwargs`) in current source. This was a -deployment-timing issue, not a code defect — see ISSUE-12, below. +```python +from pyegeria import DataDesigner +mgr = DataDesigner(view_server="qs-view-server", platform_url="https://localhost:9443", + user_id="erinoverview", user_pwd="secret") +mgr.create_egeria_bearer_token() +mgr.find_data_value_specifications(search_string="*") +``` --- ## Not a bug / n/a +--- + ### ISSUE-120: Textual apps (My Profile, MyEgeria) crash on Python 3.14 with `assert task is not None` in `textual/rlock.py` **Layer:** not pyegeria (Textual / Python 3.14) · **Status:** n/a, workaround: use @@ -8730,6 +9091,8 @@ as working until someone tries to browse by zone. correction above it's solving a problem `create_governance_definition` already handles more simply — worth switching to that instead. +--- + ### ISSUE-65: `GlossaryManager.get_term_by_guid`'s naming-standards classifications (PrimeWord/ClassWord/Modifier) don't surface as individually-named `elementHeader` keys like most classifications do — undocumented response-shape gotcha, not a data-missing bug **Status:** n/a — not a bug, a response-shape gotcha worth documenting so @@ -8880,69 +9243,91 @@ qualified names. --- -### ISSUE-11 (PY-3): `find_all_solution_blueprints`/`find_all_solution_components` missing in 6.0.12.2 - -**Status:** n/a — re-verified 2026-07-31, both methods exist in current -source. Was a version-pinning issue at the time (container pinned to -6.0.12.2, methods added in 6.0.12.4), not a code defect. - ---- +### ISSUE-37 (PY-17): `MetadataExpert.get_metadata_element_by_guid` never returns relationships, at any `graph_query_depth` — use `get_all_related_elements` instead -### ISSUE-12 (PY-9): local `as_of_time` fixes not shipped to a deployed container +**Status: not a bug — working as designed** (confirmed 2026-07-17, Dan). +Consolidated in from `egeria-workspaces-fs/PYEGERIA_ISSUES.md` 2026-08-05. -**Status:** n/a — same root cause class as ISSUE-11. The fix landed in -source; the failure mode was a deployed container `pip install`-ing an -older released version instead of using an editable local checkout. Cut a -release / mount the local checkout as editable for dev+test to avoid this -recurring. +`get_metadata_element_by_guid` is deliberately scoped to the element +itself; `get_all_related_elements` is the correct, separate call for +relationships — a two-call design, not a gap in the by-guid method. +Originally flagged 2026-07-16 while fixing egeria-workspaces-fs's Action +Center pane cross-links (`action_center_handler.py`), because the accepted +`graph_query_depth` parameter suggested it should affect relationship +inclusion; it doesn't, and that's correct behavior. ---- +**How to trigger:** +```python +guid = "" +for depth in (0, 1, 2, 3): + el = mgr.get_metadata_element_by_guid(guid, graph_query_depth=depth, output_format="JSON") + print(depth, sorted(el.keys())) +# every depth prints the same 8 keys -- no relationship key ever appears: +# ['classifications', 'elementGUID', 'elementProperties', 'headerVersion', +# 'origin', 'status', 'type', 'versions'] +``` -### ISSUE-13 (GAP-4): `pyegeria.core.mcp_server` pinned to a stale `mcp` package API +**Working alternative:** `MetadataExpert.get_all_related_elements(guid, output_format="JSON")` +returns `{"startingElement": , "elementList": [...], "mermaidGraph": ...}`. +Each `elementList` entry is a relationship-header dict with its own +`type.typeName` (the *relationship* type) and a nested `element` key +holding the *other* end, in the same raw `elementGUID`/ +`elementProperties.propertyValueMap` shape everything else uses. Confirmed +live against a real Notification with 3 genuine relationships — all three +showed up via this call, none via `get_metadata_element_by_guid` at any +depth. -**Status:** n/a — re-verified 2026-07-31. `pyegeria/core/mcp_server.py` at -HEAD already imports `from mcp.server.mcpserver import MCPServer` (the -current API) and imports/runs cleanly with `mcp==2.0.0` installed in this -repo's venv (confirmed: -`python3 -c "from pyegeria.core import mcp_server"` succeeds). The -originally-reported `ModuleNotFoundError` was observed against a -**different, stale installed pyegeria wheel** inside the separate -`quickstart-pyegeria-web` container, not this dev tree — a -deployment/versioning gap in that other project, not a fix needed here. +**Usage note (not a fix, a caller guideline):** any by-guid detail call +that needs relationships should use `get_all_related_elements` (or a +type-specific OMVS method that already merges both, e.g. +`ActorManager.get_actor_role_by_guid`), never +`get_metadata_element_by_guid` alone, plus `graph_query_depth` as if it +controlled relationship inclusion. --- -### ISSUE-15 (PY-6, PY-14): `find_*`/list-style methods are O(n × graph-computation-cost) at default `graph_query_depth` +### ISSUE-29: Information Supply Chain search results/mermaid graphs differ by user identity — governance zone visibility, not a pyegeria bug -**Status:** n/a / reclassified as expected Egeria behavior — not a defect. -`graph_query_depth` controls how much relationship graph the server -computes per returned element; defaulting to depth 3 is a deliberate -trade-off (rich results by default). Confirmed on both `find_note_logs` -(PY-6, ~30-70s at default depth on demo data with large logs) and -`find_specification_property` (PY-14, ~50s vs ~0.6-2s at depth 0 for a -1000-element result set) — same root cause, two different methods. +**Status:** n/a (Egeria Server — deliberate zone-based visibility +filtering), found 2026-08-04 chasing down the last piece of ISSUE-25's +original repro. -**Workaround, already in use:** always pass `graph_query_depth=0` on any -bulk-listing call unless the caller genuinely needs the graph/mermaid -output — it's accepted via `**kwargs` even on methods that don't list it in -their signature. +**Layer:** Egeria Server (governance zone / visible-zones configuration). -**Possible future Egeria-side improvement** (not a bug, a suggestion): -default `graph_query_depth` to `0` for list-style methods (opt-in to the -expensive graph rather than opt-out), or split "list" and "graph" into -separate REST operations. +**What:** the exact same `find_information_supply_chains` search, run at +the same instant against the same view server, returns materially +different data depending on which user's credentials are used. Live- +verified side-by-side for "New Employee Onboarding Information Supply +Chain": +- `peterprofile`: implementation graph shows 5 `SoftwareServer`s + (`coco-hrim`, `NL payroll`, `CA payroll`, `UK payroll`, `ocopages`) +- `erinoverview`: implementation graph shows only 2 (`coco-hrim`, + `ocopages`) — the 3 payroll systems are filtered out entirely ---- +Confirmed via raw `curl` against `/api/token` + the `by-search-string` +endpoint directly, with zero pyegeria code involved — same request body, +same moment, different user, different result. Not a caching artifact +(re-ran both back-to-back) and not explained by anything on the client +side. -### ISSUE-16 (PY-10): asset detail by-guid appeared to "reject" `asOfTime` +**Where seen:** originally surfaced as part of ISSUE-25's repro +investigation — initially looked like a "3 nodes vs 5 nodes" pyegeria bug, +but turned out to be two unrelated effects layered together: a +node-counting-convention difference (see ISSUE-25's postscript, resolved, +not a bug) plus this genuine zone-visibility difference (this entry). -**Status:** n/a — investigated and confirmed not a defect. -`get_asset_graph_by_guid`/`get_asset_by_guid` do honor `asOfTime` correctly; -the original 404/500 reports were caused by test timestamps that predated -the entity's repository version after demo data was reloaded — Egeria -correctly reports "not found at that time." The graph endpoint returning -500 instead of 404 for this case is a minor rough edge, not the originally -reported bug. +**Why not a bug:** pyegeria faithfully returns whatever the server sends +for the credentials it's given. The 3 payroll systems are presumably +scoped to a governance zone `erinoverview`/`IvorPadlock` aren't members of, +while `peterprofile` (likely a broader-access/admin-style demo user) can +see them. This is Egeria's zone-based security model working as designed, +not a pyegeria defect — nothing here is client-side fixable. + +**Worth knowing:** if comparing pyegeria output against a raw REST call or +Egeria Explorer UI result and results don't match, check whether the same +user/credentials were used in both before assuming a client-side bug — +this cost real investigation time in ISSUE-25's process. Added a pointer +to this in the Quick Reference table above and in this file's intro. --- @@ -8974,91 +9359,69 @@ an Egeria REST API change to add `asOfTime` support to this endpoint first. --- -### ISSUE-29: Information Supply Chain search results/mermaid graphs differ by user identity — governance zone visibility, not a pyegeria bug - -**Status:** n/a (Egeria Server — deliberate zone-based visibility -filtering), found 2026-08-04 chasing down the last piece of ISSUE-25's -original repro. +### ISSUE-16 (PY-10): asset detail by-guid appeared to "reject" `asOfTime` -**Layer:** Egeria Server (governance zone / visible-zones configuration). +**Status:** n/a — investigated and confirmed not a defect. +`get_asset_graph_by_guid`/`get_asset_by_guid` do honor `asOfTime` correctly; +the original 404/500 reports were caused by test timestamps that predated +the entity's repository version after demo data was reloaded — Egeria +correctly reports "not found at that time." The graph endpoint returning +500 instead of 404 for this case is a minor rough edge, not the originally +reported bug. -**What:** the exact same `find_information_supply_chains` search, run at -the same instant against the same view server, returns materially -different data depending on which user's credentials are used. Live- -verified side-by-side for "New Employee Onboarding Information Supply -Chain": -- `peterprofile`: implementation graph shows 5 `SoftwareServer`s - (`coco-hrim`, `NL payroll`, `CA payroll`, `UK payroll`, `ocopages`) -- `erinoverview`: implementation graph shows only 2 (`coco-hrim`, - `ocopages`) — the 3 payroll systems are filtered out entirely +--- -Confirmed via raw `curl` against `/api/token` + the `by-search-string` -endpoint directly, with zero pyegeria code involved — same request body, -same moment, different user, different result. Not a caching artifact -(re-ran both back-to-back) and not explained by anything on the client -side. +### ISSUE-15 (PY-6, PY-14): `find_*`/list-style methods are O(n × graph-computation-cost) at default `graph_query_depth` -**Where seen:** originally surfaced as part of ISSUE-25's repro -investigation — initially looked like a "3 nodes vs 5 nodes" pyegeria bug, -but turned out to be two unrelated effects layered together: a -node-counting-convention difference (see ISSUE-25's postscript, resolved, -not a bug) plus this genuine zone-visibility difference (this entry). +**Status:** n/a / reclassified as expected Egeria behavior — not a defect. +`graph_query_depth` controls how much relationship graph the server +computes per returned element; defaulting to depth 3 is a deliberate +trade-off (rich results by default). Confirmed on both `find_note_logs` +(PY-6, ~30-70s at default depth on demo data with large logs) and +`find_specification_property` (PY-14, ~50s vs ~0.6-2s at depth 0 for a +1000-element result set) — same root cause, two different methods. -**Why not a bug:** pyegeria faithfully returns whatever the server sends -for the credentials it's given. The 3 payroll systems are presumably -scoped to a governance zone `erinoverview`/`IvorPadlock` aren't members of, -while `peterprofile` (likely a broader-access/admin-style demo user) can -see them. This is Egeria's zone-based security model working as designed, -not a pyegeria defect — nothing here is client-side fixable. +**Workaround, already in use:** always pass `graph_query_depth=0` on any +bulk-listing call unless the caller genuinely needs the graph/mermaid +output — it's accepted via `**kwargs` even on methods that don't list it in +their signature. -**Worth knowing:** if comparing pyegeria output against a raw REST call or -Egeria Explorer UI result and results don't match, check whether the same -user/credentials were used in both before assuming a client-side bug — -this cost real investigation time in ISSUE-25's process. Added a pointer -to this in the Quick Reference table above and in this file's intro. +**Possible future Egeria-side improvement** (not a bug, a suggestion): +default `graph_query_depth` to `0` for list-style methods (opt-in to the +expensive graph rather than opt-out), or split "list" and "graph" into +separate REST operations. --- -### ISSUE-37 (PY-17): `MetadataExpert.get_metadata_element_by_guid` never returns relationships, at any `graph_query_depth` — use `get_all_related_elements` instead +### ISSUE-13 (GAP-4): `pyegeria.core.mcp_server` pinned to a stale `mcp` package API -**Status: not a bug — working as designed** (confirmed 2026-07-17, Dan). -Consolidated in from `egeria-workspaces-fs/PYEGERIA_ISSUES.md` 2026-08-05. +**Status:** n/a — re-verified 2026-07-31. `pyegeria/core/mcp_server.py` at +HEAD already imports `from mcp.server.mcpserver import MCPServer` (the +current API) and imports/runs cleanly with `mcp==2.0.0` installed in this +repo's venv (confirmed: +`python3 -c "from pyegeria.core import mcp_server"` succeeds). The +originally-reported `ModuleNotFoundError` was observed against a +**different, stale installed pyegeria wheel** inside the separate +`quickstart-pyegeria-web` container, not this dev tree — a +deployment/versioning gap in that other project, not a fix needed here. -`get_metadata_element_by_guid` is deliberately scoped to the element -itself; `get_all_related_elements` is the correct, separate call for -relationships — a two-call design, not a gap in the by-guid method. -Originally flagged 2026-07-16 while fixing egeria-workspaces-fs's Action -Center pane cross-links (`action_center_handler.py`), because the accepted -`graph_query_depth` parameter suggested it should affect relationship -inclusion; it doesn't, and that's correct behavior. +--- -**How to trigger:** -```python -guid = "" -for depth in (0, 1, 2, 3): - el = mgr.get_metadata_element_by_guid(guid, graph_query_depth=depth, output_format="JSON") - print(depth, sorted(el.keys())) -# every depth prints the same 8 keys -- no relationship key ever appears: -# ['classifications', 'elementGUID', 'elementProperties', 'headerVersion', -# 'origin', 'status', 'type', 'versions'] -``` +### ISSUE-12 (PY-9): local `as_of_time` fixes not shipped to a deployed container -**Working alternative:** `MetadataExpert.get_all_related_elements(guid, output_format="JSON")` -returns `{"startingElement": , "elementList": [...], "mermaidGraph": ...}`. -Each `elementList` entry is a relationship-header dict with its own -`type.typeName` (the *relationship* type) and a nested `element` key -holding the *other* end, in the same raw `elementGUID`/ -`elementProperties.propertyValueMap` shape everything else uses. Confirmed -live against a real Notification with 3 genuine relationships — all three -showed up via this call, none via `get_metadata_element_by_guid` at any -depth. +**Status:** n/a — same root cause class as ISSUE-11. The fix landed in +source; the failure mode was a deployed container `pip install`-ing an +older released version instead of using an editable local checkout. Cut a +release / mount the local checkout as editable for dev+test to avoid this +recurring. -**Usage note (not a fix, a caller guideline):** any by-guid detail call -that needs relationships should use `get_all_related_elements` (or a -type-specific OMVS method that already merges both, e.g. -`ActorManager.get_actor_role_by_guid`), never -`get_metadata_element_by_guid` alone, plus `graph_query_depth` as if it -controlled relationship inclusion. +--- + +### ISSUE-11 (PY-3): `find_all_solution_blueprints`/`find_all_solution_components` missing in 6.0.12.2 + +**Status:** n/a — re-verified 2026-07-31, both methods exist in current +source. Was a version-pinning issue at the time (container pinned to +6.0.12.2, methods added in 6.0.12.4), not a code defect. --- diff --git a/commands/cat/glossary_actions.py b/commands/cat/glossary_actions.py index fb5f21cf..a8245ed0 100644 --- a/commands/cat/glossary_actions.py +++ b/commands/cat/glossary_actions.py @@ -127,7 +127,7 @@ def create_term(server, url, userid, password, glossary_name, term_name, summary "anchorScopeGUID" : glossary_guid, "isOwnAnchor": True, "parentAtEnd1": True, - "elementProperties": { + "properties": { "class": "GlossaryTermProperties", "qualifiedName": qualified_name, "displayName": term_name, @@ -137,8 +137,8 @@ def create_term(server, url, userid, password, glossary_name, term_name, summary "examples": examples, "usage": usage, "publishVersionIdentifier": version, + "contentStatus": status, }, - "initialStatus": status, } term_guid = m_client.create_glossary_term(body_slimmer(body) diff --git a/commands/tech/gen_report_specs.py b/commands/tech/gen_report_specs.py index 87bc50ec..403d9dac 100644 --- a/commands/tech/gen_report_specs.py +++ b/commands/tech/gen_report_specs.py @@ -379,20 +379,17 @@ def build_format_sets_from_commands( extra_constraints = _safe_parse_constraints(cmd_obj.get("extra_constraints")) action = None if find_method or extra_find: - # `function` is a required (non-Optional) field on ActionParameter, - # but a command can legitimately have only an analytic_function and - # no find_method at all (e.g. a dedicated "View Growth"-style command - # with no element-listing path) -- an empty string is a valid, if - # unused, value; exec_report_spec's SERIES path never reads it. + # A command can have only an analytic_function and no find_method + # (e.g. a "View Growth"-style command with no element-listing path); + # `function` is then left unset (ISSUE-141). action = ActionParameter( - function=find_method or "", + function=find_method or None, required_params=["search_string"] if find_method else [], optional_params=OPTIONAL_SEARCH_PARAMS if find_method else [], spec_params=constraints or {}, + analytic_function=extra_find or None, + analytic_spec_params=(extra_constraints or {}) if extra_find else {}, ) - if extra_find: - action.analytic_function = extra_find - action.analytic_spec_params = extra_constraints or {} formats = _build_generated_formats(columns, default_types=default_types) if not formats: diff --git a/md_processing/md_processing_utils/common_md_utils.py b/md_processing/md_processing_utils/common_md_utils.py index 29389e79..b2effb01 100644 --- a/md_processing/md_processing_utils/common_md_utils.py +++ b/md_processing/md_processing_utils/common_md_utils.py @@ -14,6 +14,7 @@ from pyegeria.core.utils import (camel_to_title_case) from pyegeria.core._globals import DEBUG_LEVEL, GovernanceDomains, resolve_enum from md_processing.md_processing_utils.message_constants import message_types +from md_processing.md_processing_utils.element_status import element_status_properties # Constants EGERIA_METADATA_STORE = os.environ.get("EGERIA_METADATA_STORE", "active-metadata-store") @@ -303,7 +304,6 @@ def set_create_body(object_type: str, attributes: dict)->dict: "parentAtEnd1": attributes.get('Parent at End1', {}).get('value', True), "anchorScopeGUID": attributes.get('Anchor Scope ID', {}).get('guid', None), "properties": "", - "initialStatus": attributes.get('Status', {}).get('value', "ACTIVE"), "initialClassifications": {}} return body @@ -394,7 +394,7 @@ def set_element_prop_body(object_type: str, qualified_name: str, attributes: dic prop_name = _to_egeria_type_name(object_type) display_name = attributes.get('Display Name', {}).get('value', None) - return { + body = { "class": prop_name + "Properties", "typeName": prop_name, "displayName": attributes.get('Display Name', {}).get('value', None), @@ -404,7 +404,6 @@ def set_element_prop_body(object_type: str, qualified_name: str, attributes: dic "identifier": attributes.get('Identifier', {}).get('value', None), "url": attributes.get('URL', {}).get('value', None), "authors": attributes.get('Authors', {}).get('value', []), - "contentStatus": attributes.get('Content Status', {}).get('value', None) or attributes.get('Status', {}).get('value', None), "userDefinedContentStatus": attributes.get('User Defined Content Status', {}).get('value', None), "versionIdentifier": attributes.get('Version Identifier', {}).get('value', None), "effectiveFrom": attributes.get('Effective From', {}).get('value', None), @@ -415,6 +414,9 @@ def set_element_prop_body(object_type: str, qualified_name: str, attributes: dic "serviceLevels": attributes.get('Service Levels', {}).get('value', None), "legal": attributes.get('Legal', {}).get('value', None), } + # contentStatus / activityStatus / deploymentStatus, whichever this type has (ISSUE-113). + body.update(element_status_properties(body["class"], attributes)) + return body def set_collection_manager_body(object_type: str, qualified_name: str, attributes: dict) -> dict: """ diff --git a/md_processing/md_processing_utils/element_status.py b/md_processing/md_processing_utils/element_status.py new file mode 100644 index 00000000..d50d9394 --- /dev/null +++ b/md_processing/md_processing_utils/element_status.py @@ -0,0 +1,109 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright Contributors to the ODPi Egeria project. +""" +Map Dr.Egeria's status attributes onto the status property an element type actually has. + +Egeria no longer accepts `initialStatus` on new-element requests (removed 2025-12-15). +Each type carries its own kind of status in its properties instead: `contentStatus` +(authored content -- glossary terms, collections, data structures, ...), +`activityStatus` (actions and processes -- notes, to-dos, meetings, ...), +`deploymentStatus` (infrastructure, solution components, capabilities, ...), or +none at all (actors, plain assets, comments). See ISSUE-113. +""" +from __future__ import annotations + +from loguru import logger + +from md_processing.md_processing_utils.element_status_fields import STATUS_FIELDS_BY_PROPERTIES_CLASS + +# Dr.Egeria attribute -> Egeria property, for the type-specific status attributes. +EXPLICIT_STATUS_ATTRIBUTES = { + "Content Status": "contentStatus", + "Activity Status": "activityStatus", + "Deployment Status": "deploymentStatus", +} + +# Valid values of each status enum (ContentStatus, ActivityStatus, DeploymentStatus). +VALID_STATUS_VALUES = { + "contentStatus": {"DRAFT", "PREPARED", "PROPOSED", "APPROVED", "REJECTED", "ACTIVE", "DEPRECATED", + "OBSOLETE", "OTHER"}, + "activityStatus": {"REQUESTED", "APPROVED", "WAITING", "ACTIVATING", "IN_PROGRESS", "PAUSED", "FOR_INFO", + "COMPLETED", "INVALID", "IGNORED", "FAILED", "CANCELLED", "ABANDONED", "OTHER"}, + "deploymentStatus": {"PROPOSED", "UNDER_DEVELOPMENT", "DEVELOPMENT_COMPLETE", "APPROVED_FOR_DEPLOYMENT", + "REJECTED", "STANDBY", "ACTIVE", "DISABLED", "FAILED", "OTHER"}, +} + +# Where a written `Status` value fits more than one of a type's fields, the first wins. +_PREFERENCE = ("contentStatus", "activityStatus", "deploymentStatus") + + +def _value(attributes: dict, name: str): + value = (attributes.get(name) or {}).get("value") + return value if value not in (None, "") else None + + +def _is_default(attributes: dict, name: str) -> bool: + return bool((attributes.get(name) or {}).get("is_default")) + + +def element_status_properties(properties_class: str, attributes: dict) -> dict: + """Return the status properties to put in a `properties_class` body. + + Values the user wrote come first; values the parser filled in from spec + defaults only fill a property nothing written has set. + + 1. A written `Content Status` / `Activity Status` / `Deployment Status` is sent + when the type has that property, and dropped with a warning when it does not. + 2. A written `Status` goes to the first of the type's status properties not yet + set whose enum accepts the value. + 3. Defaulted type-specific attributes fill their property if it is still unset. + 4. A defaulted `Status` only ever fills `contentStatus`, as before, so a default + never starts overwriting an activity or deployment status (an upsert re-run + would otherwise reset it). + + A properties class this table does not know keeps the old behaviour: the + type-specific attributes pass through and `Status` goes to `contentStatus`. + """ + known = properties_class in STATUS_FIELDS_BY_PROPERTIES_CLASS + supported = STATUS_FIELDS_BY_PROPERTIES_CLASS.get(properties_class, ()) + has = (lambda field: field in supported) if known else (lambda field: True) + result: dict = {} + + def explicit(defaults: bool): + for attr, field in EXPLICIT_STATUS_ATTRIBUTES.items(): + value = _value(attributes, attr) + if value is None or _is_default(attributes, attr) != defaults or field in result: + continue + if not has(field): + if not defaults: + logger.warning(f"'{attr}' ignored: {properties_class} has no {field} property.") + continue + result[field] = value + + explicit(defaults=False) + + status = _value(attributes, "Status") + status_is_default = _is_default(attributes, "Status") + if status is not None and not status_is_default: + upper = str(status).strip().upper() + candidates = [f for f in _PREFERENCE if f in supported] if known else ["contentStatus"] + placed = False + for field in candidates: + if field not in result and upper in VALID_STATUS_VALUES[field]: + result[field] = upper + placed = True + break + if not placed and known: + if not supported: + logger.warning(f"'Status' ignored: {properties_class} has no status property.") + elif not any(f in result for f in candidates): + logger.warning(f"'Status' value '{status}' ignored: not valid for {properties_class}'s " + f"{' / '.join(supported)}.") + + explicit(defaults=True) + + if status is not None and status_is_default and "contentStatus" not in result and has("contentStatus"): + upper = str(status).strip().upper() + if upper in VALID_STATUS_VALUES["contentStatus"]: + result["contentStatus"] = upper + return result diff --git a/md_processing/md_processing_utils/element_status_fields.py b/md_processing/md_processing_utils/element_status_fields.py new file mode 100644 index 00000000..2a204d2d --- /dev/null +++ b/md_processing/md_processing_utils/element_status_fields.py @@ -0,0 +1,672 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright Contributors to the ODPi Egeria project. +""" +Status properties each Egeria Properties class has (ISSUE-113). + +GENERATED by scripts/gen_element_status_fields.py from Egeria source at +af400039c2 2026-10-06 -- do not edit by hand; re-run the script instead. A class with +no status property (actors, plain assets, comments, ...) maps to (). +""" + +STATUS_FIELDS_BY_PROPERTIES_CLASS: dict[str, tuple[str, ...]] = { + 'APIEndpointProperties': (), + 'APIHeaderProperties': (), + 'APIManagerProperties': ('deploymentStatus',), + 'APIOperationProperties': ('contentStatus',), + 'APIOperationsProperties': (), + 'APIParameterListProperties': ('contentStatus',), + 'APIParameterProperties': ('contentStatus',), + 'APIRequestProperties': (), + 'APIResponseProperties': (), + 'APISchemaTypeProperties': ('contentStatus',), + 'AbstractConceptProperties': (), + 'AcceptedAnswerProperties': (), + 'AccountingCodesProperties': (), + 'ActionProperties': ('activityStatus',), + 'ActionRequesterProperties': (), + 'ActionTargetProperties': ('activityStatus',), + 'ActionsProperties': (), + 'ActivityDescriptionProperties': (), + 'ActivityEntryProperties': ('activityStatus',), + 'ActorProfileProperties': (), + 'ActorProperties': (), + 'ActorRoleProperties': (), + 'AdjacentLocationProperties': (), + 'AgreementActorProperties': (), + 'AgreementItemProperties': (), + 'AgreementProperties': ('contentStatus',), + 'AnalyticalActionProcessProperties': ('contentStatus',), + 'AnalyticsEngineProperties': ('deploymentStatus',), + 'AnalyticsModelRunProperties': ('activityStatus',), + 'AnchorsProperties': (), + 'AnnotationExtensionProperties': (), + 'AnnotationMatchProperties': (), + 'AnnotationProperties': ('contentStatus',), + 'AnnotationReviewProperties': ('activityStatus',), + 'AntonymProperties': (), + 'ApplicationProperties': ('deploymentStatus',), + 'ApprovedPurposeProperties': (), + 'ArchiveContentsProperties': (), + 'ArchiveFileProperties': ('contentStatus',), + 'AssetConnectionProperties': (), + 'AssetProperties': (), + 'AssignmentScopeProperties': (), + 'AssociatedAnnotationProperties': (), + 'AssociatedLogProperties': (), + 'AssociatedSecurityListProperties': (), + 'AssociatedSkillSetProperties': (), + 'AttachedCommentProperties': (), + 'AttachedLikeProperties': (), + 'AttachedNoteLogEntryProperties': (), + 'AttachedNoteLogProperties': (), + 'AttachedRatingProperties': (), + 'AttachedStorageProperties': (), + 'AttachedTagProperties': (), + 'AttributeForSchemaProperties': (), + 'AudioFileProperties': ('contentStatus',), + 'AuditLogProperties': (), + 'AuthoredReferenceableProperties': ('contentStatus',), + 'AuthorizationManagerProperties': ('deploymentStatus',), + 'AvroFileProperties': ('contentStatus',), + 'BareMetalComputerProperties': ('deploymentStatus',), + 'BlogEntryProperties': ('activityStatus',), + 'BuildInstructionFileProperties': ('contentStatus',), + 'BusinessCapabilityDependencyProperties': (), + 'BusinessCapabilityProperties': ('contentStatus',), + 'BusinessImperativeProperties': ('contentStatus',), + 'BusinessSignificantProperties': (), + 'CSVFileProperties': ('contentStatus',), + 'CalculatedValueProperties': (), + 'CampaignProperties': (), + 'CanonicalVocabularyProperties': (), + 'CapabilityAssetUseProperties': (), + 'CatalogTargetProperties': (), + 'CatalogTemplateProperties': (), + 'CataloguingActionProcessProperties': ('contentStatus',), + 'CertificationProperties': (), + 'CertificationTypeProperties': ('contentStatus',), + 'ChangeManagementLibraryProperties': ('deploymentStatus',), + 'CitedDocumentLinkProperties': (), + 'CitedDocumentProperties': ('contentStatus',), + 'ClassWordProperties': (), + 'ClassificationAnnotationProperties': ('contentStatus',), + 'ClassificationBeanProperties': (), + 'ClassificationElementProperties': (), + 'CloudPlatformProperties': (), + 'CloudProviderProperties': (), + 'CloudServiceProperties': (), + 'CloudTenantProperties': (), + 'CodeAnalysisAnnotationProperties': ('contentStatus',), + 'CohortMemberProperties': ('deploymentStatus',), + 'CohortRegistryStoreProperties': ('contentStatus',), + 'CollectionFolderProperties': ('contentStatus',), + 'CollectionKindProperties': (), + 'CollectionMembershipProperties': (), + 'CollectionProperties': ('contentStatus',), + 'CommentProperties': (), + 'CommunityProperties': (), + 'ComplexSchemaTypeProperties': ('contentStatus',), + 'ConceptBeadAttributeLinkProperties': (), + 'ConceptBeadAttributeProperties': ('contentStatus',), + 'ConceptBeadExtensionProperties': (), + 'ConceptBeadProperties': ('contentStatus',), + 'ConceptBeadRelationshipEndProperties': (), + 'ConceptBeadRelationshipProperties': ('contentStatus',), + 'ConceptDesignProperties': (), + 'ConceptModelElementProperties': ('contentStatus',), + 'ConceptModelProperties': ('contentStatus',), + 'ConfidenceProperties': (), + 'ConfidentialityProperties': (), + 'ConnectToEndpointProperties': (), + 'ConnectionConnectorTypeProperties': (), + 'ConnectionProperties': (), + 'ConnectorActivityReportProperties': ('contentStatus',), + 'ConnectorTypeProperties': (), + 'ConsistentValidValuesProperties': (), + 'ConsolidatedDuplicateLinkProperties': (), + 'ConsolidatedDuplicateProperties': (), + 'ContactDetailsProperties': (), + 'ContactThroughProperties': (), + 'ContentCollectionManagerProperties': ('deploymentStatus',), + 'ContextDefinitionProperties': (), + 'ContextEventCollectionProperties': ('contentStatus',), + 'ContextEventEvidenceProperties': (), + 'ContextEventForTimelineEffectsProperties': (), + 'ContextEventImpactProperties': (), + 'ContextEventProperties': (), + 'ContractLinkProperties': (), + 'ContributionProperties': (), + 'ContributionRecordProperties': (), + 'ContributorAnalysisAnnotationProperties': ('contentStatus',), + 'ControlFlowProperties': (), + 'ControlPointProperties': (), + 'CriticalityProperties': (), + 'CrowdSourcingContributionProperties': (), + 'CyberLocationProperties': (), + 'DataAccessManagerProperties': ('deploymentStatus',), + 'DataAssetEncodingProperties': (), + 'DataAssetProperties': ('contentStatus',), + 'DataClassAnnotationProperties': ('contentStatus',), + 'DataClassCompositionProperties': (), + 'DataClassProperties': ('contentStatus',), + 'DataDescriptionProperties': (), + 'DataDictionaryProperties': ('contentStatus',), + 'DataFeedProperties': ('contentStatus',), + 'DataFieldAnnotationProperties': ('contentStatus',), + 'DataFieldProperties': ('contentStatus',), + 'DataFileCollectionProperties': ('contentStatus',), + 'DataFileProperties': ('contentStatus',), + 'DataFlowProperties': (), + 'DataFolderProperties': ('contentStatus',), + 'DataGrainAnnotationProperties': ('contentStatus',), + 'DataGrainProperties': ('contentStatus',), + 'DataLensProperties': ('contentStatus',), + 'DataLineageRelationshipProperties': (), + 'DataManagerProperties': ('deploymentStatus',), + 'DataMappingProperties': (), + 'DataMovementEngineProperties': ('deploymentStatus',), + 'DataProcessingActionProperties': ('contentStatus',), + 'DataProcessingDescriptionProperties': ('contentStatus',), + 'DataProcessingPurposeProperties': ('contentStatus',), + 'DataProcessingSpecificationProperties': (), + 'DataProcessingTargetProperties': (), + 'DataQualityRuleProperties': ('contentStatus',), + 'DataScopeProperties': (), + 'DataSetContentProperties': (), + 'DataSetProperties': ('contentStatus',), + 'DataSharingAgreementProperties': (), + 'DataSharingHubProperties': ('contentStatus',), + 'DataSharingRequestProperties': ('activityStatus',), + 'DataSpecProperties': ('contentStatus',), + 'DataStoreProperties': ('contentStatus',), + 'DataStructureDefinitionProperties': (), + 'DataStructureProperties': ('contentStatus',), + 'DataValueAssignmentProperties': (), + 'DataValueDefinitionProperties': (), + 'DataValueHierarchyProperties': (), + 'DataValueMeaningProperties': (), + 'DataValueSpecificationProperties': ('contentStatus',), + 'DataVirtualizationEngineProperties': ('deploymentStatus',), + 'DatabaseManagerProperties': ('deploymentStatus',), + 'DatabaseProperties': ('contentStatus',), + 'DeletingActionProcessProperties': ('contentStatus',), + 'DependentContextEventProperties': (), + 'DependentSoftwareComponentProperties': (), + 'DeployedAPIProperties': ('activityStatus',), + 'DeployedAnalyticsModelProperties': ('activityStatus',), + 'DeployedConnectorProperties': ('activityStatus',), + 'DeployedDatabaseSchemaProperties': ('contentStatus',), + 'DeployedOnProperties': (), + 'DeployedSoftwareComponentProperties': ('activityStatus',), + 'DerivedSchemaTypeQueryTargetProperties': (), + 'DesignModelElementProperties': ('contentStatus',), + 'DesignModelProperties': ('contentStatus',), + 'DesignPatternProperties': ('contentStatus',), + 'DetailedProcessingActionProperties': (), + 'DigitalProductCatalogProperties': ('contentStatus',), + 'DigitalProductDependencyProperties': (), + 'DigitalProductFamilyProperties': ('contentStatus', 'deploymentStatus',), + 'DigitalProductProperties': ('contentStatus', 'deploymentStatus',), + 'DigitalResourceOriginProperties': (), + 'DigitalSubscriberProperties': (), + 'DigitalSubscriptionProperties': ('contentStatus',), + 'DigitalSupportProperties': (), + 'DisplayDataContainerProperties': ('contentStatus',), + 'DisplayDataFieldProperties': ('contentStatus',), + 'DisplayDataSchemaTypeProperties': ('contentStatus',), + 'DocumentProperties': ('contentStatus',), + 'DocumentSchemaAttributeProperties': ('contentStatus',), + 'DocumentSchemaTypeProperties': ('contentStatus',), + 'DocumentStoreProperties': ('contentStatus',), + 'EditingCollectionProperties': (), + 'ElementSupplementProperties': (), + 'EmbeddedConnectionProperties': (), + 'EmbeddedProcessProperties': ('activityStatus',), + 'EndpointProperties': (), + 'EnforcementPointProperties': (), + 'EngineActionProperties': ('activityStatus',), + 'EngineProperties': ('deploymentStatus',), + 'EntityElementProperties': (), + 'EnumSchemaTypeProperties': ('contentStatus',), + 'EventBrokerProperties': ('deploymentStatus',), + 'EventManagerProperties': ('deploymentStatus',), + 'EventSchemaAttributeProperties': ('contentStatus',), + 'EventSetProperties': ('contentStatus',), + 'EventTypeListProperties': ('contentStatus',), + 'EventTypeProperties': ('contentStatus',), + 'ExceptionBacklogProperties': (), + 'ExceptionProperties': (), + 'ExceptionTypeProperties': ('contentStatus',), + 'ExecutableFileProperties': ('contentStatus',), + 'ExecutionPointProperties': (), + 'ExperimentProperties': (), + 'ExplorerActionEngineProperties': ('deploymentStatus',), + 'ExplorerActionServiceProperties': ('activityStatus',), + 'ExploringActionProcessProperties': ('contentStatus',), + 'ExternalDataSourceProperties': ('contentStatus',), + 'ExternalIdLinkProperties': (), + 'ExternalIdProperties': (), + 'ExternalModelSourceProperties': ('contentStatus',), + 'ExternalReferenceLinkProperties': (), + 'ExternalReferenceProperties': ('contentStatus',), + 'ExternalSchemaTypeProperties': ('contentStatus',), + 'ExternalSourceCodeProperties': ('contentStatus',), + 'ExternalStandardProperties': ('contentStatus',), + 'FileFolderProperties': ('contentStatus',), + 'FileManagerProperties': ('deploymentStatus',), + 'FileSystemProperties': ('deploymentStatus',), + 'FindDigitalResourceOriginProperties': (), + 'FindProjectClassificationProperties': (), + 'FindPropertyNameProperties': (), + 'FindPropertyNamesProperties': (), + 'FingerprintAnnotationProperties': ('contentStatus',), + 'FixedLocationProperties': (), + 'FolderHierarchyProperties': (), + 'FolioProperties': ('contentStatus',), + 'ForeignKeyProperties': (), + 'FormProperties': ('contentStatus',), + 'FunctionCallProperties': ('activityStatus',), + 'GeneratedTargetProperties': (), + 'GlossaryProjectProperties': (), + 'GlossaryProperties': ('contentStatus',), + 'GlossaryTermProperties': ('contentStatus',), + 'GlossaryTermRelationshipProperties': (), + 'GovernanceActionEngineProperties': ('deploymentStatus',), + 'GovernanceActionExecutorProperties': (), + 'GovernanceActionProcessFlowProperties': (), + 'GovernanceActionProcessInstanceProperties': ('activityStatus',), + 'GovernanceActionProcessProperties': ('contentStatus',), + 'GovernanceActionProcessStepProperties': ('contentStatus',), + 'GovernanceActionProperties': ('contentStatus',), + 'GovernanceActionServiceProperties': ('activityStatus',), + 'GovernanceActionTypeProperties': ('contentStatus',), + 'GovernanceApproachProperties': ('contentStatus',), + 'GovernanceControlLinkProperties': (), + 'GovernanceControlProperties': ('contentStatus',), + 'GovernanceDefinitionProperties': ('contentStatus',), + 'GovernanceDriverLinkProperties': (), + 'GovernanceDriverProperties': ('contentStatus',), + 'GovernanceEngineProperties': ('deploymentStatus',), + 'GovernanceExpectationsProperties': (), + 'GovernanceMeasurementsProperties': (), + 'GovernanceMechanismProperties': (), + 'GovernanceMetricProperties': ('contentStatus',), + 'GovernanceObligationProperties': ('contentStatus',), + 'GovernancePolicyLinkProperties': (), + 'GovernancePolicyProperties': ('contentStatus',), + 'GovernancePrincipleProperties': ('contentStatus',), + 'GovernanceProcedureProperties': ('contentStatus',), + 'GovernanceProjectProperties': (), + 'GovernanceResponseProperties': (), + 'GovernanceResponsibilityProperties': ('contentStatus',), + 'GovernanceResultsProperties': (), + 'GovernanceRoleProperties': (), + 'GovernanceRuleProperties': ('contentStatus',), + 'GovernanceServiceProperties': ('activityStatus',), + 'GovernanceStrategyProperties': ('contentStatus',), + 'GovernanceZoneProperties': ('contentStatus',), + 'GovernedByProperties': (), + 'GraphEdgeLinkProperties': (), + 'GraphEdgeProperties': ('contentStatus',), + 'GraphSchemaTypeProperties': ('contentStatus',), + 'GraphStoreProperties': ('contentStatus',), + 'GraphVertexProperties': ('contentStatus',), + 'HomeCollectionProperties': ('contentStatus',), + 'HostClusterMemberProperties': (), + 'HostClusterProperties': ('deploymentStatus',), + 'HostProperties': ('deploymentStatus',), + 'ISARelationshipProperties': (), + 'ITInfrastructureProfileProperties': (), + 'ITInfrastructureProperties': ('deploymentStatus',), + 'ITProfileProperties': (), + 'ITProfileRoleAppointmentProperties': (), + 'ITProfileRoleProperties': (), + 'ITSubsystemProperties': ('contentStatus',), + 'ImpactProperties': (), + 'ImpactedResourceProperties': (), + 'ImplementationResourceProperties': (), + 'ImplementationSnippetProperties': (), + 'ImplementedByProperties': (), + 'IncidentReportProperties': ('contentStatus',), + 'IncompleteProperties': (), + 'InformalTagProperties': (), + 'InformationSupplyChainLinkProperties': (), + 'InformationSupplyChainProperties': ('contentStatus',), + 'InformationViewProperties': ('contentStatus',), + 'InfrastructureProperties': ('deploymentStatus',), + 'InstanceMetadataProperties': (), + 'IntegrationConnectorProperties': ('activityStatus',), + 'IntegrationGroupProperties': ('deploymentStatus',), + 'InventoryCatalogProperties': ('deploymentStatus',), + 'InvestigationProperties': (), + 'IsAConceptBeadProperties': (), + 'JSONFileProperties': ('contentStatus',), + 'JournalEntryProperties': ('activityStatus',), + 'KeyStoreFileProperties': ('contentStatus',), + 'KnownDuplicateProperties': (), + 'KnownLocationProperties': (), + 'LabeledRelationshipProperties': (), + 'LevelIdentifierQueryProperties': (), + 'LicenseProperties': (), + 'LicenseTypeProperties': ('contentStatus',), + 'LikeProperties': (), + 'LineageBoundaryProperties': (), + 'LineageLogProperties': (), + 'LineageMappingProperties': (), + 'LineageRelationshipProperties': (), + 'LinkedDataFieldProperties': (), + 'LinkedExternalSchemaTypeProperties': (), + 'LinkedFileProperties': (), + 'LinkedMediaProperties': (), + 'ListenerInterfaceProperties': (), + 'LiteralSchemaTypeProperties': ('contentStatus',), + 'LocationProperties': (), + 'LogAnalysisProperties': (), + 'LogFileProperties': ('contentStatus',), + 'MapFromElementTypeProperties': (), + 'MapSchemaTypeProperties': ('contentStatus',), + 'MapToElementTypeProperties': (), + 'MasterDataManagerProperties': ('deploymentStatus',), + 'MediaCollectionProperties': ('contentStatus',), + 'MediaFileProperties': ('contentStatus',), + 'MediaReferenceProperties': (), + 'MeetingProperties': ('activityStatus',), + 'MemberDataFieldProperties': (), + 'MementoProperties': (), + 'MetadataCohortPeerProperties': (), + 'MetadataCollectionProperties': ('contentStatus',), + 'MetadataRepositoryCohortProperties': (), + 'MetadataRepositoryProperties': ('contentStatus',), + 'MetamodelInstanceProperties': (), + 'MeteringLogProperties': (), + 'MethodologyProperties': ('contentStatus',), + 'MobileResourceProperties': (), + 'ModifierProperties': (), + 'MonitoredResourceProperties': (), + 'MoreInformationProperties': (), + 'NamespaceProperties': ('contentStatus',), + 'NamingStandardRuleProperties': ('contentStatus',), + 'NamingStandardRuleSetProperties': ('contentStatus',), + 'NamingStandardsVocabularyProperties': (), + 'NestedDataFieldProperties': (), + 'NestedDesignPatternProperties': (), + 'NestedFileProperties': (), + 'NestedLocationProperties': (), + 'NestedSchemaAttributeProperties': (), + 'NetworkGatewayLinkProperties': (), + 'NetworkGatewayProperties': ('deploymentStatus',), + 'NetworkProperties': ('deploymentStatus',), + 'NewElementProperties': (), + 'NextGovernanceActionProcessStepProperties': (), + 'NoteLogProperties': (), + 'NoteProperties': ('activityStatus',), + 'NotificationManagerProperties': ('deploymentStatus',), + 'NotificationProperties': ('activityStatus',), + 'NotificationSubscriberProperties': ('activityStatus',), + 'NotificationTypeProperties': ('contentStatus',), + 'ObjectAttributeProperties': ('contentStatus',), + 'ObjectIdentifierProperties': (), + 'ObjectSchemaTypeProperties': ('contentStatus',), + 'OpenMetadataRootProperties': (), + 'OpenMetadataTypeDefProperties': (), + 'OperatingPlatformManifestProperties': (), + 'OperatingPlatformProperties': ('contentStatus',), + 'OperatingPlatformUseProperties': (), + 'OrganizationProperties': (), + 'OwnershipProperties': (), + 'ParquetFileProperties': ('contentStatus',), + 'PartOfRelationshipProperties': (), + 'PeerDefinitionProperties': (), + 'PeerDuplicateLinkProperties': (), + 'PeerProperties': (), + 'PermittedProcessingProperties': (), + 'PersonProperties': (), + 'PersonRoleAppointmentProperties': (), + 'PersonRoleProperties': (), + 'PersonalProjectProperties': (), + 'PerspectiveProperties': (), + 'PolicyAdministrationPointProperties': (), + 'PolicyDecisionPointProperties': (), + 'PolicyEnforcementPointProperties': (), + 'PolicyInformationPointProperties': (), + 'PolicyManagementPointProperties': (), + 'PolicyRetrievalPointProperties': (), + 'PortAliasProperties': (), + 'PortDelegationProperties': (), + 'PortImplementationProperties': (), + 'PortProperties': (), + 'PreferredTermProperties': (), + 'PrimaryKeyProperties': (), + 'PrimeWordProperties': (), + 'PrimitiveSchemaTypeProperties': ('contentStatus',), + 'ProcessCallProperties': (), + 'ProcessHierarchyProperties': (), + 'ProcessPortProperties': (), + 'ProcessProperties': ('activityStatus',), + 'ProfileIdentityProperties': (), + 'ProjectClassificationProperties': (), + 'ProjectDependencyProperties': (), + 'ProjectHierarchyProperties': (), + 'ProjectKindProperties': (), + 'ProjectProperties': ('contentStatus',), + 'PromiseProperties': ('deploymentStatus',), + 'PropertiesFileProperties': ('contentStatus',), + 'PropertyFacetProperties': (), + 'ProvisioningActionProcessProperties': ('contentStatus',), + 'PublisherInterfaceProperties': (), + 'QualityAnnotationProperties': ('contentStatus',), + 'QueryDataContainerProperties': ('contentStatus',), + 'QueryDataFieldProperties': ('contentStatus',), + 'QuerySchemaTypeProperties': ('contentStatus',), + 'QuestionProperties': (), + 'RESTAPIManagerProperties': ('deploymentStatus',), + 'RasterFileProperties': ('contentStatus',), + 'RatingProperties': (), + 'RecentAccessProperties': ('contentStatus',), + 'ReferenceCodeMappingTableProperties': ('contentStatus',), + 'ReferenceCodeTableProperties': ('contentStatus',), + 'ReferenceDataProperties': (), + 'ReferenceDataSetProperties': ('contentStatus',), + 'ReferenceDataValueProperties': ('contentStatus',), + 'ReferenceListProperties': ('contentStatus',), + 'ReferenceValueAssignmentProperties': (), + 'ReferenceableFacetProperties': (), + 'ReferenceableProperties': (), + 'RegisteredIntegrationConnectorProperties': (), + 'RegulationArticleProperties': ('contentStatus',), + 'RegulationCertificationTypeProperties': (), + 'RegulationProperties': ('contentStatus',), + 'RegulatorProperties': (), + 'RelatedContextEventProperties': (), + 'RelatedDesignPatternProperties': (), + 'RelatedMediaProperties': ('contentStatus',), + 'RelatedTermProperties': (), + 'RelationalColumnProperties': ('contentStatus',), + 'RelationalDBSchemaProperties': (), + 'RelationalDBSchemaTypeListProperties': ('contentStatus',), + 'RelationalDBSchemaTypeProperties': ('contentStatus',), + 'RelationalDatabaseProperties': ('contentStatus',), + 'RelationalTableProperties': ('contentStatus',), + 'RelationalTableTypeProperties': ('contentStatus',), + 'RelationshipAdviceAnnotationProperties': ('contentStatus',), + 'RelationshipBeanProperties': (), + 'RelationshipElementProperties': (), + 'ReplacementTermProperties': (), + 'ReportDependencyProperties': (), + 'ReportOriginatorProperties': (), + 'ReportProperties': ('contentStatus',), + 'ReportSubjectProperties': (), + 'ReportTypeProperties': ('contentStatus',), + 'ReportedAnnotationProperties': (), + 'ReportingEngineProperties': ('deploymentStatus',), + 'RepositoryGovernanceEngineProperties': ('deploymentStatus',), + 'RepositoryGovernanceServiceProperties': ('activityStatus',), + 'RequestForActionProperties': ('contentStatus',), + 'RequestForActionTargetProperties': (), + 'RequestResponseInterfaceProperties': (), + 'RequirementProperties': ('contentStatus',), + 'ResearchQuestionProperties': ('contentStatus',), + 'ResourceConnectionProperties': (), + 'ResourceListProperties': (), + 'ResourceManagerProperties': ('deploymentStatus',), + 'ResourceMeasureAnnotationProperties': ('contentStatus',), + 'ResourcePermissionsProperties': (), + 'ResourcePhysicalStatusAnnotationProperties': ('contentStatus',), + 'ResourceProfileAnnotationProperties': ('contentStatus',), + 'ResourceProfileDataProperties': (), + 'ResourceProfileLogAnnotationProperties': ('contentStatus',), + 'ResultsSetProperties': ('contentStatus',), + 'RetentionProperties': (), + 'ReusableTechniqueProperties': (), + 'ReusableTechniqueUseProperties': (), + 'ReviewProperties': ('activityStatus',), + 'RiskProperties': ('contentStatus',), + 'RoledRelationshipProperties': (), + 'RootCollectionProperties': ('contentStatus',), + 'RootSchemaTypeProperties': ('contentStatus',), + 'RunMetricsProperties': (), + 'RunnableSoftwareComponentProperties': (), + 'SampleDataProperties': (), + 'SavedQueryProperties': ('contentStatus',), + 'SchemaAnalysisAnnotationProperties': ('contentStatus',), + 'SchemaAttributeDefinitionProperties': (), + 'SchemaAttributeProperties': ('contentStatus',), + 'SchemaAttributeRelationshipProperties': (), + 'SchemaElementProperties': ('contentStatus',), + 'SchemaProperties': (), + 'SchemaTypeChoiceProperties': ('contentStatus',), + 'SchemaTypeDefinitionProperties': (), + 'SchemaTypeOptionProperties': (), + 'SchemaTypeProperties': ('contentStatus',), + 'ScopedByProperties': (), + 'ScopingCollectionProperties': (), + 'ScriptFileProperties': ('contentStatus',), + 'SearchKeywordLinkProperties': (), + 'SearchKeywordProperties': (), + 'SecretsCollectionProperties': ('contentStatus',), + 'SecretsCollectionSecurityListProperties': (), + 'SecureLocationProperties': (), + 'SecurityAccessControlProperties': ('contentStatus',), + 'SecurityGroupProperties': ('contentStatus',), + 'SecurityListMembershipProperties': (), + 'SecurityListProperties': ('contentStatus',), + 'SecurityLogProperties': (), + 'SecurityRoleProperties': ('contentStatus',), + 'SecurityTagQueryProperties': (), + 'SecurityTagsProperties': (), + 'SemanticAnnotationProperties': ('contentStatus',), + 'SemanticAssignmentProperties': (), + 'SemanticAssignmentQueryProperties': (), + 'SemanticDefinitionProperties': (), + 'ServerEndpointProperties': (), + 'ServiceAccessControlProperties': ('contentStatus',), + 'ServiceLevelObjectiveProperties': ('contentStatus',), + 'SimpleSchemaTypeProperties': ('contentStatus',), + 'SkillProperties': ('contentStatus',), + 'SkillSetProperties': ('contentStatus',), + 'SmartQueryProperties': (), + 'SoftwareArchiveProperties': ('contentStatus',), + 'SoftwareCapabilityProperties': ('deploymentStatus',), + 'SoftwareComponentProperties': ('contentStatus',), + 'SoftwareLibraryProperties': ('deploymentStatus',), + 'SoftwareModuleProperties': ('contentStatus',), + 'SoftwarePackageDependencyProperties': (), + 'SoftwarePackageManifestProperties': (), + 'SoftwareServerPlatformProperties': ('deploymentStatus',), + 'SoftwareServerProperties': ('deploymentStatus',), + 'SoftwareServiceProperties': ('deploymentStatus',), + 'SoftwareSourceProperties': (), + 'SolutionActorRoleProperties': (), + 'SolutionBlueprintProperties': ('contentStatus',), + 'SolutionComponentActorProperties': (), + 'SolutionComponentPortProperties': (), + 'SolutionComponentProperties': ('contentStatus', 'deploymentStatus',), + 'SolutionCompositionProperties': (), + 'SolutionDesignProperties': (), + 'SolutionLinkingWireProperties': (), + 'SolutionPortDelegationProperties': (), + 'SolutionPortProperties': ('contentStatus',), + 'SourceCodeFileProperties': ('contentStatus',), + 'SourceControlLibraryProperties': ('deploymentStatus',), + 'SourcedFromProperties': (), + 'SpecializedDesignPatternProperties': (), + 'SpecificationPropertyAssignmentProperties': (), + 'SpecificationPropertyValueProperties': ('contentStatus',), + 'SpreadsheetFileProperties': ('contentStatus',), + 'StagingCollectionProperties': (), + 'StorageVolumeProperties': (), + 'StoredOnProperties': (), + 'StructSchemaTypeProperties': ('contentStatus',), + 'StudyProjectProperties': (), + 'SubjectAreaHierarchyProperties': (), + 'SubjectAreaProperties': ('contentStatus',), + 'SubscribingActionProcessProperties': ('contentStatus',), + 'SupplementaryPropertiesProperties': (), + 'SupportedGovernanceServiceProperties': (), + 'SupportedSoftwareCapabilityProperties': (), + 'SupportingDefinitionProperties': (), + 'SurveyActionEngineProperties': ('deploymentStatus',), + 'SurveyActionServiceProperties': ('activityStatus',), + 'SurveyReportProperties': ('contentStatus',), + 'SurveyingActionProcessProperties': ('contentStatus',), + 'SynonymProperties': (), + 'TabularColumnProperties': ('contentStatus',), + 'TabularDataSetCollectionProperties': ('contentStatus',), + 'TabularDataSetProperties': ('contentStatus',), + 'TabularFileColumnProperties': ('contentStatus',), + 'TabularSchemaTypeProperties': ('contentStatus',), + 'TargetForGovernanceActionProperties': (), + 'TaskProperties': (), + 'TaxonomyProperties': (), + 'TeamProperties': (), + 'TeamRoleAppointmentProperties': (), + 'TeamRoleProperties': (), + 'TeamStructureProperties': (), + 'TechnologyTypeProperties': ('contentStatus',), + 'TemplateProperties': (), + 'TemplateSubstituteProperties': (), + 'TermsAndConditionsProperties': ('contentStatus',), + 'ThreatProperties': ('contentStatus',), + 'ThreeDImageFileProperties': ('contentStatus',), + 'ToDoProperties': ('activityStatus',), + 'TopicProperties': ('contentStatus',), + 'TransientEmbeddedProcessProperties': ('activityStatus',), + 'TranslationDetailProperties': (), + 'TranslationLinkProperties': (), + 'TypeEmbeddedAttributeProperties': (), + 'TypedByConceptBeadProperties': (), + 'UltimateDestinationProperties': (), + 'UltimateSourceProperties': (), + 'UsedInContextProperties': (), + 'UserAccessDirectoryProperties': ('deploymentStatus',), + 'UserAccountProfileProperties': (), + 'UserAccountProperties': (), + 'UserAuthenticationManagerProperties': ('deploymentStatus',), + 'UserIdentityProperties': (), + 'UserProfileManagerProperties': ('deploymentStatus',), + 'ValidMetadataValueProperties': ('contentStatus',), + 'ValidValueAssociationProperties': (), + 'ValidValueDefinitionProperties': ('contentStatus',), + 'ValidValueMemberProperties': (), + 'ValidValuesAssignmentProperties': (), + 'ValidValuesImplementationProperties': (), + 'ValidValuesMappingProperties': (), + 'VectorFileProperties': ('contentStatus',), + 'VerificationPointProperties': (), + 'VideoFileProperties': ('contentStatus',), + 'VirtualConnectionProperties': (), + 'VirtualContainerProperties': ('deploymentStatus',), + 'VirtualMachineProperties': ('deploymentStatus',), + 'VirtualRelationalTableProperties': ('contentStatus',), + 'VisibleEndpointProperties': (), + 'WatchdogActionEngineProperties': ('deploymentStatus',), + 'WatchdogActionServiceProperties': ('activityStatus',), + 'WorkItemListProperties': ('contentStatus',), + 'WorkflowEngineProperties': ('deploymentStatus',), + 'WorkingSetProperties': ('contentStatus',), + 'XMLFileProperties': ('contentStatus',), + 'YAMLFileProperties': ('contentStatus',), + 'ZoneHierarchyProperties': (), + 'ZoneMembershipProfileProperties': (), + 'ZoneMembershipProperties': (), +} diff --git a/md_processing/v2/asset_maker.py b/md_processing/v2/asset_maker.py index 36da7419..0404ed34 100644 --- a/md_processing/v2/asset_maker.py +++ b/md_processing/v2/asset_maker.py @@ -48,13 +48,18 @@ async def apply_changes(self) -> str: if command_key == "Element" and not om_type: # Generic 'Create Element' -- the raw TemplateRequestBody surface. + # Egeria dropped initialStatus (ISSUE-113); the copy keeps the template's + # own status, which Template Properties can override. + status = attributes.get("Initial Status") or {} + if status.get("value") and not status.get("is_default"): + logger.warning("'Initial Status' ignored: Egeria no longer accepts it. Set contentStatus " + "(or the type's own status) in Template Properties instead.") body = { "class": "TemplateRequestBody", "typeName": _v(attributes, "Element Type Name"), "templateGUID": _v(attributes, "Template GUID"), "placeholderPropertyValues": _v(attributes, "Placeholder Property Values") or {}, "replacementProperties": _v(attributes, "Template Properties"), - "initialStatus": _v(attributes, "Initial Status"), "initialClassifications": _v(attributes, "Generic Initial Classifications"), "anchorGUID": _guid(attributes, "Anchor ID"), "isOwnAnchor": _v(attributes, "Is Own Anchor", True), diff --git a/md_processing/v2/solution_architect.py b/md_processing/v2/solution_architect.py index d8c7c921..bf215c87 100644 --- a/md_processing/v2/solution_architect.py +++ b/md_processing/v2/solution_architect.py @@ -197,7 +197,6 @@ async def apply_changes(self) -> str: prop_body.update({ "solutionComponentType": attributes.get('Solution Component Type', {}).get('value'), "plannedDeployedImplementationType": attributes.get('Planned Deployed Implementation Type', {}).get('value'), - "deploymentStatus": attributes.get('Deployment Status', {}).get('value'), "userDefinedDeploymentStatus": attributes.get('User Defined Deployment Status', {}).get('value'), "canonicalName": attributes.get('Canonical Name', {}).get('value'), }) @@ -250,7 +249,6 @@ async def apply_changes(self) -> str: body = body_slimmer({ "class": "NewElementRequestBody", "properties": prop_body, - "initialStatus": attributes.get('Status', {}).get('value', 'ACTIVE') }) # Anchor/Parent logic body["anchorGUID"] = attributes.get('Anchor ID', {}).get('guid') diff --git a/pyegeria/__init__.py b/pyegeria/__init__.py index 15c6a237..f0a72a6e 100644 --- a/pyegeria/__init__.py +++ b/pyegeria/__init__.py @@ -53,7 +53,7 @@ ) from pyegeria.core.config import settings -from pyegeria.core.utils import body_slimmer +from pyegeria.core.utils import body_slimmer, guid_or_none from pyegeria.core.clipboard import copy_to_clipboard, get_from_clipboard # OMVS Clients - Promoted to package level for ease of use from pyegeria.omvs.action_author import ActionAuthor @@ -220,6 +220,7 @@ "init_logging", "exec_report_spec", "body_slimmer", + "guid_or_none", "copy_to_clipboard", "get_from_clipboard", "print_basic_exception", diff --git a/pyegeria/core/_server_client.py b/pyegeria/core/_server_client.py index 399397f0..5421e262 100644 --- a/pyegeria/core/_server_client.py +++ b/pyegeria/core/_server_client.py @@ -3422,7 +3422,7 @@ async def _async_find_notes( starts_with: bool = True, ends_with: bool = False, ignore_case: bool = True, anchor_domain: Optional[str] = None, - metadata_element_type: Optional[str] = "Action", + metadata_element_type: Optional[str] = "Note", metadata_element_subtypes: Optional[list[str]] = None, skip_relationships: Optional[list[str]] = None, include_only_relationships: Optional[list[str]] = None, @@ -3566,7 +3566,7 @@ async def _async_find_notes( # Filter out None values, but keep search_string even if None (it's required) params = {k: v for k, v in params.items() if v is not None or k == 'search_string'} - response = await self._async_find_request(url, _type=metadata_element_type or "Action", _gen_output=self._generate_feedback_output, + response = await self._async_find_request(url, _type=metadata_element_type or "Note", _gen_output=self._generate_feedback_output, **params) return response @@ -3625,7 +3625,7 @@ def find_notes( async def _async_get_note_by_guid( self, guid: str, - metadata_element_type_name: str | None = "Action", + metadata_element_type_name: str | None = "Note", graph_query_depth: int = 3, output_format: str = "JSON", report_spec: Optional[str | dict] = None, @@ -3640,7 +3640,7 @@ async def _async_get_note_by_guid( guid: str - unique identifier for the note object. metadata_element_type_name : str, optional - - type name for the note object (default is "Action"). + - type name for the note object (default is "Note"). body - optional effective time @@ -3667,7 +3667,7 @@ async def _async_get_note_by_guid( def get_note_by_guid( self, guid: str, - metadata_element_type_name: str | None = "Action", + metadata_element_type_name: str | None = "Note", graph_query_depth: int = 3, output_format: str = "JSON", report_spec: Optional[str | dict] = None, @@ -3682,7 +3682,7 @@ def get_note_by_guid( guid: str - unique identifier of the requested metadata element metadata_element_type_name : str, optional - - type name for the note object (default is "Action"). + - type name for the note object (default is "Note"). body: dict | GetRequestBody - optional details of the request. output_format: str, default = "JSON" @@ -3707,7 +3707,7 @@ def get_note_by_guid( async def _async_get_notes_for_note_log( self, note_log_guid: str, - metadata_element_type_name: str | None = "Action", + metadata_element_type_name: str | None = "Note", graph_query_depth: int = 3, start_from: int = 0, page_size: int = 0, output_format: str = "JSON", @@ -3722,6 +3722,9 @@ async def _async_get_notes_for_note_log( ---------- note_log_guid: str - id of the note log to retrieve notes for. + metadata_element_type_name : str, optional + - type of log entry to return (default is "Note"). A note log can also hold other + Notification entries; pass "Notification" to get every entry. body: dict | ResultsRequestBody - optional details of the request. output_format: str, default = "JSON" @@ -3753,7 +3756,7 @@ async def _async_get_notes_for_note_log( def get_notes_for_note_log( self, note_log_guid: str, - metadata_element_type_name: str | None = "Action", + metadata_element_type_name: str | None = "Note", graph_query_depth: int = 3, start_from: int = 0, page_size: int = 0, output_format: str = "JSON", @@ -3768,6 +3771,9 @@ def get_notes_for_note_log( ---------- note_log_guid: str - note_log id to find notes for. + metadata_element_type_name : str, optional + - type of log entry to return (default is "Note"). A note log can also hold other + Notification entries; pass "Notification" to get every entry. body: dict | ResultsRequestBody - optional details of the request. output_format: str, default = "JSON" diff --git a/pyegeria/core/utils.py b/pyegeria/core/utils.py index 0314e090..69942bf2 100644 --- a/pyegeria/core/utils.py +++ b/pyegeria/core/utils.py @@ -60,6 +60,29 @@ def print_guid_list(guids): +def guid_or_none(result: Any) -> str | None: + """Return `result` if it is a GUID, else None. + + Lookups such as `get_guid_for_name` signal a miss with a sentinel string + ("No elements found", "No guid returned", ...). The sentinel is truthy, so + `if not guid` and `guid or default` read a miss as found (ISSUE-114). Pass + the result through this helper first:: + + guid = guid_or_none(client.get_guid_for_name(qualified_name)) + if guid is None: + guid = client.create_...(...) + + A GUID is a non-empty string with no whitespace; every sentinel has spaces. + Anything else (None, "", a list, a dict) also gives None. + """ + if not isinstance(result, str): + return None + result = result.strip() + if not result or any(ch.isspace() for ch in result): + return None + return result + + def body_slimmer(body: dict) -> dict: """body_slimmer is a little function to remove unused keys from a dict and recursively slim embedded dicts @@ -438,7 +461,10 @@ def parse_to_dict(input_str: str) -> dict | None: def dynamic_catch(func: T) -> T: if getattr(app_settings.Debug, "enable_logger_catch", False): - return logger.catch(func) # Apply the logger.catch decorator + # reraise=True: log the exception with loguru's full traceback, then let it propagate. + # Without it a failure returns None, which callers read as "nothing found" (ISSUE-134). + # Pass options first: logger.catch(func, reraise=True) silently drops the keyword. + return logger.catch(reraise=True)(func) else: return func # Return the function unwrapped diff --git a/pyegeria/egeria_tech_client.py b/pyegeria/egeria_tech_client.py index 6df9723a..a13cdc56 100644 --- a/pyegeria/egeria_tech_client.py +++ b/pyegeria/egeria_tech_client.py @@ -253,6 +253,9 @@ def get_report_spec_schema(self, report_spec_name: str, **kwargs) -> list: action_params["output_format"] = "JSON" func_name = spec.action.function + if not func_name: + return [{"attribute_path": "Error", + "data_type": f"Report spec '{report_spec_name}' has only an analytic function; no element schema to discover."}] if "." in func_name: # E.g., 'GovernanceOfficer.find_governance_definitions' -> method_name _, method_name = func_name.split(".", 1) diff --git a/pyegeria/omvs/automated_curation.py b/pyegeria/omvs/automated_curation.py index 2514633a..c8f2421c 100644 --- a/pyegeria/omvs/automated_curation.py +++ b/pyegeria/omvs/automated_curation.py @@ -602,7 +602,6 @@ async def _async_create_elem_from_template(self, body: Optional[dict | TemplateR The full description of the body is shown below: { "typeName" : "", - "initialStatus" : "", "initialClassifications" : "", "anchorGUID" : "", "isOwnAnchor" : "", @@ -650,7 +649,6 @@ def create_elem_from_template(self, body: Optional[dict | TemplateRequestBody] = The full description of the body is shown below: { "typeName" : "", - "initialStatus" : "", "initialClassifications" : "", "anchorGUID" : "", "isOwnAnchor" : "", diff --git a/pyegeria/omvs/collection_manager.py b/pyegeria/omvs/collection_manager.py index 927e1a42..1d59ff8d 100644 --- a/pyegeria/omvs/collection_manager.py +++ b/pyegeria/omvs/collection_manager.py @@ -4332,14 +4332,12 @@ async def _async_create_digital_product(self, body: dict | NewElementRequestBody "externalSourceName": "add qualified name here", "effectiveTime" : "{{$isoTimestamp}}", "forLineage" : false, - "forDuplicateProcessing" : false, - "initialStatus" : "ACTIVE" + "forDuplicateProcessing" : false } - The valid values for initialStatus are: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, APPROVED_CONCEPT, - UNDER_DEVELOPMENT, DEVELOPMENT_COMPLETE, APPROVED_FOR_DEPLOYMENT, ACTIVE, DISABLED, DEPRECATED, - OTHER. If using OTHER, set the userDefinedStatus with the status value you want. If not specified, will - default to ACTIVE. + Egeria no longer accepts initialStatus (removed 2025-12-15). Set a lifecycle status with + `contentStatus` in `properties`: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, ACTIVE, DEPRECATED, + OBSOLETE or OTHER (with OTHER, also set `userDefinedContentStatus`). """ url = f"{self.collection_command_root}" return await self._async_create_element_body_request(url, ["DigitalProductProperties"], body) @@ -4457,10 +4455,9 @@ async def _async_create_digital_product_catalog(self, body: dict | NewElementReq } } - The valid values for initialStatus are: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, APPROVED_CONCEPT, - UNDER_DEVELOPMENT, DEVELOPMENT_COMPLETE, APPROVED_FOR_DEPLOYMENT, ACTIVE, DISABLED, DEPRECATED, - OTHER. If using OTHER, set the userDefinedStatus with the status value you want. If not specified, will - default to ACTIVE. + Egeria no longer accepts initialStatus (removed 2025-12-15). Set a lifecycle status with + `contentStatus` in `properties`: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, ACTIVE, DEPRECATED, + OBSOLETE or OTHER (with OTHER, also set `userDefinedContentStatus`). """ url = f"{self.collection_command_root}" return await self._async_create_element_body_request(url, "DigitalProductCatalogProperties", body) @@ -4506,10 +4503,9 @@ def create_digital_product_catalog(self, body: Optional[dict | NewElementRequest } } - The valid values for initialStatus are: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, APPROVED_CONCEPT, - UNDER_DEVELOPMENT, DEVELOPMENT_COMPLETE, APPROVED_FOR_DEPLOYMENT, ACTIVE, DISABLED, DEPRECATED, - OTHER. If using OTHER, set the userDefinedStatus with the status value you want. If not specified, will - default to ACTIVE. + Egeria no longer accepts initialStatus (removed 2025-12-15). Set a lifecycle status with + `contentStatus` in `properties`: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, ACTIVE, + DEPRECATED, OBSOLETE or OTHER (with OTHER, also set `userDefinedContentStatus`). """ return asyncio.get_event_loop().run_until_complete( @@ -4702,14 +4698,12 @@ async def _async_create_agreement(self, body: dict | NewElementRequestBody) -> s "externalSourceName": "add qualified name here", "effectiveTime" : "{{$isoTimestamp}}", "forLineage" : false, - "forDuplicateProcessing" : false, - "initialStatus" : "ACTIVE" + "forDuplicateProcessing" : false } - The valid values for initialStatus are: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, APPROVED_CONCEPT, - UNDER_DEVELOPMENT, DEVELOPMENT_COMPLETE, APPROVED_FOR_DEPLOYMENT, ACTIVE, DISABLED, DEPRECATED, - OTHER. If using OTHER, set the userDefinedStatus with the status value you want. If not specified, will - default to ACTIVE. + Egeria no longer accepts initialStatus (removed 2025-12-15). Set a lifecycle status with + `contentStatus` in `properties`: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, ACTIVE, DEPRECATED, + OBSOLETE or OTHER (with OTHER, also set `userDefinedContentStatus`). """ url = f"{self.collection_command_root}" return await self._async_create_element_body_request(url, "AgreementProperties", body) @@ -4769,14 +4763,12 @@ def create_agreement(self, body: Optional[dict | NewElementRequestBody] = None) "externalSourceName": "add qualified name here", "effectiveTime" : "{{$isoTimestamp}}", "forLineage" : false, - "forDuplicateProcessing" : false, - "initialStatus" : "ACTIVE" + "forDuplicateProcessing" : false } - The valid values for initialStatus are: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, APPROVED_CONCEPT, - UNDER_DEVELOPMENT, DEVELOPMENT_COMPLETE, APPROVED_FOR_DEPLOYMENT, ACTIVE, DISABLED, DEPRECATED, - OTHER. If using OTHER, set the userDefinedStatus with the status value you want. If not specified, will - default to ACTIVE. + Egeria no longer accepts initialStatus (removed 2025-12-15). Set a lifecycle status with + `contentStatus` in `properties`: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, ACTIVE, + DEPRECATED, OBSOLETE or OTHER (with OTHER, also set `userDefinedContentStatus`). """ return asyncio.get_event_loop().run_until_complete( @@ -4831,7 +4823,6 @@ def create_agreement(self, body: Optional[dict | NewElementRequestBody] = None) # "property2Name" : "property2Value" # } # }, - # "initialStatus" : "ACTIVE", # "externalSourceGUID": "add guid here", # "externalSourceName": "add qualified name here", # "effectiveTime" : "{{$isoTimestamp}}", @@ -4839,10 +4830,9 @@ def create_agreement(self, body: Optional[dict | NewElementRequestBody] = None) # "forDuplicateProcessing" : false # } # - # The valid values for initialStatus are: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, APPROVED_CONCEPT, - # UNDER_DEVELOPMENT, DEVELOPMENT_COMPLETE, APPROVED_FOR_DEPLOYMENT, ACTIVE, DISABLED, DEPRECATED, - # OTHER. If using OTHER, set the userDefinedStatus with the status value you want. If not specified, will - # default to ACTIVE. + # Egeria no longer accepts initialStatus (removed 2025-12-15). Set a lifecycle status with + # `contentStatus` in `properties`: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, ACTIVE, DEPRECATED, + # OBSOLETE or OTHER (with OTHER, also set `userDefinedContentStatus`). # """ # # url = f"{self.collection_command_root}/data-sharing-agreement" @@ -4906,14 +4896,12 @@ def create_agreement(self, body: Optional[dict | NewElementRequestBody] = None) # "externalSourceName": "add qualified name here", # "effectiveTime" : "{{$isoTimestamp}}", # "forLineage" : false, - # "forDuplicateProcessing" : false, - # "initialStatus" : "ACTIVE" + # "forDuplicateProcessing" : false # } # - # The valid values for initialStatus are: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, APPROVED_CONCEPT, - # UNDER_DEVELOPMENT, DEVELOPMENT_COMPLETE, APPROVED_FOR_DEPLOYMENT, ACTIVE, DISABLED, DEPRECATED, - # OTHER. If using OTHER, set the userDefinedStatus with the status value you want. If not specified, will - # default to ACTIVE. + # Egeria no longer accepts initialStatus (removed 2025-12-15). Set a lifecycle status with + # `contentStatus` in `properties`: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, ACTIVE, + # DEPRECATED, OBSOLETE or OTHER (with OTHER, also set `userDefinedContentStatus`). # """ # # return asyncio.get_event_loop().run_until_complete( @@ -5737,8 +5725,9 @@ async def _async_create_digital_subscription(self, body: dict | NewElementReques The DigitalSubscription is a type of Agreement and so can have lifecycle states. Note: the three dates: introductionDate, nextVersionDate and withdrawDate must be valid dates if specified, otherwise you will get a 400 error response. - The valid values for initialStatus are: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED - ACTIVE, DEPRECATED, OTHER. If using OTHER, set the userDefinedStatus with the statu value you want. + Egeria no longer accepts initialStatus (removed 2025-12-15). Set a lifecycle status with + `contentStatus` in `properties`: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, ACTIVE, DEPRECATED, + OBSOLETE or OTHER (with OTHER, also set `userDefinedContentStatus`). { "class" : "NewAgreementRequestBody", @@ -5764,7 +5753,6 @@ async def _async_create_digital_subscription(self, body: dict | NewElementReques "property2Name" : "property2Value" } }, - "initialStatus" : "OTHER", "externalSourceGUID": "add guid here", "externalSourceName": "add qualified name here", "effectiveTime" : "{{$isoTimestamp}}", @@ -5808,8 +5796,9 @@ def create_digital_subscription(self, body: dict) -> str: The DigitalSubscription is a type of Agreement and so can have lifecycle states. Note: the three dates: introductionDate, nextVersionDate and withdrawDate must be valid dates if specified, otherwise you will get a 400 error response. - The valid values for initialStatus are: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED - ACTIVE, DEPRECATED, OTHER. If using OTHER, set the userDefinedStatus with the statu value you want. + Egeria no longer accepts initialStatus (removed 2025-12-15). Set a lifecycle status with + `contentStatus` in `properties`: DRAFT, PREPARED, PROPOSED, APPROVED, REJECTED, ACTIVE, DEPRECATED, + OBSOLETE or OTHER (with OTHER, also set `userDefinedContentStatus`). { "class" : "NewAgreementRequestBody", @@ -5835,7 +5824,6 @@ def create_digital_subscription(self, body: dict) -> str: "property2Name" : "property2Value" } }, - "initialStatus" : "OTHER", "externalSourceGUID": "add guid here", "externalSourceName": "add qualified name here", "effectiveTime" : "{{$isoTimestamp}}", diff --git a/pyegeria/omvs/connection_maker.py b/pyegeria/omvs/connection_maker.py index 794cf604..7cc298c4 100644 --- a/pyegeria/omvs/connection_maker.py +++ b/pyegeria/omvs/connection_maker.py @@ -950,7 +950,7 @@ def get_endpoints_for_asset( async def _async_link_connection_connector_type(self, connection_guid: str, connector_type_guid: str, body: Union[NewRelationshipRequestBody, dict] = None): url = f"{self.base_url}/connections/{connection_guid}/connector-types/{connector_type_guid}/attach" - await self._async_new_relationship_request(url, ["ConnectionConnectorType"], body) + await self._async_new_relationship_request(url, ["ConnectionConnectorTypeProperties"], body) def link_connection_connector_type(self, connection_guid: str, connector_type_guid: str, body: Union[NewRelationshipRequestBody, dict] = None): loop = asyncio.get_event_loop() @@ -966,7 +966,7 @@ def detach_connection_connector_type(self, connection_guid: str, connector_type_ async def _async_link_connection_endpoint(self, connection_guid: str, endpoint_guid: str, body: Union[NewRelationshipRequestBody, dict] = None): url = f"{self.base_url}/connections/{connection_guid}/endpoints/{endpoint_guid}/attach" - await self._async_new_relationship_request(url, ["ConnectionEndpoint"], body) + await self._async_new_relationship_request(url, ["ConnectToEndpointProperties"], body) def link_connection_endpoint(self, connection_guid: str, endpoint_guid: str, body: Union[NewRelationshipRequestBody, dict] = None): loop = asyncio.get_event_loop() @@ -982,7 +982,7 @@ def detach_connection_endpoint(self, connection_guid: str, endpoint_guid: str, b async def _async_link_embedded_connection(self, connection_guid: str, embedded_connection_guid: str, body: Union[NewRelationshipRequestBody, dict] = None): url = f"{self.base_url}/connections/{connection_guid}/embedded-connections/{embedded_connection_guid}/attach" - await self._async_new_relationship_request(url, ["EmbeddedConnection"], body) + await self._async_new_relationship_request(url, ["EmbeddedConnectionProperties"], body) def link_embedded_connection(self, connection_guid: str, embedded_connection_guid: str, body: Union[NewRelationshipRequestBody, dict] = None): loop = asyncio.get_event_loop() @@ -998,7 +998,7 @@ def detach_embedded_connection(self, connection_guid: str, embedded_connection_g async def _async_link_asset_to_connection(self, asset_guid: str, connection_guid: str, body: Union[NewRelationshipRequestBody, dict] = None): url = f"{self.base_url}/assets/{asset_guid}/connections/{connection_guid}/attach" - await self._async_new_relationship_request(url, ["AssetConnection"], body) + await self._async_new_relationship_request(url, ["ResourceConnectionProperties"], body) def link_asset_to_connection(self, asset_guid: str, connection_guid: str, body: Union[NewRelationshipRequestBody, dict] = None): loop = asyncio.get_event_loop() @@ -1054,7 +1054,7 @@ def detach_resource_from_connection(self, element_guid: str, connection_guid: st async def _async_link_endpoint_to_it_asset(self, asset_guid: str, endpoint_guid: str, body: Union[NewRelationshipRequestBody, dict] = None): url = f"{self.base_url}/assets/{asset_guid}/endpoints/{endpoint_guid}/attach" - await self._async_new_relationship_request(url, ["ITAssetEndpoint"], body) + await self._async_new_relationship_request(url, ["ServerEndpointProperties"], body) def link_endpoint_to_it_asset(self, asset_guid: str, endpoint_guid: str, body: Union[NewRelationshipRequestBody, dict] = None): loop = asyncio.get_event_loop() diff --git a/pyegeria/omvs/data_discovery.py b/pyegeria/omvs/data_discovery.py index bbebc625..688d633e 100644 --- a/pyegeria/omvs/data_discovery.py +++ b/pyegeria/omvs/data_discovery.py @@ -356,7 +356,7 @@ async def _async_attach_annotation_to_report( None """ url = f"{self.platform_url}/servers/{self.view_server}/api/open-metadata/data-discovery/survey-reports/{survey_report_guid}/new-annotations/{annotation_guid}/attach" - await self._async_new_relationship_request(url, ["SurveyReportAnnotationRelationship"], body) + await self._async_new_relationship_request(url, ["ReportedAnnotationProperties"], body) def attach_annotation_to_report( self, @@ -456,7 +456,7 @@ async def _async_link_annotation_to_described_element( None """ url = f"{self.platform_url}/servers/{self.view_server}/api/open-metadata/data-discovery/elements/{element_guid}/associated-annotations/{annotation_guid}/attach" - await self._async_new_relationship_request(url, ["AnnotationDescribedElementRelationship"], body) + await self._async_new_relationship_request(url, ["AssociatedAnnotationProperties"], body) def link_annotation_to_described_element( self, @@ -529,7 +529,7 @@ async def _async_link_annotation_to_its_predecessor( None """ url = f"{self.platform_url}/servers/{self.view_server}/api/open-metadata/data-discovery/annotations/{predecessor_guid}/later-annotations/{annotation_guid}/attach" - await self._async_new_relationship_request(url, ["AnnotationPredecessorRelationship"], body) + await self._async_new_relationship_request(url, ["AnnotationExtensionProperties"], body) def link_annotation_to_its_predecessor( self, @@ -629,7 +629,7 @@ async def _async_link_resource_profile_data( None """ url = f"{self.platform_url}/servers/{self.view_server}/api/open-metadata/data-discovery/annotations/{annotation_guid}/resource-profile-data-assets/{element_guid}/attach" - await self._async_new_relationship_request(url, ["ResourceProfileDataRelationship"], body) + await self._async_new_relationship_request(url, ["ResourceProfileDataProperties"], body) def link_resource_profile_data( self, @@ -729,7 +729,7 @@ async def _async_link_annotation_match( None """ url = f"{self.platform_url}/servers/{self.view_server}/api/open-metadata/data-discovery/annotations/{annotation_guid}/matched-elements/{element_guid}/attach" - await self._async_new_relationship_request(url, ["AnnotationMatchRelationship"], body) + await self._async_new_relationship_request(url, ["AnnotationMatchProperties"], body) def link_annotation_match( self, @@ -1785,7 +1785,7 @@ def detach_analysis_report_from_asset(self, analysis_report_guid: str, asset_gui async def _async_link_request_for_action_target(self, annotation_guid: str, element_guid: str, body: Optional[dict | NewRelationshipRequestBody] = None) -> None: """Link a request for action target. Async version.""" url = f"{self.platform_url}/servers/{self.view_server}/api/open-metadata/data-discovery/annotations/{annotation_guid}/request-for-action-targets/{element_guid}/attach" - await self._async_new_relationship_request(url, ["RequestForActionProperties"], body) + await self._async_new_relationship_request(url, ["RequestForActionTargetProperties"], body) def link_request_for_action_target(self, annotation_guid: str, element_guid: str, body: Optional[dict | NewRelationshipRequestBody] = None) -> None: """Link a request for action target.""" diff --git a/pyegeria/omvs/glossary_manager.py b/pyegeria/omvs/glossary_manager.py index c452d1c1..dce02163 100644 --- a/pyegeria/omvs/glossary_manager.py +++ b/pyegeria/omvs/glossary_manager.py @@ -1045,8 +1045,8 @@ async def _async_import_glossary_terms_from_csv( "examples": examples, "usage": usage, "publishVersionIdentifier": version, + "contentStatus": status, }, - "initialStatus": status, }) resp = await self._async_make_request("POST", create_url, create_body) term_guid = resp.json().get("guid", NO_GUID_RETURNED) @@ -1221,7 +1221,8 @@ async def _async_export_glossary_to_csv( examples = props.get("examples", "---") or "---" usage = props.get("usage", "---") or "---" version = props.get("publishVersionIdentifier", "---") or "---" - status = term.get("elementHeader", {}).get("status", "DRAFT") + # A term's status is its contentStatus; Egeria removed initialStatus (ISSUE-113). + status = props.get("contentStatus") or term.get("elementHeader", {}).get("status", "DRAFT") csv_writer.writerow( { @@ -1340,7 +1341,7 @@ async def _async_create_term_copy( "class": "GlossaryTermProperties", "qualifiedName": qualified_name, "displayName": new_display_name, - "status": term_status, + "contentStatus": term_status, "versionIdentifier": version_id, }, } diff --git a/pyegeria/omvs/governance_officer.py b/pyegeria/omvs/governance_officer.py index 877b0111..dd7750a8 100644 --- a/pyegeria/omvs/governance_officer.py +++ b/pyegeria/omvs/governance_officer.py @@ -372,8 +372,7 @@ async def _async_create_governance_definition(self, body: dict | NewElementReque "property1" : "propertyValue1", "property2" : "propertyValue2" } - }, - "initialStatus": "DRAFT" + } } Generic governance Body structure with Parent: @@ -418,8 +417,7 @@ async def _async_create_governance_definition(self, body: dict | NewElementReque }, "effectiveFrom": "{{$isoTimestamp}}", "effectiveTo": "{{$isoTimestamp}}" - }, - "initialStatus": "DRAFT" + } } """ @@ -500,8 +498,7 @@ def create_governance_definition(self, body: dict | NewElementRequestBody) -> st "property1" : "propertyValue1", "property2" : "propertyValue2" } - }, - "initialStatus": "DRAFT" + } } Body structure with Parent: @@ -546,8 +543,7 @@ def create_governance_definition(self, body: dict | NewElementRequestBody) -> st }, "effectiveFrom": "{{$isoTimestamp}}", "effectiveTo": "{{$isoTimestamp}}" - }, - "initialStatus": "DRAFT" + } } diff --git a/pyegeria/omvs/metadata_expert.py b/pyegeria/omvs/metadata_expert.py index f4af34e8..9220b526 100644 --- a/pyegeria/omvs/metadata_expert.py +++ b/pyegeria/omvs/metadata_expert.py @@ -27,6 +27,21 @@ def base_path(client: ServerClient, view_server: str): return f"{client.platform_url}/servers/{view_server}/api/open-metadata/metadata-expert" +def _with_query_options(body: dict | None, **options) -> dict: + """Copy keyword arguments the caller passed separately into a ResultsRequestBody dict. + + Only fields the body does not already set are added, and None values are skipped, + so a field set in the body always wins (ISSUE-129). + """ + if body is not None and not isinstance(body, dict): + body = body.model_dump(by_alias=True, exclude_none=True) + merged = dict(body) if body else {"class": "ResultsRequestBody"} + for key, value in options.items(): + if value is not None and key not in merged: + merged[key] = value + return merged + + @dynamic_catch def process_related_element_list( response: Response, mermaid_only: bool, relationship_list: bool = False @@ -2955,7 +2970,10 @@ async def _async_get_related_metadata_elements( url = ( f"{base_path(self, self.view_server)}/related-elements/{guid}/type/{relationship_type}" + f"?startingAtEnd={starting_at_end}" ) + body = _with_query_options(body, startFrom=start_from, pageSize=page_size, forLineage=for_lineage, + forDuplicateProcessing=for_duplicate_processing) response: Response = await self._async_make_request( "POST", url, body_slimmer(body), timeout=timeout @@ -3081,7 +3099,7 @@ async def _async_get_all_metadata_element_relationships( for_duplicate_processing: bool, default is set by server - Normally false. Set true when the caller is part of a deduplication function starting_at_end: int, default = 0 - - Relationship end to start from. + - Ignored: this endpoint fixes both ends in the URL. Kept so existing calls still work. start_from: int, default = 0 - index of the list to start from (0 for start). page_size @@ -3128,6 +3146,8 @@ async def _async_get_all_metadata_element_relationships( f"{base_path(self, self.view_server)}/metadata-elements/{end1_guid}/linked-by-any-type/" f"to-elements/{end2_guid}" ) + body = _with_query_options(body, startFrom=start_from, pageSize=page_size, forLineage=for_lineage, + forDuplicateProcessing=for_duplicate_processing) response: Response = await self._async_make_request( "POST", url, body_slimmer(body), timeout=timeout @@ -3165,7 +3185,7 @@ def get_all_metadata_element_relationships( for_duplicate_processing: bool, default is set by server - Normally false. Set true when the caller is part of a deduplication function starting_at_end: int, default = 0 - - Relationship end to start from. + - Ignored: this endpoint fixes both ends in the URL. Kept so existing calls still work. start_from: int, default = 0 - index of the list to start from (0 for start). page_size @@ -3256,7 +3276,7 @@ async def _async_get_metadata_element_relationships( for_duplicate_processing: bool, default is set by server - Normally false. Set true when the caller is part of a deduplication function starting_at_end: int, default = 0 - - Relationship end to start from. + - Ignored: this endpoint fixes both ends in the URL. Kept so existing calls still work. start_from: int, default = 0 - index of the list to start from (0 for start). page_size @@ -3302,6 +3322,8 @@ async def _async_get_metadata_element_relationships( f"{base_path(self, self.view_server)}/metadata-elements/{end1_guid}/linked-by-type/" f"{relationship_type}/to-elements/{end2_guid}" ) + body = _with_query_options(body, startFrom=start_from, pageSize=page_size, forLineage=for_lineage, + forDuplicateProcessing=for_duplicate_processing) response: Response = await self._async_make_request( "POST", url, body_slimmer(body), timeout=timeout @@ -3342,7 +3364,7 @@ def get_metadata_element_relationships( for_duplicate_processing: bool, default is set by server - Normally false. Set true when the caller is part of a deduplication function starting_at_end: int, default = 0 - - Relationship end to start from. + - Ignored: this endpoint fixes both ends in the URL. Kept so existing calls still work. start_from: int, default = 0 - index of the list to start from (0 for start). page_size diff --git a/pyegeria/view/_output_format_models.py b/pyegeria/view/_output_format_models.py index 4115a590..1efe9de0 100644 --- a/pyegeria/view/_output_format_models.py +++ b/pyegeria/view/_output_format_models.py @@ -68,7 +68,7 @@ import os from pathlib import Path from typing import Dict, List, Optional, Union, Any -from pydantic import BaseModel, Field, validator, root_validator +from pydantic import BaseModel, Field, model_validator, validator, root_validator from loguru import logger __all__ = [ 'Column', @@ -237,7 +237,7 @@ class ActionParameter(BaseModel): here as more analytic functions are added. Attributes: - function: The (find) function to call + function: The (find) function to call. Optional when analytic_function is set. required_params: Parameters that are required from the user, for `function` optional_params: Parameters that are optional from the user, for `function` spec_params: Parameters that are fixed for this action, for `function` @@ -246,7 +246,7 @@ class ActionParameter(BaseModel): (from `extra_constraints`) -- layered under whatever arbitrary parameters the caller supplies at execution time. """ - function: str + function: Optional[str] = None required_params: List[str] = Field(default_factory=list) optional_params: Optional[List[str]] = Field(default_factory=list) spec_params: Dict[str, Any] = Field(default_factory=dict) @@ -261,6 +261,16 @@ def _migrate_legacy_user_params(cls, values): values['required_params'] = values.pop('user_params') return values + @model_validator(mode="after") + def _needs_a_function(self): + """`function` may be omitted only when `analytic_function` is set (ISSUE-141). + + An explicit empty string is still accepted, as it was when `function` was required. + """ + if self.function is None and not self.analytic_function: + raise ValueError("ActionParameter needs a function or an analytic_function") + return self + class QuestionSpec(BaseModel): """ diff --git a/pyegeria/view/analytic_demo_specs.py b/pyegeria/view/analytic_demo_specs.py index 2dd5d43b..99d13cb4 100644 --- a/pyegeria/view/analytic_demo_specs.py +++ b/pyegeria/view/analytic_demo_specs.py @@ -75,7 +75,7 @@ def _demo( # never matches the data. formats=[Format(types=types, attributes=[])], action=ActionParameter( - function="", analytic_function=spec.function, analytic_spec_params=demo_params, + analytic_function=spec.function, analytic_spec_params=demo_params, ), ) diff --git a/pyegeria/view/format_set_executor.py b/pyegeria/view/format_set_executor.py index 5a85c345..cd6a2d1d 100644 --- a/pyegeria/view/format_set_executor.py +++ b/pyegeria/view/format_set_executor.py @@ -111,18 +111,23 @@ def _resolve_analytic_function(func_decl: str): _ANALYTIC_CLIENT_PARAM_NAMES = {"mgr", "ce", "expert", "client"} -def _bind_client_args(func, client: Any) -> list: - """Return the positional args to pass before **kwargs: one copy of `client` - per leading parameter named like a client (mgr/ce/expert/client).""" - args = [] +def _count_client_params(func) -> int: + """Number of leading parameters named like a client (mgr/ce/expert/client).""" + count = 0 for name, param in inspect.signature(func).parameters.items(): if name in _ANALYTIC_CLIENT_PARAM_NAMES and param.kind in ( inspect.Parameter.POSITIONAL_ONLY, inspect.Parameter.POSITIONAL_OR_KEYWORD, ): - args.append(client) + count += 1 else: break - return args + return count + + +def _bind_client_args(func, client: Any) -> list: + """Return the positional args to pass before **kwargs: one copy of `client` + per leading parameter named like a client (mgr/ce/expert/client).""" + return [client] * _count_client_params(func) def run_analytic_action( @@ -197,13 +202,16 @@ def _run_analytic_function( call_params.update({k: v for k, v in params.items() if v not in (None, "")}) func = _resolve_analytic_function(func_decl) + n_clients = _count_client_params(func) + if n_clients == 0: + # The routine takes no client, so don't connect to Egeria or mint a token (ISSUE-137). + return func(**call_params) + client = EgeriaTech(view_server, view_url, user_id=user, user_pwd=user_pass) if token: - client = EgeriaTech(view_server, view_url, user_id=user, user_pwd=user_pass) client.set_bearer_token(token) else: - client = EgeriaTech(view_server, view_url, user_id=user, user_pwd=user_pass) client.create_egeria_bearer_token() - return func(*_bind_client_args(func, client), **call_params) + return func(*([client] * n_clients), **call_params) def _resolve_action_target_client(egeria_client: EgeriaTech, client_class: type) -> Any: diff --git a/scripts/gen_element_status_fields.py b/scripts/gen_element_status_fields.py new file mode 100644 index 00000000..2ecc7106 --- /dev/null +++ b/scripts/gen_element_status_fields.py @@ -0,0 +1,81 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +# Copyright Contributors to the ODPi Egeria project. +""" +Generate md_processing/md_processing_utils/element_status_fields.py from Egeria's +Java source: for every open-metadata Properties class, which of the element +status properties (contentStatus, activityStatus, deploymentStatus) it has, +directly or through its superclasses. + +Egeria removed `initialStatus` from new-element requests (2025-12-15, +deb8d14dc0); each type now carries its own kind of status, or none (ISSUE-113). +Re-run this when Egeria's type model changes: + + python scripts/gen_element_status_fields.py [--egeria ~/localGit/egeria-v6/egeria] +""" +import argparse +import glob +import os +import re +import subprocess + +STATUS_FIELDS = ("contentStatus", "activityStatus", "deploymentStatus") +OUT = os.path.join(os.path.dirname(__file__), "..", "md_processing", "md_processing_utils", + "element_status_fields.py") + + +def scan(egeria_root: str) -> tuple[dict, dict]: + parent, declares = {}, {} + pattern = os.path.join(egeria_root, "open-metadata-implementation", "frameworks", + "**", "src", "main", "java", "**", "*Properties.java") + for path in glob.glob(pattern, recursive=True): + src = open(path, encoding="utf-8", errors="ignore").read() + m = re.search(r"public\s+(?:abstract\s+)?class\s+(\w+)\s+extends\s+(\w+)", src) + if not m: + continue + cls = m.group(1) + parent[cls] = m.group(2) + declares[cls] = {f for f in STATUS_FIELDS + if re.search(rf"(private|protected)\s+[\w<>]+\s+{f}\b", src)} + return parent, declares + + +def fields_for(cls: str, parent: dict, declares: dict) -> list[str]: + found, seen = set(), set() + while cls and cls not in seen: + seen.add(cls) + found |= declares.get(cls, set()) + cls = parent.get(cls) + return [f for f in STATUS_FIELDS if f in found] + + +def main(): + ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("--egeria", default=os.path.expanduser("~/localGit/egeria-v6/egeria")) + args = ap.parse_args() + parent, declares = scan(args.egeria) + table = {c: fields_for(c, parent, declares) for c in sorted(parent)} + head = subprocess.run(["git", "-C", args.egeria, "log", "-1", "--format=%h %cs"], + capture_output=True, text=True).stdout.strip() or "unknown" + lines = [ + "# SPDX-License-Identifier: Apache-2.0", + "# Copyright Contributors to the ODPi Egeria project.", + '"""', + "Status properties each Egeria Properties class has (ISSUE-113).", + "", + "GENERATED by scripts/gen_element_status_fields.py from Egeria source at", + f"{head} -- do not edit by hand; re-run the script instead. A class with", + "no status property (actors, plain assets, comments, ...) maps to ().", + '"""', + "", + "STATUS_FIELDS_BY_PROPERTIES_CLASS: dict[str, tuple[str, ...]] = {", + ] + lines += [f" {c!r}: ({''.join(repr(x) + ', ' for x in f).rstrip()})," for c, f in table.items()] + lines += ["}", ""] + with open(OUT, "w") as out: + out.write("\n".join(lines)) + print(f"wrote {len(table)} classes ({sum(1 for f in table.values() if f)} with a status) to {os.path.normpath(OUT)} (Egeria {head})") + + +if __name__ == "__main__": + main() diff --git a/tests/micro-tests/test_format_set_executor_token.py b/tests/micro-tests/test_format_set_executor_token.py index d4ea5360..6f691ded 100644 --- a/tests/micro-tests/test_format_set_executor_token.py +++ b/tests/micro-tests/test_format_set_executor_token.py @@ -138,6 +138,24 @@ def test_chart_path_threads_token(monkeypatch): assert client.token == "user-token" +def test_analytic_routine_without_client_param_builds_no_client(monkeypatch): + """ISSUE-137: a routine that takes no mgr/ce/expert/client parameter must not + cause an EgeriaTech to be built or a token minted -- an unreachable view_url + must not stop it running.""" + _patch_analytic_path(monkeypatch) + + def _local_only(window=0): + return {"count": 7, "window": window} + + monkeypatch.setattr(fse, "_resolve_analytic_function", lambda decl: _local_only) + result = fse.exec_report_spec( + "Anything", output_format="DICT", view_server="vs", view_url="https://unreachable", + user="svc", user_pass="svc-pw", params={"window": 2}, + ) + assert result == {"kind": "json", "data": {"count": 7, "window": 2}} + assert _RecordingClient.instances == [] + + # --- MCP adapter passthrough -------------------------------------------------- def test_mcp_run_report_forwards_token(monkeypatch): diff --git a/tests/micro-tests/test_issue_111_link_default_properties_class.py b/tests/micro-tests/test_issue_111_link_default_properties_class.py new file mode 100644 index 00000000..02f5e880 --- /dev/null +++ b/tests/micro-tests/test_issue_111_link_default_properties_class.py @@ -0,0 +1,61 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright Contributors to the ODPi Egeria project. +""" +ISSUE-111: with body=None, _async_new_relationship_request builds +{"properties": {"class": prop[0]}}. Eleven ConnectionMaker/DataDiscovery link +methods passed the relationship type name (or a near-miss) as prop[0], so the +default body named a properties class Egeria does not have and the link was +refused. Each must name the class the Egeria REST service checks for with +`instanceof` (ConnectionMakerRESTServices / DataDiscoveryRESTServices). +""" +import json + +import pytest + +from pyegeria.omvs.connection_maker import ConnectionMaker +from pyegeria.omvs.data_discovery import DataDiscovery + + +class _Response: + def json(self): + return {"class": "GUIDResponse", "guid": "rel-guid"} + + +def _client(cls, sent: list, monkeypatch): + monkeypatch.setattr(cls, "check_connection", lambda self: "test") + client = cls("view-server", "https://localhost:1", "user", "pwd") + + async def fake_make_request(method, url, body=None, *args, **kwargs): + sent.append(json.loads(body) if isinstance(body, str) else body) + return _Response() + + client._async_make_request = fake_make_request + return client + + +CASES = [ + (ConnectionMaker, "_async_link_connection_connector_type", "ConnectionConnectorTypeProperties"), + (ConnectionMaker, "_async_link_connection_endpoint", "ConnectToEndpointProperties"), + (ConnectionMaker, "_async_link_embedded_connection", "EmbeddedConnectionProperties"), + (ConnectionMaker, "_async_link_asset_to_connection", "ResourceConnectionProperties"), + (ConnectionMaker, "_async_link_resource_to_connection", "ResourceConnectionProperties"), + (ConnectionMaker, "_async_link_endpoint_to_it_asset", "ServerEndpointProperties"), + (DataDiscovery, "_async_attach_annotation_to_report", "ReportedAnnotationProperties"), + (DataDiscovery, "_async_link_annotation_to_described_element", "AssociatedAnnotationProperties"), + (DataDiscovery, "_async_link_annotation_to_its_predecessor", "AnnotationExtensionProperties"), + (DataDiscovery, "_async_link_resource_profile_data", "ResourceProfileDataProperties"), + (DataDiscovery, "_async_link_annotation_match", "AnnotationMatchProperties"), + (DataDiscovery, "_async_link_request_for_action_target", "RequestForActionTargetProperties"), +] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("cls, method, expected", CASES, ids=[c[1] for c in CASES]) +async def test_default_body_names_real_properties_class(cls, method, expected, monkeypatch): + sent: list = [] + client = _client(cls, sent, monkeypatch) + + await getattr(client, method)("guid-1", "guid-2", None) + + assert sent[0]["class"] == "NewRelationshipRequestBody" + assert sent[0]["properties"]["class"] == expected diff --git a/tests/micro-tests/test_issue_113_element_status.py b/tests/micro-tests/test_issue_113_element_status.py new file mode 100644 index 00000000..ced6ef80 --- /dev/null +++ b/tests/micro-tests/test_issue_113_element_status.py @@ -0,0 +1,124 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright Contributors to the ODPi Egeria project. +""" +ISSUE-113: Egeria removed `initialStatus` from new-element requests, so Dr.Egeria +must put a status in the property each type actually has -- contentStatus, +activityStatus, deploymentStatus -- or nowhere, for types with none. +""" +from md_processing.md_processing_utils.common_md_utils import set_create_body, set_element_prop_body +from md_processing.md_processing_utils.element_status import element_status_properties + + +def _attrs(**kv): + out = {} + for k, v in kv.items(): + name = k.replace("_", " ") + out[name] = v if isinstance(v, dict) else {"value": v} + return out + + +def test_written_status_goes_to_content_status_for_authored_content(): + assert element_status_properties("GlossaryTermProperties", _attrs(Status="DRAFT")) == {"contentStatus": "DRAFT"} + + +def test_written_status_goes_to_activity_status_for_actions(): + assert element_status_properties("ToDoProperties", _attrs(Status="IN_PROGRESS")) == { + "activityStatus": "IN_PROGRESS"} + + +def test_written_status_goes_to_deployment_status_for_infrastructure(): + assert element_status_properties("SoftwareServerProperties", _attrs(Status="STANDBY")) == { + "deploymentStatus": "STANDBY"} + + +def test_type_with_two_fields_takes_the_first_that_accepts_the_value(): + # SolutionComponent has contentStatus and deploymentStatus; DRAFT is only a content value. + assert element_status_properties("SolutionComponentProperties", _attrs(Status="DRAFT")) == { + "contentStatus": "DRAFT"} + assert element_status_properties("SolutionComponentProperties", _attrs(Status="UNDER_DEVELOPMENT")) == { + "deploymentStatus": "UNDER_DEVELOPMENT"} + + +def test_type_without_a_status_gets_none(): + assert element_status_properties("ActorProfileProperties", _attrs(Status="DRAFT")) == {} + assert element_status_properties("AssetProperties", _attrs(Content_Status="DRAFT")) == {} + + +def test_value_invalid_for_the_type_is_dropped(): + assert element_status_properties("ToDoProperties", _attrs(Status="DRAFT")) == {} + + +def test_explicit_attribute_wins_over_generic_status(): + attrs = _attrs(Status="ACTIVE", Content_Status="PROPOSED") + assert element_status_properties("GlossaryTermProperties", attrs) == {"contentStatus": "PROPOSED"} + + +def test_default_status_only_ever_sets_content_status(): + default = {"Status": {"value": "ACTIVE", "is_default": True}} + assert element_status_properties("GlossaryTermProperties", default) == {"contentStatus": "ACTIVE"} + # ACTIVE is a valid deployment status, but a default must not start overwriting it. + assert element_status_properties("SoftwareServerProperties", default) == {} + + +def test_written_status_beats_a_defaulted_content_status(): + # What `dr_egeria --validate` showed: Content Status carries its own spec default of ACTIVE. + attrs = {"Status": {"value": "DRAFT"}, "Content Status": {"value": "ACTIVE", "is_default": True}} + assert element_status_properties("DataStructureProperties", attrs) == {"contentStatus": "DRAFT"} + + +def test_written_status_reaches_deployment_status_past_a_defaulted_content_status(): + attrs = {"Status": {"value": "UNDER_DEVELOPMENT"}, "Content Status": {"value": "ACTIVE", "is_default": True}} + assert element_status_properties("SolutionComponentProperties", attrs) == { + "deploymentStatus": "UNDER_DEVELOPMENT", "contentStatus": "ACTIVE"} + + +def test_unknown_properties_class_keeps_old_behaviour(): + attrs = _attrs(Status="DRAFT", Activity_Status="WAITING") + assert element_status_properties("NoSuchThingProperties", attrs) == { + "contentStatus": "DRAFT", "activityStatus": "WAITING"} + + +def test_set_element_prop_body_uses_the_mapping(): + term = set_element_prop_body("Glossary Term", "qn", _attrs(Status="DRAFT")) + assert term["contentStatus"] == "DRAFT" + todo = set_element_prop_body("To Do", "qn", _attrs(Status="COMPLETED")) + assert todo["class"] == "ToDoProperties" + assert todo["activityStatus"] == "COMPLETED" and "contentStatus" not in todo + actor = set_element_prop_body("Actor Profile", "qn", _attrs(Status="DRAFT")) + assert "contentStatus" not in actor + + +def test_create_body_no_longer_sends_initial_status(): + assert "initialStatus" not in set_create_body("Glossary Term", _attrs(Status="DRAFT")) + + +async def test_template_create_no_longer_sends_initial_status(): + from typing import Any, cast + + from md_processing.v2.asset_maker import AssetMakerProcessor + from md_processing.v2.extraction import DrECommand + + bodies = [] + + class _Curation: + async def _async_create_elem_from_template(self, body): + bodies.append(body) + return "cccccccc-0000-0000-0000-000000000003" + + class _Client: + automated_curation = _Curation() + + async def _rendered(guid): + return guid + + p = AssetMakerProcessor(client=cast(Any, _Client()), + command=DrECommand(verb="Create", object_type="Element", attributes={}, + raw_block="## Create Element"), context={}) + p.get_command_spec = lambda: {} + p.parsed_output = {"qualified_name": None, "attributes": { + "Template GUID": {"value": "3f9a0ab3-072c-4cb1-a14f-6e0492e00dd7"}, "Initial Status": {"value": "DRAFT"}}} + p.render_result_markdown = _rendered + + await p.apply_changes() + + assert "initialStatus" not in bodies[0] diff --git a/tests/micro-tests/test_issue_114_guid_or_none.py b/tests/micro-tests/test_issue_114_guid_or_none.py new file mode 100644 index 00000000..7cbf5957 --- /dev/null +++ b/tests/micro-tests/test_issue_114_guid_or_none.py @@ -0,0 +1,22 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright Contributors to the ODPi Egeria project. +""" +ISSUE-114: get_guid_for_name signals a miss with a truthy sentinel string, so +`if not guid` reads a miss as found. guid_or_none turns a miss into None. +""" +import pytest + +from pyegeria import guid_or_none +from pyegeria.core._globals import NO_ELEMENTS_FOUND, NO_ELEMENT_FOUND, NO_GUID_RETURNED + + +def test_guid_passes_through(): + g = "a7d4c3b2-1e0f-4a5b-9c8d-7e6f5a4b3c2d" + assert guid_or_none(g) == g + assert guid_or_none(f" {g}\n") == g + + +@pytest.mark.parametrize("miss", [NO_ELEMENTS_FOUND, NO_ELEMENT_FOUND, NO_GUID_RETURNED, + None, "", " ", [], ["g1", "g2"], {"guid": "x"}]) +def test_miss_is_none(miss): + assert guid_or_none(miss) is None diff --git a/tests/micro-tests/test_issue_129_metadata_expert_relationship_paging.py b/tests/micro-tests/test_issue_129_metadata_expert_relationship_paging.py new file mode 100644 index 00000000..90fd7107 --- /dev/null +++ b/tests/micro-tests/test_issue_129_metadata_expert_relationship_paging.py @@ -0,0 +1,76 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright Contributors to the ODPi Egeria project. +""" +ISSUE-129: MetadataExpert's related-elements/type and linked-by-* queries took +start_from, page_size, for_lineage, for_duplicate_processing (and, for +related-elements, starting_at_end) and sent none of them. They now go in the +ResultsRequestBody (and startingAtEnd in the query string), without +overriding a field the caller's body already sets. +""" +import json + +import pytest + +from pyegeria.omvs.metadata_expert import MetadataExpert + + +class _Response: + def json(self): + return {"elementList": []} + + +@pytest.fixture +def client_and_calls(monkeypatch): + monkeypatch.setattr(MetadataExpert, "check_connection", lambda self: "test") + client = MetadataExpert("view-server", "https://localhost:1", "user", "pwd") + calls = [] + + async def fake_make_request(method, url, body=None, *args, **kwargs): + calls.append((url, json.loads(body) if isinstance(body, str) else body)) + return _Response() + + client._async_make_request = fake_make_request + return client, calls + + +@pytest.mark.asyncio +async def test_related_elements_by_type_sends_paging_end_and_lineage(client_and_calls): + client, calls = client_and_calls + + await client._async_get_related_metadata_elements( + "g1", "SolutionCompositionLink", None, for_lineage=True, starting_at_end=2, start_from=10, page_size=5) + + url, body = calls[0] + assert url.endswith("/related-elements/g1/type/SolutionCompositionLink?startingAtEnd=2") + assert body["startFrom"] == 10 and body["pageSize"] == 5 and body["forLineage"] is True + assert "forDuplicateProcessing" not in body + + +@pytest.mark.asyncio +async def test_body_fields_win_over_keyword_arguments(client_and_calls): + client, calls = client_and_calls + + await client._async_get_related_metadata_elements( + "g1", "T", {"class": "ResultsRequestBody", "pageSize": 7}, page_size=500) + + assert calls[0][1]["pageSize"] == 7 + + +@pytest.mark.asyncio +@pytest.mark.parametrize("method, args, path", [ + ("_async_get_all_metadata_element_relationships", ("e1", "e2"), "/metadata-elements/e1/linked-by-any-type/to-elements/e2"), + ("_async_get_metadata_element_relationships", ("e1", "e2", "T"), "/metadata-elements/e1/linked-by-type/T/to-elements/e2"), +]) +async def test_linked_by_queries_send_paging(client_and_calls, method, args, path): + client, calls = client_and_calls + fn = getattr(client, method) + + if method == "_async_get_metadata_element_relationships": + await fn(end1_guid=args[0], end2_guid=args[1], relationship_type=args[2], body=None, + start_from=3, page_size=4) + else: + await fn(args[0], args[1], None, start_from=3, page_size=4) + + url, body = calls[0] + assert url.endswith(path) + assert body["startFrom"] == 3 and body["pageSize"] == 4 diff --git a/tests/micro-tests/test_issue_134_dynamic_catch_reraise.py b/tests/micro-tests/test_issue_134_dynamic_catch_reraise.py new file mode 100644 index 00000000..d93119b4 --- /dev/null +++ b/tests/micro-tests/test_issue_134_dynamic_catch_reraise.py @@ -0,0 +1,43 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright Contributors to the ODPi Egeria project. +""" +ISSUE-134: with PYEGERIA_ENABLE_LOGGER_CATCH on, @dynamic_catch wrapped methods +in loguru's logger.catch, whose default reraise=False logged the exception and +returned None -- a failed read looked like an empty one. The exception must +still reach the caller, sync and async. +""" +import pytest + +from pyegeria.core import utils + + +@pytest.fixture +def logger_catch_on(monkeypatch): + monkeypatch.setattr(utils.app_settings.Debug, "enable_logger_catch", True) + + +def test_sync_failure_still_raises(logger_catch_on): + @utils.dynamic_catch + def boom(): + raise ValueError("server said no") + + with pytest.raises(ValueError, match="server said no"): + boom() + + +@pytest.mark.asyncio +async def test_async_failure_still_raises(logger_catch_on): + @utils.dynamic_catch + async def boom(): + raise ValueError("server said no") + + with pytest.raises(ValueError, match="server said no"): + await boom() + + +def test_success_value_unchanged(logger_catch_on): + @utils.dynamic_catch + def ok(): + return "value" + + assert ok() == "value" diff --git a/tests/micro-tests/test_issue_136_note_type_defaults.py b/tests/micro-tests/test_issue_136_note_type_defaults.py new file mode 100644 index 00000000..cb3f4d1c --- /dev/null +++ b/tests/micro-tests/test_issue_136_note_type_defaults.py @@ -0,0 +1,81 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright Contributors to the ODPi Egeria project. +""" +ISSUE-136: the note reads defaulted their type filter to "Action". Note is a +subtype of Notification, which is a subtype of Action, so Notes still came +back -- but so did every other Action subtype (a note log can hold +Notifications that are not Notes, and find_notes searches all assets). The +defaults are now "Note"; a caller can still pass another type. +""" +import json + +import pytest + +from pyegeria.core._server_client import ServerClient + + +class _Response: + def __init__(self, payload): + self._payload = payload + + def json(self): + return self._payload + + +def _client(calls: list, payload: dict, monkeypatch) -> ServerClient: + monkeypatch.setattr(ServerClient, "check_connection", lambda self: "test") + client = ServerClient("view-server", "https://localhost:1", "user", "pwd") + + async def fake_make_request(method, url, body=None, *args, **kwargs): + calls.append((url, json.loads(body) if isinstance(body, str) else body)) + return _Response(payload) + + client._async_make_request = fake_make_request + return client + + +@pytest.mark.asyncio +async def test_get_notes_for_note_log_defaults_to_note(monkeypatch): + calls: list = [] + client = _client(calls, {"elements": []}, monkeypatch) + + await client._async_get_notes_for_note_log("log-guid") + + url, body = calls[0] + assert url.endswith("feedback-manager/note-logs/log-guid/notes/retrieve") + assert body["metadataElementTypeName"] == "Note" + + +@pytest.mark.asyncio +async def test_get_notes_for_note_log_type_can_be_widened(monkeypatch): + calls: list = [] + client = _client(calls, {"elements": []}, monkeypatch) + + await client._async_get_notes_for_note_log("log-guid", metadata_element_type_name="Notification") + + assert calls[0][1]["metadataElementTypeName"] == "Notification" + + +@pytest.mark.asyncio +async def test_get_note_by_guid_defaults_to_note(monkeypatch): + calls: list = [] + client = _client(calls, {"element": {}}, monkeypatch) + + await client._async_get_note_by_guid("note-guid") + + url, body = calls[0] + # Served by Asset Maker's /assets/{assetGUID}/retrieve under the feedback-manager url marker. + assert url.endswith("feedback-manager/assets/note-guid/retrieve") + assert body["metadataElementTypeName"] == "Note" + + +@pytest.mark.asyncio +async def test_find_notes_defaults_to_note(monkeypatch): + calls: list = [] + client = _client(calls, {"elements": []}, monkeypatch) + + await client._async_find_notes("*") + + url, body = calls[0] + assert url.endswith("feedback-manager/assets/by-search-string") + assert body["metadataElementTypeName"] == "Note" diff --git a/tests/micro-tests/test_issue_141_action_function_optional.py b/tests/micro-tests/test_issue_141_action_function_optional.py new file mode 100644 index 00000000..91da2124 --- /dev/null +++ b/tests/micro-tests/test_issue_141_action_function_optional.py @@ -0,0 +1,36 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright Contributors to the ODPi Egeria project. +""" +ISSUE-141: ActionParameter.function was a required string, so an analytic-only +spec had to pass function="". It is now optional when analytic_function is set. +""" +import pytest +from pydantic import ValidationError + +from pyegeria.view._output_format_models import ActionParameter + + +def test_analytic_only_action_needs_no_function(): + action = ActionParameter(analytic_function="pyegeria.view.overview_metrics.growth_series") + assert action.function is None + + +def test_find_only_action_unchanged(): + assert ActionParameter(function="EgeriaTech.find_projects").analytic_function is None + + +def test_empty_string_function_still_accepted(): + assert ActionParameter(function="").function == "" + + +def test_action_with_neither_function_is_rejected(): + with pytest.raises(ValidationError): + ActionParameter() + + +def test_analytic_demo_specs_build_without_function(): + from pyegeria.view.analytic_demo_specs import get_analytic_demo_specs + + specs = get_analytic_demo_specs() + assert specs + assert all(s.action.function is None and s.action.analytic_function for s in specs.values())