Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion tools/migrate-apim/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@ Declares how APIM shapes map to target capabilities. Never versioned separately;

- **`structural[]`** — Rules for cross-asset things (workspaces, subscriptions, telemetry, coverage gaps).

- **`policies[]`** — How APIM policies map to AI Gateway policies. Matched by element name, then by family, with fallbacks for expressions and unknowns.
- **`policies[]`** — How APIM policies map to AI Gateway policies. Four passes in order: by element name, then by family, then a fallback for anything carrying a policy expression, then one terminal rule (`match: {anyElement: true}`) that answers for everything else.

The last pass is what makes an unmapped element impossible rather than unlikely: before it, an element matching no rule was dropped where it was classified, and the ones that vanished were those carrying *no* expression, since an expression at least reached the pass above. It is a net and not a mapping, so `compile` holds every element in `policy.FAMILIES` to a rule of its own or a family rule and fails if one is left to it. The planner counts what reaches it into `policyCoverage.unrecognised` on the plan.

- **`publishableRoutes[]`** — The four routes AI Gateway publishes (`/openai/v1/chat/completions`, `/v1/messages`, etc.). Used to decide what part of a model API surface can move.

Expand Down
35 changes: 31 additions & 4 deletions tools/migrate-apim/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -436,9 +436,29 @@ model, or a tool server. APIM has well over a hundred. Most of the migration's r
| `azure-openai-content-safety`, `llm-content-safety` | — | **Gap.** Content-safety flags exist in the CLI but are not supported on typed `aigateways`. The policy also pulls in a separate Content Safety backend, surfaced as a `supportingBackends` dependency |
| Transformations: `set-body`, `set-header`, `rewrite-uri`, `json-to-xml` | — | **Gap.** No request/response rewriting |
| `send-request`, `return-response`, custom `@{…}` expressions | — | **Gap.** `agentHooks` is the nearest mechanism, and only for tool traffic |
| `authentication-basic`, `authentication-certificate`, `get-authorization-context` | — | **Gap.** A provider authenticates with a managed identity or an API key; `model-provider create/update` exposes no basic credential, no client certificate, and no credential-manager equivalent |
| `check-header`, `validate-client-certificate` | — | **Gap.** Caller authentication is Entra auth plus RBAC, or an API key. A gate on a shared header secret or a client certificate is neither |
| `emit-metric`, `log-to-eventhub` | — | **Gap.** `telemetry-exporter create` takes a kind and an endpoint, not a metric to emit or a sink to write to |
| `set-service-url` | — | **Gap.** An alias backend pool holds backends the gateway knows; overriding the URL outright names no backend |
| `mcp-server` | `mcp create` | **Handled.** Collected as an MCP asset from the API that carries it, and registered by the MCP server item |
| `validate-mcp-request` | — | **Gap.** `mcp create` registers a server, its endpoints and its versions; guarding a tool call on the way through is not part of that surface |
| `cors`, `trace`, correlation headers | — | **Not applicable** |
| Anything else APIM ships | — | **Gap, blocking.** Reported rather than assumed harmless — see below |
| — | `costLimit` | AI Gateway capability with no APIM source; an upgrade, not a gap |

**Nothing is dropped for want of a rule.** Rule selection ends in a terminal rule that matches
anything the four passes above it did not, so an element this catalog has never heard of reaches the
plan as a blocking decision instead of being discarded where it was classified. That hole was real:
eleven elements matched no rule at all, three of them carrying backend credentials, and because an
element containing an expression at least fell into the pass above, the policies that vanished most
completely were the simplest ones.

The net is not allowed to become the mapping. `compile` holds every element in `policy.FAMILIES` —
the vocabulary the collector classifies against — to a rule of its own or a rule for its family, and
fails naming any that are left to the terminal rule. What legitimately reaches it is counted on the
plan under `policyCoverage`, which accounts for every element walked under exactly one disposition
and sums them, so a reader can check the planner against the inventory rather than trust it.

### Known structural gaps

These are not policy details; they change what a migration *means*, and each one should surface as
Expand Down Expand Up @@ -759,9 +779,15 @@ checked, so a journal carrying a stale or wrong phase status repairs itself on t
There is no transaction across steps, so the cost of finding a problem halfway is a half-built
gateway. `--preflight` refuses while any of these stands: the plan names no target; an item still
needs a decision; a settled decision has been orphaned; a referenced credential is missing or not
mechanically retrievable; or a `Microsoft.ApiManagement/service` shares the target's name, which
matters because several CLI commands dispatch by name without `--ids`. Manual prerequisites are
raised as notes and acknowledged with `--acknowledge <id>`.
mechanically retrievable; the plan describes work it cannot itself produce; a step names a provider
nothing in the plan registers; an object the planner marked as not fully configurable would become a
step; or a `Microsoft.ApiManagement/service` shares the target's name, which matters because several
CLI commands dispatch by name without `--ids`. Manual prerequisites are raised as notes and
acknowledged with `--acknowledge <id>`.

That last one is the plan's own judgement, read back. `plan` records why it could not finish shaping
an object, and until `apply` consulted it the condition surfaced as a step failing part-way through
a phase — under a cause the builder guessed at, with every step queued behind it skipped.

### Phases

Expand Down Expand Up @@ -792,7 +818,8 @@ Phase models -- EXECUTING

**`policies` writes two different things**, because the gateway holds policy in two places. A token
limit lives on a *model registration*, so an APIM policy scoped to an API, a product or the service
is expanded at planning time into one object per model it covered — one object is one
is expanded at planning time into one object per model it covered — a policy factored into a
*fragment* covers whatever includes it, and is followed there first — one object is one
`az aigateway model policy set`. Policies batch by counter key, so a batch can hold two limits with
different ceilings; each member keeps its own. On a real instance that was 10000/minute on one API
and 1000 on another, and shaping only the first member applied one ceiling to both.
Expand Down
92 changes: 90 additions & 2 deletions tools/migrate-apim/capabilities/rules.json
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,61 @@
"rationale": "No schema, header, or status validation on the target.",
"proposal": "retain-in-apim"
},
{
"match": {
"family": "backendAuthentication"
},
"verdict": "gap",
"itemKind": "accept-loss",
"rationale": "A model provider authenticates with a managed identity or an API key. `model-provider create` and `model-provider update` expose no basic credential, no client certificate, and nothing equivalent to an APIM credential manager, so a backend secured any other way has no target. `authentication-managed-identity` is the one member of this family that does map, and it has a rule of its own.",
"proposal": "retain-in-apim"
},
{
"match": {
"family": "authentication"
},
"verdict": "gap",
"itemKind": "accept-loss",
Comment on lines +290 to +294
"rationale": "Caller authentication on the gateway is Entra auth plus data-plane RBAC, or an API key. A gate on a shared secret in a header, or on a client certificate, is neither. `validate-jwt` and `validate-azure-ad-token` are the members of this family that do map, and they have a rule of their own.",
"proposal": "retain-in-apim"
},
{
"match": {
"family": "observability"
},
"verdict": "gap",
"itemKind": "accept-loss",
"rationale": "The gateway reports through a telemetry exporter it owns: `telemetry-exporter create` takes a kind and an endpoint, not a metric to emit or a sink to write to. There is no Event Hub destination and no way to author a custom metric. Where the gateway's own telemetry reports is a separate decision, on the telemetry destination item.",
"proposal": "accept"
},
{
"match": {
"element": [
"mcp-server"
]
},
"verdict": "handled",
"capabilityId": "mcp.create",
"rationale": "The tool server this policy exposes is collected as an MCP asset from the API that carries it, and registered by the MCP server item rather than raised again here."
},
{
"match": {
"family": "mcp"
},
"verdict": "gap",
"itemKind": "accept-loss",
"rationale": "`mcp create` registers a tool server, its endpoints and its supported versions. Guarding a tool call on the way through is not part of that surface. `mcp-server`, which declares the server itself, is carried by the MCP server item and has a rule of its own.",
"proposal": "retain-in-apim"
},
{
"match": {
"family": "routing"
},
"verdict": "gap",
"itemKind": "accept-loss",
"rationale": "An alias backend pool holds backends the gateway knows, with a priority and a weight each. Overriding the destination URL outright names no backend, so there is nothing for the pool to hold. `set-backend-service` and `forward-request` choose among backends and have a rule of their own.",
"proposal": "retain-in-apim"
},
{
"match": {
"element": [
Expand All @@ -300,7 +355,30 @@
},
"verdict": "handled",
"capabilityId": "alias.create",
"rationale": "Backend selection becomes the alias backend pool."
"rationale": "Backend selection becomes the alias backend pool.",
"partialWhen": {
"hasExpression": true,
"underConditional": true
},
"partial": {
"itemKind": "accept-loss",
"options": [
{
"value": "accept",
"consequence": "Every call takes the alias backend pool in its own order, whatever the source would have chosen for it."
},
{
"value": "compensate",
"consequence": "Say what the condition was for in the priority and weight of the alias backend models, as far as it can be said that way."
Comment on lines +371 to +372
},
{
"value": "retain-in-apim",
"consequence": "Keep this workload on APIM, where the routing logic still runs."
}
],
"proposal": "compensate",
"rationale": "The source decides where the request goes with logic the target has no place for: a policy expression, a branch of a `choose`, or both. The alias backend pool carries the set of backends and their priority and weight, not the rule that picked among them."
}
},
{
"match": {
Expand All @@ -325,9 +403,19 @@
},
"verdict": "gap",
"itemKind": "accept-loss",
"note": "Last of the three passes in capability.policy_rule, so it is reached only by an element with no rule of its own and no family rule. The ordering is in the code, not in a priority here.",
"note": "Third of the four passes in capability.policy_rule, so it is reached only by an element with no rule of its own and no family rule. The ordering is in the code, not in a priority here.",
"rationale": "A policy expression doing more than reading a well-known context property has no equivalent. The target's policy surface is six declarative types.",
"proposal": "retain-in-apim"
},
{
"match": {
"anyElement": true
},
"verdict": "gap",
"itemKind": "accept-loss",
"note": "Last of the four passes in capability.policy_rule, and the one that makes an unmapped element impossible rather than merely unlikely. `compile` holds every element in policy.FAMILIES to a rule of its own or a family rule, so nothing reaching here is a policy the tool has ever been taught; it is a policy APIM has that this catalog does not know about. Deliberately blocking: the alternative is a migration that silently drops a policy nobody has read.",
"rationale": "This policy is not in the vocabulary this catalog was written against, so nothing can be said about what it does or whether the target has an equivalent. It is reported rather than assumed harmless.",
"proposal": "retain-in-apim"
}
],
"structural": [
Expand Down
53 changes: 53 additions & 0 deletions tools/migrate-apim/migrate_apim/apply.py
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,35 @@ def finding(code: str, blocking: bool, message: str, remedy: str = "") -> None:
"Re-plan from a current inventory, or settle the items that register those providers.",
)

# Work the plan has already judged unappliable. `plan` detects it, explains
# it and writes the reason on the object, and nothing read it back: the
# condition surfaced as a step failing part-way through a phase, under a
# builder's guess at a cause the plan could name exactly. A failed step stops
# its phase, so one such object also costs every step queued behind it.
#
# Asked of `steps_for` rather than re-derived from the items, so the objects
# checked are the objects that run. One no phase performs cannot fail a
# phase, and blocking on it would stop a whole migration over work the
# operator was going to apply by hand -- which `notExecutable` already names.
incomplete = [
(step, obj)
for step in steps_for(plan)
for obj in step.objects
# Only an explicit false. The field is absent from most objects, and
# reading absence as incomplete would block every plan there is.
if obj.get("configurationComplete") is False
]
if incomplete:
finding(
"incompleteObject", True,
f"{len(incomplete)} object(s) the plan marked as not fully configurable: "
+ "; ".join(_incomplete_note(step, obj) for step, obj in incomplete[:3])
+ ("..." if len(incomplete) > 3 else ""),
"Re-plan from a current inventory, or exclude the items named above to leave their "
"objects out of the run. Applied as it stands, each one fails its step and stops the "
"rest of its phase.",
Comment on lines +272 to +274
)

unbuildable: dict[str, list[str]] = {}
for item in plan.get("items") or []:
capability_id = item.get("capabilityId")
Expand Down Expand Up @@ -277,6 +306,30 @@ def finding(code: str, blocking: bool, message: str, remedy: str = "") -> None:
return findings


def _incomplete_note(step: Step, obj: dict[str, Any]) -> str:
"""One unappliable object, in the planner's own words.

The reason is quoted rather than restated, because the plan knows it and a
builder does not: `model policy set` can say only that an object names no
model registration, where the plan can say which scope covered none.
`fullConfigurationAt` comes with it, since the remedy usually starts at the
policy document the element was read from.
"""
label = _object_label(obj)
where = f"{step.item_id} ({label})" if label else step.item_id
reason = str(obj.get("incompleteReason") or "").strip()
if not reason:
# Two planner paths mark an object incomplete from empty settings alone
# and write no reason. Reporting that silence beats quoting an empty
# string, and naming a cause this check cannot see would be a guess.
reason = "the plan marked it incomplete and said no more"
if not obj.get("settings"):
reason += "; it carries no settings"
at = obj.get("fullConfigurationAt") or {}
source = " ".join(str(part) for part in (at.get("policyId"), at.get("path")) if part)
return f"{where}: {reason}" + (f" [{source}]" if source else "")


def _role_grant_check(
plan: dict[str, Any], target: Target, runner: Runner
) -> list[dict[str, Any]]:
Expand Down
Loading
Loading