Skip to content

knowledge(web-services): describe API objects intended for MCP tools - #233

Open
Javier Armesto Gonzalez (javiarmesto) wants to merge 4 commits into
microsoft:mainfrom
javiarmesto:claude/api-page-abouttext-for-mcp
Open

Javier Armesto Gonzalez (javiarmesto) wants to merge 4 commits into
microsoft:mainfrom
javiarmesto:claude/api-page-abouttext-for-mcp

Conversation

@javiarmesto

@javiarmesto Javier Armesto Gonzalez (javiarmesto) commented Oct 16, 2026 •

Copy link
Copy Markdown

API objects intended for the Business Central MCP server can expose a bare entity caption where an agent needs to understand document filters or computed values. Add web-services knowledge for the documented description precedence: AboutText, EntitySetCaption, then camel-cased EntityName. The good/bad samples cover both API pages and queries, with descriptions and actual document-type filters kept consistent.

The article preserves valid alternatives: missing AboutText is not a defect when the fallback is adequate or MCP use is not established, and AboutTitle is not required solely for MCP. Sources include the Learn API page/query documentation and a pinned Microsoft API query with AboutText and no AboutTitle. Its scope is BC 28+ online environments with the relevant MCP support; it does not claim that extraction first appeared in 28.0 or cover the earlier preview.

The contribution is motivated by a 22-run API generation evaluation whose syntax rubric flagged AboutText omissions in 13 runs, including one incomplete solution. These are omission flags, not measured tool-selection failures. The missing metadata is not diagnosed by the tested standard analyzers. The owning domain is web-services because the concern is the API-to-tool metadata contract, distinct from UI teaching tips.

The evaluation now supplies Business Central online version 28 and established MCP exposure as per-article context for both the positive and clean samples. This context survives comment removal in every prepared request format and does not spill into other web-services articles. A required non-MCP clean control reuses the positive sample's exact neutral code under a REST-only context. This tests the applicability boundary without changing the article's rule or AL companions.

The harness accepts optional articleOverrides with a shared pair context and named cleanContexts. These deployment facts contain no expected answer, fix, or article ID. Invalid registrations fail validation; model-facing case IDs, result shape, candidate ranking, and scoring thresholds retain their existing contracts. The new regression check is wired into fixture CI.

The branch incorporates upstream main at a4593a1. Conflict resolution keeps
the upstream stream-encoding registration, documentation, and retrieval tokens
alongside the MCP article and context cases. A compatibility comparison
preserves all upstream fixture registrations and all 292 prepared cases, with
unchanged neutral code and deployment contexts; only the three MCP applicability
cases are added. The MCP article and its AL companions are unchanged from the
head approved by Jesper.

Validation:

  • Reproduced the original failure: the prepared positive case lacked deployment context. The regression check now passes for all three request formats, isolation from other articles, existing domain contexts, identical neutral code in the positive/non-MCP cases, clean-control scoring, missing-control rejection, and invalid overrides.
  • Frontmatter: zero errors; two existing keyword-count warnings concern unrelated articles. Fixture preparation and changed-file coverage pass with 295 cases, 147 selected paired articles, and 21 domains. Review-contract acceptance checks pass.
  • Skill-index/report-schema checks pass, preserving the declared review-leaf order and existing plan-guidance checks. Knowledge-index and bounded-retrieval checks pass for all 436 articles and 759 samples.
  • The unchanged good/bad AL companions were previously compiled independently against BC 28.2 symbols with AL compiler 18.0.43.1464, CodeCop, UICop, and PerTenantExtensionCop; only expected sample-filename warnings remained. Focused page/query probes accepted AboutText without AboutTitle.

No new model-driven review evaluation, live MCP extraction, or historical deployment test was run. The review-skill edit only selects this knowledge; the facts and exceptions remain in the article.

These validation results are local; GitHub workflow authorization and maintainer review remain separate from preparing this branch.

…t for MCP tools

The Business Central MCP server takes the tool description from
AboutText, then EntitySetCaption, then EntityName. API pages without
AboutText reach agents as a bare entity name. LLMs treat AboutText as
a UI teaching-tip property and omit it on API objects; no analyzer
flags it. Observed in 13 of 22 generated solutions of a read-only
SalesMargin query API.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed draft head 38bb24502a4a2806800f12d2d99691e7883b61f6. The documented MCP description precedence and carefully bounded rule are sound, but the registered positive fixture loses its prerequisite during neutral preparation.

The article explicitly requires established Business Central online MCP use and says missing AboutText alone is not a finding. In both companions the only evidence of intended MCP exposure is a full-line comment. Test-ReviewFixtures strips those comments, and the prepared requests for this pair contain only opaque id/file entries with no MCP context. The positive API page/query is then indistinguishable from an ordinary non-MCP endpoint that the rule correctly exempts, yet the evaluation requires this article as a finding. Passing static retrieval does not establish that applicability.

Provide evaluation-visible, non-answer-revealing context establishing MCP exposure and supported version (for example a per-case/article context applied to both positive and clean inputs), or another self-contained visible registration source. Include a non-MCP clean control if expanding fixture coverage. Keep the rule's explicit intent/effective-description requirement; do not make missing AboutText on every API object a defect. Exact-head structural validators pass, but the prepared positive case is semantically under-specified.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed draft head 9279168d397d968dd4185569dbb5e704b1783fed. The applicability blocker is resolved: explicit BC28 online/MCP context survives each prepared request format and applies equally to the positive and clean inputs, without leaking the expected rule/outcome or contaminating other articles. The REST-only control uses the same neutralized positive source while rejecting MCP-specific findings, so the established-MCP-use requirement is genuinely exercised rather than inferred from removed comments. All changed pairs remain selected without waiver.

The new context regression suite passes request-format, isolation, neutralization, clean-control scoring and invalid-override checks. I also ran it directly in an isolated native-filesystem snapshot with actual child subprocesses, alongside the actual changed-path preparation (255 cases) and skill/schema checks. No model-scored precision or AL compilation claim is implied. No remaining merge-critical issue found; content is ready when the draft is made ready and substantive GitHub workflows are permitted to run.

@javiarmesto
Javier Armesto Gonzalez (javiarmesto) marked this pull request as ready for review October 23, 2026 16:12
Copilot AI balanced review requested due to automatic review settings October 23, 2026 16:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Static review found no blocking issues in the guidance, samples, or bounded fixture changes; validation commands were not rerun.

No open issues 🎉

What changed in this PR

Adds web-services guidance for API page and query descriptions exposed through Business Central’s MCP server, with context-aware evaluation fixtures that distinguish MCP use from REST-only integrations.

Changes:

  • Documents description precedence, applicability boundaries, and good/bad AL examples.
  • Adds per-article deployment contexts and clean controls that reuse the same source under different deployment conditions.
  • Adds regression checks for context preservation, isolation, validation, and scoring, and runs them in CI.
File Description
tools/​Test-ReviewFixtures.ps1 Validates article overrides and generates context-specific cases.
tools/​Test-ReviewFixtureContexts.ps1 Checks prepared contexts, clean-control scoring, and invalid registrations.
microsoft/​skills/​review/​al-web-services-review.md Adds selection signals for MCP description guidance.
microsoft/​knowledge/​web-services/​describe-api-pages-for-mcp-with-abouttext.md Defines description precedence and applicability exceptions.
microsoft/​knowledge/​web-services/​describe-api-pages-for-mcp-with-abouttext.good.al Demonstrates descriptions consistent with page and query filters.
microsoft/​knowledge/​web-services/​describe-api-pages-for-mcp-with-abouttext.bad.al Demonstrates captions that omit material endpoint scope.
evaluation/​review-fixtures.json Registers MCP contexts and a REST-only clean control.
evaluation/​README.md Documents article overrides, context controls, and regression checks.
.github/​workflows/​review-fixtures.yml Runs context regression checks in fixture CI.

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants