Repository navigation
knowledge(web-services): describe API objects intended for MCP tools - #233
Javier Armesto Gonzalez (javiarmesto) wants to merge 4 commits into
Conversation
…t for MCP tools The Business Central MCP server takes the tool description from AboutText, then EntitySetCaption, then EntityName. API pages without AboutText reach agents as a bare entity name. LLMs treat AboutText as a UI teaching-tip property and omit it on API objects; no analyzer flags it. Observed in 13 of 22 generated solutions of a read-only SalesMargin query API. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Jesper Schulz-Wedde (JesperSchulz)
left a comment
There was a problem hiding this comment.
Reviewed draft head 38bb24502a4a2806800f12d2d99691e7883b61f6. The documented MCP description precedence and carefully bounded rule are sound, but the registered positive fixture loses its prerequisite during neutral preparation.
The article explicitly requires established Business Central online MCP use and says missing AboutText alone is not a finding. In both companions the only evidence of intended MCP exposure is a full-line comment. Test-ReviewFixtures strips those comments, and the prepared requests for this pair contain only opaque id/file entries with no MCP context. The positive API page/query is then indistinguishable from an ordinary non-MCP endpoint that the rule correctly exempts, yet the evaluation requires this article as a finding. Passing static retrieval does not establish that applicability.
Provide evaluation-visible, non-answer-revealing context establishing MCP exposure and supported version (for example a per-case/article context applied to both positive and clean inputs), or another self-contained visible registration source. Include a non-MCP clean control if expanding fixture coverage. Keep the rule's explicit intent/effective-description requirement; do not make missing AboutText on every API object a defect. Exact-head structural validators pass, but the prepared positive case is semantically under-specified.
Jesper Schulz-Wedde (JesperSchulz)
left a comment
There was a problem hiding this comment.
Reviewed draft head 9279168d397d968dd4185569dbb5e704b1783fed. The applicability blocker is resolved: explicit BC28 online/MCP context survives each prepared request format and applies equally to the positive and clean inputs, without leaking the expected rule/outcome or contaminating other articles. The REST-only control uses the same neutralized positive source while rejecting MCP-specific findings, so the established-MCP-use requirement is genuinely exercised rather than inferred from removed comments. All changed pairs remain selected without waiver.
The new context regression suite passes request-format, isolation, neutralization, clean-control scoring and invalid-override checks. I also ran it directly in an isolated native-filesystem snapshot with actual child subprocesses, alongside the actual changed-path preparation (255 cases) and skill/schema checks. No model-scored precision or AL compilation claim is implied. No remaining merge-critical issue found; content is ready when the draft is made ready and substantive GitHub workflows are permitted to run.
There was a problem hiding this comment.
🟢 Approval recommended
Static review found no blocking issues in the guidance, samples, or bounded fixture changes; validation commands were not rerun.
No open issues 🎉
What changed in this PR
Adds web-services guidance for API page and query descriptions exposed through Business Central’s MCP server, with context-aware evaluation fixtures that distinguish MCP use from REST-only integrations.
Changes:
- Documents description precedence, applicability boundaries, and good/bad AL examples.
- Adds per-article deployment contexts and clean controls that reuse the same source under different deployment conditions.
- Adds regression checks for context preservation, isolation, validation, and scoring, and runs them in CI.
| File | Description |
|---|---|
| tools/Test-ReviewFixtures.ps1 | Validates article overrides and generates context-specific cases. |
| tools/Test-ReviewFixtureContexts.ps1 | Checks prepared contexts, clean-control scoring, and invalid registrations. |
| microsoft/skills/review/al-web-services-review.md | Adds selection signals for MCP description guidance. |
| microsoft/knowledge/web-services/describe-api-pages-for-mcp-with-abouttext.md | Defines description precedence and applicability exceptions. |
| microsoft/knowledge/web-services/describe-api-pages-for-mcp-with-abouttext.good.al | Demonstrates descriptions consistent with page and query filters. |
| microsoft/knowledge/web-services/describe-api-pages-for-mcp-with-abouttext.bad.al | Demonstrates captions that omit material endpoint scope. |
| evaluation/review-fixtures.json | Registers MCP contexts and a REST-only clean control. |
| evaluation/README.md | Documents article overrides, context controls, and regression checks. |
| .github/workflows/review-fixtures.yml | Runs context regression checks in fixture CI. |
Review effort: Balanced
API objects intended for the Business Central MCP server can expose a bare entity caption where an agent needs to understand document filters or computed values. Add web-services knowledge for the documented description precedence:
AboutText,EntitySetCaption, then camel-casedEntityName. The good/bad samples cover both API pages and queries, with descriptions and actual document-type filters kept consistent.The article preserves valid alternatives: missing
AboutTextis not a defect when the fallback is adequate or MCP use is not established, andAboutTitleis not required solely for MCP. Sources include the Learn API page/query documentation and a pinned Microsoft API query withAboutTextand noAboutTitle. Its scope is BC 28+ online environments with the relevant MCP support; it does not claim that extraction first appeared in 28.0 or cover the earlier preview.The contribution is motivated by a 22-run API generation evaluation whose syntax rubric flagged AboutText omissions in 13 runs, including one incomplete solution. These are omission flags, not measured tool-selection failures. The missing metadata is not diagnosed by the tested standard analyzers. The owning domain is web-services because the concern is the API-to-tool metadata contract, distinct from UI teaching tips.
The evaluation now supplies Business Central online version 28 and established MCP exposure as per-article context for both the positive and clean samples. This context survives comment removal in every prepared request format and does not spill into other web-services articles. A required non-MCP clean control reuses the positive sample's exact neutral code under a REST-only context. This tests the applicability boundary without changing the article's rule or AL companions.
The harness accepts optional
articleOverrideswith a shared pair context and namedcleanContexts. These deployment facts contain no expected answer, fix, or article ID. Invalid registrations fail validation; model-facing case IDs, result shape, candidate ranking, and scoring thresholds retain their existing contracts. The new regression check is wired into fixture CI.The branch incorporates upstream main at
a4593a1. Conflict resolution keepsthe upstream stream-encoding registration, documentation, and retrieval tokens
alongside the MCP article and context cases. A compatibility comparison
preserves all upstream fixture registrations and all 292 prepared cases, with
unchanged neutral code and deployment contexts; only the three MCP applicability
cases are added. The MCP article and its AL companions are unchanged from the
head approved by Jesper.
Validation:
No new model-driven review evaluation, live MCP extraction, or historical deployment test was run. The review-skill edit only selects this knowledge; the facts and exceptions remain in the article.
These validation results are local; GitHub workflow authorization and maintainer review remain separate from preparing this branch.