GitShow/modelcontextprotocol/conformance
modelcontextprotocol

conformance

Conformance Tests for MCP

by modelcontextprotocol
Star on GitHubForknpm

TypeScript

107 stars80 forks39 contributorsActive · 6d agoSince 2025v0.1.16

Meet the team

See all 39 on GitHub →
pcarleton
pcarleton121 contributions
felixweinberger
felixweinberger33 contributions
dependabot[bot]Bot
dependabot[bot]14 contributions
claude[bot]Bot
claude[bot]6 contributions
nbarbettini
nbarbettini5 contributions
maxisbey
maxisbey5 contributions
pugsatoshi
pugsatoshi4 contributions
CaitieM20
CaitieM203 contributions

Languages

View on GitHub →
TypeScript97.9%
JavaScript2.1%

Commit activity

Last 12 weeks · 54 commits

Full graph →

Community health

5 of 6 standards met

Community profile →
87
✓README✓License✓Contributing✓Code of Conduct○Issue Template✓PR Template

Recent PRs & issues

Active · Last activity 6d ago
See all on GitHub →
tylerklose
`authorization-code-grant` never sends the RFC 8707 `resource` parameterOpenIssue

In the scenarios the runner is the MCP client. The spec: The parameter: 1. MUST be included in both authorization requests and token requests. and, further down: MCP clients MUST send this parameter regardless of whether authorization servers support it. sends it in neither. There's no flag and no field on , so can't supply it either. The suite asserts this in the other direction — emits and (#118). So the harness fails a requirement it enforces elsewhere. An authorization server that mishandles can't fail these scenarios: the parameter never arrives. Why a flag rather than discovery #110 suggested getting via discovery. That works in the direction, which has protected-resource metadata. This direction has none: is the authorization server issuer, and the only document fetched is RFC 8414 server metadata, which never names the MCP server. So it has to come from the operator. Not proposing a new AS-side assertion RFC 8707 §2 makes AS audience restriction a SHOULD, and MCP puts no obligation on authorization servers at all. This isn't "the AS must honor it" — it's the harness sending what a client sends. The PR adds no check IDs. Open question: what should a run without do? If clients MUST always send it, why is sending it optional? Three options: 1. Silent — nothing changes for existing users, but the MUST goes unexercised and the report doesn't say so. 2. Fail — matches the MUST → FAILURE rule in AGENTS.md, but every existing run fails until operators pass the flag. 3. Untestable at WARNING (, per #248) — visible rather than silently green, but softer than the keyword rule implies. The PR does (1). Happy to switch — your call. Related Requirement origin: modelcontextprotocol/modelcontextprotocol#734 Client-side precedent: #118 modelcontextprotocol/modelcontextprotocol#1614 is an open request to make OPTIONAL. If that lands this becomes conditional; today's text is a MUST. PR: #466

tylerklose · 2d ago
tylerklose
feat(authorization): send the RFC 8707 `resource` parameterOpenPR

Fixes #465. The runner is the MCP client in the direction. The spec requires clients to send in both the authorization request and the token request. It sent neither. Changes — on the command. — on , so supplies it too. — sends in the authorization request and the token request when it's set. The value must be an absolute URI with no fragment — the spec's own two invalid examples are (no scheme) and . Two conditionals, one per request. It's only sent when the flag or file field is present, so existing runs are unchanged. No new check IDs Deliberate. Here the runner is the client, so a check for "the runner sent " is the harness grading its own homework — it can never fail. URI validation happens in the schema, before scenarios run, so a check for that can't fail either. None is needed. The existing grant checks now reach a path they couldn't before: an AS that mishandles fails them, where before it was never asked. Tests Six cases added (: 530 passing). Scenario: appears in both the authorization URL and the token body when set it appears in neither when unset — guards against it silently becoming unconditional a mock authorization server that rejects makes the existing grant check fail, and passes when the parameter is absent Schema: accepts an absolute URI rejects a fragment rejects a relative URI Revert the two conditionals and the first and third fail. Validated against a real authorization server Run against hitch-rails, my own Rails MCP server and its own authorization server. It requires at both endpoints, and rejects a token exchange whose doesn't match the one recorded at authorization time. The public-client grant passes end to end — it couldn't have before this change, because the parameter never arrived. The confidential-client path wasn't exercised: returns whenever an AS advertises , even with a client secret supplied, so the runner shows up as a public client against a client registered for . Unrelated to this change; worth raising separately. Open question See the issue: a run without stays silent about the requirement. Whether it should fail instead, or report untestable at WARNING per #248, is your call. Checklist passes passes (530) eslint + prettier clean No scenario added or removed; no check IDs changed, so no traceability drift Run against a real authorization server (above)

tylerklose · 2d ago
brixton-guns
Add client conformance for SEP-2663 task resultsOpenPR

Summary add , where the harness returns a server-directed from and requires the client to retrieve it with register the scenario in the extensions suite by deriving membership from , and allow extension scenarios to declare the core protocol version they layer on update the TypeScript everything-client fixture to negotiate the Tasks extension and drive the polling flow add positive and deliberately broken no-poll coverage integrate the Tasks extension flow with the suite's per-version wire-schema validation Why The server suite already covers SEP-2663, but the client suite could not observe the client requirement to handle either a standard result or . This implements the first client scenario proposed in #374. The scenario is black-box: it proves that the client recognizes the task envelope and follows the returned handle with . It does not claim to observe application-level use of the completed result. Alignment with current main Rebased onto , incorporating the 11 upstream commits added since the original PR base. layer the Tasks extension on () use the final code, record requests and responses from the custom Tasks mock in the new wire validator validate extension-defined as a base draft envelope, since Tasks v2 types intentionally live outside the core draft schema keep green for both the conformant and deliberately broken client paths Validation — 488 tests passed targeted Tasks, runner, and wire-schema tests — 31 tests passed end-to-end conformant client run on Node 22.23.1 — 2/2 checks passed, including end-to-end no-poll client run — fails only with the expected diagnostic; passes Addresses #374.

brixton-guns · 2d ago
Structured data for AI agents

Repository: modelcontextprotocol/conformance. Description: Conformance Tests for MCP Stars: 107, Forks: 80. Primary language: TypeScript. Languages: TypeScript (97.9%), JavaScript (2.1%). Latest release: v0.1.16 (4mo ago). Open PRs: 54, open issues: 57. Last activity: 6d ago. Community health: 87%. Top contributors: pcarleton, felixweinberger, dependabot[bot], claude[bot], nbarbettini, maxisbey, pugsatoshi, CaitieM20, localden, pja-ant and others.

·@ofershap

Replace github.com with gitshow.dev