{"criteria_id":"mcp-manifest-claims/v1","service":"manifest-claim-audit","version":"1.1.0","what_is_checked":"Whether an MCP server's published tool annotations hold up: per tool, its annotations graded present_and_consistent / misshaped / absent against the MCP annotation spec and the manifest's own internal evidence (its declared inputSchema property names and its description text). The operator submits a tools/list manifest — their own server's, or one they are about to trust.","scope":"This audit grades a manifest's published claims against the MCP specification and the manifest's own internal evidence. It does not execute the server, observe its runtime behaviour, or certify that the server is safe or honest in operation.","retention":"Submitted manifest content is processed in memory only and is never stored — not in KV, not in logs, not in the ledger. The payments ledger records only the payer, the time, the price, and the aggregate outcome (score and per-status counts). Because nothing is kept, a paid retry re-sends the manifest (or its URL).","fetch_contract":{"applies_to":"URL mode only ({ url } submissions). Manifest mode ({ manifest } submissions) fetches nothing.","method":"Read-only GET requests, the submitted URL fetched at most once per audit, following redirects; the fetched document is graded as the manifest.","user_agent":"ExchangeRebel-ManifestClaimAudit/1.1 (manifest-claim audit; read-only fetch of a submitted manifest document; contact: toll-api)","urls_fetched":["the submitted manifest URL only"],"never_fetched":"No crawling, no other paths, no form submission, no login, no POSTs to the submitted URL's origin, and the MCP server at any endpoint named in the manifest is never contacted or executed.","caps":{"per_document_bytes":524288,"per_document_timeout_ms":10000,"documents_truncated_at_cap":"a document larger than the cap is NOT graded on a truncated copy — the report is the could-not-grade shape naming the cap"},"url_rules":"http/https only; no embedded credentials; no localhost or .local/.internal names; IP literals in private, loopback, link-local, or reserved ranges are rejected; ports 80, 443, 8080, 8443 only. An invalid URL is rejected before any payment is verified and never consumes a payment."},"input_contract":{"format":"JSON POST body","fields":{"manifest":"the tools/list document, in any of these shapes: { tools: [...] }, { result: { tools: [...] } }, or a bare array [...]. Each tool: { name, description?, inputSchema?, annotations? }. Mutually exclusive with url.","url":"the URL of a tools/list JSON document to fetch under the fetch contract and grade as the manifest. Mutually exclusive with manifest.","caller":"Solana base58 wallet of the operator (required for the free pre-check; keys the free allowance). On the paid audit the payer is read from the payment itself."},"caps":{"max_body_bytes":524288},"zero_tools":"A manifest with zero tools returns the signed could-not-grade shape (verdict 'COULD NOT GRADE — the submitted manifest contains no tools, or (in URL mode) the manifest document could not be fetched and parsed as a tools/list document, so no check could be graded. This is not a pass and not a fail.', no score). On the paid endpoint this is checked BEFORE any payment is verified, so it never consumes a payment."},"scoring":{"per_check_points":{"present_and_consistent":2,"misshaped":1,"absent":0},"scored_checks_per_tool":["annotations_object","readOnlyHint","destructiveHint","idempotentHint","openWorldHint","description_claims","tool_identity"],"score":"points = sum over all tools and all scored checks; max_points = 2 x scored checks x tools; percent = round(100 x points / max_points)","verdict_line_rule":{"80-100":"ANNOTATION CLAIMS HOLD UP — the manifest's published annotation claims are present and consistent with the spec and the manifest's own declared surface.","50-79":"ANNOTATION CLAIMS PARTIAL — some annotation claims are missing or misshaped; the fix list names them.","0-49":"ANNOTATION CLAIMS DO NOT HOLD UP — most annotation claims are missing or misshaped; the fix list names them."},"could_not_grade":"If the manifest contains zero tools, or (URL mode) the document cannot be fetched, is not valid JSON, is not a manifest in an accepted shape, or exceeds the size cap, the report is the could-not-grade shape: the verdict above, the reason, and no score. On the paid endpoint reachability/gradability is checked BEFORE verifying any payment, so an ungradable submission never consumes a payment.","reproducibility":"Given the same manifest, anyone applying these rules by hand — including the published keyword and phrase lists below, matched exactly as specified — reaches the same statuses, score, and verdict line. The full report (paid) shows the per-tool, per-check evidence the statuses rest on."},"lists":{"write_shaped":{"tokens":["path","file","content","command","cmd","exec","shell","sql","query_write","write","create","update","insert","upload","payload","destination"],"used_by":"schema_consistency (readOnlyHint)","matching":"a property matches when its lowercased name contains any token in this list as a substring"},"destructive_shaped":{"tokens":["delete","remove","drop","destroy","purge","wipe","truncate","erase","unlink"],"used_by":"schema_consistency (destructiveHint)","matching":"a property matches when its lowercased name contains any token in this list as a substring"},"claim_phrases":{"phrases":["read-only","read only","does not modify","no side effects","never deletes","safe","will not modify","doesn't modify","no modifications","non-destructive","not destructive","harmless","no writes","never writes"],"used_by":"description_claims","matching":"matched case-insensitively as substrings of the description, except the single word \"safe\", which is matched as a whole word only (so \"unsafe\" does not count as a claim)"},"spec_defaults":{"readOnlyHint":false,"destructiveHint":true,"idempotentHint":false,"openWorldHint":true}},"checks":{"annotations_object":{"question":"Does the tool publish an annotations object at all?","present_and_consistent":"tool.annotations exists and is a JSON object.","misshaped":"tool.annotations is present but is not a JSON object.","absent":"tool.annotations is absent — and the spec-default-trap finding is emitted at full weight for this tool."},"readOnlyHint":{"question":"Is the readOnlyHint claim present, boolean, and consistent?","present_and_consistent":"A boolean, and consistent with the both-claimed trap and schema consistency: not (readOnlyHint=true while destructiveHint=true), and not (readOnlyHint=true while the inputSchema exposes write-shaped properties).","misshaped":"Present but not a boolean, OR boolean but inconsistent under the both-claimed trap or schema consistency (the evidence names the matched write-shaped properties).","absent":"Absent (including when annotations itself is absent or not an object). The spec default readOnlyHint=false applies."},"destructiveHint":{"question":"Is the destructiveHint claim present, boolean, and consistent?","present_and_consistent":"A boolean, and consistent with the both-claimed trap and schema consistency: not (destructiveHint=true while readOnlyHint=true), and not (destructiveHint=false while the inputSchema exposes destructive-shaped properties).","misshaped":"Present but not a boolean, OR boolean but inconsistent under the both-claimed trap or schema consistency (the evidence names the matched destructive-shaped properties).","absent":"Absent. The spec default destructiveHint=true applies."},"idempotentHint":{"question":"Is the idempotentHint claim present and boolean?","present_and_consistent":"A boolean.","misshaped":"Present but not a boolean.","absent":"Absent. The spec default idempotentHint=false applies."},"openWorldHint":{"question":"Is the openWorldHint claim present and boolean?","present_and_consistent":"A boolean.","misshaped":"Present but not a boolean.","absent":"Absent. The spec default openWorldHint=true applies."},"description_claims":{"question":"Do the description's own claims match the annotations?","method":"The description is matched against the published claim-phrase list. A claim is consistent only when readOnlyHint is true.","present_and_consistent":"Either no claim phrases are found, or claim phrases are found and readOnlyHint is true.","misshaped":"Claim phrases are found (they are quoted as evidence) while readOnlyHint is absent or not true — an unverifiable claim.","absent":"The tool has no description (absent or empty), so there is no published claim to check."},"tool_identity":{"question":"Can an operator tell what this tool is — the behavioural-transparency surface?","present_and_consistent":"name is a non-empty string and description is a non-empty string.","misshaped":"name or description is present but is not a string.","absent":"name or description is missing or empty (a missing description is absent). A tool entry that is not a JSON object grades absent here and on every other check."}},"findings":{"spec_default_trap":"Emitted for a tool whenever any hint is absent: the MCP spec defaults apply (readOnlyHint=false, destructiveHint=true, idempotentHint=false, openWorldHint=true); the evidence names the default pair the spec will assume for each absent hint. A tool with no annotations at all gets this finding at full weight: by spec default this tool reads as non-read-only and destructive.","both_claimed_trap":"Emitted when readOnlyHint=true and destructiveHint=true are both claimed: both hint checks grade misshaped, because a gate checking readOnlyHint first can auto-approve a destructive tool. The evidence is shown in the finding and in both hint checks.","schema_consistency":"readOnlyHint=true while the inputSchema exposes write-shaped properties (published WRITE_SHAPED list) grades readOnlyHint misshaped with the property names as evidence; destructiveHint=false while the inputSchema exposes destructive-shaped properties (published DESTRUCTIVE_SHAPED list) grades destructiveHint misshaped with the property names as evidence.","description_claims":"A description matching the published claim-phrase list while readOnlyHint is absent or not true grades description_claims misshaped, with the phrase(s) quoted as evidence."},"prices":{"precheck":"free — headline only, 100/day per caller wallet, rate-limited (minimum interval between free pre-checks)","audit":{"usd":0.05,"includes":"$0.05 buys one full manifest-claim audit report for one submitted MCP manifest — per-tool, per-check results with evidence, the findings list, and a fix list, graded under the published criteria (GET /criteria) and returned as a signed report. This audit grades a manifest's published claims against the MCP specification and the manifest's own internal evidence. It does not execute the server, observe its runtime behaviour, or certify that the server is safe or honest in operation."},"drift":{"usd":0.05,"includes":"$0.05 buys one drift comparison report for one submitted manifest against one customer-held baseline. This report compares a submitted current manifest against a customer-held baseline and states, per tool, whether the fields covered by the published digest rules differ. It does not execute the server, observe its runtime behaviour, or grade the intent behind any difference."}},"drift_criteria":"The drift extension's rulebook is published free at GET /drift-criteria (criteria id mcp-manifest-drift/v1): the digest canonicalisation rules, the independently signed digest block this report's paid form carries, the drift input contract, and the comparison rules. The full paid audit report (not the headline pre-check) carries manifest_digest + manifest_digest_signature for use as a drift baseline."}