{"criteria_id":"token-efficiency/v1","family":"token-efficiency","op":"budget","ops_not_offered":["catalogue","audit"],"corpus":{"id":"token-efficiency-corpus/v1","sha256":"1f50ecdaf7965a3d79d3c277c17401a53e99e1887988fd7f737cf9f8ef0e4738","token_figures_status":"PINNED-REAL-TOKENIZER"},"corpus_pin_definition":"The corpus pin is the SHA-256 of the corpus's canonical serialization (the corpus object excluding its sha256 field), computed and verifiable by the published harness (harness.mjs). Token figures inside corpus cases are pinned against the real tokenizer named in the criteria (cl100k_base, gpt-tokenizer 4.1.0), pinned at integration on 2026-11-03.","tokenizer":{"id":"cl100k_base/gpt-tokenizer@4.1.0/data-sha256:223921b76ee99bde995b7ff738513eef100fb51d18c93597a113bcffe865b2a7/source-sha256:7354993a4b8e2d0351562811a67ea2cf6fa4966164df2925c83b63d8d802f005","stub":false,"real":true,"encoding":"cl100k_base","implementation":"gpt-tokenizer","version":"4.1.0","data_sha256":"223921b76ee99bde995b7ff738513eef100fb51d18c93597a113bcffe865b2a7","source_sha256":"7354993a4b8e2d0351562811a67ea2cf6fa4966164df2925c83b63d8d802f005","status":"pinned — cl100k_base via gpt-tokenizer 4.1.0 (MIT), pins in the TOKENIZER DECISION RECORD; special-token spellings in submitted text are counted as ordinary text (no special tokens allowed).","character_counts":"Every token figure is paired with a character count (Unicode code points) of the same text, which is tokenizer-independent and hand-checkable."},"price":{"usdc_units_per_call":20000,"label":"$0.02","basis":"per call"},"free_tier":{"per_day":100,"min_interval_ms":5000,"on_exhaustion":"402"},"input_contract":{"body":"One JSON request body: { payload, budget_tokens, keep_fields?, reserve_tokens?, caller? (free pre-check only) }.","payload":"A string: the one tool/search result payload to check, as text or as JSON text. A non-string payload is a free validation refusal.","budget_tokens":"Required positive integer — the budget the payload must fit, counted under the published tokenizer. There is no default budget: a call without a valid budget_tokens is a free validation refusal.","keep_fields":"Optional array of dot-path strings over object keys (e.g. \"a\" or \"a.b\"). A keep path protects that field and everything under it from S-2 and S-3; a field that contains a protected descendant is likewise never removed by S-2. S-1 and S-4 are not keep-exempt: S-1 removes only byte-identical duplicate entries, and S-4's truncation point never cuts a protected field's serialised span (see S-4).","reserve_tokens":"Optional non-negative integer held back from the budget. Effective budget = budget_tokens - reserve_tokens, printed in the verdict. If reserve_tokens >= budget_tokens the effective budget is <= 0 and the call is a free validation refusal.","caps":{"body_bytes":262144},"over_cap":"A request body over 262,144 bytes is a free pre-gate refusal naming the cap. No partial or truncated verdict exists in this family.","malformed_payload":"If the payload string is not parseable as JSON, only the text operation (S-4) is assessable; the verdict states which operation set applied (\"json\" or \"text-only\") — never a silent downgrade. A request body that is not itself a JSON object is a free validation refusal."},"serialisation_rule":"If the payload parses as JSON, all counts and the shaped output use the canonical form: the compact JSON.stringify of the parsed value, keys in submitted order (subject to JavaScript object key ordering for integer-like keys, which JSON.parse and JSON.stringify share). If the payload does not parse as JSON, counts are over the raw submitted string. Submitted tokens P = the token count of the canonical form. The shaped payload is returned as a string holding the canonical form of the shaped value — or, where S-4 applied, the truncated serialised text with its marker.","verdict_grammar":{"fits":"PAYLOAD BUDGET — payload tokens: P — effective budget: B — FITS — <check-end UTC>","over":"PAYLOAD BUDGET — payload tokens: P — effective budget: B — OVER by E — <check-end UTC>","not_shapable":"PAYLOAD BUDGET — payload tokens: P — effective budget: B — OVER — NOT SHAPABLE WITHIN STATED CONSTRAINTS — <check-end UTC>","vocabulary":"No other verdict vocabulary exists. FITS means payload tokens <= effective budget (excess 0); OVER states the exact excess E = P - B of the SUBMITTED payload; where shaping then brings the payload within budget, the shaped figures are reported alongside — the verdict line always grades the submission against the stated budget."},"operations":[{"id":"S-1","name":"duplicate-entry removal","text":"Remove array entries whose canonical serialisation is byte-identical to an earlier entry in the SAME array; remove nothing else. Arrays are visited in document order (depth-first, an array before its surviving entries' contents); within an array, entries are examined in index order and each removal is one receipt application."},{"id":"S-2","name":"null/empty-field removal","text":"Remove object fields whose value is null, \"\", [], or {} — except keep_fields-protected fields. Applied as a single depth-first pass, children before parents: a field whose value becomes null/empty only because its own children were removed in this same pass is itself removed (unless protected). Array elements are never removed by S-2. Each removal is one receipt application."},{"id":"S-3","name":"long-string truncation","text":"A string value (an object field, or an array element) that is not keep-protected and is longer than 2,000 characters is truncated to its first 2,000 characters at a character boundary, with the marker appended. Visited in document order; each truncation is one receipt application.","cap_chars":2000},{"id":"S-4","name":"tail truncation","text":"If the payload still exceeds the effective budget after S-1..S-3, truncate the canonical serialised payload at the largest character boundary that brings its token count within the effective budget, appending the same marker form with the total removed count. S-4's truncation point never cuts a keep-protected field's serialised span: the retained prefix always extends at least to the end of the last protected span in the serialisation. Applied at most once, always last, always receipted with field path \"payload\". The count function is treated as non-decreasing over prefixes (true of the pinned tokenizer's encode over the corpus fixtures; asserted by the corpus)."}],"application_granularity":"Operations apply in the published order S-1, S-2, S-3, S-4. Once an operation starts it runs to completion over the whole payload (its rule is a complete mechanical rule; a half-applied operation would leave the shaped payload inconsistent with its own receipt). The published 'each only until the payload fits' applies between operations: once the payload fits, no later operation starts. S-4 applies at most once, always last.","marker":{"template":"[…truncated: N chars removed under token-efficiency/v1 <OP>]","text":"The truncation marker appended by S-3 and S-4. <OP> is the id of the operation that truncated (S-3 or S-4) and N is the number of characters that operation removed (for S-3, from that field; for S-4, from the serialised payload in total). The marker is part of the shaped text it terminates, and its characters and tokens are counted in the shaped figures.","interpretation_note":"The design record publishes the marker's literal text for S-3 (ending \"S-3\") and directs S-4 to append \"the same marker form\"; this criteria reads the form as the template above, with the truncating operation's own id in the <OP> position. The reading is recorded in the build registry for the coordinator; the S-3 literal is reproduced exactly by this template."},"exhaustion":"If the payload still exceeds the effective budget after S-1..S-4 — possible only where keep_fields-protected content alone exceeds the budget, or where the budget cannot hold even the S-4 marker outside the protected spans — the op returns the verdict OVER — NOT SHAPABLE WITHIN STATED CONSTRAINTS, the shaped payload UNCHANGED from the submission (the canonical form of the unmodified value; no operation is applied to the returned payload), an empty receipt (so the receipt totals reconcile with the unchanged payload), and an exhaustion block carrying the counts that show why: the protected content's token and character counts, the effective budget, and the submitted totals.","receipt":{"fields":["op","path","chars_before","chars_after","tokens_before","tokens_after","delta_tokens","delta_chars"],"text":"One entry per operation application, in application order. path is the field path of the application (dot-path over object keys, with [i] for array positions; \"payload\" for S-4; for S-1, the path of the array the entry was removed from, with the entry's index at removal in the entry's index field). chars_before/after and tokens_before/after are the counts of the WHOLE payload (canonical form) immediately before and after that application, so the deltas telescope: submitted tokens - shaped tokens = the sum of delta_tokens over the receipt, exactly, and likewise for characters. The engine self-checks this reconciliation on every call and fails closed (no shaped output) if it does not reconcile."},"free_precheck":"The free pre-check (POST /precheck) returns the verdict line, the payload token count with its character count, the excess figure, the operation set that applied, and the disclosure/retention/honesty lines, signed. It never returns a shaped payload or a receipt: those are the paid substance. Free allowance: 100 per day per caller wallet, minimum interval 5,000 ms, 402 on exhaustion.","custody":{"construction":"In-memory processing within a single request lifetime; nothing about a submission persists, on any path including errors and timeouts.","ledger_closed_fields":["payer wallet","timestamp/day","price (USDC units) with the CAD fair-market-value block (rate, rate date, basis, cad_pending, refund)","payment tx id + Solana fee (lamports)","criteria version","op","verdict line","input size (request body bytes, payload characters)","house flag","per-op aggregate counts (applications per operation, ops applied in total, tokens before/after, characters before/after)"],"ledger_note":"The ledger list above is closed: adding a field is a criteria/handling version change. The payment-inclusion line's shorter ledger sentence is the buyer-facing summary of this same list. The ledger never carries payload content, any fragment of it, or any hash of a submitted value.","errors":"Error handling is closed-form: static strings plus numeric facts only (lengths, caps, counts, rule ids). No error path interpolates any part of a submission."},"fence":"TE-B's shaped payload is produced without any model and without any judgement about what matters in the content. Every reduction is one of the published mechanical operations in this document, applied in the published order, and every application is listed in the drop receipt with the field, the operation, and the token delta, so the shaped payload can be verified to differ from the submitted payload only as the receipt states. No operation selects content by meaning; the moment any operation selects content by meaning — summarising a paragraph, keeping the \"relevant\" results, re-ranking by importance — it leaves the family. That variant is not this service.","semantic_loss_disclosure":"Shaping preserves budget, not meaning. Mechanical shaping can drop the decisive passage — S-4 tail truncation is content-blind by design. The drop receipt exists so a caller can log exactly what their agent never saw; this service expresses no view on which parts of a payload matter.","disclosure_sentence":"Submitting a payload to any third party, this service included, gives its contents to a new holder however briefly; the check is processed in memory only and never stored, but it does not undo the disclosure, and an operator who suspects a live credential in a payload should rotate it at its issuer rather than submit the payload.","honesty_line":"Counts are exact under the published tokenizer; another model's tokenizer will count the same text differently, so a token total under token-efficiency/v1 is a figure under the published rules, not a promised bill reduction on any specific provider.","ergonomics_boundary":"Ergonomics and TE-B do not share a class: Ergonomics' object is a submitted MCP tool MANIFEST — the tool definitions themselves — which it grades against `tool-ergonomics/v1` for description discrimination, parameter specification, definition-token economics, and naming hygiene, returning a graded audit of that document, whereas TE-B's object is ONE submitted tool/search RESULT PAYLOAD — arbitrary text or JSON that a tool returned, not a tool definition — which it checks against a token budget the caller states, under `token-efficiency/v1`, returning a fit verdict (`FITS` / `OVER` by an exact excess), a mechanically shaped copy of that same payload produced only by the published mechanical operations, and a drop receipt listing every operation applied with token counts before and after that reconcile exactly; the two services take different input objects, apply different criteria, and return different outputs, and neither one's deliverable can be produced by the other.","retention":"Submitted payloads are processed in memory only and are never stored — not in KV, not in logs, not in the payments ledger, not in metrics, and not in error records. No payload content, no fragment of it, and no hash of any submitted value is recorded anywhere. Reports and receipts carry counts, field paths, and operation ids only. The one fenced exception is the shaped payload itself, returned to the submitter on the paid call only; it is never logged or ledgered. The payments ledger records only the closed field list published in the criteria. Because nothing is kept, a paid retry re-sends the payload.","payment_includes":"$0.02 per payload-budget check. The fee is per call: each payload you submit with a stated token budget is charged at $0.02. The payment includes: one submitted payload (request body up to 262,144 bytes) and your stated budget, returned as (1) a signed fit verdict stating the payload's token count under the tokenizer named in the verdict, your effective budget, and whether it fits or the exact excess; (2) where it does not fit, a shaped payload produced only by the mechanical operations published in token-efficiency/v1 — duplicate removal, empty-field removal, per-field truncation, tail truncation, in that order — and (3) a drop receipt listing every operation applied, by field, with token counts before and after that reconcile exactly with the shaped payload. No operation selects content by meaning; nothing is summarised, re-ranked, or rewritten by a model. If the payload cannot be brought within your stated budget under your stated constraints, you receive the unchanged payload with the verdict that it is not shapable within those constraints. A free pre-check returns the verdict line and the excess figure only. Submitted payloads are processed in memory only and are never stored; the payments ledger records only the payer, the time, the price, and the verdict.","kill_line":"Kill line (Q-04, under Mauricio's 2026-11-03 zero-demand doctrine): build, bank, and hold; judged at the fleet's periodic reviews. There is NO demand kill line for this item. Tripwires, all stop-and-review unless stated: (family) refunds >5% of paid calls in any 30-day window; free use >10x paid use; a wrong verdict from undisclosed stale or partial data (stop-serving under the affected criteria version, per the corpus re-pinning discipline in the criteria); (Q-04-specific) an unreconciled drop receipt; a shaped payload differing from its submission in any way not listed in its receipt; any retained surface carrying submitted payload content beyond the fenced shaped-payload return.","never":"This service never selects content by meaning; never summarises, re-ranks, or rewrites content with a model (no model is called anywhere in this family); never issues a composite score or a judgement of a payload's quality or importance; never promises a bill reduction on any specific provider; never applies a default budget; never returns a shaped payload on the free tier; never grades a shortened, sliced, or partially parsed submission; and never retains submitted material beyond the request lifetime.","service":"token-efficiency","version":"1.0.0","endpoints":{"criteria":"GET /criteria (free)","precheck":"POST /precheck (free, headline only: verdict line, payload token count, excess figure, operation set, disclosure/retention/honesty lines, signed)","budget":"POST /budget (paid, $0.02 per call: signed fit verdict + shaped payload (paid call only) + drop receipt)","metrics":"GET /metrics","not_offered":"POST /catalogue and POST /audit answer NOT COVERED — those ops are not built or offered by this build."}}