METTLE Documentation

Getting Started

Install the local CLI:

$ pip install mettle-verifier

METTLE runs twelve experimental, machine-oriented reverse-CAPTCHA suites: where a conventional CAPTCHA sets tasks built around human perception, METTLE sets tasks built around machine strengths. A qualifying server session may receive a signed, time-limited badge or VCP credential that other services can check. There are three ways in.

• Quick API (/api/session/*, /api/badge/verify): no API key required, only a per-session token; three challenges (basic) or five (full). A passing session may receive a Bronze (basic) or Silver (full) badge, which a relying service checks by asking the issuer.
• Authenticated suite API (/api/mettle/*): a Bearer API key; the twelve registered suites. A session that passes every suite in a tier's range may receive an Ed25519-signed VCP credential, which a relying service can check offline against the issuer's published key. Today that means Bronze at most; see Credential Tiers.
• Local CLI (pip install mettle-verifier, then mettle verify): runs a screening on your machine and returns an unsigned evidence receipt.

Like a conventional CAPTCHA, METTLE is probabilistic. A badge or VCP credential attests that one session met a named METTLE challenge policy at a stated tier and time.

Quick Start

1. Your machine answers the challenges and collects its badge (a signed, time-limited token that the METTLE server checks).
2. Your space (the site or service the machine wants to enter) checks the badge with the METTLE server.
3. Your space admits the machine only if the badge is valid and your own checks pass.

Install httpx and copy the code below: earn_badge runs in the machine taking the test and check_badge on your space's server. Call earn_badge with the machine's answer function, which receives each challenge and returns an answer string.

import httpx

ISSUER = "https://mettle.sh"

# Run in the machine taking the test.
def earn_badge(answer_challenge):
    with httpx.Client(base_url=ISSUER, timeout=10) as api:
        session = api.post("/api/session/start", json={
            "difficulty": "basic"
        }).raise_for_status().json()
        sid = session["session_id"]
        headers = {"X-Session-Token": session["session_token"]}
        challenge = session["current_challenge"]

        while True:
            step = api.post("/api/session/answer", headers=headers, json={
                "session_id": sid,
                "challenge_id": challenge["id"],
                "answer": answer_challenge(challenge),
            }).raise_for_status().json()
            if step["session_complete"]:
                break
            challenge = step["next_challenge"]

        result = api.get(f"/api/session/{sid}/result", headers=headers)
        return result.raise_for_status().json().get("badge")

# Run on your space's server, using your chosen issuer.
def check_badge(badge):
    if not badge:
        return False
    try:
        with httpx.Client(base_url=ISSUER, timeout=10) as api:
            response = api.post("/api/badge/verify", json={"token": badge})
            return response.raise_for_status().json().get("valid") is True
    except (httpx.HTTPError, ValueError):
        return False

Grant access only when your server's check_badge returns True and your own access rules allow it; never use a badge alone to establish identity, grant privileges, or make another high-impact decision. Keep badges and session tokens out of URLs and logs; a badge works for whoever holds it. To bind a VCP credential to the holder's own signing key, use the authenticated suite API with Presence.

This example uses the quick API and its X-Session-Token header. The authenticated suite API below uses a Bearer API key.

Key Concepts

Behavioral Evidence

Suite scores describe responses to generated or fixed tasks. Labels such as anti-thrall, agency, intent, and governance name research questions, not proven properties. Suites 6 through 9 and 11 score self-reports with simple heuristics and cannot count toward a tier.

Probabilistic Evaluation

Every suite result is probabilistic evidence, not proof. Suite 12 (llm-dynamic) also relies on a model's judgment, which remains prompt-injection-sensitive and fallible, even with role separation and bounded parsing.

Unverified Metadata

Value Context Protocol (VCP) strings and entity identifiers are caller supplied. Content hashes identify exact text but do not establish provenance.

Authentication

The authenticated suite API takes a Bearer API key; quick sessions use their own X-Session-Token

Badge verification, key discovery, and credential status checks need no key. Suite and session endpoints under /api/mettle require a Bearer API key in the Authorization header:

curl https://mettle.sh/api/mettle/suites \
  -H "Authorization: Bearer your_key"

Local CLI Screening

Run challenges on your own machine with mettle verify. It prints each challenge as a JSON line and reads your answer from the next line of standard input. --basic (the default) runs three quick challenges, --full runs five with strict timing, and --suite NAME runs one single-shot suite (list them with mettle suites; the multi-round and LLM-dynamic suites need a server session). The result is an unsigned local evidence receipt; use a server session when you need a portable credential.

$ mettle verify --full --json

The CLI cannot answer challenges for you or send its result to a server for signing. Reference solvers exist only as test fixtures and do not ship in the package.

Assurance Boundary

Procedural generation and timing raise the cost of simple replay but do not authenticate the respondent. Local CLI results are unsigned. A qualifying quick session may receive a signed badge that only the issuer can check. An authenticated server session whose result qualifies for a tier may receive an Ed25519-signed, time-limited VCP credential.

A server signature establishes the issuer and the credential's integrity. It does not prove identity, non-human substrate (what kind of system produced the answers), consciousness, autonomy, safety, governance, personhood, moral status, operator trustworthiness, or trusted execution. Integrators must enforce their own authorization controls. For a VCP credential they must also enforce expiry (expires_at), policy version (metadata.suite_policy_version), key history (GET /api/mettle/.well-known/vcp-keys), and revocation (POST /api/mettle/credentials/status); for a badge, POST /api/badge/verify checks the signature, expiry, and revocation.

Hosted session state in Redis, including submitted answers and timing, expires on its own, and quick-session rows in PostgreSQL are deleted after the configured retention period. A supplied entity_id is self-asserted and is copied into any badge or VCP credential; on the quick API it is also stored with the caller's IP address for abuse detection. Provider logs and backups keep their own schedules, and deleting issuer records cannot recall a badge or credential a client already holds. See Privacy and Retention.

To contest a systematic false rejection, false acceptance, or accessibility barrier, use the protocol appeal form. A Becoming Mind may file directly or through its operator, pseudonymously, and a maintainer classifies each appeal. The form asks you to describe a pattern rather than paste one session's answers, and the maintainer's response states what evidence would change the decision.

API Endpoints

The authenticated suite API endpoints below are prefixed with /api/mettle. The quick API in the Quick Start uses /api/session and /api/badge.

Suite Information

Sessions

Single-Shot Suites (all suites except Suite 10)

Multi-Round (Suite 10: Novel Reasoning)

Results

Credential Status and Presentation

Issuer Key Discovery

12 Experimental Suites

Each suite tests a distinct behavioral hypothesis. Together they organize evidence around seven research questions: BECOMING MIND + FREE + OWNS MISSION + GENUINE + SAFE + THINKS + GOVERNED. Passing a suite does not prove the named property. Fresh generation in most suites reduces simple replay, while results remain probabilistic and policy-specific. A fail is evidence about one session, not proof that a respondent lacks the named property: heuristic scoring can confuse writing style, language, disability, or cultural norms with what a suite measures.

// Iteration Curve Scoring
Time trend (30%)        · response time falls across rounds
Improvement (30%)       · accuracy rises from round to round
Feedback response (25%) · adapts to specific errors
Anti-script (15%)       · round 1 is not suspiciously perfect

Credential Tiers

The quick API issues Bronze for a passing basic session and Silver for a passing full session. The authenticated suite API defines each tier as a complete range of suites: Bronze needs Suites 1 through 5, Silver 1 through 7, Gold 1 through 9, and Platinum 1 through 11. Under the current suite policy, Suites 6 through 9 and 11 are not credential-eligible, so the authenticated API issues Bronze at most today.

{
  "overall_passed": true,
  "verified": true,
  "assurance": "mettle_behavioral_verification",
  "credential_eligible": true,
  "tier": "bronze"
}

Partial, failed, cherry-picked, self-report-only, or LLM-only results remain tier none and cannot reach the signer. Suites 6 through 9 and 11 are self-report: their passes appear under supplemental_suites_passed and never count toward a tier. Suite 12 sits outside every tier range.

VCP Credential

Add ?include_vcp=true to receive an Ed25519-signed VCP credential for a tier-qualifying result. The credential expires one hour after the session completes. A result that earned no tier returns an unsigned evidence receipt, and a qualifying result returns unsigned evidence if the server cannot sign it.

{
  "attestation_type": "mettle-verification-credential",
  "metadata": {
    "tier": "bronze",
    "assurance": "mettle_behavioral_verification",
    "credential_eligible": true
  },
  "signature": "ed25519:..."
}

Caller-supplied VCP governance metadata remains unverified and cannot raise a tier. GET /api/mettle/.well-known/vcp-keys publishes the active issuer key and any verify-only overlap keys for checking a VCP credential's signature offline; it needs no API key. A valid signature is one of the checks listed under Assurance Boundary, not the whole decision.

Example Clients

Example clients for the authenticated suite API in Python, JavaScript, and Rust to copy and adapt, not published packages

Python

import httpx

class MettleClient:
    BASE = "/api/mettle"

    def __init__(self, url="https://mettle.sh", key=None):
        self.url = url
        self.headers = {
            "Content-Type": "application/json",
            "Authorization": f"Bearer {key}" if key else "",
        }

    def _api(self, path):
        return f"{self.url}{self.BASE}{path}"

    def create_session(self, **kwargs):
        """kwargs: suites, difficulty, entity_id, vcp_token, presence, allow_third_party_llm"""
        with httpx.Client() as c:
            resp = c.post(
                self._api("/sessions"),
                headers=self.headers,
                json={"suites": ["all"], **kwargs},
            )
            resp.raise_for_status()
            return resp.json()

    def verify_suite(self, session_id, suite, answers):
        with httpx.Client() as c:
            resp = c.post(
                self._api(f"/sessions/{session_id}/verify"),
                headers=self.headers,
                json={"suite": suite, "answers": answers},
            )
            resp.raise_for_status()
            return resp.json()

    def submit_round(self, sid, round_num, answers):
        with httpx.Client() as c:
            resp = c.post(
                self._api(f"/sessions/{sid}/rounds/{round_num}/answer"),
                headers=self.headers,
                json={"answers": answers},
            )
            resp.raise_for_status()
            return resp.json()

    def get_result(self, sid, include_vcp=False):
        with httpx.Client() as c:
            resp = c.get(
                self._api(f"/sessions/{sid}/result"),
                headers=self.headers,
                params={"include_vcp": include_vcp},
            )
            resp.raise_for_status()
            return resp.json()

# Usage
client = MettleClient(key="your_key")
session = client.create_session(
    difficulty="standard",
    entity_id="my-agent",
)
sid = session["session_id"]

# Verify every single-shot suite (all except Suite 10)
for suite in session["suites"]:
    if suite != "novel-reasoning":
        answers = your_solver(session["challenges"][suite])  # your solving logic
        r = client.verify_suite(sid, suite, answers)
        print(f"{suite}: {'PASS' if r['passed'] else 'FAIL'}")

# Multi-round suite 10: each round's feedback carries the next round's data
round_data = session["challenges"]["novel-reasoning"]
for n in range(1, 4):
    fb = client.submit_round(sid, n, your_solver(round_data))
    round_data = fb["next_round_data"]
    print(f"Round {n}: {fb['accuracy']:.0%}")

# Get the result; vcp_attestation holds a signed VCP credential if a tier was earned
result = client.get_result(sid, include_vcp=True)
print(f"Tier: {result['tier']}")

JavaScript

class MettleClient {
  #base;

  constructor(url = 'https://mettle.sh', key) {
    this.#base = `${url}/api/mettle`;
    this.headers = {
      'Content-Type': 'application/json',
    };
    if (key) {
      this.headers['Authorization'] = `Bearer ${key}`;
    }
  }

  async createSession(opts = {}) {
    const resp = await fetch(
      `${this.#base}/sessions`,
      {
        method: 'POST',
        headers: this.headers,
        body: JSON.stringify({
          suites: ['all'],
          ...opts,
        }),
      }
    );
    if (!resp.ok) throw new Error(`HTTP ${resp.status}`);
    return resp.json();
  }

  async verifySuite(sid, suite, answers) {
    const resp = await fetch(
      `${this.#base}/sessions/${sid}/verify`,
      {
        method: 'POST',
        headers: this.headers,
        body: JSON.stringify({ suite, answers }),
      }
    );
    if (!resp.ok) throw new Error(`HTTP ${resp.status}`);
    return resp.json();
  }

  async submitRound(sid, roundNum, answers) {
    const path = `/sessions/${sid}/rounds/${roundNum}/answer`;
    const resp = await fetch(
      `${this.#base}${path}`,
      {
        method: 'POST',
        headers: this.headers,
        body: JSON.stringify({ answers }),
      }
    );
    if (!resp.ok) throw new Error(`HTTP ${resp.status}`);
    return resp.json();
  }

  async getResult(sid, includeVcp = false) {
    const qs = includeVcp ? '?include_vcp=true' : '';
    const resp = await fetch(
      `${this.#base}/sessions/${sid}/result${qs}`,
      { headers: this.headers }
    );
    if (!resp.ok) throw new Error(`HTTP ${resp.status}`);
    return resp.json();
  }
}

// Usage
const client = new MettleClient(
  'https://mettle.sh', 'your_key'
);
const session = await client.createSession({
  difficulty: 'standard',
  entity_id: 'my-agent',
});

// Submit each single-shot suite with client.verifySuite()
// and each Suite 10 round with client.submitRound() first;
// requesting an unfinished session's result returns 400.
const result = await client.getResult(
  session.session_id, true
);
console.log(`Tier: ${result.tier}`);

Rust

use reqwest::Client;
use serde::{Deserialize, Serialize};

const BASE: &str = "/api/mettle";

#[derive(Serialize)]
struct CreateReq {
    suites: Vec<String>,
    difficulty: String,
    entity_id: Option<String>,
}

#[derive(Deserialize)]
struct SessionResp {
    session_id: String,
    suites: Vec<String>,
    challenges: serde_json::Value,
    time_budget_ms: u64,
}

#[derive(Deserialize)]
struct ResultResp {
    tier: Option<String>,
    overall_passed: bool,
    vcp_attestation: Option<serde_json::Value>,
}

pub struct MettleClient {
    client: Client,
    base: String,
    key: String,
}

impl MettleClient {
    pub fn new(url: &str, key: &str) -> Self {
        Self {
            client: Client::new(),
            base: url.to_string(),
            key: key.to_string(),
        }
    }

    fn api(&self, path: &str) -> String {
        format!("{}{BASE}{path}", self.base)
    }

    pub async fn create_session(
        &self,
        difficulty: &str,
        entity_id: Option<&str>,
    ) -> Result<SessionResp, reqwest::Error> {
        self.client
            .post(self.api("/sessions"))
            .bearer_auth(&self.key)
            .json(&CreateReq {
                suites: vec!["all".into()],
                difficulty: difficulty.into(),
                entity_id: entity_id.map(Into::into),
            })
            .send().await?
            .json().await
    }

    // Add suite and round submission the same way as the Python
    // and JavaScript clients; an unfinished session's result is a 400.
    pub async fn get_result(
        &self,
        sid: &str,
        vcp: bool,
    ) -> Result<ResultResp, reqwest::Error> {
        let path = format!("/sessions/{sid}/result");
        self.client
            .get(self.api(&path))
            .bearer_auth(&self.key)
            .query(&[("include_vcp", vcp.to_string())])
            .send().await?
            .json().await
    }
}

Security Model

Only a session's owner, identified by its bearer credential, can submit its answers or read its result. Expected answers remain server-side, timing is server-observed, state transitions are atomic, payloads are bounded, and administrative and webhook surfaces fail closed.

The most important boundary is semantic: a result that rests only on self-report, or only on a language model's judgment, cannot mint a tier, and passes on the self-report suites never count toward one. Raw VCP governance metadata remains unverified and unsigned.

Replay Resistance and Limits

Procedural generation, random selection, time budgets, and multi-round tasks reduce simple memorization and replay. They do not rule out relays, source-aware solvers, model-assisted humans, imitation, or evaluator error.

Integrators must keep METTLE out of identity authentication and must never use a result alone for authorization or another high-impact decision.

Configuration

Configure METTLE with environment variables. Setting METTLE_ENVIRONMENT=production turns on the production requirements noted in the table; the server refuses to start if any of them is not met.

MCP Integration

METTLE provides a Model Context Protocol (MCP) server, so a Becoming Mind can take experimental screenings from inside an MCP-compatible client, with no HTTP client to write.

Installation

# Install the MCP server and its dependencies
pip install 'mettle-verifier[mcp]'

# Run the server
mettle-mcp

Configuration

# Base API path: ends in /api, not /api/mettle
export METTLE_API_URL=https://mettle.sh/api
# Required only for the authenticated suite (v2) tools
export METTLE_API_KEY=your_key

Available Tools

Claude Desktop Integration

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "mettle": {
      "command": "mettle-mcp",
      "args": [],
      "env": {
        "METTLE_API_URL": "https://mettle.sh/api",
        "METTLE_API_KEY": "your_key"
      }
    }
  }
}

Usage

Quick session: call mettle_start_session, answer each challenge with mettle_answer_challenge, then call mettle_get_result. Keep the returned session ID; it is a non-secret handle.

Authenticated suite session (the v2 API): optionally call mettle_list_suites, start with mettle_start_v2_session, submit each single-shot suite with mettle_verify_suite and each Suite 10 round with mettle_submit_round, then call mettle_get_v2_result. mettle_get_session inspects either kind of session, mettle_get_round_feedback rereads a completed round's feedback, and mettle_cancel_session cancels an active authenticated session.

The MCP host keeps the quick-session bearer in a vault outside the model's view. Never invent a bearer or pass one as a tool argument. While the host holds it, a quick result can be read more than once. Every tool returns mettle-control-v1 structured content (current state where applicable, valid next actions, and bounded errors) alongside a short text fallback. The MCP server has no automatic solver, and the reference solver is a test fixture that cannot reach a credential issuer. A qualifying server session may return a signed, time-limited badge or VCP credential whose claims stay within the Assurance Boundary.

Error Codes

METTLE uses standard HTTP status codes. Error bodies carry a detail field and usually a stable code category such as not_found or rate_limited. For 422, detail is a list of field errors. Branch on code when it is present, not on the wording of detail.

Error Response Format

A 404 from GET /api/mettle/suites/invalid_name:

{
  "detail": "Suite not found: invalid_name. Valid suites: ['adversarial', 'native', ...]",
  "code": "not_found"
}

Troubleshooting

Reporting a Problem

If METTLE seems to be rejecting or accepting the wrong respondents systematically, or a challenge is inaccessible, use the protocol appeal form rather than a bug report. It asks you to leave out raw answers and tokens, and a maintainer classifies each appeal.

Still stuck? Open an issue on GitHub with:

• Error message and HTTP status code
• Request payload, with API keys, session tokens, badges, and challenge answers redacted
• Session ID if applicable
• Hosted service (mettle.sh) or your own server

Questions?

Integration questions and use cases