Anthropic-Compatible Inference

MinT exposes an HTTP endpoint compatible with the Anthropic Messages API, so anything built for Claude — including Claude Code and the Anthropic SDKs — can point at a MinT model with only a base-URL and API-key change. It works for MinT base models and for your own fine-tuned sampler checkpoints.

For inference inside a training run (e.g. RL rollouts), prefer the native SamplingClient.

Beta, low-traffic

Tuned for testing, evaluation, and internal tooling, not high-throughput production. Latency and throughput can vary by model during the beta. Contact the MinT team for larger setups or to enable access for your organisation.

Point an Anthropic client at MinT

1. Base URL — set your Anthropic client's base URL to:

https://mintcn.macaron.xin/services/tinker-prod/anthropic/api

The client appends /v1/messages (and /v1/messages/count_tokens).

1. Model — use a MinT base model name (e.g. Qwen/Qwen3-8B) or a MinT sampler checkpoint path:

mindlab-toolkit://0034d8c9-0a88-52a9-b2b7-bce7cb1e6fef:train:0/sampler_weights/000080

Any valid sampler path works; you can keep training and sampling from the same checkpoint at once.

1. Auth — pass your MINT_API_KEY. The endpoint accepts the Anthropic SDK's native x-api-key header (no rewriting needed); Authorization: Bearer <key> also works.

Example

from os import getenv
from anthropic import Anthropic

BASE_URL = "https://mintcn.macaron.xin/services/tinker-prod/anthropic/api"
MODEL = "Qwen/Qwen3-8B"

client = Anthropic(base_url=BASE_URL, api_key=getenv("MINT_API_KEY"))
response = client.messages.create(
    model=MODEL,
    max_tokens=512,
    messages=[{"role": "user", "content": "The capital of France is"}],
)
print(response.content[0].text)

max_tokens is required (as with Anthropic). messages, system, temperature, top_p, top_k, and stop_sequences behave as in the Anthropic Messages API. Swap MODEL for any sampler path to compare runs.

Thinking effort

MinT's reasoning models are controlled with an effort value rather than Anthropic's thinking.budget_tokens. Pass it as a MinT-specific output_config.effort through extra_body:

response = client.messages.create(
    model=MODEL,
    max_tokens=1024,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    extra_body={"output_config": {"effort": "high"}},
)
for block in response.content:
    if block.type == "thinking":
        print("Reasoning:", block.thinking)
    elif block.type == "text":
        print("Answer:", block.text)

Turn reasoning off with the standard Anthropic thinking field:

response = client.messages.create(
    model=MODEL,
    max_tokens=512,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    thinking={"type": "disabled"},
)

• effort accepts "low", "medium", "high", "xhigh", or "max" ("max" behaves like "xhigh").
• Not all models support it; unsupported models return HTTP 400.
• thinking={"type": "disabled"} forces reasoning off regardless of output_config. thinking.budget_tokens is accepted for wire compatibility but ignored — use output_config.effort.
• When omitted on a supporting model, a default effort applies.

Tool use

The endpoint round-trips Anthropic tool use. Pass tools with an input_schema; the model replies with tool_use blocks, and you send results back as tool_result blocks in a follow-up user message.

response = client.messages.create(
    model=MODEL,
    max_tokens=1024,
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=[{
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "input_schema": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    }],
)

tool_choice supports auto, any, a named tool, and none. When the model calls a tool, stop_reason is tool_use.

Streaming

Set stream=True (or use client.messages.stream(...)) for standard Anthropic SSE: message_start, then per-block content_block_start / _delta / _stop, then message_delta and message_stop. Thinking blocks stream as thinking_delta, text as text_delta, tool arguments as input_json_delta.

with client.messages.stream(
    model=MODEL,
    max_tokens=512,
    messages=[{"role": "user", "content": "Write a haiku about the ocean."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Counting tokens

POST /v1/messages/count_tokens returns the input token count using the model's real tokenizer and renderer, without sampling:

count = client.messages.count_tokens(
    model=MODEL,
    messages=[{"role": "user", "content": "How many tokens is this?"}],
)
print(count.input_tokens)

Supported and unsupported

Supported: system prompts (string or content-block list), multi-turn, streaming, tool use / results / tool_choice, extended thinking (as thinking blocks via output_config.effort), image inputs (base64 or url), token counting, temperature / top_p / top_k / stop_sequences.

Not supported: prompt caching (cache_control ignored; cache_creation_input_tokens / cache_read_input_tokens always null), citations, audio input, thinking.budget_tokens (accepted, ignored — use output_config.effort), and thinking-block signature values (returned as empty strings).

Related docs

• OpenAI-compatible inference
• Quickstart — get a MINT_API_KEY.
• SamplingClient API — native, in-loop sampling.