Skip to content

OpenAI-Compatible Inference

Mint Recursive exposes an HTTP endpoint compatible with the OpenAI Completions API, so you can sample from any checkpoint — a base model or one of your own fine-tuned samplers — using any OpenAI SDK or plain HTTP. It is the quickest way to try a model while training.

For inference inside a training run (e.g. RL rollouts), prefer the native SamplingClient — it is faster and gives you token-level control.

Beta, low-traffic

The endpoint is currently tuned for testing, evaluation, and internal tooling, not high-throughput production traffic. Latency and throughput can vary by model during the beta. For larger or more stable throughput, contact the Mint Recursive team.

Point an OpenAI client at Mint Recursive

  1. Base URL — set your OpenAI client's base URL to the Mint Recursive endpoint:
https://mintcn.macaron.xin/services/tinker-prod/oai/api/v1
  1. Model — pass a Mint Recursive sampler checkpoint path (or a base model name) as the model:
mindlab-toolkit://0034d8c9-0a88-52a9-b2b7-bce7cb1e6fef:train:0/sampler_weights/000080

Any valid sampler path works, and you can keep training and sampling from the same checkpoint at once.

  1. Auth — use your MINT_API_KEY as the OpenAI API key.

Both /completions and /chat/completions are supported:

  • /chat/completions — chat-formatted input. You pass messages; the server applies the model's chat template. The default for instruction-tuned checkpoints.
  • /completions — raw text continuation, no chat template. Typical for base (pretrained) models or completion-style prompting.
  • Need a non-default renderer? Render the prompt to token IDs yourself (see Rendering) and sample with the native SamplingClient (ModelInput.from_ints(...)), which uses your exact tokens.

Example

from os import getenv
from openai import OpenAI

BASE_URL = "https://mintcn.macaron.xin/services/tinker-prod/oai/api/v1"
MODEL_PATH = "mindlab-toolkit://0034d8c9-0a88-52a9-b2b7-bce7cb1e6fef:train:0/sampler_weights/000080"

client = OpenAI(base_url=BASE_URL, api_key=getenv("MINT_API_KEY"))
response = client.completions.create(
    model=MODEL_PATH,
    prompt="The capital of France is",
    max_tokens=50,
    temperature=0.7,
    top_p=0.9,
)
print(response.choices[0].text)

prompt, max_tokens, temperature, and top_p behave like the OpenAI API. The temperature/top_p values above are illustrative, not recommended defaults. Swap MODEL_PATH for any other sampler to compare runs in evals or notebooks.

Splitting reasoning from the answer

For reasoning models, /chat/completions accepts a non-standard separate_reasoning flag. When true (the default), the server pulls the chain-of-thought out of content and returns it on a dedicated reasoning_content field:

response = client.chat.completions.create(
    model=MODEL_PATH,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    extra_body={"separate_reasoning": True},
)
msg = response.choices[0].message
print("Reasoning:", msg.reasoning_content)
print("Answer:", msg.content)
  • Set separate_reasoning=False to keep the reasoning inlined in content.
  • Only on /chat/completions.
  • In streaming mode, reasoning and answer arrive as separate SSE event streams (reasoning first, then the answer).
  • No-op for models without a separable reasoning trace.

Thinking effort

For models that support it, /chat/completions accepts the OpenAI reasoning_effort parameter to set how much the model thinks before answering. Use one of the OpenAI strings — "none", "minimal", "low", "medium", "high", "xhigh" — or a raw float in [0.0, 0.99]:

response = client.chat.completions.create(
    model=MODEL_PATH,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    reasoning_effort="high",
)
# …or a fine-grained float:
response = client.chat.completions.create(
    model=MODEL_PATH,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    extra_body={"reasoning_effort": 0.8},
)
  • The strings map to fixed floats internally: "none" → 0.0, "minimal" → 0.1, "low" → 0.2, "medium" → 0.7, "high" → 0.9, "xhigh" → 0.99. Default 0.9.
  • Not all models support it; unsupported models return HTTP 400.
  • Only on /chat/completions.
  • Quickstart — get a MINT_API_KEY.
  • Rendering — non-default chat templates.
  • SamplingClient API — native, in-loop sampling.