Skip to content

Using a trained model

When training finishes, MindForge produces a sampler_path — a reference like tinker://<session>/sampler_weights/<name>. Find it in the Training page's version records or the Recent training artifacts list. Three ways to use it, ordered by simplicity:

The simplest. The trained model is exposed as an OpenAI-compatible API on the MinT server; you only need the openai SDK.

pip install openai

from openai import OpenAI

client = OpenAI(
    base_url="https://mintcn.macaron.xin/oai/api/v1",  # China
    api_key="sk-your-api-key"
)

# List available models (find your sampler)
models = client.models.list()
for m in models.data:
    print(m.id)

# Inference with the trained model (text completion)
sampler_path = "tinker://xxx/sampler_weights/final-sampler"
response = client.completions.create(
    model=sampler_path,
    prompt="Implement a fast Fourier transform in Python:\n",
    temperature=0.7,
    max_tokens=2048
)
print(response.choices[0].text)

model is the sampler_path, not a HF name

The model field takes the full sampler_path from training, not a HuggingFace model name. The first request waits for MinT to load the LoRA; run models.list() to confirm the path is in the server's index. The server supports both /completions (text completion) and /chat/completions (chat format).

base_url suffix

The OpenAI-compatible base_url must include the /oai/api/v1 suffix. MINT_BASE_URL (used by the mint SDK) does not include it.

Method 2 — mint SDK Sampling Client

For lower-level control (batch sampling, logprobs), use the mint SDK:

pip install git+https://github.com/MindLab-Research/mindlab-toolkit.git

import mint

sc = mint.ServiceClient()

sampler_path = "tinker://xxx/sampler_weights/final-sampler"
sampler = sc.create_sampling_client(model_path=sampler_path).result()

tokenizer = sampler.get_tokenizer()
prompt_tokens = tokenizer.encode("Implement FFT in Python:\n")

result = sampler.sample(
    prompt=mint.ModelInput.from_ints(prompt_tokens),
    sampling_params=mint.SamplingParams(
        max_tokens=1024,
        temperature=0.7,
        stop=["\n\n"],
    ),
    num_samples=1,
).result()

for seq in result.sequences:
    print(tokenizer.decode(seq.tokens))

Method 3 — Download the checkpoint locally

For offline use or migration to other frameworks (vLLM, SGLang), fetch a signed download URL for the checkpoint archive through the MinT REST client:

from mint import ServiceClient

client = ServiceClient()
rest = client.create_rest_client()

url = rest.get_checkpoint_archive_url(
    training_run_id="run-id",
    checkpoint_id="final-sampler",
).result().url  # signed URL, time-limited

# Download the checkpoint archive and extract the LoRA adapter
curl -L "$URL" -o checkpoint.tar.gz
mkdir -p ./my_adapter && tar -xzf checkpoint.tar.gz -C ./my_adapter

# Serve on vLLM with the LoRA adapter
vllm serve Qwen/Qwen3-4B --lora-modules my_adapter=./my_adapter

To merge the adapter into a full HuggingFace model (e.g. for offline export), use peft + transformers:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B")
model = PeftModel.from_pretrained(base, "./my_adapter").merge_and_unload()
model.save_pretrained("./merged_model")
AutoTokenizer.from_pretrained("Qwen/Qwen3-4B").save_pretrained("./merged_model")

Local merge needs the base model

Merging downloads the base model locally (needs enough memory/VRAM), and the signed download URL expires. If you only need online inference, prefer Method 1 or 2.

Endpoint and environment

Network Endpoint
Mainland China https://mintcn.macaron.xin/
Outside Mainland China https://mint.macaron.im/ 
MINT_API_KEY=sk-your-api-key-here
MINT_BASE_URL=https://mintcn.macaron.xin/
MINT_READ_TIMEOUT=180

MINT_BASE_URL is the base for the mint SDK (no /oai/api/v1 suffix). MINT_READ_TIMEOUT bounds both sampler warm-up and subsequent inference requests. See Run settings for pointing the MindForge web console at a different backend at runtime.