Skip to content

SFT Overview

Supervised fine-tuning (SFT) teaches a language model to produce a target response for a given prompt. You supply labeled (prompt, response) pairs, and Mint Recursive updates the model weights to minimize prediction loss on the response tokens. Prompt tokens carry no loss weight and are masked out.

Configuration

SFT needs a ServiceClient, a LoRA training client, and an optimizer. A minimal setup:

from mint import types

service_client = mint.ServiceClient()
training_client = service_client.create_lora_training_client(
    base_model="Qwen/Qwen3-0.6B",
    rank=16,
    train_mlp=True,
    train_attn=True,
    train_unembed=True,
)

tokenizer = training_client.get_tokenizer()
adam_params = types.AdamParams(learning_rate=5e-5)

Environment variables:

  • MINT_API_KEY: Your Mint Recursive API key (required). Request one from mint.macaron.im.
  • MINT_BASE_URL: Mint Recursive server endpoint. Default: https://mint.macaron.im/train (mainland China: mintcn.macaron.xin/train).
  • MINT_BASE_MODEL: Base model name. Default: Qwen/Qwen3-0.6B.
  • MINT_LORA_RANK: LoRA rank. Default: 16.

Training runs on the remote Mint Recursive server. Your Python script calls forward_backward(...), then blocks on .result() until the batch finishes.

Prompting Guide

SFT computes loss only on the response tokens. To build a Datum from a (prompt, response) pair:

  1. Encode the prompt and the response separately.
  2. Concatenate the token IDs.
  3. Set loss weights to 0.0 on prompt tokens and 1.0 on response tokens.

The canonical pattern, from quickstart.py:

def process_sft_example(ex: dict, tokenizer) -> types.Datum:
    # ex = {"question": "What is 3 * 4?"}
    a, b = map(int, re.findall(r"\d+", ex["question"]))
    answer = str(a * b)
    prompt = f"Question: {ex['question']}\nAnswer:"
    completion = f" {answer}"

    prompt_tokens = tokenizer.encode(prompt, add_special_tokens=True)
    completion_tokens = tokenizer.encode(completion, add_special_tokens=False)
    completion_tokens.append(tokenizer.eos_token_id)

    all_tokens = prompt_tokens + completion_tokens
    all_weights = [0] * len(prompt_tokens) + [1] * len(completion_tokens)

    # Shift for teacher-forcing: input = all_tokens[:-1], target = all_tokens[1:]
    input_tokens = all_tokens[:-1]
    target_tokens = all_tokens[1:]
    weights = all_weights[1:]

    return types.Datum(
        model_input=types.ModelInput.from_ints(tokens=input_tokens),
        loss_fn_inputs={"target_tokens": target_tokens, "weights": weights},
    )

Key points:

  • The prompt is always masked (loss_weight=0.0), so no gradients flow through it.
  • The response is always active (loss_weight=1.0), so the model learns to predict the target tokens.
  • For chat-style instruction tuning, use your tokenizer's apply_chat_template(...) method (see Rendering).

Using Renderers

You can skip the manual token encoding and let mint.recipe renderers turn chat-style messages into Datum objects. The renderer handles the chat template, loss masking, and teacher-forcing shift for you:

renderer = recipe.renderers.get_renderer(
    recipe.get_recommended_renderer_name("Qwen/Qwen3-0.6B"),
    tokenizer,
)

messages = [
    {"role": "user", "content": "What is 3 * 4?"},
    {"role": "assistant", "content": "12"},
]

model_input, weights = renderer.build_supervised_example(messages)
datum = recipe.datum_from_model_input_weights(model_input, weights, max_length=2048)

build_supervised_example() returns (ModelInput, weights), with prompt tokens weighted 0 and response tokens weighted 1. datum_from_model_input_weights() wraps them into a Datum with the correct loss_fn_inputs structure.

For multi-turn conversations, the renderer trains only on the final assistant message by default. Pass train_on_what=TrainOnWhat.ALL_ASSISTANT_MESSAGES to train on every assistant turn.

Output Format

SFT does not generate text; it updates model weights. After each forward_backward step you get back a ForwardBackwardResult with:

  • loss_fn_outputs: a list of per-datum dicts containing logprobs and other loss-function metadata.
  • metrics: aggregated training metrics, when available.

To generate text from the trained model, save the LoRA weights and create a sampling client:

# After SFT training completes:
checkpoint = training_client.save_weights_and_get_sampling_client(name="my-sft-v1")

prompt_ids = tokenizer.encode("3 * 7 =")
samples = checkpoint.sample(
    prompt=types.ModelInput.from_ints(prompt_ids),
    sampling_params=types.SamplingParams(max_tokens=16, temperature=0.7),
    num_samples=4,
)

for seq in samples.sequences:
    print(tokenizer.decode(seq.tokens))

For systematic evaluation — hold-out test sets, benchmark metrics, sampling logs — see Concepts → Evaluations.

All Parameters

Parameter Type Default Meaning
base_model str "Qwen/Qwen3-0.6B" Hugging Face model ID for the base model.
rank int 16 LoRA rank. Higher values are more expressive but use more memory. Typical range: 8–64.
train_mlp bool True Update the MLP (feed-forward) layers.
train_attn bool True Update the attention layers.
train_unembed bool True Update the unembedding (output) layer.
loss_fn str "cross_entropy" Loss function. SFT always uses "cross_entropy"; no alternatives.
learning_rate float 5e-5 Adam learning rate. Typical range for instruction tuning: 1e-5 to 1e-4.
betas tuple[float, float] (0.9, 0.999) Adam exponential decay rates for the 1st and 2nd moments.
eps float 1e-8 Adam numerical stability term.
weight_decay float 0.0 L2 regularization coefficient. Typical range for LoRA: 0.0–0.01.

Usage:

adam_params = types.AdamParams(
    learning_rate=5e-5,
    betas=(0.9, 0.999),
    eps=1e-8,
    weight_decay=0.0,
)

for step, batch in enumerate(batches):
    result = training_client.forward_backward(batch, loss_fn="cross_entropy").result()
    training_client.optim_step(adam_params).result()
    print(f"Step {step}: metrics={result.metrics}")

Tinker compatibility notes:

  • Do not call zero_grad_async(). The Mint Recursive server zeros gradients automatically.
  • The loss_fn parameter is passed to forward_backward(...), not stored in AdamParams.
  • save_weights_for_sampler(...) and save_weights_and_get_sampling_client(...) have the same semantics: both serialize the LoRA weights.