Skip to content

SamplingClient

Derived from the Tinker SDK (Apache-2.0)

Reference adapted from the Tinker SDK; changes have been made for MinT. See the attribution notice and the Apache 2.0 license.

class mint.SamplingClient ( holder , sampling_session_id , shadow=False , retry_config=None , subprocess_sampling=None )

Client for text generation and inference from trained or base models.

The SamplingClient lets you generate text tokens from either a base model or from weights you've saved using a TrainingClient. You typically get one by calling service_client.create_sampling_client() or training_client.save_weights_and_get_sampling_client() .

Key methods:

  • sample() - generate text completions with customizable parameters
  • compute_logprobs() - get log probabilities for prompt tokens

Create method parameters:

  • model_path : Path to saved model weights (starts with 'mindlab-toolkit://')
  • base_model : Name of base model to use for inference (e.g., 'Qwen/Qwen3-8B')
  • retry_config : Configuration for retrying failed requests
sampling_client = service_client.create_sampling_client(base_model="Qwen/Qwen3-8B")
prompt = types.ModelInput.from_ints(tokenizer.encode("The weather today is"))
params = types.SamplingParams(max_tokens=20, temperature=0.7)
future = sampling_client.sample(prompt=prompt, sampling_params=params, num_samples=1)
result = future.result()

Parameters:

  • holder ( InternalClientHolder )
  • sampling_session_id ( str )
  • shadow ( bool , default: False )
  • retry_config ( RetryConfig | None , default: None )
  • subprocess_sampling ( bool | None , default: None )

sample ( prompt , num_samples , sampling_params , include_prompt_logprobs=False , topk_prompt_logprobs=0 )

Generate text completions from the model.

Parameters:

  • prompt ( types. ModelInput ) – The input tokens as ModelInput
  • num_samples ( int ) – Number of independent samples to generate
  • sampling_params ( types. SamplingParams ) – Parameters controlling generation (temperature, max_tokens, etc.)
  • include_prompt_logprobs ( bool , default: False ) – Whether to include log probabilities for prompt tokens
  • topk_prompt_logprobs ( int , default: 0 ) – Number of top token log probabilities to return per position

Returns: A Future containing the SampleResponse with generated text

prompt = types.ModelInput.from_ints(tokenizer.encode("The weather today is"))
params = types.SamplingParams(max_tokens=20, temperature=0.7)
future = sampling_client.sample(prompt=prompt, sampling_params=params, num_samples=1)
result = future.result()
for sample in result.samples:
print(tokenizer.decode(sample.tokens))

Async variant: sample_async()

compute_logprobs ( prompt )

Compute log probabilities for prompt tokens.

Parameters:

  • prompt ( types. ModelInput ) – The input tokens as ModelInput

Returns: A Future containing a list of log probabilities for each token in the prompt. None values indicate tokens where log probabilities couldn't be computed.

prompt = types.ModelInput.from_ints(tokenizer.encode("Hello world"))
future = sampling_client.compute_logprobs(prompt)
logprobs = future.result()
for i, logprob in enumerate(logprobs):
if logprob is not None:
print(f"Token {i}: logprob = {logprob:.4f}")

Async variant: compute_logprobs_async()

get_tokenizer ()

Get the tokenizer for the current model.

Returns: PreTrainedTokenizer compatible with the model

get_base_model ()

Get the base model name for the current sampling session.

Returns: str

Async variant: get_base_model_async()

Referenced by

  • ServiceClient.create_sampling_client
  • TrainingClient.create_sampling_client
  • TrainingClient.save_weights_and_get_sampling_client