What is Macaron¶
Macaron is an open agent-model family that combines a shared base with specialist LoRA adapters for Chat, Agent, Coding, and Generative UI. Macaron-V1.1 uses a GLM-5.3 base with the Mixture-of-LoRA (MoL) architecture introduced in Macaron-V1, in which a router selects one specialist per user turn.
This page introduces V1.1 and its benchmark results, followed by the MoL architecture from the original V1 release. For the hosted API surface, see Macaron Model. For the model lineup and pricing, see Models.
Macaron-V1.1¶
Macaron-V1.1 is a 752B-parameter agent model post-trained from GLM-5.3. It combines a 744B base model with four 2B LoRA specialists for Chat, Agent, Coding, and Generative UI.
Building on Macaron-V1, V1.1 upgrades the base from GLM-5.2 to GLM-5.3. Using Mint Recursive, our serverless training and inference platform, we completed the iteration from Macaron-V1 to Macaron-V1.1 in two weeks.
Updates and evaluation¶
Long-horizon coding. V1.1 reduces the steps needed to complete extended coding tasks, helping the agent reach a verified result sooner with fewer tokens. At the data level, we standardize software-engineering (SWE) trajectories into five stages: reproduction, localization, editing, validation, and recovery. This reduces rounds of low-relevance exploration. V1.1 scores 71.7 on DeepSWE v1.1 and 50.0 on SWE-Marathon.
Complex office tasks. We improved the model's judgment about when and how to use tools, with particular attention to avoiding unnecessary or inappropriate actions. In workflows involving documents, messages, and business systems, the model needs to recognize which tool to use, when to use it, and when to hold back. V1.1 is better at completing these tasks while staying within the user's intent and authorization, reaching 53.7 on AutomationBench.
We also expanded ChatBench and UI4A-Bench with more real-world cases, bringing both to V2. V1.1 scores 66.7 on ChatBench v2 and 83.2 on UI4ABench v2.
Benchmarks¶
| Category | Benchmark | Macaron-V1.1 | Macaron-V1 | GLM-5.3 | DeepSeek V4 Pro 0813 | Qwen 3.8 Max | Kimi K3 | Claude Opus 5 |
|---|---|---|---|---|---|---|---|---|
| Chat | ChatBench v2 | 66.7 | 63.6 | 65.8 | 65.7 | 62.2 | 57.7 | 66.2 |
| Agent | AutomationBench | 53.7 | 31.8 | 48.2* | 43.2* | 39.8* | 46.7* | 50.3* |
| Agent | Toolathlon-Verified | 76.0 | 63.0 | 73.0* | 74.1* | 72.5* | 73.2* | 80.6* |
| Coding | DeepSWE v1.1 | 71.7 | 58.4 | 66.9* | 62.7* | 57.0* | 69.0* | 68.8* |
| Coding | SWE-Marathon | 50.0 | 15.0 | 42.5* | 10.6* | — | 48.1* | 50.0* |
| Coding | Terminal-Bench 3.0 | 31.4 | 5.7 | 28.7 | 11.8* | 29.0* | 17.7* | 42.7* |
| GenUI | UI4ABench v2 | 83.2 | 75.8 | 77.6 | 77.7 | 79.3 | 80.6 | 81.4 |
Higher is better. Bold marks the highest score per row; * marks an externally sourced result; — means unavailable.
Each benchmark uses the following setup:
| Benchmark | Setup and metric |
|---|---|
| ChatBench v2 | Three independent runs per model–case pair, using the production system prompt, user persona, and relevant conversation history. A privately deployed GLM-5.2 judge rates responses on a 1–5 scale; ratings are averaged over runs and cases and multiplied by 20. |
| AutomationBench | Public 600-task split of v1.0.6 across six business domains, using the API toolset with at most 50 model-response steps per task. |
| Toolathlon-Verified | Official Toolathlon evaluation service; pass@1. |
| DeepSWE v1.1 | Claude Code agent harness; pass@3. External official leaderboard results use mini-swe-agent. |
| SWE-Marathon | pass@2. |
| Terminal-Bench 3.0 | Claude Code v2.1.207; pass@3. Output limit of 32,000 tokens per response; agent timeouts are 10× each task's configured timeout. |
| UI4ABench v2 | 60 UI-generation tasks of varying difficulty. Delivery measures first-attempt success; functional completeness is assessed through execution tests and visual quality through rendered screenshots. |
Externally sourced results use their published settings. See the Macaron-V1.1 model card for the full evaluation protocols and sources.
Training with Mint Recursive¶
Mint Recursive is Mind Lab's automated model training platform. It brings data preparation, training, checkpoint management, and evaluation into one workflow, with one-click deployment for trained models. Evaluation feedback guides updates to training data and configurations, helping teams run and compare successive experiments.
Chat case study¶
We used Mint Recursive to connect capability diagnosis, targeted data updates, and training. One measured iteration covered 16 dimensions. For the second variant of the next training round, the following changes were measured relative to the initial policy on a 1–5 scale:
| Dimension | Change in score |
|---|---|
| Task progress | +0.50 |
| Emotional signal recognition | +0.56 |
| Resistance to sycophancy | +0.56 |
| Timely closure | −0.50 |
Timely closure declined in this iteration. Regressions like this informed the next round of data selection.
UI4A case study¶
We applied the same training workflow to delivery failures, interaction issues, and unclear information hierarchy. These findings guided the selection of training examples, and the updated model was evaluated on the same tasks.
Across 60 tasks, first-attempt delivery success rose from 71.67% (43/60) for GLM-5.3 to 96.67% (58/60) for Macaron-V1.1, a gain of 25 percentage points. Tasks requiring a retry fell from 17 to 2. In a supplier-comparison task, the updated model produced clearer charts and information hierarchy.
How to access¶
Open weights. Find released model weights and model cards in the Macaron-V1 collection on Hugging Face.
Official API. For hosted inference, visit Mint Recursive at mint.macaron.im outside mainland China or mintcn.macaron.xin in mainland China. Use the model ID shown in the dashboard, check Models & Pricing for current rates, and check your account's rate limits. See Models, Parameters, and Usage for the relevant documentation.
Two bets¶
The original Macaron-V1 design is built around two bets:
- Adaptation. A versioned recursive-improvement loop: experience generated by one model-harness revision is evaluated under an external contract and used to construct a successor. Adaptation is a property sought across successive revisions, not in any single checkpoint.
- Collaboration. Composition through MoL: specialists remain separable so that heterogeneous objectives do not interfere through shared parameters. A specialist trained by one team or personalized for one user can be composed with others on the same runtime.
The model family¶
The original Macaron-V1 lineup includes:
| Model | Aggregate params | Base | Specialists | Serving |
|---|---|---|---|---|
| Macaron-V1-Venti | 748B | 744B GLM-5.2 (frozen) | 4 × 1B LoRA | MoL harness |
| Macaron-V1-Tall | 50B | Qwen3.6-35B-A3B (frozen) | 4 × 1B LoRA | MoL harness |
| Macaron-V1-Coding-Venti | 748B | 744B GLM-5.2 | coding LoRA merged into base | single-specialist |
Venti is the flagship: a 744B GLM-5.2 base plus four 1B LoRA specialists, exposed through one OpenAI-compatible endpoint. Tall ships the same four-adapter design on a Qwen3.6-35B-A3B base for local and lower-latency deployment, with LoRA ranks sized for the smaller base. Coding-Venti is the single-specialist exception: the coding LoRA is merged directly into the base rather than served as a MoL adapter.
Mixture-of-LoRA architecture¶
Agent workloads combine tasks with very different chain-of-thought patterns: chat, agentic tool use, coding, and generative UI each ask the model to think in a different shape. When these behaviors share one parameter space, gradients from one family push the model in directions that harm another. MoL exposes this trade-off explicitly through one rule:
Cluster tasks that share skills and thinking patterns into one LoRA, and keep tasks whose skills diverge sharply in separate LoRAs.
Two properties follow:
- The base is frozen. New capabilities are added by training and registering another adapter, not by re-training the base. Base knowledge is not overwritten by later specialization.
- Adapters are portable. Because the base is shared, a specialist trained by one team or personalized for one user can be composed with specialists from another team or user on the same runtime.
The four specialists (Venti / Tall)¶
| Adapter | Role |
|---|---|
| L0, Chat | Conversational backbone, instruction following, model identity. Also the routing entry point. |
| L1, Agent | Long-horizon, heavy tool-use tasks; personal-agent workflows and service integrations. |
| L2, Coding | Code generation, SWE-style tasks, and terminal use. |
| L3, GenUI | UI4A rendering and UI-driven action; specialized on TSX (React and SolidJS). |
The four adapters are resident in the engine under exactly the names L0-L3; L0 is also the routing entry.
The routing loop¶
MoL does not train a separate router model. Adapter selection is decided per user turn by the entry adapter L0's own reasoning and executed by the MoL Proxy. A client request runs a three-stage lifecycle:
- Route. L0 classifies the incoming request into exactly one canonical adapter label (
L0-L3) under a tight decode budget (24 tokens). The router prompt frames the request as quoted, untrusted text and returns a single label under a constrained-decoding grammar. If L0 selects itself, the request stays on L0; otherwise the Proxy switches to the target adapter. - Answer. The chosen specialist responds from its own conversation view, seeded with cross-adapter summaries from prior turns.
- Summary. The specialist emits a short summary of what it just did, capped at 192 output tokens. The Proxy stores this summary server-side and never returns it to the client; it becomes shared context any adapter can inherit on subsequent turns.
The route is decided entirely by L0's reasoning: there is no separate router model and no keyword-based rule library overriding the decision. Routing accuracy is therefore a property of the chat specialist's understanding of the request, and it improves with the base and the L0 adapter rather than with a separate artifact.
Routing cost and accuracy¶
Routing is not free: every user turn adds a routing hop and a summary hop on top of the specialist's own generation. Measured on Venti over 48 multi-turn requests (temperature 0):
| Hop | Venti avg | Share | Tall avg | Share |
|---|---|---|---|---|
| Route (L0 constrained-decode, 24 tok) | 0.54 s | 11% | 0.20 s | 12% |
| Answer (specialist generation) | 3.17 s | 68% | 1.24 s | 70% |
| Summary (192-tok cap) | 0.97 s | 21% | 0.32 s | 18% |
| Total | 4.68 s | 100% | 1.76 s | 100% |
The overhead share (~30%) is stable across base sizes. Routing accuracy on a 6,448-sample trace set is 99.12% (Venti) / 99.04% (Tall) with 100% canonical-label compliance and zero parse errors.
Continual learning and collective intelligence¶
A frozen base plus a plug-in adapter registry is what makes MoL a fit for continual learning and collective intelligence:
- Additive capability. A new capability is added by training a new LoRA and registering it. The router sees one more candidate label; nothing already deployed is retrained.
- No catastrophic forgetting of the base. Gradients enter only adapters, so the base cannot drift as a side effect of specialization.
- Live harness updates. The harness layer (routing rules, tool exposure, configs) can be edited without redeploying weights.
Because the base is shared and adapters are portable, the same properties enable composition across teams and users: a team can ship a specialist for a domain Macaron does not cover in-house, and a user-specific adapter can be trained cheaply and mounted alongside the shipping specialists.
Reference¶
The Macaron-V1.1 section summarizes its release introduction, reported experiments, and the benchmark results in the Macaron-V1.1 model card. The original Macaron-V1 architecture and routing measurements are sourced from the Macaron-V1 technical report. The serving harness is open-sourced at MindLab-Research/Mixture-of-LoRA-Harness.