vertex-ai-samples



Post-Training Conversational Agents via Self-Evolving Data Synthesis

Authors: Guanghao Mei, Weiran Zhao, Jun-Liang Lin, Siqi Miao, Xuehan Xiong

Table of Contents

1. Introduction

Welcome to the fourth installment of our blog series on model training best practices for customers of Gemini Enterprise Agent Platform training clusters. Large language models (LLMs) have evolved from generic next-token predictors to autonomous agents. Unlike question-answering chatbots, enterprise conversational agents in domains such as banking support and IT helpdesks must coordinate unstructured knowledge retrieval with multi-step tool execution. Across multi-turn interactions, an agent must retrieve domain-specific rules from interconnected policy documents, proactively elicit missing user information, and invoke internal tools to execute policy-compliant state updates.

Despite their potential in business, executing such long-horizon, rule-constrained interactions reliably remains a major challenge. On τ3-Banking, a realistic banking support benchmark, even the highest-performing model solves only about half of the tasks. Errors frequently arise when an agent must resolve implicit cross-turn dependencies, orchestrate interdependent function calls, or enforce policies that conflict with user preferences.

Post-training provides a promising path to adapting models to such domain-specific policies and multi-turn tool workflows. However, post-training open-weight models into effective conversational agents remains challenging due to three fundamental constraints: (i) public corpora lack high-quality multi-turn tool-use trajectories; (ii) manually collecting and verifying domain-specific interaction traces is prohibitively slow and costly; and (iii) multi-turn agentic reinforcement learning (RL) requires long interactive rollouts that bottleneck training throughput. To overcome these constraints, we introduce a self-evolving data synthesis pipeline that synthesizes verifiable multi-turn tasks from an unstructured knowledge base and tools, paired with an end-to-end post-training recipe combining supervised fine-tuning (SFT), RL, and a unified execution harness on Gemini Enterprise Agent Platform training clusters. We apply this recipe to the NVIDIA Nemotron family of open models including recently released NVIDIA Nemotron 3.5 Super VL and NVIDIA Nemotron 3.5 Lightning models using NVIDIA NeMo RL and NeMo Gym, open-source post-training and evaluation libraries within the NVIDIA NeMo Framework, running on NVIDIA AI infrastructure managed by Gemini Enterprise Agent Platform training clusters training clusters.

The main contributions of this blog are summarized as follows:

  1. State-of-the-art performance within size classes: We improve Nemotron-3.5-Lightning-30B-A3B from 7.6% to 32.5% and Nemotron-3.5-Super-VL-120B-A12B from 15.1% to 44.0% on τ3-Banking, outperforming all 2026 open-weight models in their respective size classes, with our post-trained Nemotron-3.5-Super-VL-120B-A12B model achieving performance competitive with frontier models.
  2. A self-evolving data synthesis pipeline: We compile unstructured policy documents and tools into structural dependency and entity graphs, decoupling programmatic state construction from natural-language generation with task-level and pipeline-level evolution.
  3. An efficient end-to-end post-training recipe: We deliver SFT and RL recipes within a unified training and evaluation harness, incorporating systems optimizations that accelerate agentic RL.
Figure 1: τ3-Banking pass@1 comparison with frontier models. Post-trained with our self-evolving synthetic data recipe, Nemotron-3.5-Super-VL-120B-A12B and Nemotron-3.5-Lightning-30B-A3B achieve competitive performance against proprietary frontier models. This figure is for reference only. †: evaluated in our harness; all other results are from the public Artificial Analysis τ3-Banking leaderboard.

2. Task Formulation and Benchmark

2.1 Task Formulation

Multi-turn interactions between an AI assistant and a user present unique challenges that go beyond single-turn tasks. We formulate each task as a two-player turn-taking interaction between an agent and a user within a decentralized partially observable Markov decision process (Dec-POMDP). A task instance \(q = (u, s^{\text{db}}_0, A^\star) \sim \mathcal{D}\) sampled from the task distribution \(\mathcal{D}\) comprises a natural-language user scenario \(u\), an initial relational database state \(s^{\text{db}}_0\), and a sequence of ground-truth reference actions \(A^\star\). At turn \(t\), the environment state \(s_t = (s^{\text{db}}_t, h_t)\) pairs the mutable database state \(s^{\text{db}}_t\) with the cumulative interaction history \(h_t\), initialized at \(s^{\text{db}}_0\) with an empty history \(h_0\).

At each turn \(t\), the active player executes an action \(a_t\), which is either an agent action \(a^{\text{agent}}_t\) or a user action \(a^{\text{user}}_t\). The agent emits a message or invokes a tool from its active toolset \(\mathcal{T}^{\text{agent}}_t\); the user emits a message or invokes a delegated user-side tool from the user toolset \(\mathcal{T}^{\text{user}}\). The transition operator \(P(s_{t+1} \mid s_t, a_t)\) updates \(s^{\text{db}}_t\) upon write-tool execution, leaves \(s^{\text{db}}_t\) invariant under read tools and messages, and appends the action and its output to \(h_t\). At the terminal turn \(T\) (triggered by user termination or step-budget exhaustion), the environment assigns a binary exact-match reward \(R(s_T) \in \{0, 1\}\):

\[R(s_T) = \mathbf{1}\big[\, s^{\text{db}}_T = s^{\text{db}}_\star \,\big], \qquad s^{\text{db}}_\star = \mathrm{Exec}(s^{\text{db}}_0, A^\star), \tag{1}\]

where \(\mathbf{1}[\cdot]\) denotes the indicator function, \(s^{\text{db}}_T\) is the terminal database state, and \(s^{\text{db}}_\star\) is the target database state produced by the deterministic execution operator \(\mathrm{Exec}(s^{\text{db}}_0, A^\star)\) when replaying \(A^\star\) on \(s^{\text{db}}_0\). We evaluate policies via pass@1, the expectation of \(R(s_T)\) across benchmark tasks and independent trials.

2.2 The τ3-Banking Benchmark

Overview

We evaluate our work on τ3-Banking, a fintech customer-support benchmark comprising tasks over a relational database (Figure 2, Table 1). To determine actions and parameter values, the agent must retrieve rules from an unstructured knowledge base \(\mathcal{K}\) of 698 free-text documents spanning FAQs, product specifications, fee schedules, and internal procedures. Because rules governing a single decision are distributed across multiple documents, solving a task requires retrieving an average of 18.6 documents and executing up to 33 tool calls. An LLM simulates the customer conditioned solely on scenario \(u\), revealing private constraints incrementally across turns and executing user-side tools \(\mathcal{T}^{\text{user}}\) when delegated by the agent.
Figure 2: Overview of the τ3-Banking environment. To resolve multi-turn customer requests, the agent retrieves domain policies from an unstructured knowledge base and executes state-mutating actions on the backend database via discovered tools.
Statistic τ3-Banking
Documents (tokens) 698 (~195k)
Tasks 97
Required documents per task (mean) 18.6
Required tool calls per task (mean / max) 9.5 / 33

Table 1. Summary statistics of the τ3-Banking benchmark.

Why τ3-Banking Is Challenging

As mentioned above, even frontier models do not perform well on this benchmark. Trajectory analysis across frontier models identifies four dominant failure modes on τ3-Banking (Figure 3): (i) complex interdependencies, where agents optimize local conditions rather than reasoning across interacting multi-document rules; (ii) implicit subtask ordering, where agents execute multi-part requests in the user’s stated sequence rather than inferring precondition dependencies; (iii) overtrust in user assertions, where agents accept unverified user claims over ground-truth database records; and (iv) premature retrieval and assumption bias, where agents hallucinate constraints on underspecified queries rather than searching broadly. Because three of these failure modes arise from multi-hop rule execution and state verification and the fourth from search under ambiguity, addressing them requires a training distribution \(\mathcal{D}\) that explicitly exercises these behaviors.
Figure 3: Representative failure modes on τ3-Banking. Common errors across frontier models: multi-document rule interdependencies (top-left), implicit subtask ordering violations (top-right), overtrust in unverified user claims (bottom-left), and premature retrieval under ambiguity (bottom-right).

3. Self-Evolving Data Synthesis Pipeline

Synthesizing high-fidelity, multi-turn trajectories that faithfully reflect the dynamics of human–agent collaboration remains a nontrivial problem. Generating an entire task in a single pass is unreliable due to two main bottlenecks. First, multi-step trajectories suffer from severe error propagation: a single hallucination or invalid action at any intermediate state can invalidate the entire downstream task. Second, multi-turn dialogues have strict state dependencies, where each new turn must stay consistent with earlier tool calls and their returned states.

To overcome these limitations, we introduce a self-evolving synthesis pipeline that iteratively synthesizes, verifies, and refines complex training tasks (Figure 4). The pipeline connects five core modules: (i) prebuilt knowledge graphs, (ii) task planning, (iii) an iterative generation-and-review loop, (iv) self-reflection, and (v) final task composition.
Figure 4: Overview of the self-evolving data synthesis pipeline. (1–3) Prebuilt entity and tool graphs ground task planning and generation, paired with deterministic state verification and LLM review for targeted task-level refinement. (4–6) Batch-level reflection evolves generator prompts and sampling weights, while graph-guided composition merges verified single-request tasks into multi-request sessions for SFT and RL.

3.1 Prebuilt Entity Graph and Tool Graph

We construct an entity graph to structure domain knowledge for efficient querying, providing the synthesis agent with accurate context for task generation. In parallel, we build a tool graph to model the complex dependencies among tool calls (Figure 5).

Entity Graph. The entity graph \(\mathcal{G}_{\text{ent}}\) indexes domain products, features, and every numerical or categorical constant \(v\) specified in \(\mathcal{K}\) (e.g., fees, interest rates, and credit limits). Each constant \(v\) is keyed by its determining attributes and paired with an exact provenance citation \(c(v) = (d, \ell)\) pointing to line \(\ell\) of document \(d \in \mathcal{K}\).

Tool Graph. The directed graph \(\mathcal{G}_{\text{tool}} = (\mathcal{V}, \mathcal{E}_{\text{dep}}, \mathcal{E}_{\text{conf}})\) represents each tool call as a node \(a \in \mathcal{V}\) annotated with its state preconditions and legal argument domains. Dependency edges \(\mathcal{E}_{\text{dep}}\) link each tool call to mandatory prerequisite checks (e.g., identity verification) and required post-actions. Conflict edges \(\mathcal{E}_{\text{conf}}\) connect tool calls that cannot co-occur within a session due to mutual exclusion, violated preconditions, or incompatible state mutations.
Figure 5: Prebuilt entity graph and tool graph. (a) The entity graph links products to the rule constants that govern them. (b) The tool graph links each tool call to its preconditions, argument domains, and dependent or conflicting calls.

3.2 Task Generation and Evolution

Task Plan. The planner samples a structured specification \(p\) from \(\mathcal{G}_{\text{ent}}\) and \(\mathcal{G}_{\text{tool}}\) that assembles relevant context, including available tools, domain-specific rules and policies, and constraints. This context grounds the subsequent generation step in the specific constraints and capabilities of the target environment. To ensure plan diversity, we pre-construct a diverse set of plans, each defined by a unique combination of target domains, required tools, user intents, and constraints.

Synthesis Agent. The synthesis agent instantiates \(q = (u, s^{\text{db}}_0, A^\star)\) from each task plan \(p\). We first programmatically construct a task template based on the specific plan, populating randomized identifiers (such as user IDs and transaction IDs) and initializing the evaluation chain with the target tools specified in the plan. Conditioned on this structured context, an LLM samples a user persona and populates the remaining semantic fields in accordance with the plan.

Task Verification. A deterministic verifier validates each candidate task by enforcing format validity, confirming that all required task components are present, and verifying that all database states and tool calls are formatted in valid JSON. The verifier then executes \(s^{\text{db}}_\star = \mathrm{Exec}(s^{\text{db}}_0, A^\star)\) on an isolated database instance to confirm error-free replay.

Review Agent. An LLM reviewer inspects verified tasks alongside their cited policy \(c(v)\) to evaluate conversational coherence and realism, returning one of two verdicts: accept or reject (identifying specific components for targeted regeneration).

Task-Level and Pipeline-Level Evolution. When the task fails verification or is rejected, the task-level loop routes the reviewer’s critique back to the synthesis agent to regenerate only the defective component before re-running verification and review. Across batch \(k\), a self-reflection module aggregates the recorded verification and review failures, identifies recurring defect patterns, and updates the generator prompts and task plan.

3.3 Evolving Towards Real-World Tasks

Real-world customer sessions frequently bundle multiple requests whose preconditions interact or impose implicit execution orderings. Synthesizing such multi-request tasks monolithically causes constraint violations to compound with each added action. Instead, we compose multi-request sessions from verified single-request tasks along \(\mathcal{G}_{\text{tool}}\) (Figure 4, Stage 5).

Specifically, we sample a sequence of actions \(a_1, \dots, a_k\) from \(\mathcal{G}_{\text{tool}}\) such that no pair is connected by a conflict edge in \(\mathcal{E}_{\text{conf}}\) (Figure 6), retrieve verified single-request tasks \(q_1, \dots, q_k\) instantiating those actions, and merge them into a composite task \(q_c = \mathrm{Merge}(q_1, \dots, q_k)\) sharing a unified customer profile, collision-free record IDs, and a single initial identity verification step. An LLM rewrites the individual user scenarios into a coherent multi-intent scenario \(u_c\), after which \(q_c\) undergoes the full verification and review loop.
Figure 6: Composing a multi-request task along the tool graph. Starting from identity verification, the sampled chain (1–3) excludes actions that conflict with an earlier step.

3.4 Constructing SFT and RL Datasets

SFT Dataset (\(\mathcal{D}_{\text{SFT}}\)). We roll out the teacher model against the fixed user simulator across verified single-request tasks within our unified harness, retaining only successful trajectories (\(R(s_T) = 1\)) that exhibit no repetitive turn loops.

RL Dataset (\(\mathcal{D}_{\text{RL}}\)). We use the verified composite multi-request tasks directly as \(\mathcal{D}_{\text{RL}}\). Because each task carries a deterministic target database state \(s^{\text{db}}_\star\), Equation (1) supplies an exact binary reward. We filter out tasks that are too simple or too difficult, enabling RL to optimize the policy on multi-request compositions with a positive learning signal.

4. Training Recipe

We post-train student policies on \(\mathcal{D} = (\mathcal{D}_{\text{SFT}}, \mathcal{D}_{\text{RL}})\) in two stages: supervised fine-tuning on single-request teacher trajectories followed by reinforcement learning on composite multi-request tasks.

4.1 Policy Parameterization and Objective

We parameterize the agent as an autoregressive LLM policy \(\pi_\theta\) with trainable parameters \(\theta\). Conditioned on observation \(o^{\text{agent}}_t\), the policy generates a token sequence \(y_t = (y_{t,1}, \dots, y_{t,\lvert y_t \rvert})\) of length \(\lvert y_t \rvert\) encoding both internal reasoning and an executable action \(a^{\text{agent}}_t = \mathrm{parse}(y_t)\):

\[\pi_\theta\big(y_t \mid o^{\text{agent}}_t\big) = \prod_{i=1}^{\lvert y_t \rvert} \pi_\theta\big(y_{t,i} \mid o^{\text{agent}}_t, y_{t,\lt i}\big), \tag{2}\]

where \(y_{t,\lt i}\) denotes the token prefix preceding index \(i\). The user is modeled by a fixed LLM simulator \(\pi_{\text{user}}(a^{\text{user}}_t \mid o^{\text{user}}_t)\); at each turn \(t\), only the active player invokes its model. Over full trajectories \(\tau = (s_0, a_0, s_1, a_1, \dots, s_T)\), we maximize the expected terminal success rate:

\[\max_\theta \; J(\theta) = \mathbb{E}_{q \sim \mathcal{D},\; \tau \sim P(\cdot \mid q;\, \pi_\theta, \pi_{\text{user}})} \big[ R(s_T) \big]. \tag{3}\]

4.2 Two-Stage Optimization: SFT and GRPO

Supervised Fine-Tuning. We first train \(\pi_\theta\) on \(\mathcal{D}_{\text{SFT}}\) by minimizing the token-level cross-entropy loss over agent completions \(y_t\). User messages and tool outputs remain in the conditioning context \(o^{\text{agent}}_t\) but are masked from the loss gradient, ensuring the model learns agent policy execution rather than environment dynamics.

Reinforcement Learning. Initializing from the SFT checkpoint, we optimize Equation (3) over \(\mathcal{D}_{\text{RL}}\) using Group Relative Policy Optimization (GRPO). For each task \(q\), we sample \(G\) rollouts from the inference engine \(\pi_{\text{rollout}}\), which serves the weights \(\theta_{\text{old}}\) but computes token probabilities that differ numerically from the training engine’s \(\pi_{\theta_{\text{old}}}\). We maximize the token-normalized clipped surrogate objective with an importance sampling correction for this mismatch:

\[J_{\text{RL}}(\theta) = \mathbb{E}_{q \sim \mathcal{D}_{\text{RL}}} \Bigg[ \frac{1}{\sum_{g=1}^{G} N^{(g)}} \sum_{g=1}^{G} m^{(g)} \sum_{t,i} w^{(g)}_{t,i}\, \min\Big( \rho^{(g)}_{t,i}(\theta)\, \hat{A}^{(g)},\; \mathrm{clip}\big(\rho^{(g)}_{t,i}(\theta), 1-\epsilon, 1+\epsilon\big)\, \hat{A}^{(g)} \Big) \Bigg], \tag{4}\]

where \(\rho^{(g)}_{t,i}(\theta) = \pi_\theta(y^{(g)}_{t,i} \mid o^{\text{agent}}_t, y^{(g)}_{t,\lt i}) / \pi_{\theta_{\text{old}}}(y^{(g)}_{t,i} \mid o^{\text{agent}}_t, y^{(g)}_{t,\lt i})\) is the token-level policy ratio, \(w^{(g)}_{t,i} = \mathrm{sg}\big[\pi_{\theta_{\text{old}}}(y^{(g)}_{t,i} \mid o^{\text{agent}}_t, y^{(g)}_{t,\lt i}) / \pi_{\text{rollout}}(y^{(g)}_{t,i} \mid o^{\text{agent}}_t, y^{(g)}_{t,\lt i})\big]\) is the importance sampling correction term, and \(m^{(g)} = \mathbb{1}\big[\, 1-\delta \le \exp\big(\tfrac{1}{N^{(g)}} \sum_{t,i} \log w^{(g)}_{t,i}\big) \le 1+\delta \,\big]\) discards rollouts whose geometric-mean value deviates from one by more than \(\delta\). \(\epsilon\) is the clipping threshold, \(N^{(g)}\) is the count of agent-generated tokens in rollout \(g\), and \(\hat{A}^{(g)} = (R^{(g)} - b^{(g)}) / \sigma^{(g)}\) is the leave-one-out group advantage, with \(R^{(g)}\) denoting the terminal reward of rollout \(g\), and \(b^{(g)}\) and \(\sigma^{(g)}\) denoting the mean and standard deviation of terminal rewards across the remaining \(G-1\) rollouts (omitting division when \(\sigma^{(g)} = 0\)). Equation (4) restricts gradient updates to agent tokens and assigns zero gradient to rollout groups with zero reward variance (\(R^{(1)} = \dots = R^{(G)}\)).

4.3 Harness

We use one harness built with NeMo Gym for teacher trajectory generation, evaluation, and RL rollouts. It runs the interaction of Section 2.1 as three services: the user simulator, the τ3-Banking environment, which relays turns between the two players and executes their tool calls, including knowledge-base search, and the agent policy on vLLM replicas. When an episode ends, the grader computes \(R(s_T)\) (Equation (1)), which gives pass@1 in evaluation and the reward in GRPO. Because an evaluation or a GRPO step finishes only when its slowest episode does, long turns dominate wall time.
Figure 7: Unified rollout harness. The same loop produces teacher trajectories, evaluation scores, and RL rewards.

4.4 Efficiency

We optimized each stage of the pipeline using the NVIDIA NeMo RL post-training library on NVIDIA Hopper GPUs (H100 and H200). Figure 8 shows the optimization ladder for one workload per stage; below, we summarize the key optimizations, with speedups relative to the configuration before each change. Most comparisons use one run per configuration, so differences of a few percent are within noise.
Figure 8: Optimization ladder for each stage. Each bar is the cumulative speedup after adding the change on its row, relative to the first row, on fixed hardware. (a) Qwen3.8-27B evaluation on 4 H100 nodes and one user-simulator node, measured as the time until the first 450 of 485 episodes complete. (b) Nemotron-3.5-Super-120B-A12B SFT on 8 H100 nodes connected over TCP with 262k-token samples, one run per rung. (c) Nemotron-3.5-Lightning-30B-A3B GRPO rollout time on 4 H100 nodes and one user-simulator node with the same batches and a 32,768-token per-turn cap, one run per rung; dropping the per-task seed (gray) removes loop-induced early terminations and therefore lengthens rollouts.

Teacher inference (Qwen3.8-27B).

Supervised fine-tuning (Nemotron-3.5-Super-120B-A12B, 8 nodes over TCP).

Reinforcement learning (Nemotron-3.5-Lightning-30B-A3B).

5. Experiments

5.1 Experimental Setup

Gemini Enterprise Agent Platform training clusters. We orchestrate our large-scale training workloads on Gemini Enterprise Agent Platform training clusters, a managed Google Cloud infrastructure for distributed AI training. Built around an open-source Slurm interface, the platform provides optimized GPU scheduling, automated fault recovery, and hardware-level resiliency, substantially reducing the overhead of provisioning and maintaining production training runs.

Our compute infrastructure spans three hardware generations—A3-Mega, A3-Ultra, and A4 GPU nodes. To minimize inter-node communication bottlenecks during distributed training, the network fabric is tailored to each hardware architecture:

Pairing these specialized interconnect topologies with resilient cluster orchestration provides the communication throughput and scaling efficiency needed to reliably train and fine-tune frontier-scale models.

Models. We evaluate our post-training pipeline across two Mixture-of-Experts (MoE) models: Nemotron-3.5-Lightning-30B-A3B and Nemotron-3.5-Super-VL-120B-A12B. For data synthesis and trajectory generation, we employ Qwen3.8-27B. Throughout both training and evaluation, Gemma-4-31B-it serves as the user simulator \(\pi_{\text{user}}\), generating responses with a sampling temperature of \(1.0\).

Training Framework and Hyperparameters. We integrate the τ3-Banking environment into the NVIDIA NeMo RL framework backed by Megatron-Core. During SFT, models are trained using the Muon optimizer with a peak learning rate of \(5 \times 10^{-5}\) and a maximum context length of \(262{,}144\) tokens. For RL, we retain the \(262{,}144\)-token context window and train with a learning rate of \(2 \times 10^{-6}\), a batch size of \(64\) tasks with \(G = 8\) rollouts per task, and a maximum generation budget of \(32{,}768\) tokens per turn.

Evaluation Protocol. We benchmark all model checkpoints on the full suite of \(97\) tasks in τ3-Banking, reporting \(\text{pass@1}\) accuracy averaged across \(4\) independent trials per task.

5.2 Main Results

Both post-training stages produce pass@1 gains across both student scales (Figure 9). On Nemotron-3.5-Lightning-30B-A3B, SFT improves pass@1 from a base score of 7.6% to 24.2%, and RL yields an additional improvement to reach 32.5%.

On Nemotron-3.5-Super-VL-120B-A12B, SFT raises pass@1 from 15.1% to 38.6%, and RL improves further to reach 44.0%.
Figure 9: τ3-Banking pass@1 across post-training stages. Two-stage SFT and GRPO post-training yields substantial gains over the base checkpoints for both Nemotron-3.5-Lightning-30B-A3B (left) and Nemotron-3.5-Super-VL-120B-A12B (right).

5.3 Comparison with Open-Weight and Frontier Models

Both post-trained models achieve the highest reported pass@1 among 2026 open-weight models in their respective model sizes on τ3-Banking (Figure 10). Within the sub-40B MoE class (Figure 10a), post-trained Nemotron-3.5-Lightning-30B-A3B (32.5%) outperforms K2 Horizon MoVA 36B A4B and Qwen3.6-35B-A3B. Within the 40–300B MoE and 25–35B dense class (Figure 10b), post-trained Nemotron-3.5-Super-VL-120B-A12B (44.0%) surpasses available open-weight models, outperforming the domain-specialized Ling-3.0-flash-Fin and Qwen3.8-27B.


(a) Nemotron 3.5 Lightning size class.

(b) Nemotron 3.5 Super VL size class.
Figure 10: τ3-Banking pass@1 comparison across open-weight model size classes. Both post-trained models outperform all 2026 open-weight baselines in their respective size classes.

Figure 11: τ3-Banking pass@1 comparison of the post-trained models with frontier models. †: evaluated in our harness; all other results are from the public Artificial Analysis τ3-Banking leaderboard.

Compared against frontier models on τ3-Banking (Figure 11), our post-trained Nemotron-3.5-Super-VL-120B-A12B outperforms DeepSeek V4 Pro, achieving competitive performance with Claude Opus 5 and Gemini 3.8 Flash. Similarly, our post-trained Nemotron-3.5-Lightning-30B-A3B performs similarly to Gemini 3.7 Flash.

6. Key Takeaways

Our experiments highlight three practical principles for post-training tool-using agents:

Acknowledgements

We would like to express our sincere gratitude to the NVIDIA NeMo RL team—specifically Terry Kong—as well as the NVIDIA NeMo Gym team—specifically Brian Yu and Chris Wing—for their invaluable support throughout this project.

We would also like to express our gratitude to our Gemini Enterprise Agent Platform training clusters teammates: Youbao Tang, Genquan Duan, Dustin Luong, Mohammadreza Mohseni, Jiuqiang Tang, Jingya Liu, and Pooya Moradi for their infrastructure support, feedback, and insightful discussions throughout the project. We also thank Bo Wu, Ting Yu, Shengyang Dai, and Peng Xu for their leadership and support.

This site is open source. Improve this page.