ICML 2026 Papers

Reinforcement learning (RL) has advanced LLM agents on verifiable tasks but remains challenging for open-ended tasks with vast solution spaces (e.g., complex travel planning). Lacking objective ground truth, current RL algorithms rely on reward models assigning scalar scores to individual responses. We contend such pointwise scoring induces discrimination collapse: reward model fails to distinguish subtle advantages among trajectories, compressing intra-group rewards into a narrow range. This drowns effective reward signals in reward model noise, causing optimization stagnation. To address this, we propose ArenaRL, a reinforcement learning paradigm shifting from pointwise scalar scoring to intra-group relative ranking. ArenaRL introduces a process-aware pairwise evaluation with multi-level rubrics for fine-grained relative scoring. Meanwhile, we construct an intra-group adversarial arena and devise a tournament-based ranking scheme to obtain stable advantage signals. ArenaRL achieves high-precision advantage estimation with only $O(N)$ computational complexity, striking a favourable balance between efficiency and accuracy. Furthermore, to address the lack of full-cycle benchmarks for open-ended agents, we introduce two high-quality benchmarks: Open-Travel and Open-DeepResearch, encompassing full training and multi-dimensional evaluation pipelines. Extensive experiments across three open-ended tasks validate the effectiveness of ArenaRL.

Value factorization eases non-stationarity in MARL, but its static coordination assumptions hinder generalization on long-horizon tasks with shifting dependencies. Prior VQ-VAE methods abstract trajectories yet miss time-varying inter-agent dependencies. We present TACTIC, a CTDE framework with three components: (i) VQ-VAE-based trajectory abstraction that learns discrete task-semantic classes; (ii) semantic-conditioned sparse coordination graphs that adapt dependencies by pruning edges according to variance-based pairwise payoff sensitivity; and (iii) a pretrained, frozen trajectory-class predictor that conditions local policies while decoupling task recognition from control. On SMAC and SUMO, TACTIC shows strong overall competitiveness and adaptive coordination under sparse rewards and dynamic task structures.

Scientific reasoning inherently demands integrating sophisticated toolkits to navigate domain-specific knowledge. Yet, current benchmarks largely overlook agents' ability to orchestrate tools for such rigorous workflows. To bridge this gap, we introduce **SciAgentGym**, a scalable interactive environment featuring 1,780 domain-specific tools across four natural science disciplines, supported by a robust execution infrastructure. Complementing this, we present **SciAgentBench**, a tiered evaluation suite designed to stress-test agentic capabilities from elementary actions to long-horizon workflows. Our evaluation identifies a critical bottleneck: state-of-the-art models still struggle with complex scientific tool-use, and their performance degrades substantially as interaction horizons extend. To address this, we propose **SciForge**, a data synthesis method that models the tool action space as a dependency graph to generate logic-aware training trajectories. By fine-tuning on these trajectories, our SciAgent-8B outperforms the significantly larger Qwen3-VL-235B-Instruct while exhibiting positive cross-domain transfer of scientific tool-use capabilities. These results underscore the promising potential of next-generation autonomous scientific agents.