NeurIPS 2026 Papers

Positional encoding in Vision Transformers presents a recurring trade-off: spatial relative-position tables (Swin) carry thousands of parameters per block and require ad-hoc interpolation to transfer across resolutions, while no-bias designs leave the attention logits geometry-agnostic. We propose a relative positional bias parameterized directly in the frequency domain. This parameterization yields three benefits over the standard spatial-table position encoding: (i) it expresses the same translation-equivariant bias with $3.25\times$ fewer parameters than Swin's spatial table at the same head count; (ii) it admits a closed-form, structurally exact transfer to a different patch grid, in contrast to bilinear interpolation of spatial tables which smooths the learned structure; and (iii) it shares the same parameterization style as a frequency-domain spectral mixer, enabling co-design of positional encoding and token mixing in a common substrate. We demonstrate the bias inside our **Dual-Domain Residual (DDR)** framework, in which a spectral mixer and a gated attention head act as parallel residuals within each block. Head-to-head, the frequency-domain bias recovers $+1.98$ pp over Swin RelPos and $+2.74$ pp over no positional bias; removing the entire attention branch (which carries the bias) drops accuracy by $5.35$ pp, while removing the spectral branch drops it by $2.22$ pp. On ImageNet-1K from scratch, DDR outperforms SpectFormer at every comparable parameter budget, by $+2.0$ pp at Tiny (DDR-Ti-SF, 10.7M, 78.9\%) and $+0.4$ pp at Base (DDR-B-Deep, 63.1M, 82.5\%); at sub-parity parameters, DDR-Ti+ (8.9M, 77.9\%) exceeds SpectFormer-Ti (9.0M, 76.9\%) by $+1.0$ pp. The learned per-layer gates concentrate spectral processing in early blocks and attention in late blocks, qualitatively matching the schedules that are hand-coded in the previous hybrid models.

Despite advances in vision-language-action (VLA) models, they remain limited in highly dynamic game settings such as 3D open worlds and competitive player versus player (PvP) settings, where agents must efficiently extract sparse, actionable signals from dense visual input while integrating multimodal cues to track off-screen high-value targets in real time. To address this, we introduce 3A-VLA (Abstraction-Aligned Action VLA), a framework that grounds action learning in explicit intention and environment abstractions rather than superficial pattern matching. We introduce dual task-agnostic abstractions: the intention abstraction (IA), which condenses verbose instructions and reasoning into explicit semantic primitives, and the environment semantics abstraction (ESA), which structures dense visual streams into a spatial--functional affordance representation to guide grounded actions. We further propose an abstraction alignment reweighting (AAR) module that adaptively reweights the action imitation loss. A continuous intention--environment alignment signal is used to emphasize reliable action supervision when abstractions agree and reduce the influence of ambiguous demonstrations when they diverge, thereby learning a policy that balances fine-grained control and high-level reasoning without the need for manual rules. Extensive experiments show that 3A-VLA yields state-of-the-art results in both open-world (Minecraft) and competitive PvP (Game for Peace) settings. It also demonstrates strong zero-shot generalizability to high-fidelity games across different domains, including Valorant, CS2, GTA V, Elden Ring, and Mount \& Blade. Code and models will be publicly released at https://3a-vla.github.io/.

Vision-Language-Action Models (VLAs) enable robots to perform manipulation tasks by leveraging Vision-Language Models (VLMs) to process visual observations and language instructions. However, the action prediction objective itself lacks fine-grained task-specific perceptual supervision, making it prone to relying on task-irrelevant visual shortcuts, leading attention away from target objects and key interaction areas, thereby limiting its operational performance and generalization ability. To address this, we propose Automatic Perception-Guided Attention Alignment for Robot Manipulation ($\text{A}^3$-VLA), a framework that derives task-specific visual guidance from the manipulation process itself. Specifically, $\text{A}^3$-VLA first generates heuristic candidate regions for target objects and the robot from interaction-induced visual changes, and refines them with a visual segmentation model offline. These regions serve as explicit perceptual guidance through two complementary objectives: an attention allocation loss that encourages sufficient focus on task-critical regions when language or robot state tokens query the visual input, and a contrastive learning loss that aligns the representations of candidate target regions with language instructions while separating them from task-irrelevant regions. Extensive experiments on simulated and real-world manipulation tasks demonstrate that $\text{A}^3$-VLA consistently improves multiple VLA backbones, with an average success rate gain of 6.5\% - 25.2\% across benchmarks.