ECCV 2026 论文列表

Robotic manipulation with deformable objects represents adata-intensive regime in embodied learning, where shape, contact, andtopology co-evolve in ways that far exceed the variability of rigids. Al-though simulation promises relief from the cost of real-world data acqui-sition, prevailing sim-to-real pipelines remain rooted in rigid-body ab-stractions, producing mismatched geometry, fragile soft dynamics, andmotion primitives poorly suited for cloth interaction. We posit that simu-lation fails not for being synthetic, but for being ungrounded. To addressthis, we introduce SIM1, a physics-aligned real-to-sim-to-real data enginethat grounds simulation in the physical world. Given limited demon-strations, the system digitizes scenes into metric-consistent twins, cali-brates deformable dynamics through elastic modeling, and expands be-haviors via diffusion-based trajectory generation with quality filtering.vision with near-demonstration fidelity. Experiments show that policiestrained on purely synthetic data achieve parity with real-data baselinesat a 1:15 equivalence ratio, while delivering 90% zero-shot success and50% generalization gains in real-world deployment. These results vali-date physics-aligned simulation as scalable supervision for deformablemanipulation and a practical pathway for data-efficient policy learning.

We revisit the memory update mechanism in SAM2-basedvisual object tracking and identify confidence-only mask selection asthe dominant cause of drift under occlusion, rapid motion, and dis-tractors. We introduce SENTRY, a training-free, plug-and-play, refine-before-write module that validates each memory update for short-horizontemporal consistency before committing it. SENTRY aggregates diversesegmentation hypotheses per frame, backtracks them into short tracklets,and uses neighbor-aware cycle-consistent matching against recent trajec-tories to favor temporally and geometrically consistent masks. It leavesthe base architecture untouched, replacing confidence-driven writes withconsistency-validated ones. For fair evaluation, we re-evaluate major open-source SAM2-based trackers across all available scales and datasets, fill-ing gaps in prior reports. Integrated into five strong baselines, SENTRYdelivers consistent gains across nine benchmarks, achieving new zero-shotSOTA on LaSOT, LaSOText , GOT-10k, VOT20, VOT22, and DiDi. De-spite these checks, the SAM2-L version runs at 32.8 FPS on an A100, andacross compatible hosts adds only about 0.4–0.6 GB VRAM. Our resultsprovide the first unified all-scale evaluation of SAM2-based trackers andshow that enforcing temporal validity at write time stabilizes memory-augmented tracking without retraining.

The Segment Anything Model (SAM) relies heavily on mas-sive manual annotations, creating a fundamental bottleneck for modelscaling. While unsupervised methods attempt to learn object conceptsfrom motion, they typically overfit to moving entities, lacking both multi-granularity understanding and the ability to generalize to static objects.To overcome this, we introduce Motion-Grounded Segment Anything(MoSA), a highly scalable unsupervised framework that learns a trans-ferable objectness prior from unlabeled videos. MoSA operates in threeprogressive stages: (1) automatically generating multi-granularity mo-tion pseudo-labels from large-scale video data; (2) training a PerceptualGrouping Model (PGM) via contrastive learning to internalize a gen-eralized, appearance-driven concept of objects; and (3) transferring thislearned prior into a prompt-guided architecture for segment-anything-style inference on images. Extensive zero-shot evaluations across sevenchallenging benchmarks (e.g., COCO and ADE20K) demonstrate thatMoSA significantly outperforms existing unsupervised methods. Notably,despite using zero manual annotations, MoSA achieves segmentation per-formance comparable to the fully supervised SAM. Our findings revealthat harnessing large-scale unlabeled motion is a feasible and highly scal-able alternative to annotation-driven segment-anything pipelines.

Predicting how a scene may evolve from partial observationsrequires reasoning about multiple possible futures rather than committingto a single trajectory. Existing approaches either generate appearance-dominated video predictions or sample a small number of trajectorieswithout explicitly modeling the distribution of possible motion. We intro-duce Goal Aware Representations of Future kInEmatic Latent Distribu-tions (GARFIELD), a probabilistic model of scene kinematics that learnsa structured spatio-temporal latent representation of the distribution overpossible futures given an image and optional spatio-temporally sparseconstraints. The same latent representation enables both joint samplingof all trajectories and direct access to the underlying motion distributionthrough an efficient deterministic density decoder. As a result, uncer-tainty about future motion can be localized to specific scene elementsand timesteps and progressively refined through additional constraints.Experiments demonstrate strong motion planning performance competi-tive with large video generation models while sampling trajectories 97×faster. Our method further estimates motion densities two orders of mag-nitude faster than Monte-Carlo sampling from motion generation models,enabling interactive exploration and uncertainty-aware planning.Project page: https://compvis.github.io/schroedingers_cat/

As video generation paradigms evolve from localized manip-ulation to full-scene synthesis, AI-generated video detection becomes in-creasingly challenging, as forgeries exhibit coherent global structure andhigh perceptual realism. However, existing benchmarks are biased towardperceptual fidelity and primarily evaluate detectors based on perceptualartifacts, providing limited coverage of scenarios that require reasoningabout violations of physical laws, structural coherence, or social logic.This dataset bias shapes current approaches and results in a Percep-tion–Reasoning Gap: artifact-centric models capture low-level statisticalirregularities yet lack semantic inference, whereas vision-language mod-els perform semantic reasoning but remain insensitive to fine-grainedforensic cues. To bridge this gap, we propose SafeGuard, a multi-agentframework that enables collaborative specialization between forensic per-ception and semantic reasoning. A hierarchical perceptual solver extractsfine-grained forensic evidence, while a self-reflective verifier enforces con-sistency between semantic inference and physical plausibility, forming aninterpretable evidence chain. To support evaluation, we introduce Safe-Vid, a novel AI-generated video detection benchmark comprising 20Kvideos spanning 10 social risk categories, designed to evaluate physicalplausibility, structural consistency, and the rationality of social behav-iors. Extensive experiments demonstrate the generalization of SafeGuard,improving accuracy on SafeVid by +18.7% and consistently outperform-ing prior methods across four public benchmarks. The code and datasetare publicly available at https://github.com/williamw99/SafeGuard.

Representation Alignment (REPA) has emerged as a simpleway to accelerate Diffusion Transformers training in latent space. At thesame time, pixel-space diffusion transformers such as Just image Trans-formers (JiT) have attracted growing attention because they remove adependency on a pretrained tokenizer, and then avoid the reconstruc-tion bottleneck of latent diffusion. This paper shows that the REPA canfail for JiT. REPA yields worse FID for JiT as training proceeds andcollapses diversity on image subsets that are tightly clustered in the rep-resentation space of pretrained semantic encoder on ImageNet. We tracethe failure to an information asymmetry: denoising occurs in the high di-mensional image space, while the semantic target is strongly compressed,making direct regression a shortcut objective. We propose PixelREPA,which transforms the alignment target and constrains alignment witha Masked Transformer Adapter that combines a shallow transformeradapter with partial token masking. PixelREPA improves both train-ing convergence and final quality. PixelREPA reduces FID from 3.66 to3.17 for JiT-B/16 and improves Inception Score (IS) from 275.1 to 284.6on ImageNet 256×256, while achieving > 2× faster convergence. Finally,PixelREPA-H/16 achieves FID= 1.81 and IS= 317.2.

Test-time scaling (TTS) has emerged as a promising paradigm for im-proving the performance of large-scale models. However, existing vision TTSmethods largely rely on global exploration—e.g., resampling noise or rewritingprompts—and often struggle to efficiently search the vast compositional space.As a result, they can exhibit a “scaling plateau,” where additional computationyields diminishing returns in semantic alignment. In this paper, we propose Region-Aware Scaling (RAS), a framework that bridges region-aware generation andtest-time scaling. By treating regional decomposition as a powerful and previ-ously overlooked scaling axis, RAS converts complex compositional promptsinto coordinated regional sub-tasks, effectively increasing the density of validcandidates during inference. At its core, RAS builds on a training-free Region-Aware Generation (RAG). Unlike many layout-based methods that incur sub-stantial overhead, RAG injects regional guidance only during early denoising, en-abling precise attribute binding while preserving global structural coherence. Weevaluate RAS on the GenEval benchmark and observe consistent improvementsin scaling efficiency across diverse compositional challenges. RAS achieves anoverall score of 0.85 with only 2 samples, matching a 32-sample noise-scalingbaseline; with 4 samples, it reaches 0.88, surpassing the combination of 32-sample noise and prompt scaling. Overall, our results suggest that structuringthe search space via regional decomposition provides a principled and compu-tationally efficient direction for scaling compositional alignment.

Colour is one of the most perceptually salient yet least con-trollable attributes in image generation. Although recent diffusion mod-els can modify object colours from user instructions, their results of-ten deviate from the intended hue, especially for fine-grained and localedits. Early text-driven methods rely on discrete language descriptionsthat cannot accurately represent continuous chromatic variations. Toovercome this limitation, we propose ColourCrafter, a unified diffusionframework that transforms colour editing from global tone transfer intoa structured, region-aware generation process. Unlike traditional colour-driven methods, ColourCrafter performs token-level fusion of RGB colourtokens and image tokens in latent space, selectively propagating colourinformation to semantically relevant regions while preserving structuralfidelity. A perceptual Lab-space Loss further enhances pixel-level pre-cision by decoupling luminance and chrominance and constraining ed-its within masked areas. Additionally, we build ColourfulSet, a large-scale dataset of high-quality image pairs with continuous and diversecolour variations. Extensive experiments demonstrate that ColourCrafterachieves state-of-the-art colour accuracy, controllability and perceptualfidelity in fine-grained colour editing. Our project is available at https://yangyuqi317.github.io/ColourCrafter.github.io/

Rapid advancements in image generation have led models likeGPT-Image and Qwen-Image to excel in text-to-image consistency andworld knowledge. However, they still struggle with photorealism, oftenproducing "fake" images laden with obvious AI artifacts, such as overlysmooth skin or unnatural oily sheens. To recapture the original goal of"indistinguishable-from-reality" generation, we propose RealGen, a photo-realistic text-to-image framework. RealGen integrates an LLM componentfor prompt optimization and a diffusion model for realistic image genera-tion. Inspired by adversarial generation, RealGen introduces a "DetectorReward" mechanism, which quantifies artifacts and assesses realism usingboth semantic and feature-level synthetic image detectors. We leveragethis reward with the GRPO algorithm to optimize the entire generationpipeline, significantly enhancing image realism and detail. Furthermore,we propose RealBench, an automated evaluation benchmark employingDetector-Scoring and Arena-Scoring. It enables human-free photorealismassessment, yielding results that are more accurate and aligned with realuser experience. Experiments show that RealGen produces photorealisticimages with enhanced realism, detail, and aesthetics.

Producing 3D human representations from input views onthe fly is essential for immersive live streaming systems, where repre-sentation compactness is as critical as high fidelity given limited com-putational power and transmission bandwidth. Although recent feed-forward reconstruction methods achieve impressive quality through theview-centric prediction of 3D representations, they repeatedly encode thesame subject content across multiple views, leading to significant inter-view redundancy. Our key insight is to perform predictions directly in 3Dspace, enabling the network to learn and produce a highly compact rep-resentation. To this end, we propose PointSplat, a novel human-centricapproach that directly infers Gaussian primitives from an input pointset. The proposed method first estimates a coarse geometric proxy andperforms ray casting to prune redundant points and establish explicit2D–3D correspondences. Subsequently, it employs a Point-Image Trans-former to fuse appearance and geometry features, predicting Gaussianattributes in a single forward pass. This design restricts predictions toforeground regions of interest, substantially reducing the total numberof Gaussians while improving novel-view rendering quality. Extensiveexperiments demonstrate that PointSplat achieves higher efficiency andquality while exhibiting strong robustness to variations in view countand image resolution across multiple datasets.

Reconstructing high-fidelity surfaces of dynamic scenes re-mains a critical challenge. While current methods excel at novel viewsynthesis, they often struggle to recover accurate and stable geometry,particularly under large non-rigid deformations. This results in noisymeshes that are unsuitable for downstream simulation or editing. In thiswork, we introduce a new paradigm for dynamic surface reconstructionbased on a parametric Signed Distance Function (p-SDF). Our key in-sight is to generalize static SDF fields—where each spatial point stores aconstant value—into time-dependent parametric curves where each curvemodels a temporally evolving SDF trajectory. This parametric SDF rep-resentation provides a principled way to capture complex temporal varia-tions, naturally enforcing smoothness and continuity in shape dynamics.At each timestamp, a static SDF field can be queried from p-SDF andconverted into an explicit surface mesh via differentiable iso-surfacing.By rendering these meshes with a physically based differentiable ren-derer, we optimize the underlying parametric curves end-to-end against2D image observations. Our framework produces high-fidelity surfacesand inherently disentangles geometry, material, and lighting from multi-view videos. It robustly reconstructs geometry under large-scale motionsand resolves appearance ambiguities caused by lighting and occlusions.Experiments on both synthetic and real-world scenes demonstrate thatour method achieves state-of-the-art geometric accuracy. The projectpage for p-SDF is available at https://timchonggao.github.io/psdf.

The quality and diversity of instruction-based image editing datasets are continuously increasing, yet large-scale, high-quality datasets for instruction-based video editing remain scarce. To address this gap, we introduce OpenVE-3M, an open-source, large-scale, and high-quality dataset for instruction-based video editing. It comprises two primary categories: spatially-aligned edits (Global Style, Background Change, Local Change, Local Remove, Local Add, and Subtitles Edit) and non-spatially-aligned edits (Camera Multi-Shot Edit and Creative Edit). All edit types are generated via a meticulously designed data pipeline with rigorous quality filtering, a process consuming in excess of 10,000 GPU-days. OpenVE-3M surpasses existing open-source datasets in terms of scale, diversity of edit types, instruction length, and overall quality. Furthermore, to address the lack of a unified benchmark in the field, we construct OpenVE-Bench, containing 431 video-edit pairs that cover a diverse range of editing tasks with three key metrics highly aligned with human judgment. We present OpenVE-Edit, a 5B model trained on our dataset that demonstrates remarkable efficiency and effectiveness by setting a new state-of-the-art on OpenVE-Bench, outperforming all prior open-source models including a 14B baseline.

Autoregressive video diffusion models hold promise for worldsimulation but are vulnerable to exposure bias arising from the train–testmismatch. While recent works address this via post-training, they typi-cally rely on a bidirectional teacher model or discriminator. To achievean end-to-end solution, we introduce Resampling Forcing, a teacher-free framework that enables training autoregressive video models fromscratch and at scale. Central to our approach is a self-resampling schemethat simulates inference-time model errors on history frames during train-ing. Conditioned on these degraded histories, a sparse causal mask en-forces temporal causality while enabling parallel training with frame-leveldiffusion loss. To facilitate efficient long-horizon generation, we furtherintroduce history routing, a parameter-free mechanism that dynamicallyretrieves the top-k most relevant history frames for each query. Experi-ments demonstrate that our approach achieves performance comparableto distillation-based baselines while exhibiting superior temporal consis-tency on longer videos owing to native-length training. See our ProjectPage for more details.

Aligning generative real-world image super-resolution mod-els with human visual preference is challenging due to the perception–fidelity trade-off and diverse, unknown degradations. Prior approachesrely on offline preference optimization and static metric aggregation,which are often non-interpretable and prone to pseudo-diversity understrong conditioning. We propose OARS, a process-aware online align-ment framework built on COMPASS, a MLLM-based reward that eval-uates the LR→SR transition by jointly modeling fidelity preservation andperceptual gain with an input-quality-adaptive trade-off. To train COM-PASS, we curate COMPASS-20K spanning synthetic and real degrada-tions, and introduce a three-stage perceptual annotation pipeline thatyields calibrated, fine-grained training labels. Guided by COMPASS,OARS performs progressive online alignment from cold-start flow match-ing to full-reference and finally reference-free RL via shallow LoRA opti-mization for on-policy exploration. Extensive experiments and user stud-ies demonstrate consistent perceptual improvements while maintainingfidelity, achieving state-of-the-art performance on Real-ISR benchmarks.

Accurate 3D neuron segmentation in fluorescence microscopyis critical for neuroscience. However, the sparse and elongated morphol-ogy of neurons poses significant challenges to existing segmentation meth-ods. These methods struggle to preserve both local details and globaltopology, leading to fragmented results. To address this, we proposeNeuroRefiner, a multi-agent system that formalizes the human expertworkflow involving iterative global observation and local editing. Specifi-cally, NeuroRefiner comprises three collaborative agents dedicated to di-agnosing topological errors, generating correction instructions, and vali-dating refinement quality. To facilitate agent instruction-guided segmen-tation refinement, we propose TopoRefineNet, a dedicated 3D U-Net-based tool that leverages cross-modality feature fusion to generate refinedmasks. Through multi-round agent reasoning and voxel-level editing,NeuroRefiner produces topologically more accurate segmentations withenhanced interpretability. Experiments on the BigNeuron, CWMBS, andZBFWB datasets demonstrate that NeuroRefiner outperforms state-of-the-art methods, notably achieving a 3.02% improvement in F1 score onthe challenging ZBFWB dataset.

Low-light image enhancement (LLIE) on pretrained diffu-sion models is commonly achieved through weighted pixel-space guidancelosses, whose competing gradients often destabilize sampling and inducecolor distortion. We present LUCE, a training-free framework that re-formulates sampler-side guidance as a constrained curve-domain lumi-nance control problem. Instead of directly optimizing pixel intensities,LUCE projects luminance guidance into a low-dimensional monotoniccurve space and enforces a unified luminance target via a single scalarenergy defined on a piecewise-linear LUT. This constrained parameteri-zation restricts guidance to a globally consistent exposure trajectory andremoves the need for balancing multiple heterogeneous losses. To preservechromatic fidelity, we introduce a hue-preserving chromatic decouplingmechanism that realizes luminance edits through per-pixel scalar gainswhile maintaining RGB direction. Implemented as a gradient-based cor-rection within a frozen DDPM sampler, LUCE requires no retraining orarchitectural modification. Experiments on paired and unpaired bench-marks demonstrate stable exposure adjustment and improved color fi-delity over prior training-free diffusion baselines.

In this paper, we present Generative Language-Image Pre-training (GenLIP), a simplified generative pretraining approach for Vi-sion Transformers (ViTs) tailored to multimodal large language models(MLLMs). To better align ViTs with the autoregressive nature of LLMs,GenLIP trains a ViT to predict language tokens directly from visual to-kens using a standard language modeling objective—without contrastivebatch construction or an additional text decoder. GenLIP offers three keyadvantages: (1) Simplicity: a single transformer jointly models visualand linguistic tokens; (2) Scalability: it scales efficiently with both dataand model size; (3) Performance: it achieves competitive or superiorresults across diverse multimodal benchmarks. With only 1/5 of theseen samples, GenLIP matches or surpasses strong baselines such asSigLIP2. With continued pretraining on multi-resolution images at nativeaspect ratios, GenLIP further excels at detail-sensitive tasks such as OCR,chart understanding, and visual question answering, making it a strongfoundation for vision encoders in MLLMs.

Text-guided 3D motion editing has seen success in single-person scenarios, but its extension to multi-person settings is less ex-plored due to limited paired data and the complexity of inter-personinteractions. We introduce the task of multi-person 3D motion editing,where a target motion is generated from a source and a text instruction.To support this, we propose InterEdit3D, a new dataset with man-ual two-person motion change annotations, and a Text-guided Multi-human Motion Editing (TMME) benchmark. We present InterEdit,a synchronized classifier-free conditional diffusion model for TMME. Itintroduces Semantic-Aware Plan Token Alignment with learnable to-kens to capture high-level interaction cues and an Interaction-AwareFrequency Token Alignment strategy using DCT and energy poolingto model periodic motion dynamics. Experiments show that InterEditimproves text-to-motion consistency and edit fidelity, achieving state-of-the-art TMME performance. The dataset and code will be released athttps://github.com/YNG916/InterEdit.

This paper proposes InstaEdit, an acceleration frameworkfor image editing models that reduces the inference steps while preserv-ing, or even improving, generation quality. Unlike existing approaches,the proposed method employed a noise prediction strategy that incorpo-rates both textual instructions and visual conditions into the noise initial-ization process, producing latent variables that are closer to intermediatestates along the generative trajectory. These optimized latent variableswere then used as the starting point for the sampling process. The keycomponent of the framework is a deep noise predictor that estimates amore suitable initial noise state for the generation procedure, therebyreducing redundant denoising steps and improving inference efficiency.InstaEdit can be seamlessly integrated into existing mainstream FlowMatching inference pipelines. We applied InstaEdit to several state-of-the-art image editing models, including Step1X-Edit, FLUX.1-Kontext,and FLUX.2-klein-base-9B. Experimental results demonstrated that In-staEdit achieved acceleration factors of 4.13×, 3.88×, and 4.22×, respec-tively, while maintaining stable and high-quality generative performanceacross multiple standard image editing benchmarks.