ECCV 2026 Papers

European Conference on Computer Vision

Decision breakdown

2,834papers

Counted from the official paper list; may differ slightly from the number the conference announced.

  • Poster
    2,80398.9%
  • Other / unlabeled
    311.1%

Top 10 topics (click to filter)

No topic data yet.

Trends for all 207 research topics

Browse all papers

SPOT-E: Test-Time Entropy Shaping with Visual Spotlights for Frozen VLMs

Bo Yin, Xiaobin Hu, Chengming Xu, Ruolin Shen, Mo Yang, Jiangning Zhang, Peng-Tao Jiang, Cheng Tan, Shuicheng YAN

Expand abstract

Vision-language models (VLMs) often underperform on evi-dence intensive tasks because decisive visual evidence are small, localized,and easy to overlook, leading to failures in evidence readout even whenhigh-level reasoning is intact. Prior inference-time visual interventionscan improve grounding without retraining, but they are largely open-loop and lack a mechanism to verify whether highlighted evidence is ac-tually used. We study answer-span prediction entropy as a model-internalfeedback signal and show that naive entropy minimization is ambiguous,since low entropy may arise from evidence-grounded confidence or short-cut collapse. To resolve this ambiguity, we introduce low-entropy anchorsand an entropy-shaping objective that reduces answer uncertainty whilepreserving baseline high-confidence tokens. We instantiate this principlein SPOT-E, a plug-and-play test-time method that produces question-conditioned spotlights, optimized per instance via light-weight tuningbased on Group Relative Policy Optimization (GRPO). Across all bench-marks and different VLM families, SPOT-E yields consistent gains andimproved robustness under visual corruptions. Code is publicly availableat: https://github.com/YinBo0927/SPOT-E

Poster
MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens

Youngrae Kim, Qixin Hu, C.-C. Jay Kuo, Peter Beerel

Expand abstract

Autoregressive diffusion enables real-time frame streaming,yet existing sliding-window caches discard past context, causing fidelitydegradation, identity drift, and motion stagnation over long horizons.Current approaches preserve a fixed set of early tokens as attentionsinks, but this static anchor cannot reflect the evolving content of agrowing video. We introduce MemRoPE, a training-free framework withtwo co-designed components. Memory Tokens continuously compress allpast keys into dual long-term and short-term streams via exponentialmoving averages, maintaining both global identity and recent dynamicswithin a fixed-size cache. Online RoPE Indexing caches unrotated keysand applies positional embeddings dynamically at attention time, ensur-ing the aggregation is free of conflicting positional phases. These twomechanisms are mutually enabling: positional decoupling makes tempo-ral aggregation well-defined, while aggregation makes fixed-size cachingviable for unbounded generation. Extensive experiments validate thatMemRoPE outperforms existing methods in temporal coherence, visualfidelity, and subject consistency across minute- to hour-scale generation.

Poster
LogiCo: A Unified Framework for Logical and Structural Anomaly Detection

Ximiao Zhang, Min Xu, Xiuzhuang Zhou

Expand abstract

Current anomaly detection methods primarily focus on struc-tural anomalies, while paying insu_x001E_cient attention to anomalies that vi-olate logical constraints. Conversely, top-performing logical anomaly de-tection approaches address this by modeling global semantic consistency,but perform poorly on subtle structural anomalies due to inadequate de-tection granularity. In this paper, we propose LogiCo , a uni_x001C_ed frame-work for Logi cal and structural anomaly detection via Co mponent-levelfeature reconstruction. Unlike existing methods that rely on explicitglobal semantic modeling, LogiCo employs a novel component-level fea-ture reconstruction technique to capture inter-component logical con-straints. Speci_x001C_cally, LogiCo maps pre-trained image features into a dis-crete component-level feature space and performs collaborative featurereconstruction at both component and patch levels, enabling it to ef-fectively detect both logical and structural anomalies. Furthermore, toaddress the speci_x001C_c challenge of count-related logical anomalies, we inte-grate a segmentation-map discriminator that extends the model's capa-bility to identify quantitative inconsistencies. LogiCo achieves state-of-the-art performance on both logical and structural anomaly detectionacross four benchmarks, including MVTec-LOCO, MVTec-AD, VisA,and Real-IAD, demonstrating its superiority and practical feasibility.The code is available at https://github.com/cnulab/LogiCo.

Poster
Learning on the Manifold: Unlocking Standard Diffusion Transformers with Representation Encoders

Amandeep Kumar, Vishal Patel

Expand abstract

Leveraging representation encoders for generative modelingoffers a path for efficient, high-fidelity synthesis. However, standard diffu-sion transformers fail to converge on these representations directly. Whilerecent work attributes this to a capacity bottleneck—proposing computa-tionally expensive “width scaling” of diffusion transformers—we demon-strate that the failure is fundamentally geometric. We identify GeometricInterference as the root cause: standard Euclidean flow matching forcesprobability paths through the low-density interior of the hypersphericalfeature space of representation encoders, rather than following the mani-fold surface. To resolve this, we propose Riemannian Flow Matchingwith Jacobi Regularization (RJF). By constraining the generativeprocess to the manifold geodesics and correcting for curvature-inducederror propagation, RJF enables standard Diffusion Transformer architec-tures to converge without width scaling. Our method RJF enables thestandard DiT-B architecture (131M parameters) to converge effectively,achieving an FID of 3.37 where prior methods fail to converge.

Poster
Per‑Object IoU Forecasting for Deadline‑Aware Real‑Time Embedded Detection Control

Erfan Foorginejad, Akshar Chavan, Marco Brocanelli

Expand abstract

Real-time object detection on edge platforms is constrainedby inference latency due to limited resources. Existing runtime controllersadapt detector configurations using coarse frame-level feedback, but failto account for object-level accuracy degradation caused by detection tem-poral mismatch. This leads to suboptimal scheduling decisions under dy-namic workloads. We propose a lightweight per-object IoU decay modelthat predicts accuracy loss and derives closed-form deadline estimatesfor each detected object. These deadline predictions provide fine-grained,deadline-aware feedback that can be integrated into runtime control poli-cies. We integrate the model into a new closed-loop detector selectionframework and evaluate it under streaming conditions on Argoverse-HDand MOT17 across heterogeneous hardware platforms. Experiments onreal hardware testbeds show that our model and control algorithm im-prove deadline prediction accuracy by 56% and streaming AP by 39% onaverage across 71 video scenarios, while incurring acceptable overhead.The code is available at https://github.com/3Eerfan/OLAP-ECCV2026.

Poster
Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Grounding

Yizhou Liu, Fei Tang, Yuchen Yan, Zhengxi Lu, Songqin Nong, Tao Jiang, Wenhao XU, Wenqi Zhang, Weiming Lu, Jun Xiao, Yongliang Shen

Expand abstract

Graphical User Interface (GUI) grounding is essential for au-tonomous agents to map natural language instructions to precise screencoordinates. However, existing supervised fine-tuning and reinforcementlearning methods are constrained by the high cost of annotation, creatinga scalability bottleneck. In this paper, we introduce a label-free test-timetraining paradigm driven by two key insights: (1) confidence patterns incoordinate tokens are a better indicator than full-sequence confidence,and (2) in sparse GUI coordinate spaces, negative samples offer more re-liable learning signals than potentially noisy positive ones. We first pro-pose Confidence-Anchored Learning (CAL), which utilizes coordinate-token confidence to filter pseudo-labels and assign distance-based binaryrewards. Building on this, we develop Confidence-Anchored NegativeLearning (CANL), which exclusively optimizes the model using negativesamples to bypass the risks of incorrect positive samples. Experimen-tal results demonstrate that CANL-7B achieves 92.1% on ScreenSpot-V2. On more challenging ScreenSpot-Pro, CANL-7B reaches 33.8%, an8.9% absolute improvement over the base model. Our findings establishcoordinate-token confidence as a powerful alternative to manual annota-tions for scalable GUI agent development.

Poster
Improved Immiscible Diffusion: Accelerating Diffusion Training by Reducing Miscibility

Yiheng Li, Feng Liang, Dan Kondratyuk, Masayoshi Tomizuka, Kurt Keutzer, Chenfeng Xu

Expand abstract

The substantial training cost of di!usion models hinderstheir deployment. Immiscible Di!usion [20] showed that reducing themixing of images’ di!usion destination in the noise space via linear as-signment can accelerate di!usion training. However, concerns regardinglimited image diversity and exploding execution times under large batchsizes limit its feasibility for large-scale training. In this work, we startfrom thoroughly tackling these limitations: For the image diversity, wedemonstrate the bijective nature of the denoising process of vanilla dif-fusion, underlying that being immiscible cannot hurt the diversity. Mov-ing beyond the noise layer, we refine immiscible di!usion’s concept to abroader miscibility reduction at any layer, which enables us to proposea new family of its implementations much more e"cient to execute un-der high batch sizes, including K-nearest neighbor (KNN) noise selectionand image scaling. Overall, the immiscible di!usion family achieves upto 4→ faster training across diverse models and tasks, including uncon-ditional/conditional generation, image editing, and robotics planning.Extensive analysis shows step-by-step on how immiscibility eases denois-ing and improves e"ciency. Besides, our analysis of immiscibility o!ersa novel perspective on how optimal transport (OT) enhances di!usiontraining. By identifying trajectory miscibility as a fundamental bottle-neck, we believe this work establishes a potentially new direction forfuture research in high-e"ciency di!usion training.

Poster
Hyper-Network Neural Functional Maps for Unsupervised Robust 3D Shape Matching

Dongliang Cao, Florian Bernard

Expand abstract

Functional maps are the cornerstone of recent non-rigid 3D shape matching methods due to their efficiency and performance. However, existing methods struggle with challenging scenarios, such as partiality, topological noise, and raw point clouds. A primary bottleneck is that significant intrinsic distortion prevents truncated spectral bases from being accurately aligned via linear transformations (i.e., functional maps). To address this, we introduce a hyper-network that predicts nonlinear neural functional maps (NFM), learned in an unsupervised manner, to better align spectral bases. Specifically, we model the NFM as an MLP with skip-connection to refine standard FM and employ a hypernetwork to predict its weights, conditioned on standard FM. Our framework is trained using a novel unsupervised spectral alignment loss. Experiments demonstrate that our approach can be seamlessly integrated into state-of-the-art unsupervised deep functional map pipelines, substantially improving matching accuracy in demanding scenarios.

Poster
From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation

Rit Gangopadhyay, Alex Wong

Expand abstract

Vision foundation models are capable of generalizing across3-dimensional (3D) scenes with high-fidelity estimates; their empiricalsuccess can be attributed to training on large-scale datasets of perspec-tive images. However, when transferred to wide field-of-view (FoV) im-ages, such as those captured by fisheye cameras, they return erroneousoutputs due to a covariate shift stemming from the radial distortion onthe image pixels. We propose a method to generalize vision foundationmodels to fisheye cameras. The crux of our method lies in a set of learn-able parameters, termed Distortion Extenders (DEX), that model thefisheye distortion coefficients and the distributional shift between fish-eye and perspective images encoded in the latent space. By minimizing aself-supervised alignment loss, DEX transforms the latent embeddings offisheye images to resemble those of perspective images to recover high-fidelity estimates. DEX is architecture- and task-agnostic: We demon-strate DEX on monocular depth estimation and open-vocabulary seg-mentation for convolution- and Transformer-based architectures, wherewe consistently improve over baselines across indoor and outdoor fisheyedatasets. As a byproduct, the activations of DEX can also be decoded todistortion coefficients to support camera calibration. Code available at:https://github.com/Suchisrit/DEX.

Poster
FlowInOne: Unifying Multimodal Generation as Image-in, Image-out Flow Matching

Junchao Yi, Rui Zhao, Jiahao Tang, Weixian Lei, Linjie Li, Qisheng Su, Zhengyuan Yang, Lijuan Wang, Xiaofeng Zhu, Alex Jinpeng Wang

Expand abstract

Multimodal generation has long been dominated by text-driven pipelines where language dictates vision but cannot reason orcreate within it. We challenge this paradigm by asking whether all modal-ities, including textual descriptions, spatial layouts, and editing instruc-tions, can be unified into a single visual representation. We presentFlowInOne, a framework that reformulates multimodal generation as apurely visual flow, converting all inputs into visual prompts and enablinga clean image-in, image-out pipeline governed by a single flow matchingmodel. This vision-centric formulation naturally eliminates cross-modalalignment bottlenecks, noise scheduling, and task-specific architecturalbranches, unifying text-to-image generation, layout-guided editing, andvisual instruction following under one coherent paradigm. To supportthis, we introduce VisPrompt-5M, a large-scale dataset of 5 millionvisual prompt pairs spanning diverse tasks including physics-aware forcedynamics and trajectory prediction, alongside VP-Bench, a rigorouslycurated benchmark assessing instruction faithfulness, spatial precision,visual realism, and content consistency. Extensive experiments show thatFlowInOne achieves state-of-the-art performance across unified generationtasks, outperforming open-source models and rivaling competitive com-mercial systems. These results establish FlowInOne as a strong foundationfor fully vision-centric generative modeling, where perception and creationare unified within a single continuous visual space. Code is available atcsu-jpg.github.io/FlowInOne.github.io.

Poster
DeltaDeno: Zero-Shot Anomaly Generation via Delta-Denoising Attribution

Chaoran Xu, Chengkan Lv, Qiyu Chen, Yunkang Cao, Feng Zhang, Zhengtao Zhang

Expand abstract

Anomaly generation is often framed as few-shot fine-tuningwith anomalous samples, which contradicts the scarcity that motivatesgeneration and tends to overfit category priors. We tackle the settingwhere no real anomaly samples or training are available. We proposeDelta-Denoising (DeltaDeno), a training-free zero-shot anomaly gener-ation method that localizes and edits defects by contrasting two diffusionbranches driven by a minimal prompt pair under a shared schedule. Byaccumulating per-step denoising deltas into an image-specific localizationmap, we obtain a mask to guide the latent inpainting during later diffu-sion steps and preserve the surrounding context while generating realis-tic local defects. To improve stability and control, DeltaDeno performstoken-level prompt refinement that aligns shared content and strength-ens anomaly tokens, and applies a spatial attention bias restricted toanomaly tokens in the predicted region. Experiments on public datasetsshow that DeltaDeno achieves great generation, realism and consistentgains in downstream detection performance. Code will be made publiclyavailable at https://github.com/CROVO1026/DeltaDeno.

Poster
Fidelity- and Perception-Aware Local Implicit Attention for Arbitrary-Scale Image Super-Resolution

Yu-Syuan Xu, Hao-Lun Sun, Hao-Wei Chen, Hsien-Kai Kuo, Chun-Yi Lee

Expand abstract

Arbitrary-scale image super-resolution (ASISR) aims to recon-struct high-resolution images from low-resolution inputs over a continuousrange of upscaling factors. While traditional pixel-regression approachesoften produce overly smooth results that lack realistic details, recent dif-fusion methods can produce sharper and more realistic textures. However,these diffusion techniques frequently introduce the risk of structural hal-lucinations. To address these issues, we propose Fidelity- and Perception-Aware Local Implicit Attention (FPLIA), a framework that effectivelyintegrates fidelity-oriented features into a diffusion pipeline to produce re-alistic and faithful reconstructions for ASISR. We introduce a Fidelity andPerception Attention Module (FPAM), which applies both self-attentionand cross-attention to fidelity-oriented and perceptual features to en-hance representational capacity. To further exploit their complements,we design a Fidelity and Perception Select Module (FPSM) that adap-tively selects the most representative features for RGB values prediction.We conduct extensive experiments to validate the effectiveness of thesecomponents. Both qualitative and quantitative results show that FPLIAdelivers superior perceptual realism while maintaining reconstructionaccuracy on standard ASISR benchmarks. The source code is accessibleat the following repository: https://github.com/XUSean0118/FPLIA.

Poster
Fast Spatial Memory with Scalable Elastic Test-Time Training

Ziqiao Ma, Xueyang Yu, Haoyu Zhen, Yuncong Yang, Joyce Chai, Chuang Gan

Expand abstract

Large Chunk Test-Time Training (LaCT) has shown strongperformance on long-context 3D reconstruction, but its fully plastic inference-time updates remain vulnerable to catastrophic forgetting and overfit-ting. As a result, LaCT is typically instantiated with a single large chunkspanning the full input sequence, falling short of the broader goal ofhandling arbitrarily long sequences in a single pass. We propose ElasticTest-Time Training inspired by elastic weight consolidation, that sta-bilizes LaCT fast-weight updates with a Fisher-weighted elastic prioraround a maintained anchor state. The anchor evolves as an exponentialmoving average of past fast weights to balance stability and plasticity.Based on this updated architecture, we introduce Fast Spatial Memory(FSM), an efficient and scalable model for 4D reconstruction that learnsspatiotemporal representations from long observation sequences and ren-ders novel view-time combinations. We pre-trained FSM on large-scalecurated 3D/4D data to capture the dynamics and semantics of complexspatial environments. Extensive experiments show that FSM supportsfast adaptation over long sequences and delivers high-quality 3D/4D re-construction with smaller chunks and mitigates the camera-interpolationshortcut. Overall, we hope to advance LaCT beyond the bounded single-chunk setting toward robust multi-chunk adaptation, a necessary stepfor generalization to genuinely longer sequences, while substantially al-leviating the activation-memory bottleneck.

Poster
Event-driven Motion Deblurring via Trajectory-based Kernel Reconstruction

Zhiwei Zhong, Peilin Chen, Wei Dong, Bo Li, Anmin Liu, Shiqi Wang

Expand abstract

Recovering a sharp image from a motion-blurred observation remains challenging due to the inherently ill-posed nature of blind deblurring, especially under real-world non-uniform blur caused by complex camera and scene motion. Event cameras, with their microsecond-level temporal resolution and inherent sensitivity to motion, provide rich motion cues that can serve as strong priors for resolving such ambiguities. However, existing event-guided deblurring methods either rely on simplified blur assumptions or treat event data merely as generic features, without fully exploiting their physical relationship with the image formation process. In this paper, we propose a complete event-driven nonuniform blind deblurring framework that explicitly models the physical formation of spatially varying blur kernels. Specifically, we first estimate dense pixel-wise motion trajectories from the event stream via a differentiable event-alignment objective. These trajectories are then used to construct per-pixel point spread functions (PSFs), which provide a physically grounded blur operator. Based on this operator, we formulate a joint optimization framework that enforces blur consistency, event consistency, and image regularization. Finally, we design an unrolled optimization network that alternates between data-consistency reconstruction and learned image priors. Experiments on both synthetic and real-world datasets demonstrate that the proposed method improves deblurring performance while maintaining low computational complexity.

Poster
Enhancing Pretrained Model-based Continual Representation Learning via Guided Random Projection

Ruilin Li, Heming Zou, Xiufeng Yan, Zheming Liang, Jie Yang, Chenliang Li, Xue Yang

Expand abstract

Recent paradigms in Random Projection Layer (RPL)–basedcontinual representation learning have demonstrated superior performancewhen building upon a pre-trained model (PTM). These methods insert arandomly initialized RPL after a PTM to enhance feature representationin the initial stage. Subsequently, a linear classification head is used foranalytic updates in the continual learning stage. However, under severedomain gaps between pre-trained representations and target domains, arandomly initialized RPL exhibits limited expressivity under large do-main shifts. While largely scaling up the RPL dimension can improveexpressivity, it also induces an ill-conditioned feature matrix, therebydestabilizing the recursive analytic updates of the linear head. To this end,we propose the Stochastic Continual Learner with MemoryGuard Super-visory Mechanism (SCL-MGSM). Unlike random initialization, MGSMconstructs the projection layer via a principled, data-guided mechanismthat progressively selects target-aligned random bases to adapt the PTMrepresentation to downstream tasks. This facilitates the construction of acompact yet expressive RPL while improving the numerical stability ofanalytic updates. Extensive experiments on multiple exemplar-free ClassIncremental Learning (CIL) benchmarks demonstrate that SCL-MGSMachieves superior performance compared to state-of-the-art methods.Project Page: https://rlinl.github.io/SCL/

Poster
Dive into the implicit biases of low-rank vision-language alignment

Mingjia Shi, Shuo Wang, Xiaobo Wang, Sifan Zhou, Kai Wang, Tianyu Fu, Chenxu Zhao, Anyang Su, Ping Jiang, Minghui Wu

Expand abstract

Vision-language alignment—the stage that bridges pretrainedvision encoders and large language models—is widely treated as a formof pretraining requiring full-parameter updates. We challenge this viewand investigate what happens when low-rank adaptation is applied to theLLM during this stage instead. We find that low-rank alignment not onlyreduces computational costs but also outperforms full-parameter align-ment on most benchmarks. To understand this phenomenon, we system-atically characterize the implicit biases introduced by low-rank adap-tation during alignment. Empirically, we find that low-rank alignmentshifts model behavior from hallucinatory to conservative and preservesper-token linear separability of visual features that full-parameter align-ment disrupts—a phenomenon we term LS-curse. Geometrically, low-rank aligned models exhibit more homogeneous and structurally stablevisual representations, maintaining modality-specific knowledge ratherthan prematurely fusing entity-level semantics. Theoretically, we estab-lish two theorems showing that low-rank alignment induces preferencesfor parameter subspaces with flat gradients and feature subspaces robustto perturbations, providing a principled explanation for the observedstructure-preserving behavior. Extensive experiments cover ablation over100 alignment configurations, three families of low-rank operators, andvarious rank, encoder, and other settings.

Poster
DiT as Real-Time Rerenderer: Streaming Video Stylization with Autoregressive Diffusion Transformer

Hengye Lyu, Zisu Li, Yue Hong, Yueting Weng, Jiaxin Shi, Hanwang Zhang, Chen Liang

Expand abstract

Recent advances in video generation models have signifi-cantly accelerated video generation and related downstream tasks. Amongthese, video stylization holds important research value in areas suchas immersive applications and artistic creation, attracting widespreadattention. However, existing diffusion-based video stylization methodsstruggle to maintain stability and consistency when processing long videos,and their high computational cost and multi-step denoising make themdifficult to apply in practical scenarios. In this work, we propose RTR-DiT (DiT as Real-Time Rerenderer), a streaming video stylization frame-work built upon a Diffusion Transformer. We first fine-tune a bidirec-tional teacher model on a curated video stylization dataset, supportingboth text-guided and reference-guided video stylization tasks, and sub-sequently distill it into a few-step autoregressive model via post-trainingwith Self Forcing and Distribution Matching Distillation. Furthermore,we propose a reference-preserving KV cache update strategy that notonly enables stable and consistent processing of long videos, but alsosupports real-time switching between text prompts and reference images.Experimental results show that RTR-DiT outperforms existing meth-ods in both text-guided and reference-guided video stylization tasks, interms of quantitative metrics and visual quality, and demonstrates ex-cellent performance in real-time long video stylization and interactivestyle-switching applications.

Poster
DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space

Wenkun He, Yuchao Gu, Junyu Chen, Junyi Wu, Wenhang Ge, Dongyun Zou, Yujun Lin, Zhekai Zhang, Haocheng Xi, Muyang Li, Ligeng Zhu, Jincheng Yu, Junsong Chen, Enze Xie, Song Han, Han Cai

Expand abstract

Existing latent diffusion models excel at visual generationand editing tasks, employing autoencoders to project RGB images andvideos into latent spaces. However, their low-compression latents pro-duce redundant and excessive visual tokens, limiting scalability to high-resolution content such as 4K images and 2160p videos. One promis-ing approach for acceleration is to leverage deeply compressed latentspaces produced by deep compression autoencoders. However, trainingindustrial-scale models from scratch remains expensive and often fails tomatch the performance of pre-trained models. Alternatively, directly fine-tuning existing pre-trained models on deeply compressed latent spacesis appealing, but it introduces a critical challenge: a representation gapbetween the original latent space and the target compressed space, whichcan result in suboptimal convergence and even training instability. DC-Gen offers the first solution to tackle this challenge. Specifically, we in-troduce a novel embedding alignment process to bridge the representa-tion gap. Once the embedding spaces are aligned, only lightweight LoRAfine-tuning is required to unlock the pre-trained model’s full generativecapability. We validate the effectiveness of DC-Gen across tasks rangingfrom image and video generation to image editing. The resulting DC-Genmodels achieve quality comparable to pre-trained models while deliveringsubstantial speedups. For example, DC-Gen-FLUX reduces the latencyof 4K image generation by 53.8× on a single NVIDIA H100 GPU. Codeis available at https://github.com/dc-ai-projects/DC-Gen.

Poster
Controlling Motion Transfer in Diffusion Transformers via Attention Heads

Sunyoung Jung, Jiwoo Park, Yoonseok Choi, Kyobin Choo, Ming-Hsuan Yang, Seong Jae Hwang

Expand abstract

Diffusion Transformers (DiTs) have advanced video gener-ation with high-quality, temporally coherent results. However, extend-ing them to motion transfer, which requires following reference motionwhile aligning with a target prompt, remains challenging due to lim-ited understanding of motion and structure representations within DiTs.We analyze video DiTs at the attention-head level and identify distinctheads specialized for motion and spatial structure. Based on this insight,we propose a head-aware controllable motion transfer framework thatrequires no parameter updates. Our method refines motion cues frommotion-specialized heads via semantic correspondence guidance and pre-serves structure through selective feature injection. This head-level con-trol not only enables accurate motion transfer but also provides an in-terpretable foundation for controllable video generation with DiTs.

Poster
Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilities

Yuan Xiong, Miao Ziqi, Lijun Li, Chen Qian, Jie Li, Jing Shao

Expand abstract

While Multimodal Large Language Models (MLLMs) showremarkable capabilities, their safety alignments are susceptible to jail-break attacks. Existing attack methods typically focus on text-imageinterplay, treating the visual modality as a secondary prompt. This ap-proach underutilizes the unique potential of images to carry complex,contextual information. To address this gap, we propose a new image-centric attack method, Contextual Image Attack (CIA), which employsa multi-agent system to subtly embed harmful queries into seeminglybenign visual contexts using four distinct visualization strategies. Tofurther enhance the attack’s efficacy, the system incorporates contextualelement enhancement and automatic toxicity obfuscation techniques. Ex-perimental results on the MMSafetyBench-tiny dataset show that CIAachieves high toxicity scores of 4.62 and 4.83 against the GPT-4o andQwen2.5-VL-72B models, respectively, with Attack Success Rates (ASR)reaching 86.83% and 91.02%. Our method significantly outperforms priorwork, demonstrating that the visual modality itself is a potent vector forjailbreaking advanced MLLMs. WARNING: This paper may contain ex-amples of harmful content for research purposes.

Poster