Sensor Fusion

Radar-camera depth estimation must turn an ultra-sparse, all-weather, metric radar signal into a dense per-pixel depth map. Existing methods --- concatenation, confidence-aware gating, sparse supervision, graph-based extraction --- combine radar and image features outside the backbone's sequence operator, and even cross-modal Mamba variants leave the selection mechanism itself unimodal. We argue that the selection mechanism is the right place for radar to enter. We introduce Radar-Modulated Selection (RMS), a minimal and principled way to inject radar into Mamba's selective scan: radar modulates the scan from within, adding zero-initialised perturbations to the step size $\boldsymbol{\Delta}$ and readout $\mathbf{C}$ while leaving the input projection $\mathbf{B}$ and state dynamics $\mathbf{A}$ image-only. The construction is exactly equivalent to a pretrained image-only Mamba at initialisation, ensuring radar only influences the model where it improves accuracy. Two further properties follow that out-of-scan fusion cannot offer: linear-cost cross-modal coupling at every recurrence step, and a natural fallback to the image-only backbone when radar is absent. We deploy RMS in a Multi-View Scan Pyramid (MVSP) that matches the fusion operator to radar's spatial reach at each scale. SemoDepth achieves state-of-the-art performance on nuScenes, reducing MAE by 34.0%, 29.9%, and 29.9% over the previous best at 0-50, 0-70, and 0-80 m, while attaining the lowest single-frame latency (26.8 ms). A further ablation shows that out-of-scan feature blending adds no accuracy on top of RMS, providing empirical validation that in-scan selection can replace out-of-scan fusion.

Large-scale annotation of vectorized high-definition (HD) maps requires substantial human effort, motivating local-to-global auto-labeling that incrementally constructs global maps from local vector predictions. In this formulation, limited reliability management allows inaccurate predictions to be accumulated and reused as priors, causing persistent error propagation across iterations. We propose ReVMap, a local-to-global framework for reliable vectorized global map construction. ReVMap introduces a reliability-aware map evolution strategy that unifies uncertainty-guided prior reuse with revisable global map updates. To realize this strategy, ReVMap estimates localization uncertainty for vectorized local map elements and propagates these reliability cues to the accumulated global map. The resulting uncertainty-aware global representation guides subsequent local prediction and facilitates revisable integration through learning-based local-to-global connectivity estimation and uncertainty-aware fusion. ReVMap achieves state-of-the-art results on nuScenes and Argoverse2, with 36.1 mGAP, 43.4 miECM, and 58.0 local mAP on nuScenes, and 62.1 mGAP, 62.5 miECM, and 77.0 local mAP on Argoverse2.

Multi-modal image fusion is crucial for comprehensive scene representation, yet real-world degradations compromise its efficacy, necessitating degradation-robust fusion paradigms. Existing degradation-robust multi-modal image fusion methods are often hindered by cascaded sub-optimal solution or black-box opacity. To address these challenges, we propose a theory-inspired mutual-promoting deep unfolding framework. It reformulates degradation-robust image fusion as a joint optimization problem, utilizing the degradation model and modality generation mechanism to explicitly model both single-task priors and cross-task dependencies. By decomposing the complex multi-task optimization problem into task-related iterative subproblems, the framework establishes a bidirectional reciprocal information flow between restoration and fusion. This process is further unfolded into a multi-stage deep neural network, where each component explicitly corresponds to a specific mathematical operation. In our framework, the fused cross-modal prior actively regularizes the ill-posed restoration, while the purified modality features continuously refine the fusion output. It ensures a transparent architecture that combines the merits of both model-based and data-driven methodologies. Extensive experiments demonstrate that our paradigm achieves state-of-the-art performance while providing superior interpretability.

Real-world multi-modal image fusion faces two fundamental challenges: spatial misalignment and complex degradations induced by heterogeneous imaging sensors. These factors are intrinsically coupled, but most existing fusion methods typically address only one of them, inevitably leading to failure under the other challenge. In this paper, we propose G2Fusion, the first fusion framework that simultaneously addresses image registration and information restoration. By developing a novel geometric-to-generative paradigm, it can directly produce high-quality fused images from unregistered and degraded inputs captured by heterogeneous imaging sensors. This framework consists of two key modules: a flow-based geometric deformation reduction module (Flow-GDR) and a DiT-based generative fusion module (DiT-GF). The former reduces large non-rigid discrepancies through dense flow estimation. The latter performs generative fusion, progressively refining residual misalignments while restoring degraded content through iterative denoising. To enable effective interaction among registration, restoration, and fusion, we design two complementary mechanisms. On the one hand, a target distribution mining strategy is introduced to construct a joint objective distribution from registration, restoration, and fusion, effectively guiding the optimization of DiT-GF. On the other hand, we develop a mutual promotion mechanism that establishes a closed-loop interaction between Flow-GDR and DiT-GF by re-estimating the residual deformation between the fused output and the infrared reference. Extensive experiments demonstrate that G2Fusion consistently outperforms state-of-the-art methods in terms of registration, restoration, and fusion.

Multimodal learning has become a prominent research area, with the potential of substantial performance gains by combining information across modalities. At the same time, model development has trended toward increasingly complex deep learning architectures, motivated by the assumption that multimodal-specific methods improve performance. We challenge this assumption through a large-scale empirical study by reimplementing 19 high-impact multimodal methods across nine diverse datasets with up to 23 modalities. Under standardized experimental conditions, including hyperparameter tuning, weight initialization, cross-validation, and statistical testing, increased multimodal complexity often yields confusion rather than effective fusion of data modalities. Accordingly, complex multimodal architectures do not reliably outperform unimodal baselines and a Simple Baseline for Multimodal Learning (SimBaMM). Through a focused case study, we further demonstrate concrete methodological shortcomings even in top-tier multimodal learning publications, underscoring the need for standardized evaluation practices. In summary, we argue for a shift in focus for multimodal learning: away from the pursuit of architectural novelty and toward methodological rigor.

Class-Incremental Learning (CIL) requires models to continuously acquire new classes without forgetting previously learned ones. A dominant paradigm involves freezing a pre-trained model and training lightweight, task-specific adapters. However, maintaining task-specific parameters hinders knowledge transfer and incurs high retrieval costs, while naive parameter fusion often leads to destructive interference and catastrophic forgetting. To address these challenges, we propose Dynamical Adapter Fusion (DAF) to construct a single robust global adapter. Grounded in the PAC-Bayes theorem, we derive a fusion mechanism that explicitly integrates three components: the optimized task-specific adapter parameters, the previous global adapter parameters, and the initialization parameters. We utilize the Taylor expansion of the loss function to derive the optimal fusion coefficients, dynamically achieving the best balance between stability and plasticity. Furthermore, we propose a Robust Initialization strategy to effectively capture global knowledge patterns. Experiments on multiple CIL benchmarks demonstrate that DAF achieves state-of-the-art (SOTA) performance.

Robots operating in semantic environments often need to satisfy linear temporal logic (LTL) tasks before the semantic map is fully known. Many practical semantic-LTL planners maintain semantic beliefs but execute a single task policy induced by a point semantic interpretation, such as a maximum-a-posteriori (MAP) semantic map, possibly supported by risk checks or active perception. This premature commitment can be brittle when distant semantic observations are weak or correlated. We propose Coherence-Aware Transition-Intent Fusion, an online planner that preserves automaton-relevant semantic uncertainty using a finite top-$K$ intent abstraction. Given a conservative current automaton estimate, the planner propagates plausible semantic assignments through the automaton, converts feasible successor transitions into weighted reach-avoid intents, and fuses their risk-gated Dijkstra progress scores. When competing intents create incoherent motion, a KL-regularized calibration step shifts weight toward intents consistent with recent physical progress. Experiments on random-obstacle and room-like semantic maps show improved satisfaction over single-intent and raw-fusion variants, and over a TOAPP* MAP-plus-active-perception baseline under the same static-view sensor. Ablations attribute the gain primarily to intent-level calibration.

Multi-View Stereo (MVS) and Monocular Depth Estimation (MDE) provide complementary cues for dense 3D reconstruction. MDE offers rich contextual and structural priors but lacks explicit geometric constraints, whereas MVS exploits multi-view geometry but often struggles in ambiguous regions, such as weakly textured or reflective surfaces. Existing MDE-assisted MVS methods mainly use monocular cues in a unidirectional manner, leaving the interaction between monocular representations and multi-view cost volumes insufficiently explored. To address this limitation, we introduce BridgeMVS, a unified cascaded MVS framework that bridges monocular depth features and multi-view cost-volume representations through Bidirectional Dynamic Fusion (BDF). At each cascade stage, BDF performs dynamic mono-to-volume and volume-to-mono updates: monocular structural features are injected into cost-volume regularization, while multi-view geometric evidence recalibrates intermediate monocular representations. The enhanced monocular features are propagated across stages together with MVS representations and supervised by stage-wise monocular losses, establishing an auxiliary structural guidance path for the MVS branch. Unlike stage-isolated designs, BridgeMVS preserves fused representations from coarse to fine stages, allowing low-resolution cross-branch interactions to guide subsequent high-resolution depth estimation. Moreover, BDF is designed as a plug-and-play module and can be conveniently integrated into existing cascaded MVS frameworks. Extensive experiments show that BridgeMVS achieves state-of-the-art performance on the DTU, Tanks and Temples, and ETH3D benchmarks.

Many decision-level fusion methods implicitly assume that semantically aligned soft outputs can be directly aggregated once they share the same label simplex. However, this assumption can fail even under identical label semantics, as oracle-specific differences in confidence sharpness, boundary uncertainty, and tail-mass allocation may place soft outputs in incompatible probability geometries and bias direct soft fusion. To address this issue, we formalize this failure mode as geometric incomparability and propose a pre-fusion comparability recovery framework, which learns a shared consensus through constrained oracle-specific rectifiers on the probability simplex. The framework includes a closed-form variant with frozen geometry descriptors and an alternating-refinement variant with consensus-dependent descriptors. Theoretically, we analyze the rectifier family, prove exact recovery under matched radial distortions, characterize the bias of direct fusion, and establish stable recovery under approximate mismatch. Experiments on controlled synthetic stress tests and real black-box decision-level settings show that comparability recovery consistently reduces fusion bias and improves reliability over direct aggregation. These results indicate that geometric comparability is a necessary condition for reliable soft-output fusion. The code is available at \url{https://anonymous.4open.science/r/CFCR-ACR-B8F1}.