ReSMapRecasting Satellite Priors for Robust and Accurate Online HD Map Construction

1Abstract

Despite recent advances in camera-based online HD map construction, these systems remain susceptible to sensor failures because of their reliance on onboard cameras. Satellite images, in contrast, remain available independently of real-time sensing conditions, since they are cached offline. Existing methods that use satellite images, however, primarily leverage them under nominal conditions, without explicitly addressing them as redundancy when a sensor degrades.

We propose ReSMap, a redundancy-aware camera–satellite fusion framework for robust online HD map construction. A Predict-then-Fuse (PTF) module fuses camera and satellite BEV features using each branch’s own prediction confidence as a per-pixel reliability weight, with no learned router. An Object-Centric Cooperative Multi-modal Query (OCMQ) decoder then propagates this isolation to the query level: each modality cross-attends only to its own BEV, and per-modality updates are merged through a learned per-token gate.

ReSMap achieves state-of-the-art performance on nuScenes across all splits and range settings. More importantly, it also sets a new state of the art under camera-failure scenarios, demonstrating that satellite images serve not only as a strong prior but as practical redundancy for robust online HD mapping.

2Method

LiDAR is bolted to the same onboard stack that fails, and SD maps carry topology but not lane-level geometry. A satellite tile is different in the one way that matters: it is fetched and cached before the drive, so no onboard fault can take it away. Two pieces spend that, one at the pixel and one at the query.

2.1  Predict-then-Fuse

Each branch segments the BEV on its own, and how confident it is at a pixel becomes that branch’s weight at that pixel. The weight is detached from the gradient, so the fusion never learns a preference between the two:

We call this the Self-Confidence Gate. The fallback is a closed-form property rather than a learned behaviour: as $r_{\text{cam}}$ → 0, the fused feature converges to the satellite feature. Because the gating signal is a prediction confidence and not a raw-image statistic, it also holds up under failures never seen in training, such as cameras zeroed at inference.

2.2  Object-Centric Cooperative Multi-modal Query decoder

A standard DETR-style map decoder undoes this isolation: every query reads from a shared key/value pool that already mixes the three BEVs. The naive fix, one query pool per modality, is worse, because Hungarian matching then routes each ground-truth instance to whichever pool already explains it best, entangling classes with modalities. Dividers and boundaries become camera-only skills, crossings a fusion-only one; when a modality fails, no other branch was ever supervised to take over.

OCMQ keeps a single object-query pool, so the matcher only ever sees one unified set, and builds the modality-specific updates inside each decoder layer instead. Three copies of the query set, each offset by a learnable modality embedding, cross-attend only to their own BEV, and a per-token softmax gate merges them back into one query. There is deliberately no residual to the previous layer, which a down-weighted modality would otherwise re-enter through and accumulate over. Every class therefore receives gradients from all three modalities, while contamination at inference is still controlled per token.

3Quantitative results

The main benchmark is the geographically non-overlapping split of nuScenes, where training and validation cover different parts of each city; the tabs over each table switch to nuScenes’ original split and to Argoverse 2. On nuScenes the satellite input is ego-aligned AID4AD tiles cropped to the BEV region of interest, on Argoverse 2 our own tiles from Google Maps. Except where marked, every baseline is retrained under the same evaluation conditions, following its own paper’s schedule.

3.1  Clean conditions

AP_p, AP_d and AP_b are average precision on pedestrian crossings, lane dividers and road boundaries, each averaged over Chamfer-distance thresholds of 0.5, 1.0 and 1.5 m (1.0, 1.5 and 2.0 m at 100 × 50 m); mAP is the mean of the three.

The margin grows with range. At 60 × 30 m ReSMap leads the best prior method by 12.4 mAP; at 100 × 50 m, where camera evidence thins out with distance, by 18.5. The satellite branch is cheap to carry: with temporal memory ReSMap runs at 10.2 FPS, faster than every LiDAR- and SD-fused method, and without it at 16.1 FPS, faster than the camera-only baseline while still ahead of it by 7.0 mAP.

3.2  Camera failure

Robustness is measured by zeroing the front camera, the three front cameras, and all six cameras at inference time, replacing each dropped view with a blackout image.

4Qualitative results

Pick a scene, then switch the range and turn cameras off while it plays: every method's map changes on the same frame. Grey lines are the ground truth.

5Paper

@inproceedings{kim2026resmap,
  title     = {ReSMap: Recasting Satellite Priors for Robust and
               Accurate Online HD Map Construction},
  author    = {Kim, Kyungmin and Lee, Sumin and Jeong, Sungoh and
               Lim, DoHyun and Park, Seonghyun and Hwang, Soonmin},
  booktitle = {Advances in Neural Information Processing Systems
               (NeurIPS)},
  year      = {2026}
}

Page and volume numbers follow once the proceedings appear. Satellite imagery is the ego-aligned AID4AD release on nuScenes and our own Google Maps crops on Argoverse 2; both datasets are used under their non-commercial licences.