GeoDualNet: Geometry-Aware Dual-Latent Neural Codec with Epipolar Cross-Attention for Joint Disparity Vector and Depth Prediction in Multi-View Video Coding
Abstract
Abstract Multi-view video plus depth (MVD) coding requires efficient exploitation of temporal, inter-view, and cross-component redundancies. In traditional codecs such as 3D-HEVC, disparity vector (DV) prediction and depth map coding are handled by independent, hand-crafted modules, while the only end-to-end learned multiview video codec (LMVC) avoids explicit disparity estimation and omits depth coding entirely. Neither paradigm exploits the projective duality d = fB/Z that tightly couples disparity and depth. We propose GeoDualNet, the first endto- end neural codec that jointly learns DV prediction and depth map coding through a unified geometric latent space. The architecture introduces five novel components: (1) a Dual-Latent Mutual Refinement (DLMR) scheme in which DV and depth representations iteratively refine each other with provable fixed-point convergence; (2) Epipolar Cross-Attention (ECA) layers that restrict inter-latent interactions to projectively valid correspondences, reducing complexity by ∼6×; (3) an Implicit Geometric Field (IGF) that replaces all hand-crafted warping with a single differentiable neural field conditioned on camera parameters; (4) a Depth-Gated Disparity Flow (DGDF) module that constrains DV search within geometrically plausible corridors derived from coarse depth priors; and (5) a Joint Geometric Entropy Model (JGEM) that conditions arithmetic coding of all latent streams on shared geometric priors for cross-component bitrate savings.…
Read the paper
Related studies
- Regulation of nucleolus assembly by non-coding RNA polymerase II transcripts
- For a Healthy Histone Code, a Little SUMO in the Tail Keeps the Acetyl Away
- Discovering a binary CTCF code with a little help from BORIS
- Article Commentary: Stabilizing the Code–-Methods to Preserve RNA Prove Their Worth
- Foreword to the Special Issue for the RINTC (The Implicit Seismic Risk of Code-Conforming Structures) Project
- Commentary: Promoting Health and Well-being in Pediatric Primary Care Settings: Using Health and Behavior Codes at Routine Well-child Visits
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.