DID : DIMENSION-INVARIANT DISENTANGLING MODEL FOR LIGHT FIELD SPATIAL SUPER-RESOLUTION

4D light field (LF) images can be encoded from 3D scenes by LF cameras, have highly coupled spatial and angular information and cannot be directly utilized. To solve this problem, a disentangling mechanism is proposed in the DistgSSR method. Despite delivering encouraging results of DistgSSR, the parameters of different feature extractors in the Distg-Block of DistgSSR are manually tweaked. Moreover, the concatenated features generated by different extractors should be fused by two convolutions, which will consume unnecessary resources. To address this issue, we propose a dimension-invariant disentangling (DiD) model, which is a plug-and-play model and can adaptively allocate the dimension of the features to fit the residual-in-residual structure. The computational complexity, GPU_Memory and FLOPs of our DiD model are further reduced without losing performance compared with the DistgSSR. Furthermore, our DiD model can be adapted to features of different sizes and may be applied to some networks for other tasks. To further improve the reconstruction result, we integrate a multi-stage spectral dimensional Transformer (MST++) into the DiD model to propose the DiD-MST++ for LF super-resolution. Extensive experimental results on five public LF datasets demonstrate that the DiD and DiD-MST++ can outperform state-of-the-art methods in terms of objective and subjective quality.