Train Pipeline of Codebook-NeRF: Train SR (low-resolution) patches to mimic the characteristics of HR (high-resolution) patches. (a) Input both HR and SR patches into the codebook. (b) Learn high-resolution latent features from the HR patches through the codebook, training SR patches to imitate these features. (c) Reconstruct HR patches' latent features using a decoder, combining the output from the codebook at each deconvolution layer to produce a high-resolution image. (d) Employ a UNet structure to enhance reconstruction by incorporating high-resolution details obtained at each stage as additional input.
Test Pipeline of Codebook-NeRF: Use only SR patches to restore high-resolution images. (a) Input SR patches into the codebook to generate latent representations with high-resolution features. (b) Pass these representations to the decoder to produce the final high-resolution image. (c) Apply high-resolution details learned by the codebook to SR patches, enabling high-resolution restoration without needing reference images.
Visualization: Here are examples of HuGS on different scenes (datasets). More results can be found in the paper and the data.