论文标题
PIX2NERF：无监督的有条件$π$ - 单图像神经辐射场翻译
Pix2NeRF: Unsupervised Conditional $π$-GAN for Single Image to Neural Radiance Fields Translation
论文作者
论文摘要
我们提出了一条管道，以生成对象的神经辐射场〜（nerf）或特定类的场景，以单个输入图像为条件。这是一项具有挑战性的任务，因为培训NERF需要同一场景的多个视图，再加上相应的姿势，很难获得。我们的方法基于$π$ -GAN，这是一种无条件3D感知图像合成的生成模型，该模型将随机潜在代码映射到一类对象的辐射字段。我们共同优化（1）利用其高保真3D感知的生成和（2）精心设计的重建目标的$π$ gan目标。后者包括一个编码器，并与$π$ -GAN发电机形成自动编码器。与以前的几种NERF方法不同，我们的管道是无监督的，能够接受没有3D，多视图或姿势监督的独立图像培训。我们的管道的应用包括3D化身生成，以对象为中心的新型视图合成具有单个输入图像以及3D感知的超分辨率。
We propose a pipeline to generate Neural Radiance Fields~(NeRF) of an object or a scene of a specific class, conditioned on a single input image. This is a challenging task, as training NeRF requires multiple views of the same scene, coupled with corresponding poses, which are hard to obtain. Our method is based on $π$-GAN, a generative model for unconditional 3D-aware image synthesis, which maps random latent codes to radiance fields of a class of objects. We jointly optimize (1) the $π$-GAN objective to utilize its high-fidelity 3D-aware generation and (2) a carefully designed reconstruction objective. The latter includes an encoder coupled with $π$-GAN generator to form an auto-encoder. Unlike previous few-shot NeRF approaches, our pipeline is unsupervised, capable of being trained with independent images without 3D, multi-view, or pose supervision. Applications of our pipeline include 3d avatar generation, object-centric novel view synthesis with a single input image, and 3d-aware super-resolution, to name a few.
