论文标题
带有扰动面具的图像动画
Image Animation with Perturbed Masks
论文作者
论文摘要
我们提出了一种通过驱动视频描述相同类型对象的源图像的图像动画的新颖方法。我们不假定姿势模型的存在，我们的方法能够在不了解对象结构的情况下对任意对象进行动画动画。此外，驾驶视频和源图像都仅在测试时间内看到。我们的方法基于共享掩码生成器，该生成器将前景对象与其背景区分开，并捕获对象的一般姿势和形状。为了控制输出框架身份的源头，我们采用扰动来中断驾驶员掩模上不需要的身份信息。然后，蒙版改装模块用源的身份代替驱动程序的身份。然后，在源图像上，转换后的掩码由一个多尺度发电机解码，该发电机呈现一个逼真的图像，其中源框架的内容由驾驶视频中的姿势动画。由于缺乏完全监督的数据，我们培训从源图像中重建帧的任务。我们的方法显示出在多个基准测试上的最新方法大大胜过。我们的代码和示例可在https://github.com/itsyoavshalev/image-animation-with-with-with-with-terberted-masks获得。
We present a novel approach for image-animation of a source image by a driving video, both depicting the same type of object. We do not assume the existence of pose models and our method is able to animate arbitrary objects without the knowledge of the object's structure. Furthermore, both, the driving video and the source image are only seen during test-time. Our method is based on a shared mask generator, which separates the foreground object from its background, and captures the object's general pose and shape. To control the source of the identity of the output frame, we employ perturbations to interrupt the unwanted identity information on the driver's mask. A mask-refinement module then replaces the identity of the driver with the identity of the source. Conditioned on the source image, the transformed mask is then decoded by a multi-scale generator that renders a realistic image, in which the content of the source frame is animated by the pose in the driving video. Due to the lack of fully supervised data, we train on the task of reconstructing frames from the same video the source image is taken from. Our method is shown to greatly outperform the state-of-the-art methods on multiple benchmarks. Our code and samples are available at https://github.com/itsyoavshalev/Image-Animation-with-Perturbed-Masks.
