计算机视觉——图像修复综述篇

目录

[1. Deterministic Image Inpainting 判别器图像修复](#1. Deterministic Image Inpainting 判别器图像修复)

[1.1. sigle-shot framework](#1.1. sigle-shot framework)

[(1) Generators](#(1) Generators)

[(2) training objects / Loss Functions](#(2) training objects / Loss Functions)

[1.2. two-stage framework](#1.2. two-stage framework)

[2. Stochastic Image Inpainting 随机图像修复](#2. Stochastic Image Inpainting 随机图像修复)

[2.1. VAE-based methods](#2.1. VAE-based methods)

[2.2. GAN-based methods](#2.2. GAN-based methods)

[2.3. Flow-based methods](#2.3. Flow-based methods)

[2.4. MLM-based methods](#2.4. MLM-based methods)

[2.5. Diffusion model-based methods](#2.5. Diffusion model-based methods)

[3. text-guided image inpainting ⽂本引导的图像修复](#3. text-guided image inpainting ⽂本引导的图像修复)

[4. Inpainting Mask 掩码机制](#4. Inpainting Mask 掩码机制)

[(1) regular mask](#(1) regular mask)

[(2) irregular mask](#(2) irregular mask)

[5. Loss Function 损失函数](#5. Loss Function 损失函数)

[6. Dataset 图像修复领域数据集](#6. Dataset 图像修复领域数据集)

[(1) faces(CelebA & CelebA-HQ)](#(1) faces(CelebA & CelebA-HQ))

[(2) real-world encountered scenes(Places2)](#(2) real-world encountered scenes(Places2))

[(3) street scenes(Paris)](#(3) street scenes(Paris))

[(4) texture(DTD)](#(4) texture(DTD))

[(5) objects (ImageNet)](#(5) objects (ImageNet))

[7. Evaluation Protocol 评估指标](#7. Evaluation Protocol 评估指标)

[7.1. pixel-aware metrics](#7.1. pixel-aware metrics)

[7.2. (human) perception-aware metriics](#7.2. (human) perception-aware metriics)

[8. Performance Evaluation 表现评估](#8. Performance Evaluation 表现评估)

[8.1 Representative Image Inpainting Methods](#8.1 Representative Image Inpainting Methods)

[8.2 Loss Functions](#8.2 Loss Functions)

[9. Inpainting-based Application 基于图像修复的领域应⽤](#9. Inpainting-based Application 基于图像修复的领域应⽤)

[(1) Object Removal](#(1) Object Removal)

[(2) Text Editing](#(2) Text Editing)

[(3) Old Photo Restoration](#(3) Old Photo Restoration)

[(4) Image Compression](#(4) Image Compression)

[(5) Text-guided image editing](#(5) Text-guided image editing)

Reference


1. Deterministic Image Inpainting 判别器图像修复

1.1. sigle-shot framework
(1) Generators
  1. mask-aware design
  2. attention mechanism
  3. multi-scale aggregation
  4. transform domain
  5. encoder-decoder connection
  6. deep prior guidance
(2) training objects / Loss Functions
  1. Pixel-wise reconstruction loss
  2. perceptual loss
  3. style loss
  4. adversarial loss
  5. prevalent training objectives
1.2. two-stage framework

(1) coarse-to-fiine methods
(2) structure-then-texture methods

2. Stochastic Image Inpainting 随机图像修复

2.1. VAE-based methods
2.2. GAN-based methods
2.3. Flow-based methods
2.4. MLM-based methods
2.5. Diffusion model-based methods

(1) sample stratage design
(2) computational cost reduction

3. text-guided image inpainting ⽂本引导的图像修复

4. Inpainting Mask 掩码机制

(1) regular mask
(2) irregular mask

5. Loss Function 损失函数

同1-1.1-(2) training objects

6. Dataset 图像修复领域数据集

(1) faces(CelebA & CelebA-HQ)
(2) real-world encountered scenes(Places2)
(3) street scenes(Paris)
(4) texture(DTD)
(5) objects (ImageNet)

7. Evaluation Protocol 评估指标

7.1. pixel-aware metrics

focus on the precision of reconstructed pixels
(1) l1 error
(1) l2 error
(3) PSNR(peak signal-to-noise ratio)
(4) SSIM(the structure similarity index)
(5) MS-SSIM(muti-scale SSIM)

7.2. (human) perception-aware metriics

the visual perception quality
(1) FID(Frechet Inception diistance)
(2) LPIPS(learned perceptual image patch similarity)
(3) P/U-IDS(pair-unpair Inception discriminative score)

8. Performance Evaluation 表现评估

8.1 Representative Image Inpainting Methods

(1) Models: RFR, MADF, DSI, CR-Fill, CoModGAN, LGNet, RePaint
(2) Dataset: CeleBA-HQ, Places2
(3) Mask: M1, M2, M3, M4, M5, M6
(4) Metrics: l1, PSNR, SSIM, MS-SSIM, FID, LP-IPS
(5) Loss: pixes reconstruction loss, perceptual loss, resnetpl loss, style loss, stylemeanstd,
percept-style loss, lsgan

8.2 Loss Functions

同1-1.1-(2) training objects

9. Inpainting-based Application 基于图像修复的领域应⽤

(1) Object Removal
(2) Text Editing
(3) Old Photo Restoration
(4) Image Compression
(5) Text-guided image editing

Reference

  1. Deep Learning-based Image and Video Inpainting: A Survey
相关推荐
坤坤爱学习2.05 分钟前
求医十年,病因不明,ChatGPT:你看起来有基因突变
人工智能·ai·chatgpt·程序员·大模型·ai编程·大模型学
蹦蹦跳跳真可爱58940 分钟前
Python----循环神经网络(Transformer ----注意力机制)
人工智能·深度学习·nlp·transformer·循环神经网络
空中湖3 小时前
tensorflow武林志第二卷第九章:玄功九转
人工智能·python·tensorflow
lishaoan773 小时前
使用tensorflow的线性回归的例子(七)
人工智能·tensorflow·线性回归
千宇宙航6 小时前
闲庭信步使用SV搭建图像测试平台:第三十一课——基于神经网络的手写数字识别
图像处理·人工智能·深度学习·神经网络·计算机视觉·fpga开发
onceco6 小时前
领域LLM九讲——第5讲 为什么选择OpenManus而不是QwenAgent(附LLM免费api邀请码)
人工智能·python·深度学习·语言模型·自然语言处理·自动化
jndingxin9 小时前
OpenCV CUDA模块设备层-----高效地计算两个 uint 类型值的带权重平均值
人工智能·opencv·计算机视觉
Sweet锦9 小时前
零基础保姆级本地化部署文心大模型4.5开源系列
人工智能·语言模型·文心一言
hie9889410 小时前
MATLAB锂离子电池伪二维(P2D)模型实现
人工智能·算法·matlab
晨同学032710 小时前
opencv的颜色通道问题 & rgb & bgr
人工智能·opencv·计算机视觉