CNNs for image processing and other applications

CNNs for image processing and other applications


CNNs - convolutional neural networks

A structure to simulate the brain's visual cortex. They can both perform well in CV [object detection (classifying multiple objects in an image and placing bounding boxes around them) and semantic segmentation (classifying each pixel according to the class of the object it belongs to)] and NLP.

Why don't we use the Dense Layers? Although Dense Layers works well with MNIST dataset, they're hard to scale up (i.e., to deal with relative large image; e.g., 100 × 100) as the number of parameters will explode. CNN solves this problem by using partial connections and sharing weights.

CNNs actually use the similar "cross-correlations" operation instead of using "convolution".

Remark: these local receptive fields are rectangles

In real world, the image is hierarchical. This may be the reason that a stack of CNN layers succeed in image recognition.

Supplement 1: we have to line up input tensors as dense layers follow the principle: 1 sample, 1 vector (1D)

Supplement 2: tf.keras.layers.Conv2D allows input Tensor with the shape (batch_size, height, width, channels), which means we can pass it a 2D image directly

Spacing out the receptive fields to connect to a larger input layer. In this way, the computational burden of this model will be eased, dramatically.

The connection weights are just multiplying their values to the corresponding receptive field values, plused by respective bias terms. A set of these weights (excluding bias terms) is called a (convolutional) kernel (alias: a filter).

As usual, all these weights (which said, the filters) and the biases will be learned during training, and the layer corresponding to them will output a feature map given it a input.

In reality, a convolution layer will output one feature map per filter. Each pixel of a feature map pairs one neuron in a 2D layer (precisely we shall take a convolutional layer as a 3D layer, by stacking them together). Each feature map matches a pair of (kernel, bias), which is bijectively mapped onto a 2D layer.

In short, a convolutional layer simultaneously applies multiple trainable filters to its inputs, making it capable of detecting multiple features anywhere in its inputs.

There're many advantages (which surpass dense layers) of sharing kernels and bias terms in one "2D layer"/feature map, including:

  1. reducing the computation complexity greatly
  2. learning a feature in someplace and transfer into anywhere of input image

Computing the output of a neuron in a convolutional layer

the output of a neuron in one convolutional layer located in ( i , j , f e a t u r e m a p i d ) = b i a s ( f e a t u r e m a p i d ) + ∑ the output of one neuron of previous layer located in ( i ′ , j ′ , f e a t u r e m a p i d ′ ) × weight ( f e a t u r e m a p i d , f e a t u r e m a p i d ′ , i n t h e c e l l o f x t h r o w o f r e c e p t i v e f i e l d , i n t h e c e l l o f y t h c o l u m n o f r e c e p t i v e f i e l d ) \text{the output of a neuron in one convolutional layer located in}(i, j, feature~map~id)\\ = bias(feature~map~id) + \sum\\\text{the output of one neuron of previous layer located in}(i', j', feature~map~id')\times\text{weight}(feature~map~id, feature~map~id', in~the~cell~of~x_{th}~row~of~receptive~field, in~the~cell~of~y_{th}~column~of~receptive~field) the output of a neuron in one convolutional layer located in(i,j,feature map id)=bias(feature map id)+∑the output of one neuron of previous layer located in(i′,j′,feature map id′)×weight(feature map id,feature map id′,in the cell of xth row of receptive field,in the cell of yth column of receptive field)

w i t h with with

i ′ = i × s t r i d e h e i g h t + x , x + 1 ∈ [ 1 , f i e l d h e i g h t ] i'=i\times stride_{height}+x, x+1\in[1, field_{height}] i′=i×strideheight+x,x+1∈[1,fieldheight], j ′ = j × s t r i d e w i d t h + y , y + 1 ∈ [ 1 , f i e l d w i d t h ] j'=j\times stride_{width}+y, y+1\in[1, field_{width}] j′=j×stridewidth+y,y+1∈[1,fieldwidth]

python 复制代码
import tensorflow as tf
import matplotlib.pyplot as plt

from sklearn.datasets import load_sample_images
dataset = load_sample_images()['images']
im1, im2 = dataset

_, ax = plt.subplots(1, 2)
ax[0].imshow(im1)
ax[1].imshow(im2)
plt.show()

im1.max(), im1.min(), im2.max(), im2.min()

print(tuple(map(lambda x: x.dtype, dataset)))

print(tuple(map(lambda x: x.shape, dataset)))

dataset = tf.keras.layers.Rescaling(1/255)(tf.keras.layers.CenterCrop(height=70, width=120)(dataset))

dataset.shape

More about tf.keras.layers.Conv2D:

  1. tf.keras.layers.Conv2D = tf.keras.layers.Convolution2D
  2. under the hood, this layer relies on TensorFlow's tf.nn.conv2d() operation
  3. kernel_size defines the shape of reception field
  4. by default, strides is set to (1, 1) and padding="valid" (which actually means no zero-padding at all)

Conv2D accepts Tensors of shape (batch_size, spacial_dimension_1, spatial_dimension_2, channels)

We can consider channels as color filters.

python 复制代码
Conv2D = tf.keras.layers.Conv2D(filters=32, kernel_size=7) # equivalent to using kernel_size=(7 , 7)
feature_map = Conv2D(dataset)
feature_map.shape

64 = 70 - 7 + 1

114 = 120 - 7 + 1

python 复制代码
Conv2D = tf.keras.layers.Conv2D(filters=32, kernel_size=7, padding='same') # pad with zeros to make shapes the same
feature_map = Conv2D(dataset)
feature_map.shape

Under the hood: how to pad with 0s?

padding='valid':
find the maximal O u t p u t G r i d s s . t . 1 + S t r i d e s × ( O u t p u t G r i d s − 1 ) + ( K e r n e l S i z e − 1 ) ≤ I n p u t G r i d s O u t p u t G r i d s ≤ I n p u t G r i d s − K e r n e l S i z e + S t r i d e s S t r i d e s Therefore: O u t p u t G r i d s = ⌊ I n p u t G r i d s − K e r n e l S i z e + S t r i d e s S t r i d e s ⌋ \text{find the maximal }OutputGrids \\ s.t. \\ 1 + Strides \times (OutputGrids-1) + (KernelSize-1) \le InputGrids \\ OutputGrids \le \frac{InputGrids-KernelSize+Strides}{Strides} \\ \text{Therefore:}~OutputGrids=\lfloor\frac{InputGrids-KernelSize+Strides}{Strides}\rfloor find the maximal OutputGridss.t.1+Strides×(OutputGrids−1)+(KernelSize−1)≤InputGridsOutputGrids≤StridesInputGrids−KernelSize+StridesTherefore: OutputGrids=⌊StridesInputGrids−KernelSize+Strides⌋

padding = 'same':
we confine the O u t p u t G r i d s s . t . O u t p u t G r i d s = ⌈ I n p u t G r i d s S t r i d e s ⌉ Therefore we can compute the I n p u t G r i d s A f t e r P a d d i n g = K e r n e l S i z e + ( O u t p u t G r i d s − 1 ) × S t r i d e s So we shall pad ⌊ I n p u t G r i d s A f t e r P a d d i n g − I n p u t G r i d s 2 ⌋ , ⌈ I n p u t G r i d s A f t e r P a d d i n g − I n p u t G r i d s 2 ⌉ 0s on each side \text{we confine the }OutputGrids~s.t.~OutputGrids=\lceil\frac{InputGrids}{Strides}\rceil \\ \text{Therefore we can compute the }InputGridsAfterPadding=KernelSize+(OutputGrids-1)\times Strides \\ \text{So we shall pad }\lfloor\frac{InputGridsAfterPadding - InputGrids}2\rfloor, \lceil\frac{InputGridsAfterPadding - InputGrids}2\rceil\text{ 0s on each side} we confine the OutputGrids s.t. OutputGrids=⌈StridesInputGrids⌉Therefore we can compute the InputGridsAfterPadding=KernelSize+(OutputGrids−1)×StridesSo we shall pad ⌊2InputGridsAfterPadding−InputGrids⌋,⌈2InputGridsAfterPadding−InputGrids⌉ 0s on each side

python 复制代码
kernels, biases = Conv2D.weights # this attribute will return Tensors; if we use get_weights() method instead, numpy arrays will be returned

kernels.shape # [kernel_height, kernel_width, input_channels, output_channels]

biases.shape # [output_channels]

We can feed images of any size to this layer, as long as they are at least as large as the kernels, and if they have the right number of channels.

Specifying an activation function (such as ReLU) when creating a Conv2D layer, and also specifying the corresponding kernel initializer (such as He initialization) is useful, otherwise consecutively stacked convolutional layers are equivalent to one convolutional layer.

Hyperparameters Summary: filters, kernel_size, padding, strides, activation, kernel_initializer, etc.

相关推荐
考拉悠然科技1 小时前
考拉悠然:科技与匠心,以烟草虫情AI监测系统共筑品质未来
ai
安冬的码畜日常1 小时前
【AI 加持下的 Python 编程实战 2_10】DIY 拓展:从扫雷小游戏开发再探问题分解与 AI 代码调试能力(中)
开发语言·前端·人工智能·ai·扫雷游戏·ai辅助编程·辅助编程
Apifox.5 小时前
Apifox 4月更新|Apifox在线文档支持LLMs.txt、评论支持使用@提及成员、支持为团队配置「IP 允许访问名单」
前端·人工智能·后端·ai·ai编程
姚毛毛8 小时前
Windows上,10分钟构建一个本地知识库
python·ai·rag
HUIBUR科技10 小时前
AI与智能能源管理:如何通过AI优化能源分配和消耗?
人工智能·ai
结冰架构10 小时前
【AI提示词】艺人顾问
人工智能·ai·提示词·艺人·顾问
漫谈网络12 小时前
Ollama工具调用(Tool Calls)业务应用案例
linux·ai·aigc·工具调用·ollama·tool calls
wuhanwhite13 小时前
机器人新革命:Pi 0.5如何让智能走进千家万户
ai·机器人
MarsBighead13 小时前
Pgvector+R2R搭建RAG知识库
python·ai·postgresql·rag·pgvector
带刺的坐椅14 小时前
Java AI(智能体)编排开发就用 Solon Flow
java·ai·openai·solon·solon-flow