Convolutional Neural Networks
Introduction
Convolutional Neural Networks (CNNs) are a class of deep, feed-forward artificial neural networks that have been applied to analyze visual imagery. They are also known as shift invariant or space invariant artificial neural networks (SIANN), based on their shared-weights architecture and translation invariance characteristics.
Architecture
CNNs are composed of multiple layers of receptive fields that are then tiled to cover the entire visual field. These layers are designed to automatically and adaptively learn spatial hierarchies of features. CNNs are designed to process data with a grid-like topology, such as an image, which has a two-dimensional grid of pixels.
Layers
The layers of a CNN typically consist of convolutional layers, pooling layers, fully connected layers, and normalization layers.
Convolutional Layer
The convolutional layer is the core building block of a CNN. The layer's parameters consist of a set of learnable filters, which have a small receptive field, but extend through the full depth of the input volume. During the forward pass, each filter is convolved across the width and height of the input volume, computing the dot product between the entries of the filter and the input and producing a 2-dimensional activation map of that filter. As a result, the network learns filters that activate when it detects some specific type of feature at some spatial position in the input.
Pooling Layer
The pooling layer (also known as downsampling or subsampling) reduces the dimensionality of each feature map but retains the most important information. Pooling can be of different types: Max, Average, Sum etc.
Fully Connected Layer
The Fully Connected layer is a traditional Multi Layer Perceptron that uses a softmax activation function in the output layer. The term "Fully Connected" implies that every neuron in the previous layer is connected to every neuron on the next layer.
Normalization Layer
Normalization layers are used to maintain normalization of the activations. This helps to ensure that the model does not become too sensitive to the specific weights of neurons, and allows the model to generalize well.
Training
Training a CNN involves learning the weights that are used in the convolutional layers. This is typically done using gradient descent, where the gradient is calculated using backpropagation. The weights are then updated to minimize the loss function.
Applications
CNNs have been successfully applied to many types of visual imagery tasks. They are most commonly used in image and video recognition, but have also been applied to recommender systems, natural language processing, and medical image analysis.
Advantages and Limitations
CNNs have several advantages over other image processing methods. They eliminate the need for manual feature extraction, so they can learn complex patterns using raw pixel data. They are also very efficient to train, because they have relatively few parameters compared to fully connected networks with the same number of hidden units.
However, CNNs also have some limitations. They require a large amount of training data to avoid overfitting. They can also be computationally intensive to train, although this has been mitigated by the use of GPUs, and specialized software packages.