1.Introduction : discontinuities / sharp changes in the image brightness
- Edge detection: concept in image processing and computer vision.
- Boundaries/edges: identified within an image where there is a significant change in intensity or color.
- Correspond to boundaries between different objects or features.
- Applying an edge detection algorithm to an image can significantly reduce the amount of data to be processed and thus filter out information that may be considered less relevant while preserving the important structural properties of the image.
- Obtain a set of curves that indicate the boundaries of objects and surface markings, as well as curves that correspond to discontinuities in surface orientation.
2.Purpose
The main purpose of edge detection is to simplify the representation of an image by reducing it to its structural framework, which consists of lines and points . This simplification is crucial for higher-level image analysis tasks such as object recognition, image segmentation, and feature extraction.
in these edge-cutting techniques, it is necessary to reduce the amount of information that the algorithm should focus on. Sometimes this could be done only by passing the edges of the image.
the reason discontinuities in image brightness
Discontinuities in depth, surface orientation, scene illumination variations, and material property
3.Understanding Popular Edge Detection Algorithms
Traditional approach / Deep learning-based approach
(1)Prewitt edge detection
Similar to the Sobel operator but uses different kernel coefficients.
This method is a commonly used edge detector mostly to detect the horizontal and vertical edges in images.
(2)Sobel Operator
- Uses a pair of 3x3 convolution kernels to calculate the gradient in the horizontal and vertical directions.
This method is a commonly used edge detector mostly to detect the horizontal and vertical edges in images. This method is a commonly used edge detector mostly to detect the horizontal and vertical edges in images.
(3)Canny Edge Detector:
Canny edge detection algorithm produces smoother, thinner, and cleaner images than Sobel and Prewitt filters.
A multi-stage algorithm that uses a Gaussian filter to smooth the image, computes the intensity gradient magnitude and direction, applies non-maximum suppression to get thin edges, and then uses double thresholding to determine potential edges.
- The input image is smoothened, Sobel filter is applied to detect the edges of the image.
- Then we apply non-max suppression where the local maximum pixels in the gradient direction are retained, and the rest are suppressed.
- We apply thresholding to remove pixels below a certain threshold and retain the pixels above a certain threshold to remove edges that could be formed due to noise.
- Later we apply hysteresis tracking to make a pixel strong if any of the 8 neighboring pixels are strong.
non-max suppression(NMS):
It is designed to reduce the number of bounding boxes or potential regions of interest (ROIs) that are detected in an image, by eliminating those that do not correspond to the actual objects of interest.
For each local region, the algorithm checks whether the gradient magnitude at a given pixel is the maximum within that region. If it is not the maximum, the pixel is suppressed (i.e., its value is set to zero). This step ensures that only the local maxima of the gradient magnitude remain, which are more likely to be part of the true edges.
After applying NMS, the edges are thinned down to single-pixel width, which makes them easier to analyze and process in subsequent steps.
The purpose of non-maximum suppression is to refine the edge detection results by removing any non-maximum points along the detected edges, ensuring that the final output consists of clean, single-pixel-wide edges that accurately represent the boundaries of objects in the image. This technique is crucial for reducing noise and false positives in edge detection and object detection tasks.
hysteresis tracking:
It uses two thresholds, a high threshold and a low threshold.
Strong edge pixels are those with gradient magnitudes exceeding the high threshold.
Weak edge pixels have gradient magnitudes between the two thresholds.
Potential edge pixels have gradient magnitudes below the low threshold.
By connecting strong edge pixels and potential edge pixels, a complete edge image is formed. This tracking approach helps to better preserve edge information in the image and avoid edge loss due to factors like noise.
(The process of connecting strong edge pixels and potential edge pixels helps to filter out noise. Since noise usually has a relatively low gradient magnitude, it is less likely to be classified as a strong or potential edge. This means that hysteresis tracking can better preserve the important structural properties of an image while reducing the impact of noise)
5 steps involved in Canny edge detection, as shown in fig 1.2 above.
Image Smoothening : as the edge detection that using derivatives is sensitive to noise, we reduce it
In this step, we convert the image to grayscale as edge detection does not dependent on colors. Then we remove the noise in the image with a Gaussian filteras edge detection is prone to noise.
Finding Intensity Gradients of the Image : helps identify the edge intensity and direction.
We then apply the Sobel kernel in horizontal and vertical directions to get the first derivative in the horizontal direction (Gx) and vertical direction (Gy) on the smoothened image. We then calculate the edge gradient(G) and Angle(θ) as given below,
We know that the gradient direction is perpendicular to the edge. We round the angle to one of four angles representing vertical, horizontal, and two diagonal directions.
Non-Max Suppression : to thin the edges of the image
Now we remove all the unwanted pixels which may not constitute the edge. For this, every pixel is checked in the direction of the gradient if it is a local maximum in its neighbourhood. If it is a local maximum, it is considered for the next stage, otherwise, it is darkened with 0. This will give a thin line in the output image.
Double Threshold : identify the strong, weak and irrelevant pixels in the images.
Pixels due to noise and color variation would persist in the image. So, to remove this, we get two thresholds from the user, lowerVal and upperVal. We filter out edge pixels with a weak gradient(lowerVal) value and preserve edge pixels with a high gradient value(upperVal). Edges with an intensity gradient more than upperVal are sure to edge, and those below lowerVal are sure to be non-edges, so discarded. The pixels that have pixel value lesser than the upperVal and greater than the lowerVal are considered part of the edge if it is connected to a "sure-edge". Otherwise, they are also discarded.
Edge Tracking by Hysteresis
A pixel is made as a strong pixel if either of the 8 pixels around it is strong(pixel value=255) else it is made as 0.
That's pretty much about Canny Edge Detection. As you can see here, the edges are detected from an image.
Canny edge detection focuses only on local changes, and it does not understand the image's semantics, i.e., the content. Hence, Deep Learning based algorithms are proposed to solve these problems.
(4)Laplacian of Gaussian (LoG):
- Combines Gaussian smoothing with the Laplacian to detect edges while reducing noise
The Laplacian edge detectors vary from the previously discussed edge detectors. This method uses only one filter (also called a kernel).
In a single pass, Laplacian detection performs second-order derivatives and hence are sensitive to noise.
To avoid this sensitivity to noise, before applying this method, Gaussian smoothing is performed on the image.
Parameters: Edge detection algorithms often have parameters that need to be tuned for optimal performance. For example, the Canny edge detector has parameters for the standard deviation of the Gaussian filter and the two threshold values.
4.Applications
- Automated Inspection: To detect defects or anomalies in manufactured products.
- Medical Imaging: To identify boundaries of organs or lesions.
- Robotics and Autonomous Vehicles: To perceive the environment and navigate.
- Satellite Imagery: To analyze geographical features and changes over time.
5.Challenges and drawbacks
Edge detection can be challenging due to factors such as noise, lighting conditions, and the complexity of the scene . Advanced techniques often incorporate additional steps to handle these issues, such as adaptive thresholding or machine learning-based approaches.
6.Why DeepLearning for edge detection?
Canny edge detection focuses mainly on local changes and not on the semantics of the image i.e it focuses less on the image's content. Hence we get less accurate edges.
Deep Learning Approach for Edge Detection
A technique called Holistically Nested Edge Detection, or HED is a learning-based end-to-end edge detection system that uses a trimmed VGG-like convolutional neural network for an image-to-image prediction task. HED generates the side outputs in the neural network. All the side outputs are fused to make the final output. Let us understand the algorithm in a more detailed manner.
7.Reference
Comprehensive Guide to Edge Detection Algorithms - Analytics Vidhya
What is Edge Detection | Introduction to Edge Detection | Great Learning
Edge Detection | Papers With Code