Contents¶

  • Image Transformations
    • Brightening
    • Flipping
    • Linear Contrast Stretching
  • Linear Filtering
    • Correlation
    • Convolution
    • Why Convolution, when we already have Correlation
  • Gaussian Blurring
  • Sobel Filter
  • Associativity of Convolution
  • Exploring Linearity and Shift-invariance
    • Linearity
    • Shift-invariance
  • TODO
In [1]:
# Imports

from PIL import Image
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import scipy.datasets
from scipy.ndimage import convolve, correlate, gaussian_filter
In [2]:
def get_image(filename):
    im = Image.open(filename)
    np_im = np.array(im)
    shape = len(np_im.shape)
    if shape > 2:
        np_im = np_im[:,:,0]
    return np_im
In [3]:
grayscale_image = scipy.datasets.ascent()
In [4]:
plt.imshow(grayscale_image, cmap='gray')
Out[4]:
<matplotlib.image.AxesImage at 0x7dc2be9944c0>

Image Transformations ¶

Brightening the Image ¶

In [7]:
brighten_grayscale_image = grayscale_image + 20
plt.imshow(brighten_grayscale_image, cmap='gray')
Out[7]:
<matplotlib.image.AxesImage at 0x7dc2bafb34c0>

Flipping the image ¶

In [8]:
flipped_image = grayscale_image[:,::-1]
plt.imshow(flipped_image, cmap='gray')
Out[8]:
<matplotlib.image.AxesImage at 0x7dc2bae404c0>

Linear Contrast Stretching ¶

In [9]:
im_max = grayscale_image.max()
im_min = grayscale_image.min()
print(im_max, im_min)
lcs_im = (grayscale_image - im_min) / (im_max - im_min)
lcs_im = 255 * lcs_im
plt.imshow(lcs_im, cmap='gray')
print(lcs_im.max(), lcs_im.min())

255 0
255.0 0.0

Linear Filtering ¶

Correlation and Cross-Correlation¶

$$ G(i, j) = \underbrace{\frac{1}{(2k+1)^2}}_{\text{uniform weights}}\sum_{u=-k}^{k}\sum_{v=-k}^{k}I(i + u, j + v) $$$$ G(i, j) = \sum_{u=-k}^{k}\sum_{v=-k}^{k}{I(i + u, j + v)\underbrace{H(u,k)}_{\text{non-uniform weights}}} $$

Convolution¶

$$ G(i, j) = \sum_{u=-k}^{k}\sum_{v=-k}^{k}{I(i - u, j - v)H(u,k)} $$
In [90]:
impulse_image = np.zeros((5,5))
impulse_image[2,2] = 1
plt.imshow(impulse_image, cmap='gray')
Out[90]:
<matplotlib.image.AxesImage at 0x7dc2a13ae860>
In [15]:
impulse_image = np.pad(impulse_image, 1)
In [16]:
i,j = np.indices((3,3))
fltr = (1/9)*(3 * i + j)
plt.imshow(fltr, cmap='gray')
Out[16]:
<matplotlib.image.AxesImage at 0x7dc2baeb1a20>

Correlation Operation: Implementation 1 ¶

In [17]:
def correlation_operation(i,j,k, image, filtr):
    mid_k = k // 2
    (u,v) = np.indices((k,k))
    G = (image[i+(u-mid_k),j+(v - mid_k)] * fltr[u,v])
    return G.sum()

def correlation(image, fltr):
    im_shape = image.shape[0] - 2
    new_impulse_image = np.zeros((im_shape, im_shape))

    for i in range(0,im_shape):
        for j in range(0,im_shape):
            new_impulse_image[i, j] = correlation_operation(i+1, j+1, 3, image, fltr)
    return new_impulse_image

plt.imshow(correlation(impulse_image, fltr), cmap='gray')
Out[17]:
<matplotlib.image.AxesImage at 0x7dc2ba9c88b0>

Convolution Operation: Implementation 1 ¶

In [18]:
def convolution_operation(i,j,k, image, fltr):
    mid_k = k // 2
    (u,v) = np.indices((k,k))
    x = (image[i-(u-mid_k),j-(v-mid_k)] * fltr[u,v])
    return x.sum()

def convolution(image, fltr):
    im_shape = image.shape[0] - 2
    new_impulse_image = np.zeros((im_shape, im_shape))

    for i in range(0,im_shape):
        for j in range(0,im_shape):
            new_impulse_image[i, j] = convolution_operation(i+1, j+1, 3, image, fltr)
    
    return new_impulse_image


plt.imshow(convolution(impulse_image, fltr), cmap='gray')
Out[18]:
<matplotlib.image.AxesImage at 0x7dc2ba898910>

Why do we need convolution, when we already have correlation? ¶

Apart from filter not getting flipped, there are other benefits with Convolution.

  1. Commutative, Distributive, Associative, and also has a Identity.
  2. Convolution is seperable, if the filter is seperable.

In the interest of time and efficiency, I will be using scipy package for convolve and other allied operations¶

In [20]:
plt.imshow(convolve(impulse_image, fltr), cmap='gray')
Out[20]:
<matplotlib.image.AxesImage at 0x7dc2a1c0c970>

Gaussian Blurring ¶

In [44]:
gaussian = 1/36 * np.matrix([[1,1,2,1,1],[1,1,2,1,1],[2,2,4,2,2],[1,1,2,1,1],[1,1,2,1,1]])
print("Sum:", np.sum(gaussian),"\n\n", "Kernel:\n", gaussian)

Sum: 1.0 

 Kernel:
 [[0.02777778 0.02777778 0.05555556 0.02777778 0.02777778]
 [0.02777778 0.02777778 0.05555556 0.02777778 0.02777778]
 [0.05555556 0.05555556 0.11111111 0.05555556 0.05555556]
 [0.02777778 0.02777778 0.05555556 0.02777778 0.02777778]
 [0.02777778 0.02777778 0.05555556 0.02777778 0.02777778]]
In [62]:
def gaussian_kernel(size: int, sigma: float) -> np.ndarray:
    """Generates a (size x size) Gaussian kernel."""
    kernel = np.fromfunction(
        lambda x, y: (1/(2*np.pi*sigma**2)) * np.exp(-((x-(size-1)/2)**2 + (y-(size-1)/2)**2) / (2*sigma**2)),
        (size, size)
    )
    return kernel / np.sum(kernel)

gaussian_kernel_31x31 = gaussian_kernel(31, 5)
$$ \frac{1}{2\pi\sigma^2} e^{\frac{-((x-\mu)^2 + (y - \mu)^2)}{2\sigma^2}} $$
In [50]:
plt.imshow(convolve(grayscale_image, gaussian), cmap='gray')
Out[50]:
<matplotlib.image.AxesImage at 0x7dc2a1a97fd0>
In [52]:
plt.imshow(convolve(grayscale_image, gaussian_kernel_31x31), cmap='gray')
Out[52]:
<matplotlib.image.AxesImage at 0x7dc2a15a8940>
In [53]:
plt.imshow(grayscale_image, cmap='gray')
Out[53]:
<matplotlib.image.AxesImage at 0x7dc2a16158d0>
In [61]:
f, ax = plt.subplots(1,3)
ax[1].imshow(convolve(grayscale_image, gaussian), cmap='gray')
ax[1].title.set_text("5X5 Blur")

ax[2].imshow(convolve(grayscale_image, gaussian_kernel_31x31), cmap='gray')
ax[2].title.set_text("31X31 Blur")

ax[0].imshow(grayscale_image, cmap='gray')
ax[0].title.set_text("Original")

Thus Standard Gaussian Filter is low-pass filter

In [76]:
edge_filter = np.matrix(
    [
        [1,0,0,0,-1],
        [1,0,0,0,-1], 
        [2,0,0,0,-2],
        [1,0,0,0,-1] ,
        [1,0,0,0,-1]
    ]
)

Sobel Filter is an example of a high-pass filter, and it detects edges.

In [77]:
plt.imshow(convolve(grayscale_image, edge_filter), cmap='gray')
Out[77]:
<matplotlib.image.AxesImage at 0x7dc2a0c07160>

Prove that Correlation is not associative, but Convolution is ¶

In [126]:
a = gaussian_kernel(3, 1)
b = np.matrix([[1/9,1/9,1/9], [1/9,1/9,1/9], [1/9,1/9,1/9]])
c = fltr


# Prove (a * b) * c = a * (b * c)

a1 = convolve(convolve(a, b), c)
a2 = convolve(a, convolve(b, c))
In [127]:
plt.imshow(a1, cmap='gray')
Out[127]:
<matplotlib.image.AxesImage at 0x7dc29fe14c10>
In [128]:
plt.imshow(a2, cmap='gray')
Out[128]:
<matplotlib.image.AxesImage at 0x7dc29fcd2b30>

Conclusion: Even the convolution is not associative in the discrete domain. Yet, in the continous domain the convolution is associative.

Explore Linearity and Shift-invariance ¶

Linearity ¶
In [141]:
l1 = convolve(impulse_image, (a + b))
l2 = convolve(impulse_image,a) + convolve(impulse_image,b)
In [142]:
plt.imshow(l1, cmap='gray')
Out[142]:
<matplotlib.image.AxesImage at 0x7dc29f895180>
In [143]:
plt.imshow(l2, cmap='gray')
Out[143]:
<matplotlib.image.AxesImage at 0x7dc29f904700>
Shift-Invariance ¶

In a shift invariant system, the response to a translated stimulus is just a translation of the response to the stimulus. This means that, for example, if a view of a small light aimed at the center of the camera is a small bright blob, then if the light is moved to the periphery, we should see the same small bright blob, only translated.

In [146]:
si_image_1 = np.zeros((5,5))
si_image_2 = np.zeros((5,5))

si_image_1[2,2] = 1
si_image_2[2,3] = 1
In [148]:
si1 = convolve(si_image_1, fltr)
si2 = convolve(si_image_2, fltr)
In [154]:
f,a = plt.subplots(1,2)
a[0].imshow(si1, cmap='gray')
a[1].imshow(si2, cmap='gray')
Out[154]:
<matplotlib.image.AxesImage at 0x7dc29ed7d540>

Thus the convolution is a linear and shift-invariant operator

To-Do: ¶

Interpolation as Convolution¶

Template Matching¶