Computer vision

Feature Extraction

Turn raw pixels into a handful of meaningful measurements: gradients, textures, colours and patterns.

Feature extraction overview

Feature extraction: converting a raw image into meaningful measurements.

Low-level

Edges, corners, colours, textures.

Mid-level

SIFT, SURF, ORB descriptors.

High-level

Object parts, semantic concepts (CNNs).

Purpose: reduce dimensionality while keeping the important information.

Histogram of Oriented Gradients (HOG)

HOG describes local object appearance with gradient orientations.

Compute gradients (magnitude + direction)
Divide the image into cells (e.g. 8×8 pixels)
Compute a histogram of gradients per cell (9 bins)
Group cells into blocks (e.g. 2×2 cells)
Normalise the histograms within each block
Concatenate all histograms → the feature vector

Use case: pedestrian detection, object recognition.

HOG parameters

Cell size: 8×8 or 16×16 pixels
Block size: 2×2 or 3×3 cells
Orientations: 9 bins (0–180° for unsigned gradients)
Block stride: the overlap between blocks

Example: 64×128 window, 8×8 cells, 2×2 blocks, 9 bins

8×16 cells, stride 1 cell → (8−1) × (16−1) = 7 × 15 blocks

7 × 15 blocks × 4 cells × 9 bins

= 3780 dimensions

HOG in Python


        from skimage.feature import hog
        from skimage import exposure
        import cv2
        # load image
        img = cv2.imread('person.jpg')
        gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
        # compute HOG features
        features, hog_image = hog(gray, orientations=9, pixels_per_cell=(8, 8),  # ans:
                                  cells_per_block=(2, 2), visualize=True)  # ans:
        # features: the HOG descriptor vector
        print(features.shape)  # (N,) 1-D array  # ans:
        # hog_image: a visualisation of the gradients
        hog_image = exposure.rescale_intensity(hog_image, out_range=(0, 255))  # ans:
        hog_image = np.uint8(hog_image)  # ans:

What `hog_image` looks like

gray

hog_image

Each cell draws its 9 bins as lines along the edge direction, brighter for stronger bins.

Local Binary Patterns (LBP)

LBP is a texture descriptor.

For each pixel, compare it with its 8 neighbours
If neighbour ≥ centre: 1, else: 0
Read the bits clockwise → an 8-bit binary number
Convert to decimal (0–255)

Properties: rotation invariant (optionally) · grayscale invariant · fast to compute

Use case: texture classification, face recognition.

LBP implementation


        from skimage.feature import local_binary_pattern
        # compute LBP
        radius = 3
        n_points = 8 * radius
        lbp = local_binary_pattern(gray, n_points, radius, method='uniform')  # ans:
        # method options:
        # ans: 'default': the original LBP
        # ans: 'uniform': rotation invariant (most common)
        # ans: 'var': variance-based
        # create a histogram
        hist, _ = np.histogram(lbp.ravel(), bins=np.arange(0, n_points + 3),  # ans:
                               range=(0, n_points + 2))  # ans:
        hist = hist.astype('float')  # ans:
        hist /= (hist.sum() + 1e-7)  # normalise  # ans:

What an LBP image looks like

gray

LBP code (0–255) of every pixel

Flat regions give noisy codes; edges and textures give stable patterns. The histogram of these codes is the texture descriptor.

Colour histograms

Colour histogram: the distribution of colours in an image.

Choose a colour space (RGB, HSV, LAB)
Divide each channel into bins
Count the pixels in each bin
Normalise (make the sum = 1)

Use case: image retrieval, colour-based matching.

Colour histogram in OpenCV


        # compute a histogram for each channel
        hist_b = cv2.calcHist([img], [0], None, [256], [0, 256])  # ans:
        hist_g = cv2.calcHist([img], [1], None, [256], [0, 256])  # ans:
        hist_r = cv2.calcHist([img], [2], None, [256], [0, 256])  # ans:
        # parameters:
        # ans: [img]: the source image
        # ans: [0]: the channel index (0=B, 1=G, 2=R)
        # ans: None: no mask
        # ans: [256]: the number of bins
        # ans: [0, 256]: the range
        # normalise
        hist_b = cv2.normalize(hist_b, hist_b).flatten()  # ans:

HSV histogram


          # convert to HSV
          hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
          # compute a 2-D histogram (Hue vs Saturation)
          hist = cv2.calcHist([hsv], [0, 1], None, [180, 256],  # ans:
                              [0, 180, 0, 256])  # ans:
          # ans: channels [0, 1] = Hue and Saturation
          # ans: bins [180, 256] = 180 for H, 256 for S
          # normalise
          cv2.normalize(hist, hist, 0, 255, cv2.NORM_MINMAX)  # ans:

The dolls photo: hue (0–179) across, saturation (0–255) down; brighter = more pixels

Haar cascades

Haar cascades: pre-trained classifiers for object detection.

Haar-like features: rectangular patterns: sum of the white area minus sum of the black area
Cascade of weak classifiers: most windows are rejected by the first cheap stages
AdaBoost training picks the useful features

Pre-trained models: face, eyes, smile, full body… Advantage: fast, real-time detection.

Haar cascade in OpenCV


        # load the pre-trained face detector
        face_cascade = cv2.CascadeClassifier(  # ans:
            cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')  # ans:
        # detect faces
        gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
        faces = face_cascade.detectMultiScale(gray, scaleFactor=1.1,  # ans:
                                              minNeighbors=5, minSize=(30, 30))  # ans:
        # faces: an array of [x, y, w, h]
        for (x, y, w, h) in faces:  # ans:
            cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)  # ans:

Template matching

result: bright = good match. Click the photo to pick a new template (dashed).

Template matching: find a template image inside a larger image.

TM_CCOEFF_NORMED: correlation coefficient (best)
TM_SQDIFF_NORMED: squared difference
TM_CCORR_NORMED: cross-correlation

Limitation: not scale- or rotation-invariant.

Template matching in OpenCV


        # load the template
        template = cv2.imread('template.jpg', 0)
        w, h = template.shape[::-1]
        # template matching
        result = cv2.matchTemplate(gray, template, cv2.TM_CCOEFF_NORMED)  # ans:
        # find the location of the best match
        min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)  # ans:
        # for CCOEFF_NORMED, higher is better
        top_left = max_loc  # ans:
        bottom_right = (top_left[0] + w, top_left[1] + h)  # ans:
        cv2.rectangle(img, top_left, bottom_right, (0, 255, 0), 2)  # ans:

Exercises

Answer first, then press → to check.

Exercises 1 · concepts

What is HOG?

Histogram of Oriented Gradients: it describes object appearance.

What does LBP encode?

A texture pattern, by comparing a pixel with its neighbours.

What is a colour histogram?

The distribution of colours in an image.

What are Haar cascades?

Pre-trained classifiers built on Haar-like features.

What is the limitation of template matching?

It is not scale- or rotation-invariant.

Exercises 2 · concepts

What is a typical HOG cell size?

8×8 or 16×16 pixels.

What is the LBP output range?

0–255 (8-bit patterns from 8 neighbours).

Why normalise histograms?

To make them independent of image size: they sum to 1.

What is the "cascade" in Haar cascades?

A series of weak classifiers that reject non-objects quickly.

What is the best template matching method?

TM_CCOEFF_NORMED (normalised correlation coefficient).

Exercises 3 · coding


        # compute HOG features
        from skimage.feature import hog
        features, hog_img = hog(gray, orientations=9, pixels_per_cell=(8, 8),  # ans:
                                cells_per_block=(2, 2), visualize=True)  # ans:
        # compute LBP
        from skimage.feature import local_binary_pattern
        lbp = local_binary_pattern(gray, 24, 3, method='uniform')  # ans:
        # colour histogram
        hist_b = cv2.calcHist([img], [0], None, [256], [0, 256])  # ans:
        hist_b = cv2.normalize(hist_b, hist_b).flatten()  # ans:

Exercises 4 · coding


        # face detection with a Haar cascade
        face_cascade = cv2.CascadeClassifier(  # ans:
            cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')  # ans:
        gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
        faces = face_cascade.detectMultiScale(gray, 1.1, 5, minSize=(30, 30))  # ans:
        for (x, y, w, h) in faces:  # ans:
            cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)  # ans:

Exercises 5 · mixed


        # template matching
        template = cv2.imread('template.jpg', 0)
        w, h = template.shape[::-1]
        result = cv2.matchTemplate(gray, template, cv2.TM_CCOEFF_NORMED)  # ans:
        min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)  # ans:
        top_left = max_loc  # ans:
        cv2.rectangle(img, top_left, (top_left[0]+w, top_left[1]+h),  # ans:
                      (0, 255, 0), 2)  # ans:
        # HSV histogram
        hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
        hist = cv2.calcHist([hsv], [0, 1], None, [180, 256],  # ans:
                            [0, 180, 0, 256])  # ans:

Feature Extraction