Feature Descriptors

What are Feature Descriptors?

Descriptor: Numerical representation of feature's local appearance

Purpose:

  • Describe feature neighborhood
  • Enable matching between images
  • Invariant to transformations (rotation, scale, illumination)

Process:

  1. Detect keypoints (location)
  2. Compute descriptors (appearance)
  3. Match descriptors across images

Descriptor Properties

Good descriptor should be:

  • Distinctive: Different features have different descriptors
  • Invariant: Robust to rotation, scale, illumination
  • Compact: Small memory footprint
  • Fast: Quick to compute and match

Trade-off: Invariance vs distinctiveness vs speed

Types of Descriptors

Classical descriptors:

  • SIFT: Scale-Invariant Feature Transform
  • SURF: Speeded-Up Robust Features
  • ORB: Oriented FAST and Rotated BRIEF
  • BRIEF: Binary Robust Independent Elementary Features

Deep learning descriptors:

  • CNN-based (covered in advanced topics)

Descriptor Vector

Descriptor: Fixed-length vector of numbers

Examples:

  • SIFT: 128-dimensional float vector
  • SURF: 64 or 128 dimensions
  • ORB: 256-bit binary vector (32 bytes)
  • BRIEF: 128, 256, or 512 bits

Matching: Compare vectors using distance metric

ORB Descriptor

ORB: Oriented FAST + Rotated BRIEF

Features:

  • Binary descriptor (fast to compute and match)
  • Rotation invariant
  • Scale invariant (using pyramid)
  • Free to use (no patents)

Use case: Real-time applications, mobile devices

ORB in OpenCV

  1. import cv2
  2. # create ORB detector
  3. #ans: orb = cv2.ORB_create()
  4. # detect and compute descriptors
  5. #ans: keypoints, descriptors = orb.detectAndCompute(gray, None)
  6. # keypoints: list of cv2.KeyPoint objects
  7. #ans: contains x, y, size, angle, response
  8. # descriptors: numpy array
  9. #ans: shape (N, 32) for N keypoints, 256-bit each
  10. # draw keypoints
  11. #ans: img_kp = cv2.drawKeypoints(img, keypoints, None, (0, 255, 0), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)

ORB Parameters

  1. # create ORB with custom parameters
  2. #ans: orb = cv2.ORB_create(nfeatures=500, scaleFactor=1.2, nlevels=8)
  3. # nfeatures: maximum number of features
  4. #ans: 500 strongest features retained
  5. # scaleFactor: pyramid decimation ratio
  6. #ans: 1.2 means each level is 1.2x smaller
  7. # nlevels: number of pyramid levels
  8. #ans: 8 levels for multi-scale detection
  9. # set edge threshold
  10. orb = cv2.ORB_create()
  11. #ans: orb.setEdgeThreshold(15)
  12. #ans: ignores features near edges (15 pixels)

SIFT Descriptor

SIFT: Scale-Invariant Feature Transform

Properties:

  • 128-dimensional float vector
  • Histogram of gradient orientations
  • Highly distinctive
  • Invariant to scale, rotation, illumination

Disadvantage: Slower, patented (expired 2020)

SIFT in OpenCV

  1. # create SIFT detector
  2. #ans: sift = cv2.SIFT_create()
  3. # detect and compute
  4. #ans: keypoints, descriptors = sift.detectAndCompute(gray, None)
  5. # descriptors shape
  6. #ans: (N, 128) for N keypoints
  7. # custom parameters
  8. #ans: sift = cv2.SIFT_create(nfeatures=0, nOctaveLayers=3, contrastThreshold=0.04)
  9. # nfeatures: 0 means no limit
  10. #ans: all features above threshold retained
  11. # contrastThreshold: filters low-contrast features
  12. #ans: higher value = fewer, stronger features

SURF Descriptor

SURF: Speeded-Up Robust Features

Properties:

  • Faster than SIFT
  • 64 or 128 dimensions
  • Uses integral images for speed
  • Similar performance to SIFT

Note: Patented, not available in OpenCV by default

Descriptor Matching

Matching: Find similar descriptors across images

Distance metrics:

  • L2 (Euclidean): For float descriptors (SIFT, SURF)
    • distance = √(Σ(d1[i] - d2[i])²)
  • Hamming: For binary descriptors (ORB, BRIEF)
    • distance = count of differing bits

Lower distance = better match

Brute-Force Matcher

  1. # create BF matcher for ORB (Hamming distance)
  2. #ans: bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
  3. # detect features in both images
  4. orb = cv2.ORB_create()
  5. kp1, des1 = orb.detectAndCompute(img1_gray, None)
  6. kp2, des2 = orb.detectAndCompute(img2_gray, None)
  7. # match descriptors
  8. #ans: matches = bf.match(des1, des2)
  9. # sort by distance
  10. #ans: matches = sorted(matches, key=lambda x: x.distance)
  11. # draw matches
  12. #ans: img_matches = cv2.drawMatches(img1, kp1, img2, kp2, matches[:50], None, flags=2)

FLANN Matcher

FLANN: Fast Library for Approximate Nearest Neighbors

Advantage: Much faster for large datasets (approximate)

Trade-off: Speed vs accuracy

FLANN in OpenCV

  1. # FLANN parameters for ORB
  2. FLANN_INDEX_LSH = 6
  3. index_params = dict(algorithm=FLANN_INDEX_LSH, table_number=6, key_size=12, multi_probe_level=1)
  4. search_params = dict(checks=50)
  5. #ans: flann = cv2.FlannBasedMatcher(index_params, search_params)
  6. # match descriptors
  7. orb = cv2.ORB_create()
  8. kp1, des1 = orb.detectAndCompute(img1_gray, None)
  9. kp2, des2 = orb.detectAndCompute(img2_gray, None)
  10. #ans: matches = flann.knnMatch(des1, des2, k=2)
  11. # k=2 returns 2 best matches per descriptor

Exercises - Part 1 (Concepts)

  1. # what is a descriptor?
  2. #ans: numerical representation of feature's local appearance
  3. # why need descriptors?
  4. #ans: to match features across images
  5. # what makes a good descriptor?
  6. #ans: distinctive, invariant, compact, fast
  7. # SIFT descriptor size?
  8. #ans: 128 dimensions (float)
  9. # ORB descriptor size?
  10. #ans: 256 bits (32 bytes)

Exercises - Part 2 (Concepts)

  1. # what distance for binary descriptors?
  2. #ans: Hamming distance (count differing bits)
  3. # what distance for SIFT?
  4. #ans: L2 (Euclidean) distance
  5. # advantage of ORB over SIFT?
  6. #ans: faster, binary, free (no patent)
  7. # what is FLANN?
  8. #ans: fast approximate nearest neighbor matcher
  9. # what does crossCheck do in BFMatcher?
  10. #ans: ensures mutual best match (A→B and B→A)

Exercises - Part 3 (Coding)

  1. # create ORB and detect features
  2. gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
  3. #ans: orb = cv2.ORB_create()
  4. #ans: keypoints, descriptors = orb.detectAndCompute(gray, None)
  5. # create SIFT and detect
  6. #ans: sift = cv2.SIFT_create()
  7. #ans: keypoints, descriptors = sift.detectAndCompute(gray, None)
  8. # create BF matcher for ORB
  9. #ans: bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)

Exercises - Part 4 (Coding)

  1. # match ORB features between two images
  2. orb = cv2.ORB_create()
  3. kp1, des1 = orb.detectAndCompute(gray1, None)
  4. kp2, des2 = orb.detectAndCompute(gray2, None)
  5. bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
  6. #ans: matches = bf.match(des1, des2)
  7. #ans: matches = sorted(matches, key=lambda x: x.distance)
  8. # draw top 20 matches
  9. #ans: img_matches = cv2.drawMatches(img1, kp1, img2, kp2, matches[:20], None, flags=2)

Exercises - Part 5 (Mixed)

  1. # ORB with 1000 features
  2. #ans: orb = cv2.ORB_create(nfeatures=1000)
  3. #ans: kp, des = orb.detectAndCompute(gray, None)
  4. # BF matcher for SIFT
  5. #ans: bf = cv2.BFMatcher(cv2.NORM_L2, crossCheck=True)
  6. # FLANN for ORB with kNN
  7. FLANN_INDEX_LSH = 6
  8. index_params = dict(algorithm=FLANN_INDEX_LSH, table_number=6, key_size=12, multi_probe_level=1)
  9. search_params = dict(checks=50)
  10. flann = cv2.FlannBasedMatcher(index_params, search_params)
  11. #ans: matches = flann.knnMatch(des1, des2, k=2)

Google tag (gtag.js)