Turn the neighbourhood of a keypoint into a vector of numbers, so the same point can be found in another image.
Descriptor: a numerical representation of a feature's local appearance.
A good descriptor is…
Different features have different descriptors.
Robust to rotation, scale and illumination.
A small memory footprint.
Quick to compute and to match.
Trade-off: invariance vs distinctiveness vs speed.
A descriptor is a fixed-length vector of numbers.
SIFT: 128 floats · SURF: 64 or 128 floats
ORB: 256 bits (32 bytes) · BRIEF: 128, 256 or 512 bits
Matching: compare the vectors with a distance metric.
ORB: Oriented FAST + Rotated BRIEF.
BRIEF compares the brightness of random pairs of points in the patch: 1 if the first is darker, 0 otherwise. 256 pairs give 256 bits.
Use case: real-time applications, mobile devices.
import cv2
# create an ORB detector
orb = cv2.ORB_create() # ans:
# detect and compute descriptors
keypoints, descriptors = orb.detectAndCompute(gray, None) # ans:
# keypoints: a list of cv2.KeyPoint objects
# ans: each holds x, y, size, angle, response
# descriptors: a numpy array
# ans: shape (N, 32) for N keypoints: 256 bits each
# draw keypoints
img_kp = cv2.drawKeypoints(img, keypoints, None, (0, 255, 0),
cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS) # ans:
# create ORB with custom parameters
orb = cv2.ORB_create(nfeatures=500, scaleFactor=1.2, nlevels=8) # ans:
# nfeatures: maximum number of features
# ans: the 500 strongest features are kept
# scaleFactor: pyramid decimation ratio
# ans: 1.2 means each level is 1.2x smaller
# nlevels: number of pyramid levels
# ans: 8 levels for multi-scale detection
# set the edge threshold
orb = cv2.ORB_create()
orb.setEdgeThreshold(15) # ans:
# ans: ignores features within 15 pixels of the border
SIFT: Scale-Invariant Feature Transform.
A 128-dimensional float vector.
Histograms of gradient orientations around the keypoint.
Invariant to scale, rotation and illumination.
Disadvantage: slower; it was patented (the patent expired in 2020).
More in SIFT & SURF.
# create a SIFT detector
sift = cv2.SIFT_create() # ans:
# detect and compute
keypoints, descriptors = sift.detectAndCompute(gray, None) # ans:
# descriptors shape
# ans: (N, 128) for N keypoints
# custom parameters
sift = cv2.SIFT_create(nfeatures=0, nOctaveLayers=3,
contrastThreshold=0.04) # ans:
# nfeatures: 0 means no limit
# ans: every feature above the threshold is kept
# contrastThreshold: filters low-contrast features
# ans: higher value = fewer, stronger features
SURF: Speeded-Up Robust Features.
Uses integral images for speed.
Similar performance to SIFT.
Not available in OpenCV by default (it lives in the non-free contrib module).
Matching: find similar descriptors across images. Lower distance = better match.
For float descriptors (SIFT, SURF)
\(d = \sqrt{\sum_i (d_1[i] - d_2[i])^2}\)
For binary descriptors (ORB, BRIEF)
\(d\) = the number of differing bits
# create a BF matcher for ORB (Hamming distance)
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True) # ans:
# detect features in both images
orb = cv2.ORB_create()
kp1, des1 = orb.detectAndCompute(img1_gray, None)
kp2, des2 = orb.detectAndCompute(img2_gray, None)
# match descriptors
matches = bf.match(des1, des2) # ans:
# sort by distance
matches = sorted(matches, key=lambda x: x.distance) # ans:
# draw matches
img_matches = cv2.drawMatches(img1, kp1, img2, kp2,
matches[:50], None, flags=2) # ans:
FAST keypoints + 256-bit binary tests + brute-force Hamming matching, written in JavaScript. Green = a correct match, red = wrong.
FLANN: Fast Library for Approximate Nearest Neighbors.
Much faster for large datasets, because the search is approximate.
Speed vs accuracy.
# FLANN parameters for ORB
FLANN_INDEX_LSH = 6
index_params = dict(algorithm=FLANN_INDEX_LSH, table_number=6,
key_size=12, multi_probe_level=1)
search_params = dict(checks=50)
flann = cv2.FlannBasedMatcher(index_params, search_params) # ans:
# match descriptors
orb = cv2.ORB_create()
kp1, des1 = orb.detectAndCompute(img1_gray, None)
kp2, des2 = orb.detectAndCompute(img2_gray, None)
matches = flann.knnMatch(des1, des2, k=2) # ans:
# k=2 returns the 2 best matches per descriptor
Answer first, then press → to check.
What is a descriptor?
A numerical representation of a feature's local appearance.
Why do we need descriptors?
To match features across images.
What makes a good descriptor?
Distinctive, invariant, compact, fast.
What is the SIFT descriptor size?
128 dimensions (floats).
What is the ORB descriptor size?
256 bits (32 bytes).
Which distance for binary descriptors?
Hamming distance (count the differing bits).
Which distance for SIFT?
L2 (Euclidean) distance.
What are ORB's advantages over SIFT?
Faster, binary, free (no patent).
What is FLANN?
A fast approximate nearest-neighbour matcher.
What does crossCheck do in BFMatcher?
Keeps only mutual best matches (A→B and B→A).
# create ORB and detect features
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
orb = cv2.ORB_create() # ans:
keypoints, descriptors = orb.detectAndCompute(gray, None) # ans:
# create SIFT and detect
sift = cv2.SIFT_create() # ans:
keypoints, descriptors = sift.detectAndCompute(gray, None) # ans:
# create a BF matcher for ORB
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True) # ans:
# match ORB features between two images
orb = cv2.ORB_create()
kp1, des1 = orb.detectAndCompute(gray1, None)
kp2, des2 = orb.detectAndCompute(gray2, None)
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des1, des2) # ans:
matches = sorted(matches, key=lambda x: x.distance) # ans:
# draw the top 20 matches
img_matches = cv2.drawMatches(img1, kp1, img2, kp2,
matches[:20], None, flags=2) # ans:
# ORB with 1000 features
orb = cv2.ORB_create(nfeatures=1000) # ans:
kp, des = orb.detectAndCompute(gray, None) # ans:
# BF matcher for SIFT
bf = cv2.BFMatcher(cv2.NORM_L2, crossCheck=True) # ans:
# FLANN for ORB with kNN
FLANN_INDEX_LSH = 6
index_params = dict(algorithm=FLANN_INDEX_LSH, table_number=6,
key_size=12, multi_probe_level=1)
search_params = dict(checks=50)
flann = cv2.FlannBasedMatcher(index_params, search_params)
matches = flann.knnMatch(des1, des2, k=2) # ans: