Congratulations on finishing the final chapter of Practical Python and OpenCV + Case Studies. Awesome job!
I’ve organized this book to save what I feel is the best for last: keypoint detection, local invariant descriptors, and keypoint matching to build a system that can recognize the covers of books in images.
Scaling to millions of images: the BOVW model
Keypoint detectors and local invariant descriptors are just the start to building a scalable image search engine.
The problem is that such systems scale linearly, meaning that as more images are added to our system, the longer it will take to perform a search.
For example, if you included a few thousand book covers in our dataset, search time would slow dramatically because we need to compare our query image to every image in our dataset.
So, what’s the solution?
In practice, to build image search engines that scale to millions of images, we apply the bag of visual words (BOVW) method.
Constructing a BOVW consists of:
- Extracting keypoints and local invariant descriptors from our dataset.
- Clustering these features to form a codebook/vocabulary.
- Quantizing the local invariant descriptors from each image to form a single histogram that represents the visual contents of an image.
- Optionally applying tf-idf weighting.
- Constructing an inverted index for efficient search and retrieval.
Using such a system, you can create an image search engine that scales to millions of images.
In fact, to demonstrate this, I’ve included the following screenshot from the PyImageSearch Gurus course:
As you can see, I have constructed an in-memory inverted index using the Redis data store.
This inverted index stores 1,000 BOVW representations (1 BOVW representation for each image) using only 1.9MB of RAM. This implies that we can easily accommodate 1,000,000+ images using only 16GB of RAM, a small amount by modern-day server standards.
In terms of searching, we can both query the inverted index and apply spatial verification in less than 0.35 seconds, as the following screenshot demonstrates (click the image to see the full resolution screenshot):
If you can’t tell, I love building image search engines. They are the reason why I started the PyImageSearch blog.
If you are interested in learning more about image search engines, I highly recommend that you join the PyImageSearch Gurus course which includes over 20 lessons on building highly-scalable image search engines. I also have another 29 lessons on extracting features and quantifying the contents of an image — the starting point to building any image search engine.
To learn more about the PyImageSearch Gurus course, just use this link: