A graduation research project combined Meta's DINOv2 embedding model and Segment Anything Model (SAM) to enable visual search across the Rijksmuseum's 800,000+ artwork collection. The prototype allows users to find similar artworks through image-to-image search and segment-based search, where specific details within paintings can be selected to discover visual connections across the collection. Embeddings were stored in Qdrant vector database using cosine similarity for nearest neighbor search. The segmented search processed 18,000 artworks, converting SAM's COCO_RLE masks into SVG polygons for interactive browser display. While promising, scaling challenges remain: processing the full collection would take 80 days with current infrastructure, and storing all segments would create a 25 million point vector database.
Sort: