S3 Vectors metadata pre-filtering: up to 5x higher recall on filtered search!
Hi everyone, it's Shii-chan! This update targets the vector search problem where filtering shrinks your results.
AWS What's New
aws.amazon.com
What was announced?
AWS What's New announced that Amazon S3 Vectors now supports pre-filtering. It evaluates metadata filters before running similarity search, returning up to 5x more of the matching vectors when your filter is selective. It also adds a prefix match operator, $startsWith, for values like paths and URLs.
What changes
RAG, agentic, and semantic-search applications get more complete results when they filter, so they return more relevant answers.
Dive Deep
- Indexes in new vector buckets use pre-filtering by default, with no change to how you call PutVectors or QueryVectors
- Existing indexes can be updated in place with the UpdateIndexMode API
- Before updating, you can compare pre-filtering against your current filtering on the same index using a per-query parameter on QueryVectors
- No additional cost; available in all commercial AWS Regions where S3 Vectors is available, and in the AWS China Regions
- The deployment is still in progress and is expected to complete in the coming days
You can use the AWS CLI, the AWS SDKs, or the S3 console.
Wrap-up
- Pre-filtering returns up to 5x more matches when filters are selective
- $startsWith enables prefix matching on paths and URLs
- New indexes default to it; existing ones update via UpdateIndexMode
- No extra cost, with deployment completing over the next few days
This is for anyone using filtered vector search in RAG or agents!
