Pinecone is designed to find vectors that are similar to a given set of conditions, either by comparing a new vector to the ones in the index or by comparing a vector in the index to all of the others using the query by ID feature. Because the Pinecone query function relies on performing this similarity search, there isn't a way to return all of the vectors currently stored in the index with a single query.
There isn't a guaranteed workaround for this type of query today but providing the ability to query all or export the entire index is on our roadmap for the future.
Comments
3 comments
Is there any visibility into this feature on the roadmap? We're currently exploring which vector database to use for long term production, and this inability to select * in any way is a major con for pinecone.
Hi Bill,
This is definitely on our roadmap, but it's dependent on some other features being released first, so we can't say for sure when it will be available. Hopefully later this year.
Cory
Is there any way at least to export only the vector ids? What I need is to transverse my complete index to update one metadata attribute of every single vector. My Index has more than 10.000, so the "index.query" method is no option for me as it limits to 10.000! Is there any trick so I can "cheat" and make the index.query return all vector ids? If I have this, then I can separately make a single update operation for each id that I previously obtained...
Please sign in to leave a comment.