HN

Need help?
<- Back
Pg_vectorsearch: An index that is faster than pgvector's IVFFlat and HNSW

Pg_vectorsearch: An index that is faster than pgvector's IVFFlat and HNSW

jascha_eng

Comments (42)

  • devy
    A couple preliminary findings if you are considering adopting this extension to replace your previous pgvector's HNSW indexing.- pg_vectorsearch uses a quantized IVF index called PRISM. PRISM caps at 1968 dimensions for vectors. If your previous HNSW index has more dimensions than that, you're SOL (at least for now until they fix that).- pg_vectorsearch requires Postgres version 18 or above. If you run PG versions below that you need to migrate your database first. Many legacy PG production environment are way dated on PG 14 or 15, migration is prohibitively high cost to just try this out.- pg_vectorsearch shipped yesterday and it's obviously alpha. Hasn't been battle tested in production environments outside of Timescale's. Also since it's so new, it isn't on Azure's allowed-extension list or any other Hyperscale's managed PG instances' allowed extension list. It takes time to graduate from alpha and mature. So if you run managaed PG instances from hyperscaler's cloud, you're SOL (at least for now until this matures into those ecosystems) or you have to migrate to a PG 18 self-hosted.
  • EgregiousCube
    I'm currently running VectorChord and I love it - with a fast backing disk, it's cut my postgres cost in half. Excited to see competition in low-RAM vector index tools. RAMageddon seems to be creating incentives to find more ways to take advantage of how fast modern nvme drives are!
  • arunmu
    Didn't these guys also write pgvectorscale which seems to be unmaintained now ? Why a new index instead of adding onto or improving pgvectorscale?
  • potomushto
    Interesting, I see no mention of support for bit precision and hammer distance operator which supposed to be much faster and storage efficient
  • Rapzid
    Does it support filtering though?Edit: post filtering onlyI don't understand how people are efficiently implementing tenant/user/permission scopes with these indexes.
  • ilumanty
    It would be so awesome if SQLite supported anything along these lines.
  • thomasahle
    How does it compare to Roar graph? https://github.com/matchyc/RoarGraph
  • alexpadula
    Dark side of the PRISM! The image on the repo 100% inspired by Dark Side of The Moon by Pink Floyd.
  • dmix
    I'm curious how it compares to `pg_search` https://www.paradedb.com/blog/introducing-search
  • swe_dima
    Sadly if cloud providers don't offer it on managed postgres then not many people can use it...
  • jakobedding
    How does this compare to pgvectorscale?
  • jascha_eng
    from the accompanying blog post: https://www.tigerdata.com/blog/announcing-pg_vectorsearch-sc...> TLDR: We are launching pg_vectorsearch, a new open-source PostgreSQL extension that makes vector search dramatically faster and more cost-efficient. Its new PRISM index uses partitioning and compression instead of HNSW's graph-based approach, delivering faster queries with a fraction of the memory and significantly faster index builds. At 100 million vectors, PRISM achieved 6 ms query latency on a machine with 4× less memory than HNSW, with index builds over 20× faster.
  • esafak
    Shouldn't you lead with your benchmarks if you're fast? qps and precision/recall, please.
  • ferozelalani
    [flagged]
  • latencyharbor
    [flagged]
  • arbor-group
    [flagged]
  • hushpadapp
    [flagged]
  • srameshonline
    [flagged]
  • anon
    undefined